2024-09-15T21:36:06.840 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T21:36:06.867 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T21:36:06.958 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906191 branch: squid description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906191' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 conf: client: client mount timeout: 600 client_shutdown_timeout: 120 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds oft prefetch dirfrags: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true mds_sleep_rank_change: 5000000.0 mon client directed command retry: 5 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds osd ms inject socket failures: 2500 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon client directed command retry: 5 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 ms inject delay max: 1 ms inject delay probability: 0.005 ms inject delay type: client mds ms inject socket failures: 2500 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub - but it is still running - objects unfound and apparently lost - MDS_SLOW_METADATA_IO - MDS_TRIM sha1: 8921c8ef947807b717db041d85f0761b7914d5fc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 8921c8ef947807b717db041d85f0761b7914d5fc kclient: syntax: v2 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ad35db06e8924912f01e32b35576297c0f1352b4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 3393 sha1: 8921c8ef947807b717db041d85f0761b7914d5fc sleep_before_teardown: 0 subset: 294/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ad35db06e8924912f01e32b35576297c0f1352b4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBMAlTIStuHpllTVDu5vTqc1TSc4GR/wA2rUXLINU8t6s+N6zad/NtqWnt2V7UlL4r/j7H4SWh43jPZtTxS8lbc= smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKLhAQxZqaDOGfCotacJm7nV2Q2xqd7kK7xMd5ZtKLmIIx8322AOCgQpz6UbHVKCd9O80mluLSPBnT2PsLceozc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - thrashosds: null - workunit: clients: all: - fs/misc/trivial_sync.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_21:24:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T21:36:06.958 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa; will attempt to use it 2024-09-15T21:36:06.958 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks 2024-09-15T21:36:06.959 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T21:36:06.960 INFO:teuthology.task.internal:Checking packages... 2024-09-15T21:36:06.982 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8921c8ef947807b717db041d85f0761b7914d5fc' 2024-09-15T21:36:06.982 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T21:36:06.982 INFO:teuthology.packaging:ref: None 2024-09-15T21:36:06.982 INFO:teuthology.packaging:tag: None 2024-09-15T21:36:06.983 INFO:teuthology.packaging:branch: squid 2024-09-15T21:36:06.983 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:36:06.983 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-15T21:36:07.211 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-326.g8921c8ef 2024-09-15T21:36:07.213 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T21:36:07.220 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T21:36:07.220 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T21:36:07.228 INFO:teuthology.task.internal:Saving configuration 2024-09-15T21:36:07.249 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T21:36:07.257 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T21:36:07.278 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906191', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 21:31:09.683213', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBMAlTIStuHpllTVDu5vTqc1TSc4GR/wA2rUXLINU8t6s+N6zad/NtqWnt2V7UlL4r/j7H4SWh43jPZtTxS8lbc='} 2024-09-15T21:36:07.298 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906191', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 21:31:09.685043', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKLhAQxZqaDOGfCotacJm7nV2Q2xqd7kK7xMd5ZtKLmIIx8322AOCgQpz6UbHVKCd9O80mluLSPBnT2PsLceozc='} 2024-09-15T21:36:07.298 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T21:36:07.304 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-09-15T21:36:07.304 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-09-15T21:36:07.304 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T21:36:07.359 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6454153be0>, signals=[15]) 2024-09-15T21:36:07.359 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T21:36:07.366 INFO:teuthology.task.internal:Opening connections... 2024-09-15T21:36:07.366 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-09-15T21:36:07.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:36:07.443 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T21:36:07.443 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:36:07.517 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T21:36:07.524 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-09-15T21:36:07.542 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-09-15T21:36:07.542 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-09-15T21:36:07.601 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2024-09-15T21:36:07.601 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="9" 2024-09-15T21:36:07.601 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2024-09-15T21:36:07.601 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2024-09-15T21:36:07.601 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="9" 2024-09-15T21:36:07.602 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el9" 2024-09-15T21:36:07.602 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T21:36:07.602 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-09-15T21:36:07.602 INFO:teuthology.orchestra.run.smithi100.stdout:LOGO="fedora-logo-icon" 2024-09-15T21:36:07.602 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T21:36:07.602 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2024-09-15T21:36:07.602 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T21:36:07.602 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T21:36:07.602 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T21:36:07.603 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-09-15T21:36:07.624 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-09-15T21:36:07.642 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-09-15T21:36:07.642 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-09-15T21:36:07.699 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream" 2024-09-15T21:36:07.699 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9" 2024-09-15T21:36:07.699 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2024-09-15T21:36:07.699 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2024-09-15T21:36:07.699 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9" 2024-09-15T21:36:07.699 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9" 2024-09-15T21:36:07.699 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T21:36:07.699 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2024-09-15T21:36:07.699 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon" 2024-09-15T21:36:07.699 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T21:36:07.700 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2024-09-15T21:36:07.700 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T21:36:07.700 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T21:36:07.700 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T21:36:07.700 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-09-15T21:36:07.715 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T21:36:07.725 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T21:36:07.730 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T21:36:07.730 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-09-15T21:36:07.732 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-09-15T21:36:07.756 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T21:36:07.762 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T21:36:07.762 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-09-15T21:36:07.790 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-09-15T21:36:07.847 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T21:36:07.913 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T21:36:07.928 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-15T21:36:07.928 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-15T21:36:07.929 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-15T21:36:07.929 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T21:36:07.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:36:07.946 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-09-15T21:36:08.005 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-503.el9.x86_64 2024-09-15T21:36:08.005 INFO:teuthology.task.kernel:Running kernel on smithi100: 5.14.0-503.el9.x86_64 2024-09-15T21:36:08.006 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum install -y kernel 2024-09-15T21:36:12.778 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.2 MB 00:03 2024-09-15T21:36:15.804 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-09-15T21:36:20.388 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 8.1 MB/s | 6.5 MB 00:00 2024-09-15T21:36:21.772 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 105 kB/s | 19 kB 00:00 2024-09-15T21:36:22.843 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 24 MB/s | 23 MB 00:00 2024-09-15T21:36:28.240 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 46 kB/s | 1.7 kB 00:00 2024-09-15T21:36:30.255 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T21:36:30.255 INFO:teuthology.orchestra.run.smithi100.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T21:36:30.351 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-09-15T21:36:30.358 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-09-15T21:36:30.358 INFO:teuthology.orchestra.run.smithi100.stdout: Package Architecture Version Repository Size 2024-09-15T21:36:30.358 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-09-15T21:36:30.358 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-09-15T21:36:30.359 INFO:teuthology.orchestra.run.smithi100.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T21:36:30.359 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-09-15T21:36:30.359 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T21:36:30.359 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T21:36:30.359 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T21:36:30.359 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:36:30.359 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-09-15T21:36:30.359 INFO:teuthology.orchestra.run.smithi100.stdout:================================================================================ 2024-09-15T21:36:30.359 INFO:teuthology.orchestra.run.smithi100.stdout:Install 4 Packages 2024-09-15T21:36:30.359 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:36:30.360 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 79 M 2024-09-15T21:36:30.360 INFO:teuthology.orchestra.run.smithi100.stdout:Installed size: 126 M 2024-09-15T21:36:30.360 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-09-15T21:36:30.483 INFO:teuthology.orchestra.run.smithi100.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 288 kB/s | 23 kB 00:00 2024-09-15T21:36:32.950 INFO:teuthology.orchestra.run.smithi100.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 6.1 MB/s | 16 MB 00:02 2024-09-15T21:36:35.152 INFO:teuthology.orchestra.run.smithi100.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 6.1 MB/s | 29 MB 00:04 2024-09-15T21:36:36.584 INFO:teuthology.orchestra.run.smithi100.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 5.6 MB/s | 35 MB 00:06 2024-09-15T21:36:36.584 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:36:36.584 INFO:teuthology.orchestra.run.smithi100.stdout:Total 13 MB/s | 79 MB 00:06 2024-09-15T21:36:37.267 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-09-15T21:36:37.625 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-09-15T21:36:37.625 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-09-15T21:36:38.585 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-09-15T21:36:38.585 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-09-15T21:36:41.164 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-09-15T21:36:41.811 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T21:36:41.892 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:36:44.686 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:36:44.822 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:36:49.652 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:36:49.689 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:36:54.562 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:37:32.713 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:37:32.753 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:37:33.114 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:37:33.114 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T21:37:33.114 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:37:33.114 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:37:33.969 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:37:33.969 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:37:33.969 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-09-15T21:37:33.969 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T21:37:33.969 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T21:37:33.969 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T21:37:33.969 INFO:teuthology.orchestra.run.smithi100.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T21:37:33.970 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:37:33.970 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-09-15T21:37:34.326 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T21:37:34.421 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T21:37:34.421 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T21:37:34.421 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T21:37:34.421 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T21:37:34.422 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T21:37:34.422 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-15T21:37:34.422 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi100.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T21:37:34.422 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T21:37:34.480 INFO:teuthology.orchestra.run.smithi100.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T21:37:34.480 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T21:37:34.481 DEBUG:teuthology.orchestra.run.smithi100:> sudo rpm -qi grub2-tools 2024-09-15T21:37:34.512 INFO:teuthology.orchestra.run.smithi100.stdout:Name : grub2-tools 2024-09-15T21:37:34.512 INFO:teuthology.orchestra.run.smithi100.stdout:Epoch : 1 2024-09-15T21:37:34.512 INFO:teuthology.orchestra.run.smithi100.stdout:Version : 2.06 2024-09-15T21:37:34.512 INFO:teuthology.orchestra.run.smithi100.stdout:Release : 82.el9 2024-09-15T21:37:34.512 INFO:teuthology.orchestra.run.smithi100.stdout:Architecture: x86_64 2024-09-15T21:37:34.513 INFO:teuthology.orchestra.run.smithi100.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T21:37:34.513 INFO:teuthology.orchestra.run.smithi100.stdout:Group : Unspecified 2024-09-15T21:37:34.513 INFO:teuthology.orchestra.run.smithi100.stdout:Size : 8274670 2024-09-15T21:37:34.513 INFO:teuthology.orchestra.run.smithi100.stdout:License : GPLv3+ 2024-09-15T21:37:34.513 INFO:teuthology.orchestra.run.smithi100.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T21:37:34.513 INFO:teuthology.orchestra.run.smithi100.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T21:37:34.513 INFO:teuthology.orchestra.run.smithi100.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T21:37:34.513 INFO:teuthology.orchestra.run.smithi100.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T21:37:34.513 INFO:teuthology.orchestra.run.smithi100.stdout:Packager : builder@centos.org 2024-09-15T21:37:34.513 INFO:teuthology.orchestra.run.smithi100.stdout:Vendor : CentOS 2024-09-15T21:37:34.513 INFO:teuthology.orchestra.run.smithi100.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T21:37:34.513 INFO:teuthology.orchestra.run.smithi100.stdout:Summary : Support tools for GRUB. 2024-09-15T21:37:34.513 INFO:teuthology.orchestra.run.smithi100.stdout:Description : 2024-09-15T21:37:34.513 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:37:34.513 INFO:teuthology.orchestra.run.smithi100.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T21:37:34.514 INFO:teuthology.orchestra.run.smithi100.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T21:37:34.514 INFO:teuthology.orchestra.run.smithi100.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T21:37:34.514 INFO:teuthology.orchestra.run.smithi100.stdout:hardware devices. 2024-09-15T21:37:34.514 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:37:34.514 INFO:teuthology.orchestra.run.smithi100.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T21:37:34.515 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T21:37:34.515 INFO:teuthology.task.kernel:Updating grub on smithi100 to boot 5.14.0-505.el9.x86_64 2024-09-15T21:37:34.515 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T21:37:35.196 INFO:teuthology.orchestra.run.smithi100.stderr:Generating grub configuration file ... 2024-09-15T21:37:36.469 INFO:teuthology.orchestra.run.smithi100.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T21:37:36.491 INFO:teuthology.orchestra.run.smithi100.stderr:done 2024-09-15T21:37:36.494 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-09-15T21:37:36.510 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.IrP4bj9KKk 2024-09-15T21:37:36.510 DEBUG:teuthology.orchestra.run.smithi100:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.IrP4bj9KKk 2024-09-15T21:37:36.584 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.IrP4bj9KKk 2024-09-15T21:37:36.727 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.IrP4bj9KKk is 6KB 2024-09-15T21:37:36.775 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.IrP4bj9KKk 2024-09-15T21:37:36.790 DEBUG:teuthology.orchestra.run.smithi100:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T21:37:36.853 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T21:37:36.853 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T21:37:36.853 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T21:37:36.853 INFO:teuthology.orchestra.run.smithi100.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T21:37:36.854 DEBUG:teuthology.orchestra.run.smithi100:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T21:37:37.030 DEBUG:teuthology.orchestra.run.smithi100:> sudo shutdown -r now 2024-09-15T21:38:07.032 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-15T21:38:07.033 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-15T21:38:07.034 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:39:07.037 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T21:39:16.038 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-15T21:39:16.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:39:16.420 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-09-15T21:39:16.762 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-15T21:39:16.762 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-15T21:39:16.763 DEBUG:teuthology.orchestra.run.smithi100:> uname -r 2024-09-15T21:39:16.778 INFO:teuthology.orchestra.run.smithi100.stdout:5.14.0-505.el9.x86_64 2024-09-15T21:39:16.778 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T21:39:16.778 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T21:39:16.778 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T21:39:17.779 DEBUG:teuthology.parallel:result is None 2024-09-15T21:39:17.779 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T21:39:18.151 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T21:39:18.151 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T21:39:18.153 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T21:39:18.172 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T21:39:18.625 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T21:39:18.814 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T21:39:18.815 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T21:39:18.818 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T21:39:18.891 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T21:39:18.901 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T21:39:18.902 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T21:39:18.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:39:18.920 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T21:39:18.922 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T21:39:19.020 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:39:19.020 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:39:19.038 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:39:19.085 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:39:19.087 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T21:39:19.094 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T21:39:19.094 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T21:39:19.096 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T21:39:19.126 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T21:39:19.180 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T21:39:19.180 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T21:39:19.182 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T21:39:19.200 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:39:19.278 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:39:19.325 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:39:19.406 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:39:19.446 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:39:19.446 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T21:39:19.509 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:39:19.559 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:39:19.606 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:39:19.691 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:39:19.735 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:39:19.736 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T21:39:19.802 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-09-15T21:39:19.804 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-09-15T21:39:19.852 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T21:39:19.886 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T21:39:20.301 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T21:39:20.309 INFO:teuthology.task.internal:Starting timer... 2024-09-15T21:39:20.309 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T21:39:20.321 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T21:39:20.331 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-15T21:39:20.331 DEBUG:teuthology.orchestra.run.smithi100:> sudo service auditd rotate 2024-09-15T21:39:20.383 INFO:teuthology.orchestra.run.smithi100.stdout:Rotating logs: 2024-09-15T21:39:20.385 DEBUG:teuthology.orchestra.run.smithi122:> sudo service auditd rotate 2024-09-15T21:39:20.474 INFO:teuthology.orchestra.run.smithi122.stdout:Rotating logs: 2024-09-15T21:39:20.476 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T21:39:20.476 DEBUG:teuthology.orchestra.run.smithi100:> /usr/sbin/getenforce 2024-09-15T21:39:20.509 INFO:teuthology.orchestra.run.smithi100.stdout:Permissive 2024-09-15T21:39:20.509 DEBUG:teuthology.orchestra.run.smithi122:> /usr/sbin/getenforce 2024-09-15T21:39:20.534 INFO:teuthology.orchestra.run.smithi122.stdout:Permissive 2024-09-15T21:39:20.534 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi100.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi122.front.sepia.ceph.com': 'permissive'} 2024-09-15T21:39:20.534 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T21:39:20.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:39:20.573 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T21:39:20.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:39:20.601 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T21:39:20.601 DEBUG:teuthology.orchestra.run.smithi100:> sudo /usr/sbin/setenforce permissive 2024-09-15T21:39:20.642 DEBUG:teuthology.orchestra.run.smithi122:> sudo /usr/sbin/setenforce permissive 2024-09-15T21:39:20.675 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T21:39:20.685 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T21:39:20.699 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T21:39:20.700 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi100.front.sepia.ceph.com,smithi122.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T21:43:19.377 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi100.front.sepia.ceph.com'), Remote(name='ubuntu@smithi122.front.sepia.ceph.com')] 2024-09-15T21:43:19.379 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-15T21:43:19.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:43:19.466 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-09-15T21:43:19.545 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-09-15T21:43:19.545 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-15T21:43:19.546 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:43:19.628 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-09-15T21:43:19.716 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-15T21:43:19.717 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T21:43:19.729 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T21:43:19.729 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T21:43:19.730 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:43:19.732 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T21:43:19.732 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:43:19.760 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T21:43:19.775 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T21:43:19.786 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T21:43:19.800 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: ntpd: command not found 2024-09-15T21:43:19.801 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T21:43:19.812 INFO:teuthology.orchestra.run.smithi100.stdout:506 Cannot talk to daemon 2024-09-15T21:43:19.825 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: ntpd: command not found 2024-09-15T21:43:19.826 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T21:43:19.836 INFO:teuthology.orchestra.run.smithi122.stdout:506 Cannot talk to daemon 2024-09-15T21:43:19.840 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T21:43:19.850 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T21:43:19.864 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T21:43:19.895 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-09-15T21:43:19.899 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T21:43:19.900 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-09-15T21:43:19.900 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:43:19.900 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:43:19.900 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:43:19.900 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:43:19.920 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-09-15T21:43:19.923 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T21:43:19.924 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-09-15T21:43:19.924 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:43:19.924 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:43:19.924 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:43:19.924 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:43:19.925 INFO:teuthology.run_tasks:Running task install... 2024-09-15T21:43:19.941 DEBUG:teuthology.task.install:project ceph 2024-09-15T21:43:19.941 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}} 2024-09-15T21:43:19.941 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-15T21:43:19.941 INFO:teuthology.task.install:Using flavor: default 2024-09-15T21:43:19.946 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T21:43:19.947 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-15T21:43:19.947 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T21:43:19.948 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:43:19.950 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-15T21:43:19.951 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:43:20.097 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T21:43:20.097 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-15T21:43:20.103 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-15T21:43:20.104 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-15T21:43:20.210 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T21:43:20.210 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:43:20.210 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T21:43:20.221 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T21:43:20.221 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:43:20.221 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T21:43:20.237 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-15T21:43:20.237 DEBUG:teuthology.orchestra.run.smithi100:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T21:43:20.248 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-15T21:43:20.248 DEBUG:teuthology.orchestra.run.smithi122:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T21:43:20.303 DEBUG:teuthology.orchestra.run.smithi100:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T21:43:20.315 DEBUG:teuthology.orchestra.run.smithi122:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T21:43:20.379 DEBUG:teuthology.orchestra.run.smithi100:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T21:43:20.391 DEBUG:teuthology.orchestra.run.smithi122:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T21:43:20.443 INFO:teuthology.orchestra.run.smithi100.stdout:check_obsoletes = 1 2024-09-15T21:43:20.444 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum clean all 2024-09-15T21:43:20.455 INFO:teuthology.orchestra.run.smithi122.stdout:check_obsoletes = 1 2024-09-15T21:43:20.457 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum clean all 2024-09-15T21:43:20.677 INFO:teuthology.orchestra.run.smithi100.stdout:45 files removed 2024-09-15T21:43:20.683 INFO:teuthology.orchestra.run.smithi122.stdout:45 files removed 2024-09-15T21:43:20.703 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-15T21:43:20.710 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-15T21:43:21.261 INFO:teuthology.orchestra.run.smithi122.stdout:ceph packages for x86_64 307 kB/s | 83 kB 00:00 2024-09-15T21:43:21.261 INFO:teuthology.orchestra.run.smithi100.stdout:ceph packages for x86_64 305 kB/s | 83 kB 00:00 2024-09-15T21:43:21.536 INFO:teuthology.orchestra.run.smithi122.stdout:ceph noarch packages 50 kB/s | 12 kB 00:00 2024-09-15T21:43:21.545 INFO:teuthology.orchestra.run.smithi100.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-15T21:43:21.803 INFO:teuthology.orchestra.run.smithi122.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-15T21:43:21.812 INFO:teuthology.orchestra.run.smithi100.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-15T21:43:22.338 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-15T21:43:23.370 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.2 MB 00:01 2024-09-15T21:43:24.281 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-15T21:43:27.130 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 7.7 MB/s | 20 MB 00:02 2024-09-15T21:43:29.123 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 9.0 MB/s | 6.5 MB 00:00 2024-09-15T21:43:30.791 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 19 kB 00:00 2024-09-15T21:43:32.403 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 15 MB/s | 23 MB 00:01 2024-09-15T21:43:32.756 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-15T21:43:34.431 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-15T21:43:35.802 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-15T21:43:37.893 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-15T21:43:40.223 INFO:teuthology.orchestra.run.smithi100.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T21:43:40.224 INFO:teuthology.orchestra.run.smithi100.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T21:43:40.236 INFO:teuthology.orchestra.run.smithi100.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-15T21:43:40.236 INFO:teuthology.orchestra.run.smithi100.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-15T21:43:40.237 INFO:teuthology.orchestra.run.smithi100.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-15T21:43:40.238 INFO:teuthology.orchestra.run.smithi100.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-15T21:43:40.240 INFO:teuthology.orchestra.run.smithi100.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-15T21:43:40.243 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-15T21:43:40.244 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-15T21:43:40.309 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-09-15T21:43:40.320 INFO:teuthology.orchestra.run.smithi100.stdout:====================================================================================== 2024-09-15T21:43:40.320 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-09-15T21:43:40.320 INFO:teuthology.orchestra.run.smithi100.stdout:====================================================================================== 2024-09-15T21:43:40.320 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-09-15T21:43:40.320 INFO:teuthology.orchestra.run.smithi100.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-15T21:43:40.320 INFO:teuthology.orchestra.run.smithi100.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T21:43:40.320 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T21:43:40.320 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T21:43:40.320 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T21:43:40.320 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T21:43:40.320 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-15T21:43:40.321 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-15T21:43:40.321 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-15T21:43:40.321 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-15T21:43:40.321 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T21:43:40.321 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T21:43:40.321 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-15T21:43:40.321 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-15T21:43:40.321 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-15T21:43:40.321 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-15T21:43:40.322 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-15T21:43:40.322 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-15T21:43:40.322 INFO:teuthology.orchestra.run.smithi100.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-15T21:43:40.322 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-15T21:43:40.322 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T21:43:40.322 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T21:43:40.322 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T21:43:40.322 INFO:teuthology.orchestra.run.smithi100.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-15T21:43:40.323 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-15T21:43:40.323 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-15T21:43:40.323 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-15T21:43:40.323 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-15T21:43:40.323 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-15T21:43:40.323 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T21:43:40.324 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T21:43:40.324 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T21:43:40.324 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T21:43:40.324 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T21:43:40.324 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T21:43:40.324 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T21:43:40.324 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-15T21:43:40.324 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2024-09-15T21:43:40.325 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-15T21:43:40.325 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-15T21:43:40.325 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-15T21:43:40.325 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-15T21:43:40.325 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T21:43:40.325 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T21:43:40.325 INFO:teuthology.orchestra.run.smithi100.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-15T21:43:40.325 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-15T21:43:40.325 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-09-15T21:43:40.325 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T21:43:40.325 INFO:teuthology.orchestra.run.smithi100.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-15T21:43:40.325 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T21:43:40.326 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T21:43:40.326 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-15T21:43:40.326 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T21:43:40.326 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T21:43:40.326 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T21:43:40.326 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T21:43:40.326 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-15T21:43:40.326 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T21:43:40.326 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T21:43:40.326 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T21:43:40.326 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T21:43:40.327 INFO:teuthology.orchestra.run.smithi100.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T21:43:40.327 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T21:43:40.327 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T21:43:40.327 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T21:43:40.327 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T21:43:40.327 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T21:43:40.327 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T21:43:40.327 INFO:teuthology.orchestra.run.smithi100.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T21:43:40.327 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T21:43:40.327 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T21:43:40.327 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T21:43:40.327 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T21:43:40.328 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T21:43:40.328 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T21:43:40.328 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T21:43:40.328 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-15T21:43:40.328 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T21:43:40.328 INFO:teuthology.orchestra.run.smithi100.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T21:43:40.328 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T21:43:40.328 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T21:43:40.328 INFO:teuthology.orchestra.run.smithi100.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T21:43:40.328 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T21:43:40.328 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T21:43:40.328 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-15T21:43:40.328 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T21:43:40.329 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T21:43:40.329 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T21:43:40.329 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-15T21:43:40.329 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T21:43:40.329 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T21:43:40.329 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T21:43:40.329 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T21:43:40.329 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T21:43:40.329 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T21:43:40.329 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T21:43:40.329 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T21:43:40.330 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T21:43:40.330 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-15T21:43:40.330 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-15T21:43:40.330 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T21:43:40.330 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T21:43:40.330 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T21:43:40.330 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T21:43:40.330 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T21:43:40.330 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T21:43:40.330 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T21:43:40.330 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T21:43:40.331 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T21:43:40.331 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T21:43:40.331 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T21:43:40.331 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T21:43:40.331 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T21:43:40.331 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T21:43:40.331 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T21:43:40.331 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T21:43:40.331 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T21:43:40.331 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T21:43:40.331 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T21:43:40.331 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T21:43:40.331 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T21:43:40.332 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T21:43:40.332 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T21:43:40.332 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T21:43:40.332 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T21:43:40.332 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T21:43:40.332 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T21:43:40.332 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T21:43:40.332 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T21:43:40.332 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T21:43:40.332 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T21:43:40.332 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T21:43:40.333 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-15T21:43:40.333 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T21:43:40.333 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T21:43:40.333 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T21:43:40.333 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T21:43:40.333 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T21:43:40.333 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T21:43:40.333 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T21:43:40.333 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T21:43:40.333 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T21:43:40.333 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T21:43:40.333 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T21:43:40.333 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-15T21:43:40.334 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T21:43:40.334 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T21:43:40.334 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T21:43:40.334 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T21:43:40.334 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T21:43:40.334 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T21:43:40.334 INFO:teuthology.orchestra.run.smithi100.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T21:43:40.334 INFO:teuthology.orchestra.run.smithi100.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T21:43:40.334 INFO:teuthology.orchestra.run.smithi100.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T21:43:40.334 INFO:teuthology.orchestra.run.smithi100.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-15T21:43:40.335 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T21:43:40.335 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-15T21:43:40.335 INFO:teuthology.orchestra.run.smithi100.stdout:Installing weak dependencies: 2024-09-15T21:43:40.335 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T21:43:40.335 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:43:40.335 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-09-15T21:43:40.335 INFO:teuthology.orchestra.run.smithi100.stdout:====================================================================================== 2024-09-15T21:43:40.335 INFO:teuthology.orchestra.run.smithi100.stdout:Install 148 Packages 2024-09-15T21:43:40.335 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 8 Packages 2024-09-15T21:43:40.335 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:43:40.336 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 226 M 2024-09-15T21:43:40.336 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-09-15T21:43:40.944 INFO:teuthology.orchestra.run.smithi100.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 31 kB/s | 6.5 kB 00:00 2024-09-15T21:43:41.209 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-15T21:43:41.544 INFO:teuthology.orchestra.run.smithi100.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 1.5 MB/s | 896 kB 00:00 2024-09-15T21:43:41.686 INFO:teuthology.orchestra.run.smithi100.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 1.0 MB/s | 145 kB 00:00 2024-09-15T21:43:42.720 INFO:teuthology.orchestra.run.smithi100.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 2.7 MB/s | 5.4 MB 00:01 2024-09-15T21:43:43.073 INFO:teuthology.orchestra.run.smithi100.stdout:(5/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 1.7 MB/s | 2.4 MB 00:01 2024-09-15T21:43:43.228 INFO:teuthology.orchestra.run.smithi100.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 3.0 MB/s | 1.5 MB 00:00 2024-09-15T21:43:43.488 INFO:teuthology.orchestra.run.smithi122.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T21:43:43.488 INFO:teuthology.orchestra.run.smithi122.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-15T21:43:43.502 INFO:teuthology.orchestra.run.smithi122.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-15T21:43:43.503 INFO:teuthology.orchestra.run.smithi122.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-15T21:43:43.504 INFO:teuthology.orchestra.run.smithi122.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-15T21:43:43.505 INFO:teuthology.orchestra.run.smithi122.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-15T21:43:43.507 INFO:teuthology.orchestra.run.smithi122.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-15T21:43:43.510 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-15T21:43:43.511 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-15T21:43:43.575 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-15T21:43:43.593 INFO:teuthology.orchestra.run.smithi122.stdout:====================================================================================== 2024-09-15T21:43:43.593 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-15T21:43:43.593 INFO:teuthology.orchestra.run.smithi122.stdout:====================================================================================== 2024-09-15T21:43:43.593 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-15T21:43:43.593 INFO:teuthology.orchestra.run.smithi122.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-15T21:43:43.593 INFO:teuthology.orchestra.run.smithi122.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-15T21:43:43.593 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-15T21:43:43.593 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-15T21:43:43.593 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-15T21:43:43.593 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-15T21:43:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-15T21:43:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-15T21:43:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-15T21:43:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-15T21:43:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-15T21:43:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-15T21:43:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-15T21:43:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-15T21:43:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-15T21:43:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-15T21:43:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-15T21:43:43.594 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-15T21:43:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-15T21:43:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-15T21:43:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-15T21:43:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-15T21:43:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-15T21:43:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-15T21:43:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-15T21:43:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-15T21:43:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-15T21:43:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-15T21:43:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-15T21:43:43.595 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-15T21:43:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-15T21:43:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-15T21:43:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-15T21:43:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-15T21:43:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-15T21:43:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-15T21:43:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-15T21:43:43.596 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrading: 2024-09-15T21:43:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-15T21:43:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-15T21:43:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-15T21:43:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-15T21:43:43.596 INFO:teuthology.orchestra.run.smithi122.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-15T21:43:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-15T21:43:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-15T21:43:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-15T21:43:43.597 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-15T21:43:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T21:43:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-15T21:43:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T21:43:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-15T21:43:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-15T21:43:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-15T21:43:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-15T21:43:43.597 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-15T21:43:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-15T21:43:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-15T21:43:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-15T21:43:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T21:43:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T21:43:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T21:43:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T21:43:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T21:43:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T21:43:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T21:43:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T21:43:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-15T21:43:43.598 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T21:43:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T21:43:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T21:43:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T21:43:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T21:43:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-15T21:43:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T21:43:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-15T21:43:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T21:43:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-15T21:43:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T21:43:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T21:43:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T21:43:43.599 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T21:43:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T21:43:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T21:43:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T21:43:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-15T21:43:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T21:43:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T21:43:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T21:43:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-15T21:43:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T21:43:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T21:43:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T21:43:43.600 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-15T21:43:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-15T21:43:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T21:43:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T21:43:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T21:43:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T21:43:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-15T21:43:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-15T21:43:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T21:43:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T21:43:43.601 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T21:43:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T21:43:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T21:43:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T21:43:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T21:43:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-15T21:43:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T21:43:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T21:43:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T21:43:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T21:43:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T21:43:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T21:43:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T21:43:43.602 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T21:43:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T21:43:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T21:43:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T21:43:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T21:43:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T21:43:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T21:43:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T21:43:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T21:43:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T21:43:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T21:43:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T21:43:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T21:43:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T21:43:43.603 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T21:43:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T21:43:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T21:43:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-15T21:43:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T21:43:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T21:43:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T21:43:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T21:43:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T21:43:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T21:43:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T21:43:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T21:43:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T21:43:43.604 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T21:43:43.605 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T21:43:43.605 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-15T21:43:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T21:43:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T21:43:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T21:43:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T21:43:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T21:43:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T21:43:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T21:43:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T21:43:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T21:43:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-15T21:43:43.606 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T21:43:43.607 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-15T21:43:43.607 INFO:teuthology.orchestra.run.smithi122.stdout:Installing weak dependencies: 2024-09-15T21:43:43.607 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T21:43:43.607 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:43:43.607 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-15T21:43:43.607 INFO:teuthology.orchestra.run.smithi122.stdout:====================================================================================== 2024-09-15T21:43:43.607 INFO:teuthology.orchestra.run.smithi122.stdout:Install 148 Packages 2024-09-15T21:43:43.607 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrade 8 Packages 2024-09-15T21:43:43.607 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:43:43.608 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 226 M 2024-09-15T21:43:43.608 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-15T21:43:44.243 INFO:teuthology.orchestra.run.smithi122.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 20 kB/s | 6.5 kB 00:00 2024-09-15T21:43:45.027 INFO:teuthology.orchestra.run.smithi122.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 1.1 MB/s | 896 kB 00:00 2024-09-15T21:43:45.419 INFO:teuthology.orchestra.run.smithi122.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 370 kB/s | 145 kB 00:00 2024-09-15T21:43:45.621 INFO:teuthology.orchestra.run.smithi100.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 1.8 MB/s | 4.7 MB 00:02 2024-09-15T21:43:45.811 INFO:teuthology.orchestra.run.smithi122.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 2.9 MB/s | 5.4 MB 00:01 2024-09-15T21:43:46.219 INFO:teuthology.orchestra.run.smithi122.stdout:(5/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 3.7 MB/s | 1.5 MB 00:00 2024-09-15T21:43:47.245 INFO:teuthology.orchestra.run.smithi122.stdout:(6/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 4.6 MB/s | 4.7 MB 00:01 2024-09-15T21:43:47.595 INFO:teuthology.orchestra.run.smithi100.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 3.1 MB/s | 21 MB 00:06 2024-09-15T21:43:47.779 INFO:teuthology.orchestra.run.smithi100.stdout:(9/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86_ 3.7 MB/s | 17 MB 00:04 2024-09-15T21:43:47.804 INFO:teuthology.orchestra.run.smithi100.stdout:(10/156): ceph-selinux-19.1.1-326.g8921c8ef.el9 121 kB/s | 25 kB 00:00 2024-09-15T21:43:47.954 INFO:teuthology.orchestra.run.smithi100.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 1.5 MB/s | 228 kB 00:00 2024-09-15T21:43:48.038 INFO:teuthology.orchestra.run.smithi100.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 325 kB/s | 27 kB 00:00 2024-09-15T21:43:48.113 INFO:teuthology.orchestra.run.smithi100.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 423 kB/s | 32 kB 00:00 2024-09-15T21:43:48.322 INFO:teuthology.orchestra.run.smithi100.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 3.5 MB/s | 751 kB 00:00 2024-09-15T21:43:48.387 INFO:teuthology.orchestra.run.smithi122.stdout:(7/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 823 kB/s | 2.4 MB 00:02 2024-09-15T21:43:48.414 INFO:teuthology.orchestra.run.smithi100.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 1.7 MB/s | 163 kB 00:00 2024-09-15T21:43:48.507 INFO:teuthology.orchestra.run.smithi100.stdout:(16/156): librados-devel-19.1.1-326.g8921c8ef.e 1.4 MB/s | 127 kB 00:00 2024-09-15T21:43:48.689 INFO:teuthology.orchestra.run.smithi100.stdout:(17/156): libradosstriper1-19.1.1-326.g8921c8ef 2.7 MB/s | 506 kB 00:00 2024-09-15T21:43:50.123 INFO:teuthology.orchestra.run.smithi100.stdout:(18/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 3.7 MB/s | 5.3 MB 00:01 2024-09-15T21:43:50.206 INFO:teuthology.orchestra.run.smithi100.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 543 kB/s | 45 kB 00:00 2024-09-15T21:43:50.245 INFO:teuthology.orchestra.run.smithi122.stdout:(8/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86_ 5.6 MB/s | 17 MB 00:02 2024-09-15T21:43:50.290 INFO:teuthology.orchestra.run.smithi100.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.6 MB/s | 136 kB 00:00 2024-09-15T21:43:50.337 INFO:teuthology.orchestra.run.smithi122.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 275 kB/s | 25 kB 00:00 2024-09-15T21:43:50.382 INFO:teuthology.orchestra.run.smithi100.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.7 MB/s | 161 kB 00:00 2024-09-15T21:43:50.558 INFO:teuthology.orchestra.run.smithi100.stdout:(22/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 2.1 MB/s | 11 MB 00:04 2024-09-15T21:43:50.583 INFO:teuthology.orchestra.run.smithi100.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 1.6 MB/s | 320 kB 00:00 2024-09-15T21:43:50.675 INFO:teuthology.orchestra.run.smithi100.stdout:(24/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.1 MB/s | 99 kB 00:00 2024-09-15T21:43:50.725 INFO:teuthology.orchestra.run.smithi100.stdout:(25/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 1.8 MB/s | 300 kB 00:00 2024-09-15T21:43:50.767 INFO:teuthology.orchestra.run.smithi100.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 927 kB/s | 85 kB 00:00 2024-09-15T21:43:50.859 INFO:teuthology.orchestra.run.smithi100.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 1.8 MB/s | 170 kB 00:00 2024-09-15T21:43:50.951 INFO:teuthology.orchestra.run.smithi100.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 348 kB/s | 32 kB 00:00 2024-09-15T21:43:51.051 INFO:teuthology.orchestra.run.smithi100.stdout:(29/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.4 MB/s | 146 kB 00:00 2024-09-15T21:43:52.093 INFO:teuthology.orchestra.run.smithi100.stdout:(30/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 3.4 MB/s | 3.6 MB 00:01 2024-09-15T21:43:52.143 INFO:teuthology.orchestra.run.smithi100.stdout:(31/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 2.2 MB/s | 3.1 MB 00:01 2024-09-15T21:43:52.319 INFO:teuthology.orchestra.run.smithi100.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 1.4 MB/s | 249 kB 00:00 2024-09-15T21:43:52.419 INFO:teuthology.orchestra.run.smithi100.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 494 kB/s | 49 kB 00:00 2024-09-15T21:43:52.519 INFO:teuthology.orchestra.run.smithi100.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 164 kB/s | 16 kB 00:00 2024-09-15T21:43:52.703 INFO:teuthology.orchestra.run.smithi100.stdout:(35/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 1.5 MB/s | 285 kB 00:00 2024-09-15T21:43:53.070 INFO:teuthology.orchestra.run.smithi100.stdout:(36/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 2.0 MB/s | 765 kB 00:00 2024-09-15T21:43:53.162 INFO:teuthology.orchestra.run.smithi100.stdout:(37/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 199 kB/s | 18 kB 00:00 2024-09-15T21:43:53.287 INFO:teuthology.orchestra.run.smithi100.stdout:(38/156): fuse-2.9.9-16.el9.x86_64.rpm 639 kB/s | 80 kB 00:00 2024-09-15T21:43:53.404 INFO:teuthology.orchestra.run.smithi100.stdout:(39/156): libgfortran-11.5.0-2.el9.x86_64.rpm 6.7 MB/s | 801 kB 00:00 2024-09-15T21:43:53.471 INFO:teuthology.orchestra.run.smithi100.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-15T21:43:53.546 INFO:teuthology.orchestra.run.smithi100.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-09-15T21:43:53.655 INFO:teuthology.orchestra.run.smithi100.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-09-15T21:43:53.722 INFO:teuthology.orchestra.run.smithi100.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-15T21:43:53.781 INFO:teuthology.orchestra.run.smithi100.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-15T21:43:53.873 INFO:teuthology.orchestra.run.smithi100.stdout:(45/156): ceph-mgr-diskprediction-local-19.1.1- 4.2 MB/s | 7.4 MB 00:01 2024-09-15T21:43:53.907 INFO:teuthology.orchestra.run.smithi100.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 995 kB/s | 125 kB 00:00 2024-09-15T21:43:54.007 INFO:teuthology.orchestra.run.smithi100.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.6 MB/s | 215 kB 00:00 2024-09-15T21:43:54.478 INFO:teuthology.orchestra.run.smithi122.stdout:(10/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 1.7 MB/s | 11 MB 00:06 2024-09-15T21:43:54.578 INFO:teuthology.orchestra.run.smithi122.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 2.2 MB/s | 228 kB 00:00 2024-09-15T21:43:54.637 INFO:teuthology.orchestra.run.smithi122.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 464 kB/s | 27 kB 00:00 2024-09-15T21:43:54.687 INFO:teuthology.orchestra.run.smithi122.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 634 kB/s | 32 kB 00:00 2024-09-15T21:43:54.765 INFO:teuthology.orchestra.run.smithi100.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 803 kB/s | 689 kB 00:00 2024-09-15T21:43:54.907 INFO:teuthology.orchestra.run.smithi100.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-09-15T21:43:54.912 INFO:teuthology.orchestra.run.smithi122.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 3.3 MB/s | 751 kB 00:00 2024-09-15T21:43:55.113 INFO:teuthology.orchestra.run.smithi122.stdout:(15/156): ceph-common-19.1.1-326.g8921c8ef.el9. 1.9 MB/s | 21 MB 00:11 2024-09-15T21:43:55.172 INFO:teuthology.orchestra.run.smithi122.stdout:(16/156): libcephsqlite-19.1.1-326.g8921c8ef.el 631 kB/s | 163 kB 00:00 2024-09-15T21:43:55.214 INFO:teuthology.orchestra.run.smithi122.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 1.2 MB/s | 127 kB 00:00 2024-09-15T21:43:55.281 INFO:teuthology.orchestra.run.smithi122.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 4.5 MB/s | 506 kB 00:00 2024-09-15T21:43:55.331 INFO:teuthology.orchestra.run.smithi122.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 905 kB/s | 45 kB 00:00 2024-09-15T21:43:55.381 INFO:teuthology.orchestra.run.smithi122.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 2.7 MB/s | 136 kB 00:00 2024-09-15T21:43:55.482 INFO:teuthology.orchestra.run.smithi122.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.6 MB/s | 161 kB 00:00 2024-09-15T21:43:55.573 INFO:teuthology.orchestra.run.smithi122.stdout:(22/156): python3-rados-19.1.1-326.g8921c8ef.el 3.4 MB/s | 320 kB 00:00 2024-09-15T21:43:55.625 INFO:teuthology.orchestra.run.smithi100.stdout:(50/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 6.2 MB/s | 49 MB 00:07 2024-09-15T21:43:55.649 INFO:teuthology.orchestra.run.smithi122.stdout:(23/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 3.9 MB/s | 300 kB 00:00 2024-09-15T21:43:55.699 INFO:teuthology.orchestra.run.smithi122.stdout:(24/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 2.0 MB/s | 99 kB 00:00 2024-09-15T21:43:55.741 INFO:teuthology.orchestra.run.smithi100.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 110 kB/s | 107 kB 00:00 2024-09-15T21:43:55.749 INFO:teuthology.orchestra.run.smithi122.stdout:(25/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-15T21:43:55.866 INFO:teuthology.orchestra.run.smithi100.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 23 kB/s | 22 kB 00:00 2024-09-15T21:43:56.284 INFO:teuthology.orchestra.run.smithi100.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 60 kB/s | 33 kB 00:00 2024-09-15T21:43:56.358 INFO:teuthology.orchestra.run.smithi122.stdout:(26/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 8.1 MB/s | 49 MB 00:06 2024-09-15T21:43:56.442 INFO:teuthology.orchestra.run.smithi122.stdout:(27/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 4.4 MB/s | 5.3 MB 00:01 2024-09-15T21:43:56.467 INFO:teuthology.orchestra.run.smithi122.stdout:(28/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 1.5 MB/s | 170 kB 00:00 2024-09-15T21:43:56.467 INFO:teuthology.orchestra.run.smithi100.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 373 kB/s | 313 kB 00:00 2024-09-15T21:43:56.492 INFO:teuthology.orchestra.run.smithi122.stdout:(29/156): ceph-grafana-dashboards-19.1.1-326.g8 636 kB/s | 32 kB 00:00 2024-09-15T21:43:56.517 INFO:teuthology.orchestra.run.smithi122.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 2.9 MB/s | 146 kB 00:00 2024-09-15T21:43:56.585 INFO:teuthology.orchestra.run.smithi122.stdout:(31/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 3.7 MB/s | 3.1 MB 00:00 2024-09-15T21:43:56.651 INFO:teuthology.orchestra.run.smithi122.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 3.7 MB/s | 249 kB 00:00 2024-09-15T21:43:56.702 INFO:teuthology.orchestra.run.smithi122.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 987 kB/s | 49 kB 00:00 2024-09-15T21:43:56.752 INFO:teuthology.orchestra.run.smithi122.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 330 kB/s | 16 kB 00:00 2024-09-15T21:43:56.809 INFO:teuthology.orchestra.run.smithi100.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 35 kB/s | 18 kB 00:00 2024-09-15T21:43:56.819 INFO:teuthology.orchestra.run.smithi122.stdout:(35/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 4.2 MB/s | 285 kB 00:00 2024-09-15T21:43:56.926 INFO:teuthology.orchestra.run.smithi100.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.8 MB/s | 3.0 MB 00:01 2024-09-15T21:43:56.951 INFO:teuthology.orchestra.run.smithi100.stdout:(57/156): libaio-devel-0.3.111-13.el9.x86_64.rp 23 kB/s | 11 kB 00:00 2024-09-15T21:43:56.961 INFO:teuthology.orchestra.run.smithi122.stdout:(36/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 5.3 MB/s | 765 kB 00:00 2024-09-15T21:43:57.020 INFO:teuthology.orchestra.run.smithi122.stdout:(37/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 6.8 MB/s | 3.6 MB 00:00 2024-09-15T21:43:57.045 INFO:teuthology.orchestra.run.smithi122.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 218 kB/s | 18 kB 00:00 2024-09-15T21:43:57.145 INFO:teuthology.orchestra.run.smithi122.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 12 MB/s | 7.4 MB 00:00 2024-09-15T21:43:57.270 INFO:teuthology.orchestra.run.smithi122.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 318 kB/s | 80 kB 00:00 2024-09-15T21:43:57.304 INFO:teuthology.orchestra.run.smithi122.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-15T21:43:57.354 INFO:teuthology.orchestra.run.smithi122.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-15T21:43:57.388 INFO:teuthology.orchestra.run.smithi122.stdout:(43/156): libgfortran-11.5.0-2.el9.x86_64.rpm 2.3 MB/s | 801 kB 00:00 2024-09-15T21:43:57.415 INFO:teuthology.orchestra.run.smithi122.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.7 MB/s | 106 kB 00:00 2024-09-15T21:43:57.440 INFO:teuthology.orchestra.run.smithi122.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 135 kB 00:00 2024-09-15T21:43:57.466 INFO:teuthology.orchestra.run.smithi122.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-15T21:43:57.499 INFO:teuthology.orchestra.run.smithi122.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-15T21:43:57.501 INFO:teuthology.orchestra.run.smithi100.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 79 kB/s | 45 kB 00:00 2024-09-15T21:43:57.541 INFO:teuthology.orchestra.run.smithi122.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 5.2 MB/s | 1.2 MB 00:00 2024-09-15T21:43:57.575 INFO:teuthology.orchestra.run.smithi122.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 6.2 MB/s | 689 kB 00:00 2024-09-15T21:43:57.600 INFO:teuthology.orchestra.run.smithi122.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-15T21:43:57.626 INFO:teuthology.orchestra.run.smithi122.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 431 kB/s | 22 kB 00:00 2024-09-15T21:43:57.651 INFO:teuthology.orchestra.run.smithi122.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-15T21:43:57.745 INFO:teuthology.orchestra.run.smithi122.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 275 kB/s | 33 kB 00:00 2024-09-15T21:43:57.785 INFO:teuthology.orchestra.run.smithi122.stdout:(54/156): bison-3.7.4-5.el9.x86_64.rpm 3.2 MB/s | 943 kB 00:00 2024-09-15T21:43:57.810 INFO:teuthology.orchestra.run.smithi100.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 772 kB/s | 662 kB 00:00 2024-09-15T21:43:57.811 INFO:teuthology.orchestra.run.smithi122.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 279 kB/s | 18 kB 00:00 2024-09-15T21:43:57.835 INFO:teuthology.orchestra.run.smithi122.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-15T21:43:57.861 INFO:teuthology.orchestra.run.smithi122.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-15T21:43:57.886 INFO:teuthology.orchestra.run.smithi122.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 901 kB/s | 45 kB 00:00 2024-09-15T21:43:57.928 INFO:teuthology.orchestra.run.smithi122.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-15T21:43:57.962 INFO:teuthology.orchestra.run.smithi122.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 7.5 MB/s | 581 kB 00:00 2024-09-15T21:43:57.987 INFO:teuthology.orchestra.run.smithi122.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-15T21:43:58.012 INFO:teuthology.orchestra.run.smithi122.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-15T21:43:58.046 INFO:teuthology.orchestra.run.smithi122.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-09-15T21:43:58.080 INFO:teuthology.orchestra.run.smithi122.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-09-15T21:43:58.105 INFO:teuthology.orchestra.run.smithi122.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 660 kB/s | 38 kB 00:00 2024-09-15T21:43:58.163 INFO:teuthology.orchestra.run.smithi122.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.8 MB/s | 3.0 MB 00:00 2024-09-15T21:43:58.264 INFO:teuthology.orchestra.run.smithi122.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 27 MB/s | 4.9 MB 00:00 2024-09-15T21:43:58.314 INFO:teuthology.orchestra.run.smithi122.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-15T21:43:58.381 INFO:teuthology.orchestra.run.smithi122.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-15T21:43:58.406 INFO:teuthology.orchestra.run.smithi122.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.5 MB/s | 137 kB 00:00 2024-09-15T21:43:58.427 INFO:teuthology.orchestra.run.smithi100.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 40 kB/s | 24 kB 00:00 2024-09-15T21:43:58.457 INFO:teuthology.orchestra.run.smithi122.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 4.6 MB/s | 910 kB 00:00 2024-09-15T21:43:58.460 INFO:teuthology.orchestra.run.smithi100.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 606 kB/s | 581 kB 00:00 2024-09-15T21:43:58.490 INFO:teuthology.orchestra.run.smithi122.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-09-15T21:43:58.516 INFO:teuthology.orchestra.run.smithi122.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-15T21:43:58.549 INFO:teuthology.orchestra.run.smithi122.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.8 MB/s | 61 kB 00:00 2024-09-15T21:43:58.624 INFO:teuthology.orchestra.run.smithi122.stdout:(75/156): postgresql-server-13.16-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-09-15T21:43:58.650 INFO:teuthology.orchestra.run.smithi122.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-15T21:43:58.675 INFO:teuthology.orchestra.run.smithi122.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-15T21:43:58.692 INFO:teuthology.orchestra.run.smithi122.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 612 kB/s | 26 kB 00:00 2024-09-15T21:43:58.717 INFO:teuthology.orchestra.run.smithi122.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 458 kB/s | 19 kB 00:00 2024-09-15T21:43:58.751 INFO:teuthology.orchestra.run.smithi122.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-15T21:43:58.776 INFO:teuthology.orchestra.run.smithi122.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-09-15T21:43:58.860 INFO:teuthology.orchestra.run.smithi122.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-15T21:43:58.885 INFO:teuthology.orchestra.run.smithi122.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-09-15T21:43:58.910 INFO:teuthology.orchestra.run.smithi122.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-09-15T21:43:58.936 INFO:teuthology.orchestra.run.smithi122.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-15T21:43:58.961 INFO:teuthology.orchestra.run.smithi122.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-09-15T21:43:58.986 INFO:teuthology.orchestra.run.smithi122.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-15T21:43:59.012 INFO:teuthology.orchestra.run.smithi122.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-15T21:43:59.037 INFO:teuthology.orchestra.run.smithi122.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-15T21:43:59.062 INFO:teuthology.orchestra.run.smithi122.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-15T21:43:59.077 INFO:teuthology.orchestra.run.smithi100.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 73 kB/s | 47 kB 00:00 2024-09-15T21:43:59.087 INFO:teuthology.orchestra.run.smithi122.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-15T21:43:59.163 INFO:teuthology.orchestra.run.smithi122.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-15T21:43:59.188 INFO:teuthology.orchestra.run.smithi122.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-09-15T21:43:59.213 INFO:teuthology.orchestra.run.smithi122.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-09-15T21:43:59.238 INFO:teuthology.orchestra.run.smithi122.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-15T21:43:59.264 INFO:teuthology.orchestra.run.smithi122.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 713 kB/s | 36 kB 00:00 2024-09-15T21:43:59.289 INFO:teuthology.orchestra.run.smithi122.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-15T21:43:59.322 INFO:teuthology.orchestra.run.smithi122.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-15T21:43:59.328 INFO:teuthology.orchestra.run.smithi100.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 337 kB/s | 292 kB 00:00 2024-09-15T21:43:59.348 INFO:teuthology.orchestra.run.smithi122.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 988 kB/s | 58 kB 00:00 2024-09-15T21:43:59.373 INFO:teuthology.orchestra.run.smithi122.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-15T21:43:59.399 INFO:teuthology.orchestra.run.smithi122.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-15T21:43:59.449 INFO:teuthology.orchestra.run.smithi122.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 594 kB/s | 45 kB 00:00 2024-09-15T21:43:59.533 INFO:teuthology.orchestra.run.smithi122.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 167 kB/s | 22 kB 00:00 2024-09-15T21:43:59.716 INFO:teuthology.orchestra.run.smithi122.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-15T21:43:59.775 INFO:teuthology.orchestra.run.smithi122.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.2 MB/s | 548 kB 00:00 2024-09-15T21:43:59.800 INFO:teuthology.orchestra.run.smithi122.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 843 kB/s | 19 kB 00:00 2024-09-15T21:43:59.834 INFO:teuthology.orchestra.run.smithi122.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 2.6 MB/s | 308 kB 00:00 2024-09-15T21:43:59.859 INFO:teuthology.orchestra.run.smithi122.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-15T21:43:59.884 INFO:teuthology.orchestra.run.smithi122.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-15T21:43:59.911 INFO:teuthology.orchestra.run.smithi100.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 226 kB/s | 188 kB 00:00 2024-09-15T21:43:59.918 INFO:teuthology.orchestra.run.smithi122.stdout:(110/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-09-15T21:43:59.943 INFO:teuthology.orchestra.run.smithi122.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-15T21:43:59.961 INFO:teuthology.orchestra.run.smithi100.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 61 kB/s | 38 kB 00:00 2024-09-15T21:43:59.993 INFO:teuthology.orchestra.run.smithi122.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-15T21:44:00.035 INFO:teuthology.orchestra.run.smithi122.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.7 MB/s | 839 kB 00:00 2024-09-15T21:44:00.103 INFO:teuthology.orchestra.run.smithi122.stdout:(114/156): libarrow-9.0.0-11.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-15T21:44:00.127 INFO:teuthology.orchestra.run.smithi122.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 472 kB/s | 43 kB 00:00 2024-09-15T21:44:00.153 INFO:teuthology.orchestra.run.smithi122.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 645 kB/s | 32 kB 00:00 2024-09-15T21:44:00.178 INFO:teuthology.orchestra.run.smithi122.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 280 kB/s | 14 kB 00:00 2024-09-15T21:44:00.204 INFO:teuthology.orchestra.run.smithi122.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-09-15T21:44:00.229 INFO:teuthology.orchestra.run.smithi122.stdout:(119/156): python3-asyncssh-2.13.2-5.el9.noarch 2.3 MB/s | 548 kB 00:00 2024-09-15T21:44:00.296 INFO:teuthology.orchestra.run.smithi122.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-09-15T21:44:00.321 INFO:teuthology.orchestra.run.smithi122.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-09-15T21:44:00.346 INFO:teuthology.orchestra.run.smithi122.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 1.8 MB/s | 220 kB 00:00 2024-09-15T21:44:00.380 INFO:teuthology.orchestra.run.smithi122.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-09-15T21:44:00.405 INFO:teuthology.orchestra.run.smithi122.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-09-15T21:44:00.430 INFO:teuthology.orchestra.run.smithi122.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-09-15T21:44:00.456 INFO:teuthology.orchestra.run.smithi122.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-09-15T21:44:00.481 INFO:teuthology.orchestra.run.smithi122.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-15T21:44:00.506 INFO:teuthology.orchestra.run.smithi122.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-15T21:44:00.540 INFO:teuthology.orchestra.run.smithi122.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-15T21:44:00.574 INFO:teuthology.orchestra.run.smithi122.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-09-15T21:44:00.599 INFO:teuthology.orchestra.run.smithi122.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-09-15T21:44:00.625 INFO:teuthology.orchestra.run.smithi122.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-15T21:44:00.650 INFO:teuthology.orchestra.run.smithi122.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-15T21:44:00.675 INFO:teuthology.orchestra.run.smithi122.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-15T21:44:00.700 INFO:teuthology.orchestra.run.smithi122.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-15T21:44:00.725 INFO:teuthology.orchestra.run.smithi122.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-15T21:44:00.751 INFO:teuthology.orchestra.run.smithi122.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-15T21:44:00.776 INFO:teuthology.orchestra.run.smithi122.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-15T21:44:00.834 INFO:teuthology.orchestra.run.smithi122.stdout:(139/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.8 MB/s | 2.0 MB 00:00 2024-09-15T21:44:00.860 INFO:teuthology.orchestra.run.smithi122.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.4 MB/s | 188 kB 00:00 2024-09-15T21:44:00.885 INFO:teuthology.orchestra.run.smithi122.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 329 kB/s | 36 kB 00:00 2024-09-15T21:44:00.912 INFO:teuthology.orchestra.run.smithi122.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-15T21:44:00.936 INFO:teuthology.orchestra.run.smithi122.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-15T21:44:00.961 INFO:teuthology.orchestra.run.smithi122.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-15T21:44:00.986 INFO:teuthology.orchestra.run.smithi122.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-15T21:44:01.011 INFO:teuthology.orchestra.run.smithi122.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-15T21:44:01.037 INFO:teuthology.orchestra.run.smithi122.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-15T21:44:01.095 INFO:teuthology.orchestra.run.smithi122.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-15T21:44:01.128 INFO:teuthology.orchestra.run.smithi100.stdout:(66/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.1 MB/s | 4.9 MB 00:01 2024-09-15T21:44:01.254 INFO:teuthology.orchestra.run.smithi122.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 15 MB/s | 2.3 MB 00:00 2024-09-15T21:44:01.279 INFO:teuthology.orchestra.run.smithi100.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.4 MB/s | 4.4 MB 00:01 2024-09-15T21:44:01.321 INFO:teuthology.orchestra.run.smithi122.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-09-15T21:44:01.363 INFO:teuthology.orchestra.run.smithi122.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 982 kB/s | 41 kB 00:00 2024-09-15T21:44:01.422 INFO:teuthology.orchestra.run.smithi122.stdout:(152/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 8.2 MB/s | 3.2 MB 00:00 2024-09-15T21:44:01.447 INFO:teuthology.orchestra.run.smithi122.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 2.9 MB/s | 249 kB 00:00 2024-09-15T21:44:01.514 INFO:teuthology.orchestra.run.smithi122.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 36 MB/s | 2.4 MB 00:00 2024-09-15T21:44:01.622 INFO:teuthology.orchestra.run.smithi122.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 6.8 MB/s | 1.4 MB 00:00 2024-09-15T21:44:01.697 INFO:teuthology.orchestra.run.smithi122.stdout:(156/156): librados2-19.1.1-326.g8921c8ef.el9.x 5.0 MB/s | 3.4 MB 00:00 2024-09-15T21:44:01.703 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:44:01.703 INFO:teuthology.orchestra.run.smithi122.stdout:Total 12 MB/s | 226 MB 00:18 2024-09-15T21:44:02.196 INFO:teuthology.orchestra.run.smithi100.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-09-15T21:44:02.304 INFO:teuthology.orchestra.run.smithi100.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 888 kB/s | 910 kB 00:01 2024-09-15T21:44:02.945 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-15T21:44:03.005 INFO:teuthology.orchestra.run.smithi100.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 170 kB/s | 137 kB 00:00 2024-09-15T21:44:03.095 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-15T21:44:03.095 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-15T21:44:03.638 INFO:teuthology.orchestra.run.smithi100.stdout:(71/156): postgresql-server-13.16-1.el9.x86_64. 4.3 MB/s | 5.8 MB 00:01 2024-09-15T21:44:03.738 INFO:teuthology.orchestra.run.smithi100.stdout:(72/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 23 kB/s | 160 kB 00:06 2024-09-15T21:44:04.022 INFO:teuthology.orchestra.run.smithi100.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:01 2024-09-15T21:44:04.372 INFO:teuthology.orchestra.run.smithi100.stdout:(74/156): python3-attrs-20.3.0-7.el9.noarch.rpm 119 kB/s | 87 kB 00:00 2024-09-15T21:44:04.714 INFO:teuthology.orchestra.run.smithi100.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 89 kB/s | 61 kB 00:00 2024-09-15T21:44:04.887 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-15T21:44:04.888 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-15T21:44:05.015 INFO:teuthology.orchestra.run.smithi100.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.7 MB/s | 6.0 MB 00:01 2024-09-15T21:44:05.265 INFO:teuthology.orchestra.run.smithi100.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 274 kB/s | 245 kB 00:00 2024-09-15T21:44:05.632 INFO:teuthology.orchestra.run.smithi100.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 271 kB/s | 248 kB 00:00 2024-09-15T21:44:05.657 INFO:teuthology.orchestra.run.smithi100.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 40 kB/s | 26 kB 00:00 2024-09-15T21:44:05.807 INFO:teuthology.orchestra.run.smithi100.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 36 kB/s | 19 kB 00:00 2024-09-15T21:44:06.274 INFO:teuthology.orchestra.run.smithi100.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 56 kB/s | 35 kB 00:00 2024-09-15T21:44:06.458 INFO:teuthology.orchestra.run.smithi100.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 209 kB/s | 172 kB 00:00 2024-09-15T21:44:07.033 INFO:teuthology.orchestra.run.smithi100.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.2 MB/s | 5.1 MB 00:01 2024-09-15T21:44:07.150 INFO:teuthology.orchestra.run.smithi100.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 286 kB/s | 250 kB 00:00 2024-09-15T21:44:07.358 INFO:teuthology.orchestra.run.smithi100.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 247 kB/s | 222 kB 00:00 2024-09-15T21:44:07.742 INFO:teuthology.orchestra.run.smithi100.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 109 kB/s | 77 kB 00:00 2024-09-15T21:44:07.784 INFO:teuthology.orchestra.run.smithi100.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 67 kB/s | 42 kB 00:00 2024-09-15T21:44:08.234 INFO:teuthology.orchestra.run.smithi100.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 308 kB/s | 269 kB 00:00 2024-09-15T21:44:08.476 INFO:teuthology.orchestra.run.smithi100.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 216 kB/s | 159 kB 00:00 2024-09-15T21:44:08.593 INFO:teuthology.orchestra.run.smithi100.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 345 kB/s | 279 kB 00:00 2024-09-15T21:44:08.851 INFO:teuthology.orchestra.run.smithi100.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 83 kB/s | 51 kB 00:00 2024-09-15T21:44:09.093 INFO:teuthology.orchestra.run.smithi100.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 88 kB/s | 54 kB 00:00 2024-09-15T21:44:09.419 INFO:teuthology.orchestra.run.smithi100.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 73 kB/s | 42 kB 00:00 2024-09-15T21:44:09.661 INFO:teuthology.orchestra.run.smithi100.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 78 kB/s | 44 kB 00:00 2024-09-15T21:44:10.061 INFO:teuthology.orchestra.run.smithi100.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-15T21:44:10.269 INFO:teuthology.orchestra.run.smithi100.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 260 kB/s | 221 kB 00:00 2024-09-15T21:44:10.328 INFO:teuthology.orchestra.run.smithi100.stdout:(97/156): qatlib-service-24.02.0-1.el9.x86_64.r 54 kB/s | 36 kB 00:00 2024-09-15T21:44:10.637 INFO:teuthology.orchestra.run.smithi100.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 83 kB/s | 47 kB 00:00 2024-09-15T21:44:10.962 INFO:teuthology.orchestra.run.smithi100.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 91 kB/s | 58 kB 00:00 2024-09-15T21:44:11.079 INFO:teuthology.orchestra.run.smithi100.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 376 kB/s | 304 kB 00:00 2024-09-15T21:44:11.106 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-15T21:44:11.202 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-15T21:44:11.319 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-15T21:44:11.321 INFO:teuthology.orchestra.run.smithi100.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 140 kB/s | 96 kB 00:00 2024-09-15T21:44:11.405 INFO:teuthology.orchestra.run.smithi100.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-09-15T21:44:11.513 INFO:teuthology.orchestra.run.smithi100.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 7.8 MB/s | 863 kB 00:00 2024-09-15T21:44:11.587 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-15T21:44:11.606 INFO:teuthology.orchestra.run.smithi100.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.8 MB/s | 548 kB 00:00 2024-09-15T21:44:11.647 INFO:teuthology.orchestra.run.smithi100.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 7.3 MB/s | 308 kB 00:00 2024-09-15T21:44:11.672 INFO:teuthology.orchestra.run.smithi100.stdout:(106/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 90 kB/s | 64 kB 00:00 2024-09-15T21:44:11.678 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-15T21:44:11.698 INFO:teuthology.orchestra.run.smithi100.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 398 kB/s | 19 kB 00:00 2024-09-15T21:44:11.703 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-15T21:44:11.755 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-15T21:44:11.756 INFO:teuthology.orchestra.run.smithi100.stdout:(108/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 66 kB/s | 45 kB 00:00 2024-09-15T21:44:11.785 INFO:teuthology.orchestra.run.smithi100.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-09-15T21:44:11.815 INFO:teuthology.orchestra.run.smithi100.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-15T21:44:11.849 INFO:teuthology.orchestra.run.smithi100.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 537 kB/s | 49 kB 00:00 2024-09-15T21:44:11.874 INFO:teuthology.orchestra.run.smithi100.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-09-15T21:44:11.957 INFO:teuthology.orchestra.run.smithi100.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-09-15T21:44:11.983 INFO:teuthology.orchestra.run.smithi100.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-15T21:44:12.008 INFO:teuthology.orchestra.run.smithi100.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-15T21:44:12.033 INFO:teuthology.orchestra.run.smithi100.stdout:(116/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.4 MB/s | 839 kB 00:00 2024-09-15T21:44:12.058 INFO:teuthology.orchestra.run.smithi100.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 282 kB/s | 14 kB 00:00 2024-09-15T21:44:12.084 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-15T21:44:12.093 INFO:teuthology.orchestra.run.smithi100.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-15T21:44:12.195 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T21:44:12.253 INFO:teuthology.orchestra.run.smithi100.stdout:(119/156): libarrow-9.0.0-11.el9.x86_64.rpm 7.6 MB/s | 4.4 MB 00:00 2024-09-15T21:44:12.300 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T21:44:12.328 INFO:teuthology.orchestra.run.smithi100.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 1.3 MB/s | 358 kB 00:00 2024-09-15T21:44:12.330 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T21:44:12.354 INFO:teuthology.orchestra.run.smithi100.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.1 MB/s | 303 kB 00:00 2024-09-15T21:44:12.379 INFO:teuthology.orchestra.run.smithi100.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 1.7 MB/s | 220 kB 00:00 2024-09-15T21:44:12.404 INFO:teuthology.orchestra.run.smithi100.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-15T21:44:12.420 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T21:44:12.430 INFO:teuthology.orchestra.run.smithi100.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-09-15T21:44:12.451 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T21:44:12.455 INFO:teuthology.orchestra.run.smithi100.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-15T21:44:12.480 INFO:teuthology.orchestra.run.smithi100.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-15T21:44:12.514 INFO:teuthology.orchestra.run.smithi100.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-09-15T21:44:12.531 INFO:teuthology.orchestra.run.smithi100.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 390 kB/s | 20 kB 00:00 2024-09-15T21:44:12.556 INFO:teuthology.orchestra.run.smithi100.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-15T21:44:12.590 INFO:teuthology.orchestra.run.smithi100.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-15T21:44:12.596 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-15T21:44:12.632 INFO:teuthology.orchestra.run.smithi100.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-09-15T21:44:12.657 INFO:teuthology.orchestra.run.smithi100.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 863 kB/s | 58 kB 00:00 2024-09-15T21:44:12.665 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-15T21:44:12.701 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T21:44:12.715 INFO:teuthology.orchestra.run.smithi100.stdout:(133/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.3 MB/s | 2.0 MB 00:00 2024-09-15T21:44:12.741 INFO:teuthology.orchestra.run.smithi100.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-09-15T21:44:12.766 INFO:teuthology.orchestra.run.smithi100.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-15T21:44:12.791 INFO:teuthology.orchestra.run.smithi100.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-15T21:44:12.804 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T21:44:12.817 INFO:teuthology.orchestra.run.smithi100.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-09-15T21:44:12.827 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T21:44:12.850 INFO:teuthology.orchestra.run.smithi100.stdout:(138/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.2 MB/s | 272 kB 00:00 2024-09-15T21:44:12.876 INFO:teuthology.orchestra.run.smithi100.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-15T21:44:12.890 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T21:44:12.901 INFO:teuthology.orchestra.run.smithi100.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-15T21:44:12.922 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T21:44:12.934 INFO:teuthology.orchestra.run.smithi100.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-09-15T21:44:12.944 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T21:44:12.968 INFO:teuthology.orchestra.run.smithi100.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 809 kB/s | 74 kB 00:00 2024-09-15T21:44:12.993 INFO:teuthology.orchestra.run.smithi100.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-15T21:44:13.018 INFO:teuthology.orchestra.run.smithi100.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-09-15T21:44:13.044 INFO:teuthology.orchestra.run.smithi100.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-15T21:44:13.078 INFO:teuthology.orchestra.run.smithi100.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-15T21:44:13.111 INFO:teuthology.orchestra.run.smithi100.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-15T21:44:13.138 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T21:44:13.195 INFO:teuthology.orchestra.run.smithi100.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-15T21:44:13.287 INFO:teuthology.orchestra.run.smithi100.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 25 MB/s | 2.3 MB 00:00 2024-09-15T21:44:13.291 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T21:44:13.371 INFO:teuthology.orchestra.run.smithi100.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 22 MB/s | 1.8 MB 00:00 2024-09-15T21:44:13.410 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-15T21:44:13.471 INFO:teuthology.orchestra.run.smithi100.stdout:(151/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 8.8 MB/s | 3.2 MB 00:00 2024-09-15T21:44:13.496 INFO:teuthology.orchestra.run.smithi100.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 326 kB/s | 41 kB 00:00 2024-09-15T21:44:13.510 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T21:44:13.588 INFO:teuthology.orchestra.run.smithi100.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 15 MB/s | 1.4 MB 00:00 2024-09-15T21:44:13.603 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T21:44:13.613 INFO:teuthology.orchestra.run.smithi100.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 1.7 MB/s | 249 kB 00:00 2024-09-15T21:44:13.680 INFO:teuthology.orchestra.run.smithi100.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-09-15T21:44:13.772 INFO:teuthology.orchestra.run.smithi100.stdout:(156/156): librados2-19.1.1-326.g8921c8ef.el9.x 4.9 MB/s | 3.4 MB 00:00 2024-09-15T21:44:13.779 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:44:13.779 INFO:teuthology.orchestra.run.smithi100.stdout:Total 6.8 MB/s | 226 MB 00:33 2024-09-15T21:44:13.824 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T21:44:13.937 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-15T21:44:14.039 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-15T21:44:14.075 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-15T21:44:14.110 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-15T21:44:14.192 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-15T21:44:14.244 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-15T21:44:14.276 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-15T21:44:14.320 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-15T21:44:14.356 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-15T21:44:14.383 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-15T21:44:14.451 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-15T21:44:14.505 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-15T21:44:14.537 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-15T21:44:14.580 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-15T21:44:14.657 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-15T21:44:14.691 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-15T21:44:14.743 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-15T21:44:14.780 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-15T21:44:14.849 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-15T21:44:14.894 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-15T21:44:14.967 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-15T21:44:14.994 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-09-15T21:44:15.099 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-15T21:44:15.148 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-09-15T21:44:15.148 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-09-15T21:44:15.155 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-15T21:44:15.682 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-15T21:44:15.760 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-15T21:44:15.789 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-15T21:44:15.832 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-15T21:44:15.869 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-15T21:44:15.919 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-15T21:44:16.079 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-15T21:44:16.139 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-15T21:44:16.195 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-15T21:44:16.269 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-15T21:44:16.300 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-15T21:44:16.337 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-15T21:44:16.383 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-15T21:44:16.419 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-15T21:44:16.462 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-15T21:44:16.513 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-15T21:44:16.657 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-15T21:44:16.698 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-15T21:44:16.741 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-15T21:44:16.842 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-15T21:44:16.873 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-15T21:44:16.915 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-15T21:44:16.939 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-15T21:44:16.964 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-15T21:44:17.047 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-15T21:44:17.174 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-15T21:44:17.221 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-15T21:44:17.249 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-15T21:44:17.283 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-15T21:44:17.323 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-15T21:44:17.362 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-15T21:44:17.393 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-15T21:44:17.427 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-15T21:44:17.460 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-15T21:44:17.482 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-09-15T21:44:17.482 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-09-15T21:44:17.510 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-15T21:44:17.554 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-15T21:44:17.640 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-15T21:44:17.692 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-15T21:44:17.725 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-15T21:44:17.828 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-15T21:44:17.884 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-15T21:44:18.602 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-15T21:44:18.648 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-15T21:44:18.691 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-15T21:44:18.789 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-15T21:44:18.846 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-15T21:44:18.890 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-15T21:44:18.971 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-15T21:44:19.010 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-15T21:44:19.194 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-15T21:44:19.225 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-15T21:44:19.302 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-15T21:44:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-15T21:44:20.056 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-15T21:44:21.280 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-15T21:44:21.379 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-15T21:44:21.406 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-15T21:44:21.439 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-15T21:44:21.749 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-15T21:44:21.794 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T21:44:21.884 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T21:44:21.916 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-15T21:44:21.952 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-15T21:44:22.444 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-15T21:44:22.473 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T21:44:22.543 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T21:44:22.574 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-15T21:44:23.772 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-09-15T21:44:23.876 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-15T21:44:23.914 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-15T21:44:23.980 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-15T21:44:24.018 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-15T21:44:24.044 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-15T21:44:24.095 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-15T21:44:24.427 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-15T21:44:24.553 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T21:44:24.669 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T21:44:24.696 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T21:44:24.787 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T21:44:24.832 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T21:44:24.893 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T21:44:24.930 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T21:44:25.008 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-15T21:44:25.073 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-15T21:44:25.101 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T21:44:25.197 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T21:44:25.227 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T21:44:25.273 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-15T21:44:25.303 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T21:44:25.322 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-15T21:44:25.472 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T21:44:25.477 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T21:44:25.533 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T21:44:25.625 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T21:44:25.625 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T21:44:25.625 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T21:44:25.626 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:44:25.659 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T21:44:25.677 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-15T21:44:25.809 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-15T21:44:25.930 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T21:44:26.003 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T21:44:26.221 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-15T21:44:26.323 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-15T21:44:26.420 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-15T21:44:26.459 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-15T21:44:26.501 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-15T21:44:26.564 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-15T21:44:26.610 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-15T21:44:26.642 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-15T21:44:26.685 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-15T21:44:26.718 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-15T21:44:26.746 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-15T21:44:26.814 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-15T21:44:26.869 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-15T21:44:26.909 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-15T21:44:26.951 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-15T21:44:27.027 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-15T21:44:27.062 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-15T21:44:27.106 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-15T21:44:27.135 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-15T21:44:27.204 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-15T21:44:27.249 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-15T21:44:27.322 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-15T21:44:27.462 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-15T21:44:27.517 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-15T21:44:28.034 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-15T21:44:28.122 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-15T21:44:28.160 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-15T21:44:28.203 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-15T21:44:28.241 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-15T21:44:28.299 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-15T21:44:28.458 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-15T21:44:28.519 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-15T21:44:28.573 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-15T21:44:28.654 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-15T21:44:28.695 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-15T21:44:28.731 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-15T21:44:28.778 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-15T21:44:28.817 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-15T21:44:28.860 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-15T21:44:28.902 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-15T21:44:29.037 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-15T21:44:29.079 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-15T21:44:29.120 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-15T21:44:29.224 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-15T21:44:29.263 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-15T21:44:29.312 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-15T21:44:29.345 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-15T21:44:29.369 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-15T21:44:29.474 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-15T21:44:29.594 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-15T21:44:29.627 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-15T21:44:29.655 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-15T21:44:29.698 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-15T21:44:29.737 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-15T21:44:29.776 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-15T21:44:29.806 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-15T21:44:29.841 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-15T21:44:29.874 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-15T21:44:29.915 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-15T21:44:29.950 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-15T21:44:30.038 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-15T21:44:30.091 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-15T21:44:30.122 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-15T21:44:30.225 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-15T21:44:30.281 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-15T21:44:31.007 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-15T21:44:31.053 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-15T21:44:31.087 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-15T21:44:31.186 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-15T21:44:31.234 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-15T21:44:31.288 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-15T21:44:31.376 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-15T21:44:31.407 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-15T21:44:31.592 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-15T21:44:31.621 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-15T21:44:31.699 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-15T21:44:32.338 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-15T21:44:32.452 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-15T21:44:33.696 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-15T21:44:33.791 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-15T21:44:33.820 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-15T21:44:33.862 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-15T21:44:34.173 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-15T21:44:34.233 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T21:44:34.539 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-15T21:44:34.707 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-15T21:44:34.751 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-15T21:44:35.235 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-15T21:44:35.263 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T21:44:35.334 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-15T21:44:35.364 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-15T21:44:37.592 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T21:44:37.636 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T21:44:37.709 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T21:44:37.709 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /sys 2024-09-15T21:44:37.709 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /proc 2024-09-15T21:44:37.710 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /mnt 2024-09-15T21:44:37.710 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /var/tmp 2024-09-15T21:44:37.710 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /home 2024-09-15T21:44:37.710 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /root 2024-09-15T21:44:37.710 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /tmp 2024-09-15T21:44:37.710 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:44:37.880 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T21:44:37.899 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T21:44:37.899 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:44:37.899 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T21:44:37.899 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:44:38.184 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-15T21:44:38.222 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T21:44:38.299 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-15T21:44:38.299 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T21:44:38.299 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T21:44:38.299 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:44:38.332 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T21:44:39.367 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T21:44:39.394 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T21:44:39.394 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:44:39.394 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T21:44:39.394 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T21:44:39.394 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T21:44:39.394 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:44:39.618 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T21:44:39.641 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T21:44:39.642 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:44:39.642 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T21:44:39.642 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T21:44:39.642 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T21:44:39.642 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:44:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T21:44:40.100 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T21:44:40.100 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:44:40.100 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T21:44:40.100 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T21:44:40.101 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T21:44:40.101 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:44:40.207 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-15T21:44:40.263 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-15T21:44:40.319 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-15T21:44:40.503 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-15T21:44:40.553 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-15T21:44:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-15T21:44:40.718 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-15T21:44:40.857 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-15T21:44:40.903 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-15T21:44:40.935 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-15T21:44:40.963 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-15T21:44:41.425 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T21:44:41.453 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T21:44:41.484 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-15T21:44:41.547 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-15T21:44:41.576 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T21:44:42.620 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T21:44:42.653 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T21:44:43.614 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T21:44:43.645 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T21:44:43.747 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T21:44:43.910 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-15T21:44:43.939 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T21:44:43.963 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T21:44:43.963 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:44:43.964 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T21:44:43.964 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T21:44:43.964 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T21:44:43.964 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:44:44.014 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T21:44:44.027 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T21:44:44.123 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-15T21:44:44.170 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-15T21:44:45.097 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-15T21:44:45.125 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T21:44:45.149 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T21:44:45.150 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:44:45.150 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T21:44:45.150 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:44:45.150 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:44:45.150 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:44:45.192 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T21:44:45.216 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T21:44:45.216 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:44:45.216 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T21:44:45.216 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:44:45.260 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T21:44:45.284 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T21:44:45.284 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:44:45.284 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-15T21:44:45.284 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T21:44:45.284 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T21:44:45.284 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:44:45.586 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T21:44:45.609 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T21:44:45.609 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:44:45.610 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T21:44:45.610 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T21:44:45.610 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T21:44:45.610 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:44:50.156 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-15T21:44:50.197 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-15T21:44:50.238 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-15T21:44:50.279 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-15T21:44:50.306 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-15T21:44:50.306 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /sys 2024-09-15T21:44:50.306 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /proc 2024-09-15T21:44:50.306 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /mnt 2024-09-15T21:44:50.306 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /var/tmp 2024-09-15T21:44:50.307 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /home 2024-09-15T21:44:50.307 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /root 2024-09-15T21:44:50.307 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /tmp 2024-09-15T21:44:50.307 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:44:50.314 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-15T21:44:50.495 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T21:44:50.495 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-15T21:44:50.510 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-15T21:44:50.510 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:44:50.510 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T21:44:50.510 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:44:50.536 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-15T21:44:50.566 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-15T21:44:50.595 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-15T21:44:50.650 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-15T21:44:51.205 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T21:44:51.240 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T21:44:51.261 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T21:44:51.998 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T21:44:52.024 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-15T21:44:52.025 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:44:52.025 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T21:44:52.025 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T21:44:52.025 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T21:44:52.025 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:44:52.086 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-15T21:44:52.178 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-15T21:44:52.219 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-15T21:44:52.265 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T21:44:52.288 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-15T21:44:52.288 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:44:52.288 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T21:44:52.288 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T21:44:52.288 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T21:44:52.288 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:44:52.302 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-15T21:44:52.340 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-15T21:44:52.365 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-15T21:44:52.391 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-15T21:44:52.391 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T21:44:52.475 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T21:44:52.475 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T21:44:52.559 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T21:44:52.584 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-15T21:44:52.593 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T21:44:52.593 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T21:44:52.733 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T21:44:52.756 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-15T21:44:52.756 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:44:52.756 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T21:44:52.756 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T21:44:52.756 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T21:44:52.757 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:44:52.778 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T21:44:52.778 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T21:44:52.862 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T21:44:52.862 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-15T21:44:52.919 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-15T21:44:52.924 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T21:44:52.974 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-15T21:44:53.159 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-15T21:44:53.209 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-15T21:44:53.286 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-15T21:44:53.374 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-15T21:44:53.512 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-15T21:44:53.582 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-15T21:44:53.615 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-15T21:44:53.643 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-15T21:44:54.073 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T21:44:54.109 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-15T21:44:54.140 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-15T21:44:54.203 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-15T21:44:54.240 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T21:44:55.294 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-15T21:44:55.326 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T21:44:56.294 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-15T21:44:56.326 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T21:44:56.426 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-15T21:44:56.603 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-15T21:44:56.628 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T21:44:56.652 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-15T21:44:56.653 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:44:56.653 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T21:44:56.653 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T21:44:56.653 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T21:44:56.653 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:44:56.704 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T21:44:56.715 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-15T21:44:56.815 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-15T21:44:56.885 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-15T21:44:57.823 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-15T21:44:57.856 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T21:44:57.880 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-15T21:44:57.880 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:44:57.880 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T21:44:57.880 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:44:57.880 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:44:57.880 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:44:57.924 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T21:44:57.947 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-15T21:44:57.947 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:44:57.947 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T21:44:57.947 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:44:57.991 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T21:44:58.014 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-15T21:44:58.014 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:44:58.014 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-15T21:44:58.014 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T21:44:58.015 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-15T21:44:58.015 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:44:58.325 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T21:44:58.348 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-15T21:44:58.348 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:44:58.348 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T21:44:58.349 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T21:44:58.349 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T21:44:58.349 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:44:58.804 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T21:44:58.804 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-15T21:44:58.804 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-15T21:44:58.804 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-15T21:44:58.804 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-15T21:44:58.804 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-15T21:44:58.804 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-15T21:44:58.805 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-15T21:44:58.805 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T21:44:58.805 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T21:44:58.805 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T21:44:58.805 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-15T21:44:58.805 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-15T21:44:58.805 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T21:44:58.805 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-15T21:44:58.805 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-15T21:44:58.805 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-15T21:44:58.805 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-15T21:44:58.805 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-15T21:44:58.806 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-15T21:44:58.806 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-15T21:44:58.806 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-15T21:44:58.806 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-15T21:44:58.806 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-15T21:44:58.806 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-15T21:44:58.806 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-15T21:44:58.806 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-15T21:44:58.806 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-15T21:44:58.806 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-15T21:44:58.807 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-15T21:44:58.807 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-15T21:44:58.807 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-15T21:44:58.807 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-15T21:44:58.834 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-15T21:44:58.834 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-15T21:44:58.835 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-15T21:44:58.835 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-15T21:44:58.835 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-15T21:44:58.835 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-15T21:44:58.835 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-15T21:44:58.835 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-15T21:44:58.836 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-15T21:44:58.836 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-15T21:44:58.836 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-15T21:44:58.836 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-15T21:44:58.836 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-15T21:44:58.836 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-15T21:44:58.836 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-15T21:44:58.836 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-15T21:44:58.836 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-15T21:44:58.837 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-15T21:44:58.837 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-15T21:44:58.837 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-15T21:44:58.837 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-15T21:44:58.837 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-15T21:44:58.837 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-15T21:44:58.837 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-15T21:44:58.837 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-15T21:44:58.837 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-15T21:44:58.837 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-15T21:44:58.837 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-15T21:44:58.837 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-15T21:44:58.838 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-15T21:44:58.838 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-15T21:44:58.838 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-15T21:44:58.838 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-15T21:44:58.838 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-15T21:44:58.838 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-15T21:44:58.838 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-15T21:44:58.838 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-15T21:44:58.838 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-15T21:44:58.838 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-15T21:44:58.838 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-15T21:44:58.838 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-15T21:44:58.838 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-15T21:44:58.839 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-15T21:44:58.839 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-15T21:44:58.839 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-15T21:44:58.839 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-15T21:44:58.839 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-15T21:44:58.839 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-15T21:44:58.839 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-15T21:44:58.839 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-15T21:44:58.839 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-15T21:44:58.839 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-15T21:44:58.839 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-15T21:44:58.839 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-15T21:44:58.840 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-15T21:44:58.840 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-15T21:44:58.840 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-15T21:44:58.840 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-15T21:44:58.840 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-15T21:44:58.840 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-15T21:44:58.840 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-15T21:44:58.840 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-15T21:44:58.840 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-15T21:44:58.840 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-15T21:44:58.840 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-15T21:44:58.840 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-15T21:44:58.840 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-15T21:44:58.841 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-15T21:44:58.841 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-15T21:44:58.841 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-15T21:44:58.841 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-15T21:44:58.841 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-15T21:44:58.841 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-15T21:44:58.841 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-15T21:44:58.841 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-15T21:44:58.841 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-15T21:44:58.841 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-15T21:44:58.841 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-15T21:44:58.841 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-15T21:44:58.841 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-15T21:44:58.842 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-15T21:44:58.842 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-15T21:44:58.842 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-15T21:44:58.842 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-15T21:44:58.842 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-15T21:44:58.842 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-15T21:44:58.842 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-15T21:44:58.842 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-15T21:44:58.842 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-15T21:44:58.842 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-15T21:44:58.842 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-15T21:44:58.843 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-15T21:44:58.843 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-15T21:44:58.843 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-15T21:44:58.843 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-15T21:44:58.843 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-15T21:44:58.843 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-15T21:44:58.843 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-15T21:44:58.843 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-15T21:44:58.843 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-15T21:44:58.843 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-15T21:44:58.843 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-15T21:44:58.843 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-15T21:44:58.843 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-15T21:44:58.844 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-15T21:44:58.844 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-15T21:44:58.844 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-15T21:44:58.844 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-15T21:44:58.844 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-15T21:44:58.844 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-15T21:44:58.844 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-15T21:44:58.844 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-15T21:44:58.844 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-15T21:44:58.844 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-15T21:44:58.844 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-15T21:44:58.844 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-15T21:44:58.844 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-15T21:44:58.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-15T21:44:58.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-15T21:44:58.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-15T21:44:58.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-15T21:44:58.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-15T21:44:58.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-15T21:44:58.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-15T21:44:58.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-15T21:44:58.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-15T21:44:58.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-15T21:44:58.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-15T21:44:58.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-15T21:44:58.845 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-15T21:44:58.846 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-15T21:45:01.424 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T21:45:01.424 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:01.424 INFO:teuthology.orchestra.run.smithi122.stdout:Upgraded: 2024-09-15T21:45:01.424 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-15T21:45:01.424 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-15T21:45:01.424 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-15T21:45:01.425 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-15T21:45:01.425 INFO:teuthology.orchestra.run.smithi122.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.425 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.425 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-15T21:45:01.425 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-15T21:45:01.425 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-15T21:45:01.425 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T21:45:01.425 INFO:teuthology.orchestra.run.smithi122.stdout: automake-1.16.2-8.el9.noarch 2024-09-15T21:45:01.425 INFO:teuthology.orchestra.run.smithi122.stdout: bison-3.7.4-5.el9.x86_64 2024-09-15T21:45:01.425 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T21:45:01.425 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.425 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.425 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.425 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.426 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:01.426 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.426 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.426 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.426 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:01.426 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:01.426 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:01.426 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:01.426 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:01.426 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.426 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.426 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:01.426 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.427 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.427 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.427 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:01.427 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:01.427 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.427 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.427 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:01.427 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-15T21:45:01.427 INFO:teuthology.orchestra.run.smithi122.stdout: flex-2.6.4-9.el9.x86_64 2024-09-15T21:45:01.427 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T21:45:01.427 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T21:45:01.427 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T21:45:01.427 INFO:teuthology.orchestra.run.smithi122.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-15T21:45:01.428 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T21:45:01.428 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T21:45:01.428 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-15T21:45:01.428 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T21:45:01.428 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T21:45:01.428 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.428 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.428 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.428 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T21:45:01.428 INFO:teuthology.orchestra.run.smithi122.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T21:45:01.428 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T21:45:01.428 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T21:45:01.428 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T21:45:01.428 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.429 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.429 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T21:45:01.429 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.429 INFO:teuthology.orchestra.run.smithi122.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-15T21:45:01.429 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T21:45:01.429 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-15T21:45:01.429 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-15T21:45:01.429 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T21:45:01.429 INFO:teuthology.orchestra.run.smithi122.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T21:45:01.429 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T21:45:01.429 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T21:45:01.429 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T21:45:01.429 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T21:45:01.430 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-15T21:45:01.430 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T21:45:01.430 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-13.16-1.el9.x86_64 2024-09-15T21:45:01.430 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-15T21:45:01.430 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-15T21:45:01.430 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-15T21:45:01.430 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T21:45:01.430 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T21:45:01.430 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T21:45:01.430 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-15T21:45:01.430 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T21:45:01.430 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T21:45:01.430 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T21:45:01.431 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.431 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.431 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.431 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T21:45:01.431 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T21:45:01.431 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T21:45:01.431 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T21:45:01.431 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-15T21:45:01.431 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-15T21:45:01.431 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T21:45:01.431 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T21:45:01.431 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T21:45:01.431 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T21:45:01.431 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T21:45:01.432 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T21:45:01.432 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T21:45:01.432 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T21:45:01.432 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T21:45:01.432 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T21:45:01.432 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T21:45:01.432 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T21:45:01.432 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T21:45:01.432 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T21:45:01.432 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T21:45:01.432 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T21:45:01.432 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T21:45:01.432 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T21:45:01.433 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T21:45:01.433 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T21:45:01.433 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T21:45:01.433 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T21:45:01.433 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T21:45:01.433 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T21:45:01.433 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T21:45:01.433 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T21:45:01.433 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T21:45:01.433 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T21:45:01.433 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T21:45:01.433 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T21:45:01.433 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T21:45:01.434 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T21:45:01.434 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-15T21:45:01.434 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T21:45:01.434 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.434 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.434 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T21:45:01.434 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T21:45:01.434 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T21:45:01.434 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.434 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T21:45:01.434 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T21:45:01.434 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T21:45:01.434 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T21:45:01.434 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T21:45:01.435 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T21:45:01.435 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T21:45:01.435 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-15T21:45:01.435 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T21:45:01.435 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T21:45:01.435 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T21:45:01.435 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T21:45:01.435 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T21:45:01.435 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T21:45:01.435 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T21:45:01.435 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.435 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.436 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:01.436 INFO:teuthology.orchestra.run.smithi122.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T21:45:01.436 INFO:teuthology.orchestra.run.smithi122.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T21:45:01.436 INFO:teuthology.orchestra.run.smithi122.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T21:45:01.436 INFO:teuthology.orchestra.run.smithi122.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-15T21:45:01.436 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-15T21:45:01.436 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T21:45:01.436 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-15T21:45:01.437 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:01.437 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-15T21:45:01.827 DEBUG:teuthology.parallel:result is None 2024-09-15T21:45:02.945 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-15T21:45:02.986 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-15T21:45:03.035 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-15T21:45:03.076 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-15T21:45:03.111 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-15T21:45:03.317 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-15T21:45:03.367 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-15T21:45:03.398 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-15T21:45:03.427 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-15T21:45:03.465 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-15T21:45:04.022 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T21:45:04.063 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T21:45:04.083 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-15T21:45:04.922 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-15T21:45:05.018 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-15T21:45:05.068 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-15T21:45:05.169 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-15T21:45:05.204 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-15T21:45:05.229 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-15T21:45:05.255 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-15T21:45:05.256 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T21:45:05.340 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-15T21:45:05.340 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T21:45:05.424 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-15T21:45:05.449 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-15T21:45:05.454 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T21:45:05.454 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T21:45:05.642 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-15T21:45:05.643 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T21:45:05.732 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-15T21:45:05.801 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T21:45:11.672 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T21:45:11.672 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-15T21:45:11.672 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-15T21:45:11.672 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-15T21:45:11.672 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-15T21:45:11.672 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-15T21:45:11.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-15T21:45:11.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-15T21:45:11.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-15T21:45:11.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-15T21:45:11.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-15T21:45:11.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-15T21:45:11.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-15T21:45:11.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-15T21:45:11.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-15T21:45:11.673 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-15T21:45:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-15T21:45:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-15T21:45:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-15T21:45:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-15T21:45:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-15T21:45:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-15T21:45:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-15T21:45:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-15T21:45:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-15T21:45:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-15T21:45:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-15T21:45:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-15T21:45:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-15T21:45:11.674 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-15T21:45:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-15T21:45:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-15T21:45:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-15T21:45:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-15T21:45:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-15T21:45:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-15T21:45:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-15T21:45:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-15T21:45:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-15T21:45:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-15T21:45:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-15T21:45:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-15T21:45:11.675 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-15T21:45:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-15T21:45:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-15T21:45:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-15T21:45:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-15T21:45:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-15T21:45:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-15T21:45:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-15T21:45:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-15T21:45:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-15T21:45:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-15T21:45:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-15T21:45:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-15T21:45:11.676 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-15T21:45:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-15T21:45:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-15T21:45:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-15T21:45:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-15T21:45:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-15T21:45:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-15T21:45:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-15T21:45:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-15T21:45:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-15T21:45:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-15T21:45:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-15T21:45:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-15T21:45:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-15T21:45:11.677 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-15T21:45:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-15T21:45:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-15T21:45:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-15T21:45:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-15T21:45:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-15T21:45:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-15T21:45:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-15T21:45:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-15T21:45:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-15T21:45:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-15T21:45:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-15T21:45:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-15T21:45:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-15T21:45:11.678 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-15T21:45:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-15T21:45:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-15T21:45:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-15T21:45:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-15T21:45:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-15T21:45:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-15T21:45:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-15T21:45:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-15T21:45:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-15T21:45:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-15T21:45:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-15T21:45:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-15T21:45:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-15T21:45:11.679 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-15T21:45:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-15T21:45:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-15T21:45:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-15T21:45:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-15T21:45:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-15T21:45:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-15T21:45:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-15T21:45:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-15T21:45:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-15T21:45:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-15T21:45:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-15T21:45:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-15T21:45:11.680 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-15T21:45:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-15T21:45:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-15T21:45:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-15T21:45:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-15T21:45:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-15T21:45:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-15T21:45:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-15T21:45:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-15T21:45:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-15T21:45:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-15T21:45:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-15T21:45:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-15T21:45:11.681 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-15T21:45:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-15T21:45:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-15T21:45:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-15T21:45:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-15T21:45:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-15T21:45:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-15T21:45:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-15T21:45:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-15T21:45:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-15T21:45:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-15T21:45:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-15T21:45:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-15T21:45:11.682 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-15T21:45:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-15T21:45:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-15T21:45:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-15T21:45:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-15T21:45:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-15T21:45:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-15T21:45:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-15T21:45:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-15T21:45:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-15T21:45:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-15T21:45:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-15T21:45:11.683 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-15T21:45:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-15T21:45:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-15T21:45:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-15T21:45:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-15T21:45:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-15T21:45:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-15T21:45:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-15T21:45:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-15T21:45:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-15T21:45:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-15T21:45:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-15T21:45:11.684 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-15T21:45:11.685 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-15T21:45:11.685 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-15T21:45:11.685 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-15T21:45:14.426 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-15T21:45:14.426 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:14.427 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2024-09-15T21:45:14.427 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-15T21:45:14.427 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-15T21:45:14.427 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-15T21:45:14.427 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-15T21:45:14.427 INFO:teuthology.orchestra.run.smithi100.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.427 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.427 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-15T21:45:14.427 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-15T21:45:14.427 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-09-15T21:45:14.427 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T21:45:14.427 INFO:teuthology.orchestra.run.smithi100.stdout: automake-1.16.2-8.el9.noarch 2024-09-15T21:45:14.427 INFO:teuthology.orchestra.run.smithi100.stdout: bison-3.7.4-5.el9.x86_64 2024-09-15T21:45:14.428 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T21:45:14.428 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.428 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.428 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.428 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.428 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:14.428 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.428 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.428 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.428 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:14.428 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:14.428 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:14.428 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:14.429 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:14.429 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.429 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.429 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:14.429 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.429 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.429 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.429 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:14.429 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:14.429 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.429 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.429 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-15T21:45:14.429 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-15T21:45:14.430 INFO:teuthology.orchestra.run.smithi100.stdout: flex-2.6.4-9.el9.x86_64 2024-09-15T21:45:14.430 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T21:45:14.430 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T21:45:14.430 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T21:45:14.430 INFO:teuthology.orchestra.run.smithi100.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-15T21:45:14.430 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T21:45:14.430 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T21:45:14.430 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-15T21:45:14.430 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T21:45:14.430 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T21:45:14.430 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.430 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.430 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.431 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T21:45:14.431 INFO:teuthology.orchestra.run.smithi100.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T21:45:14.431 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T21:45:14.431 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T21:45:14.431 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T21:45:14.431 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.431 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.431 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T21:45:14.431 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.431 INFO:teuthology.orchestra.run.smithi100.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-15T21:45:14.431 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T21:45:14.431 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-15T21:45:14.431 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-15T21:45:14.431 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T21:45:14.432 INFO:teuthology.orchestra.run.smithi100.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T21:45:14.432 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T21:45:14.432 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T21:45:14.432 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T21:45:14.432 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T21:45:14.432 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-15T21:45:14.432 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T21:45:14.432 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-13.16-1.el9.x86_64 2024-09-15T21:45:14.432 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-15T21:45:14.432 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-15T21:45:14.432 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-15T21:45:14.432 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T21:45:14.432 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T21:45:14.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T21:45:14.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-15T21:45:14.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T21:45:14.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T21:45:14.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T21:45:14.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T21:45:14.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T21:45:14.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T21:45:14.433 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T21:45:14.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-15T21:45:14.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-15T21:45:14.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T21:45:14.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T21:45:14.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T21:45:14.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T21:45:14.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T21:45:14.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T21:45:14.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T21:45:14.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T21:45:14.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T21:45:14.434 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T21:45:14.435 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T21:45:14.435 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T21:45:14.435 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T21:45:14.435 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T21:45:14.435 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T21:45:14.435 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T21:45:14.435 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T21:45:14.435 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T21:45:14.435 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T21:45:14.435 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T21:45:14.435 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T21:45:14.435 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T21:45:14.435 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T21:45:14.435 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T21:45:14.436 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T21:45:14.436 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T21:45:14.436 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T21:45:14.436 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T21:45:14.436 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T21:45:14.436 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T21:45:14.436 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T21:45:14.436 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T21:45:14.436 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-15T21:45:14.436 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T21:45:14.436 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.436 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.436 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T21:45:14.437 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T21:45:14.437 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T21:45:14.437 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.437 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T21:45:14.437 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T21:45:14.437 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T21:45:14.437 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T21:45:14.437 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T21:45:14.437 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T21:45:14.437 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T21:45:14.437 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-15T21:45:14.437 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T21:45:14.437 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T21:45:14.438 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T21:45:14.438 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T21:45:14.438 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T21:45:14.438 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T21:45:14.438 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T21:45:14.438 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.438 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.438 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-15T21:45:14.438 INFO:teuthology.orchestra.run.smithi100.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T21:45:14.438 INFO:teuthology.orchestra.run.smithi100.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T21:45:14.438 INFO:teuthology.orchestra.run.smithi100.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T21:45:14.438 INFO:teuthology.orchestra.run.smithi100.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-15T21:45:14.438 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-15T21:45:14.439 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T21:45:14.439 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-15T21:45:14.439 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:14.439 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-09-15T21:45:14.863 DEBUG:teuthology.parallel:result is None 2024-09-15T21:45:14.864 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:45:15.008 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T21:45:15.030 INFO:teuthology.orchestra.run.smithi100.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T21:45:15.031 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T21:45:15.031 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T21:45:15.033 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-15T21:45:15.170 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T21:45:15.197 INFO:teuthology.orchestra.run.smithi122.stdout:19.1.1-326.g8921c8ef.el9 2024-09-15T21:45:15.197 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-15T21:45:15.197 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-15T21:45:15.200 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T21:45:15.200 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:15.200 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T21:45:15.228 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:45:15.228 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T21:45:15.267 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T21:45:15.267 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:15.268 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T21:45:15.292 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T21:45:15.357 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:45:15.357 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T21:45:15.384 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T21:45:15.451 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T21:45:15.451 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:15.451 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T21:45:15.476 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T21:45:15.544 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:45:15.544 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T21:45:15.570 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T21:45:15.637 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T21:45:15.637 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:15.637 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T21:45:15.662 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T21:45:15.728 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:45:15.728 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T21:45:15.753 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T21:45:15.819 INFO:teuthology.run_tasks:Running task ceph... 2024-09-15T21:45:15.902 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-15T21:45:15.902 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /var/log/ceph 2024-09-15T21:45:15.905 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /var/log/ceph 2024-09-15T21:45:15.930 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-15T21:45:15.930 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T21:45:15.971 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T21:45:15.996 INFO:tasks.ceph:Creating extra log directories... 2024-09-15T21:45:15.996 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T21:45:16.036 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T21:45:16.069 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-15T21:45:16.069 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-15T21:45:16.120 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-15T21:45:16.121 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:16.121 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-15T21:45:16.136 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-15T21:45:16.251 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-15T21:45:16.277 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:45:16.277 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-15T21:45:16.294 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-15T21:45:16.406 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-15T21:45:16.443 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-15T21:45:16.443 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-15T21:45:16.444 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906191', 'branch': 'squid', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7906191', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'ad35db06e8924912f01e32b35576297c0f1352b4'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 3393, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'sleep_before_teardown': 0, 'subset': '294/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'ad35db06e8924912f01e32b35576297c0f1352b4', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi100.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBMAlTIStuHpllTVDu5vTqc1TSc4GR/wA2rUXLINU8t6s+N6zad/NtqWnt2V7UlL4r/j7H4SWh43jPZtTxS8lbc=', 'smithi122.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKLhAQxZqaDOGfCotacJm7nV2Q2xqd7kK7xMd5ZtKLmIIx8322AOCgQpz6UbHVKCd9O80mluLSPBnT2PsLceozc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'thrashosds': None}, {'workunit': {'clients': {'all': ['fs/misc/trivial_sync.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_21:24:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-15T21:45:16.445 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T21:45:16.448 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T21:45:16.488 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T21:45:16.504 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T21:45:16.552 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:16.553 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T21:45:16.583 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T21:45:16.583 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-09-15T21:45:16.639 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T21:45:16.639 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:45:16.639 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-15T21:45:16.639 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:45:16.639 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:45:16.639 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-15 21:45:06.158491739 +0000 2024-09-15T21:45:16.639 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-15 21:42:43.420207041 +0000 2024-09-15T21:45:16.639 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-15 21:42:43.420207041 +0000 2024-09-15T21:45:16.639 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-15 21:42:43.420207041 +0000 2024-09-15T21:45:16.640 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T21:45:16.701 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-15T21:45:16.702 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-15T21:45:16.702 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000183858 s, 2.8 MB/s 2024-09-15T21:45:16.704 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T21:45:16.761 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-09-15T21:45:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T21:45:16.817 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:45:16.817 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-15T21:45:16.817 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:45:16.817 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:45:16.817 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-15 21:45:06.158491739 +0000 2024-09-15T21:45:16.817 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-15 21:42:43.758198106 +0000 2024-09-15T21:45:16.818 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-15 21:42:43.758198106 +0000 2024-09-15T21:45:16.818 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-15 21:42:43.758198106 +0000 2024-09-15T21:45:16.818 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T21:45:16.881 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-15T21:45:16.881 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-15T21:45:16.881 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.00024667 s, 2.1 MB/s 2024-09-15T21:45:16.882 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T21:45:16.939 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-09-15T21:45:16.995 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T21:45:16.996 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:45:16.996 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-15T21:45:16.996 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:45:16.996 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:45:16.996 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-15 21:45:06.158491739 +0000 2024-09-15T21:45:16.997 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-15 21:42:44.082189542 +0000 2024-09-15T21:45:16.997 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-15 21:42:44.082189542 +0000 2024-09-15T21:45:16.997 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-15 21:42:44.082189542 +0000 2024-09-15T21:45:16.997 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T21:45:17.059 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-15T21:45:17.059 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-15T21:45:17.059 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000227968 s, 2.2 MB/s 2024-09-15T21:45:17.060 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T21:45:17.117 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-09-15T21:45:17.173 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T21:45:17.173 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:45:17.173 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-15T21:45:17.173 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:45:17.173 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:45:17.173 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-09-15 21:45:06.159491713 +0000 2024-09-15T21:45:17.173 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-15 21:42:44.400181136 +0000 2024-09-15T21:45:17.173 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-15 21:42:44.400181136 +0000 2024-09-15T21:45:17.174 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-15 21:42:44.400181136 +0000 2024-09-15T21:45:17.174 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T21:45:17.237 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-09-15T21:45:17.237 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-09-15T21:45:17.237 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000191727 s, 2.7 MB/s 2024-09-15T21:45:17.238 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T21:45:17.295 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T21:45:17.295 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:45:17.295 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T21:45:17.311 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T21:45:17.312 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-09-15T21:45:17.367 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T21:45:17.367 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:45:17.367 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-15T21:45:17.367 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:45:17.368 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:45:17.368 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-15 21:44:53.312784062 +0000 2024-09-15T21:45:17.368 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-15 21:42:59.345275009 +0000 2024-09-15T21:45:17.368 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-15 21:42:59.345275009 +0000 2024-09-15T21:45:17.368 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-15 21:42:59.345275009 +0000 2024-09-15T21:45:17.368 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T21:45:17.431 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-15T21:45:17.431 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-15T21:45:17.431 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000263844 s, 1.9 MB/s 2024-09-15T21:45:17.432 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T21:45:17.489 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-09-15T21:45:17.545 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T21:45:17.545 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:45:17.545 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-15T21:45:17.545 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:45:17.545 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:45:17.545 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-15 21:44:53.313784038 +0000 2024-09-15T21:45:17.545 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-15 21:42:59.662267197 +0000 2024-09-15T21:45:17.545 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-15 21:42:59.662267197 +0000 2024-09-15T21:45:17.545 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-15 21:42:59.662267197 +0000 2024-09-15T21:45:17.546 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T21:45:17.609 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-15T21:45:17.609 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-15T21:45:17.609 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000240104 s, 2.1 MB/s 2024-09-15T21:45:17.610 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T21:45:17.667 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-09-15T21:45:17.722 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T21:45:17.722 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:45:17.722 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-15T21:45:17.722 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:45:17.722 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:45:17.722 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-15 21:44:53.313784038 +0000 2024-09-15T21:45:17.722 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-15 21:42:59.992259064 +0000 2024-09-15T21:45:17.722 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-15 21:42:59.992259064 +0000 2024-09-15T21:45:17.722 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-15 21:42:59.992259064 +0000 2024-09-15T21:45:17.722 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T21:45:17.784 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-15T21:45:17.784 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-15T21:45:17.784 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000217157 s, 2.4 MB/s 2024-09-15T21:45:17.785 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T21:45:17.842 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-09-15T21:45:17.897 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T21:45:17.897 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:45:17.897 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-15T21:45:17.897 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:45:17.897 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:45:17.897 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-15 21:44:53.313784038 +0000 2024-09-15T21:45:17.897 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-15 21:43:00.327250808 +0000 2024-09-15T21:45:17.897 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-15 21:43:00.327250808 +0000 2024-09-15T21:45:17.897 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-15 21:43:00.327250808 +0000 2024-09-15T21:45:17.898 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T21:45:17.959 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-15T21:45:17.960 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-15T21:45:17.960 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000200678 s, 2.6 MB/s 2024-09-15T21:45:17.961 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T21:45:18.016 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T21:45:18.017 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-15T21:45:18.017 INFO:tasks.ceph:Generating config... 2024-09-15T21:45:18.023 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-15T21:45:18.024 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-09-15T21:45:18.024 INFO:tasks.ceph:[client] debug client = 20 2024-09-15T21:45:18.024 INFO:tasks.ceph:[client] debug ms = 1 2024-09-15T21:45:18.024 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-15T21:45:18.025 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-15T21:45:18.025 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-15T21:45:18.025 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-15T21:45:18.025 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-15T21:45:18.025 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-15T21:45:18.025 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2024-09-15T21:45:18.025 INFO:tasks.ceph:[mds] mds bal merge size = 5 2024-09-15T21:45:18.025 INFO:tasks.ceph:[mds] mds bal split bits = 3 2024-09-15T21:45:18.025 INFO:tasks.ceph:[mds] mds bal split size = 100 2024-09-15T21:45:18.025 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-15T21:45:18.025 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-15T21:45:18.026 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-09-15T21:45:18.026 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-15T21:45:18.026 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-15T21:45:18.026 INFO:tasks.ceph:[mds] mds_dir_prefetch = True 2024-09-15T21:45:18.026 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-09-15T21:45:18.026 INFO:tasks.ceph:[mds] mon client directed command retry = 5 2024-09-15T21:45:18.026 INFO:tasks.ceph:[mds] ms inject delay max = 1 2024-09-15T21:45:18.026 INFO:tasks.ceph:[mds] ms inject delay probability = 0.005 2024-09-15T21:45:18.026 INFO:tasks.ceph:[mds] ms inject delay type = client mds osd 2024-09-15T21:45:18.026 INFO:tasks.ceph:[mds] ms inject socket failures = 2500 2024-09-15T21:45:18.026 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-15T21:45:18.026 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-15T21:45:18.026 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-15T21:45:18.026 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-15T21:45:18.026 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-15T21:45:18.027 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-15T21:45:18.027 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-15T21:45:18.027 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-15T21:45:18.027 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-15T21:45:18.027 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-15T21:45:18.027 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-15T21:45:18.027 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-15T21:45:18.027 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-15T21:45:18.027 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-15T21:45:18.027 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-15T21:45:18.027 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-15T21:45:18.027 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-15T21:45:18.027 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-15T21:45:18.027 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-15T21:45:18.028 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-15T21:45:18.028 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-15T21:45:18.028 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-15T21:45:18.028 INFO:tasks.ceph:[osd] mon client directed command retry = 5 2024-09-15T21:45:18.028 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-15T21:45:18.028 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-15T21:45:18.028 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-15T21:45:18.028 INFO:tasks.ceph:[osd] ms inject delay max = 1 2024-09-15T21:45:18.028 INFO:tasks.ceph:[osd] ms inject delay probability = 0.005 2024-09-15T21:45:18.028 INFO:tasks.ceph:[osd] ms inject delay type = client mds 2024-09-15T21:45:18.028 INFO:tasks.ceph:[osd] ms inject socket failures = 2500 2024-09-15T21:45:18.028 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-15T21:45:18.028 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-15T21:45:18.028 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-15T21:45:18.029 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-15T21:45:18.029 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-15T21:45:18.029 INFO:tasks.ceph:Setting up mon.a... 2024-09-15T21:45:18.029 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-15T21:45:18.087 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.keyring 2024-09-15T21:45:18.090 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-15T21:45:18.134 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T21:45:18.160 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.100'), ('mon.b', '172.21.15.122'), ('mon.c', '[v2:172.21.15.122:3301,v1:172.21.15.122:6790]')] 2024-09-15T21:45:18.160 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.100,172.21.15.122,[v2:172.21.15.122:3301,v1:172.21.15.122:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-15T21:45:18.161 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:18.161 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T21:45:18.215 DEBUG:teuthology.orchestra.run.smithi100:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.100 --add b 172.21.15.122 --addv c '[v2:172.21.15.122:3301,v1:172.21.15.122:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:45:18.303 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:45:18.304 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool: generated fsid 30e2fff4-2171-4531-8806-0e03b63f494f 2024-09-15T21:45:18.304 INFO:teuthology.orchestra.run.smithi100.stdout:setting min_mon_release = quincy 2024-09-15T21:45:18.304 INFO:teuthology.orchestra.run.smithi100.stdout:epoch 0 2024-09-15T21:45:18.304 INFO:teuthology.orchestra.run.smithi100.stdout:fsid 30e2fff4-2171-4531-8806-0e03b63f494f 2024-09-15T21:45:18.304 INFO:teuthology.orchestra.run.smithi100.stdout:last_changed 2024-09-15T21:45:18.302698+0000 2024-09-15T21:45:18.304 INFO:teuthology.orchestra.run.smithi100.stdout:created 2024-09-15T21:45:18.302698+0000 2024-09-15T21:45:18.304 INFO:teuthology.orchestra.run.smithi100.stdout:min_mon_release 17 (quincy) 2024-09-15T21:45:18.304 INFO:teuthology.orchestra.run.smithi100.stdout:election_strategy: 1 2024-09-15T21:45:18.304 INFO:teuthology.orchestra.run.smithi100.stdout:0: [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] mon.a 2024-09-15T21:45:18.304 INFO:teuthology.orchestra.run.smithi100.stdout:1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.b 2024-09-15T21:45:18.304 INFO:teuthology.orchestra.run.smithi100.stdout:2: [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] mon.c 2024-09-15T21:45:18.305 INFO:teuthology.orchestra.run.smithi100.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-15T21:45:18.306 DEBUG:teuthology.orchestra.run.smithi100:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T21:45:18.321 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 30e2fff4-2171-4531-8806-0e03b63f494f... 2024-09-15T21:45:18.322 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T21:45:18.363 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T21:45:18.403 INFO:teuthology.orchestra.run.smithi100.stdout:[global] 2024-09-15T21:45:18.404 INFO:teuthology.orchestra.run.smithi100.stdout: chdir = "" 2024-09-15T21:45:18.404 INFO:teuthology.orchestra.run.smithi100.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T21:45:18.404 INFO:teuthology.orchestra.run.smithi100.stdout: auth supported = cephx 2024-09-15T21:45:18.404 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.404 INFO:teuthology.orchestra.run.smithi100.stdout: filestore xattr use omap = true 2024-09-15T21:45:18.404 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.404 INFO:teuthology.orchestra.run.smithi100.stdout: mon clock drift allowed = 1.000 2024-09-15T21:45:18.404 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.404 INFO:teuthology.orchestra.run.smithi100.stdout: osd crush chooseleaf type = 0 2024-09-15T21:45:18.405 INFO:teuthology.orchestra.run.smithi100.stdout: auth debug = true 2024-09-15T21:45:18.405 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.405 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on old message = true 2024-09-15T21:45:18.405 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on bug = true 2024-09-15T21:45:18.405 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.405 INFO:teuthology.orchestra.run.smithi100.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T21:45:18.405 INFO:teuthology.orchestra.run.smithi100.stdout: mon pg warn max object skew = 0 2024-09-15T21:45:18.405 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.405 INFO:teuthology.orchestra.run.smithi100.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T21:45:18.405 INFO:teuthology.orchestra.run.smithi100.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T21:45:18.405 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.405 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default size = 2 2024-09-15T21:45:18.405 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.406 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow primary affinity = true 2024-09-15T21:45:18.406 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd allow pg remap = true 2024-09-15T21:45:18.406 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on legacy crush tunables = false 2024-09-15T21:45:18.406 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on crush straw calc version zero = false 2024-09-15T21:45:18.406 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on no sortbitwise = false 2024-09-15T21:45:18.406 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on osd down out interval zero = false 2024-09-15T21:45:18.406 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on too few osds = false 2024-09-15T21:45:18.406 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T21:45:18.406 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T21:45:18.406 INFO:teuthology.orchestra.run.smithi100.stdout: mon_allow_pool_size_one = true 2024-09-15T21:45:18.406 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.406 INFO:teuthology.orchestra.run.smithi100.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T21:45:18.406 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.406 INFO:teuthology.orchestra.run.smithi100.stdout: osd default data pool replay window = 5 2024-09-15T21:45:18.407 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.407 INFO:teuthology.orchestra.run.smithi100.stdout: mon allow pool delete = true 2024-09-15T21:45:18.407 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.407 INFO:teuthology.orchestra.run.smithi100.stdout: mon cluster log file level = debug 2024-09-15T21:45:18.407 INFO:teuthology.orchestra.run.smithi100.stdout: debug asserts on shutdown = true 2024-09-15T21:45:18.407 INFO:teuthology.orchestra.run.smithi100.stdout: mon health detail to clog = false 2024-09-15T21:45:18.407 INFO:teuthology.orchestra.run.smithi100.stdout: mon host = "172.21.15.100,172.21.15.122,[v2:172.21.15.122:3301,v1:172.21.15.122:6790]" 2024-09-15T21:45:18.407 INFO:teuthology.orchestra.run.smithi100.stdout: ms die on skipped message = False 2024-09-15T21:45:18.407 INFO:teuthology.orchestra.run.smithi100.stdout: fsid = 30e2fff4-2171-4531-8806-0e03b63f494f 2024-09-15T21:45:18.407 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.407 INFO:teuthology.orchestra.run.smithi100.stdout:[osd] 2024-09-15T21:45:18.407 INFO:teuthology.orchestra.run.smithi100.stdout: osd journal size = 100 2024-09-15T21:45:18.407 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.408 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub load threshold = 5.0 2024-09-15T21:45:18.408 INFO:teuthology.orchestra.run.smithi100.stdout: osd scrub max interval = 600 2024-09-15T21:45:18.408 INFO:teuthology.orchestra.run.smithi100.stdout: osd mclock profile = high_recovery_ops 2024-09-15T21:45:18.408 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.408 INFO:teuthology.orchestra.run.smithi100.stdout: osd recover clone overlap = true 2024-09-15T21:45:18.408 INFO:teuthology.orchestra.run.smithi100.stdout: osd recovery max chunk = 1048576 2024-09-15T21:45:18.408 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.408 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug shutdown = true 2024-09-15T21:45:18.408 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug op order = true 2024-09-15T21:45:18.408 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug verify stray on activate = true 2024-09-15T21:45:18.408 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug trim objects = true 2024-09-15T21:45:18.408 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.408 INFO:teuthology.orchestra.run.smithi100.stdout: osd open classes on start = true 2024-09-15T21:45:18.408 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug pg log writeout = true 2024-09-15T21:45:18.409 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.409 INFO:teuthology.orchestra.run.smithi100.stdout: osd deep scrub update digest min age = 30 2024-09-15T21:45:18.409 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.409 INFO:teuthology.orchestra.run.smithi100.stdout: osd map max advance = 10 2024-09-15T21:45:18.409 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.409 INFO:teuthology.orchestra.run.smithi100.stdout: journal zero on create = true 2024-09-15T21:45:18.409 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.409 INFO:teuthology.orchestra.run.smithi100.stdout: filestore ondisk finisher threads = 3 2024-09-15T21:45:18.409 INFO:teuthology.orchestra.run.smithi100.stdout: filestore apply finisher threads = 3 2024-09-15T21:45:18.409 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.409 INFO:teuthology.orchestra.run.smithi100.stdout: bdev debug aio = true 2024-09-15T21:45:18.409 INFO:teuthology.orchestra.run.smithi100.stdout: osd debug misdirected ops = true 2024-09-15T21:45:18.409 INFO:teuthology.orchestra.run.smithi100.stdout: bdev async discard = True 2024-09-15T21:45:18.409 INFO:teuthology.orchestra.run.smithi100.stdout: bdev enable discard = True 2024-09-15T21:45:18.410 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore allocator = bitmap 2024-09-15T21:45:18.410 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore block size = 96636764160 2024-09-15T21:45:18.410 INFO:teuthology.orchestra.run.smithi100.stdout: bluestore fsck on mount = True 2024-09-15T21:45:18.410 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluefs = 1/20 2024-09-15T21:45:18.410 INFO:teuthology.orchestra.run.smithi100.stdout: debug bluestore = 1/20 2024-09-15T21:45:18.410 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-09-15T21:45:18.410 INFO:teuthology.orchestra.run.smithi100.stdout: debug osd = 20 2024-09-15T21:45:18.410 INFO:teuthology.orchestra.run.smithi100.stdout: debug rocksdb = 4/10 2024-09-15T21:45:18.410 INFO:teuthology.orchestra.run.smithi100.stdout: mon client directed command retry = 5 2024-09-15T21:45:18.410 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T21:45:18.410 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd full ratio = 0.9 2024-09-15T21:45:18.410 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd nearfull ratio = 0.8 2024-09-15T21:45:18.410 INFO:teuthology.orchestra.run.smithi100.stdout: ms inject delay max = 1 2024-09-15T21:45:18.411 INFO:teuthology.orchestra.run.smithi100.stdout: ms inject delay probability = 0.005 2024-09-15T21:45:18.411 INFO:teuthology.orchestra.run.smithi100.stdout: ms inject delay type = client mds 2024-09-15T21:45:18.411 INFO:teuthology.orchestra.run.smithi100.stdout: ms inject socket failures = 2500 2024-09-15T21:45:18.411 INFO:teuthology.orchestra.run.smithi100.stdout: osd blocked scrub grace period = 3600 2024-09-15T21:45:18.411 INFO:teuthology.orchestra.run.smithi100.stdout: osd failsafe full ratio = 0.95 2024-09-15T21:45:18.411 INFO:teuthology.orchestra.run.smithi100.stdout: osd max markdown count = 1000 2024-09-15T21:45:18.411 INFO:teuthology.orchestra.run.smithi100.stdout: osd objectstore = bluestore 2024-09-15T21:45:18.411 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-09-15T21:45:18.411 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.411 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr] 2024-09-15T21:45:18.411 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-09-15T21:45:18.411 INFO:teuthology.orchestra.run.smithi100.stdout: debug mgr = 20 2024-09-15T21:45:18.411 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-09-15T21:45:18.411 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-09-15T21:45:18.412 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-09-15T21:45:18.412 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-09-15T21:45:18.412 INFO:teuthology.orchestra.run.smithi100.stdout: mgr/telemetry/nag = false 2024-09-15T21:45:18.412 INFO:teuthology.orchestra.run.smithi100.stdout: debug client = 20 2024-09-15T21:45:18.412 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.412 INFO:teuthology.orchestra.run.smithi100.stdout:[mon] 2024-09-15T21:45:18.412 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-09-15T21:45:18.412 INFO:teuthology.orchestra.run.smithi100.stdout: debug mon = 20 2024-09-15T21:45:18.412 INFO:teuthology.orchestra.run.smithi100.stdout: debug paxos = 20 2024-09-15T21:45:18.412 INFO:teuthology.orchestra.run.smithi100.stdout: debug auth = 20 2024-09-15T21:45:18.412 INFO:teuthology.orchestra.run.smithi100.stdout: mon data avail warn = 5 2024-09-15T21:45:18.412 INFO:teuthology.orchestra.run.smithi100.stdout: mon mgr mkfs grace = 240 2024-09-15T21:45:18.412 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min pgs per osd = 4 2024-09-15T21:45:18.413 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd reporter subtree level = osd 2024-09-15T21:45:18.413 INFO:teuthology.orchestra.run.smithi100.stdout: mon osd prime pg temp = true 2024-09-15T21:45:18.413 INFO:teuthology.orchestra.run.smithi100.stdout: mon reweight min bytes per osd = 10 2024-09-15T21:45:18.413 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.413 INFO:teuthology.orchestra.run.smithi100.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T21:45:18.413 INFO:teuthology.orchestra.run.smithi100.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T21:45:18.413 INFO:teuthology.orchestra.run.smithi100.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T21:45:18.413 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.413 INFO:teuthology.orchestra.run.smithi100.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T21:45:18.413 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T21:45:18.413 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T21:45:18.413 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.413 INFO:teuthology.orchestra.run.smithi100.stdout: # 1m isn't quite enough 2024-09-15T21:45:18.414 INFO:teuthology.orchestra.run.smithi100.stdout: mon_down_mkfs_grace = 2m 2024-09-15T21:45:18.414 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.414 INFO:teuthology.orchestra.run.smithi100.stdout: mon_warn_on_filestore_osds = false 2024-09-15T21:45:18.414 INFO:teuthology.orchestra.run.smithi100.stdout: mon down mkfs grace = 300 2024-09-15T21:45:18.414 INFO:teuthology.orchestra.run.smithi100.stdout: mon op complaint time = 120 2024-09-15T21:45:18.414 INFO:teuthology.orchestra.run.smithi100.stdout: mon warn on pool no app = False 2024-09-15T21:45:18.414 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:45:18.414 INFO:teuthology.orchestra.run.smithi100.stdout:[client] 2024-09-15T21:45:18.414 INFO:teuthology.orchestra.run.smithi100.stdout: rgw cache enabled = true 2024-09-15T21:45:18.414 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable ops log = true 2024-09-15T21:45:18.414 INFO:teuthology.orchestra.run.smithi100.stdout: rgw enable usage log = true 2024-09-15T21:45:18.414 INFO:teuthology.orchestra.run.smithi100.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T21:45:18.414 INFO:teuthology.orchestra.run.smithi100.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T21:45:18.414 INFO:teuthology.orchestra.run.smithi100.stdout: client mount timeout = 600 2024-09-15T21:45:18.415 INFO:teuthology.orchestra.run.smithi100.stdout: client_shutdown_timeout = 120 2024-09-15T21:45:18.415 INFO:teuthology.orchestra.run.smithi100.stdout: debug client = 20 2024-09-15T21:45:18.415 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-09-15T21:45:18.415 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-09-15T21:45:18.415 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-09-15T21:45:18.415 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.a] 2024-09-15T21:45:18.415 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.b] 2024-09-15T21:45:18.415 INFO:teuthology.orchestra.run.smithi100.stdout:[mon.c] 2024-09-15T21:45:18.415 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-09-15T21:45:18.415 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-09-15T21:45:18.415 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.e] 2024-09-15T21:45:18.415 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-09-15T21:45:18.415 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-09-15T21:45:18.415 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.f] 2024-09-15T21:45:18.416 INFO:teuthology.orchestra.run.smithi100.stdout:[mds] 2024-09-15T21:45:18.416 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds = 20 2024-09-15T21:45:18.416 INFO:teuthology.orchestra.run.smithi100.stdout: debug mds balancer = 20 2024-09-15T21:45:18.416 INFO:teuthology.orchestra.run.smithi100.stdout: debug ms = 1 2024-09-15T21:45:18.416 INFO:teuthology.orchestra.run.smithi100.stdout: mds bal fragment size max = 10000 2024-09-15T21:45:18.416 INFO:teuthology.orchestra.run.smithi100.stdout: mds bal merge size = 5 2024-09-15T21:45:18.416 INFO:teuthology.orchestra.run.smithi100.stdout: mds bal split bits = 3 2024-09-15T21:45:18.416 INFO:teuthology.orchestra.run.smithi100.stdout: mds bal split size = 100 2024-09-15T21:45:18.416 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug frag = True 2024-09-15T21:45:18.416 INFO:teuthology.orchestra.run.smithi100.stdout: mds debug scatterstat = True 2024-09-15T21:45:18.416 INFO:teuthology.orchestra.run.smithi100.stdout: mds oft prefetch dirfrags = True 2024-09-15T21:45:18.416 INFO:teuthology.orchestra.run.smithi100.stdout: mds op complaint time = 180 2024-09-15T21:45:18.416 INFO:teuthology.orchestra.run.smithi100.stdout: mds verify scatter = True 2024-09-15T21:45:18.416 INFO:teuthology.orchestra.run.smithi100.stdout: mds_dir_prefetch = True 2024-09-15T21:45:18.417 INFO:teuthology.orchestra.run.smithi100.stdout: mds_sleep_rank_change = 5000000.0 2024-09-15T21:45:18.417 INFO:teuthology.orchestra.run.smithi100.stdout: mon client directed command retry = 5 2024-09-15T21:45:18.417 INFO:teuthology.orchestra.run.smithi100.stdout: ms inject delay max = 1 2024-09-15T21:45:18.417 INFO:teuthology.orchestra.run.smithi100.stdout: ms inject delay probability = 0.005 2024-09-15T21:45:18.417 INFO:teuthology.orchestra.run.smithi100.stdout: ms inject delay type = client mds osd 2024-09-15T21:45:18.417 INFO:teuthology.orchestra.run.smithi100.stdout: ms inject socket failures = 2500 2024-09-15T21:45:18.417 INFO:teuthology.orchestra.run.smithi100.stdout: osd op complaint time = 180 2024-09-15T21:45:18.417 INFO:teuthology.orchestra.run.smithi100.stdout: rados mon op timeout = 900 2024-09-15T21:45:18.417 INFO:teuthology.orchestra.run.smithi100.stdout: rados osd op timeout = 900 2024-09-15T21:45:18.419 INFO:teuthology.orchestra.run.smithi122.stdout:[global] 2024-09-15T21:45:18.419 INFO:teuthology.orchestra.run.smithi122.stdout: chdir = "" 2024-09-15T21:45:18.419 INFO:teuthology.orchestra.run.smithi122.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T21:45:18.419 INFO:teuthology.orchestra.run.smithi122.stdout: auth supported = cephx 2024-09-15T21:45:18.419 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.419 INFO:teuthology.orchestra.run.smithi122.stdout: filestore xattr use omap = true 2024-09-15T21:45:18.419 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.419 INFO:teuthology.orchestra.run.smithi122.stdout: mon clock drift allowed = 1.000 2024-09-15T21:45:18.419 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.419 INFO:teuthology.orchestra.run.smithi122.stdout: osd crush chooseleaf type = 0 2024-09-15T21:45:18.419 INFO:teuthology.orchestra.run.smithi122.stdout: auth debug = true 2024-09-15T21:45:18.419 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.419 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on old message = true 2024-09-15T21:45:18.420 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on bug = true 2024-09-15T21:45:18.420 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.420 INFO:teuthology.orchestra.run.smithi122.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T21:45:18.420 INFO:teuthology.orchestra.run.smithi122.stdout: mon pg warn max object skew = 0 2024-09-15T21:45:18.420 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.420 INFO:teuthology.orchestra.run.smithi122.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T21:45:18.420 INFO:teuthology.orchestra.run.smithi122.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T21:45:18.420 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.420 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default size = 2 2024-09-15T21:45:18.420 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.420 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow primary affinity = true 2024-09-15T21:45:18.420 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow pg remap = true 2024-09-15T21:45:18.420 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on legacy crush tunables = false 2024-09-15T21:45:18.421 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on crush straw calc version zero = false 2024-09-15T21:45:18.421 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on no sortbitwise = false 2024-09-15T21:45:18.421 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on osd down out interval zero = false 2024-09-15T21:45:18.421 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on too few osds = false 2024-09-15T21:45:18.421 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T21:45:18.421 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T21:45:18.421 INFO:teuthology.orchestra.run.smithi122.stdout: mon_allow_pool_size_one = true 2024-09-15T21:45:18.421 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.421 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T21:45:18.421 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.421 INFO:teuthology.orchestra.run.smithi122.stdout: osd default data pool replay window = 5 2024-09-15T21:45:18.421 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.421 INFO:teuthology.orchestra.run.smithi122.stdout: mon allow pool delete = true 2024-09-15T21:45:18.421 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.422 INFO:teuthology.orchestra.run.smithi122.stdout: mon cluster log file level = debug 2024-09-15T21:45:18.422 INFO:teuthology.orchestra.run.smithi122.stdout: debug asserts on shutdown = true 2024-09-15T21:45:18.422 INFO:teuthology.orchestra.run.smithi122.stdout: mon health detail to clog = false 2024-09-15T21:45:18.422 INFO:teuthology.orchestra.run.smithi122.stdout: mon host = "172.21.15.100,172.21.15.122,[v2:172.21.15.122:3301,v1:172.21.15.122:6790]" 2024-09-15T21:45:18.422 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on skipped message = False 2024-09-15T21:45:18.422 INFO:teuthology.orchestra.run.smithi122.stdout: fsid = 30e2fff4-2171-4531-8806-0e03b63f494f 2024-09-15T21:45:18.422 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.422 INFO:teuthology.orchestra.run.smithi122.stdout:[osd] 2024-09-15T21:45:18.422 INFO:teuthology.orchestra.run.smithi122.stdout: osd journal size = 100 2024-09-15T21:45:18.422 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.422 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub load threshold = 5.0 2024-09-15T21:45:18.422 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub max interval = 600 2024-09-15T21:45:18.422 INFO:teuthology.orchestra.run.smithi122.stdout: osd mclock profile = high_recovery_ops 2024-09-15T21:45:18.422 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.422 INFO:teuthology.orchestra.run.smithi122.stdout: osd recover clone overlap = true 2024-09-15T21:45:18.423 INFO:teuthology.orchestra.run.smithi122.stdout: osd recovery max chunk = 1048576 2024-09-15T21:45:18.423 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.423 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug shutdown = true 2024-09-15T21:45:18.423 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug op order = true 2024-09-15T21:45:18.423 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug verify stray on activate = true 2024-09-15T21:45:18.423 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug trim objects = true 2024-09-15T21:45:18.423 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.423 INFO:teuthology.orchestra.run.smithi122.stdout: osd open classes on start = true 2024-09-15T21:45:18.423 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug pg log writeout = true 2024-09-15T21:45:18.423 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.423 INFO:teuthology.orchestra.run.smithi122.stdout: osd deep scrub update digest min age = 30 2024-09-15T21:45:18.423 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.423 INFO:teuthology.orchestra.run.smithi122.stdout: osd map max advance = 10 2024-09-15T21:45:18.423 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.424 INFO:teuthology.orchestra.run.smithi122.stdout: journal zero on create = true 2024-09-15T21:45:18.424 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.424 INFO:teuthology.orchestra.run.smithi122.stdout: filestore ondisk finisher threads = 3 2024-09-15T21:45:18.424 INFO:teuthology.orchestra.run.smithi122.stdout: filestore apply finisher threads = 3 2024-09-15T21:45:18.424 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.424 INFO:teuthology.orchestra.run.smithi122.stdout: bdev debug aio = true 2024-09-15T21:45:18.424 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug misdirected ops = true 2024-09-15T21:45:18.424 INFO:teuthology.orchestra.run.smithi122.stdout: bdev async discard = True 2024-09-15T21:45:18.424 INFO:teuthology.orchestra.run.smithi122.stdout: bdev enable discard = True 2024-09-15T21:45:18.424 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore allocator = bitmap 2024-09-15T21:45:18.424 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore block size = 96636764160 2024-09-15T21:45:18.424 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore fsck on mount = True 2024-09-15T21:45:18.424 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluefs = 1/20 2024-09-15T21:45:18.424 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluestore = 1/20 2024-09-15T21:45:18.425 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-15T21:45:18.425 INFO:teuthology.orchestra.run.smithi122.stdout: debug osd = 20 2024-09-15T21:45:18.425 INFO:teuthology.orchestra.run.smithi122.stdout: debug rocksdb = 4/10 2024-09-15T21:45:18.425 INFO:teuthology.orchestra.run.smithi122.stdout: mon client directed command retry = 5 2024-09-15T21:45:18.425 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T21:45:18.425 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd full ratio = 0.9 2024-09-15T21:45:18.425 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd nearfull ratio = 0.8 2024-09-15T21:45:18.425 INFO:teuthology.orchestra.run.smithi122.stdout: ms inject delay max = 1 2024-09-15T21:45:18.425 INFO:teuthology.orchestra.run.smithi122.stdout: ms inject delay probability = 0.005 2024-09-15T21:45:18.425 INFO:teuthology.orchestra.run.smithi122.stdout: ms inject delay type = client mds 2024-09-15T21:45:18.425 INFO:teuthology.orchestra.run.smithi122.stdout: ms inject socket failures = 2500 2024-09-15T21:45:18.425 INFO:teuthology.orchestra.run.smithi122.stdout: osd blocked scrub grace period = 3600 2024-09-15T21:45:18.425 INFO:teuthology.orchestra.run.smithi122.stdout: osd failsafe full ratio = 0.95 2024-09-15T21:45:18.426 INFO:teuthology.orchestra.run.smithi122.stdout: osd max markdown count = 1000 2024-09-15T21:45:18.426 INFO:teuthology.orchestra.run.smithi122.stdout: osd objectstore = bluestore 2024-09-15T21:45:18.426 INFO:teuthology.orchestra.run.smithi122.stdout: osd op complaint time = 180 2024-09-15T21:45:18.426 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.426 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr] 2024-09-15T21:45:18.426 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-15T21:45:18.426 INFO:teuthology.orchestra.run.smithi122.stdout: debug mgr = 20 2024-09-15T21:45:18.426 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-09-15T21:45:18.426 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-09-15T21:45:18.427 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-09-15T21:45:18.427 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-09-15T21:45:18.427 INFO:teuthology.orchestra.run.smithi122.stdout: mgr/telemetry/nag = false 2024-09-15T21:45:18.427 INFO:teuthology.orchestra.run.smithi122.stdout: debug client = 20 2024-09-15T21:45:18.427 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.427 INFO:teuthology.orchestra.run.smithi122.stdout:[mon] 2024-09-15T21:45:18.427 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-15T21:45:18.427 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-09-15T21:45:18.427 INFO:teuthology.orchestra.run.smithi122.stdout: debug paxos = 20 2024-09-15T21:45:18.427 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-09-15T21:45:18.427 INFO:teuthology.orchestra.run.smithi122.stdout: mon data avail warn = 5 2024-09-15T21:45:18.427 INFO:teuthology.orchestra.run.smithi122.stdout: mon mgr mkfs grace = 240 2024-09-15T21:45:18.427 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-09-15T21:45:18.427 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd reporter subtree level = osd 2024-09-15T21:45:18.428 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd prime pg temp = true 2024-09-15T21:45:18.428 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-09-15T21:45:18.428 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.428 INFO:teuthology.orchestra.run.smithi122.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T21:45:18.428 INFO:teuthology.orchestra.run.smithi122.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T21:45:18.428 INFO:teuthology.orchestra.run.smithi122.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T21:45:18.428 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.428 INFO:teuthology.orchestra.run.smithi122.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T21:45:18.428 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T21:45:18.428 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T21:45:18.428 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.428 INFO:teuthology.orchestra.run.smithi122.stdout: # 1m isn't quite enough 2024-09-15T21:45:18.428 INFO:teuthology.orchestra.run.smithi122.stdout: mon_down_mkfs_grace = 2m 2024-09-15T21:45:18.428 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.429 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_filestore_osds = false 2024-09-15T21:45:18.429 INFO:teuthology.orchestra.run.smithi122.stdout: mon down mkfs grace = 300 2024-09-15T21:45:18.429 INFO:teuthology.orchestra.run.smithi122.stdout: mon op complaint time = 120 2024-09-15T21:45:18.429 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on pool no app = False 2024-09-15T21:45:18.429 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-15T21:45:18.429 INFO:teuthology.orchestra.run.smithi122.stdout:[client] 2024-09-15T21:45:18.429 INFO:teuthology.orchestra.run.smithi122.stdout: rgw cache enabled = true 2024-09-15T21:45:18.429 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable ops log = true 2024-09-15T21:45:18.429 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable usage log = true 2024-09-15T21:45:18.429 INFO:teuthology.orchestra.run.smithi122.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T21:45:18.429 INFO:teuthology.orchestra.run.smithi122.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T21:45:18.429 INFO:teuthology.orchestra.run.smithi122.stdout: client mount timeout = 600 2024-09-15T21:45:18.429 INFO:teuthology.orchestra.run.smithi122.stdout: client_shutdown_timeout = 120 2024-09-15T21:45:18.429 INFO:teuthology.orchestra.run.smithi122.stdout: debug client = 20 2024-09-15T21:45:18.430 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-15T21:45:18.430 INFO:teuthology.orchestra.run.smithi122.stdout: rados mon op timeout = 900 2024-09-15T21:45:18.430 INFO:teuthology.orchestra.run.smithi122.stdout: rados osd op timeout = 900 2024-09-15T21:45:18.430 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.a] 2024-09-15T21:45:18.430 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.b] 2024-09-15T21:45:18.430 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.c] 2024-09-15T21:45:18.430 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.a] 2024-09-15T21:45:18.430 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.c] 2024-09-15T21:45:18.430 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.e] 2024-09-15T21:45:18.430 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.b] 2024-09-15T21:45:18.430 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.d] 2024-09-15T21:45:18.430 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.f] 2024-09-15T21:45:18.430 INFO:teuthology.orchestra.run.smithi122.stdout:[mds] 2024-09-15T21:45:18.430 INFO:teuthology.orchestra.run.smithi122.stdout: debug mds = 20 2024-09-15T21:45:18.431 INFO:teuthology.orchestra.run.smithi122.stdout: debug mds balancer = 20 2024-09-15T21:45:18.431 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-15T21:45:18.431 INFO:teuthology.orchestra.run.smithi122.stdout: mds bal fragment size max = 10000 2024-09-15T21:45:18.431 INFO:teuthology.orchestra.run.smithi122.stdout: mds bal merge size = 5 2024-09-15T21:45:18.431 INFO:teuthology.orchestra.run.smithi122.stdout: mds bal split bits = 3 2024-09-15T21:45:18.431 INFO:teuthology.orchestra.run.smithi122.stdout: mds bal split size = 100 2024-09-15T21:45:18.431 INFO:teuthology.orchestra.run.smithi122.stdout: mds debug frag = True 2024-09-15T21:45:18.431 INFO:teuthology.orchestra.run.smithi122.stdout: mds debug scatterstat = True 2024-09-15T21:45:18.431 INFO:teuthology.orchestra.run.smithi122.stdout: mds oft prefetch dirfrags = True 2024-09-15T21:45:18.431 INFO:teuthology.orchestra.run.smithi122.stdout: mds op complaint time = 180 2024-09-15T21:45:18.431 INFO:teuthology.orchestra.run.smithi122.stdout: mds verify scatter = True 2024-09-15T21:45:18.431 INFO:teuthology.orchestra.run.smithi122.stdout: mds_dir_prefetch = True 2024-09-15T21:45:18.431 INFO:teuthology.orchestra.run.smithi122.stdout: mds_sleep_rank_change = 5000000.0 2024-09-15T21:45:18.431 INFO:teuthology.orchestra.run.smithi122.stdout: mon client directed command retry = 5 2024-09-15T21:45:18.432 INFO:teuthology.orchestra.run.smithi122.stdout: ms inject delay max = 1 2024-09-15T21:45:18.432 INFO:teuthology.orchestra.run.smithi122.stdout: ms inject delay probability = 0.005 2024-09-15T21:45:18.432 INFO:teuthology.orchestra.run.smithi122.stdout: ms inject delay type = client mds osd 2024-09-15T21:45:18.432 INFO:teuthology.orchestra.run.smithi122.stdout: ms inject socket failures = 2500 2024-09-15T21:45:18.432 INFO:teuthology.orchestra.run.smithi122.stdout: osd op complaint time = 180 2024-09-15T21:45:18.432 INFO:teuthology.orchestra.run.smithi122.stdout: rados mon op timeout = 900 2024-09-15T21:45:18.432 INFO:teuthology.orchestra.run.smithi122.stdout: rados osd op timeout = 900 2024-09-15T21:45:18.432 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-15T21:45:18.432 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-15T21:45:18.513 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-15T21:45:18.513 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:18.513 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-15T21:45:18.528 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:18.528 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-15T21:45:18.584 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi100.front.sepia.ceph.com 2024-09-15T21:45:18.584 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:18.584 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T21:45:18.584 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T21:45:18.657 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:18.658 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:45:18.713 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T21:45:18.713 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:45:18.713 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T21:45:18.713 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T21:45:18.749 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:45:18.749 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:45:18.806 INFO:tasks.ceph:Setting up mon nodes... 2024-09-15T21:45:18.806 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-15T21:45:18.806 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-15T21:45:18.861 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-15T21:45:18.862 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T21:45:18.935 INFO:teuthology.orchestra.run.smithi122.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T21:45:18.941 INFO:tasks.ceph:Setting up mds nodes... 2024-09-15T21:45:18.942 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-15T21:45:18.994 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-15T21:45:18.996 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-15T21:45:19.020 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-15T21:45:19.111 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-15T21:45:19.113 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-15T21:45:19.138 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-09-15T21:45:19.229 INFO:teuthology.orchestra.run.smithi100.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-09-15T21:45:19.232 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-09-15T21:45:19.256 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-15T21:45:19.307 INFO:teuthology.orchestra.run.smithi122.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-15T21:45:19.309 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-15T21:45:19.334 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-15T21:45:19.425 INFO:teuthology.orchestra.run.smithi122.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-15T21:45:19.428 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-15T21:45:19.453 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-09-15T21:45:19.545 INFO:teuthology.orchestra.run.smithi122.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-09-15T21:45:19.546 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-09-15T21:45:19.571 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-15T21:45:19.571 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-15T21:45:19.612 INFO:teuthology.orchestra.run.smithi100.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-15T21:45:19.625 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-15T21:45:19.625 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi100.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-15T21:45:19.625 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-15T21:45:19.689 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T21:45:19.689 INFO:tasks.ceph:role: osd.0 2024-09-15T21:45:19.690 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-15T21:45:19.690 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T21:45:19.761 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:45:19.761 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:45:19.762 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:45:19.762 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:45:19.762 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:45:19.762 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:45:19.762 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:45:19.762 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:45:19.762 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:45:19.762 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:45:20.263 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-09-15T21:45:20.265 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-09-15T21:45:20.265 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-15T21:45:20.383 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-15T21:45:20.415 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-15T21:45:20.480 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T21:45:20.480 INFO:tasks.ceph:role: osd.1 2024-09-15T21:45:20.480 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-15T21:45:20.480 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T21:45:20.552 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:45:20.552 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:45:20.552 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:45:20.552 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:45:20.552 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:45:20.552 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:45:20.552 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:45:20.552 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:45:20.553 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:45:20.553 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:45:21.015 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-09-15T21:45:21.018 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-09-15T21:45:21.018 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-15T21:45:21.112 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-15T21:45:21.141 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-15T21:45:21.205 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T21:45:21.205 INFO:tasks.ceph:role: osd.2 2024-09-15T21:45:21.206 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-15T21:45:21.206 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T21:45:21.277 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:45:21.277 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:45:21.278 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:45:21.278 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:45:21.278 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:45:21.278 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:45:21.278 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:45:21.278 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:45:21.278 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:45:21.278 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:45:21.761 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-09-15T21:45:21.763 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-09-15T21:45:21.763 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-15T21:45:21.893 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-15T21:45:21.923 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-15T21:45:21.988 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T21:45:21.989 INFO:tasks.ceph:role: osd.3 2024-09-15T21:45:21.989 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-15T21:45:21.989 DEBUG:teuthology.orchestra.run.smithi100:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T21:45:22.061 INFO:teuthology.orchestra.run.smithi100.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:45:22.061 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:45:22.061 INFO:teuthology.orchestra.run.smithi100.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:45:22.061 INFO:teuthology.orchestra.run.smithi100.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:45:22.061 INFO:teuthology.orchestra.run.smithi100.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:45:22.062 INFO:teuthology.orchestra.run.smithi100.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:45:22.062 INFO:teuthology.orchestra.run.smithi100.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:45:22.062 INFO:teuthology.orchestra.run.smithi100.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:45:22.062 INFO:teuthology.orchestra.run.smithi100.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:45:22.062 INFO:teuthology.orchestra.run.smithi100.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:45:22.575 INFO:teuthology.orchestra.run.smithi100.stdout:Discarding blocks...Done. 2024-09-15T21:45:22.577 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi100.front.sepia.ceph.com -o noatime 2024-09-15T21:45:22.577 DEBUG:teuthology.orchestra.run.smithi100:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-15T21:45:22.687 DEBUG:teuthology.orchestra.run.smithi100:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-15T21:45:22.719 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:45:22.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:22.825+0000 7fefba042640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-15T21:45:22.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:22.825+0000 7fefba042640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-15T21:45:22.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:22.825+0000 7fefba042640 -1 bdev(0x55c1d9b4f800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-15T21:45:22.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:22.825+0000 7fefba042640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-15T21:45:26.228 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-15T21:45:26.259 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:45:26.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:26.347+0000 7f10a6883640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-15T21:45:26.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:26.348+0000 7f10a6883640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-15T21:45:26.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:26.348+0000 7f10a6883640 -1 bdev(0x55a19267b800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-15T21:45:26.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:26.348+0000 7f10a6883640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-15T21:45:30.245 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-15T21:45:30.275 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:45:30.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:30.364+0000 7f3244262640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-15T21:45:30.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:30.364+0000 7f3244262640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-15T21:45:30.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:30.364+0000 7f3244262640 -1 bdev(0x5580fdaeb800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-15T21:45:30.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:30.364+0000 7f3244262640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-15T21:45:34.270 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-15T21:45:34.300 DEBUG:teuthology.orchestra.run.smithi100:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:45:34.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:34.389+0000 7fe0e681a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-15T21:45:34.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:34.389+0000 7fe0e681a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-15T21:45:34.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:34.389+0000 7fe0e681a640 -1 bdev(0x56008cfa9800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-15T21:45:34.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:34.389+0000 7fe0e681a640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-15T21:45:38.059 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-15T21:45:38.088 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-15T21:45:38.120 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T21:45:38.121 INFO:tasks.ceph:role: osd.4 2024-09-15T21:45:38.121 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T21:45:38.121 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T21:45:38.196 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:45:38.196 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:45:38.197 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:45:38.197 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:45:38.197 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:45:38.197 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:45:38.197 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:45:38.197 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:45:38.197 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:45:38.197 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:45:38.691 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-09-15T21:45:38.693 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-15T21:45:38.693 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-15T21:45:38.829 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-15T21:45:38.865 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-15T21:45:38.939 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T21:45:38.939 INFO:tasks.ceph:role: osd.5 2024-09-15T21:45:38.939 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T21:45:38.939 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T21:45:39.016 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:45:39.016 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:45:39.016 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:45:39.017 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:45:39.017 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:45:39.017 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:45:39.017 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:45:39.017 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:45:39.017 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:45:39.017 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:45:39.461 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-09-15T21:45:39.463 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-15T21:45:39.463 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-15T21:45:39.555 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-15T21:45:39.589 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-15T21:45:39.660 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T21:45:39.660 INFO:tasks.ceph:role: osd.6 2024-09-15T21:45:39.660 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T21:45:39.660 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T21:45:39.738 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:45:39.739 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:45:39.739 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:45:39.739 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:45:39.739 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:45:39.739 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:45:39.739 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:45:39.739 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:45:39.739 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:45:39.739 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:45:40.253 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-09-15T21:45:40.255 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-15T21:45:40.255 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-15T21:45:40.384 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-15T21:45:40.419 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-15T21:45:40.490 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T21:45:40.490 INFO:tasks.ceph:role: osd.7 2024-09-15T21:45:40.490 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T21:45:40.490 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T21:45:40.568 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:45:40.569 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:45:40.569 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:45:40.569 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:45:40.569 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:45:40.569 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:45:40.569 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:45:40.569 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:45:40.569 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:45:40.569 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:45:41.059 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-09-15T21:45:41.061 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-15T21:45:41.061 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-15T21:45:41.186 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-15T21:45:41.219 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:45:41.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:45:41.315+0000 7f1e87fbd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-15T21:45:41.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:45:41.315+0000 7f1e87fbd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-15T21:45:41.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:45:41.315+0000 7f1e87fbd640 -1 bdev(0x55b6ce4c3800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-15T21:45:41.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:45:41.315+0000 7f1e87fbd640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-15T21:45:45.222 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-15T21:45:45.253 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:45:45.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:45:45.346+0000 7f6ac4c39640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-15T21:45:45.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:45:45.347+0000 7f6ac4c39640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-15T21:45:45.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:45:45.347+0000 7f6ac4c39640 -1 bdev(0x56493c065800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-15T21:45:45.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:45:45.347+0000 7f6ac4c39640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-15T21:45:46.446 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T21:45:46.449 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T21:45:49.020 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-15T21:45:49.050 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:45:49.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:45:49.137+0000 7fb0d1c0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-15T21:45:49.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:45:49.137+0000 7fb0d1c0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-15T21:45:49.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:45:49.137+0000 7fb0d1c0d640 -1 bdev(0x55adfbdc9800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-15T21:45:49.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:45:49.138+0000 7fb0d1c0d640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-15T21:45:52.791 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-15T21:45:52.821 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:45:52.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:45:52.914+0000 7f79d6789640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-15T21:45:52.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:45:52.914+0000 7f79d6789640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-15T21:45:52.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:45:52.914+0000 7f79d6789640 -1 bdev(0x555eae389800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-15T21:45:52.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:45:52.914+0000 7f79d6789640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-15T21:45:56.818 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-15T21:45:56.848 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-15T21:45:56.848 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:56.849 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-15T21:45:56.879 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:56.879 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-15T21:45:56.948 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:56.948 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-15T21:45:57.017 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:57.018 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-09-15T21:45:57.086 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:57.087 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-15T21:45:57.155 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:57.155 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-15T21:45:57.221 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:57.221 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-15T21:45:57.286 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:57.286 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-15T21:45:57.351 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:45:57.351 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-15T21:45:57.382 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:45:57.382 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-15T21:45:57.451 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:45:57.452 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-15T21:45:57.522 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:45:57.522 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-09-15T21:45:57.592 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:45:57.593 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-15T21:45:57.663 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:45:57.663 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-15T21:45:57.731 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:45:57.731 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-15T21:45:57.799 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:45:57.799 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-15T21:45:57.866 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:57.866 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-15T21:45:57.882 INFO:tasks.ceph:Adding keys to all mons... 2024-09-15T21:45:57.882 DEBUG:teuthology.orchestra.run.smithi100:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T21:45:57.925 DEBUG:teuthology.orchestra.run.smithi122:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T21:45:57.946 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.y] 2024-09-15T21:45:57.946 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBuVedmVscMMxAAdTNWtriuIFDyfm49Ou/rEQ== 2024-09-15T21:45:57.946 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.a] 2024-09-15T21:45:57.946 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBuVedm/MY3OxAA072UmbY0GsI4Q5akJkXqgQ== 2024-09-15T21:45:57.946 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.c] 2024-09-15T21:45:57.946 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBvVedmmHiXBhAA0HLLhjb54pzaxWdmrOKe8w== 2024-09-15T21:45:57.946 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.e] 2024-09-15T21:45:57.946 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBvVedmty2fDRAAYyD3kDwLgWZEPICXijceTw== 2024-09-15T21:45:57.946 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.0] 2024-09-15T21:45:57.947 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQByVedmpnZBMRAAwxdS16NYeSAbW75p5FeX8g== 2024-09-15T21:45:57.947 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.1] 2024-09-15T21:45:57.947 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQB2VedmpJrbFBAAISL3pO8hRj2fa+eXwvxeKw== 2024-09-15T21:45:57.947 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.2] 2024-09-15T21:45:57.947 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQB6VedmeObCFRAAugfHp6Hte7n2gBOw4QUR+g== 2024-09-15T21:45:57.947 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.3] 2024-09-15T21:45:57.947 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQB+Vedmwn5CFxAAUI/sovm29wCbC3O9Ka4SEA== 2024-09-15T21:45:57.947 INFO:teuthology.orchestra.run.smithi100.stdout:[mgr.x] 2024-09-15T21:45:57.947 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBuVedmjpG6NxAA56Z2HGPT1EkFLeOS7dS04w== 2024-09-15T21:45:57.947 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.b] 2024-09-15T21:45:57.947 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBvVedmdVpGEhAAPiDJpT3/gBxn9n7NloXCVA== 2024-09-15T21:45:57.947 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.d] 2024-09-15T21:45:57.947 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBvVedmXphTGRAAIpVvOd5FuUsoifv5VUDuTQ== 2024-09-15T21:45:57.948 INFO:teuthology.orchestra.run.smithi100.stdout:[mds.f] 2024-09-15T21:45:57.948 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBvVedmD5lmIBAAmMvm+PdIxVy77jwRsMWqyw== 2024-09-15T21:45:57.948 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.4] 2024-09-15T21:45:57.948 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCFVedmnFzaEhAA5UCGrLdpYJykanzZZlBhAg== 2024-09-15T21:45:57.948 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.5] 2024-09-15T21:45:57.948 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCJVedmDeC7FBAAbI63NnrXwbMtHm0n+smzlQ== 2024-09-15T21:45:57.948 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.6] 2024-09-15T21:45:57.948 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCNVedmeA1BCBAAQYodB2GLlEy0O1MDM7+fbA== 2024-09-15T21:45:57.948 INFO:teuthology.orchestra.run.smithi100.stdout:[osd.7] 2024-09-15T21:45:57.948 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQCQVedmvayINhAASmc+0n2JbO+9HjARCwKhiA== 2024-09-15T21:45:57.948 INFO:teuthology.orchestra.run.smithi100.stdout:[client.0] 2024-09-15T21:45:57.948 INFO:teuthology.orchestra.run.smithi100.stdout: key = AQBvVedm67N7JBAAk6I6hjrgnvdL4bjoTMz8og== 2024-09-15T21:45:57.949 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr.y] 2024-09-15T21:45:57.949 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBuVedmVscMMxAAdTNWtriuIFDyfm49Ou/rEQ== 2024-09-15T21:45:57.950 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.a] 2024-09-15T21:45:57.950 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBuVedm/MY3OxAA072UmbY0GsI4Q5akJkXqgQ== 2024-09-15T21:45:57.950 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.c] 2024-09-15T21:45:57.950 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBvVedmmHiXBhAA0HLLhjb54pzaxWdmrOKe8w== 2024-09-15T21:45:57.950 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.e] 2024-09-15T21:45:57.950 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBvVedmty2fDRAAYyD3kDwLgWZEPICXijceTw== 2024-09-15T21:45:57.950 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.0] 2024-09-15T21:45:57.950 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQByVedmpnZBMRAAwxdS16NYeSAbW75p5FeX8g== 2024-09-15T21:45:57.950 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.1] 2024-09-15T21:45:57.950 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQB2VedmpJrbFBAAISL3pO8hRj2fa+eXwvxeKw== 2024-09-15T21:45:57.951 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.2] 2024-09-15T21:45:57.951 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQB6VedmeObCFRAAugfHp6Hte7n2gBOw4QUR+g== 2024-09-15T21:45:57.951 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.3] 2024-09-15T21:45:57.951 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQB+Vedmwn5CFxAAUI/sovm29wCbC3O9Ka4SEA== 2024-09-15T21:45:57.951 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr.x] 2024-09-15T21:45:57.951 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBuVedmjpG6NxAA56Z2HGPT1EkFLeOS7dS04w== 2024-09-15T21:45:57.951 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.b] 2024-09-15T21:45:57.951 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBvVedmdVpGEhAAPiDJpT3/gBxn9n7NloXCVA== 2024-09-15T21:45:57.951 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.d] 2024-09-15T21:45:57.951 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBvVedmXphTGRAAIpVvOd5FuUsoifv5VUDuTQ== 2024-09-15T21:45:57.951 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.f] 2024-09-15T21:45:57.951 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBvVedmD5lmIBAAmMvm+PdIxVy77jwRsMWqyw== 2024-09-15T21:45:57.951 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.4] 2024-09-15T21:45:57.951 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCFVedmnFzaEhAA5UCGrLdpYJykanzZZlBhAg== 2024-09-15T21:45:57.952 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.5] 2024-09-15T21:45:57.952 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCJVedmDeC7FBAAbI63NnrXwbMtHm0n+smzlQ== 2024-09-15T21:45:57.952 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.6] 2024-09-15T21:45:57.952 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCNVedmeA1BCBAAQYodB2GLlEy0O1MDM7+fbA== 2024-09-15T21:45:57.952 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.7] 2024-09-15T21:45:57.952 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCQVedmvayINhAASmc+0n2JbO+9HjARCwKhiA== 2024-09-15T21:45:57.952 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-09-15T21:45:57.952 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQBvVedm67N7JBAAk6I6hjrgnvdL4bjoTMz8og== 2024-09-15T21:45:57.953 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T21:45:57.992 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T21:45:58.046 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T21:45:58.048 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T21:45:58.102 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T21:45:58.104 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T21:45:58.157 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T21:45:58.160 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T21:45:58.213 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:45:58.215 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:45:58.265 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:45:58.268 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:45:58.318 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:45:58.321 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:45:58.371 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:45:58.374 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:45:58.424 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T21:45:58.427 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T21:45:58.477 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T21:45:58.480 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T21:45:58.533 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T21:45:58.536 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T21:45:58.589 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T21:45:58.591 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-15T21:45:58.646 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:45:58.649 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:45:58.699 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:45:58.701 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:45:58.752 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:45:58.754 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:45:58.804 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:45:58.807 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:45:58.858 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T21:45:58.861 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T21:45:58.917 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-15T21:45:58.917 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-15T21:45:58.942 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T21:45:59.038 DEBUG:teuthology.orchestra.run.smithi100:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-15T21:45:59.062 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-15T21:45:59.088 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T21:45:59.184 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-15T21:45:59.209 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-15T21:45:59.273 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T21:45:59.368 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-15T21:45:59.394 DEBUG:teuthology.orchestra.run.smithi100:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:45:59.396 DEBUG:teuthology.orchestra.run.smithi122:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:45:59.449 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-15T21:45:59.449 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T21:45:59.450 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T21:45:59.452 INFO:tasks.ceph.mon.a:Started 2024-09-15T21:45:59.452 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:45:59.452 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:45:59.492 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:45:59.492 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T21:45:59.492 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T21:45:59.494 INFO:tasks.ceph.mon.c:Started 2024-09-15T21:45:59.495 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-15T21:45:59.495 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-15T21:45:59.495 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-15T21:45:59.497 INFO:tasks.ceph.mgr.y:Started 2024-09-15T21:45:59.497 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-15T21:45:59.498 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-15T21:45:59.499 INFO:tasks.ceph.mgr.x:Started 2024-09-15T21:45:59.500 INFO:tasks.ceph:Setting crush tunables to default 2024-09-15T21:45:59.500 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph osd crush tunables default 2024-09-15T21:45:59.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.596+0000 7fbe832a5640 1 Processor -- start 2024-09-15T21:45:59.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.597+0000 7fbe832a5640 1 -- start start 2024-09-15T21:45:59.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.597+0000 7fbe832a5640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe7c15b370 0x7fbe7c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:45:59.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.597+0000 7fbe832a5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbe7c06db40 0x7fbe7c15b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:45:59.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.597+0000 7fbe832a5640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe7c0c9cf0 con 0x7fbe7c0c46e0 2024-09-15T21:45:59.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.597+0000 7fbe832a5640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fbe7c0c9200 con 0x7fbe7c06db40 2024-09-15T21:45:59.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.597+0000 7fbe832a5640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbe7c0c60f0 con 0x7fbe7c15b370 2024-09-15T21:45:59.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.598+0000 7fbe81842640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fbe7c0c46e0 0x7fbe7c15bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:51508/0 (socket says 172.21.15.100:51508) 2024-09-15T21:45:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.598+0000 7fbe81842640 1 -- 172.21.15.100:0/1771443768 learned_addr learned my addr 172.21.15.100:0/1771443768 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:45:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.598+0000 7fbe80840640 1 --2- 172.21.15.100:0/1771443768 >> v2:172.21.15.100:3300/0 conn(0x7fbe7c06db40 0x7fbe7c15b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:45:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.598+0000 7fbe80840640 1 -- 172.21.15.100:0/1771443768 >> v2:172.21.15.100:3300/0 conn(0x7fbe7c06db40 msgr2=0x7fbe7c15b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-09-15T21:45:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.598+0000 7fbe80840640 1 -- 172.21.15.100:0/1771443768 >> v2:172.21.15.100:3300/0 conn(0x7fbe7c06db40 msgr2=0x7fbe7c15b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:45:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.598+0000 7fbe80840640 1 --2- 172.21.15.100:0/1771443768 >> v2:172.21.15.100:3300/0 conn(0x7fbe7c06db40 0x7fbe7c15b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:45:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.598+0000 7fbe81041640 1 --2- 172.21.15.100:0/1771443768 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe7c15b370 0x7fbe7c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:45:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.598+0000 7fbe80840640 1 --2- 172.21.15.100:0/1771443768 >> v2:172.21.15.100:3300/0 conn(0x7fbe7c06db40 0x7fbe7c15b750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:45:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.598+0000 7fbe81041640 1 -- 172.21.15.100:0/1771443768 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe7c15b370 msgr2=0x7fbe7c06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-15T21:45:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.598+0000 7fbe81041640 1 -- 172.21.15.100:0/1771443768 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe7c15b370 msgr2=0x7fbe7c06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-15T21:45:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.598+0000 7fbe81041640 1 --2- 172.21.15.100:0/1771443768 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe7c15b370 0x7fbe7c06d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:45:59.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.598+0000 7fbe81041640 1 --2- 172.21.15.100:0/1771443768 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe7c15b370 0x7fbe7c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:45:59.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.758+0000 7fbe73fff640 1 -- 172.21.15.100:0/1771443768 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3719476420 0 0) 0x7fbe7c0c9cf0 con 0x7fbe7c0c46e0 2024-09-15T21:45:59.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.758+0000 7fbe73fff640 1 -- 172.21.15.100:0/1771443768 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe68003520 con 0x7fbe7c0c46e0 2024-09-15T21:45:59.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe73fff640 1 -- 172.21.15.100:0/1771443768 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 366438782 0 0) 0x7fbe68003520 con 0x7fbe7c0c46e0 2024-09-15T21:45:59.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe73fff640 1 -- 172.21.15.100:0/1771443768 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe7c15b370 msgr2=0x7fbe7c06d600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:45:59.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe73fff640 1 --2- 172.21.15.100:0/1771443768 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe7c15b370 0x7fbe7c06d600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:45:59.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe73fff640 1 -- 172.21.15.100:0/1771443768 >> v2:172.21.15.100:3300/0 conn(0x7fbe7c06db40 msgr2=0x7fbe7c15b750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:45:59.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe73fff640 1 --2- 172.21.15.100:0/1771443768 >> v2:172.21.15.100:3300/0 conn(0x7fbe7c06db40 0x7fbe7c15b750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:45:59.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe73fff640 1 -- 172.21.15.100:0/1771443768 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7c0c9730 con 0x7fbe7c0c46e0 2024-09-15T21:45:59.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe73fff640 1 -- 172.21.15.100:0/1771443768 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7fbe740019f0 con 0x7fbe7c0c46e0 2024-09-15T21:45:59.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe73fff640 1 -- 172.21.15.100:0/1771443768 >> v1:172.21.15.100:6789/0 conn(0x7fbe7c0c46e0 legacy=0x7fbe7c15bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:45:59.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe73fff640 1 --2- 172.21.15.100:0/1771443768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe680039b0 0x7fbe68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:45:59.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe73fff640 1 --2- 172.21.15.100:0/1771443768 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe7c15b370 0x7fbe68004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:45:59.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe73fff640 1 --2- 172.21.15.100:0/1771443768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe68004d40 0x7fbe680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:45:59.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe73fff640 1 -- 172.21.15.100:0/1771443768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c0c9200 con 0x7fbe68004d40 2024-09-15T21:45:59.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe73fff640 1 -- 172.21.15.100:0/1771443768 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c0c60f0 con 0x7fbe680039b0 2024-09-15T21:45:59.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe73fff640 1 -- 172.21.15.100:0/1771443768 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbe6800bbd0 con 0x7fbe7c15b370 2024-09-15T21:45:59.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe81842640 1 --2- 172.21.15.100:0/1771443768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe68004d40 0x7fbe680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:45:59.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe80840640 1 --2- 172.21.15.100:0/1771443768 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe7c15b370 0x7fbe68004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:45:59.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe81041640 1 --2- 172.21.15.100:0/1771443768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe680039b0 0x7fbe68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:45:59.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe81842640 1 -- 172.21.15.100:0/1771443768 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe7c15b370 msgr2=0x7fbe68004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:45:59.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe81842640 1 --2- 172.21.15.100:0/1771443768 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe7c15b370 0x7fbe68004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:45:59.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe81842640 1 -- 172.21.15.100:0/1771443768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe680039b0 msgr2=0x7fbe68003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:45:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe81842640 1 --2- 172.21.15.100:0/1771443768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe680039b0 0x7fbe68003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:45:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.759+0000 7fbe81842640 1 -- 172.21.15.100:0/1771443768 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe68003520 con 0x7fbe68004d40 2024-09-15T21:45:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.760+0000 7fbe81842640 1 --2- 172.21.15.100:0/1771443768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe68004d40 0x7fbe680051c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbe740013d0 tx=0x7fbe68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f845fb8893f55fc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:45:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.760+0000 7fbe73fff640 1 -- 172.21.15.100:0/1771443768 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7400a4e0 con 0x7fbe68004d40 2024-09-15T21:45:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.760+0000 7fbe73fff640 1 -- 172.21.15.100:0/1771443768 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe7402f070 con 0x7fbe68004d40 2024-09-15T21:45:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.760+0000 7fbe73fff640 1 -- 172.21.15.100:0/1771443768 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7402bce0 con 0x7fbe68004d40 2024-09-15T21:45:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.761+0000 7fbe832a5640 1 -- 172.21.15.100:0/1771443768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe68004d40 msgr2=0x7fbe680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:45:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.761+0000 7fbe832a5640 1 --2- 172.21.15.100:0/1771443768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe68004d40 0x7fbe680051c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbe740013d0 tx=0x7fbe68005830 comp rx=0 tx=0).stop 2024-09-15T21:45:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.761+0000 7fbe81842640 1 -- 172.21.15.100:0/1771443768 reap_dead start 2024-09-15T21:45:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.761+0000 7fbe832a5640 1 -- 172.21.15.100:0/1771443768 shutdown_connections 2024-09-15T21:45:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.761+0000 7fbe832a5640 1 -- 172.21.15.100:0/1771443768 >> 172.21.15.100:0/1771443768 conn(0x7fbe7c0ca390 msgr2=0x7fbe7c0ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:45:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.761+0000 7fbe832a5640 1 -- 172.21.15.100:0/1771443768 shutdown_connections 2024-09-15T21:45:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.761+0000 7fbe832a5640 1 -- 172.21.15.100:0/1771443768 wait complete. 2024-09-15T21:45:59.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.762+0000 7fbe832a5640 1 Processor -- start 2024-09-15T21:45:59.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.762+0000 7fbe832a5640 1 -- start start 2024-09-15T21:45:59.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.763+0000 7fbe832a5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe68004d40 0x7fbe7c1edaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:45:59.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.763+0000 7fbe832a5640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe7c143a50 0x7fbe7c0cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:45:59.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.763+0000 7fbe832a5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe7c15aee0 0x7fbe7c141100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:45:59.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.763+0000 7fbe832a5640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c06d0e0 con 0x7fbe7c15aee0 2024-09-15T21:45:59.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.763+0000 7fbe832a5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbe7c06ce00 con 0x7fbe68004d40 2024-09-15T21:45:59.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.763+0000 7fbe832a5640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbe7c1523d0 con 0x7fbe7c143a50 2024-09-15T21:45:59.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.763+0000 7fbe81041640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe68004d40 0x7fbe7c1edaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:45:59.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.763+0000 7fbe81041640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe68004d40 0x7fbe7c1edaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:48254/0 (socket says 172.21.15.100:48254) 2024-09-15T21:45:59.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.763+0000 7fbe81041640 1 -- 172.21.15.100:0/4271324853 learned_addr learned my addr 172.21.15.100:0/4271324853 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:45:59.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.763+0000 7fbe80840640 1 --2- 172.21.15.100:0/4271324853 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe7c143a50 0x7fbe7c0cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:45:59.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.763+0000 7fbe81842640 1 --2- 172.21.15.100:0/4271324853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe7c15aee0 0x7fbe7c141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:45:59.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.763+0000 7fbe81041640 1 -- 172.21.15.100:0/4271324853 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe7c143a50 msgr2=0x7fbe7c0cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:45:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.763+0000 7fbe81041640 1 --2- 172.21.15.100:0/4271324853 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe7c143a50 0x7fbe7c0cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:45:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.763+0000 7fbe81041640 1 -- 172.21.15.100:0/4271324853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe7c15aee0 msgr2=0x7fbe7c141100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:45:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.763+0000 7fbe81041640 1 --2- 172.21.15.100:0/4271324853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe7c15aee0 0x7fbe7c141100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:45:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.763+0000 7fbe81041640 1 -- 172.21.15.100:0/4271324853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7c152770 con 0x7fbe68004d40 2024-09-15T21:45:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.764+0000 7fbe81041640 1 --2- 172.21.15.100:0/4271324853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe68004d40 0x7fbe7c1edaa0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbe780108c0 tx=0x7fbe78010d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:45:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.764+0000 7fbe71ffb640 1 -- 172.21.15.100:0/4271324853 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7801a020 con 0x7fbe68004d40 2024-09-15T21:45:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.764+0000 7fbe71ffb640 1 -- 172.21.15.100:0/4271324853 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe7800e9c0 con 0x7fbe68004d40 2024-09-15T21:45:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.764+0000 7fbe71ffb640 1 -- 172.21.15.100:0/4271324853 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7800edb0 con 0x7fbe68004d40 2024-09-15T21:45:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.764+0000 7fbe832a5640 1 -- 172.21.15.100:0/4271324853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe7c152a50 con 0x7fbe68004d40 2024-09-15T21:45:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.764+0000 7fbe832a5640 1 -- 172.21.15.100:0/4271324853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe7c153010 con 0x7fbe68004d40 2024-09-15T21:45:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.765+0000 7fbe71ffb640 1 -- 172.21.15.100:0/4271324853 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbe7801d020 con 0x7fbe68004d40 2024-09-15T21:45:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.765+0000 7fbe71ffb640 1 -- 172.21.15.100:0/4271324853 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbe78014810 con 0x7fbe68004d40 2024-09-15T21:45:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.766+0000 7fbe832a5640 1 -- 172.21.15.100:0/4271324853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe7c0c8c70 con 0x7fbe68004d40 2024-09-15T21:45:59.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.768+0000 7fbe71ffb640 1 -- 172.21.15.100:0/4271324853 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fbe78024020 con 0x7fbe68004d40 2024-09-15T21:45:59.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.831+0000 7fbe832a5640 1 -- 172.21.15.100:0/4271324853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fbe7c153510 con 0x7fbe68004d40 2024-09-15T21:45:59.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.839+0000 7fbe71ffb640 1 -- 172.21.15.100:0/4271324853 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fbe78014a60 con 0x7fbe68004d40 2024-09-15T21:45:59.841 INFO:teuthology.orchestra.run.smithi100.stderr:adjusted tunables profile to default 2024-09-15T21:45:59.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.840+0000 7fbe832a5640 1 -- 172.21.15.100:0/4271324853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe68004d40 msgr2=0x7fbe7c1edaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:45:59.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.840+0000 7fbe832a5640 1 --2- 172.21.15.100:0/4271324853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe68004d40 0x7fbe7c1edaa0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbe780108c0 tx=0x7fbe78010d80 comp rx=0 tx=0).stop 2024-09-15T21:45:59.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.840+0000 7fbe832a5640 1 -- 172.21.15.100:0/4271324853 shutdown_connections 2024-09-15T21:45:59.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.840+0000 7fbe832a5640 1 --2- 172.21.15.100:0/4271324853 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbe7c15aee0 0x7fbe7c141100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:45:59.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.840+0000 7fbe832a5640 1 --2- 172.21.15.100:0/4271324853 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbe7c143a50 0x7fbe7c0cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:45:59.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.840+0000 7fbe832a5640 1 --2- 172.21.15.100:0/4271324853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe68004d40 0x7fbe7c1edaa0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:45:59.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.840+0000 7fbe832a5640 1 -- 172.21.15.100:0/4271324853 >> 172.21.15.100:0/4271324853 conn(0x7fbe7c0ca390 msgr2=0x7fbe7c147740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:45:59.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.840+0000 7fbe832a5640 1 -- 172.21.15.100:0/4271324853 shutdown_connections 2024-09-15T21:45:59.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:45:59.840+0000 7fbe832a5640 1 -- 172.21.15.100:0/4271324853 wait complete. 2024-09-15T21:45:59.852 INFO:tasks.ceph:check_enable_crimson: False 2024-09-15T21:45:59.852 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-15T21:45:59.853 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:59.853 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-15T21:45:59.862 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:45:59.860+0000 7ffadb000040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T21:45:59.865 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:45:59.864+0000 7efd4012b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-15T21:45:59.878 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:59.878 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-15T21:45:59.901 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:45:59.900+0000 7ffadb000040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T21:45:59.908 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:45:59.906+0000 7efd4012b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-15T21:45:59.933 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:59.933 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-15T21:45:59.998 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:45:59.998 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-15T21:46:00.041 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:00.039+0000 7ffadb000040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T21:46:00.053 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:00.052+0000 7efd4012b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-15T21:46:00.063 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:46:00.064 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-15T21:46:00.088 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:46:00.088 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-15T21:46:00.093 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:00.091+0000 7ffadb000040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T21:46:00.105 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:00.104+0000 7efd4012b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-15T21:46:00.117 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:46:00.117 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-15T21:46:00.186 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-15T21:46:00.186 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-15T21:46:00.251 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 5887900b-522d-476f-9704-cc2c5ee87a60 0 2024-09-15T21:46:00.258 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:00.257+0000 7ffadb000040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T21:46:00.272 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:00.271+0000 7efd4012b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-15T21:46:00.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.355+0000 7f437701e640 1 Processor -- start 2024-09-15T21:46:00.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.355+0000 7f437701e640 1 -- start start 2024-09-15T21:46:00.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.355+0000 7f437701e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f437014f3b0 0x7f437014f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:00.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.355+0000 7f437701e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4370071070 con 0x7f43700c8490 2024-09-15T21:46:00.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.355+0000 7f437701e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4370159ca0 con 0x7f43700c68c0 2024-09-15T21:46:00.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.355+0000 7f437701e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4370073b10 con 0x7f437014f3b0 2024-09-15T21:46:00.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.356+0000 7f43755bb640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f437014f3b0 0x7f437014f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:00.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.356+0000 7f43755bb640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f437014f3b0 0x7f437014f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.122:39252/0 (socket says 172.21.15.122:39252) 2024-09-15T21:46:00.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.356+0000 7f43755bb640 1 -- 172.21.15.122:0/2719331641 learned_addr learned my addr 172.21.15.122:0/2719331641 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-15T21:46:00.358 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.356+0000 7f436f7fe640 1 -- 172.21.15.122:0/2719331641 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3950389710 0 0) 0x7f4370159ca0 con 0x7f43700c68c0 2024-09-15T21:46:00.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.356+0000 7f43755bb640 1 -- 172.21.15.122:0/2719331641 >> v1:172.21.15.122:6789/0 conn(0x7f43700c68c0 legacy=0x7f4370070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.356+0000 7f43755bb640 1 -- 172.21.15.122:0/2719331641 >> v1:172.21.15.100:6789/0 conn(0x7f43700c8490 legacy=0x7f43700c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.356+0000 7f43755bb640 1 -- 172.21.15.122:0/2719331641 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4370158ee0 con 0x7f437014f3b0 2024-09-15T21:46:00.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.357+0000 7f43755bb640 1 --2- 172.21.15.122:0/2719331641 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f437014f3b0 0x7f437014f840 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4368009e80 tx=0x7f436802a830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de5c66850f2f67ab server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:00.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.357+0000 7f436f7fe640 1 -- 172.21.15.122:0/2719331641 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4368037070 con 0x7f437014f3b0 2024-09-15T21:46:00.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.357+0000 7f436f7fe640 1 -- 172.21.15.122:0/2719331641 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4368032420 con 0x7f437014f3b0 2024-09-15T21:46:00.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.357+0000 7f436f7fe640 1 -- 172.21.15.122:0/2719331641 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43680327f0 con 0x7f437014f3b0 2024-09-15T21:46:00.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.358+0000 7f437701e640 1 -- 172.21.15.122:0/2719331641 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f437014f3b0 msgr2=0x7f437014f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.358+0000 7f437701e640 1 --2- 172.21.15.122:0/2719331641 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f437014f3b0 0x7f437014f840 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4368009e80 tx=0x7f436802a830 comp rx=0 tx=0).stop 2024-09-15T21:46:00.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.358+0000 7f437701e640 1 -- 172.21.15.122:0/2719331641 shutdown_connections 2024-09-15T21:46:00.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.358+0000 7f437701e640 1 --2- 172.21.15.122:0/2719331641 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f437014f3b0 0x7f437014f840 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.358+0000 7f437701e640 1 -- 172.21.15.122:0/2719331641 >> 172.21.15.122:0/2719331641 conn(0x7f4370067980 msgr2=0x7f4370067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:00.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.358+0000 7f437701e640 1 -- 172.21.15.122:0/2719331641 shutdown_connections 2024-09-15T21:46:00.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.358+0000 7f437701e640 1 -- 172.21.15.122:0/2719331641 wait complete. 2024-09-15T21:46:00.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.359+0000 7f437701e640 1 Processor -- start 2024-09-15T21:46:00.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.359+0000 7f437701e640 1 -- start start 2024-09-15T21:46:00.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.360+0000 7f437701e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43700c68c0 0x7f4370146040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:00.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.360+0000 7f437701e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43700c8490 0x7f43701f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:00.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.360+0000 7f437701e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f437014f3b0 0x7f43701431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:00.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.360+0000 7f437701e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4370159780 con 0x7f43700c8490 2024-09-15T21:46:00.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.360+0000 7f437701e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43701594a0 con 0x7f43700c68c0 2024-09-15T21:46:00.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.360+0000 7f437701e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f43700737b0 con 0x7f437014f3b0 2024-09-15T21:46:00.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.360+0000 7f43755bb640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f437014f3b0 0x7f43701431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:00.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.360+0000 7f4374dba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43700c68c0 0x7f4370146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:00.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.360+0000 7f43755bb640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f437014f3b0 0x7f43701431c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.122:39254/0 (socket says 172.21.15.122:39254) 2024-09-15T21:46:00.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.360+0000 7f43755bb640 1 -- 172.21.15.122:0/995822899 learned_addr learned my addr 172.21.15.122:0/995822899 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-15T21:46:00.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.360+0000 7f436ffff640 1 --2- 172.21.15.122:0/995822899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43700c8490 0x7f43701f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:00.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.361+0000 7f4374dba640 1 -- 172.21.15.122:0/995822899 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f437014f3b0 msgr2=0x7f43701431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.361+0000 7f4374dba640 1 --2- 172.21.15.122:0/995822899 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f437014f3b0 0x7f43701431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.361+0000 7f4374dba640 1 -- 172.21.15.122:0/995822899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43700c8490 msgr2=0x7f43701f1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.361+0000 7f4374dba640 1 --2- 172.21.15.122:0/995822899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43700c8490 0x7f43701f1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.361+0000 7f4374dba640 1 -- 172.21.15.122:0/995822899 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4370146c30 con 0x7f43700c68c0 2024-09-15T21:46:00.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.361+0000 7f43755bb640 1 --2- 172.21.15.122:0/995822899 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f437014f3b0 0x7f43701431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:46:00.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.361+0000 7f4374dba640 1 --2- 172.21.15.122:0/995822899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43700c68c0 0x7f4370146040 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f436400c390 tx=0x7f436400c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:00.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.361+0000 7f436d7fa640 1 -- 172.21.15.122:0/995822899 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364017070 con 0x7f43700c68c0 2024-09-15T21:46:00.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.361+0000 7f436d7fa640 1 -- 172.21.15.122:0/995822899 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43640052d0 con 0x7f43700c68c0 2024-09-15T21:46:00.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.361+0000 7f436d7fa640 1 -- 172.21.15.122:0/995822899 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364003830 con 0x7f43700c68c0 2024-09-15T21:46:00.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.361+0000 7f437701e640 1 -- 172.21.15.122:0/995822899 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43701f28a0 con 0x7f43700c68c0 2024-09-15T21:46:00.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.361+0000 7f437701e640 1 -- 172.21.15.122:0/995822899 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43700c6370 con 0x7f43700c68c0 2024-09-15T21:46:00.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.361+0000 7f436d7fa640 1 -- 172.21.15.122:0/995822899 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f436401a070 con 0x7f43700c68c0 2024-09-15T21:46:00.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.361+0000 7f436d7fa640 1 -- 172.21.15.122:0/995822899 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f436401c440 con 0x7f43700c68c0 2024-09-15T21:46:00.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.362+0000 7f437701e640 1 -- 172.21.15.122:0/995822899 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4370073df0 con 0x7f43700c68c0 2024-09-15T21:46:00.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.366+0000 7f436d7fa640 1 -- 172.21.15.122:0/995822899 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4364021080 con 0x7f43700c68c0 2024-09-15T21:46:00.377 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:00.375+0000 7ffadb000040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T21:46:00.381 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:00.379+0000 7efd4012b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-15T21:46:00.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.436+0000 7f437701e640 1 -- 172.21.15.122:0/995822899 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5887900b-522d-476f-9704-cc2c5ee87a60", "id": 0} v 0) -- 0x7f4370156fc0 con 0x7f43700c68c0 2024-09-15T21:46:00.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.442+0000 7f436d7fa640 1 -- 172.21.15.122:0/995822899 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5887900b-522d-476f-9704-cc2c5ee87a60", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f4364011590 con 0x7f43700c68c0 2024-09-15T21:46:00.443 INFO:teuthology.orchestra.run.smithi122.stdout:0 2024-09-15T21:46:00.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.442+0000 7f437701e640 1 -- 172.21.15.122:0/995822899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43700c68c0 msgr2=0x7f4370146040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.442+0000 7f437701e640 1 --2- 172.21.15.122:0/995822899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43700c68c0 0x7f4370146040 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f436400c390 tx=0x7f436400c850 comp rx=0 tx=0).stop 2024-09-15T21:46:00.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.443+0000 7f437701e640 1 -- 172.21.15.122:0/995822899 shutdown_connections 2024-09-15T21:46:00.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.443+0000 7f437701e640 1 --2- 172.21.15.122:0/995822899 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f437014f3b0 0x7f43701431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.443+0000 7f437701e640 1 --2- 172.21.15.122:0/995822899 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43700c8490 0x7f43701f1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.443+0000 7f437701e640 1 --2- 172.21.15.122:0/995822899 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43700c68c0 0x7f4370146040 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.443+0000 7f437701e640 1 -- 172.21.15.122:0/995822899 >> 172.21.15.122:0/995822899 conn(0x7f4370067980 msgr2=0x7f43700c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:00.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.443+0000 7f437701e640 1 -- 172.21.15.122:0/995822899 shutdown_connections 2024-09-15T21:46:00.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.443+0000 7f437701e640 1 -- 172.21.15.122:0/995822899 wait complete. 2024-09-15T21:46:00.455 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new adc17993-5c50-4251-85e1-4c7ac62e25e6 1 2024-09-15T21:46:00.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.551+0000 7f138ce2e640 1 Processor -- start 2024-09-15T21:46:00.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.551+0000 7f138ce2e640 1 -- start start 2024-09-15T21:46:00.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.552+0000 7f138ce2e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f13881554b0 0x7f138815c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:00.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.552+0000 7f138ce2e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1388073d30 0x7f138815cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:00.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.552+0000 7f138ce2e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13880737e0 0x7f138815e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:00.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.552+0000 7f138ce2e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f1388073630 con 0x7f13881554b0 2024-09-15T21:46:00.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.552+0000 7f138ce2e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f138815bf90 con 0x7f1388073d30 2024-09-15T21:46:00.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.552+0000 7f138ce2e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f13880724c0 con 0x7f13880737e0 2024-09-15T21:46:00.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.552+0000 7f1385d9b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1388073d30 0x7f138815cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.552+0000 7f1385d9b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1388073d30 0x7f138815cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60592/0 (socket says 172.21.15.122:60592) 2024-09-15T21:46:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.552+0000 7f1386d9d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13880737e0 0x7f138815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.552+0000 7f1385d9b640 1 -- 172.21.15.122:0/2009101643 learned_addr learned my addr 172.21.15.122:0/2009101643 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-15T21:46:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.552+0000 7f138659c640 1 --2- 172.21.15.122:0/2009101643 >> v2:172.21.15.100:3300/0 conn(0x7f13881554b0 0x7f138815c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.552+0000 7f1386d9d640 1 -- 172.21.15.122:0/2009101643 >> v2:172.21.15.122:3300/0 conn(0x7f1388073d30 msgr2=0x7f138815cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.552+0000 7f1386d9d640 1 --2- 172.21.15.122:0/2009101643 >> v2:172.21.15.122:3300/0 conn(0x7f1388073d30 0x7f138815cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.552+0000 7f1386d9d640 1 -- 172.21.15.122:0/2009101643 >> v2:172.21.15.100:3300/0 conn(0x7f13881554b0 msgr2=0x7f138815c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.552+0000 7f1386d9d640 1 --2- 172.21.15.122:0/2009101643 >> v2:172.21.15.100:3300/0 conn(0x7f13881554b0 0x7f138815c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.552+0000 7f1386d9d640 1 -- 172.21.15.122:0/2009101643 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138806dbb0 con 0x7f13880737e0 2024-09-15T21:46:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.552+0000 7f138659c640 1 --2- 172.21.15.122:0/2009101643 >> v2:172.21.15.100:3300/0 conn(0x7f13881554b0 0x7f138815c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.552+0000 7f1385d9b640 1 --2- 172.21.15.122:0/2009101643 >> v2:172.21.15.122:3300/0 conn(0x7f1388073d30 0x7f138815cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:46:00.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.553+0000 7f1386d9d640 1 --2- 172.21.15.122:0/2009101643 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13880737e0 0x7f138815e680 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f13880cb3d0 tx=0x7f138002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d339712d739cab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:00.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.553+0000 7f138559a640 1 -- 172.21.15.122:0/2009101643 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380007740 con 0x7f13880737e0 2024-09-15T21:46:00.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.553+0000 7f138559a640 1 -- 172.21.15.122:0/2009101643 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13800078e0 con 0x7f13880737e0 2024-09-15T21:46:00.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.553+0000 7f138559a640 1 -- 172.21.15.122:0/2009101643 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380007cb0 con 0x7f13880737e0 2024-09-15T21:46:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.554+0000 7f138ce2e640 1 -- 172.21.15.122:0/2009101643 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13880737e0 msgr2=0x7f138815e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.554+0000 7f138ce2e640 1 --2- 172.21.15.122:0/2009101643 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13880737e0 0x7f138815e680 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f13880cb3d0 tx=0x7f138002a660 comp rx=0 tx=0).stop 2024-09-15T21:46:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.554+0000 7f138ce2e640 1 -- 172.21.15.122:0/2009101643 shutdown_connections 2024-09-15T21:46:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.554+0000 7f138ce2e640 1 --2- 172.21.15.122:0/2009101643 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13880737e0 0x7f138815e680 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.554+0000 7f138ce2e640 1 --2- 172.21.15.122:0/2009101643 >> v2:172.21.15.122:3300/0 conn(0x7f1388073d30 0x7f138815cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.554+0000 7f138ce2e640 1 --2- 172.21.15.122:0/2009101643 >> v2:172.21.15.100:3300/0 conn(0x7f13881554b0 0x7f138815c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.554+0000 7f138ce2e640 1 -- 172.21.15.122:0/2009101643 >> 172.21.15.122:0/2009101643 conn(0x7f13880ca390 msgr2=0x7f13880ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.554+0000 7f138ce2e640 1 -- 172.21.15.122:0/2009101643 shutdown_connections 2024-09-15T21:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.554+0000 7f138ce2e640 1 -- 172.21.15.122:0/2009101643 wait complete. 2024-09-15T21:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.555+0000 7f138ce2e640 1 Processor -- start 2024-09-15T21:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.555+0000 7f138ce2e640 1 -- start start 2024-09-15T21:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.556+0000 7f138ce2e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13880737e0 0x7f13880c4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.556+0000 7f138ce2e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1388073d30 0x7f13881f1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.556+0000 7f138ce2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13881554b0 0x7f13881f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.556+0000 7f138ce2e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f138815bc20 con 0x7f13881554b0 2024-09-15T21:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.556+0000 7f138ce2e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f13880721e0 con 0x7f1388073d30 2024-09-15T21:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.556+0000 7f138ce2e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f138815ebc0 con 0x7f13880737e0 2024-09-15T21:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.556+0000 7f138659c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13880737e0 0x7f13880c4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.556+0000 7f1385d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1388073d30 0x7f13881f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.556+0000 7f138659c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13880737e0 0x7f13880c4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.122:39264/0 (socket says 172.21.15.122:39264) 2024-09-15T21:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.556+0000 7f138659c640 1 -- 172.21.15.122:0/3753701969 learned_addr learned my addr 172.21.15.122:0/3753701969 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-15T21:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.556+0000 7f1386d9d640 1 --2- 172.21.15.122:0/3753701969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13881554b0 0x7f13881f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.557+0000 7f138659c640 1 -- 172.21.15.122:0/3753701969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1388073d30 msgr2=0x7f13881f1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.557+0000 7f138659c640 1 --2- 172.21.15.122:0/3753701969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1388073d30 0x7f13881f1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.557+0000 7f138659c640 1 -- 172.21.15.122:0/3753701969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13881554b0 msgr2=0x7f13881f4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.557+0000 7f138659c640 1 --2- 172.21.15.122:0/3753701969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13881554b0 0x7f13881f4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.557+0000 7f138659c640 1 -- 172.21.15.122:0/3753701969 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13881f1e80 con 0x7f13880737e0 2024-09-15T21:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.557+0000 7f1386d9d640 1 --2- 172.21.15.122:0/3753701969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13881554b0 0x7f13881f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.557+0000 7f138659c640 1 --2- 172.21.15.122:0/3753701969 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13880737e0 0x7f13880c4450 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f137c00d800 tx=0x7f137c00dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.557+0000 7f1376ffd640 1 -- 172.21.15.122:0/3753701969 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c004030 con 0x7f13880737e0 2024-09-15T21:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.557+0000 7f1376ffd640 1 -- 172.21.15.122:0/3753701969 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f137c0041d0 con 0x7f13880737e0 2024-09-15T21:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.557+0000 7f138ce2e640 1 -- 172.21.15.122:0/3753701969 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f138815a7b0 con 0x7f13880737e0 2024-09-15T21:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.557+0000 7f1376ffd640 1 -- 172.21.15.122:0/3753701969 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c0096a0 con 0x7f13880737e0 2024-09-15T21:46:00.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.557+0000 7f138ce2e640 1 -- 172.21.15.122:0/3753701969 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f138815a1f0 con 0x7f13880737e0 2024-09-15T21:46:00.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.557+0000 7f1376ffd640 1 -- 172.21.15.122:0/3753701969 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f137c009840 con 0x7f13880737e0 2024-09-15T21:46:00.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.557+0000 7f1376ffd640 1 -- 172.21.15.122:0/3753701969 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f137c0147e0 con 0x7f13880737e0 2024-09-15T21:46:00.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.558+0000 7f138ce2e640 1 -- 172.21.15.122:0/3753701969 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13880c6ef0 con 0x7f13880737e0 2024-09-15T21:46:00.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.560+0000 7f1376ffd640 1 -- 172.21.15.122:0/3753701969 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f13881f1e80 con 0x7f13880737e0 2024-09-15T21:46:00.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.629+0000 7f138ce2e640 1 -- 172.21.15.122:0/3753701969 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd new", "uuid": "adc17993-5c50-4251-85e1-4c7ac62e25e6", "id": 1} v 0) -- 0x7f1388158ab0 con 0x7f13880737e0 2024-09-15T21:46:00.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.632+0000 7f1376ffd640 1 -- 172.21.15.122:0/3753701969 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "adc17993-5c50-4251-85e1-4c7ac62e25e6", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f137c009c20 con 0x7f13880737e0 2024-09-15T21:46:00.634 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-15T21:46:00.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.633+0000 7f138ce2e640 1 -- 172.21.15.122:0/3753701969 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13880737e0 msgr2=0x7f13880c4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.633+0000 7f138ce2e640 1 --2- 172.21.15.122:0/3753701969 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13880737e0 0x7f13880c4450 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f137c00d800 tx=0x7f137c00dcc0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.633+0000 7f138ce2e640 1 -- 172.21.15.122:0/3753701969 shutdown_connections 2024-09-15T21:46:00.635 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.633+0000 7f138ce2e640 1 --2- 172.21.15.122:0/3753701969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1388073d30 0x7f13881f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.633+0000 7f138ce2e640 1 --2- 172.21.15.122:0/3753701969 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13881554b0 0x7f13881f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.633+0000 7f138ce2e640 1 --2- 172.21.15.122:0/3753701969 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13880737e0 0x7f13880c4450 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.633+0000 7f138ce2e640 1 -- 172.21.15.122:0/3753701969 >> 172.21.15.122:0/3753701969 conn(0x7f13880ca390 msgr2=0x7f13880c7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:00.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.633+0000 7f138ce2e640 1 -- 172.21.15.122:0/3753701969 shutdown_connections 2024-09-15T21:46:00.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.633+0000 7f138ce2e640 1 -- 172.21.15.122:0/3753701969 wait complete. 2024-09-15T21:46:00.645 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 8de3b7d5-1980-4a3b-a934-264f867c7c8d 2 2024-09-15T21:46:00.697 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:00.695+0000 7ffadb000040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T21:46:00.697 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:00.696+0000 7efd4012b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-15T21:46:00.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.742+0000 7f2f74f9f640 1 Processor -- start 2024-09-15T21:46:00.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.742+0000 7f2f74f9f640 1 -- start start 2024-09-15T21:46:00.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.743+0000 7f2f74f9f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2f7015b370 0x7f2f7006d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:00.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.743+0000 7f2f74f9f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f7006db40 0x7f2f7015b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:00.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.743+0000 7f2f74f9f640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f700c9cf0 con 0x7f2f700c46e0 2024-09-15T21:46:00.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.743+0000 7f2f74f9f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f2f700c9200 con 0x7f2f7015b370 2024-09-15T21:46:00.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.743+0000 7f2f74f9f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2f700c60f0 con 0x7f2f7006db40 2024-09-15T21:46:00.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.744+0000 7f2f6dd9b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f7006db40 0x7f2f7015b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:00.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.744+0000 7f2f6ed9d640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f2f700c46e0 0x7f2f7015bc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:46184/0 (socket says 172.21.15.122:46184) 2024-09-15T21:46:00.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.744+0000 7f2f6ed9d640 1 -- 172.21.15.122:0/702821741 learned_addr learned my addr 172.21.15.122:0/702821741 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-09-15T21:46:00.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.744+0000 7f2f6e59c640 1 --2- 172.21.15.122:0/702821741 >> v2:172.21.15.100:3300/0 conn(0x7f2f7015b370 0x7f2f7006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:00.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.744+0000 7f2f6d59a640 1 -- 172.21.15.122:0/702821741 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 387095916 0 0) 0x7f2f700c9cf0 con 0x7f2f700c46e0 2024-09-15T21:46:00.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.744+0000 7f2f6dd9b640 1 -- 172.21.15.122:0/702821741 >> v2:172.21.15.100:3300/0 conn(0x7f2f7015b370 msgr2=0x7f2f7006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:00.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.744+0000 7f2f6dd9b640 1 --2- 172.21.15.122:0/702821741 >> v2:172.21.15.100:3300/0 conn(0x7f2f7015b370 0x7f2f7006d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.744+0000 7f2f6dd9b640 1 -- 172.21.15.122:0/702821741 >> v1:172.21.15.122:6789/0 conn(0x7f2f700c46e0 legacy=0x7f2f7015bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.744+0000 7f2f6dd9b640 1 -- 172.21.15.122:0/702821741 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f700c9730 con 0x7f2f7006db40 2024-09-15T21:46:00.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.745+0000 7f2f6dd9b640 1 --2- 172.21.15.122:0/702821741 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f7006db40 0x7f2f7015b750 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2f640097a0 tx=0x7f2f6402a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd3b17dff0505ca3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:00.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.745+0000 7f2f6d59a640 1 -- 172.21.15.122:0/702821741 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64037070 con 0x7f2f7006db40 2024-09-15T21:46:00.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.745+0000 7f2f6d59a640 1 -- 172.21.15.122:0/702821741 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f6402adb0 con 0x7f2f7006db40 2024-09-15T21:46:00.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.745+0000 7f2f6d59a640 1 -- 172.21.15.122:0/702821741 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f64032660 con 0x7f2f7006db40 2024-09-15T21:46:00.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.745+0000 7f2f74f9f640 1 -- 172.21.15.122:0/702821741 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f7006db40 msgr2=0x7f2f7015b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.745+0000 7f2f74f9f640 1 --2- 172.21.15.122:0/702821741 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f7006db40 0x7f2f7015b750 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2f640097a0 tx=0x7f2f6402a060 comp rx=0 tx=0).stop 2024-09-15T21:46:00.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.745+0000 7f2f74f9f640 1 -- 172.21.15.122:0/702821741 shutdown_connections 2024-09-15T21:46:00.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.745+0000 7f2f74f9f640 1 --2- 172.21.15.122:0/702821741 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f7006db40 0x7f2f7015b750 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.745+0000 7f2f74f9f640 1 --2- 172.21.15.122:0/702821741 >> v2:172.21.15.100:3300/0 conn(0x7f2f7015b370 0x7f2f7006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.745+0000 7f2f74f9f640 1 -- 172.21.15.122:0/702821741 >> 172.21.15.122:0/702821741 conn(0x7f2f700ca390 msgr2=0x7f2f700ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.745+0000 7f2f74f9f640 1 -- 172.21.15.122:0/702821741 shutdown_connections 2024-09-15T21:46:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.745+0000 7f2f74f9f640 1 -- 172.21.15.122:0/702821741 wait complete. 2024-09-15T21:46:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.747+0000 7f2f74f9f640 1 Processor -- start 2024-09-15T21:46:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.747+0000 7f2f74f9f640 1 -- start start 2024-09-15T21:46:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.747+0000 7f2f74f9f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f7006db40 0x7f2f700c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.747+0000 7f2f74f9f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f700c46e0 0x7f2f701ef330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.747+0000 7f2f74f9f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f7015b370 0x7f2f70142240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.747+0000 7f2f74f9f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7006d0e0 con 0x7f2f7015b370 2024-09-15T21:46:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.747+0000 7f2f74f9f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2f7006ce00 con 0x7f2f700c46e0 2024-09-15T21:46:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.747+0000 7f2f74f9f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2f701523d0 con 0x7f2f7006db40 2024-09-15T21:46:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.747+0000 7f2f6dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f700c46e0 0x7f2f701ef330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.747+0000 7f2f6e59c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f7006db40 0x7f2f700c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:00.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.748+0000 7f2f6e59c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f7006db40 0x7f2f700c7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.122:39278/0 (socket says 172.21.15.122:39278) 2024-09-15T21:46:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.748+0000 7f2f6dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f700c46e0 0x7f2f701ef330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60604/0 (socket says 172.21.15.122:60604) 2024-09-15T21:46:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.748+0000 7f2f6e59c640 1 -- 172.21.15.122:0/1292990900 learned_addr learned my addr 172.21.15.122:0/1292990900 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-15T21:46:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.748+0000 7f2f6ed9d640 1 --2- 172.21.15.122:0/1292990900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f7015b370 0x7f2f70142240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.748+0000 7f2f6e59c640 1 -- 172.21.15.122:0/1292990900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f700c46e0 msgr2=0x7f2f701ef330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.748+0000 7f2f6e59c640 1 --2- 172.21.15.122:0/1292990900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f700c46e0 0x7f2f701ef330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.748+0000 7f2f6e59c640 1 -- 172.21.15.122:0/1292990900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f7015b370 msgr2=0x7f2f70142240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.748+0000 7f2f6e59c640 1 --2- 172.21.15.122:0/1292990900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f7015b370 0x7f2f70142240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.748+0000 7f2f6e59c640 1 -- 172.21.15.122:0/1292990900 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f701f2ad0 con 0x7f2f7006db40 2024-09-15T21:46:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.748+0000 7f2f6e59c640 1 --2- 172.21.15.122:0/1292990900 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f7006db40 0x7f2f700c7f30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000e9b0 tx=0x7f2f6000ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.748+0000 7f2f5effd640 1 -- 172.21.15.122:0/1292990900 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f60017070 con 0x7f2f7006db40 2024-09-15T21:46:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.748+0000 7f2f5effd640 1 -- 172.21.15.122:0/1292990900 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f6000c9a0 con 0x7f2f7006db40 2024-09-15T21:46:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.748+0000 7f2f74f9f640 1 -- 172.21.15.122:0/1292990900 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f70143aa0 con 0x7f2f7006db40 2024-09-15T21:46:00.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.748+0000 7f2f5effd640 1 -- 172.21.15.122:0/1292990900 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6000cd90 con 0x7f2f7006db40 2024-09-15T21:46:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.748+0000 7f2f74f9f640 1 -- 172.21.15.122:0/1292990900 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f70154300 con 0x7f2f7006db40 2024-09-15T21:46:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.748+0000 7f2f5effd640 1 -- 172.21.15.122:0/1292990900 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2f6001a070 con 0x7f2f7006db40 2024-09-15T21:46:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.748+0000 7f2f5effd640 1 -- 172.21.15.122:0/1292990900 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f2f6001ca40 con 0x7f2f7006db40 2024-09-15T21:46:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.749+0000 7f2f74f9f640 1 -- 172.21.15.122:0/1292990900 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f700c8c70 con 0x7f2f7006db40 2024-09-15T21:46:00.754 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:00.749+0000 7efd4012b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T21:46:00.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.751+0000 7f2f5effd640 1 -- 172.21.15.122:0/1292990900 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f2f701f2ad0 con 0x7f2f7006db40 2024-09-15T21:46:00.766 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:00.764+0000 7ffadb000040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-15T21:46:00.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.842+0000 7f2f74f9f640 1 -- 172.21.15.122:0/1292990900 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8de3b7d5-1980-4a3b-a934-264f867c7c8d", "id": 2} v 0) -- 0x7f2f700c42b0 con 0x7f2f7006db40 2024-09-15T21:46:00.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.845+0000 7f2f5effd640 1 -- 172.21.15.122:0/1292990900 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8de3b7d5-1980-4a3b-a934-264f867c7c8d", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f2f6001c5e0 con 0x7f2f7006db40 2024-09-15T21:46:00.847 INFO:teuthology.orchestra.run.smithi122.stdout:2 2024-09-15T21:46:00.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.846+0000 7f2f74f9f640 1 -- 172.21.15.122:0/1292990900 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f7006db40 msgr2=0x7f2f700c7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.846+0000 7f2f74f9f640 1 --2- 172.21.15.122:0/1292990900 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f7006db40 0x7f2f700c7f30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000e9b0 tx=0x7f2f6000ee70 comp rx=0 tx=0).stop 2024-09-15T21:46:00.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.846+0000 7f2f74f9f640 1 -- 172.21.15.122:0/1292990900 shutdown_connections 2024-09-15T21:46:00.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.846+0000 7f2f74f9f640 1 --2- 172.21.15.122:0/1292990900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f700c46e0 0x7f2f701ef330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.846+0000 7f2f74f9f640 1 --2- 172.21.15.122:0/1292990900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f7015b370 0x7f2f70142240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.846+0000 7f2f74f9f640 1 --2- 172.21.15.122:0/1292990900 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f7006db40 0x7f2f700c7f30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.846+0000 7f2f74f9f640 1 -- 172.21.15.122:0/1292990900 >> 172.21.15.122:0/1292990900 conn(0x7f2f700ca390 msgr2=0x7f2f7006c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:00.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.846+0000 7f2f74f9f640 1 -- 172.21.15.122:0/1292990900 shutdown_connections 2024-09-15T21:46:00.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.846+0000 7f2f74f9f640 1 -- 172.21.15.122:0/1292990900 wait complete. 2024-09-15T21:46:00.858 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 93dbcd1c-6a6f-49cf-bc94-e64b883de5f6 3 2024-09-15T21:46:00.900 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:00.898+0000 7efd4012b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T21:46:00.936 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:00.934+0000 7ffadb000040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-15T21:46:00.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.956+0000 7f5cbf82e640 1 Processor -- start 2024-09-15T21:46:00.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.956+0000 7f5cbf82e640 1 -- start start 2024-09-15T21:46:00.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.957+0000 7f5cbf82e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5cb8073d30 0x7f5cb806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:00.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.957+0000 7f5cbf82e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cb80724c0 con 0x7f5cb81554b0 2024-09-15T21:46:00.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.957+0000 7f5cbf82e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5cb806dbb0 con 0x7f5cb815ce40 2024-09-15T21:46:00.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.957+0000 7f5cbf82e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f5cb8073630 con 0x7f5cb8073d30 2024-09-15T21:46:00.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.957+0000 7f5cbddcb640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f5cb815ce40 0x7f5cb806c120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:46200/0 (socket says 172.21.15.122:46200) 2024-09-15T21:46:00.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.957+0000 7f5cbddcb640 1 -- 172.21.15.122:0/2697758620 learned_addr learned my addr 172.21.15.122:0/2697758620 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-09-15T21:46:00.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.957+0000 7f5cbcdc9640 1 --2- 172.21.15.122:0/2697758620 >> v2:172.21.15.100:3300/0 conn(0x7f5cb8073d30 0x7f5cb806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:00.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.957+0000 7f5caffff640 1 -- 172.21.15.122:0/2697758620 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3746919883 0 0) 0x7f5cb0001490 con 0x7f5cb815ce40 2024-09-15T21:46:00.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.957+0000 7f5caffff640 1 -- 172.21.15.122:0/2697758620 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ca4003520 con 0x7f5cb815ce40 2024-09-15T21:46:00.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.957+0000 7f5caffff640 1 -- 172.21.15.122:0/2697758620 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 925274328 0 0) 0x7f5cb80724c0 con 0x7f5cb81554b0 2024-09-15T21:46:00.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.957+0000 7f5caffff640 1 -- 172.21.15.122:0/2697758620 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5cb0001490 con 0x7f5cb81554b0 2024-09-15T21:46:00.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.957+0000 7f5cbcdc9640 1 -- 172.21.15.122:0/2697758620 >> v1:172.21.15.122:6789/0 conn(0x7f5cb815ce40 legacy=0x7f5cb806c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.957+0000 7f5caffff640 1 -- 172.21.15.122:0/2697758620 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1201663829 0 0) 0x7f5ca4003520 con 0x7f5cb815ce40 2024-09-15T21:46:00.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.957+0000 7f5cbcdc9640 1 -- 172.21.15.122:0/2697758620 >> v1:172.21.15.100:6789/0 conn(0x7f5cb81554b0 legacy=0x7f5cb815c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.957+0000 7f5cbcdc9640 1 -- 172.21.15.122:0/2697758620 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb8073350 con 0x7f5cb8073d30 2024-09-15T21:46:00.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.958+0000 7f5cbcdc9640 1 --2- 172.21.15.122:0/2697758620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cb8073d30 0x7f5cb806bbe0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009720 tx=0x7f5ca802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79411dacdf03dfa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:00.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.959+0000 7f5caffff640 1 -- 172.21.15.122:0/2697758620 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8037070 con 0x7f5cb8073d30 2024-09-15T21:46:00.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.959+0000 7f5caffff640 1 -- 172.21.15.122:0/2697758620 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ca802ac10 con 0x7f5cb8073d30 2024-09-15T21:46:00.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.959+0000 7f5caffff640 1 -- 172.21.15.122:0/2697758620 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4001670 con 0x7f5cb8073d30 2024-09-15T21:46:00.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.959+0000 7f5cbf82e640 1 -- 172.21.15.122:0/2697758620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cb8073d30 msgr2=0x7f5cb806bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.959+0000 7f5cbf82e640 1 --2- 172.21.15.122:0/2697758620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cb8073d30 0x7f5cb806bbe0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009720 tx=0x7f5ca802a180 comp rx=0 tx=0).stop 2024-09-15T21:46:00.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.959+0000 7f5cbf82e640 1 -- 172.21.15.122:0/2697758620 shutdown_connections 2024-09-15T21:46:00.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.959+0000 7f5cbf82e640 1 --2- 172.21.15.122:0/2697758620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cb8073d30 0x7f5cb806bbe0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.959+0000 7f5cbf82e640 1 -- 172.21.15.122:0/2697758620 >> 172.21.15.122:0/2697758620 conn(0x7f5cb80ca390 msgr2=0x7f5cb80ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:00.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.959+0000 7f5cbf82e640 1 -- 172.21.15.122:0/2697758620 shutdown_connections 2024-09-15T21:46:00.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.959+0000 7f5cbf82e640 1 -- 172.21.15.122:0/2697758620 wait complete. 2024-09-15T21:46:00.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.961+0000 7f5cbf82e640 1 Processor -- start 2024-09-15T21:46:00.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.961+0000 7f5cbf82e640 1 -- start start 2024-09-15T21:46:00.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.961+0000 7f5cbf82e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5cb8073d30 0x7f5cb8155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:00.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.961+0000 7f5cbf82e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cb81554b0 0x7f5cb815a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:00.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.961+0000 7f5cbf82e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cb815ce40 0x7f5cb814fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:00.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.961+0000 7f5cbf82e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb815bc20 con 0x7f5cb81554b0 2024-09-15T21:46:00.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.961+0000 7f5cbf82e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5cb806c830 con 0x7f5cb815ce40 2024-09-15T21:46:00.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.961+0000 7f5cbf82e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5cb806c9b0 con 0x7f5cb8073d30 2024-09-15T21:46:00.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.961+0000 7f5cbd5ca640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5cb8073d30 0x7f5cb8155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:00.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.961+0000 7f5cbddcb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cb815ce40 0x7f5cb814fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:00.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.961+0000 7f5cbd5ca640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5cb8073d30 0x7f5cb8155980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.122:39286/0 (socket says 172.21.15.122:39286) 2024-09-15T21:46:00.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.961+0000 7f5cbd5ca640 1 -- 172.21.15.122:0/4252642456 learned_addr learned my addr 172.21.15.122:0/4252642456 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-15T21:46:00.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.961+0000 7f5cbcdc9640 1 --2- 172.21.15.122:0/4252642456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cb81554b0 0x7f5cb815a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:00.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.962+0000 7f5cbd5ca640 1 -- 172.21.15.122:0/4252642456 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cb815ce40 msgr2=0x7f5cb814fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.962+0000 7f5cbd5ca640 1 --2- 172.21.15.122:0/4252642456 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cb815ce40 0x7f5cb814fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.962+0000 7f5cbd5ca640 1 -- 172.21.15.122:0/4252642456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cb81554b0 msgr2=0x7f5cb815a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:00.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.962+0000 7f5cbd5ca640 1 --2- 172.21.15.122:0/4252642456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cb81554b0 0x7f5cb815a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:00.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.962+0000 7f5cbd5ca640 1 -- 172.21.15.122:0/4252642456 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cb80bff30 con 0x7f5cb8073d30 2024-09-15T21:46:00.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.962+0000 7f5cbd5ca640 1 --2- 172.21.15.122:0/4252642456 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5cb8073d30 0x7f5cb8155980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400c7b0 tx=0x7f5cb400cc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:00.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.962+0000 7f5cadffb640 1 -- 172.21.15.122:0/4252642456 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4017070 con 0x7f5cb8073d30 2024-09-15T21:46:00.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.962+0000 7f5cadffb640 1 -- 172.21.15.122:0/4252642456 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb400a640 con 0x7f5cb8073d30 2024-09-15T21:46:00.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.962+0000 7f5cadffb640 1 -- 172.21.15.122:0/4252642456 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb400ce60 con 0x7f5cb8073d30 2024-09-15T21:46:00.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.962+0000 7f5cbf82e640 1 -- 172.21.15.122:0/4252642456 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cb80bed20 con 0x7f5cb8073d30 2024-09-15T21:46:00.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.962+0000 7f5cadffb640 1 -- 172.21.15.122:0/4252642456 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5cb401a070 con 0x7f5cb8073d30 2024-09-15T21:46:00.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.962+0000 7f5cbf82e640 1 -- 172.21.15.122:0/4252642456 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cb80cd3e0 con 0x7f5cb8073d30 2024-09-15T21:46:00.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.963+0000 7f5cadffb640 1 -- 172.21.15.122:0/4252642456 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f5cb4003d30 con 0x7f5cb8073d30 2024-09-15T21:46:00.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.963+0000 7f5cbf82e640 1 -- 172.21.15.122:0/4252642456 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cb80741b0 con 0x7f5cb8073d30 2024-09-15T21:46:00.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:00.965+0000 7f5cadffb640 1 -- 172.21.15.122:0/4252642456 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5cb80741b0 con 0x7f5cb8073d30 2024-09-15T21:46:01.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.034+0000 7f5cbf82e640 1 -- 172.21.15.122:0/4252642456 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd new", "uuid": "93dbcd1c-6a6f-49cf-bc94-e64b883de5f6", "id": 3} v 0) -- 0x7f5cb80737e0 con 0x7f5cb8073d30 2024-09-15T21:46:01.039 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.038+0000 7f5cadffb640 1 -- 172.21.15.122:0/4252642456 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "93dbcd1c-6a6f-49cf-bc94-e64b883de5f6", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f5cb4022020 con 0x7f5cb8073d30 2024-09-15T21:46:01.040 INFO:teuthology.orchestra.run.smithi122.stdout:3 2024-09-15T21:46:01.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.039+0000 7f5cbf82e640 1 -- 172.21.15.122:0/4252642456 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5cb8073d30 msgr2=0x7f5cb8155980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.039+0000 7f5cbf82e640 1 --2- 172.21.15.122:0/4252642456 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5cb8073d30 0x7f5cb8155980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400c7b0 tx=0x7f5cb400cc70 comp rx=0 tx=0).stop 2024-09-15T21:46:01.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.039+0000 7f5cbf82e640 1 -- 172.21.15.122:0/4252642456 shutdown_connections 2024-09-15T21:46:01.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.039+0000 7f5cbf82e640 1 --2- 172.21.15.122:0/4252642456 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cb815ce40 0x7f5cb814fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.039+0000 7f5cbf82e640 1 --2- 172.21.15.122:0/4252642456 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5cb81554b0 0x7f5cb815a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.039+0000 7f5cbf82e640 1 --2- 172.21.15.122:0/4252642456 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5cb8073d30 0x7f5cb8155980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.039+0000 7f5cbf82e640 1 -- 172.21.15.122:0/4252642456 >> 172.21.15.122:0/4252642456 conn(0x7f5cb80ca390 msgr2=0x7f5cb806bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:01.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.039+0000 7f5cbf82e640 1 -- 172.21.15.122:0/4252642456 shutdown_connections 2024-09-15T21:46:01.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.039+0000 7f5cbf82e640 1 -- 172.21.15.122:0/4252642456 wait complete. 2024-09-15T21:46:01.051 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 7caee2a0-bab4-45a9-8109-724948c863c6 4 2024-09-15T21:46:01.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.148+0000 7f1d4a665640 1 Processor -- start 2024-09-15T21:46:01.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.149+0000 7f1d4a665640 1 -- start start 2024-09-15T21:46:01.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.149+0000 7f1d4a665640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1d4415b370 0x7f1d4406d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.149+0000 7f1d4a665640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1d440c46e0 0x7f1d4415be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.149+0000 7f1d4a665640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d440c9cf0 con 0x7f1d4406db40 2024-09-15T21:46:01.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.149+0000 7f1d4a665640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f1d440c9200 con 0x7f1d4415b370 2024-09-15T21:46:01.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.149+0000 7f1d4a665640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f1d440c60f0 con 0x7f1d440c46e0 2024-09-15T21:46:01.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.149+0000 7f1d437fe640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f1d4406db40 0x7f1d4415b750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:46206/0 (socket says 172.21.15.122:46206) 2024-09-15T21:46:01.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.149+0000 7f1d437fe640 1 -- 172.21.15.122:0/89884471 learned_addr learned my addr 172.21.15.122:0/89884471 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-09-15T21:46:01.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.150+0000 7f1d43fff640 1 --2- 172.21.15.122:0/89884471 >> v2:172.21.15.100:3300/0 conn(0x7f1d4415b370 0x7f1d4406d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.150+0000 7f1d48c02640 1 --2- 172.21.15.122:0/89884471 >> v2:172.21.15.122:3300/0 conn(0x7f1d440c46e0 0x7f1d4415be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.150+0000 7f1d42ffd640 1 -- 172.21.15.122:0/89884471 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1855152306 0 0) 0x7f1d440c9cf0 con 0x7f1d4406db40 2024-09-15T21:46:01.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.150+0000 7f1d42ffd640 1 -- 172.21.15.122:0/89884471 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d30003520 con 0x7f1d4406db40 2024-09-15T21:46:01.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.150+0000 7f1d48c02640 1 -- 172.21.15.122:0/89884471 >> v2:172.21.15.100:3300/0 conn(0x7f1d4415b370 msgr2=0x7f1d4406d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:01.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.150+0000 7f1d48c02640 1 --2- 172.21.15.122:0/89884471 >> v2:172.21.15.100:3300/0 conn(0x7f1d4415b370 0x7f1d4406d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.150+0000 7f1d48c02640 1 -- 172.21.15.122:0/89884471 >> v1:172.21.15.122:6789/0 conn(0x7f1d4406db40 legacy=0x7f1d4415b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.150+0000 7f1d48c02640 1 -- 172.21.15.122:0/89884471 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d440c9730 con 0x7f1d440c46e0 2024-09-15T21:46:01.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.151+0000 7f1d48c02640 1 --2- 172.21.15.122:0/89884471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d440c46e0 0x7f1d4415be60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1d440cb720 tx=0x7f1d3c02c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed16168a55ef28dd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:01.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.151+0000 7f1d42ffd640 1 -- 172.21.15.122:0/89884471 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c038070 con 0x7f1d440c46e0 2024-09-15T21:46:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.151+0000 7f1d42ffd640 1 -- 172.21.15.122:0/89884471 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d3c009540 con 0x7f1d440c46e0 2024-09-15T21:46:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.151+0000 7f1d42ffd640 1 -- 172.21.15.122:0/89884471 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c009910 con 0x7f1d440c46e0 2024-09-15T21:46:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.152+0000 7f1d4a665640 1 -- 172.21.15.122:0/89884471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d440c46e0 msgr2=0x7f1d4415be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.152+0000 7f1d4a665640 1 --2- 172.21.15.122:0/89884471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d440c46e0 0x7f1d4415be60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1d440cb720 tx=0x7f1d3c02c760 comp rx=0 tx=0).stop 2024-09-15T21:46:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.152+0000 7f1d4a665640 1 -- 172.21.15.122:0/89884471 shutdown_connections 2024-09-15T21:46:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.152+0000 7f1d4a665640 1 --2- 172.21.15.122:0/89884471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d440c46e0 0x7f1d4415be60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.152+0000 7f1d4a665640 1 --2- 172.21.15.122:0/89884471 >> v2:172.21.15.100:3300/0 conn(0x7f1d4415b370 0x7f1d4406d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.152+0000 7f1d4a665640 1 -- 172.21.15.122:0/89884471 >> 172.21.15.122:0/89884471 conn(0x7f1d440ca390 msgr2=0x7f1d440ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.152+0000 7f1d4a665640 1 -- 172.21.15.122:0/89884471 shutdown_connections 2024-09-15T21:46:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.152+0000 7f1d4a665640 1 -- 172.21.15.122:0/89884471 wait complete. 2024-09-15T21:46:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.153+0000 7f1d4a665640 1 Processor -- start 2024-09-15T21:46:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.153+0000 7f1d4a665640 1 -- start start 2024-09-15T21:46:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.154+0000 7f1d4a665640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1d4406db40 0x7f1d440c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.154+0000 7f1d4a665640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d440c46e0 0x7f1d440cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.154+0000 7f1d4a665640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d4415b370 0x7f1d440cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.154+0000 7f1d4a665640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4406d0e0 con 0x7f1d4415b370 2024-09-15T21:46:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.154+0000 7f1d4a665640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4406ce00 con 0x7f1d440c46e0 2024-09-15T21:46:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.154+0000 7f1d4a665640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1d441523d0 con 0x7f1d4406db40 2024-09-15T21:46:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.154+0000 7f1d437fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d440c46e0 0x7f1d440cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.154+0000 7f1d43fff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1d4406db40 0x7f1d440c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.154+0000 7f1d437fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d440c46e0 0x7f1d440cbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60620/0 (socket says 172.21.15.122:60620) 2024-09-15T21:46:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.154+0000 7f1d437fe640 1 -- 172.21.15.122:0/245311140 learned_addr learned my addr 172.21.15.122:0/245311140 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-15T21:46:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.154+0000 7f1d48c02640 1 --2- 172.21.15.122:0/245311140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d4415b370 0x7f1d440cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.154+0000 7f1d437fe640 1 -- 172.21.15.122:0/245311140 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1d4406db40 msgr2=0x7f1d440c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.154+0000 7f1d437fe640 1 --2- 172.21.15.122:0/245311140 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1d4406db40 0x7f1d440c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.154+0000 7f1d437fe640 1 -- 172.21.15.122:0/245311140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d4415b370 msgr2=0x7f1d440cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.159 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.154+0000 7f1d437fe640 1 --2- 172.21.15.122:0/245311140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d4415b370 0x7f1d440cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.154+0000 7f1d437fe640 1 -- 172.21.15.122:0/245311140 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d440ccb80 con 0x7f1d440c46e0 2024-09-15T21:46:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.154+0000 7f1d43fff640 1 --2- 172.21.15.122:0/245311140 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1d4406db40 0x7f1d440c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:46:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.154+0000 7f1d437fe640 1 --2- 172.21.15.122:0/245311140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d440c46e0 0x7f1d440cbce0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00c390 tx=0x7f1d2c00c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.155+0000 7f1d40ff9640 1 -- 172.21.15.122:0/245311140 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c017070 con 0x7f1d440c46e0 2024-09-15T21:46:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.155+0000 7f1d40ff9640 1 -- 172.21.15.122:0/245311140 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d2c005ce0 con 0x7f1d440c46e0 2024-09-15T21:46:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.155+0000 7f1d4a665640 1 -- 172.21.15.122:0/245311140 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d440cce00 con 0x7f1d440c46e0 2024-09-15T21:46:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.155+0000 7f1d40ff9640 1 -- 172.21.15.122:0/245311140 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c0072f0 con 0x7f1d440c46e0 2024-09-15T21:46:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.155+0000 7f1d4a665640 1 -- 172.21.15.122:0/245311140 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d441547a0 con 0x7f1d440c46e0 2024-09-15T21:46:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.155+0000 7f1d40ff9640 1 -- 172.21.15.122:0/245311140 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1d2c01a070 con 0x7f1d440c46e0 2024-09-15T21:46:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.155+0000 7f1d40ff9640 1 -- 172.21.15.122:0/245311140 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f1d2c012910 con 0x7f1d440c46e0 2024-09-15T21:46:01.160 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.156+0000 7f1d4a665640 1 -- 172.21.15.122:0/245311140 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d440c42b0 con 0x7f1d440c46e0 2024-09-15T21:46:01.161 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.158+0000 7f1d40ff9640 1 -- 172.21.15.122:0/245311140 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1d2c021070 con 0x7f1d440c46e0 2024-09-15T21:46:01.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.229+0000 7f1d4a665640 1 -- 172.21.15.122:0/245311140 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7caee2a0-bab4-45a9-8109-724948c863c6", "id": 4} v 0) -- 0x7f1d440c7e60 con 0x7f1d440c46e0 2024-09-15T21:46:01.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.234+0000 7f1d40ff9640 1 -- 172.21.15.122:0/245311140 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7caee2a0-bab4-45a9-8109-724948c863c6", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f1d440ccb80 con 0x7f1d440c46e0 2024-09-15T21:46:01.236 INFO:teuthology.orchestra.run.smithi122.stdout:4 2024-09-15T21:46:01.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.235+0000 7f1d4a665640 1 -- 172.21.15.122:0/245311140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d440c46e0 msgr2=0x7f1d440cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.235+0000 7f1d4a665640 1 --2- 172.21.15.122:0/245311140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d440c46e0 0x7f1d440cbce0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c00c390 tx=0x7f1d2c00c850 comp rx=0 tx=0).stop 2024-09-15T21:46:01.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.235+0000 7f1d4a665640 1 -- 172.21.15.122:0/245311140 shutdown_connections 2024-09-15T21:46:01.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.235+0000 7f1d4a665640 1 --2- 172.21.15.122:0/245311140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1d440c46e0 0x7f1d440cbce0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.235+0000 7f1d4a665640 1 --2- 172.21.15.122:0/245311140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d4415b370 0x7f1d440cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.235+0000 7f1d4a665640 1 --2- 172.21.15.122:0/245311140 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1d4406db40 0x7f1d440c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.235+0000 7f1d4a665640 1 -- 172.21.15.122:0/245311140 >> 172.21.15.122:0/245311140 conn(0x7f1d440ca390 msgr2=0x7f1d44072560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:01.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.235+0000 7f1d4a665640 1 -- 172.21.15.122:0/245311140 shutdown_connections 2024-09-15T21:46:01.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.235+0000 7f1d4a665640 1 -- 172.21.15.122:0/245311140 wait complete. 2024-09-15T21:46:01.247 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 6e64070a-671a-444b-a061-b3051a29b121 5 2024-09-15T21:46:01.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.344+0000 7ff4e9359640 1 Processor -- start 2024-09-15T21:46:01.346 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.344+0000 7ff4e9359640 1 -- start start 2024-09-15T21:46:01.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.345+0000 7ff4e9359640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff4e4152fa0 0x7ff4e4153380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.345+0000 7ff4e9359640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff4e40c4870 0x7ff4e4071420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.345+0000 7ff4e9359640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4e4071960 con 0x7ff4e40c9550 2024-09-15T21:46:01.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.345+0000 7ff4e9359640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff4e415d300 con 0x7ff4e40c4870 2024-09-15T21:46:01.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.345+0000 7ff4e9359640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e4071bf0 con 0x7ff4e4152fa0 2024-09-15T21:46:01.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.345+0000 7ff4e2ffd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff4e4152fa0 0x7ff4e4153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.345+0000 7ff4e37fe640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7ff4e40c9550 0x7ff4e4159060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:46216/0 (socket says 172.21.15.122:46216) 2024-09-15T21:46:01.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.345+0000 7ff4e37fe640 1 -- 172.21.15.122:0/1975669351 learned_addr learned my addr 172.21.15.122:0/1975669351 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-09-15T21:46:01.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.345+0000 7ff4e27fc640 1 --2- 172.21.15.122:0/1975669351 >> v2:172.21.15.100:3300/0 conn(0x7ff4e40c4870 0x7ff4e4071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.345+0000 7ff4e2ffd640 1 -- 172.21.15.122:0/1975669351 >> v2:172.21.15.100:3300/0 conn(0x7ff4e40c4870 msgr2=0x7ff4e4071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:01.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.345+0000 7ff4e2ffd640 1 --2- 172.21.15.122:0/1975669351 >> v2:172.21.15.100:3300/0 conn(0x7ff4e40c4870 0x7ff4e4071420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.345+0000 7ff4e2ffd640 1 -- 172.21.15.122:0/1975669351 >> v1:172.21.15.122:6789/0 conn(0x7ff4e40c9550 legacy=0x7ff4e4159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.345+0000 7ff4e2ffd640 1 -- 172.21.15.122:0/1975669351 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e415c6f0 con 0x7ff4e4152fa0 2024-09-15T21:46:01.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.345+0000 7ff4e27fc640 1 --2- 172.21.15.122:0/1975669351 >> v2:172.21.15.100:3300/0 conn(0x7ff4e40c4870 0x7ff4e4071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:01.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.346+0000 7ff4e2ffd640 1 --2- 172.21.15.122:0/1975669351 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff4e4152fa0 0x7ff4e4153380 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40097a0 tx=0x7ff4d402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7b82e863b12137 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:01.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.346+0000 7ff4e1ffb640 1 -- 172.21.15.122:0/1975669351 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4037070 con 0x7ff4e4152fa0 2024-09-15T21:46:01.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.346+0000 7ff4e1ffb640 1 -- 172.21.15.122:0/1975669351 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d4032440 con 0x7ff4e4152fa0 2024-09-15T21:46:01.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.346+0000 7ff4e1ffb640 1 -- 172.21.15.122:0/1975669351 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4032810 con 0x7ff4e4152fa0 2024-09-15T21:46:01.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.347+0000 7ff4e9359640 1 -- 172.21.15.122:0/1975669351 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff4e4152fa0 msgr2=0x7ff4e4153380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.347+0000 7ff4e9359640 1 --2- 172.21.15.122:0/1975669351 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff4e4152fa0 0x7ff4e4153380 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40097a0 tx=0x7ff4d402a180 comp rx=0 tx=0).stop 2024-09-15T21:46:01.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.347+0000 7ff4e9359640 1 -- 172.21.15.122:0/1975669351 shutdown_connections 2024-09-15T21:46:01.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.347+0000 7ff4e9359640 1 --2- 172.21.15.122:0/1975669351 >> v2:172.21.15.100:3300/0 conn(0x7ff4e40c4870 0x7ff4e4071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.347+0000 7ff4e9359640 1 --2- 172.21.15.122:0/1975669351 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff4e4152fa0 0x7ff4e4153380 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.347+0000 7ff4e9359640 1 -- 172.21.15.122:0/1975669351 >> 172.21.15.122:0/1975669351 conn(0x7ff4e4067980 msgr2=0x7ff4e4067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:01.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.347+0000 7ff4e9359640 1 -- 172.21.15.122:0/1975669351 shutdown_connections 2024-09-15T21:46:01.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.347+0000 7ff4e9359640 1 -- 172.21.15.122:0/1975669351 wait complete. 2024-09-15T21:46:01.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.348+0000 7ff4e9359640 1 Processor -- start 2024-09-15T21:46:01.354 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.348+0000 7ff4e9359640 1 -- start start 2024-09-15T21:46:01.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.349+0000 7ff4e9359640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff4e40c4870 0x7ff4e4073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.349+0000 7ff4e9359640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4e40c9550 0x7ff4e41eed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.349+0000 7ff4e9359640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e4152fa0 0x7ff4e414d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.349+0000 7ff4e9359640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e415cf90 con 0x7ff4e40c9550 2024-09-15T21:46:01.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.349+0000 7ff4e9359640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e415ccb0 con 0x7ff4e4152fa0 2024-09-15T21:46:01.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.349+0000 7ff4e9359640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e40c4c70 con 0x7ff4e40c4870 2024-09-15T21:46:01.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.349+0000 7ff4e2ffd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff4e40c4870 0x7ff4e4073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.349+0000 7ff4e2ffd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff4e40c4870 0x7ff4e4073670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.122:39328/0 (socket says 172.21.15.122:39328) 2024-09-15T21:46:01.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.349+0000 7ff4e2ffd640 1 -- 172.21.15.122:0/2103621859 learned_addr learned my addr 172.21.15.122:0/2103621859 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-15T21:46:01.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.349+0000 7ff4e37fe640 1 --2- 172.21.15.122:0/2103621859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e4152fa0 0x7ff4e414d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.349+0000 7ff4e27fc640 1 --2- 172.21.15.122:0/2103621859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4e40c9550 0x7ff4e41eed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.349+0000 7ff4e2ffd640 1 -- 172.21.15.122:0/2103621859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e4152fa0 msgr2=0x7ff4e414d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.349+0000 7ff4e2ffd640 1 --2- 172.21.15.122:0/2103621859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e4152fa0 0x7ff4e414d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.349+0000 7ff4e2ffd640 1 -- 172.21.15.122:0/2103621859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4e40c9550 msgr2=0x7ff4e41eed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.349+0000 7ff4e2ffd640 1 --2- 172.21.15.122:0/2103621859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4e40c9550 0x7ff4e41eed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.349+0000 7ff4e2ffd640 1 -- 172.21.15.122:0/2103621859 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e40c82c0 con 0x7ff4e40c4870 2024-09-15T21:46:01.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.350+0000 7ff4e2ffd640 1 --2- 172.21.15.122:0/2103621859 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff4e40c4870 0x7ff4e4073670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff4d402a1c0 tx=0x7ff4d40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:01.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.350+0000 7ff4db7fe640 1 -- 172.21.15.122:0/2103621859 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4037040 con 0x7ff4e40c4870 2024-09-15T21:46:01.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.350+0000 7ff4db7fe640 1 -- 172.21.15.122:0/2103621859 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d4042070 con 0x7ff4e40c4870 2024-09-15T21:46:01.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.350+0000 7ff4e9359640 1 -- 172.21.15.122:0/2103621859 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e41f1fd0 con 0x7ff4e40c4870 2024-09-15T21:46:01.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.350+0000 7ff4db7fe640 1 -- 172.21.15.122:0/2103621859 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d4030bb0 con 0x7ff4e40c4870 2024-09-15T21:46:01.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.350+0000 7ff4e9359640 1 -- 172.21.15.122:0/2103621859 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e41f2570 con 0x7ff4e40c4870 2024-09-15T21:46:01.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.350+0000 7ff4db7fe640 1 -- 172.21.15.122:0/2103621859 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff4d4047460 con 0x7ff4e40c4870 2024-09-15T21:46:01.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.350+0000 7ff4db7fe640 1 -- 172.21.15.122:0/2103621859 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7ff4d4030df0 con 0x7ff4e40c4870 2024-09-15T21:46:01.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.351+0000 7ff4e9359640 1 -- 172.21.15.122:0/2103621859 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e40b50d0 con 0x7ff4e40c4870 2024-09-15T21:46:01.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.353+0000 7ff4db7fe640 1 -- 172.21.15.122:0/2103621859 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ff4d403d440 con 0x7ff4e40c4870 2024-09-15T21:46:01.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.422+0000 7ff4e9359640 1 -- 172.21.15.122:0/2103621859 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6e64070a-671a-444b-a061-b3051a29b121", "id": 5} v 0) -- 0x7ff4e40c3fe0 con 0x7ff4e40c4870 2024-09-15T21:46:01.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.428+0000 7ff4db7fe640 1 -- 172.21.15.122:0/2103621859 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6e64070a-671a-444b-a061-b3051a29b121", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7ff4d403d620 con 0x7ff4e40c4870 2024-09-15T21:46:01.429 INFO:teuthology.orchestra.run.smithi122.stdout:5 2024-09-15T21:46:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.429+0000 7ff4e9359640 1 -- 172.21.15.122:0/2103621859 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff4e40c4870 msgr2=0x7ff4e4073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.429+0000 7ff4e9359640 1 --2- 172.21.15.122:0/2103621859 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff4e40c4870 0x7ff4e4073670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff4d402a1c0 tx=0x7ff4d40046c0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.429+0000 7ff4e9359640 1 -- 172.21.15.122:0/2103621859 shutdown_connections 2024-09-15T21:46:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.429+0000 7ff4e9359640 1 --2- 172.21.15.122:0/2103621859 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff4e4152fa0 0x7ff4e414d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.429+0000 7ff4e9359640 1 --2- 172.21.15.122:0/2103621859 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4e40c9550 0x7ff4e41eed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.429+0000 7ff4e9359640 1 --2- 172.21.15.122:0/2103621859 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff4e40c4870 0x7ff4e4073670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.429+0000 7ff4e9359640 1 -- 172.21.15.122:0/2103621859 >> 172.21.15.122:0/2103621859 conn(0x7ff4e4067980 msgr2=0x7ff4e40ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.429+0000 7ff4e9359640 1 -- 172.21.15.122:0/2103621859 shutdown_connections 2024-09-15T21:46:01.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.429+0000 7ff4e9359640 1 -- 172.21.15.122:0/2103621859 wait complete. 2024-09-15T21:46:01.441 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3 6 2024-09-15T21:46:01.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.537+0000 7f8ddf9d8640 1 Processor -- start 2024-09-15T21:46:01.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.537+0000 7f8ddf9d8640 1 -- start start 2024-09-15T21:46:01.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.538+0000 7f8ddf9d8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8dd80c46e0 0x7f8dd815b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.538+0000 7f8ddf9d8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8dd80c8c70 0x7f8dd815bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.538+0000 7f8ddf9d8640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8dd80c9cf0 con 0x7f8dd815b370 2024-09-15T21:46:01.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.538+0000 7f8ddf9d8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f8dd80c9200 con 0x7f8dd80c8c70 2024-09-15T21:46:01.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.538+0000 7f8ddf9d8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd80c60f0 con 0x7f8dd80c46e0 2024-09-15T21:46:01.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.538+0000 7f8ddcf73640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8dd80c46e0 0x7f8dd815b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.538+0000 7f8ddd774640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f8dd815b370 0x7f8dd806d600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:46222/0 (socket says 172.21.15.122:46222) 2024-09-15T21:46:01.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.538+0000 7f8ddd774640 1 -- 172.21.15.122:0/832150371 learned_addr learned my addr 172.21.15.122:0/832150371 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-09-15T21:46:01.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.538+0000 7f8dddf75640 1 --2- 172.21.15.122:0/832150371 >> v2:172.21.15.100:3300/0 conn(0x7f8dd80c8c70 0x7f8dd815bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.538+0000 7f8dcffff640 1 -- 172.21.15.122:0/832150371 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1218998085 0 0) 0x7f8dd80c9cf0 con 0x7f8dd815b370 2024-09-15T21:46:01.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.538+0000 7f8dcffff640 1 -- 172.21.15.122:0/832150371 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8dc4003520 con 0x7f8dd815b370 2024-09-15T21:46:01.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.538+0000 7f8ddcf73640 1 -- 172.21.15.122:0/832150371 >> v2:172.21.15.100:3300/0 conn(0x7f8dd80c8c70 msgr2=0x7f8dd815bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:01.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.538+0000 7f8ddcf73640 1 --2- 172.21.15.122:0/832150371 >> v2:172.21.15.100:3300/0 conn(0x7f8dd80c8c70 0x7f8dd815bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.538+0000 7f8ddcf73640 1 -- 172.21.15.122:0/832150371 >> v1:172.21.15.122:6789/0 conn(0x7f8dd815b370 legacy=0x7f8dd806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.538+0000 7f8ddcf73640 1 -- 172.21.15.122:0/832150371 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd80c9730 con 0x7f8dd80c46e0 2024-09-15T21:46:01.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.539+0000 7f8dddf75640 1 --2- 172.21.15.122:0/832150371 >> v2:172.21.15.100:3300/0 conn(0x7f8dd80c8c70 0x7f8dd815bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:01.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.539+0000 7f8ddcf73640 1 --2- 172.21.15.122:0/832150371 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8dd80c46e0 0x7f8dd815b750 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40097a0 tx=0x7f8dd402a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b8cbd689b9cf045 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:01.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.539+0000 7f8dcffff640 1 -- 172.21.15.122:0/832150371 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4037070 con 0x7f8dd80c46e0 2024-09-15T21:46:01.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.539+0000 7f8dcffff640 1 -- 172.21.15.122:0/832150371 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd402adb0 con 0x7f8dd80c46e0 2024-09-15T21:46:01.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.539+0000 7f8dcffff640 1 -- 172.21.15.122:0/832150371 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4032640 con 0x7f8dd80c46e0 2024-09-15T21:46:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.540+0000 7f8ddf9d8640 1 -- 172.21.15.122:0/832150371 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8dd80c46e0 msgr2=0x7f8dd815b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.540+0000 7f8ddf9d8640 1 --2- 172.21.15.122:0/832150371 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8dd80c46e0 0x7f8dd815b750 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40097a0 tx=0x7f8dd402a060 comp rx=0 tx=0).stop 2024-09-15T21:46:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.540+0000 7f8ddf9d8640 1 -- 172.21.15.122:0/832150371 shutdown_connections 2024-09-15T21:46:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.540+0000 7f8ddf9d8640 1 --2- 172.21.15.122:0/832150371 >> v2:172.21.15.100:3300/0 conn(0x7f8dd80c8c70 0x7f8dd815bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.540+0000 7f8ddf9d8640 1 --2- 172.21.15.122:0/832150371 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8dd80c46e0 0x7f8dd815b750 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.540+0000 7f8ddf9d8640 1 -- 172.21.15.122:0/832150371 >> 172.21.15.122:0/832150371 conn(0x7f8dd80ca390 msgr2=0x7f8dd80ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.540+0000 7f8ddf9d8640 1 -- 172.21.15.122:0/832150371 shutdown_connections 2024-09-15T21:46:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.540+0000 7f8ddf9d8640 1 -- 172.21.15.122:0/832150371 wait complete. 2024-09-15T21:46:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.541+0000 7f8ddf9d8640 1 Processor -- start 2024-09-15T21:46:01.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.541+0000 7f8ddf9d8640 1 -- start start 2024-09-15T21:46:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.542+0000 7f8ddf9d8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dd80c46e0 0x7f8dd814cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.542+0000 7f8ddf9d8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8dd80c8c70 0x7f8dd81f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.542+0000 7f8ddf9d8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dd815b370 0x7f8dd8141cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.542+0000 7f8ddf9d8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd806d0e0 con 0x7f8dd815b370 2024-09-15T21:46:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.542+0000 7f8ddf9d8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8dd806ce00 con 0x7f8dd80c46e0 2024-09-15T21:46:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.542+0000 7f8ddf9d8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8dd815c1d0 con 0x7f8dd80c8c70 2024-09-15T21:46:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.542+0000 7f8ddd774640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dd80c46e0 0x7f8dd814cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.542+0000 7f8ddcf73640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8dd80c8c70 0x7f8dd81f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.542+0000 7f8ddcf73640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8dd80c8c70 0x7f8dd81f4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.122:39352/0 (socket says 172.21.15.122:39352) 2024-09-15T21:46:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.542+0000 7f8ddcf73640 1 -- 172.21.15.122:0/3089297204 learned_addr learned my addr 172.21.15.122:0/3089297204 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-15T21:46:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.542+0000 7f8ddd774640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dd80c46e0 0x7f8dd814cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60644/0 (socket says 172.21.15.122:60644) 2024-09-15T21:46:01.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.542+0000 7f8dddf75640 1 --2- 172.21.15.122:0/3089297204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dd815b370 0x7f8dd8141cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.542+0000 7f8ddcf73640 1 -- 172.21.15.122:0/3089297204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dd80c46e0 msgr2=0x7f8dd814cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.542+0000 7f8ddcf73640 1 --2- 172.21.15.122:0/3089297204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dd80c46e0 0x7f8dd814cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.542+0000 7f8ddcf73640 1 -- 172.21.15.122:0/3089297204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dd815b370 msgr2=0x7f8dd8141cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.542+0000 7f8ddcf73640 1 --2- 172.21.15.122:0/3089297204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dd815b370 0x7f8dd8141cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.542+0000 7f8ddcf73640 1 -- 172.21.15.122:0/3089297204 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dd80cd3d0 con 0x7f8dd80c8c70 2024-09-15T21:46:01.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.543+0000 7f8ddcf73640 1 --2- 172.21.15.122:0/3089297204 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8dd80c8c70 0x7f8dd81f4f80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4035040 tx=0x7f8dd40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:01.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.543+0000 7f8dcdffb640 1 -- 172.21.15.122:0/3089297204 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4041020 con 0x7f8dd80c8c70 2024-09-15T21:46:01.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.543+0000 7f8dcdffb640 1 -- 172.21.15.122:0/3089297204 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dd402abb0 con 0x7f8dd80c8c70 2024-09-15T21:46:01.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.543+0000 7f8dcdffb640 1 -- 172.21.15.122:0/3089297204 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4037040 con 0x7f8dd80c8c70 2024-09-15T21:46:01.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.543+0000 7f8ddf9d8640 1 -- 172.21.15.122:0/3089297204 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dd80cccb0 con 0x7f8dd80c8c70 2024-09-15T21:46:01.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.543+0000 7f8ddf9d8640 1 -- 172.21.15.122:0/3089297204 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dd81f54c0 con 0x7f8dd80c8c70 2024-09-15T21:46:01.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.543+0000 7f8dcdffb640 1 -- 172.21.15.122:0/3089297204 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8dd4044020 con 0x7f8dd80c8c70 2024-09-15T21:46:01.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.543+0000 7f8dcdffb640 1 -- 172.21.15.122:0/3089297204 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f8dd403d640 con 0x7f8dd80c8c70 2024-09-15T21:46:01.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.544+0000 7f8ddf9d8640 1 -- 172.21.15.122:0/3089297204 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dd81523d0 con 0x7f8dd80c8c70 2024-09-15T21:46:01.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.546+0000 7f8dcdffb640 1 -- 172.21.15.122:0/3089297204 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f8dd404b030 con 0x7f8dd80c8c70 2024-09-15T21:46:01.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.615+0000 7f8ddf9d8640 1 -- 172.21.15.122:0/3089297204 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd new", "uuid": "47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3", "id": 6} v 0) -- 0x7f8dd80c42b0 con 0x7f8dd80c8c70 2024-09-15T21:46:01.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.621+0000 7f8dcdffb640 1 -- 172.21.15.122:0/3089297204 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f8dd402ad80 con 0x7f8dd80c8c70 2024-09-15T21:46:01.623 INFO:teuthology.orchestra.run.smithi122.stdout:6 2024-09-15T21:46:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.622+0000 7f8ddf9d8640 1 -- 172.21.15.122:0/3089297204 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8dd80c8c70 msgr2=0x7f8dd81f4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.622+0000 7f8ddf9d8640 1 --2- 172.21.15.122:0/3089297204 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8dd80c8c70 0x7f8dd81f4f80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8dd4035040 tx=0x7f8dd40046c0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.622+0000 7f8ddf9d8640 1 -- 172.21.15.122:0/3089297204 shutdown_connections 2024-09-15T21:46:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.622+0000 7f8ddf9d8640 1 --2- 172.21.15.122:0/3089297204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8dd815b370 0x7f8dd8141cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.622+0000 7f8ddf9d8640 1 --2- 172.21.15.122:0/3089297204 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8dd80c8c70 0x7f8dd81f4f80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.622+0000 7f8ddf9d8640 1 --2- 172.21.15.122:0/3089297204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dd80c46e0 0x7f8dd814cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.622+0000 7f8ddf9d8640 1 -- 172.21.15.122:0/3089297204 >> 172.21.15.122:0/3089297204 conn(0x7f8dd80ca390 msgr2=0x7f8dd80c7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.622+0000 7f8ddf9d8640 1 -- 172.21.15.122:0/3089297204 shutdown_connections 2024-09-15T21:46:01.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.622+0000 7f8ddf9d8640 1 -- 172.21.15.122:0/3089297204 wait complete. 2024-09-15T21:46:01.634 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 9a916b6d-af82-486f-af40-4a7291c900fe 7 2024-09-15T21:46:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.731+0000 7fb8444f8640 1 Processor -- start 2024-09-15T21:46:01.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.731+0000 7fb8444f8640 1 -- start start 2024-09-15T21:46:01.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.732+0000 7fb8444f8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb83c15b370 0x7fb83c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.732+0000 7fb8444f8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb83c0c9cf0 con 0x7fb83c06db40 2024-09-15T21:46:01.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.732+0000 7fb8444f8640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb83c0c60f0 con 0x7fb83c0c46e0 2024-09-15T21:46:01.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.732+0000 7fb8444f8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb83c0c9200 con 0x7fb83c15b370 2024-09-15T21:46:01.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.732+0000 7fb842294640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb83c15b370 0x7fb83c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.732+0000 7fb842a95640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fb83c0c46e0 0x7fb83c1626a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:46232/0 (socket says 172.21.15.122:46232) 2024-09-15T21:46:01.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.732+0000 7fb842a95640 1 -- 172.21.15.122:0/777800784 learned_addr learned my addr 172.21.15.122:0/777800784 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-09-15T21:46:01.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.733+0000 7fb842294640 1 -- 172.21.15.122:0/777800784 >> v1:172.21.15.122:6789/0 conn(0x7fb83c0c46e0 legacy=0x7fb83c1626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.733+0000 7fb841292640 1 -- 172.21.15.122:0/777800784 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3018286520 0 0) 0x7fb83c0c60f0 con 0x7fb83c0c46e0 2024-09-15T21:46:01.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.733+0000 7fb842294640 1 -- 172.21.15.122:0/777800784 >> v1:172.21.15.100:6789/0 conn(0x7fb83c06db40 legacy=0x7fb83c15b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.733+0000 7fb842294640 1 -- 172.21.15.122:0/777800784 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83c0c9730 con 0x7fb83c15b370 2024-09-15T21:46:01.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.733+0000 7fb842294640 1 --2- 172.21.15.122:0/777800784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb83c15b370 0x7fb83c06d600 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0097d0 tx=0x7fb82c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77f8a0abe8183906 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:01.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.733+0000 7fb841292640 1 -- 172.21.15.122:0/777800784 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c037070 con 0x7fb83c15b370 2024-09-15T21:46:01.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.733+0000 7fb841292640 1 -- 172.21.15.122:0/777800784 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb82c02ae50 con 0x7fb83c15b370 2024-09-15T21:46:01.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.733+0000 7fb841292640 1 -- 172.21.15.122:0/777800784 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82c032660 con 0x7fb83c15b370 2024-09-15T21:46:01.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.734+0000 7fb8444f8640 1 -- 172.21.15.122:0/777800784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb83c15b370 msgr2=0x7fb83c06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.734+0000 7fb8444f8640 1 --2- 172.21.15.122:0/777800784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb83c15b370 0x7fb83c06d600 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0097d0 tx=0x7fb82c02a100 comp rx=0 tx=0).stop 2024-09-15T21:46:01.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.734+0000 7fb8444f8640 1 -- 172.21.15.122:0/777800784 shutdown_connections 2024-09-15T21:46:01.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.734+0000 7fb8444f8640 1 --2- 172.21.15.122:0/777800784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb83c15b370 0x7fb83c06d600 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.734+0000 7fb8444f8640 1 -- 172.21.15.122:0/777800784 >> 172.21.15.122:0/777800784 conn(0x7fb83c0ca390 msgr2=0x7fb83c0ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:01.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.734+0000 7fb8444f8640 1 -- 172.21.15.122:0/777800784 shutdown_connections 2024-09-15T21:46:01.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.734+0000 7fb8444f8640 1 -- 172.21.15.122:0/777800784 wait complete. 2024-09-15T21:46:01.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.735+0000 7fb8444f8640 1 Processor -- start 2024-09-15T21:46:01.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.735+0000 7fb8444f8640 1 -- start start 2024-09-15T21:46:01.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.736+0000 7fb8444f8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb83c06db40 0x7fb83c1e4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.736+0000 7fb8444f8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb83c0c46e0 0x7fb83c153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.736+0000 7fb8444f8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb83c15b370 0x7fb83c1541e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.736+0000 7fb8444f8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c06d0e0 con 0x7fb83c06db40 2024-09-15T21:46:01.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.736+0000 7fb8444f8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb83c06ce00 con 0x7fb83c0c46e0 2024-09-15T21:46:01.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.736+0000 7fb8444f8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb83c1523d0 con 0x7fb83c15b370 2024-09-15T21:46:01.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.736+0000 7fb841a93640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb83c0c46e0 0x7fb83c153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.736+0000 7fb842a95640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb83c15b370 0x7fb83c1541e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.736+0000 7fb841a93640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb83c0c46e0 0x7fb83c153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:60658/0 (socket says 172.21.15.122:60658) 2024-09-15T21:46:01.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.736+0000 7fb841a93640 1 -- 172.21.15.122:0/423959542 learned_addr learned my addr 172.21.15.122:0/423959542 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-15T21:46:01.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.736+0000 7fb842294640 1 --2- 172.21.15.122:0/423959542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb83c06db40 0x7fb83c1e4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.736+0000 7fb842a95640 1 -- 172.21.15.122:0/423959542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb83c0c46e0 msgr2=0x7fb83c153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.736+0000 7fb842a95640 1 --2- 172.21.15.122:0/423959542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb83c0c46e0 0x7fb83c153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.737+0000 7fb842a95640 1 -- 172.21.15.122:0/423959542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb83c06db40 msgr2=0x7fb83c1e4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.737+0000 7fb842a95640 1 --2- 172.21.15.122:0/423959542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb83c06db40 0x7fb83c1e4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.737+0000 7fb842a95640 1 -- 172.21.15.122:0/423959542 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb83c0c7630 con 0x7fb83c15b370 2024-09-15T21:46:01.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.737+0000 7fb841a93640 1 --2- 172.21.15.122:0/423959542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb83c0c46e0 0x7fb83c153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:46:01.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.737+0000 7fb842a95640 1 --2- 172.21.15.122:0/423959542 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb83c15b370 0x7fb83c1541e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb83400c780 tx=0x7fb83400cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:01.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.737+0000 7fb832ffd640 1 -- 172.21.15.122:0/423959542 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb834018070 con 0x7fb83c15b370 2024-09-15T21:46:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.737+0000 7fb832ffd640 1 -- 172.21.15.122:0/423959542 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb834004d60 con 0x7fb83c15b370 2024-09-15T21:46:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.737+0000 7fb832ffd640 1 -- 172.21.15.122:0/423959542 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83400a880 con 0x7fb83c15b370 2024-09-15T21:46:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.737+0000 7fb8444f8640 1 -- 172.21.15.122:0/423959542 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb83c1f1530 con 0x7fb83c15b370 2024-09-15T21:46:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.737+0000 7fb8444f8640 1 -- 172.21.15.122:0/423959542 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb83c1f1af0 con 0x7fb83c15b370 2024-09-15T21:46:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.737+0000 7fb832ffd640 1 -- 172.21.15.122:0/423959542 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb83401b070 con 0x7fb83c15b370 2024-09-15T21:46:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.737+0000 7fb832ffd640 1 -- 172.21.15.122:0/423959542 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fb834007760 con 0x7fb83c15b370 2024-09-15T21:46:01.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.738+0000 7fb8444f8640 1 -- 172.21.15.122:0/423959542 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb83c147e50 con 0x7fb83c15b370 2024-09-15T21:46:01.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.740+0000 7fb832ffd640 1 -- 172.21.15.122:0/423959542 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb83c147e50 con 0x7fb83c15b370 2024-09-15T21:46:01.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.810+0000 7fb8444f8640 1 -- 172.21.15.122:0/423959542 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9a916b6d-af82-486f-af40-4a7291c900fe", "id": 7} v 0) -- 0x7fb83c0c42b0 con 0x7fb83c15b370 2024-09-15T21:46:01.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.818+0000 7fb832ffd640 1 -- 172.21.15.122:0/423959542 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9a916b6d-af82-486f-af40-4a7291c900fe", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fb8340070d0 con 0x7fb83c15b370 2024-09-15T21:46:01.820 INFO:teuthology.orchestra.run.smithi122.stdout:7 2024-09-15T21:46:01.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.819+0000 7fb8444f8640 1 -- 172.21.15.122:0/423959542 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb83c15b370 msgr2=0x7fb83c1541e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.819+0000 7fb8444f8640 1 --2- 172.21.15.122:0/423959542 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb83c15b370 0x7fb83c1541e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb83400c780 tx=0x7fb83400cc40 comp rx=0 tx=0).stop 2024-09-15T21:46:01.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.819+0000 7fb8444f8640 1 -- 172.21.15.122:0/423959542 shutdown_connections 2024-09-15T21:46:01.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.819+0000 7fb8444f8640 1 --2- 172.21.15.122:0/423959542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb83c0c46e0 0x7fb83c153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.819+0000 7fb8444f8640 1 --2- 172.21.15.122:0/423959542 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb83c15b370 0x7fb83c1541e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.819+0000 7fb8444f8640 1 --2- 172.21.15.122:0/423959542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb83c06db40 0x7fb83c1e4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.819+0000 7fb8444f8640 1 -- 172.21.15.122:0/423959542 >> 172.21.15.122:0/423959542 conn(0x7fb83c0ca390 msgr2=0x7fb83c150180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:01.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.819+0000 7fb8444f8640 1 -- 172.21.15.122:0/423959542 shutdown_connections 2024-09-15T21:46:01.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-15T21:46:01.819+0000 7fb8444f8640 1 -- 172.21.15.122:0/423959542 wait complete. 2024-09-15T21:46:01.831 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T21:46:01.831 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-15T21:46:01.833 INFO:tasks.ceph.osd.0:Started 2024-09-15T21:46:01.833 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T21:46:01.833 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-15T21:46:01.835 INFO:tasks.ceph.osd.1:Started 2024-09-15T21:46:01.835 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T21:46:01.836 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-15T21:46:01.837 INFO:tasks.ceph.osd.2:Started 2024-09-15T21:46:01.838 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T21:46:01.838 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-15T21:46:01.840 INFO:tasks.ceph.osd.3:Started 2024-09-15T21:46:01.840 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-15T21:46:01.840 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-15T21:46:01.843 INFO:tasks.ceph.osd.4:Started 2024-09-15T21:46:01.843 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-15T21:46:01.843 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-15T21:46:01.845 INFO:tasks.ceph.osd.5:Started 2024-09-15T21:46:01.845 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-15T21:46:01.845 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-15T21:46:01.848 INFO:tasks.ceph.osd.6:Started 2024-09-15T21:46:01.848 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-15T21:46:01.848 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-15T21:46:01.850 INFO:tasks.ceph.osd.7:Started 2024-09-15T21:46:01.850 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:46:01.858 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:01.856+0000 7efd4012b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T21:46:01.932 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:01.930+0000 7efd4012b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T21:46:01.949 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:01.948+0000 7ffadb000040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-15T21:46:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.970+0000 7f393aad9640 1 Processor -- start 2024-09-15T21:46:01.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.970+0000 7f393aad9640 1 -- start start 2024-09-15T21:46:01.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.971+0000 7f393aad9640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f39340cb7d0 0x7f39340cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.971+0000 7f393aad9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f393406d0e0 0x7f39340cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.971+0000 7f393aad9640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f393415e7a0 con 0x7f3934155440 2024-09-15T21:46:01.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.971+0000 7f393aad9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f393415b470 con 0x7f393406d0e0 2024-09-15T21:46:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.971+0000 7f393aad9640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f3934155820 con 0x7f39340cb7d0 2024-09-15T21:46:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.971+0000 7f3939076640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f393406d0e0 0x7f39340cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.972+0000 7f3939076640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f393406d0e0 0x7f39340cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41428/0 (socket says 172.21.15.100:41428) 2024-09-15T21:46:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.972+0000 7f3939076640 1 -- 172.21.15.100:0/3705765868 learned_addr learned my addr 172.21.15.100:0/3705765868 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.972+0000 7f3933fff640 1 --2- 172.21.15.100:0/3705765868 >> v2:172.21.15.122:3300/0 conn(0x7f39340cb7d0 0x7f39340cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.972+0000 7f3939076640 1 -- 172.21.15.100:0/3705765868 >> v2:172.21.15.122:3300/0 conn(0x7f39340cb7d0 msgr2=0x7f39340cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.972+0000 7f3939076640 1 --2- 172.21.15.100:0/3705765868 >> v2:172.21.15.122:3300/0 conn(0x7f39340cb7d0 0x7f39340cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.972+0000 7f3939076640 1 -- 172.21.15.100:0/3705765868 >> v1:172.21.15.100:6789/0 conn(0x7f3934155440 legacy=0x7f39340cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.972+0000 7f3939076640 1 -- 172.21.15.100:0/3705765868 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393415ea40 con 0x7f393406d0e0 2024-09-15T21:46:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.972+0000 7f3939076640 1 --2- 172.21.15.100:0/3705765868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f393406d0e0 0x7f39340cc720 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3934158ca0 tx=0x7f391c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26865af46361d158 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.973+0000 7f39337fe640 1 -- 172.21.15.100:0/3705765868 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c037070 con 0x7f393406d0e0 2024-09-15T21:46:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.973+0000 7f39337fe640 1 -- 172.21.15.100:0/3705765868 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f391c032440 con 0x7f393406d0e0 2024-09-15T21:46:01.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.973+0000 7f39337fe640 1 -- 172.21.15.100:0/3705765868 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c032850 con 0x7f393406d0e0 2024-09-15T21:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.973+0000 7f393aad9640 1 -- 172.21.15.100:0/3705765868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f393406d0e0 msgr2=0x7f39340cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.973+0000 7f393aad9640 1 --2- 172.21.15.100:0/3705765868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f393406d0e0 0x7f39340cc720 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3934158ca0 tx=0x7f391c02a210 comp rx=0 tx=0).stop 2024-09-15T21:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.973+0000 7f393aad9640 1 -- 172.21.15.100:0/3705765868 shutdown_connections 2024-09-15T21:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.973+0000 7f393aad9640 1 --2- 172.21.15.100:0/3705765868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f393406d0e0 0x7f39340cc720 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.973+0000 7f393aad9640 1 --2- 172.21.15.100:0/3705765868 >> v2:172.21.15.122:3300/0 conn(0x7f39340cb7d0 0x7f39340cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.973+0000 7f393aad9640 1 -- 172.21.15.100:0/3705765868 >> 172.21.15.100:0/3705765868 conn(0x7f39340680d0 msgr2=0x7f393406d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.973+0000 7f393aad9640 1 -- 172.21.15.100:0/3705765868 shutdown_connections 2024-09-15T21:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.973+0000 7f393aad9640 1 -- 172.21.15.100:0/3705765868 wait complete. 2024-09-15T21:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.975+0000 7f393aad9640 1 Processor -- start 2024-09-15T21:46:01.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.975+0000 7f393aad9640 1 -- start start 2024-09-15T21:46:01.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.976+0000 7f393aad9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f393406d0e0 0x7f393414edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.976+0000 7f393aad9640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39340cb7d0 0x7f3934148a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.976+0000 7f393aad9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3934155440 0x7f3934148f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:01.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.976+0000 7f393aad9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f393415f2e0 con 0x7f393406d0e0 2024-09-15T21:46:01.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.976+0000 7f393aad9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f393415f000 con 0x7f3934155440 2024-09-15T21:46:01.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.976+0000 7f393aad9640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f393415b070 con 0x7f39340cb7d0 2024-09-15T21:46:01.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.976+0000 7f3938875640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f393406d0e0 0x7f393414edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.976+0000 7f3938875640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f393406d0e0 0x7f393414edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41442/0 (socket says 172.21.15.100:41442) 2024-09-15T21:46:01.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.976+0000 7f3938875640 1 -- 172.21.15.100:0/2867786476 learned_addr learned my addr 172.21.15.100:0/2867786476 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:01.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.976+0000 7f3933fff640 1 --2- 172.21.15.100:0/2867786476 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39340cb7d0 0x7f3934148a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.976+0000 7f3939076640 1 --2- 172.21.15.100:0/2867786476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3934155440 0x7f3934148f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:01.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.976+0000 7f3938875640 1 -- 172.21.15.100:0/2867786476 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39340cb7d0 msgr2=0x7f3934148a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.976+0000 7f3938875640 1 --2- 172.21.15.100:0/2867786476 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39340cb7d0 0x7f3934148a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.976+0000 7f3938875640 1 -- 172.21.15.100:0/2867786476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3934155440 msgr2=0x7f3934148f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:01.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.976+0000 7f3938875640 1 --2- 172.21.15.100:0/2867786476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3934155440 0x7f3934148f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:01.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.976+0000 7f3938875640 1 -- 172.21.15.100:0/2867786476 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393414a960 con 0x7f393406d0e0 2024-09-15T21:46:01.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.976+0000 7f3938875640 1 --2- 172.21.15.100:0/2867786476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f393406d0e0 0x7f393414edd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f392c00c430 tx=0x7f392c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:01.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.976+0000 7f39317fa640 1 -- 172.21.15.100:0/2867786476 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c0052d0 con 0x7f393406d0e0 2024-09-15T21:46:01.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.977+0000 7f393aad9640 1 -- 172.21.15.100:0/2867786476 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39340b5980 con 0x7f393406d0e0 2024-09-15T21:46:01.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.977+0000 7f39317fa640 1 -- 172.21.15.100:0/2867786476 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f392c005490 con 0x7f393406d0e0 2024-09-15T21:46:01.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.977+0000 7f39317fa640 1 -- 172.21.15.100:0/2867786476 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c007260 con 0x7f393406d0e0 2024-09-15T21:46:01.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.977+0000 7f393aad9640 1 -- 172.21.15.100:0/2867786476 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39340b5170 con 0x7f393406d0e0 2024-09-15T21:46:01.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.978+0000 7f39317fa640 1 -- 172.21.15.100:0/2867786476 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f392c007400 con 0x7f393406d0e0 2024-09-15T21:46:01.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.978+0000 7f39317fa640 1 -- 172.21.15.100:0/2867786476 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f392c011c60 con 0x7f393406d0e0 2024-09-15T21:46:01.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.978+0000 7f393aad9640 1 -- 172.21.15.100:0/2867786476 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f393415bcf0 con 0x7f393406d0e0 2024-09-15T21:46:01.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:01.980+0000 7f39317fa640 1 -- 172.21.15.100:0/2867786476 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f393415bcf0 con 0x7f393406d0e0 2024-09-15T21:46:02.003 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:02.001+0000 7ffadb000040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-15T21:46:02.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:02.038+0000 7f393aad9640 1 -- 172.21.15.100:0/2867786476 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39340b3940 con 0x7f393406d0e0 2024-09-15T21:46:02.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:02.039+0000 7f39317fa640 1 -- 172.21.15.100:0/2867786476 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f392c0035f0 con 0x7f393406d0e0 2024-09-15T21:46:02.042 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:02.042 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":10,"fsid":"30e2fff4-2171-4531-8806-0e03b63f494f","created":"2024-09-15T21:45:59.746897+0000","modified":"2024-09-15T21:46:01.812129+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T21:46:01.812129+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5887900b-522d-476f-9704-cc2c5ee87a60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"adc17993-5c50-4251-85e1-4c7ac62e25e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8de3b7d5-1980-4a3b-a934-264f867c7c8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"93dbcd1c-6a6f-49cf-bc94-e64b883de5f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7caee2a0-bab4-45a9-8109-724948c863c6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6e64070a-671a-444b-a061-b3051a29b121","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9a916b6d-af82-486f-af40-4a7291c900fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:46:02.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:02.040+0000 7f393aad9640 1 -- 172.21.15.100:0/2867786476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f393406d0e0 msgr2=0x7f393414edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:02.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:02.040+0000 7f393aad9640 1 --2- 172.21.15.100:0/2867786476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f393406d0e0 0x7f393414edd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f392c00c430 tx=0x7f392c00c8f0 comp rx=0 tx=0).stop 2024-09-15T21:46:02.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:02.040+0000 7f393aad9640 1 -- 172.21.15.100:0/2867786476 shutdown_connections 2024-09-15T21:46:02.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:02.040+0000 7f393aad9640 1 --2- 172.21.15.100:0/2867786476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3934155440 0x7f3934148f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:02.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:02.040+0000 7f393aad9640 1 --2- 172.21.15.100:0/2867786476 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39340cb7d0 0x7f3934148a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:02.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:02.040+0000 7f393aad9640 1 --2- 172.21.15.100:0/2867786476 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f393406d0e0 0x7f393414edd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:02.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:02.040+0000 7f393aad9640 1 -- 172.21.15.100:0/2867786476 >> 172.21.15.100:0/2867786476 conn(0x7f39340680d0 msgr2=0x7f39340cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:02.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:02.040+0000 7f393aad9640 1 -- 172.21.15.100:0/2867786476 shutdown_connections 2024-09-15T21:46:02.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:02.040+0000 7f393aad9640 1 -- 172.21.15.100:0/2867786476 wait complete. 2024-09-15T21:46:02.052 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-15T21:46:02.052 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-15T21:46:02.307 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:02.306+0000 7efd4012b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T21:46:02.366 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:02.365+0000 7ffadb000040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-15T21:46:02.392 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:02.390+0000 7efd4012b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T21:46:02.450 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:02.448+0000 7ffadb000040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-15T21:46:02.500 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:02.498+0000 7efd4012b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T21:46:02.546 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:02.544+0000 7efd4012b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T21:46:02.558 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:02.556+0000 7ffadb000040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-15T21:46:02.603 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:02.601+0000 7ffadb000040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-15T21:46:02.657 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:02.655+0000 7efd4012b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T21:46:02.709 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:02.707+0000 7efd4012b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T21:46:02.713 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:02.712+0000 7ffadb000040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-15T21:46:02.755 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:02.753+0000 7efd4012b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T21:46:02.768 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:02.766+0000 7ffadb000040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-15T21:46:02.813 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:02.811+0000 7ffadb000040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-15T21:46:02.858 DEBUG:teuthology.orchestra.run.smithi100:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T21:46:02.868 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:02.867+0000 7efd4012b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T21:46:02.921 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:02.919+0000 7efd4012b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T21:46:02.922 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:02.921+0000 7ffadb000040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-15T21:46:02.948 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.946+0000 7fd78af1a640 1 Processor -- start 2024-09-15T21:46:02.949 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.947+0000 7fd78af1a640 1 -- start start 2024-09-15T21:46:02.950 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.947+0000 7fd78af1a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd7840cef90 0x7fd784070b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:02.950 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.947+0000 7fd78af1a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd7840c8640 0x7fd7840c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:02.951 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.947+0000 7fd78af1a640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7840c9c70 con 0x7fd7840cebb0 2024-09-15T21:46:02.951 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.947+0000 7fd78af1a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fd784158c80 con 0x7fd7840c8640 2024-09-15T21:46:02.951 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.947+0000 7fd78af1a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd7840c8140 con 0x7fd7840cef90 2024-09-15T21:46:02.951 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.947+0000 7fd7894b7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd7840c8640 0x7fd7840c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:02.951 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.948+0000 7fd7894b7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd7840c8640 0x7fd7840c8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41458/0 (socket says 172.21.15.100:41458) 2024-09-15T21:46:02.951 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.948+0000 7fd7894b7640 1 -- 172.21.15.100:0/983831467 learned_addr learned my addr 172.21.15.100:0/983831467 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:02.951 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.948+0000 7fd783fff640 1 --2- 172.21.15.100:0/983831467 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd7840cef90 0x7fd784070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:02.951 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.948+0000 7fd7894b7640 1 -- 172.21.15.100:0/983831467 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd7840cef90 msgr2=0x7fd784070b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:02.951 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.948+0000 7fd7837fe640 1 -- 172.21.15.100:0/983831467 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 527211806 0 0) 0x7fd778001110 con 0x7fd7840cebb0 2024-09-15T21:46:02.951 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.948+0000 7fd7894b7640 1 --2- 172.21.15.100:0/983831467 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd7840cef90 0x7fd784070b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:02.951 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.948+0000 7fd7894b7640 1 -- 172.21.15.100:0/983831467 >> v1:172.21.15.122:6789/0 conn(0x7fd7840cebb0 legacy=0x7fd7840c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:02.951 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.948+0000 7fd7894b7640 1 -- 172.21.15.100:0/983831467 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7840c5ea0 con 0x7fd7840c8640 2024-09-15T21:46:02.951 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.949+0000 7fd7894b7640 1 --2- 172.21.15.100:0/983831467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7840c8640 0x7fd7840c8ab0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009e50 tx=0x7fd77c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98bccbabc2ce4bef server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:02.952 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.949+0000 7fd7837fe640 1 -- 172.21.15.100:0/983831467 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c036070 con 0x7fd7840c8640 2024-09-15T21:46:02.952 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.949+0000 7fd7837fe640 1 -- 172.21.15.100:0/983831467 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd77c031600 con 0x7fd7840c8640 2024-09-15T21:46:02.952 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.949+0000 7fd7837fe640 1 -- 172.21.15.100:0/983831467 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c0319d0 con 0x7fd7840c8640 2024-09-15T21:46:02.955 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.949+0000 7fd78af1a640 1 -- 172.21.15.100:0/983831467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7840c8640 msgr2=0x7fd7840c8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:02.955 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.949+0000 7fd78af1a640 1 --2- 172.21.15.100:0/983831467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7840c8640 0x7fd7840c8ab0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009e50 tx=0x7fd77c02a7a0 comp rx=0 tx=0).stop 2024-09-15T21:46:02.955 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.949+0000 7fd78af1a640 1 -- 172.21.15.100:0/983831467 shutdown_connections 2024-09-15T21:46:02.955 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.949+0000 7fd78af1a640 1 --2- 172.21.15.100:0/983831467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7840c8640 0x7fd7840c8ab0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:02.955 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.949+0000 7fd78af1a640 1 --2- 172.21.15.100:0/983831467 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd7840cef90 0x7fd784070b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:02.955 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.949+0000 7fd78af1a640 1 -- 172.21.15.100:0/983831467 >> 172.21.15.100:0/983831467 conn(0x7fd784067aa0 msgr2=0x7fd784067e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:02.955 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.949+0000 7fd78af1a640 1 -- 172.21.15.100:0/983831467 shutdown_connections 2024-09-15T21:46:02.956 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.949+0000 7fd78af1a640 1 -- 172.21.15.100:0/983831467 wait complete. 2024-09-15T21:46:02.956 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.950+0000 7fd78af1a640 1 Processor -- start 2024-09-15T21:46:02.956 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.951+0000 7fd78af1a640 1 -- start start 2024-09-15T21:46:02.956 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.951+0000 7fd78af1a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd7840c8640 0x7fd7841f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:02.956 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.951+0000 7fd78af1a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7840cebb0 0x7fd7840cd580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:02.956 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.951+0000 7fd78af1a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7840cef90 0x7fd7840cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:02.956 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.951+0000 7fd78af1a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd7840c6740 con 0x7fd7840cebb0 2024-09-15T21:46:02.956 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.951+0000 7fd78af1a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd7840c6460 con 0x7fd7840cef90 2024-09-15T21:46:02.956 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.951+0000 7fd78af1a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd7840c6a20 con 0x7fd7840c8640 2024-09-15T21:46:02.956 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.951+0000 7fd783fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7840cebb0 0x7fd7840cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:02.956 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.952+0000 7fd783fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7840cebb0 0x7fd7840cd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:41468/0 (socket says 172.21.15.100:41468) 2024-09-15T21:46:02.956 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.952+0000 7fd783fff640 1 -- 172.21.15.100:0/684853900 learned_addr learned my addr 172.21.15.100:0/684853900 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:02.956 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.952+0000 7fd788cb6640 1 --2- 172.21.15.100:0/684853900 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd7840c8640 0x7fd7841f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:02.957 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.952+0000 7fd7894b7640 1 --2- 172.21.15.100:0/684853900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7840cef90 0x7fd7840cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:02.957 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.952+0000 7fd783fff640 1 -- 172.21.15.100:0/684853900 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd7840c8640 msgr2=0x7fd7841f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:02.957 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.952+0000 7fd783fff640 1 --2- 172.21.15.100:0/684853900 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd7840c8640 0x7fd7841f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:02.957 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.952+0000 7fd783fff640 1 -- 172.21.15.100:0/684853900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7840cef90 msgr2=0x7fd7840cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:02.957 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.952+0000 7fd783fff640 1 --2- 172.21.15.100:0/684853900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7840cef90 0x7fd7840cdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:02.957 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.952+0000 7fd783fff640 1 -- 172.21.15.100:0/684853900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7841e2330 con 0x7fd7840cebb0 2024-09-15T21:46:02.957 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.952+0000 7fd788cb6640 1 --2- 172.21.15.100:0/684853900 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd7840c8640 0x7fd7841f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:46:02.957 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.952+0000 7fd783fff640 1 --2- 172.21.15.100:0/684853900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7840cebb0 0x7fd7840cd580 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd77000e870 tx=0x7fd77000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:02.957 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.952+0000 7fd7817fa640 1 -- 172.21.15.100:0/684853900 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770017070 con 0x7fd7840cebb0 2024-09-15T21:46:02.957 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.952+0000 7fd7817fa640 1 -- 172.21.15.100:0/684853900 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd77000c940 con 0x7fd7840cebb0 2024-09-15T21:46:02.957 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.952+0000 7fd78af1a640 1 -- 172.21.15.100:0/684853900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7841e2610 con 0x7fd7840cebb0 2024-09-15T21:46:02.957 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.952+0000 7fd7817fa640 1 -- 172.21.15.100:0/684853900 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77000cd30 con 0x7fd7840cebb0 2024-09-15T21:46:02.957 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.952+0000 7fd78af1a640 1 -- 172.21.15.100:0/684853900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7840ccd80 con 0x7fd7840cebb0 2024-09-15T21:46:02.958 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.952+0000 7fd7817fa640 1 -- 172.21.15.100:0/684853900 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd77001a070 con 0x7fd7840cebb0 2024-09-15T21:46:02.958 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.952+0000 7fd7817fa640 1 -- 172.21.15.100:0/684853900 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fd77001cb20 con 0x7fd7840cebb0 2024-09-15T21:46:02.958 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.953+0000 7fd78af1a640 1 -- 172.21.15.100:0/684853900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7840c6de0 con 0x7fd7840cebb0 2024-09-15T21:46:02.958 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:02.955+0000 7fd7817fa640 1 -- 172.21.15.100:0/684853900 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd7840c6de0 con 0x7fd7840cebb0 2024-09-15T21:46:02.973 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:02.971+0000 7ffadb000040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-15T21:46:02.976 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:02.974+0000 7efd4012b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T21:46:03.016 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:03.014+0000 7fd78af1a640 1 -- 172.21.15.100:0/684853900 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd784157dc0 con 0x7fd7840cebb0 2024-09-15T21:46:03.017 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:03.015+0000 7fd7817fa640 1 -- 172.21.15.100:0/684853900 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fd784157dc0 con 0x7fd7840cebb0 2024-09-15T21:46:03.017 INFO:teuthology.misc.health.smithi100.stdout: 2024-09-15T21:46:03.017 INFO:teuthology.misc.health.smithi100.stdout:{"epoch":10,"fsid":"30e2fff4-2171-4531-8806-0e03b63f494f","created":"2024-09-15T21:45:59.746897+0000","modified":"2024-09-15T21:46:01.812129+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T21:46:01.812129+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5887900b-522d-476f-9704-cc2c5ee87a60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"adc17993-5c50-4251-85e1-4c7ac62e25e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8de3b7d5-1980-4a3b-a934-264f867c7c8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"93dbcd1c-6a6f-49cf-bc94-e64b883de5f6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7caee2a0-bab4-45a9-8109-724948c863c6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6e64070a-671a-444b-a061-b3051a29b121","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9a916b6d-af82-486f-af40-4a7291c900fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:46:03.018 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:03.016+0000 7fd78af1a640 1 -- 172.21.15.100:0/684853900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7840cebb0 msgr2=0x7fd7840cd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:03.018 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:03.016+0000 7fd78af1a640 1 --2- 172.21.15.100:0/684853900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7840cebb0 0x7fd7840cd580 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd77000e870 tx=0x7fd77000ed30 comp rx=0 tx=0).stop 2024-09-15T21:46:03.018 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:03.016+0000 7fd78af1a640 1 -- 172.21.15.100:0/684853900 shutdown_connections 2024-09-15T21:46:03.018 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:03.016+0000 7fd78af1a640 1 --2- 172.21.15.100:0/684853900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7840cef90 0x7fd7840cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:03.018 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:03.016+0000 7fd78af1a640 1 --2- 172.21.15.100:0/684853900 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd7840cebb0 0x7fd7840cd580 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:03.018 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:03.016+0000 7fd78af1a640 1 --2- 172.21.15.100:0/684853900 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd7840c8640 0x7fd7841f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:03.018 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:03.016+0000 7fd78af1a640 1 -- 172.21.15.100:0/684853900 >> 172.21.15.100:0/684853900 conn(0x7fd784067aa0 msgr2=0x7fd7840c9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:03.019 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:03.016+0000 7fd78af1a640 1 -- 172.21.15.100:0/684853900 shutdown_connections 2024-09-15T21:46:03.019 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:03.016+0000 7fd78af1a640 1 -- 172.21.15.100:0/684853900 wait complete. 2024-09-15T21:46:03.024 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:03.022+0000 7ffadb000040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-15T21:46:03.027 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-15T21:46:03.047 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:03.045+0000 7efd4012b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T21:46:03.092 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:03.090+0000 7ffadb000040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-15T21:46:03.093 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:03.091+0000 7efd4012b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T21:46:03.135 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:03.134+0000 7ffadb000040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-15T21:46:03.136 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:03.135+0000 7efd4012b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T21:46:03.179 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:03.177+0000 7ffadb000040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-15T21:46:03.556 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:46:03.554+0000 7f44e10ae640 -1 Falling back to public interface 2024-09-15T21:46:03.569 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:46:03.567+0000 7fa4cf6b0640 -1 Falling back to public interface 2024-09-15T21:46:03.797 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:03.794+0000 7efd4012b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T21:46:03.805 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:46:03.802+0000 7f9853545640 -1 Falling back to public interface 2024-09-15T21:46:03.808 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:46:03.805+0000 7f55154ef640 -1 Falling back to public interface 2024-09-15T21:46:03.809 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:46:03.806+0000 7f1318d1e640 -1 Falling back to public interface 2024-09-15T21:46:03.810 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:46:03.807+0000 7f1317cc6640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6822/0: (98) Address already in use 2024-09-15T21:46:03.816 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:46:03.814+0000 7fa8fea42640 -1 Falling back to public interface 2024-09-15T21:46:03.818 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:46:03.816+0000 7f7c65810640 -1 Falling back to public interface 2024-09-15T21:46:03.824 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:46:03.822+0000 7f7434a16640 -1 Falling back to public interface 2024-09-15T21:46:03.834 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-15T21:46:03.833+0000 7ffadb000040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-15T21:46:04.812 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:04.809+0000 7efcdff96640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-15T21:46:04.812 INFO:tasks.ceph.mgr.y.smithi100.stderr:2024-09-15T21:46:04.809+0000 7efcdff96640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-15T21:46:07.569 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:46:07.567+0000 7f9853545640 -1 osd.3 0 log_to_monitors true 2024-09-15T21:46:07.890 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:46:07.888+0000 7f44e10ae640 -1 osd.4 0 log_to_monitors true 2024-09-15T21:46:07.904 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:46:07.902+0000 7fa4cf6b0640 -1 osd.1 0 log_to_monitors true 2024-09-15T21:46:08.122 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:46:08.120+0000 7f55154ef640 -1 osd.0 0 log_to_monitors true 2024-09-15T21:46:08.130 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:46:08.128+0000 7f1318d1e640 -1 osd.2 0 log_to_monitors true 2024-09-15T21:46:08.170 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:46:08.167+0000 7f7434a16640 -1 osd.5 0 log_to_monitors true 2024-09-15T21:46:08.391 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:46:08.388+0000 7f7c65810640 -1 osd.7 0 log_to_monitors true 2024-09-15T21:46:08.436 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:46:08.433+0000 7fa8fea42640 -1 osd.6 0 log_to_monitors true 2024-09-15T21:46:09.732 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:46:09.730+0000 7f984fce8640 -1 osd.3 0 waiting for initial osdmap 2024-09-15T21:46:09.742 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:46:09.741+0000 7f98472e3640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:46:09.833 DEBUG:teuthology.orchestra.run.smithi100:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T21:46:09.940 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.939+0000 7f0d6da15640 1 Processor -- start 2024-09-15T21:46:09.941 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.939+0000 7f0d6da15640 1 -- start start 2024-09-15T21:46:09.942 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.939+0000 7f0d6da15640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0d68152200 0x7f0d68158920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:09.943 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.939+0000 7f0d6da15640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0d680c67e0 0x7f0d68162790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:09.943 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.939+0000 7f0d6da15640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d680c3480 con 0x7f0d6806e640 2024-09-15T21:46:09.943 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.939+0000 7f0d6da15640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f0d6806d740 con 0x7f0d680c67e0 2024-09-15T21:46:09.943 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.939+0000 7f0d6da15640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0d680c36f0 con 0x7f0d68152200 2024-09-15T21:46:09.943 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.940+0000 7f0d677fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0d680c67e0 0x7f0d68162790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:09.943 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.940+0000 7f0d677fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0d680c67e0 0x7f0d68162790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:34552/0 (socket says 172.21.15.100:34552) 2024-09-15T21:46:09.943 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.940+0000 7f0d677fe640 1 -- 172.21.15.100:0/3932334591 learned_addr learned my addr 172.21.15.100:0/3932334591 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:09.943 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.940+0000 7f0d667fc640 1 --1- 172.21.15.100:0/3932334591 >> v1:172.21.15.100:6789/0 conn(0x7f0d6806e640 0x7f0d680c60d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42104/0 (socket says 172.21.15.100:42104) 2024-09-15T21:46:09.943 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.940+0000 7f0d66ffd640 1 --2- 172.21.15.100:0/3932334591 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0d68152200 0x7f0d68158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:09.943 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.940+0000 7f0d677fe640 1 -- 172.21.15.100:0/3932334591 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0d68152200 msgr2=0x7f0d68158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:09.943 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.940+0000 7f0d677fe640 1 --2- 172.21.15.100:0/3932334591 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0d68152200 0x7f0d68158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:09.943 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.940+0000 7f0d677fe640 1 -- 172.21.15.100:0/3932334591 >> v1:172.21.15.100:6789/0 conn(0x7f0d6806e640 legacy=0x7f0d680c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:09.944 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.940+0000 7f0d677fe640 1 -- 172.21.15.100:0/3932334591 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d68158e60 con 0x7f0d680c67e0 2024-09-15T21:46:09.944 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.940+0000 7f0d65ffb640 1 -- 172.21.15.100:0/3932334591 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3831808738 0 0) 0x7f0d680c3480 con 0x7f0d6806e640 2024-09-15T21:46:09.944 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.941+0000 7f0d677fe640 1 --2- 172.21.15.100:0/3932334591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d680c67e0 0x7f0d68162790 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009e50 tx=0x7f0d6002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e52e5220f37f8c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:09.944 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.941+0000 7f0d65ffb640 1 -- 172.21.15.100:0/3932334591 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d60036070 con 0x7f0d680c67e0 2024-09-15T21:46:09.944 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.941+0000 7f0d65ffb640 1 -- 172.21.15.100:0/3932334591 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d60031600 con 0x7f0d680c67e0 2024-09-15T21:46:09.944 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.941+0000 7f0d65ffb640 1 -- 172.21.15.100:0/3932334591 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d600319d0 con 0x7f0d680c67e0 2024-09-15T21:46:09.945 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.941+0000 7f0d6da15640 1 -- 172.21.15.100:0/3932334591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d680c67e0 msgr2=0x7f0d68162790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:09.945 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.941+0000 7f0d6da15640 1 --2- 172.21.15.100:0/3932334591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d680c67e0 0x7f0d68162790 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009e50 tx=0x7f0d6002a7a0 comp rx=0 tx=0).stop 2024-09-15T21:46:09.945 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.942+0000 7f0d6da15640 1 -- 172.21.15.100:0/3932334591 shutdown_connections 2024-09-15T21:46:09.945 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.942+0000 7f0d6da15640 1 --2- 172.21.15.100:0/3932334591 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d680c67e0 0x7f0d68162790 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:09.945 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.942+0000 7f0d6da15640 1 --2- 172.21.15.100:0/3932334591 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0d68152200 0x7f0d68158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:09.945 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.942+0000 7f0d6da15640 1 -- 172.21.15.100:0/3932334591 >> 172.21.15.100:0/3932334591 conn(0x7f0d6806bc10 msgr2=0x7f0d6806c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:09.945 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.942+0000 7f0d6da15640 1 -- 172.21.15.100:0/3932334591 shutdown_connections 2024-09-15T21:46:09.946 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.942+0000 7f0d6da15640 1 -- 172.21.15.100:0/3932334591 wait complete. 2024-09-15T21:46:09.946 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.943+0000 7f0d6da15640 1 Processor -- start 2024-09-15T21:46:09.946 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.943+0000 7f0d6da15640 1 -- start start 2024-09-15T21:46:09.948 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.944+0000 7f0d6da15640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d6806e640 0x7f0d68156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:09.948 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.944+0000 7f0d6da15640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0d680c67e0 0x7f0d681f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:09.948 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.944+0000 7f0d6da15640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d68152200 0x7f0d681f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:09.948 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.944+0000 7f0d6da15640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0d680c7730 con 0x7f0d68152200 2024-09-15T21:46:09.948 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.944+0000 7f0d6da15640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0d680c5e30 con 0x7f0d6806e640 2024-09-15T21:46:09.948 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.944+0000 7f0d6da15640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0d6806d990 con 0x7f0d680c67e0 2024-09-15T21:46:09.949 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.944+0000 7f0d667fc640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0d680c67e0 0x7f0d681f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:09.949 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.944+0000 7f0d677fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d68152200 0x7f0d681f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:09.949 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.944+0000 7f0d66ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d6806e640 0x7f0d68156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:09.949 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.944+0000 7f0d667fc640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0d680c67e0 0x7f0d681f12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:50234/0 (socket says 172.21.15.100:50234) 2024-09-15T21:46:09.949 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.944+0000 7f0d667fc640 1 -- 172.21.15.100:0/2887105452 learned_addr learned my addr 172.21.15.100:0/2887105452 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:09.949 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.944+0000 7f0d667fc640 1 -- 172.21.15.100:0/2887105452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d6806e640 msgr2=0x7f0d68156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:09.949 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.944+0000 7f0d667fc640 1 --2- 172.21.15.100:0/2887105452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d6806e640 0x7f0d68156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:09.949 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.944+0000 7f0d667fc640 1 -- 172.21.15.100:0/2887105452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d68152200 msgr2=0x7f0d681f1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:09.949 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.944+0000 7f0d667fc640 1 --2- 172.21.15.100:0/2887105452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d68152200 0x7f0d681f1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:09.949 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.944+0000 7f0d667fc640 1 -- 172.21.15.100:0/2887105452 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d681f4ee0 con 0x7f0d680c67e0 2024-09-15T21:46:09.949 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.944+0000 7f0d677fe640 1 --2- 172.21.15.100:0/2887105452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d68152200 0x7f0d681f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:46:09.949 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.944+0000 7f0d667fc640 1 --2- 172.21.15.100:0/2887105452 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0d680c67e0 0x7f0d681f12c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00c4b0 tx=0x7f0d5c00c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:09.949 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.944+0000 7f0d4ffff640 1 -- 172.21.15.100:0/2887105452 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c017070 con 0x7f0d680c67e0 2024-09-15T21:46:09.950 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.945+0000 7f0d4ffff640 1 -- 172.21.15.100:0/2887105452 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d5c006480 con 0x7f0d680c67e0 2024-09-15T21:46:09.950 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.945+0000 7f0d6da15640 1 -- 172.21.15.100:0/2887105452 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d681f0dc0 con 0x7f0d680c67e0 2024-09-15T21:46:09.950 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.945+0000 7f0d4ffff640 1 -- 172.21.15.100:0/2887105452 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c0049e0 con 0x7f0d680c67e0 2024-09-15T21:46:09.950 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.945+0000 7f0d6da15640 1 -- 172.21.15.100:0/2887105452 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d68157ba0 con 0x7f0d680c67e0 2024-09-15T21:46:09.950 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.946+0000 7f0d6da15640 1 -- 172.21.15.100:0/2887105452 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d681525e0 con 0x7f0d680c67e0 2024-09-15T21:46:09.950 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.946+0000 7f0d4ffff640 1 -- 172.21.15.100:0/2887105452 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 4) ==== 90790+0+0 (secure 0 0 0) 0x7f0d5c01a070 con 0x7f0d680c67e0 2024-09-15T21:46:09.950 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.946+0000 7f0d4ffff640 1 --2- 172.21.15.100:0/2887105452 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0d5406c220 0x7f0d5406e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:09.950 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.946+0000 7f0d4ffff640 1 -- 172.21.15.100:0/2887105452 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(13..13 src has 1..13) ==== 2511+0+0 (secure 0 0 0) 0x7f0d5c089a50 con 0x7f0d680c67e0 2024-09-15T21:46:09.950 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.946+0000 7f0d66ffd640 1 --2- 172.21.15.100:0/2887105452 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0d5406c220 0x7f0d5406e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:09.950 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.947+0000 7f0d66ffd640 1 --2- 172.21.15.100:0/2887105452 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0d5406c220 0x7f0d5406e6e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009aa0 tx=0x7f0d58009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:09.953 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:09.951+0000 7f0d4ffff640 1 -- 172.21.15.100:0/2887105452 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0d5c0588e0 con 0x7f0d680c67e0 2024-09-15T21:46:10.184 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:10.183+0000 7f0d6da15640 1 -- 172.21.15.100:0/2887105452 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d68073a60 con 0x7f0d680c67e0 2024-09-15T21:46:10.199 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:10.197+0000 7f0d4ffff640 1 -- 172.21.15.100:0/2887105452 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7567 (secure 0 0 0) 0x7f0d5c05cbe0 con 0x7f0d680c67e0 2024-09-15T21:46:10.199 INFO:teuthology.misc.health.smithi100.stdout: 2024-09-15T21:46:10.199 INFO:teuthology.misc.health.smithi100.stdout:{"epoch":13,"fsid":"30e2fff4-2171-4531-8806-0e03b63f494f","created":"2024-09-15T21:45:59.746897+0000","modified":"2024-09-15T21:46:09.819080+0000","last_up_change":"2024-09-15T21:46:09.819080+0000","last_in_change":"2024-09-15T21:46:01.812129+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5887900b-522d-476f-9704-cc2c5ee87a60","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"adc17993-5c50-4251-85e1-4c7ac62e25e6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"8de3b7d5-1980-4a3b-a934-264f867c7c8d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"93dbcd1c-6a6f-49cf-bc94-e64b883de5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6811","nonce":1287562741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6813","nonce":1287562741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6817","nonce":1287562741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6815","nonce":1287562741}]},"public_addr":"172.21.15.100:6811/1287562741","cluster_addr":"172.21.15.100:6813/1287562741","heartbeat_back_addr":"172.21.15.100:6817/1287562741","heartbeat_front_addr":"172.21.15.100:6815/1287562741","state":["exists","up"]},{"osd":4,"uuid":"7caee2a0-bab4-45a9-8109-724948c863c6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6e64070a-671a-444b-a061-b3051a29b121","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"9a916b6d-af82-486f-af40-4a7291c900fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:46:10.201 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:10.199+0000 7f0d6da15640 1 -- 172.21.15.100:0/2887105452 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0d5406c220 msgr2=0x7f0d5406e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:10.201 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:10.199+0000 7f0d6da15640 1 --2- 172.21.15.100:0/2887105452 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0d5406c220 0x7f0d5406e6e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009aa0 tx=0x7f0d58009190 comp rx=0 tx=0).stop 2024-09-15T21:46:10.201 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:10.199+0000 7f0d6da15640 1 -- 172.21.15.100:0/2887105452 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0d680c67e0 msgr2=0x7f0d681f12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:10.201 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:10.199+0000 7f0d6da15640 1 --2- 172.21.15.100:0/2887105452 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0d680c67e0 0x7f0d681f12c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00c4b0 tx=0x7f0d5c00c970 comp rx=0 tx=0).stop 2024-09-15T21:46:10.201 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:10.199+0000 7f0d6da15640 1 -- 172.21.15.100:0/2887105452 shutdown_connections 2024-09-15T21:46:10.201 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:10.199+0000 7f0d6da15640 1 --2- 172.21.15.100:0/2887105452 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0d5406c220 0x7f0d5406e6e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:10.201 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:10.199+0000 7f0d6da15640 1 --2- 172.21.15.100:0/2887105452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0d68152200 0x7f0d681f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:10.201 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:10.199+0000 7f0d6da15640 1 --2- 172.21.15.100:0/2887105452 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0d680c67e0 0x7f0d681f12c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:10.201 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:10.199+0000 7f0d6da15640 1 --2- 172.21.15.100:0/2887105452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0d6806e640 0x7f0d68156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:10.202 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:10.199+0000 7f0d6da15640 1 -- 172.21.15.100:0/2887105452 >> 172.21.15.100:0/2887105452 conn(0x7f0d6806bc10 msgr2=0x7f0d680c4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:10.202 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:10.199+0000 7f0d6da15640 1 -- 172.21.15.100:0/2887105452 shutdown_connections 2024-09-15T21:46:10.202 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:10.199+0000 7f0d6da15640 1 -- 172.21.15.100:0/2887105452 wait complete. 2024-09-15T21:46:10.210 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-15T21:46:11.083 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:46:11.082+0000 7fa4cae1c640 -1 osd.1 0 waiting for initial osdmap 2024-09-15T21:46:11.117 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:46:11.115+0000 7fa4c344e640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:46:11.130 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:46:11.128+0000 7f131448a640 -1 osd.2 0 waiting for initial osdmap 2024-09-15T21:46:11.249 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:46:11.247+0000 7f130cabc640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:46:11.643 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:46:11.639+0000 7f742fb77640 -1 osd.5 0 waiting for initial osdmap 2024-09-15T21:46:11.655 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:46:11.653+0000 7f74281a9640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:46:11.833 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:46:11.832+0000 7fa8fa1ae640 -1 osd.6 0 waiting for initial osdmap 2024-09-15T21:46:11.844 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:46:11.842+0000 7fa8f27e0640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:46:11.845 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:46:11.842+0000 7f44dd851640 -1 osd.4 0 waiting for initial osdmap 2024-09-15T21:46:11.850 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:46:11.848+0000 7f7c619ed640 -1 osd.7 0 waiting for initial osdmap 2024-09-15T21:46:11.856 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:46:11.855+0000 7f44d4e4c640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:46:11.864 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:46:11.863+0000 7f7c58fe8640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:46:12.214 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:46:12.212+0000 7f5510c5b640 -1 osd.0 0 waiting for initial osdmap 2024-09-15T21:46:12.224 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:46:12.223+0000 7f550928d640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:46:16.545 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T21:46:16.548 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T21:46:17.017 DEBUG:teuthology.orchestra.run.smithi100:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T21:46:17.109 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.107+0000 7f7f15323640 1 Processor -- start 2024-09-15T21:46:17.109 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.107+0000 7f7f15323640 1 -- start start 2024-09-15T21:46:17.112 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.108+0000 7f7f15323640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f1006dbe0 0x7f7f100ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:17.112 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.108+0000 7f7f15323640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7f100ccf50 0x7f7f1015b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:17.112 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.108+0000 7f7f15323640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f100cc7e0 con 0x7f7f1006dfc0 2024-09-15T21:46:17.112 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.108+0000 7f7f15323640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f7f100cc2b0 con 0x7f7f100ccf50 2024-09-15T21:46:17.112 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.108+0000 7f7f15323640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7f100cb9b0 con 0x7f7f1006dbe0 2024-09-15T21:46:17.113 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.108+0000 7f7f0f7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7f100ccf50 0x7f7f1015b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:17.113 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.108+0000 7f7f0f7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7f100ccf50 0x7f7f1015b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47490/0 (socket says 172.21.15.100:47490) 2024-09-15T21:46:17.113 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.108+0000 7f7f0f7fe640 1 -- 172.21.15.100:0/3190589331 learned_addr learned my addr 172.21.15.100:0/3190589331 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:17.113 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.108+0000 7f7f0effd640 1 --2- 172.21.15.100:0/3190589331 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f1006dbe0 0x7f7f100ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:17.113 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.108+0000 7f7f0f7fe640 1 -- 172.21.15.100:0/3190589331 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f1006dbe0 msgr2=0x7f7f100ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:17.113 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.108+0000 7f7f0f7fe640 1 --2- 172.21.15.100:0/3190589331 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f1006dbe0 0x7f7f100ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:17.113 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.108+0000 7f7f0f7fe640 1 -- 172.21.15.100:0/3190589331 >> v1:172.21.15.122:6789/0 conn(0x7f7f1006dfc0 legacy=0x7f7f101550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:17.113 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.108+0000 7f7f0f7fe640 1 -- 172.21.15.100:0/3190589331 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1015f550 con 0x7f7f100ccf50 2024-09-15T21:46:17.113 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.109+0000 7f7f0f7fe640 1 --2- 172.21.15.100:0/3190589331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f100ccf50 0x7f7f1015b410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009e50 tx=0x7f7f0802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2375e1aa928e2275 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:17.113 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.109+0000 7f7f0dffb640 1 -- 172.21.15.100:0/3190589331 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f08036070 con 0x7f7f100ccf50 2024-09-15T21:46:17.113 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.109+0000 7f7f0dffb640 1 -- 172.21.15.100:0/3190589331 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f08031600 con 0x7f7f100ccf50 2024-09-15T21:46:17.113 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.109+0000 7f7f0dffb640 1 -- 172.21.15.100:0/3190589331 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f080319d0 con 0x7f7f100ccf50 2024-09-15T21:46:17.113 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.110+0000 7f7f15323640 1 -- 172.21.15.100:0/3190589331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f100ccf50 msgr2=0x7f7f1015b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:17.114 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.110+0000 7f7f15323640 1 --2- 172.21.15.100:0/3190589331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f100ccf50 0x7f7f1015b410 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009e50 tx=0x7f7f0802a710 comp rx=0 tx=0).stop 2024-09-15T21:46:17.114 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.110+0000 7f7f15323640 1 -- 172.21.15.100:0/3190589331 shutdown_connections 2024-09-15T21:46:17.114 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.110+0000 7f7f15323640 1 --2- 172.21.15.100:0/3190589331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f100ccf50 0x7f7f1015b410 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:17.114 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.110+0000 7f7f15323640 1 --2- 172.21.15.100:0/3190589331 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f1006dbe0 0x7f7f100ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:17.114 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.110+0000 7f7f15323640 1 -- 172.21.15.100:0/3190589331 >> 172.21.15.100:0/3190589331 conn(0x7f7f1006d1c0 msgr2=0x7f7f10073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:17.114 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.110+0000 7f7f15323640 1 -- 172.21.15.100:0/3190589331 shutdown_connections 2024-09-15T21:46:17.114 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.110+0000 7f7f15323640 1 -- 172.21.15.100:0/3190589331 wait complete. 2024-09-15T21:46:17.116 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.111+0000 7f7f15323640 1 Processor -- start 2024-09-15T21:46:17.116 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.111+0000 7f7f15323640 1 -- start start 2024-09-15T21:46:17.116 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.112+0000 7f7f15323640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f1006dbe0 0x7f7f101f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:17.116 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.112+0000 7f7f15323640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1006dfc0 0x7f7f101f1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:17.116 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.112+0000 7f7f15323640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f100ccf50 0x7f7f1014d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:17.117 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.112+0000 7f7f15323640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1015f1c0 con 0x7f7f1006dfc0 2024-09-15T21:46:17.117 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.112+0000 7f7f15323640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7f100cb640 con 0x7f7f100ccf50 2024-09-15T21:46:17.117 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.112+0000 7f7f15323640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7f100ccbc0 con 0x7f7f1006dbe0 2024-09-15T21:46:17.117 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.112+0000 7f7f0e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1006dfc0 0x7f7f101f1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:17.117 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.112+0000 7f7f0e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1006dfc0 0x7f7f101f1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47500/0 (socket says 172.21.15.100:47500) 2024-09-15T21:46:17.117 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.112+0000 7f7f0effd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f1006dbe0 0x7f7f101f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:17.117 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.112+0000 7f7f0e7fc640 1 -- 172.21.15.100:0/328679389 learned_addr learned my addr 172.21.15.100:0/328679389 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:17.117 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.112+0000 7f7f0f7fe640 1 --2- 172.21.15.100:0/328679389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f100ccf50 0x7f7f1014d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:17.117 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.112+0000 7f7f0e7fc640 1 -- 172.21.15.100:0/328679389 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f1006dbe0 msgr2=0x7f7f101f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:17.117 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.112+0000 7f7f0e7fc640 1 --2- 172.21.15.100:0/328679389 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f1006dbe0 0x7f7f101f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:17.117 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.112+0000 7f7f0e7fc640 1 -- 172.21.15.100:0/328679389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f100ccf50 msgr2=0x7f7f1014d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:17.117 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.112+0000 7f7f0e7fc640 1 --2- 172.21.15.100:0/328679389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f100ccf50 0x7f7f1014d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:17.118 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.112+0000 7f7f0e7fc640 1 -- 172.21.15.100:0/328679389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1015e7c0 con 0x7f7f1006dfc0 2024-09-15T21:46:17.118 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.112+0000 7f7f0e7fc640 1 --2- 172.21.15.100:0/328679389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1006dfc0 0x7f7f101f1780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400c3c0 tx=0x7f7f0400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:17.118 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.113+0000 7f7ef77fe640 1 -- 172.21.15.100:0/328679389 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f040052d0 con 0x7f7f1006dfc0 2024-09-15T21:46:17.118 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.113+0000 7f7ef77fe640 1 -- 172.21.15.100:0/328679389 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f04005490 con 0x7f7f1006dfc0 2024-09-15T21:46:17.118 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.113+0000 7f7f15323640 1 -- 172.21.15.100:0/328679389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f101f1cc0 con 0x7f7f1006dfc0 2024-09-15T21:46:17.118 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.113+0000 7f7ef77fe640 1 -- 172.21.15.100:0/328679389 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f04006a20 con 0x7f7f1006dfc0 2024-09-15T21:46:17.118 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.113+0000 7f7f15323640 1 -- 172.21.15.100:0/328679389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f1015ed40 con 0x7f7f1006dfc0 2024-09-15T21:46:17.118 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.113+0000 7f7f15323640 1 -- 172.21.15.100:0/328679389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f100ce180 con 0x7f7f1006dfc0 2024-09-15T21:46:17.118 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.114+0000 7f7ef77fe640 1 -- 172.21.15.100:0/328679389 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7f04003640 con 0x7f7f1006dfc0 2024-09-15T21:46:17.118 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.114+0000 7f7ef77fe640 1 --2- 172.21.15.100:0/328679389 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7efc06c2e0 0x7f7efc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:17.118 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.114+0000 7f7f0effd640 1 --2- 172.21.15.100:0/328679389 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7efc06c2e0 0x7f7efc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:17.118 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.114+0000 7f7ef77fe640 1 -- 172.21.15.100:0/328679389 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f7f0408a460 con 0x7f7f1006dfc0 2024-09-15T21:46:17.118 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.115+0000 7f7f0effd640 1 --2- 172.21.15.100:0/328679389 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7efc06c2e0 0x7f7efc06e7a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f00006fd0 tx=0x7f7f00008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:17.121 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.119+0000 7f7ef77fe640 1 -- 172.21.15.100:0/328679389 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7f04058960 con 0x7f7f1006dfc0 2024-09-15T21:46:17.288 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.287+0000 7f7f15323640 1 -- 172.21.15.100:0/328679389 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f1014c310 con 0x7f7f1006dfc0 2024-09-15T21:46:17.289 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.287+0000 7f7ef77fe640 1 -- 172.21.15.100:0/328679389 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) ==== 74+0+12968 (secure 0 0 0) 0x7f7f0405cc60 con 0x7f7f1006dfc0 2024-09-15T21:46:17.289 INFO:teuthology.misc.health.smithi100.stdout: 2024-09-15T21:46:17.290 INFO:teuthology.misc.health.smithi100.stdout:{"epoch":18,"fsid":"30e2fff4-2171-4531-8806-0e03b63f494f","created":"2024-09-15T21:45:59.746897+0000","modified":"2024-09-15T21:46:15.233141+0000","last_up_change":"2024-09-15T21:46:13.224709+0000","last_in_change":"2024-09-15T21:46:01.812129+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:46:11.837603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5887900b-522d-476f-9704-cc2c5ee87a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6819","nonce":3920435546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6821","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6822","nonce":3920435546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6829","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6831","nonce":3920435546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6825","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6827","nonce":3920435546}]},"public_addr":"172.21.15.100:6819/3920435546","cluster_addr":"172.21.15.100:6822/3920435546","heartbeat_back_addr":"172.21.15.100:6831/3920435546","heartbeat_front_addr":"172.21.15.100:6827/3920435546","state":["exists","up"]},{"osd":1,"uuid":"adc17993-5c50-4251-85e1-4c7ac62e25e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6801","nonce":3802679972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6803","nonce":3802679972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6807","nonce":3802679972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6805","nonce":3802679972}]},"public_addr":"172.21.15.100:6801/3802679972","cluster_addr":"172.21.15.100:6803/3802679972","heartbeat_back_addr":"172.21.15.100:6807/3802679972","heartbeat_front_addr":"172.21.15.100:6805/3802679972","state":["exists","up"]},{"osd":2,"uuid":"8de3b7d5-1980-4a3b-a934-264f867c7c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6823","nonce":2316787974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6826","nonce":2316787974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6833","nonce":2316787974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6830","nonce":2316787974}]},"public_addr":"172.21.15.100:6823/2316787974","cluster_addr":"172.21.15.100:6826/2316787974","heartbeat_back_addr":"172.21.15.100:6833/2316787974","heartbeat_front_addr":"172.21.15.100:6830/2316787974","state":["exists","up"]},{"osd":3,"uuid":"93dbcd1c-6a6f-49cf-bc94-e64b883de5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6811","nonce":1287562741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6813","nonce":1287562741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6817","nonce":1287562741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6815","nonce":1287562741}]},"public_addr":"172.21.15.100:6811/1287562741","cluster_addr":"172.21.15.100:6813/1287562741","heartbeat_back_addr":"172.21.15.100:6817/1287562741","heartbeat_front_addr":"172.21.15.100:6815/1287562741","state":["exists","up"]},{"osd":4,"uuid":"7caee2a0-bab4-45a9-8109-724948c863c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6801","nonce":4045445512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6803","nonce":4045445512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6807","nonce":4045445512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6805","nonce":4045445512}]},"public_addr":"172.21.15.122:6801/4045445512","cluster_addr":"172.21.15.122:6803/4045445512","heartbeat_back_addr":"172.21.15.122:6807/4045445512","heartbeat_front_addr":"172.21.15.122:6805/4045445512","state":["exists","up"]},{"osd":5,"uuid":"6e64070a-671a-444b-a061-b3051a29b121","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6825","nonce":2400668532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6827","nonce":2400668532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6831","nonce":2400668532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6829","nonce":2400668532}]},"public_addr":"172.21.15.122:6825/2400668532","cluster_addr":"172.21.15.122:6827/2400668532","heartbeat_back_addr":"172.21.15.122:6831/2400668532","heartbeat_front_addr":"172.21.15.122:6829/2400668532","state":["exists","up"]},{"osd":6,"uuid":"47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6809","nonce":1437167311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6811","nonce":1437167311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6815","nonce":1437167311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6813","nonce":1437167311}]},"public_addr":"172.21.15.122:6809/1437167311","cluster_addr":"172.21.15.122:6811/1437167311","heartbeat_back_addr":"172.21.15.122:6815/1437167311","heartbeat_front_addr":"172.21.15.122:6813/1437167311","state":["exists","up"]},{"osd":7,"uuid":"9a916b6d-af82-486f-af40-4a7291c900fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6817","nonce":2431377162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6819","nonce":2431377162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6823","nonce":2431377162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6821","nonce":2431377162}]},"public_addr":"172.21.15.122:6817/2431377162","cluster_addr":"172.21.15.122:6819/2431377162","heartbeat_back_addr":"172.21.15.122:6823/2431377162","heartbeat_front_addr":"172.21.15.122:6821/2431377162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.168771+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.886614+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.154129+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.570531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.926773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.146320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.409137+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.373359+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:46:17.292 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.290+0000 7f7f15323640 1 -- 172.21.15.100:0/328679389 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7efc06c2e0 msgr2=0x7f7efc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:17.292 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.290+0000 7f7f15323640 1 --2- 172.21.15.100:0/328679389 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7efc06c2e0 0x7f7efc06e7a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7f00006fd0 tx=0x7f7f00008040 comp rx=0 tx=0).stop 2024-09-15T21:46:17.292 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.290+0000 7f7f15323640 1 -- 172.21.15.100:0/328679389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1006dfc0 msgr2=0x7f7f101f1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:17.292 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.290+0000 7f7f15323640 1 --2- 172.21.15.100:0/328679389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1006dfc0 0x7f7f101f1780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400c3c0 tx=0x7f7f0400c880 comp rx=0 tx=0).stop 2024-09-15T21:46:17.292 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.290+0000 7f7f15323640 1 -- 172.21.15.100:0/328679389 shutdown_connections 2024-09-15T21:46:17.292 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.290+0000 7f7f15323640 1 --2- 172.21.15.100:0/328679389 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7efc06c2e0 0x7f7efc06e7a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:17.292 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.290+0000 7f7f15323640 1 --2- 172.21.15.100:0/328679389 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7f100ccf50 0x7f7f1014d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:17.292 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.290+0000 7f7f15323640 1 --2- 172.21.15.100:0/328679389 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f1006dfc0 0x7f7f101f1780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:17.292 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.290+0000 7f7f15323640 1 --2- 172.21.15.100:0/328679389 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7f1006dbe0 0x7f7f101f4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:17.293 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.290+0000 7f7f15323640 1 -- 172.21.15.100:0/328679389 >> 172.21.15.100:0/328679389 conn(0x7f7f1006d1c0 msgr2=0x7f7f10073340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:17.293 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.290+0000 7f7f15323640 1 -- 172.21.15.100:0/328679389 shutdown_connections 2024-09-15T21:46:17.293 INFO:teuthology.misc.health.smithi100.stderr:2024-09-15T21:46:17.290+0000 7f7f15323640 1 -- 172.21.15.100:0/328679389 wait complete. 2024-09-15T21:46:17.301 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-15T21:46:17.301 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-15T21:46:17.302 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-15T21:46:17.302 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-15T21:46:17.304 INFO:tasks.ceph.mds.a:Started 2024-09-15T21:46:17.304 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-15T21:46:17.305 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-15T21:46:17.306 INFO:tasks.ceph.mds.c:Started 2024-09-15T21:46:17.306 INFO:tasks.ceph.mds.e:Restarting daemon 2024-09-15T21:46:17.307 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-09-15T21:46:17.308 INFO:tasks.ceph.mds.e:Started 2024-09-15T21:46:17.308 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-15T21:46:17.309 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-15T21:46:17.311 INFO:tasks.ceph.mds.b:Started 2024-09-15T21:46:17.311 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-15T21:46:17.312 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-15T21:46:17.313 INFO:tasks.ceph.mds.d:Started 2024-09-15T21:46:17.314 INFO:tasks.ceph.mds.f:Restarting daemon 2024-09-15T21:46:17.314 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-09-15T21:46:17.316 INFO:tasks.ceph.mds.f:Started 2024-09-15T21:46:17.316 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-15T21:46:17.317 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T21:46:17.381 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-15T21:46:17.382 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-09-15T21:46:17.382 INFO:tasks.ceph.mds.e.smithi100.stdout:starting mds.e at 2024-09-15T21:46:17.394 INFO:tasks.ceph.mds.d.smithi122.stdout:starting mds.d at 2024-09-15T21:46:17.396 INFO:tasks.ceph.mds.f.smithi122.stdout:starting mds.f at 2024-09-15T21:46:17.397 INFO:tasks.ceph.mds.b.smithi122.stdout:starting mds.b at 2024-09-15T21:46:17.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.421+0000 7f34d6823640 1 Processor -- start 2024-09-15T21:46:17.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.421+0000 7f34d6823640 1 -- start start 2024-09-15T21:46:17.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.422+0000 7f34d6823640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34d0073320 0x7f34d0158e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:17.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.422+0000 7f34d6823640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f34d0073700 0x7f34d00c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:17.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.422+0000 7f34d6823640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34d00c91a0 con 0x7f34d00c8a10 2024-09-15T21:46:17.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.422+0000 7f34d6823640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f34d00c9440 con 0x7f34d0073700 2024-09-15T21:46:17.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.422+0000 7f34d6823640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f34d00c9cc0 con 0x7f34d0073320 2024-09-15T21:46:17.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.422+0000 7f34d4dc0640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f34d00c8a10 0x7f34d0152b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46340/0 (socket says 172.21.15.100:46340) 2024-09-15T21:46:17.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.422+0000 7f34d4dc0640 1 -- 172.21.15.100:0/657400562 learned_addr learned my addr 172.21.15.100:0/657400562 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:17.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.422+0000 7f34cffff640 1 --2- 172.21.15.100:0/657400562 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34d0073320 0x7f34d0158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:17.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.422+0000 7f34cf7fe640 1 --2- 172.21.15.100:0/657400562 >> v2:172.21.15.122:3300/0 conn(0x7f34d0073700 0x7f34d00c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:17.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.422+0000 7f34ceffd640 1 -- 172.21.15.100:0/657400562 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2292005048 0 0) 0x7f34d00c91a0 con 0x7f34d00c8a10 2024-09-15T21:46:17.425 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.422+0000 7f34ceffd640 1 -- 172.21.15.100:0/657400562 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34b8003520 con 0x7f34d00c8a10 2024-09-15T21:46:17.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.422+0000 7f34cffff640 1 -- 172.21.15.100:0/657400562 >> v2:172.21.15.122:3300/0 conn(0x7f34d0073700 msgr2=0x7f34d00c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:17.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.422+0000 7f34cffff640 1 --2- 172.21.15.100:0/657400562 >> v2:172.21.15.122:3300/0 conn(0x7f34d0073700 0x7f34d00c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:17.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.422+0000 7f34cffff640 1 -- 172.21.15.100:0/657400562 >> v1:172.21.15.100:6789/0 conn(0x7f34d00c8a10 legacy=0x7f34d0152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:17.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.422+0000 7f34cffff640 1 -- 172.21.15.100:0/657400562 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d00c9700 con 0x7f34d0073320 2024-09-15T21:46:17.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.423+0000 7f34cffff640 1 --2- 172.21.15.100:0/657400562 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34d0073320 0x7f34d0158e70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f34c00097a0 tx=0x7f34c002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80b7e70b5065b9c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:17.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.423+0000 7f34ceffd640 1 -- 172.21.15.100:0/657400562 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0037070 con 0x7f34d0073320 2024-09-15T21:46:17.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.423+0000 7f34ceffd640 1 -- 172.21.15.100:0/657400562 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c0032440 con 0x7f34d0073320 2024-09-15T21:46:17.426 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.423+0000 7f34ceffd640 1 -- 172.21.15.100:0/657400562 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c00327f0 con 0x7f34d0073320 2024-09-15T21:46:17.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.423+0000 7f34d6823640 1 -- 172.21.15.100:0/657400562 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34d0073320 msgr2=0x7f34d0158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:17.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.423+0000 7f34d6823640 1 --2- 172.21.15.100:0/657400562 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34d0073320 0x7f34d0158e70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f34c00097a0 tx=0x7f34c002a180 comp rx=0 tx=0).stop 2024-09-15T21:46:17.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.423+0000 7f34d6823640 1 -- 172.21.15.100:0/657400562 shutdown_connections 2024-09-15T21:46:17.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.423+0000 7f34d6823640 1 --2- 172.21.15.100:0/657400562 >> v2:172.21.15.122:3300/0 conn(0x7f34d0073700 0x7f34d00c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:17.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.424+0000 7f34d6823640 1 --2- 172.21.15.100:0/657400562 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34d0073320 0x7f34d0158e70 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:17.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.424+0000 7f34d6823640 1 -- 172.21.15.100:0/657400562 >> 172.21.15.100:0/657400562 conn(0x7f34d0070ca0 msgr2=0x7f34d0071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:17.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.424+0000 7f34d6823640 1 -- 172.21.15.100:0/657400562 shutdown_connections 2024-09-15T21:46:17.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.424+0000 7f34d6823640 1 -- 172.21.15.100:0/657400562 wait complete. 2024-09-15T21:46:17.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.425+0000 7f34d6823640 1 Processor -- start 2024-09-15T21:46:17.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.425+0000 7f34d6823640 1 -- start start 2024-09-15T21:46:17.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.425+0000 7f34d6823640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34d0073320 0x7f34d01e2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:17.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.425+0000 7f34d6823640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34d0073700 0x7f34d0142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:17.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.425+0000 7f34d6823640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34d00c8a10 0x7f34d00cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:17.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.425+0000 7f34d6823640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f34d00c6020 con 0x7f34d0073700 2024-09-15T21:46:17.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.425+0000 7f34d6823640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f34d00c5d40 con 0x7f34d0073320 2024-09-15T21:46:17.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.425+0000 7f34d6823640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f34d00c78c0 con 0x7f34d00c8a10 2024-09-15T21:46:17.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.426+0000 7f34cf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34d0073700 0x7f34d0142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:17.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.426+0000 7f34cf7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34d0073700 0x7f34d0142b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47572/0 (socket says 172.21.15.100:47572) 2024-09-15T21:46:17.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.426+0000 7f34cf7fe640 1 -- 172.21.15.100:0/2468634191 learned_addr learned my addr 172.21.15.100:0/2468634191 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:17.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.426+0000 7f34cffff640 1 --2- 172.21.15.100:0/2468634191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34d0073320 0x7f34d01e2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:17.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.426+0000 7f34d4dc0640 1 --2- 172.21.15.100:0/2468634191 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34d00c8a10 0x7f34d00cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:17.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.426+0000 7f34cf7fe640 1 -- 172.21.15.100:0/2468634191 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34d00c8a10 msgr2=0x7f34d00cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:17.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.426+0000 7f34cf7fe640 1 --2- 172.21.15.100:0/2468634191 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34d00c8a10 0x7f34d00cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:17.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.426+0000 7f34cf7fe640 1 -- 172.21.15.100:0/2468634191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34d0073320 msgr2=0x7f34d01e2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:17.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.426+0000 7f34cf7fe640 1 --2- 172.21.15.100:0/2468634191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34d0073320 0x7f34d01e2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:17.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.426+0000 7f34cf7fe640 1 -- 172.21.15.100:0/2468634191 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d00cc060 con 0x7f34d0073700 2024-09-15T21:46:17.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.426+0000 7f34cf7fe640 1 --2- 172.21.15.100:0/2468634191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34d0073700 0x7f34d0142b90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f34c400e930 tx=0x7f34c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:17.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.426+0000 7f34ccff9640 1 -- 172.21.15.100:0/2468634191 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4017070 con 0x7f34d0073700 2024-09-15T21:46:17.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.426+0000 7f34ccff9640 1 -- 172.21.15.100:0/2468634191 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34c400ca00 con 0x7f34d0073700 2024-09-15T21:46:17.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.426+0000 7f34d6823640 1 -- 172.21.15.100:0/2468634191 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d01ef730 con 0x7f34d0073700 2024-09-15T21:46:17.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.426+0000 7f34ccff9640 1 -- 172.21.15.100:0/2468634191 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c400cdb0 con 0x7f34d0073700 2024-09-15T21:46:17.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.426+0000 7f34d6823640 1 -- 172.21.15.100:0/2468634191 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d01ef9d0 con 0x7f34d0073700 2024-09-15T21:46:17.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.427+0000 7f34d6823640 1 -- 172.21.15.100:0/2468634191 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34d00ced90 con 0x7f34d0073700 2024-09-15T21:46:17.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.428+0000 7f34ccff9640 1 -- 172.21.15.100:0/2468634191 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f34c401a070 con 0x7f34d0073700 2024-09-15T21:46:17.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.428+0000 7f34ccff9640 1 --2- 172.21.15.100:0/2468634191 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f34bc06c2e0 0x7f34bc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:17.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.428+0000 7f34ccff9640 1 -- 172.21.15.100:0/2468634191 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f34c408b370 con 0x7f34d0073700 2024-09-15T21:46:17.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.428+0000 7f34cffff640 1 --2- 172.21.15.100:0/2468634191 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f34bc06c2e0 0x7f34bc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:17.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.428+0000 7f34cffff640 1 --2- 172.21.15.100:0/2468634191 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f34bc06c2e0 0x7f34bc06e7a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f34c0034000 tx=0x7f34c002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:17.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.432+0000 7f34ccff9640 1 -- 172.21.15.100:0/2468634191 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f34c4059930 con 0x7f34d0073700 2024-09-15T21:46:17.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.632+0000 7f34d6823640 1 -- 172.21.15.100:0/2468634191 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34d0156890 con 0x7f34d0073700 2024-09-15T21:46:17.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.632+0000 7f34ccff9640 1 -- 172.21.15.100:0/2468634191 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1437 (secure 0 0 0) 0x7f34c405dc30 con 0x7f34d0073700 2024-09-15T21:46:17.634 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:17.634 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":2,"btime":"2024-09-15T21:46:17:385130+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1809406468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1809406468},{"type":"v1","addr":"172.21.15.100:6835","nonce":1809406468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-15T21:46:17.635 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 2 2024-09-15T21:46:17.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.634+0000 7f34d6823640 1 -- 172.21.15.100:0/2468634191 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f34bc06c2e0 msgr2=0x7f34bc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:17.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.634+0000 7f34d6823640 1 --2- 172.21.15.100:0/2468634191 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f34bc06c2e0 0x7f34bc06e7a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f34c0034000 tx=0x7f34c002e000 comp rx=0 tx=0).stop 2024-09-15T21:46:17.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.634+0000 7f34d6823640 1 -- 172.21.15.100:0/2468634191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34d0073700 msgr2=0x7f34d0142b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:17.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.634+0000 7f34d6823640 1 --2- 172.21.15.100:0/2468634191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34d0073700 0x7f34d0142b90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f34c400e930 tx=0x7f34c400edf0 comp rx=0 tx=0).stop 2024-09-15T21:46:17.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.634+0000 7f34d6823640 1 -- 172.21.15.100:0/2468634191 shutdown_connections 2024-09-15T21:46:17.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.634+0000 7f34d6823640 1 --2- 172.21.15.100:0/2468634191 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f34bc06c2e0 0x7f34bc06e7a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:17.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.634+0000 7f34d6823640 1 --2- 172.21.15.100:0/2468634191 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f34d00c8a10 0x7f34d00cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:17.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.634+0000 7f34d6823640 1 --2- 172.21.15.100:0/2468634191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f34d0073700 0x7f34d0142b90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:17.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.634+0000 7f34d6823640 1 --2- 172.21.15.100:0/2468634191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34d0073320 0x7f34d01e2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:17.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.634+0000 7f34d6823640 1 -- 172.21.15.100:0/2468634191 >> 172.21.15.100:0/2468634191 conn(0x7f34d0070ca0 msgr2=0x7f34d014d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:17.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.634+0000 7f34d6823640 1 -- 172.21.15.100:0/2468634191 shutdown_connections 2024-09-15T21:46:17.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:17.634+0000 7f34d6823640 1 -- 172.21.15.100:0/2468634191 wait complete. 2024-09-15T21:46:19.647 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T21:46:19.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.756+0000 7f3f9098d640 1 Processor -- start 2024-09-15T21:46:19.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.756+0000 7f3f9098d640 1 -- start start 2024-09-15T21:46:19.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.757+0000 7f3f9098d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3f8c1552e0 0x7f3f8c06d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:19.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.757+0000 7f3f9098d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f3f8c15c5e0 0x7f3f8c15c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:19.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.757+0000 7f3f9098d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f8c06ec60 con 0x7f3f8c0cee00 2024-09-15T21:46:19.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.757+0000 7f3f9098d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f3f8c15af40 con 0x7f3f8c1552e0 2024-09-15T21:46:19.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.757+0000 7f3f9098d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f3f8c15b200 con 0x7f3f8c15c5e0 2024-09-15T21:46:19.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.758+0000 7f3f8a59c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3f8c1552e0 0x7f3f8c06d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:19.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.758+0000 7f3f89d9b640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f3f8c0cee00 0x7f3f8c0730b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46344/0 (socket says 172.21.15.100:46344) 2024-09-15T21:46:19.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.758+0000 7f3f89d9b640 1 -- 172.21.15.100:0/3661653724 learned_addr learned my addr 172.21.15.100:0/3661653724 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:19.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.758+0000 7f3f8ad9d640 1 --2- 172.21.15.100:0/3661653724 >> v2:172.21.15.122:3300/0 conn(0x7f3f8c15c5e0 0x7f3f8c15c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:19.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.758+0000 7f3f8a59c640 1 -- 172.21.15.100:0/3661653724 >> v2:172.21.15.122:3300/0 conn(0x7f3f8c15c5e0 msgr2=0x7f3f8c15c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:19.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.758+0000 7f3f8a59c640 1 --2- 172.21.15.100:0/3661653724 >> v2:172.21.15.122:3300/0 conn(0x7f3f8c15c5e0 0x7f3f8c15c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:19.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.758+0000 7f3f8a59c640 1 -- 172.21.15.100:0/3661653724 >> v1:172.21.15.100:6789/0 conn(0x7f3f8c0cee00 legacy=0x7f3f8c0730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:19.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.758+0000 7f3f8a59c640 1 -- 172.21.15.100:0/3661653724 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8c15b4e0 con 0x7f3f8c1552e0 2024-09-15T21:46:19.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.758+0000 7f3f8959a640 1 -- 172.21.15.100:0/3661653724 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 948061449 0 0) 0x7f3f7c001290 con 0x7f3f8c0cee00 2024-09-15T21:46:19.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.758+0000 7f3f8ad9d640 1 --2- 172.21.15.100:0/3661653724 >> v2:172.21.15.122:3300/0 conn(0x7f3f8c15c5e0 0x7f3f8c15c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:19.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.759+0000 7f3f8a59c640 1 --2- 172.21.15.100:0/3661653724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8c1552e0 0x7f3f8c06d700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c0ced00 tx=0x7f3f7402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a1332089c28d711 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:19.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.759+0000 7f3f8959a640 1 -- 172.21.15.100:0/3661653724 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74037070 con 0x7f3f8c1552e0 2024-09-15T21:46:19.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.759+0000 7f3f8959a640 1 -- 172.21.15.100:0/3661653724 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f74032440 con 0x7f3f8c1552e0 2024-09-15T21:46:19.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.759+0000 7f3f8959a640 1 -- 172.21.15.100:0/3661653724 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74032810 con 0x7f3f8c1552e0 2024-09-15T21:46:19.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.759+0000 7f3f9098d640 1 -- 172.21.15.100:0/3661653724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8c1552e0 msgr2=0x7f3f8c06d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:19.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.759+0000 7f3f9098d640 1 --2- 172.21.15.100:0/3661653724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8c1552e0 0x7f3f8c06d700 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c0ced00 tx=0x7f3f7402a1a0 comp rx=0 tx=0).stop 2024-09-15T21:46:19.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.759+0000 7f3f9098d640 1 -- 172.21.15.100:0/3661653724 shutdown_connections 2024-09-15T21:46:19.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.759+0000 7f3f9098d640 1 --2- 172.21.15.100:0/3661653724 >> v2:172.21.15.122:3300/0 conn(0x7f3f8c15c5e0 0x7f3f8c15c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:19.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.759+0000 7f3f9098d640 1 --2- 172.21.15.100:0/3661653724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8c1552e0 0x7f3f8c06d700 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:19.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.759+0000 7f3f9098d640 1 -- 172.21.15.100:0/3661653724 >> 172.21.15.100:0/3661653724 conn(0x7f3f8c070ca0 msgr2=0x7f3f8c071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:19.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.759+0000 7f3f9098d640 1 -- 172.21.15.100:0/3661653724 shutdown_connections 2024-09-15T21:46:19.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.759+0000 7f3f9098d640 1 -- 172.21.15.100:0/3661653724 wait complete. 2024-09-15T21:46:19.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.761+0000 7f3f9098d640 1 Processor -- start 2024-09-15T21:46:19.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.761+0000 7f3f9098d640 1 -- start start 2024-09-15T21:46:19.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f9098d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8c0cee00 0x7f3f8c152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:19.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f9098d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f8c1552e0 0x7f3f8c146e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:19.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f9098d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3f8c15c5e0 0x7f3f8c1f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:19.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f9098d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c15cf30 con 0x7f3f8c0cee00 2024-09-15T21:46:19.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f9098d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c15baa0 con 0x7f3f8c1552e0 2024-09-15T21:46:19.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f9098d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3f8c0c91a0 con 0x7f3f8c15c5e0 2024-09-15T21:46:19.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f8a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8c0cee00 0x7f3f8c152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:19.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f8a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8c0cee00 0x7f3f8c152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47594/0 (socket says 172.21.15.100:47594) 2024-09-15T21:46:19.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f8a59c640 1 -- 172.21.15.100:0/3961722274 learned_addr learned my addr 172.21.15.100:0/3961722274 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f89d9b640 1 --2- 172.21.15.100:0/3961722274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f8c1552e0 0x7f3f8c146e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f8ad9d640 1 --2- 172.21.15.100:0/3961722274 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3f8c15c5e0 0x7f3f8c1f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f8a59c640 1 -- 172.21.15.100:0/3961722274 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3f8c15c5e0 msgr2=0x7f3f8c1f1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f8a59c640 1 --2- 172.21.15.100:0/3961722274 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3f8c15c5e0 0x7f3f8c1f1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f8a59c640 1 -- 172.21.15.100:0/3961722274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f8c1552e0 msgr2=0x7f3f8c146e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f8a59c640 1 --2- 172.21.15.100:0/3961722274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f8c1552e0 0x7f3f8c146e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f8a59c640 1 -- 172.21.15.100:0/3961722274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8c146660 con 0x7f3f8c0cee00 2024-09-15T21:46:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f8a59c640 1 --2- 172.21.15.100:0/3961722274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8c0cee00 0x7f3f8c152300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3f74000c00 tx=0x7f3f740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f82ffd640 1 -- 172.21.15.100:0/3961722274 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74037040 con 0x7f3f8c0cee00 2024-09-15T21:46:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f82ffd640 1 -- 172.21.15.100:0/3961722274 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f74042070 con 0x7f3f8c0cee00 2024-09-15T21:46:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f9098d640 1 -- 172.21.15.100:0/3961722274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8c0cba20 con 0x7f3f8c0cee00 2024-09-15T21:46:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f82ffd640 1 -- 172.21.15.100:0/3961722274 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74030bd0 con 0x7f3f8c0cee00 2024-09-15T21:46:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.762+0000 7f3f9098d640 1 -- 172.21.15.100:0/3961722274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8c0cbfe0 con 0x7f3f8c0cee00 2024-09-15T21:46:19.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.763+0000 7f3f9098d640 1 -- 172.21.15.100:0/3961722274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f8c15c100 con 0x7f3f8c0cee00 2024-09-15T21:46:19.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.764+0000 7f3f82ffd640 1 -- 172.21.15.100:0/3961722274 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3f74047460 con 0x7f3f8c0cee00 2024-09-15T21:46:19.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.764+0000 7f3f82ffd640 1 --2- 172.21.15.100:0/3961722274 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3f6c06c2e0 0x7f3f6c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:19.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.764+0000 7f3f82ffd640 1 -- 172.21.15.100:0/3961722274 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7f3f740abac0 con 0x7f3f8c0cee00 2024-09-15T21:46:19.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.764+0000 7f3f89d9b640 1 --2- 172.21.15.100:0/3961722274 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3f6c06c2e0 0x7f3f6c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:19.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.765+0000 7f3f89d9b640 1 --2- 172.21.15.100:0/3961722274 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3f6c06c2e0 0x7f3f6c06e7a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c1e63a0 tx=0x7f3f7c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:19.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.769+0000 7f3f82ffd640 1 -- 172.21.15.100:0/3961722274 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3f8c15c100 con 0x7f3f8c0cee00 2024-09-15T21:46:19.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.972+0000 7f3f9098d640 1 -- 172.21.15.100:0/3961722274 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f8c0769a0 con 0x7f3f8c0cee00 2024-09-15T21:46:19.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.972+0000 7f3f82ffd640 1 -- 172.21.15.100:0/3961722274 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+5563 (secure 0 0 0) 0x7f3f8c0769a0 con 0x7f3f8c0cee00 2024-09-15T21:46:19.974 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:19.974 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":3,"btime":"2024-09-15T21:46:18:386273+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2992633397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2992633397},{"type":"v1","addr":"172.21.15.122:6833","nonce":2992633397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1809406468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1809406468},{"type":"v1","addr":"172.21.15.100:6835","nonce":1809406468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4288,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/371886422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":371886422},{"type":"v1","addr":"172.21.15.122:6837","nonce":371886422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6838/276075809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":276075809},{"type":"v1","addr":"172.21.15.100:6838","nonce":276075809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6836/2947743626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6835","nonce":2947743626},{"type":"v1","addr":"172.21.15.122:6836","nonce":2947743626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4293,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/602064326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6837","nonce":602064326},{"type":"v1","addr":"172.21.15.100:6839","nonce":602064326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-15T21:46:19.974 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 3 2024-09-15T21:46:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.974+0000 7f3f9098d640 1 -- 172.21.15.100:0/3961722274 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3f6c06c2e0 msgr2=0x7f3f6c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.974+0000 7f3f9098d640 1 --2- 172.21.15.100:0/3961722274 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3f6c06c2e0 0x7f3f6c06e7a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c1e63a0 tx=0x7f3f7c009040 comp rx=0 tx=0).stop 2024-09-15T21:46:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.974+0000 7f3f9098d640 1 -- 172.21.15.100:0/3961722274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8c0cee00 msgr2=0x7f3f8c152300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.974+0000 7f3f9098d640 1 --2- 172.21.15.100:0/3961722274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8c0cee00 0x7f3f8c152300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3f74000c00 tx=0x7f3f740046c0 comp rx=0 tx=0).stop 2024-09-15T21:46:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.974+0000 7f3f9098d640 1 -- 172.21.15.100:0/3961722274 shutdown_connections 2024-09-15T21:46:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.974+0000 7f3f9098d640 1 --2- 172.21.15.100:0/3961722274 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3f6c06c2e0 0x7f3f6c06e7a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.974+0000 7f3f9098d640 1 --2- 172.21.15.100:0/3961722274 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f8c1552e0 0x7f3f8c146e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.974+0000 7f3f9098d640 1 --2- 172.21.15.100:0/3961722274 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3f8c15c5e0 0x7f3f8c1f1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.974+0000 7f3f9098d640 1 --2- 172.21.15.100:0/3961722274 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f8c0cee00 0x7f3f8c152300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.974+0000 7f3f9098d640 1 -- 172.21.15.100:0/3961722274 >> 172.21.15.100:0/3961722274 conn(0x7f3f8c070ca0 msgr2=0x7f3f8c1541c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.974+0000 7f3f9098d640 1 -- 172.21.15.100:0/3961722274 shutdown_connections 2024-09-15T21:46:19.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:19.974+0000 7f3f9098d640 1 -- 172.21.15.100:0/3961722274 wait complete. 2024-09-15T21:46:19.987 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-15T21:46:20.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.083+0000 7fa91bab7640 1 Processor -- start 2024-09-15T21:46:20.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.083+0000 7fa91bab7640 1 -- start start 2024-09-15T21:46:20.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.084+0000 7fa91bab7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa9141552e0 0x7fa91406d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:20.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.084+0000 7fa91bab7640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa91415c5e0 0x7fa91415ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:20.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.084+0000 7fa91bab7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa91406ec60 con 0x7fa9140cee00 2024-09-15T21:46:20.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.084+0000 7fa91bab7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa91415cf70 con 0x7fa9141552e0 2024-09-15T21:46:20.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.084+0000 7fa91bab7640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa91415af40 con 0x7fa91415c5e0 2024-09-15T21:46:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.084+0000 7fa919853640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa9141552e0 0x7fa91406d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.084+0000 7fa919052640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fa9140cee00 0x7fa9140730b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46348/0 (socket says 172.21.15.100:46348) 2024-09-15T21:46:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.084+0000 7fa919052640 1 -- 172.21.15.100:0/1513115647 learned_addr learned my addr 172.21.15.100:0/1513115647 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.084+0000 7fa91a054640 1 --2- 172.21.15.100:0/1513115647 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa91415c5e0 0x7fa91415ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.084+0000 7fa919853640 1 -- 172.21.15.100:0/1513115647 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa91415c5e0 msgr2=0x7fa91415ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.084+0000 7fa919853640 1 --2- 172.21.15.100:0/1513115647 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa91415c5e0 0x7fa91415ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.084+0000 7fa919853640 1 -- 172.21.15.100:0/1513115647 >> v1:172.21.15.100:6789/0 conn(0x7fa9140cee00 legacy=0x7fa9140730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.084+0000 7fa919853640 1 -- 172.21.15.100:0/1513115647 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91415b200 con 0x7fa9141552e0 2024-09-15T21:46:20.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.084+0000 7fa918851640 1 -- 172.21.15.100:0/1513115647 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2187985956 0 0) 0x7fa904001160 con 0x7fa9140cee00 2024-09-15T21:46:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.085+0000 7fa919853640 1 --2- 172.21.15.100:0/1513115647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9141552e0 0x7fa91406d700 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa9100097a0 tx=0x7fa91002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e2068edab2741b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.085+0000 7fa918851640 1 -- 172.21.15.100:0/1513115647 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910037070 con 0x7fa9141552e0 2024-09-15T21:46:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.085+0000 7fa918851640 1 -- 172.21.15.100:0/1513115647 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa910032440 con 0x7fa9141552e0 2024-09-15T21:46:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.085+0000 7fa918851640 1 -- 172.21.15.100:0/1513115647 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910032810 con 0x7fa9141552e0 2024-09-15T21:46:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.086+0000 7fa91bab7640 1 -- 172.21.15.100:0/1513115647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9141552e0 msgr2=0x7fa91406d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.086+0000 7fa91bab7640 1 --2- 172.21.15.100:0/1513115647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9141552e0 0x7fa91406d700 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa9100097a0 tx=0x7fa91002a180 comp rx=0 tx=0).stop 2024-09-15T21:46:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.086+0000 7fa91bab7640 1 -- 172.21.15.100:0/1513115647 shutdown_connections 2024-09-15T21:46:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.086+0000 7fa91bab7640 1 --2- 172.21.15.100:0/1513115647 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa91415c5e0 0x7fa91415ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.086+0000 7fa91bab7640 1 --2- 172.21.15.100:0/1513115647 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9141552e0 0x7fa91406d700 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.086+0000 7fa91bab7640 1 -- 172.21.15.100:0/1513115647 >> 172.21.15.100:0/1513115647 conn(0x7fa914070ca0 msgr2=0x7fa914071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.086+0000 7fa91bab7640 1 -- 172.21.15.100:0/1513115647 shutdown_connections 2024-09-15T21:46:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.086+0000 7fa91bab7640 1 -- 172.21.15.100:0/1513115647 wait complete. 2024-09-15T21:46:20.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.087+0000 7fa91bab7640 1 Processor -- start 2024-09-15T21:46:20.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.087+0000 7fa91bab7640 1 -- start start 2024-09-15T21:46:20.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa91bab7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9140cee00 0x7fa9140cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa91bab7640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9141552e0 0x7fa914153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa91bab7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa91415c5e0 0x7fa91414f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa91bab7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa91415baa0 con 0x7fa9140cee00 2024-09-15T21:46:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa91bab7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa91415b7c0 con 0x7fa91415c5e0 2024-09-15T21:46:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa91bab7640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa9140c91a0 con 0x7fa9141552e0 2024-09-15T21:46:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa919853640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9140cee00 0x7fa9140cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa919853640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9140cee00 0x7fa9140cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47610/0 (socket says 172.21.15.100:47610) 2024-09-15T21:46:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa919853640 1 -- 172.21.15.100:0/594589747 learned_addr learned my addr 172.21.15.100:0/594589747 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa919052640 1 --2- 172.21.15.100:0/594589747 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9141552e0 0x7fa914153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa91a054640 1 --2- 172.21.15.100:0/594589747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa91415c5e0 0x7fa91414f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:20.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa919853640 1 -- 172.21.15.100:0/594589747 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9141552e0 msgr2=0x7fa914153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:20.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa919853640 1 --2- 172.21.15.100:0/594589747 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9141552e0 0x7fa914153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:20.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa919853640 1 -- 172.21.15.100:0/594589747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa91415c5e0 msgr2=0x7fa91414f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:20.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa919853640 1 --2- 172.21.15.100:0/594589747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa91415c5e0 0x7fa91414f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:20.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa919853640 1 -- 172.21.15.100:0/594589747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9141f11b0 con 0x7fa9140cee00 2024-09-15T21:46:20.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa919853640 1 --2- 172.21.15.100:0/594589747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9140cee00 0x7fa9140cd9c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9100097a0 tx=0x7fa9100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:20.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa90a7fc640 1 -- 172.21.15.100:0/594589747 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910037050 con 0x7fa9140cee00 2024-09-15T21:46:20.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa90a7fc640 1 -- 172.21.15.100:0/594589747 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa910038040 con 0x7fa9140cee00 2024-09-15T21:46:20.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa90a7fc640 1 -- 172.21.15.100:0/594589747 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910004490 con 0x7fa9140cee00 2024-09-15T21:46:20.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa91bab7640 1 -- 172.21.15.100:0/594589747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9141f1450 con 0x7fa9140cee00 2024-09-15T21:46:20.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.088+0000 7fa91bab7640 1 -- 172.21.15.100:0/594589747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9141f1a10 con 0x7fa9140cee00 2024-09-15T21:46:20.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.089+0000 7fa91bab7640 1 -- 172.21.15.100:0/594589747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9140cf250 con 0x7fa9140cee00 2024-09-15T21:46:20.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.090+0000 7fa90a7fc640 1 -- 172.21.15.100:0/594589747 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa91003c6d0 con 0x7fa9140cee00 2024-09-15T21:46:20.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.090+0000 7fa90a7fc640 1 --2- 172.21.15.100:0/594589747 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa8f806c2b0 0x7fa8f806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:20.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.090+0000 7fa919052640 1 --2- 172.21.15.100:0/594589747 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa8f806c2b0 0x7fa8f806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:20.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.090+0000 7fa90a7fc640 1 -- 172.21.15.100:0/594589747 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7fa9100abb00 con 0x7fa9140cee00 2024-09-15T21:46:20.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.091+0000 7fa919052640 1 --2- 172.21.15.100:0/594589747 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa8f806c2b0 0x7fa8f806e770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa904007fd0 tx=0x7fa904007ca0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:20.096 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.095+0000 7fa90a7fc640 1 -- 172.21.15.100:0/594589747 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa91007a5c0 con 0x7fa9140cee00 2024-09-15T21:46:20.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.264+0000 7fa91bab7640 1 -- 172.21.15.100:0/594589747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa914158960 con 0x7fa9140cee00 2024-09-15T21:46:20.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.264+0000 7fa90a7fc640 1 -- 172.21.15.100:0/594589747 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) ==== 84+0+65 (secure 0 0 0) 0x7fa910079ec0 con 0x7fa9140cee00 2024-09-15T21:46:20.266 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:20.266 INFO:teuthology.orchestra.run.smithi100.stdout:[ 2024-09-15T21:46:20.266 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:46:20.267 INFO:teuthology.orchestra.run.smithi100.stdout: "poolnum": 1, 2024-09-15T21:46:20.267 INFO:teuthology.orchestra.run.smithi100.stdout: "poolname": ".mgr" 2024-09-15T21:46:20.267 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:46:20.267 INFO:teuthology.orchestra.run.smithi100.stdout:] 2024-09-15T21:46:20.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.266+0000 7fa91bab7640 1 -- 172.21.15.100:0/594589747 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa8f806c2b0 msgr2=0x7fa8f806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:20.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.266+0000 7fa91bab7640 1 --2- 172.21.15.100:0/594589747 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa8f806c2b0 0x7fa8f806e770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa904007fd0 tx=0x7fa904007ca0 comp rx=0 tx=0).stop 2024-09-15T21:46:20.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.266+0000 7fa91bab7640 1 -- 172.21.15.100:0/594589747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9140cee00 msgr2=0x7fa9140cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:20.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.266+0000 7fa91bab7640 1 --2- 172.21.15.100:0/594589747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9140cee00 0x7fa9140cd9c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa9100097a0 tx=0x7fa9100046c0 comp rx=0 tx=0).stop 2024-09-15T21:46:20.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.266+0000 7fa91bab7640 1 -- 172.21.15.100:0/594589747 shutdown_connections 2024-09-15T21:46:20.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.266+0000 7fa91bab7640 1 --2- 172.21.15.100:0/594589747 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa8f806c2b0 0x7fa8f806e770 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:20.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.266+0000 7fa91bab7640 1 --2- 172.21.15.100:0/594589747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa91415c5e0 0x7fa91414f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:20.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.266+0000 7fa91bab7640 1 --2- 172.21.15.100:0/594589747 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9141552e0 0x7fa914153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:20.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.266+0000 7fa91bab7640 1 --2- 172.21.15.100:0/594589747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9140cee00 0x7fa9140cd9c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:20.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.266+0000 7fa91bab7640 1 -- 172.21.15.100:0/594589747 >> 172.21.15.100:0/594589747 conn(0x7fa914070ca0 msgr2=0x7fa91415d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:20.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.266+0000 7fa91bab7640 1 -- 172.21.15.100:0/594589747 shutdown_connections 2024-09-15T21:46:20.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.266+0000 7fa91bab7640 1 -- 172.21.15.100:0/594589747 wait complete. 2024-09-15T21:46:20.279 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-15T21:46:20.279 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-15T21:46:20.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.380+0000 7ff92ed9f640 1 Processor -- start 2024-09-15T21:46:20.382 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.380+0000 7ff92ed9f640 1 -- start start 2024-09-15T21:46:20.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.381+0000 7ff92ed9f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff9281552e0 0x7ff9280c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:20.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.381+0000 7ff92ed9f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff92815eed0 con 0x7ff92815bbb0 2024-09-15T21:46:20.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.381+0000 7ff92ed9f640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff92815f1b0 con 0x7ff92806bbe0 2024-09-15T21:46:20.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.381+0000 7ff92ed9f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff9280c9400 con 0x7ff9281552e0 2024-09-15T21:46:20.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.381+0000 7ff92dd9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff9281552e0 0x7ff9280c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:20.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.381+0000 7ff92d59c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff92815bbb0 0x7ff92815e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46364/0 (socket says 172.21.15.100:46364) 2024-09-15T21:46:20.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.381+0000 7ff92d59c640 1 -- 172.21.15.100:0/2143517930 learned_addr learned my addr 172.21.15.100:0/2143517930 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:20.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.381+0000 7ff92dd9d640 1 -- 172.21.15.100:0/2143517930 >> v1:172.21.15.122:6789/0 conn(0x7ff92806bbe0 legacy=0x7ff92806bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:20.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.381+0000 7ff92dd9d640 1 -- 172.21.15.100:0/2143517930 >> v1:172.21.15.100:6789/0 conn(0x7ff92815bbb0 legacy=0x7ff92815e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:20.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.381+0000 7ff92cd9b640 1 -- 172.21.15.100:0/2143517930 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1240767394 0 0) 0x7ff92815eed0 con 0x7ff92815bbb0 2024-09-15T21:46:20.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.381+0000 7ff92dd9d640 1 -- 172.21.15.100:0/2143517930 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92815b0d0 con 0x7ff9281552e0 2024-09-15T21:46:20.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.382+0000 7ff92dd9d640 1 --2- 172.21.15.100:0/2143517930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9281552e0 0x7ff9280c8ec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff9240097a0 tx=0x7ff92402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0593f0a85114a34 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:20.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.382+0000 7ff92cd9b640 1 -- 172.21.15.100:0/2143517930 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924037070 con 0x7ff9281552e0 2024-09-15T21:46:20.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.382+0000 7ff92cd9b640 1 -- 172.21.15.100:0/2143517930 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff924032440 con 0x7ff9281552e0 2024-09-15T21:46:20.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.382+0000 7ff92cd9b640 1 -- 172.21.15.100:0/2143517930 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff924032810 con 0x7ff9281552e0 2024-09-15T21:46:20.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.383+0000 7ff92ed9f640 1 -- 172.21.15.100:0/2143517930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9281552e0 msgr2=0x7ff9280c8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:20.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.383+0000 7ff92ed9f640 1 --2- 172.21.15.100:0/2143517930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9281552e0 0x7ff9280c8ec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff9240097a0 tx=0x7ff92402a1e0 comp rx=0 tx=0).stop 2024-09-15T21:46:20.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.383+0000 7ff92ed9f640 1 -- 172.21.15.100:0/2143517930 shutdown_connections 2024-09-15T21:46:20.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.383+0000 7ff92ed9f640 1 --2- 172.21.15.100:0/2143517930 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff9281552e0 0x7ff9280c8ec0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:20.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.383+0000 7ff92ed9f640 1 -- 172.21.15.100:0/2143517930 >> 172.21.15.100:0/2143517930 conn(0x7ff9280ca120 msgr2=0x7ff9280ca530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:20.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.383+0000 7ff92ed9f640 1 -- 172.21.15.100:0/2143517930 shutdown_connections 2024-09-15T21:46:20.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.383+0000 7ff92ed9f640 1 -- 172.21.15.100:0/2143517930 wait complete. 2024-09-15T21:46:20.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.384+0000 7ff92ed9f640 1 Processor -- start 2024-09-15T21:46:20.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.384+0000 7ff92ed9f640 1 -- start start 2024-09-15T21:46:20.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff92ed9f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff92806bbe0 0x7ff928142310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:20.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff92ed9f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9281552e0 0x7ff928155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:20.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff92ed9f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff92815bbb0 0x7ff928157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:20.388 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff92ed9f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff92815b690 con 0x7ff92815bbb0 2024-09-15T21:46:20.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff92ed9f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff9280c85b0 con 0x7ff9281552e0 2024-09-15T21:46:20.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff92ed9f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff92815ebc0 con 0x7ff92806bbe0 2024-09-15T21:46:20.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff92e59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff92815bbb0 0x7ff928157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:20.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff92e59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff92815bbb0 0x7ff928157450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47626/0 (socket says 172.21.15.100:47626) 2024-09-15T21:46:20.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff92e59e640 1 -- 172.21.15.100:0/1802261535 learned_addr learned my addr 172.21.15.100:0/1802261535 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:20.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff92d59c640 1 --2- 172.21.15.100:0/1802261535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9281552e0 0x7ff928155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:20.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff92dd9d640 1 --2- 172.21.15.100:0/1802261535 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff92806bbe0 0x7ff928142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:20.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff92e59e640 1 -- 172.21.15.100:0/1802261535 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff92806bbe0 msgr2=0x7ff928142310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:20.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff92e59e640 1 --2- 172.21.15.100:0/1802261535 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff92806bbe0 0x7ff928142310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:20.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff92e59e640 1 -- 172.21.15.100:0/1802261535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9281552e0 msgr2=0x7ff928155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:20.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff92e59e640 1 --2- 172.21.15.100:0/1802261535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9281552e0 0x7ff928155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:20.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff92e59e640 1 -- 172.21.15.100:0/1802261535 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9281f0c00 con 0x7ff92815bbb0 2024-09-15T21:46:20.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff92e59e640 1 --2- 172.21.15.100:0/1802261535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff92815bbb0 0x7ff928157450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff92000b9a0 tx=0x7ff92000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:20.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff91e7fc640 1 -- 172.21.15.100:0/1802261535 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920019070 con 0x7ff92815bbb0 2024-09-15T21:46:20.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff91e7fc640 1 -- 172.21.15.100:0/1802261535 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff92000cba0 con 0x7ff92815bbb0 2024-09-15T21:46:20.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff91e7fc640 1 -- 172.21.15.100:0/1802261535 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920014440 con 0x7ff92815bbb0 2024-09-15T21:46:20.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff92ed9f640 1 -- 172.21.15.100:0/1802261535 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9281f0ec0 con 0x7ff92815bbb0 2024-09-15T21:46:20.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.385+0000 7ff92ed9f640 1 -- 172.21.15.100:0/1802261535 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9281f1480 con 0x7ff92815bbb0 2024-09-15T21:46:20.390 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.386+0000 7ff92ed9f640 1 -- 172.21.15.100:0/1802261535 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff928068a90 con 0x7ff92815bbb0 2024-09-15T21:46:20.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.387+0000 7ff91e7fc640 1 -- 172.21.15.100:0/1802261535 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff92001c070 con 0x7ff92815bbb0 2024-09-15T21:46:20.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.387+0000 7ff91e7fc640 1 --2- 172.21.15.100:0/1802261535 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff90c06c2b0 0x7ff90c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:20.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.387+0000 7ff91e7fc640 1 -- 172.21.15.100:0/1802261535 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 4857+0+0 (secure 0 0 0) 0x7ff92008abb0 con 0x7ff92815bbb0 2024-09-15T21:46:20.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.387+0000 7ff92dd9d640 1 --2- 172.21.15.100:0/1802261535 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff90c06c2b0 0x7ff90c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:20.391 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.388+0000 7ff92dd9d640 1 --2- 172.21.15.100:0/1802261535 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff90c06c2b0 0x7ff90c06e770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff924034000 tx=0x7ff92402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:20.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.391+0000 7ff91e7fc640 1 -- 172.21.15.100:0/1802261535 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff928068a90 con 0x7ff92815bbb0 2024-09-15T21:46:20.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:20.559+0000 7ff92ed9f640 1 -- 172.21.15.100:0/1802261535 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff9280c8ec0 con 0x7ff92815bbb0 2024-09-15T21:46:21.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.400+0000 7ff91e7fc640 1 -- 172.21.15.100:0/1802261535 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) ==== 138+0+0 (secure 0 0 0) 0x7ff9200590b0 con 0x7ff92815bbb0 2024-09-15T21:46:21.403 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' created 2024-09-15T21:46:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.403+0000 7ff92ed9f640 1 -- 172.21.15.100:0/1802261535 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff90c06c2b0 msgr2=0x7ff90c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.403+0000 7ff92ed9f640 1 --2- 172.21.15.100:0/1802261535 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff90c06c2b0 0x7ff90c06e770 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff924034000 tx=0x7ff92402e000 comp rx=0 tx=0).stop 2024-09-15T21:46:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.403+0000 7ff92ed9f640 1 -- 172.21.15.100:0/1802261535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff92815bbb0 msgr2=0x7ff928157450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.403+0000 7ff92ed9f640 1 --2- 172.21.15.100:0/1802261535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff92815bbb0 0x7ff928157450 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff92000b9a0 tx=0x7ff92000be60 comp rx=0 tx=0).stop 2024-09-15T21:46:21.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.403+0000 7ff92ed9f640 1 -- 172.21.15.100:0/1802261535 shutdown_connections 2024-09-15T21:46:21.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.403+0000 7ff92ed9f640 1 --2- 172.21.15.100:0/1802261535 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff90c06c2b0 0x7ff90c06e770 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:21.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.403+0000 7ff92ed9f640 1 --2- 172.21.15.100:0/1802261535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9281552e0 0x7ff928155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:21.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.403+0000 7ff92ed9f640 1 --2- 172.21.15.100:0/1802261535 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff92815bbb0 0x7ff928157450 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:21.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.403+0000 7ff92ed9f640 1 --2- 172.21.15.100:0/1802261535 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff92806bbe0 0x7ff928142310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:21.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.403+0000 7ff92ed9f640 1 -- 172.21.15.100:0/1802261535 >> 172.21.15.100:0/1802261535 conn(0x7ff9280ca120 msgr2=0x7ff9280cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:21.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.403+0000 7ff92ed9f640 1 -- 172.21.15.100:0/1802261535 shutdown_connections 2024-09-15T21:46:21.406 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.403+0000 7ff92ed9f640 1 -- 172.21.15.100:0/1802261535 wait complete. 2024-09-15T21:46:21.422 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-15T21:46:21.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.532+0000 7fcb63847640 1 Processor -- start 2024-09-15T21:46:21.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.532+0000 7fcb63847640 1 -- start start 2024-09-15T21:46:21.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.533+0000 7fcb63847640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fcb5c15f190 0x7fcb5c071070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:21.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.533+0000 7fcb63847640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb5c15b250 con 0x7fcb5c0c4950 2024-09-15T21:46:21.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.533+0000 7fcb63847640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb5c15b530 con 0x7fcb5c1551f0 2024-09-15T21:46:21.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.533+0000 7fcb63847640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fcb5c1555d0 con 0x7fcb5c15f190 2024-09-15T21:46:21.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.533+0000 7fcb60de2640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fcb5c0c4950 0x7fcb5c15ea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46380/0 (socket says 172.21.15.100:46380) 2024-09-15T21:46:21.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.533+0000 7fcb60de2640 1 -- 172.21.15.100:0/2839374927 learned_addr learned my addr 172.21.15.100:0/2839374927 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:21.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.533+0000 7fcb5bfff640 1 -- 172.21.15.100:0/2839374927 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2438169715 0 0) 0x7fcb5c15b250 con 0x7fcb5c0c4950 2024-09-15T21:46:21.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.533+0000 7fcb5bfff640 1 -- 172.21.15.100:0/2839374927 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb44003520 con 0x7fcb5c0c4950 2024-09-15T21:46:21.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.533+0000 7fcb61de4640 1 --2- 172.21.15.100:0/2839374927 >> v2:172.21.15.122:3300/0 conn(0x7fcb5c15f190 0x7fcb5c071070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:21.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb5bfff640 1 -- 172.21.15.100:0/2839374927 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3611188109 0 0) 0x7fcb44003520 con 0x7fcb5c0c4950 2024-09-15T21:46:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb5bfff640 1 -- 172.21.15.100:0/2839374927 >> v2:172.21.15.122:3300/0 conn(0x7fcb5c15f190 msgr2=0x7fcb5c071070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb5bfff640 1 --2- 172.21.15.100:0/2839374927 >> v2:172.21.15.122:3300/0 conn(0x7fcb5c15f190 0x7fcb5c071070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb5bfff640 1 -- 172.21.15.100:0/2839374927 >> v1:172.21.15.122:6789/0 conn(0x7fcb5c1551f0 legacy=0x7fcb5c0cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb5bfff640 1 -- 172.21.15.100:0/2839374927 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5c15b810 con 0x7fcb5c0c4950 2024-09-15T21:46:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb5bfff640 1 -- 172.21.15.100:0/2839374927 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7fcb500016c0 con 0x7fcb5c0c4950 2024-09-15T21:46:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb5bfff640 1 -- 172.21.15.100:0/2839374927 >> v1:172.21.15.100:6789/0 conn(0x7fcb5c0c4950 legacy=0x7fcb5c15ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb5bfff640 1 --2- 172.21.15.100:0/2839374927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb440039b0 0x7fcb44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb5bfff640 1 --2- 172.21.15.100:0/2839374927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb44004800 0x7fcb44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb5bfff640 1 --2- 172.21.15.100:0/2839374927 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcb440051c0 0x7fcb4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb5bfff640 1 -- 172.21.15.100:0/2839374927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c1555d0 con 0x7fcb440039b0 2024-09-15T21:46:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb5bfff640 1 -- 172.21.15.100:0/2839374927 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4400c070 con 0x7fcb44004800 2024-09-15T21:46:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb5bfff640 1 -- 172.21.15.100:0/2839374927 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fcb4400c270 con 0x7fcb440051c0 2024-09-15T21:46:21.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb615e3640 1 --2- 172.21.15.100:0/2839374927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb440039b0 0x7fcb44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:21.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb60de2640 1 --2- 172.21.15.100:0/2839374927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb44004800 0x7fcb44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:21.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb61de4640 1 --2- 172.21.15.100:0/2839374927 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcb440051c0 0x7fcb4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:21.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb615e3640 1 -- 172.21.15.100:0/2839374927 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcb440051c0 msgr2=0x7fcb4400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:21.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb615e3640 1 --2- 172.21.15.100:0/2839374927 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcb440051c0 0x7fcb4400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:21.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb615e3640 1 -- 172.21.15.100:0/2839374927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb44004800 msgr2=0x7fcb44004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:21.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb615e3640 1 --2- 172.21.15.100:0/2839374927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb44004800 0x7fcb44004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:21.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb615e3640 1 -- 172.21.15.100:0/2839374927 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb44003520 con 0x7fcb440039b0 2024-09-15T21:46:21.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb61de4640 1 --2- 172.21.15.100:0/2839374927 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcb440051c0 0x7fcb4400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:21.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.534+0000 7fcb61de4640 1 -- 172.21.15.100:0/2839374927 reap_dead start 2024-09-15T21:46:21.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.535+0000 7fcb615e3640 1 --2- 172.21.15.100:0/2839374927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb440039b0 0x7fcb44003dd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c001e50 tx=0x7fcb44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733bc44de4bc66fe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:21.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.535+0000 7fcb5bfff640 1 -- 172.21.15.100:0/2839374927 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c001a70 con 0x7fcb440039b0 2024-09-15T21:46:21.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.535+0000 7fcb5bfff640 1 -- 172.21.15.100:0/2839374927 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb4c02ae40 con 0x7fcb440039b0 2024-09-15T21:46:21.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.535+0000 7fcb5bfff640 1 -- 172.21.15.100:0/2839374927 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c003bb0 con 0x7fcb440039b0 2024-09-15T21:46:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.535+0000 7fcb63847640 1 -- 172.21.15.100:0/2839374927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb440039b0 msgr2=0x7fcb44003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.535+0000 7fcb63847640 1 --2- 172.21.15.100:0/2839374927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb440039b0 0x7fcb44003dd0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c001e50 tx=0x7fcb44005830 comp rx=0 tx=0).stop 2024-09-15T21:46:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.535+0000 7fcb63847640 1 -- 172.21.15.100:0/2839374927 shutdown_connections 2024-09-15T21:46:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.536+0000 7fcb63847640 1 --2- 172.21.15.100:0/2839374927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb440039b0 0x7fcb44003dd0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.536+0000 7fcb63847640 1 -- 172.21.15.100:0/2839374927 >> 172.21.15.100:0/2839374927 conn(0x7fcb5c067980 msgr2=0x7fcb5c067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.536+0000 7fcb63847640 1 -- 172.21.15.100:0/2839374927 shutdown_connections 2024-09-15T21:46:21.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.536+0000 7fcb63847640 1 -- 172.21.15.100:0/2839374927 wait complete. 2024-09-15T21:46:21.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.537+0000 7fcb63847640 1 Processor -- start 2024-09-15T21:46:21.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.537+0000 7fcb63847640 1 -- start start 2024-09-15T21:46:21.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.537+0000 7fcb63847640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb440039b0 0x7fcb5c150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:21.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.537+0000 7fcb63847640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb5c15c270 0x7fcb5c0c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:21.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.537+0000 7fcb63847640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcb5c15c6d0 0x7fcb5c1f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:21.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.537+0000 7fcb63847640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c15ea80 con 0x7fcb5c15c270 2024-09-15T21:46:21.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.537+0000 7fcb63847640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcb5c15bdd0 con 0x7fcb440039b0 2024-09-15T21:46:21.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.537+0000 7fcb63847640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fcb5c15ec00 con 0x7fcb5c15c6d0 2024-09-15T21:46:21.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.538+0000 7fcb60de2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb5c15c270 0x7fcb5c0c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:21.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.538+0000 7fcb60de2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb5c15c270 0x7fcb5c0c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47640/0 (socket says 172.21.15.100:47640) 2024-09-15T21:46:21.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.538+0000 7fcb60de2640 1 -- 172.21.15.100:0/3276872493 learned_addr learned my addr 172.21.15.100:0/3276872493 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:21.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.538+0000 7fcb615e3640 1 --2- 172.21.15.100:0/3276872493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb440039b0 0x7fcb5c150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:21.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.538+0000 7fcb61de4640 1 --2- 172.21.15.100:0/3276872493 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcb5c15c6d0 0x7fcb5c1f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:21.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.538+0000 7fcb60de2640 1 -- 172.21.15.100:0/3276872493 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcb5c15c6d0 msgr2=0x7fcb5c1f3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:21.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.538+0000 7fcb60de2640 1 --2- 172.21.15.100:0/3276872493 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcb5c15c6d0 0x7fcb5c1f3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:21.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.538+0000 7fcb60de2640 1 -- 172.21.15.100:0/3276872493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb440039b0 msgr2=0x7fcb5c150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:21.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.538+0000 7fcb60de2640 1 --2- 172.21.15.100:0/3276872493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb440039b0 0x7fcb5c150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:21.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.538+0000 7fcb60de2640 1 -- 172.21.15.100:0/3276872493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb5c1e4950 con 0x7fcb5c15c270 2024-09-15T21:46:21.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.538+0000 7fcb615e3640 1 --2- 172.21.15.100:0/3276872493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb440039b0 0x7fcb5c150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:21.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.538+0000 7fcb60de2640 1 --2- 172.21.15.100:0/3276872493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb5c15c270 0x7fcb5c0c84e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcb500072c0 tx=0x7fcb50007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:21.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.538+0000 7fcb59ffb640 1 -- 172.21.15.100:0/3276872493 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5000f940 con 0x7fcb5c15c270 2024-09-15T21:46:21.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.538+0000 7fcb59ffb640 1 -- 172.21.15.100:0/3276872493 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb50018070 con 0x7fcb5c15c270 2024-09-15T21:46:21.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.538+0000 7fcb63847640 1 -- 172.21.15.100:0/3276872493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb5c15d7f0 con 0x7fcb5c15c270 2024-09-15T21:46:21.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.538+0000 7fcb59ffb640 1 -- 172.21.15.100:0/3276872493 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb5000fd40 con 0x7fcb5c15c270 2024-09-15T21:46:21.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.538+0000 7fcb63847640 1 -- 172.21.15.100:0/3276872493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb5c0c7df0 con 0x7fcb5c15c270 2024-09-15T21:46:21.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.539+0000 7fcb63847640 1 -- 172.21.15.100:0/3276872493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb5c1525c0 con 0x7fcb5c15c270 2024-09-15T21:46:21.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.540+0000 7fcb59ffb640 1 -- 172.21.15.100:0/3276872493 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcb5001d460 con 0x7fcb5c15c270 2024-09-15T21:46:21.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.540+0000 7fcb59ffb640 1 --2- 172.21.15.100:0/3276872493 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fcb4806c2e0 0x7fcb4806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:21.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.540+0000 7fcb615e3640 1 --2- 172.21.15.100:0/3276872493 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fcb4806c2e0 0x7fcb4806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:21.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.540+0000 7fcb59ffb640 1 -- 172.21.15.100:0/3276872493 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5227+0+0 (secure 0 0 0) 0x7fcb5008b220 con 0x7fcb5c15c270 2024-09-15T21:46:21.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.541+0000 7fcb615e3640 1 --2- 172.21.15.100:0/3276872493 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fcb4806c2e0 0x7fcb4806e7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c032000 tx=0x7fcb4c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:21.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.545+0000 7fcb59ffb640 1 -- 172.21.15.100:0/3276872493 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcb5c1e4950 con 0x7fcb5c15c270 2024-09-15T21:46:21.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:21.711+0000 7fcb63847640 1 -- 172.21.15.100:0/3276872493 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcb5c0ce080 con 0x7fcb5c15c270 2024-09-15T21:46:22.409 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.406+0000 7fcb59ffb640 1 -- 172.21.15.100:0/3276872493 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) ==== 170+0+0 (secure 0 0 0) 0x7fcb50059c00 con 0x7fcb5c15c270 2024-09-15T21:46:22.409 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' created 2024-09-15T21:46:22.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.410+0000 7fcb63847640 1 -- 172.21.15.100:0/3276872493 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fcb4806c2e0 msgr2=0x7fcb4806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:22.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.410+0000 7fcb63847640 1 --2- 172.21.15.100:0/3276872493 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fcb4806c2e0 0x7fcb4806e7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c032000 tx=0x7fcb4c02e000 comp rx=0 tx=0).stop 2024-09-15T21:46:22.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.410+0000 7fcb63847640 1 -- 172.21.15.100:0/3276872493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb5c15c270 msgr2=0x7fcb5c0c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:22.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.410+0000 7fcb63847640 1 --2- 172.21.15.100:0/3276872493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb5c15c270 0x7fcb5c0c84e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcb500072c0 tx=0x7fcb50007780 comp rx=0 tx=0).stop 2024-09-15T21:46:22.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.410+0000 7fcb63847640 1 -- 172.21.15.100:0/3276872493 shutdown_connections 2024-09-15T21:46:22.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.410+0000 7fcb63847640 1 --2- 172.21.15.100:0/3276872493 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fcb4806c2e0 0x7fcb4806e7a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:22.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.410+0000 7fcb63847640 1 --2- 172.21.15.100:0/3276872493 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcb5c15c6d0 0x7fcb5c1f3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:22.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.410+0000 7fcb63847640 1 --2- 172.21.15.100:0/3276872493 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcb5c15c270 0x7fcb5c0c84e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:22.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.410+0000 7fcb63847640 1 --2- 172.21.15.100:0/3276872493 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb440039b0 0x7fcb5c150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:22.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.410+0000 7fcb63847640 1 -- 172.21.15.100:0/3276872493 >> 172.21.15.100:0/3276872493 conn(0x7fcb5c067980 msgr2=0x7fcb5c0c9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:22.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.410+0000 7fcb63847640 1 -- 172.21.15.100:0/3276872493 shutdown_connections 2024-09-15T21:46:22.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.410+0000 7fcb63847640 1 -- 172.21.15.100:0/3276872493 wait complete. 2024-09-15T21:46:22.438 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-15T21:46:22.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.541+0000 7fafd3fe4640 1 Processor -- start 2024-09-15T21:46:22.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.541+0000 7fafd3fe4640 1 -- start start 2024-09-15T21:46:22.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.542+0000 7fafd3fe4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fafcc155550 0x7fafcc0cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:22.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.542+0000 7fafd3fe4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fafcc0cdfb0 0x7fafcc0cc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:22.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.542+0000 7fafd3fe4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafcc0c61b0 con 0x7fafcc0cceb0 2024-09-15T21:46:22.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.542+0000 7fafd3fe4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fafcc15bec0 con 0x7fafcc0cdfb0 2024-09-15T21:46:22.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.542+0000 7fafd3fe4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fafcc0ce4b0 con 0x7fafcc155550 2024-09-15T21:46:22.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.542+0000 7fafd2581640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fafcc0cceb0 0x7fafcc15e7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46392/0 (socket says 172.21.15.100:46392) 2024-09-15T21:46:22.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.542+0000 7fafd2581640 1 -- 172.21.15.100:0/3400685401 learned_addr learned my addr 172.21.15.100:0/3400685401 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:22.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.542+0000 7fafd157f640 1 --2- 172.21.15.100:0/3400685401 >> v2:172.21.15.122:3300/0 conn(0x7fafcc0cdfb0 0x7fafcc0cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:22.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.542+0000 7fafd1d80640 1 --2- 172.21.15.100:0/3400685401 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fafcc155550 0x7fafcc0cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:22.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.542+0000 7fafd0d7e640 1 -- 172.21.15.100:0/3400685401 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3779227551 0 0) 0x7fafc4001490 con 0x7fafcc0cceb0 2024-09-15T21:46:22.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.542+0000 7fafd0d7e640 1 -- 172.21.15.100:0/3400685401 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafb8003520 con 0x7fafcc0cceb0 2024-09-15T21:46:22.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd0d7e640 1 -- 172.21.15.100:0/3400685401 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4169186155 0 0) 0x7fafb8003520 con 0x7fafcc0cceb0 2024-09-15T21:46:22.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd0d7e640 1 -- 172.21.15.100:0/3400685401 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fafcc155550 msgr2=0x7fafcc0cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:22.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd0d7e640 1 --2- 172.21.15.100:0/3400685401 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fafcc155550 0x7fafcc0cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:22.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd0d7e640 1 -- 172.21.15.100:0/3400685401 >> v2:172.21.15.122:3300/0 conn(0x7fafcc0cdfb0 msgr2=0x7fafcc0cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:22.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd0d7e640 1 --2- 172.21.15.100:0/3400685401 >> v2:172.21.15.122:3300/0 conn(0x7fafcc0cdfb0 0x7fafcc0cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:22.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd0d7e640 1 -- 172.21.15.100:0/3400685401 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafcc15b590 con 0x7fafcc0cceb0 2024-09-15T21:46:22.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd0d7e640 1 -- 172.21.15.100:0/3400685401 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7fafc4001bc0 con 0x7fafcc0cceb0 2024-09-15T21:46:22.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd0d7e640 1 -- 172.21.15.100:0/3400685401 >> v1:172.21.15.100:6789/0 conn(0x7fafcc0cceb0 legacy=0x7fafcc15e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:22.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd0d7e640 1 --2- 172.21.15.100:0/3400685401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafb80039b0 0x7fafb8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:22.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd0d7e640 1 --2- 172.21.15.100:0/3400685401 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fafcc155550 0x7fafb8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:22.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd0d7e640 1 --2- 172.21.15.100:0/3400685401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb8004d40 0x7fafb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:22.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd0d7e640 1 -- 172.21.15.100:0/3400685401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc15bec0 con 0x7fafb80039b0 2024-09-15T21:46:22.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd0d7e640 1 -- 172.21.15.100:0/3400685401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc0ce4b0 con 0x7fafb8004d40 2024-09-15T21:46:22.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd0d7e640 1 -- 172.21.15.100:0/3400685401 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fafb800b9c0 con 0x7fafcc155550 2024-09-15T21:46:22.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd1d80640 1 --2- 172.21.15.100:0/3400685401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafb80039b0 0x7fafb8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:22.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd2581640 1 --2- 172.21.15.100:0/3400685401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb8004d40 0x7fafb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:22.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd157f640 1 --2- 172.21.15.100:0/3400685401 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fafcc155550 0x7fafb8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:22.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd1d80640 1 -- 172.21.15.100:0/3400685401 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fafcc155550 msgr2=0x7fafb8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:22.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd1d80640 1 --2- 172.21.15.100:0/3400685401 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fafcc155550 0x7fafb8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:22.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd1d80640 1 -- 172.21.15.100:0/3400685401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb8004d40 msgr2=0x7fafb80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:22.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd1d80640 1 --2- 172.21.15.100:0/3400685401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb8004d40 0x7fafb80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:22.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd1d80640 1 -- 172.21.15.100:0/3400685401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb8003520 con 0x7fafb80039b0 2024-09-15T21:46:22.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd157f640 1 --2- 172.21.15.100:0/3400685401 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fafcc155550 0x7fafb8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:22.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.543+0000 7fafd2581640 1 --2- 172.21.15.100:0/3400685401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafb8004d40 0x7fafb80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.544+0000 7fafd1d80640 1 --2- 172.21.15.100:0/3400685401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafb80039b0 0x7fafb8003dd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009ac0 tx=0x7fafb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad61d4b32e432a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.544+0000 7fafd0d7e640 1 -- 172.21.15.100:0/3400685401 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8036070 con 0x7fafb80039b0 2024-09-15T21:46:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.544+0000 7fafd0d7e640 1 -- 172.21.15.100:0/3400685401 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc8031400 con 0x7fafb80039b0 2024-09-15T21:46:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.544+0000 7fafd0d7e640 1 -- 172.21.15.100:0/3400685401 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8009690 con 0x7fafb80039b0 2024-09-15T21:46:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.544+0000 7fafd3fe4640 1 -- 172.21.15.100:0/3400685401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafb80039b0 msgr2=0x7fafb8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.544+0000 7fafd3fe4640 1 --2- 172.21.15.100:0/3400685401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafb80039b0 0x7fafb8003dd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009ac0 tx=0x7fafb8005830 comp rx=0 tx=0).stop 2024-09-15T21:46:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.544+0000 7fafd2581640 1 -- 172.21.15.100:0/3400685401 reap_dead start 2024-09-15T21:46:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.544+0000 7fafd3fe4640 1 -- 172.21.15.100:0/3400685401 shutdown_connections 2024-09-15T21:46:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.544+0000 7fafd3fe4640 1 -- 172.21.15.100:0/3400685401 >> 172.21.15.100:0/3400685401 conn(0x7fafcc067b60 msgr2=0x7fafcc067f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.545+0000 7fafd3fe4640 1 -- 172.21.15.100:0/3400685401 shutdown_connections 2024-09-15T21:46:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.545+0000 7fafd3fe4640 1 -- 172.21.15.100:0/3400685401 wait complete. 2024-09-15T21:46:22.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.546+0000 7fafd3fe4640 1 Processor -- start 2024-09-15T21:46:22.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.546+0000 7fafd3fe4640 1 -- start start 2024-09-15T21:46:22.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.546+0000 7fafd3fe4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafb80039b0 0x7fafcc0c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:22.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.546+0000 7fafd3fe4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fafcc1e4480 0x7fafcc1f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:22.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.546+0000 7fafd3fe4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafcc1e4900 0x7fafcc14c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:22.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.546+0000 7fafd3fe4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc15b280 con 0x7fafb80039b0 2024-09-15T21:46:22.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.546+0000 7fafd3fe4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc15bb50 con 0x7fafcc1e4900 2024-09-15T21:46:22.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.546+0000 7fafd3fe4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fafcc155930 con 0x7fafcc1e4480 2024-09-15T21:46:22.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.546+0000 7fafd1d80640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafb80039b0 0x7fafcc0c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:22.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.546+0000 7fafd1d80640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafb80039b0 0x7fafcc0c6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47662/0 (socket says 172.21.15.100:47662) 2024-09-15T21:46:22.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.546+0000 7fafd1d80640 1 -- 172.21.15.100:0/1755167419 learned_addr learned my addr 172.21.15.100:0/1755167419 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:22.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.547+0000 7fafd1d80640 1 -- 172.21.15.100:0/1755167419 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fafcc1e4480 msgr2=0x7fafcc1f42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:22.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.547+0000 7fafd1d80640 1 --2- 172.21.15.100:0/1755167419 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fafcc1e4480 0x7fafcc1f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:22.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.547+0000 7fafd1d80640 1 -- 172.21.15.100:0/1755167419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafcc1e4900 msgr2=0x7fafcc14c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:22.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.547+0000 7fafd2581640 1 --2- 172.21.15.100:0/1755167419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafcc1e4900 0x7fafcc14c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:22.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.547+0000 7fafd1d80640 1 --2- 172.21.15.100:0/1755167419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafcc1e4900 0x7fafcc14c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:22.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.547+0000 7fafd1d80640 1 -- 172.21.15.100:0/1755167419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafcc1e5030 con 0x7fafb80039b0 2024-09-15T21:46:22.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.547+0000 7fafd2581640 1 --2- 172.21.15.100:0/1755167419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafcc1e4900 0x7fafcc14c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:22.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.547+0000 7fafd1d80640 1 --2- 172.21.15.100:0/1755167419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafb80039b0 0x7fafcc0c6ba0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009ac0 tx=0x7fafc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:22.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.547+0000 7fafc27fc640 1 -- 172.21.15.100:0/1755167419 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8040020 con 0x7fafb80039b0 2024-09-15T21:46:22.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.547+0000 7fafc27fc640 1 -- 172.21.15.100:0/1755167419 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc8003bc0 con 0x7fafb80039b0 2024-09-15T21:46:22.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.547+0000 7fafd3fe4640 1 -- 172.21.15.100:0/1755167419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafcc15a0b0 con 0x7fafb80039b0 2024-09-15T21:46:22.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.547+0000 7fafc27fc640 1 -- 172.21.15.100:0/1755167419 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8036040 con 0x7fafb80039b0 2024-09-15T21:46:22.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.547+0000 7fafd3fe4640 1 -- 172.21.15.100:0/1755167419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafcc0ceba0 con 0x7fafb80039b0 2024-09-15T21:46:22.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.548+0000 7fafd3fe4640 1 -- 172.21.15.100:0/1755167419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafcc0cda70 con 0x7fafb80039b0 2024-09-15T21:46:22.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.549+0000 7fafc27fc640 1 -- 172.21.15.100:0/1755167419 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fafc8043020 con 0x7fafb80039b0 2024-09-15T21:46:22.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.549+0000 7fafc27fc640 1 --2- 172.21.15.100:0/1755167419 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fafb006c2e0 0x7fafb006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:22.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.549+0000 7fafd157f640 1 --2- 172.21.15.100:0/1755167419 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fafb006c2e0 0x7fafb006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:22.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.549+0000 7fafc27fc640 1 -- 172.21.15.100:0/1755167419 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5609+0+0 (secure 0 0 0) 0x7fafc80ab7b0 con 0x7fafb80039b0 2024-09-15T21:46:22.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.550+0000 7fafd157f640 1 --2- 172.21.15.100:0/1755167419 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fafb006c2e0 0x7fafb006e7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0cb280 tx=0x7fafbc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:22.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.554+0000 7fafc27fc640 1 -- 172.21.15.100:0/1755167419 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fafc804a030 con 0x7fafb80039b0 2024-09-15T21:46:22.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:22.752+0000 7fafd3fe4640 1 -- 172.21.15.100:0/1755167419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fafcc0cc150 con 0x7fafb80039b0 2024-09-15T21:46:23.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.408+0000 7fafc27fc640 1 -- 172.21.15.100:0/1755167419 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7fafc80044e0 con 0x7fafb80039b0 2024-09-15T21:46:23.410 INFO:teuthology.orchestra.run.smithi100.stderr:new fs with metadata pool 2 and data pool 3 2024-09-15T21:46:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.411+0000 7fafd3fe4640 1 -- 172.21.15.100:0/1755167419 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fafb006c2e0 msgr2=0x7fafb006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.411+0000 7fafd3fe4640 1 --2- 172.21.15.100:0/1755167419 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fafb006c2e0 0x7fafb006e7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0cb280 tx=0x7fafbc008040 comp rx=0 tx=0).stop 2024-09-15T21:46:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.411+0000 7fafd3fe4640 1 -- 172.21.15.100:0/1755167419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafb80039b0 msgr2=0x7fafcc0c6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.411+0000 7fafd3fe4640 1 --2- 172.21.15.100:0/1755167419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafb80039b0 0x7fafcc0c6ba0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009ac0 tx=0x7fafc8004730 comp rx=0 tx=0).stop 2024-09-15T21:46:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.411+0000 7fafd3fe4640 1 -- 172.21.15.100:0/1755167419 shutdown_connections 2024-09-15T21:46:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.411+0000 7fafd3fe4640 1 --2- 172.21.15.100:0/1755167419 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fafb006c2e0 0x7fafb006e7a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:23.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.411+0000 7fafd3fe4640 1 --2- 172.21.15.100:0/1755167419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fafcc1e4900 0x7fafcc14c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:23.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.411+0000 7fafd3fe4640 1 --2- 172.21.15.100:0/1755167419 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fafcc1e4480 0x7fafcc1f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:23.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.411+0000 7fafd3fe4640 1 --2- 172.21.15.100:0/1755167419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fafb80039b0 0x7fafcc0c6ba0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:23.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.411+0000 7fafd3fe4640 1 -- 172.21.15.100:0/1755167419 >> 172.21.15.100:0/1755167419 conn(0x7fafcc067b60 msgr2=0x7fafcc1e3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:23.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.411+0000 7fafd3fe4640 1 -- 172.21.15.100:0/1755167419 shutdown_connections 2024-09-15T21:46:23.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.411+0000 7fafd3fe4640 1 -- 172.21.15.100:0/1755167419 wait complete. 2024-09-15T21:46:23.434 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:46:23.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.531+0000 7f3d2f311640 1 Processor -- start 2024-09-15T21:46:23.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.531+0000 7f3d2f311640 1 -- start start 2024-09-15T21:46:23.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.532+0000 7f3d2f311640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3d28155440 0x7f3d280cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:23.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.532+0000 7f3d2f311640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3d280cc8f0 0x7f3d280cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:23.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.532+0000 7f3d2f311640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d2815e7a0 con 0x7f3d280cb7d0 2024-09-15T21:46:23.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.532+0000 7f3d2f311640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f3d2815b470 con 0x7f3d280cc8f0 2024-09-15T21:46:23.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.532+0000 7f3d2f311640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3d28155820 con 0x7f3d28155440 2024-09-15T21:46:23.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.532+0000 7f3d2d8ae640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3d280cc8f0 0x7f3d280cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:23.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.532+0000 7f3d2d8ae640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3d280cc8f0 0x7f3d280cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47702/0 (socket says 172.21.15.100:47702) 2024-09-15T21:46:23.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.532+0000 7f3d2d8ae640 1 -- 172.21.15.100:0/89136886 learned_addr learned my addr 172.21.15.100:0/89136886 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:23.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.532+0000 7f3d2d0ad640 1 --2- 172.21.15.100:0/89136886 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3d28155440 0x7f3d280cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:23.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.532+0000 7f3d2d8ae640 1 -- 172.21.15.100:0/89136886 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3d28155440 msgr2=0x7f3d280cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:23.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.532+0000 7f3d2d8ae640 1 --2- 172.21.15.100:0/89136886 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3d28155440 0x7f3d280cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:23.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.532+0000 7f3d2d8ae640 1 -- 172.21.15.100:0/89136886 >> v1:172.21.15.122:6789/0 conn(0x7f3d280cb7d0 legacy=0x7f3d280cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:23.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.532+0000 7f3d2d8ae640 1 -- 172.21.15.100:0/89136886 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d2815ea40 con 0x7f3d280cc8f0 2024-09-15T21:46:23.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.533+0000 7f3d2d8ae640 1 --2- 172.21.15.100:0/89136886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d280cc8f0 0x7f3d280cd960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009870 tx=0x7f3d2402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3248443f03d0ae16 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:23.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.533+0000 7f3d1ffff640 1 -- 172.21.15.100:0/89136886 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24037070 con 0x7f3d280cc8f0 2024-09-15T21:46:23.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.533+0000 7f3d1ffff640 1 -- 172.21.15.100:0/89136886 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d24032440 con 0x7f3d280cc8f0 2024-09-15T21:46:23.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.533+0000 7f3d1ffff640 1 -- 172.21.15.100:0/89136886 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d24032810 con 0x7f3d280cc8f0 2024-09-15T21:46:23.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.533+0000 7f3d2f311640 1 -- 172.21.15.100:0/89136886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d280cc8f0 msgr2=0x7f3d280cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:23.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.533+0000 7f3d2f311640 1 --2- 172.21.15.100:0/89136886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d280cc8f0 0x7f3d280cd960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009870 tx=0x7f3d2402a270 comp rx=0 tx=0).stop 2024-09-15T21:46:23.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.533+0000 7f3d2f311640 1 -- 172.21.15.100:0/89136886 shutdown_connections 2024-09-15T21:46:23.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.533+0000 7f3d2f311640 1 --2- 172.21.15.100:0/89136886 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d280cc8f0 0x7f3d280cd960 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:23.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.533+0000 7f3d2f311640 1 --2- 172.21.15.100:0/89136886 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3d28155440 0x7f3d280cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:23.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.533+0000 7f3d2f311640 1 -- 172.21.15.100:0/89136886 >> 172.21.15.100:0/89136886 conn(0x7f3d280680d0 msgr2=0x7f3d2806d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:23.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.534+0000 7f3d2f311640 1 -- 172.21.15.100:0/89136886 shutdown_connections 2024-09-15T21:46:23.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.534+0000 7f3d2f311640 1 -- 172.21.15.100:0/89136886 wait complete. 2024-09-15T21:46:23.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.535+0000 7f3d2f311640 1 Processor -- start 2024-09-15T21:46:23.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.535+0000 7f3d2f311640 1 -- start start 2024-09-15T21:46:23.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.535+0000 7f3d2f311640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d280cb7d0 0x7f3d28149500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:23.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.535+0000 7f3d2f311640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d280cc8f0 0x7f3d280cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:23.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.535+0000 7f3d2f311640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3d28155440 0x7f3d281442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:23.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.535+0000 7f3d2f311640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2815f2e0 con 0x7f3d280cc8f0 2024-09-15T21:46:23.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.535+0000 7f3d2f311640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2815f000 con 0x7f3d280cb7d0 2024-09-15T21:46:23.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.535+0000 7f3d2f311640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3d2815b070 con 0x7f3d28155440 2024-09-15T21:46:23.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.535+0000 7f3d2c8ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d280cc8f0 0x7f3d280cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:23.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.535+0000 7f3d2c8ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d280cc8f0 0x7f3d280cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47710/0 (socket says 172.21.15.100:47710) 2024-09-15T21:46:23.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.535+0000 7f3d2c8ac640 1 -- 172.21.15.100:0/3671953576 learned_addr learned my addr 172.21.15.100:0/3671953576 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:23.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.536+0000 7f3d2d8ae640 1 --2- 172.21.15.100:0/3671953576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3d28155440 0x7f3d281442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:23.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.536+0000 7f3d2d0ad640 1 --2- 172.21.15.100:0/3671953576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d280cb7d0 0x7f3d28149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:23.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.536+0000 7f3d2c8ac640 1 -- 172.21.15.100:0/3671953576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3d28155440 msgr2=0x7f3d281442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:23.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.536+0000 7f3d2c8ac640 1 --2- 172.21.15.100:0/3671953576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3d28155440 0x7f3d281442e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:23.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.536+0000 7f3d2c8ac640 1 -- 172.21.15.100:0/3671953576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d280cb7d0 msgr2=0x7f3d28149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:23.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.536+0000 7f3d2c8ac640 1 --2- 172.21.15.100:0/3671953576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d280cb7d0 0x7f3d28149500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:23.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.536+0000 7f3d2c8ac640 1 -- 172.21.15.100:0/3671953576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d280c83a0 con 0x7f3d280cc8f0 2024-09-15T21:46:23.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.536+0000 7f3d2c8ac640 1 --2- 172.21.15.100:0/3671953576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d280cc8f0 0x7f3d280cd670 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800c470 tx=0x7f3d18001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:23.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.536+0000 7f3d1dffb640 1 -- 172.21.15.100:0/3671953576 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d180052d0 con 0x7f3d280cc8f0 2024-09-15T21:46:23.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.536+0000 7f3d1dffb640 1 -- 172.21.15.100:0/3671953576 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d18005470 con 0x7f3d280cc8f0 2024-09-15T21:46:23.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.536+0000 7f3d2f311640 1 -- 172.21.15.100:0/3671953576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d280c7de0 con 0x7f3d280cc8f0 2024-09-15T21:46:23.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.536+0000 7f3d1dffb640 1 -- 172.21.15.100:0/3671953576 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18007260 con 0x7f3d280cc8f0 2024-09-15T21:46:23.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.536+0000 7f3d2f311640 1 -- 172.21.15.100:0/3671953576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d281f4bd0 con 0x7f3d280cc8f0 2024-09-15T21:46:23.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.537+0000 7f3d2f311640 1 -- 172.21.15.100:0/3671953576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d280b3940 con 0x7f3d280cc8f0 2024-09-15T21:46:23.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.538+0000 7f3d1dffb640 1 -- 172.21.15.100:0/3671953576 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3d18007400 con 0x7f3d280cc8f0 2024-09-15T21:46:23.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.538+0000 7f3d1dffb640 1 --2- 172.21.15.100:0/3671953576 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3d0c06c2b0 0x7f3d0c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:23.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.538+0000 7f3d1dffb640 1 -- 172.21.15.100:0/3671953576 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3d1808a7c0 con 0x7f3d280cc8f0 2024-09-15T21:46:23.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.538+0000 7f3d2d0ad640 1 --2- 172.21.15.100:0/3671953576 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3d0c06c2b0 0x7f3d0c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:23.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.539+0000 7f3d2d0ad640 1 --2- 172.21.15.100:0/3671953576 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3d0c06c2b0 0x7f3d0c06e770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d20011340 tx=0x7f3d2000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:23.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.543+0000 7f3d1dffb640 1 -- 172.21.15.100:0/3671953576 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3d18058970 con 0x7f3d280cc8f0 2024-09-15T21:46:23.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.710+0000 7f3d2f311640 1 -- 172.21.15.100:0/3671953576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d2814f4c0 con 0x7f3d280cc8f0 2024-09-15T21:46:23.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.711+0000 7f3d1dffb640 1 -- 172.21.15.100:0/3671953576 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16750 (secure 0 0 0) 0x7f3d1805cc70 con 0x7f3d280cc8f0 2024-09-15T21:46:23.713 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:23.713 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":21,"fsid":"30e2fff4-2171-4531-8806-0e03b63f494f","created":"2024-09-15T21:45:59.746897+0000","modified":"2024-09-15T21:46:22.753668+0000","last_up_change":"2024-09-15T21:46:13.224709+0000","last_in_change":"2024-09-15T21:46:01.812129+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:46:11.837603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T21:46:20.577982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T21:46:21.729231+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5887900b-522d-476f-9704-cc2c5ee87a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6819","nonce":3920435546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6821","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6822","nonce":3920435546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6829","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6831","nonce":3920435546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6825","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6827","nonce":3920435546}]},"public_addr":"172.21.15.100:6819/3920435546","cluster_addr":"172.21.15.100:6822/3920435546","heartbeat_back_addr":"172.21.15.100:6831/3920435546","heartbeat_front_addr":"172.21.15.100:6827/3920435546","state":["exists","up"]},{"osd":1,"uuid":"adc17993-5c50-4251-85e1-4c7ac62e25e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6801","nonce":3802679972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6803","nonce":3802679972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6807","nonce":3802679972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6805","nonce":3802679972}]},"public_addr":"172.21.15.100:6801/3802679972","cluster_addr":"172.21.15.100:6803/3802679972","heartbeat_back_addr":"172.21.15.100:6807/3802679972","heartbeat_front_addr":"172.21.15.100:6805/3802679972","state":["exists","up"]},{"osd":2,"uuid":"8de3b7d5-1980-4a3b-a934-264f867c7c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6823","nonce":2316787974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6826","nonce":2316787974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6833","nonce":2316787974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6830","nonce":2316787974}]},"public_addr":"172.21.15.100:6823/2316787974","cluster_addr":"172.21.15.100:6826/2316787974","heartbeat_back_addr":"172.21.15.100:6833/2316787974","heartbeat_front_addr":"172.21.15.100:6830/2316787974","state":["exists","up"]},{"osd":3,"uuid":"93dbcd1c-6a6f-49cf-bc94-e64b883de5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6811","nonce":1287562741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6813","nonce":1287562741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6817","nonce":1287562741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6815","nonce":1287562741}]},"public_addr":"172.21.15.100:6811/1287562741","cluster_addr":"172.21.15.100:6813/1287562741","heartbeat_back_addr":"172.21.15.100:6817/1287562741","heartbeat_front_addr":"172.21.15.100:6815/1287562741","state":["exists","up"]},{"osd":4,"uuid":"7caee2a0-bab4-45a9-8109-724948c863c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6801","nonce":4045445512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6803","nonce":4045445512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6807","nonce":4045445512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6805","nonce":4045445512}]},"public_addr":"172.21.15.122:6801/4045445512","cluster_addr":"172.21.15.122:6803/4045445512","heartbeat_back_addr":"172.21.15.122:6807/4045445512","heartbeat_front_addr":"172.21.15.122:6805/4045445512","state":["exists","up"]},{"osd":5,"uuid":"6e64070a-671a-444b-a061-b3051a29b121","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6825","nonce":2400668532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6827","nonce":2400668532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6831","nonce":2400668532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6829","nonce":2400668532}]},"public_addr":"172.21.15.122:6825/2400668532","cluster_addr":"172.21.15.122:6827/2400668532","heartbeat_back_addr":"172.21.15.122:6831/2400668532","heartbeat_front_addr":"172.21.15.122:6829/2400668532","state":["exists","up"]},{"osd":6,"uuid":"47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6809","nonce":1437167311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6811","nonce":1437167311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6815","nonce":1437167311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6813","nonce":1437167311}]},"public_addr":"172.21.15.122:6809/1437167311","cluster_addr":"172.21.15.122:6811/1437167311","heartbeat_back_addr":"172.21.15.122:6815/1437167311","heartbeat_front_addr":"172.21.15.122:6813/1437167311","state":["exists","up"]},{"osd":7,"uuid":"9a916b6d-af82-486f-af40-4a7291c900fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6817","nonce":2431377162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6819","nonce":2431377162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6823","nonce":2431377162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6821","nonce":2431377162}]},"public_addr":"172.21.15.122:6817/2431377162","cluster_addr":"172.21.15.122:6819/2431377162","heartbeat_back_addr":"172.21.15.122:6823/2431377162","heartbeat_front_addr":"172.21.15.122:6821/2431377162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.168771+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.886614+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.154129+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.570531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.926773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.146320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.409137+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.373359+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:46:23.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.713+0000 7f3d2f311640 1 -- 172.21.15.100:0/3671953576 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3d0c06c2b0 msgr2=0x7f3d0c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:23.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.713+0000 7f3d2f311640 1 --2- 172.21.15.100:0/3671953576 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3d0c06c2b0 0x7f3d0c06e770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3d20011340 tx=0x7f3d2000b040 comp rx=0 tx=0).stop 2024-09-15T21:46:23.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.713+0000 7f3d2f311640 1 -- 172.21.15.100:0/3671953576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d280cc8f0 msgr2=0x7f3d280cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:23.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.713+0000 7f3d2f311640 1 --2- 172.21.15.100:0/3671953576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d280cc8f0 0x7f3d280cd670 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800c470 tx=0x7f3d18001e70 comp rx=0 tx=0).stop 2024-09-15T21:46:23.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.713+0000 7f3d2f311640 1 -- 172.21.15.100:0/3671953576 shutdown_connections 2024-09-15T21:46:23.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.713+0000 7f3d2f311640 1 --2- 172.21.15.100:0/3671953576 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3d0c06c2b0 0x7f3d0c06e770 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:23.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.713+0000 7f3d2f311640 1 --2- 172.21.15.100:0/3671953576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3d28155440 0x7f3d281442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:23.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.713+0000 7f3d2f311640 1 --2- 172.21.15.100:0/3671953576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3d280cc8f0 0x7f3d280cd670 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:23.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.713+0000 7f3d2f311640 1 --2- 172.21.15.100:0/3671953576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3d280cb7d0 0x7f3d28149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:23.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.713+0000 7f3d2f311640 1 -- 172.21.15.100:0/3671953576 >> 172.21.15.100:0/3671953576 conn(0x7f3d280680d0 msgr2=0x7f3d28159880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:23.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.714+0000 7f3d2f311640 1 -- 172.21.15.100:0/3671953576 shutdown_connections 2024-09-15T21:46:23.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.714+0000 7f3d2f311640 1 -- 172.21.15.100:0/3671953576 wait complete. 2024-09-15T21:46:23.726 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:46:23.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.823+0000 7fc8966e4640 1 Processor -- start 2024-09-15T21:46:23.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.823+0000 7fc8966e4640 1 -- start start 2024-09-15T21:46:23.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.823+0000 7fc8966e4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc890155440 0x7fc8900cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:23.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.823+0000 7fc8966e4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc8900cb7d0 0x7fc8900cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:23.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.823+0000 7fc8966e4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc89015e7a0 con 0x7fc89006d0e0 2024-09-15T21:46:23.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.823+0000 7fc8966e4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc89015b470 con 0x7fc890155440 2024-09-15T21:46:23.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.823+0000 7fc8966e4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fc890155820 con 0x7fc8900cb7d0 2024-09-15T21:46:23.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.824+0000 7fc88ffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc890155440 0x7fc8900cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:23.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.824+0000 7fc88ffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc890155440 0x7fc8900cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47720/0 (socket says 172.21.15.100:47720) 2024-09-15T21:46:23.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.824+0000 7fc88ffff640 1 -- 172.21.15.100:0/2998555499 learned_addr learned my addr 172.21.15.100:0/2998555499 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:23.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.824+0000 7fc88f7fe640 1 --2- 172.21.15.100:0/2998555499 >> v2:172.21.15.122:3300/0 conn(0x7fc8900cb7d0 0x7fc8900cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:23.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.824+0000 7fc88ffff640 1 -- 172.21.15.100:0/2998555499 >> v2:172.21.15.122:3300/0 conn(0x7fc8900cb7d0 msgr2=0x7fc8900cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:23.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.824+0000 7fc88ffff640 1 --2- 172.21.15.100:0/2998555499 >> v2:172.21.15.122:3300/0 conn(0x7fc8900cb7d0 0x7fc8900cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:23.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.824+0000 7fc88ffff640 1 -- 172.21.15.100:0/2998555499 >> v1:172.21.15.100:6789/0 conn(0x7fc89006d0e0 legacy=0x7fc8900cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:23.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.824+0000 7fc88ffff640 1 -- 172.21.15.100:0/2998555499 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89015ea40 con 0x7fc890155440 2024-09-15T21:46:23.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.824+0000 7fc88ffff640 1 --2- 172.21.15.100:0/2998555499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc890155440 0x7fc8900cd420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc890158ca0 tx=0x7fc88002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abd20b14267075c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:23.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.825+0000 7fc88effd640 1 -- 172.21.15.100:0/2998555499 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880037070 con 0x7fc890155440 2024-09-15T21:46:23.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.825+0000 7fc88effd640 1 -- 172.21.15.100:0/2998555499 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc880032440 con 0x7fc890155440 2024-09-15T21:46:23.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.825+0000 7fc88effd640 1 -- 172.21.15.100:0/2998555499 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc880032810 con 0x7fc890155440 2024-09-15T21:46:23.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.825+0000 7fc8966e4640 1 -- 172.21.15.100:0/2998555499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc890155440 msgr2=0x7fc8900cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:23.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.825+0000 7fc8966e4640 1 --2- 172.21.15.100:0/2998555499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc890155440 0x7fc8900cd420 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc890158ca0 tx=0x7fc88002a1e0 comp rx=0 tx=0).stop 2024-09-15T21:46:23.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.825+0000 7fc8966e4640 1 -- 172.21.15.100:0/2998555499 shutdown_connections 2024-09-15T21:46:23.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.825+0000 7fc8966e4640 1 --2- 172.21.15.100:0/2998555499 >> v2:172.21.15.122:3300/0 conn(0x7fc8900cb7d0 0x7fc8900cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:23.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.825+0000 7fc8966e4640 1 --2- 172.21.15.100:0/2998555499 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc890155440 0x7fc8900cd420 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:23.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.825+0000 7fc8966e4640 1 -- 172.21.15.100:0/2998555499 >> 172.21.15.100:0/2998555499 conn(0x7fc8900680d0 msgr2=0x7fc89006d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:23.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.825+0000 7fc8966e4640 1 -- 172.21.15.100:0/2998555499 shutdown_connections 2024-09-15T21:46:23.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.825+0000 7fc8966e4640 1 -- 172.21.15.100:0/2998555499 wait complete. 2024-09-15T21:46:23.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.826+0000 7fc8966e4640 1 Processor -- start 2024-09-15T21:46:23.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.827+0000 7fc8966e4640 1 -- start start 2024-09-15T21:46:23.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.827+0000 7fc8966e4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc89006d0e0 0x7fc890151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:23.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.827+0000 7fc8966e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8900cb7d0 0x7fc890150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:23.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.827+0000 7fc8966e4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc890155440 0x7fc89014b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:23.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.827+0000 7fc8966e4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc89015f2e0 con 0x7fc890155440 2024-09-15T21:46:23.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.827+0000 7fc8966e4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc89015f000 con 0x7fc8900cb7d0 2024-09-15T21:46:23.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.827+0000 7fc8966e4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc89015b070 con 0x7fc89006d0e0 2024-09-15T21:46:23.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.827+0000 7fc894c81640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc890155440 0x7fc89014b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:23.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.827+0000 7fc894c81640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc890155440 0x7fc89014b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47730/0 (socket says 172.21.15.100:47730) 2024-09-15T21:46:23.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.827+0000 7fc894c81640 1 -- 172.21.15.100:0/924218968 learned_addr learned my addr 172.21.15.100:0/924218968 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:23.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.828+0000 7fc88ffff640 1 --2- 172.21.15.100:0/924218968 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc89006d0e0 0x7fc890151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:23.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.828+0000 7fc88f7fe640 1 --2- 172.21.15.100:0/924218968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8900cb7d0 0x7fc890150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:23.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.828+0000 7fc894c81640 1 -- 172.21.15.100:0/924218968 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc89006d0e0 msgr2=0x7fc890151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:23.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.828+0000 7fc894c81640 1 --2- 172.21.15.100:0/924218968 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc89006d0e0 0x7fc890151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:23.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.828+0000 7fc894c81640 1 -- 172.21.15.100:0/924218968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8900cb7d0 msgr2=0x7fc890150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:23.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.828+0000 7fc894c81640 1 --2- 172.21.15.100:0/924218968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8900cb7d0 0x7fc890150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:23.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.828+0000 7fc894c81640 1 -- 172.21.15.100:0/924218968 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8901f20a0 con 0x7fc890155440 2024-09-15T21:46:23.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.828+0000 7fc88f7fe640 1 --2- 172.21.15.100:0/924218968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8900cb7d0 0x7fc890150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:23.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.828+0000 7fc894c81640 1 --2- 172.21.15.100:0/924218968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc890155440 0x7fc89014b2e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc88800c6b0 tx=0x7fc88800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:23.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.828+0000 7fc88cff9640 1 -- 172.21.15.100:0/924218968 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888003b70 con 0x7fc890155440 2024-09-15T21:46:23.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.828+0000 7fc88cff9640 1 -- 172.21.15.100:0/924218968 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc888003d30 con 0x7fc890155440 2024-09-15T21:46:23.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.828+0000 7fc88cff9640 1 -- 172.21.15.100:0/924218968 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc888005310 con 0x7fc890155440 2024-09-15T21:46:23.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.828+0000 7fc8966e4640 1 -- 172.21.15.100:0/924218968 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89015c870 con 0x7fc890155440 2024-09-15T21:46:23.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.828+0000 7fc8966e4640 1 -- 172.21.15.100:0/924218968 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc89014f860 con 0x7fc890155440 2024-09-15T21:46:23.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.829+0000 7fc8966e4640 1 -- 172.21.15.100:0/924218968 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8900b3940 con 0x7fc890155440 2024-09-15T21:46:23.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.830+0000 7fc88cff9640 1 -- 172.21.15.100:0/924218968 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc8880054b0 con 0x7fc890155440 2024-09-15T21:46:23.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.830+0000 7fc88cff9640 1 --2- 172.21.15.100:0/924218968 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc87006c2b0 0x7fc87006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:23.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.830+0000 7fc88cff9640 1 -- 172.21.15.100:0/924218968 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc88808b1a0 con 0x7fc890155440 2024-09-15T21:46:23.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.830+0000 7fc88ffff640 1 --2- 172.21.15.100:0/924218968 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc87006c2b0 0x7fc87006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:23.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.830+0000 7fc88ffff640 1 --2- 172.21.15.100:0/924218968 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc87006c2b0 0x7fc87006e770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc880034000 tx=0x7fc88002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:23.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:23.834+0000 7fc88cff9640 1 -- 172.21.15.100:0/924218968 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc8880599a0 con 0x7fc890155440 2024-09-15T21:46:24.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.002+0000 7fc8966e4640 1 -- 172.21.15.100:0/924218968 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc89014fa30 con 0x7fc890155440 2024-09-15T21:46:24.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.003+0000 7fc88cff9640 1 -- 172.21.15.100:0/924218968 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16750 (secure 0 0 0) 0x7fc8880592a0 con 0x7fc890155440 2024-09-15T21:46:24.005 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:24.005 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":21,"fsid":"30e2fff4-2171-4531-8806-0e03b63f494f","created":"2024-09-15T21:45:59.746897+0000","modified":"2024-09-15T21:46:22.753668+0000","last_up_change":"2024-09-15T21:46:13.224709+0000","last_in_change":"2024-09-15T21:46:01.812129+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:46:11.837603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T21:46:20.577982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T21:46:21.729231+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5887900b-522d-476f-9704-cc2c5ee87a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6819","nonce":3920435546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6821","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6822","nonce":3920435546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6829","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6831","nonce":3920435546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6825","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6827","nonce":3920435546}]},"public_addr":"172.21.15.100:6819/3920435546","cluster_addr":"172.21.15.100:6822/3920435546","heartbeat_back_addr":"172.21.15.100:6831/3920435546","heartbeat_front_addr":"172.21.15.100:6827/3920435546","state":["exists","up"]},{"osd":1,"uuid":"adc17993-5c50-4251-85e1-4c7ac62e25e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6801","nonce":3802679972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6803","nonce":3802679972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6807","nonce":3802679972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6805","nonce":3802679972}]},"public_addr":"172.21.15.100:6801/3802679972","cluster_addr":"172.21.15.100:6803/3802679972","heartbeat_back_addr":"172.21.15.100:6807/3802679972","heartbeat_front_addr":"172.21.15.100:6805/3802679972","state":["exists","up"]},{"osd":2,"uuid":"8de3b7d5-1980-4a3b-a934-264f867c7c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6823","nonce":2316787974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6826","nonce":2316787974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6833","nonce":2316787974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6830","nonce":2316787974}]},"public_addr":"172.21.15.100:6823/2316787974","cluster_addr":"172.21.15.100:6826/2316787974","heartbeat_back_addr":"172.21.15.100:6833/2316787974","heartbeat_front_addr":"172.21.15.100:6830/2316787974","state":["exists","up"]},{"osd":3,"uuid":"93dbcd1c-6a6f-49cf-bc94-e64b883de5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6811","nonce":1287562741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6813","nonce":1287562741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6817","nonce":1287562741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6815","nonce":1287562741}]},"public_addr":"172.21.15.100:6811/1287562741","cluster_addr":"172.21.15.100:6813/1287562741","heartbeat_back_addr":"172.21.15.100:6817/1287562741","heartbeat_front_addr":"172.21.15.100:6815/1287562741","state":["exists","up"]},{"osd":4,"uuid":"7caee2a0-bab4-45a9-8109-724948c863c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6801","nonce":4045445512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6803","nonce":4045445512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6807","nonce":4045445512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6805","nonce":4045445512}]},"public_addr":"172.21.15.122:6801/4045445512","cluster_addr":"172.21.15.122:6803/4045445512","heartbeat_back_addr":"172.21.15.122:6807/4045445512","heartbeat_front_addr":"172.21.15.122:6805/4045445512","state":["exists","up"]},{"osd":5,"uuid":"6e64070a-671a-444b-a061-b3051a29b121","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6825","nonce":2400668532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6827","nonce":2400668532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6831","nonce":2400668532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6829","nonce":2400668532}]},"public_addr":"172.21.15.122:6825/2400668532","cluster_addr":"172.21.15.122:6827/2400668532","heartbeat_back_addr":"172.21.15.122:6831/2400668532","heartbeat_front_addr":"172.21.15.122:6829/2400668532","state":["exists","up"]},{"osd":6,"uuid":"47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6809","nonce":1437167311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6811","nonce":1437167311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6815","nonce":1437167311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6813","nonce":1437167311}]},"public_addr":"172.21.15.122:6809/1437167311","cluster_addr":"172.21.15.122:6811/1437167311","heartbeat_back_addr":"172.21.15.122:6815/1437167311","heartbeat_front_addr":"172.21.15.122:6813/1437167311","state":["exists","up"]},{"osd":7,"uuid":"9a916b6d-af82-486f-af40-4a7291c900fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6817","nonce":2431377162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6819","nonce":2431377162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6823","nonce":2431377162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6821","nonce":2431377162}]},"public_addr":"172.21.15.122:6817/2431377162","cluster_addr":"172.21.15.122:6819/2431377162","heartbeat_back_addr":"172.21.15.122:6823/2431377162","heartbeat_front_addr":"172.21.15.122:6821/2431377162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.168771+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.886614+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.154129+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.570531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.926773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.146320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.409137+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.373359+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:46:24.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.005+0000 7fc8966e4640 1 -- 172.21.15.100:0/924218968 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc87006c2b0 msgr2=0x7fc87006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:24.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.005+0000 7fc8966e4640 1 --2- 172.21.15.100:0/924218968 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc87006c2b0 0x7fc87006e770 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc880034000 tx=0x7fc88002e000 comp rx=0 tx=0).stop 2024-09-15T21:46:24.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.005+0000 7fc8966e4640 1 -- 172.21.15.100:0/924218968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc890155440 msgr2=0x7fc89014b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:24.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.005+0000 7fc8966e4640 1 --2- 172.21.15.100:0/924218968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc890155440 0x7fc89014b2e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc88800c6b0 tx=0x7fc88800cb70 comp rx=0 tx=0).stop 2024-09-15T21:46:24.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.005+0000 7fc8966e4640 1 -- 172.21.15.100:0/924218968 shutdown_connections 2024-09-15T21:46:24.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.005+0000 7fc8966e4640 1 --2- 172.21.15.100:0/924218968 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc87006c2b0 0x7fc87006e770 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:24.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.005+0000 7fc8966e4640 1 --2- 172.21.15.100:0/924218968 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8900cb7d0 0x7fc890150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:24.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.005+0000 7fc8966e4640 1 --2- 172.21.15.100:0/924218968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc890155440 0x7fc89014b2e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:24.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.005+0000 7fc8966e4640 1 --2- 172.21.15.100:0/924218968 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc89006d0e0 0x7fc890151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:24.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.005+0000 7fc8966e4640 1 -- 172.21.15.100:0/924218968 >> 172.21.15.100:0/924218968 conn(0x7fc8900680d0 msgr2=0x7fc8900c6360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:24.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.005+0000 7fc8966e4640 1 -- 172.21.15.100:0/924218968 shutdown_connections 2024-09-15T21:46:24.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.005+0000 7fc8966e4640 1 -- 172.21.15.100:0/924218968 wait complete. 2024-09-15T21:46:24.018 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-15T21:46:24.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.117+0000 7fe3deae6640 1 Processor -- start 2024-09-15T21:46:24.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.118+0000 7fe3deae6640 1 -- start start 2024-09-15T21:46:24.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.118+0000 7fe3deae6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe3d80c8490 0x7fe3d80c8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:24.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.118+0000 7fe3deae6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe3d814f3b0 0x7fe3d814f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:24.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.118+0000 7fe3deae6640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3d8071070 con 0x7fe3d80c68c0 2024-09-15T21:46:24.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.118+0000 7fe3deae6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fe3d8159ca0 con 0x7fe3d814f3b0 2024-09-15T21:46:24.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.118+0000 7fe3deae6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d8073b10 con 0x7fe3d80c8490 2024-09-15T21:46:24.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.118+0000 7fe3dd083640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe3d814f3b0 0x7fe3d814f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.118+0000 7fe3dd083640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe3d814f3b0 0x7fe3d814f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47734/0 (socket says 172.21.15.100:47734) 2024-09-15T21:46:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.118+0000 7fe3dd083640 1 -- 172.21.15.100:0/2445741047 learned_addr learned my addr 172.21.15.100:0/2445741047 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.119+0000 7fe3dc882640 1 --2- 172.21.15.100:0/2445741047 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe3d80c8490 0x7fe3d80c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.119+0000 7fe3dd083640 1 -- 172.21.15.100:0/2445741047 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe3d80c8490 msgr2=0x7fe3d80c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.119+0000 7fe3dd083640 1 --2- 172.21.15.100:0/2445741047 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe3d80c8490 0x7fe3d80c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.119+0000 7fe3dd083640 1 -- 172.21.15.100:0/2445741047 >> v1:172.21.15.122:6789/0 conn(0x7fe3d80c68c0 legacy=0x7fe3d8070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.119+0000 7fe3dd083640 1 -- 172.21.15.100:0/2445741047 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d8158ee0 con 0x7fe3d814f3b0 2024-09-15T21:46:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.119+0000 7fe3dd083640 1 --2- 172.21.15.100:0/2445741047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d814f3b0 0x7fe3d814f860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009e50 tx=0x7fe3d002a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8cc77a34c56aea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.120+0000 7fe3d77fe640 1 -- 172.21.15.100:0/2445741047 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0036070 con 0x7fe3d814f3b0 2024-09-15T21:46:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.120+0000 7fe3d77fe640 1 -- 172.21.15.100:0/2445741047 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3d0031600 con 0x7fe3d814f3b0 2024-09-15T21:46:24.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.120+0000 7fe3d77fe640 1 -- 172.21.15.100:0/2445741047 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d00319d0 con 0x7fe3d814f3b0 2024-09-15T21:46:24.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.120+0000 7fe3deae6640 1 -- 172.21.15.100:0/2445741047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d814f3b0 msgr2=0x7fe3d814f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:24.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.120+0000 7fe3deae6640 1 --2- 172.21.15.100:0/2445741047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d814f3b0 0x7fe3d814f860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe3d0009e50 tx=0x7fe3d002a750 comp rx=0 tx=0).stop 2024-09-15T21:46:24.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.120+0000 7fe3deae6640 1 -- 172.21.15.100:0/2445741047 shutdown_connections 2024-09-15T21:46:24.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.120+0000 7fe3deae6640 1 --2- 172.21.15.100:0/2445741047 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d814f3b0 0x7fe3d814f860 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:24.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.120+0000 7fe3deae6640 1 --2- 172.21.15.100:0/2445741047 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe3d80c8490 0x7fe3d80c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:24.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.120+0000 7fe3deae6640 1 -- 172.21.15.100:0/2445741047 >> 172.21.15.100:0/2445741047 conn(0x7fe3d8067980 msgr2=0x7fe3d8067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:24.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.120+0000 7fe3deae6640 1 -- 172.21.15.100:0/2445741047 shutdown_connections 2024-09-15T21:46:24.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.120+0000 7fe3deae6640 1 -- 172.21.15.100:0/2445741047 wait complete. 2024-09-15T21:46:24.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.121+0000 7fe3deae6640 1 Processor -- start 2024-09-15T21:46:24.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.121+0000 7fe3deae6640 1 -- start start 2024-09-15T21:46:24.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.122+0000 7fe3deae6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d80c68c0 0x7fe3d80cca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:24.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.122+0000 7fe3deae6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3d80c8490 0x7fe3d8148f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:24.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.122+0000 7fe3deae6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe3d814f3b0 0x7fe3d806e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:24.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.122+0000 7fe3deae6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d8159780 con 0x7fe3d80c68c0 2024-09-15T21:46:24.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.122+0000 7fe3deae6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81594a0 con 0x7fe3d80c8490 2024-09-15T21:46:24.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.122+0000 7fe3deae6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe3d80737b0 con 0x7fe3d814f3b0 2024-09-15T21:46:24.126 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.122+0000 7fe3dc882640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d80c68c0 0x7fe3d80cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:24.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.122+0000 7fe3dc882640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d80c68c0 0x7fe3d80cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47748/0 (socket says 172.21.15.100:47748) 2024-09-15T21:46:24.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.122+0000 7fe3dc882640 1 -- 172.21.15.100:0/3003555885 learned_addr learned my addr 172.21.15.100:0/3003555885 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:24.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.122+0000 7fe3d7fff640 1 --2- 172.21.15.100:0/3003555885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3d80c8490 0x7fe3d8148f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:24.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.122+0000 7fe3dd083640 1 --2- 172.21.15.100:0/3003555885 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe3d814f3b0 0x7fe3d806e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:24.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.123+0000 7fe3dc882640 1 -- 172.21.15.100:0/3003555885 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe3d814f3b0 msgr2=0x7fe3d806e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:24.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.123+0000 7fe3dc882640 1 --2- 172.21.15.100:0/3003555885 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe3d814f3b0 0x7fe3d806e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:24.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.123+0000 7fe3dc882640 1 -- 172.21.15.100:0/3003555885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3d80c8490 msgr2=0x7fe3d8148f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:24.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.123+0000 7fe3dc882640 1 --2- 172.21.15.100:0/3003555885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3d80c8490 0x7fe3d8148f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:24.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.123+0000 7fe3dc882640 1 -- 172.21.15.100:0/3003555885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d806e5e0 con 0x7fe3d80c68c0 2024-09-15T21:46:24.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.123+0000 7fe3dc882640 1 --2- 172.21.15.100:0/3003555885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d80c68c0 0x7fe3d80cca50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800c820 tx=0x7fe3c800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:24.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.123+0000 7fe3d57fa640 1 -- 172.21.15.100:0/3003555885 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8015070 con 0x7fe3d80c68c0 2024-09-15T21:46:24.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.123+0000 7fe3d57fa640 1 -- 172.21.15.100:0/3003555885 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3c8004240 con 0x7fe3d80c68c0 2024-09-15T21:46:24.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.123+0000 7fe3deae6640 1 -- 172.21.15.100:0/3003555885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d8158b70 con 0x7fe3d80c68c0 2024-09-15T21:46:24.127 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.123+0000 7fe3d57fa640 1 -- 172.21.15.100:0/3003555885 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c800ce10 con 0x7fe3d80c68c0 2024-09-15T21:46:24.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.123+0000 7fe3deae6640 1 -- 172.21.15.100:0/3003555885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d806eb80 con 0x7fe3d80c68c0 2024-09-15T21:46:24.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.124+0000 7fe3deae6640 1 -- 172.21.15.100:0/3003555885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3d8073df0 con 0x7fe3d80c68c0 2024-09-15T21:46:24.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.124+0000 7fe3d57fa640 1 -- 172.21.15.100:0/3003555885 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe3c8007830 con 0x7fe3d80c68c0 2024-09-15T21:46:24.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.125+0000 7fe3d57fa640 1 --2- 172.21.15.100:0/3003555885 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe3b806c2b0 0x7fe3b806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:24.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.125+0000 7fe3d57fa640 1 -- 172.21.15.100:0/3003555885 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe3c808b300 con 0x7fe3d80c68c0 2024-09-15T21:46:24.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.125+0000 7fe3d7fff640 1 --2- 172.21.15.100:0/3003555885 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe3b806c2b0 0x7fe3b806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:24.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.125+0000 7fe3d7fff640 1 --2- 172.21.15.100:0/3003555885 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe3b806c2b0 0x7fe3b806e770 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80ca060 tx=0x7fe3c0009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:24.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.129+0000 7fe3d57fa640 1 -- 172.21.15.100:0/3003555885 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe3c8059390 con 0x7fe3d80c68c0 2024-09-15T21:46:24.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.328+0000 7fe3deae6640 1 -- 172.21.15.100:0/3003555885 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe3d80c9db0 con 0x7fe3d80c68c0 2024-09-15T21:46:24.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.422+0000 7fe3d57fa640 1 -- 172.21.15.100:0/3003555885 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7fe3c805d690 con 0x7fe3d80c68c0 2024-09-15T21:46:24.428 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.424+0000 7fe3deae6640 1 -- 172.21.15.100:0/3003555885 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe3b806c2b0 msgr2=0x7fe3b806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:24.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.424+0000 7fe3deae6640 1 --2- 172.21.15.100:0/3003555885 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe3b806c2b0 0x7fe3b806e770 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe3d80ca060 tx=0x7fe3c0009060 comp rx=0 tx=0).stop 2024-09-15T21:46:24.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.424+0000 7fe3deae6640 1 -- 172.21.15.100:0/3003555885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d80c68c0 msgr2=0x7fe3d80cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:24.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.424+0000 7fe3deae6640 1 --2- 172.21.15.100:0/3003555885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d80c68c0 0x7fe3d80cca50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800c820 tx=0x7fe3c800cce0 comp rx=0 tx=0).stop 2024-09-15T21:46:24.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.424+0000 7fe3deae6640 1 -- 172.21.15.100:0/3003555885 shutdown_connections 2024-09-15T21:46:24.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.424+0000 7fe3deae6640 1 --2- 172.21.15.100:0/3003555885 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe3b806c2b0 0x7fe3b806e770 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:24.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.424+0000 7fe3deae6640 1 --2- 172.21.15.100:0/3003555885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe3d80c8490 0x7fe3d8148f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:24.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.424+0000 7fe3deae6640 1 --2- 172.21.15.100:0/3003555885 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe3d814f3b0 0x7fe3d806e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:24.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.424+0000 7fe3deae6640 1 --2- 172.21.15.100:0/3003555885 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe3d80c68c0 0x7fe3d80cca50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:24.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.424+0000 7fe3deae6640 1 -- 172.21.15.100:0/3003555885 >> 172.21.15.100:0/3003555885 conn(0x7fe3d8067980 msgr2=0x7fe3d80c5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:24.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.424+0000 7fe3deae6640 1 -- 172.21.15.100:0/3003555885 shutdown_connections 2024-09-15T21:46:24.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.424+0000 7fe3deae6640 1 -- 172.21.15.100:0/3003555885 wait complete. 2024-09-15T21:46:24.437 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300} 2024-09-15T21:46:24.437 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-09-15T21:46:24.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.537+0000 7fe67141e640 1 Processor -- start 2024-09-15T21:46:24.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.537+0000 7fe67141e640 1 -- start start 2024-09-15T21:46:24.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.537+0000 7fe67141e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe66c155680 0x7fe66c15b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:24.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.537+0000 7fe67141e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe66c15b7f0 con 0x7fe66c0cb910 2024-09-15T21:46:24.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.537+0000 7fe67141e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe66c15bab0 con 0x7fe66c1552a0 2024-09-15T21:46:24.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.537+0000 7fe67141e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe66c15bd90 con 0x7fe66c155680 2024-09-15T21:46:24.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe66b7fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe66c0cb910 0x7fe66c0c6d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:46410/0 (socket says 172.21.15.100:46410) 2024-09-15T21:46:24.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe66b7fe640 1 -- 172.21.15.100:0/564212164 learned_addr learned my addr 172.21.15.100:0/564212164 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:24.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe66a7fc640 1 --2- 172.21.15.100:0/564212164 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe66c155680 0x7fe66c15b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:24.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe669ffb640 1 -- 172.21.15.100:0/564212164 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1036934136 0 0) 0x7fe66c15b7f0 con 0x7fe66c0cb910 2024-09-15T21:46:24.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe669ffb640 1 -- 172.21.15.100:0/564212164 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe658003520 con 0x7fe66c0cb910 2024-09-15T21:46:24.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe669ffb640 1 -- 172.21.15.100:0/564212164 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1911439639 0 0) 0x7fe66c15bab0 con 0x7fe66c1552a0 2024-09-15T21:46:24.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe669ffb640 1 -- 172.21.15.100:0/564212164 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe66c15b7f0 con 0x7fe66c1552a0 2024-09-15T21:46:24.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe669ffb640 1 -- 172.21.15.100:0/564212164 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3214799413 0 0) 0x7fe658003520 con 0x7fe66c0cb910 2024-09-15T21:46:24.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe669ffb640 1 -- 172.21.15.100:0/564212164 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe66c155680 msgr2=0x7fe66c15b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:24.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe669ffb640 1 --2- 172.21.15.100:0/564212164 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe66c155680 0x7fe66c15b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:24.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe669ffb640 1 -- 172.21.15.100:0/564212164 >> v1:172.21.15.122:6789/0 conn(0x7fe66c1552a0 legacy=0x7fe66c0c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:24.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe669ffb640 1 -- 172.21.15.100:0/564212164 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c15c070 con 0x7fe66c0cb910 2024-09-15T21:46:24.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe669ffb640 1 -- 172.21.15.100:0/564212164 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7fe6640019f0 con 0x7fe66c0cb910 2024-09-15T21:46:24.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe669ffb640 1 -- 172.21.15.100:0/564212164 >> v1:172.21.15.100:6789/0 conn(0x7fe66c0cb910 legacy=0x7fe66c0c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:24.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe669ffb640 1 --2- 172.21.15.100:0/564212164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe658003b70 0x7fe658003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:24.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe669ffb640 1 --2- 172.21.15.100:0/564212164 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe66c155680 0x7fe658004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:24.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe669ffb640 1 --2- 172.21.15.100:0/564212164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe658004d40 0x7fe6580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:24.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe669ffb640 1 -- 172.21.15.100:0/564212164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c15bd90 con 0x7fe658004d40 2024-09-15T21:46:24.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe669ffb640 1 -- 172.21.15.100:0/564212164 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe65800bb00 con 0x7fe658003b70 2024-09-15T21:46:24.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe669ffb640 1 -- 172.21.15.100:0/564212164 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe65800bd00 con 0x7fe66c155680 2024-09-15T21:46:24.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe66b7fe640 1 --2- 172.21.15.100:0/564212164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe658004d40 0x7fe6580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:24.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe66a7fc640 1 --2- 172.21.15.100:0/564212164 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe66c155680 0x7fe658004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:24.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.538+0000 7fe66affd640 1 --2- 172.21.15.100:0/564212164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe658003b70 0x7fe658003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:24.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.539+0000 7fe66b7fe640 1 -- 172.21.15.100:0/564212164 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe66c155680 msgr2=0x7fe658004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:24.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.539+0000 7fe66b7fe640 1 --2- 172.21.15.100:0/564212164 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe66c155680 0x7fe658004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:24.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.539+0000 7fe66b7fe640 1 -- 172.21.15.100:0/564212164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe658003b70 msgr2=0x7fe658003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:24.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.539+0000 7fe66b7fe640 1 --2- 172.21.15.100:0/564212164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe658003b70 0x7fe658003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:24.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.539+0000 7fe66b7fe640 1 -- 172.21.15.100:0/564212164 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe658003520 con 0x7fe658004d40 2024-09-15T21:46:24.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.539+0000 7fe66b7fe640 1 --2- 172.21.15.100:0/564212164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe658004d40 0x7fe6580051c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe66400a710 tx=0x7fe658005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b5b5d516965ecb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:24.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.539+0000 7fe669ffb640 1 -- 172.21.15.100:0/564212164 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66400a330 con 0x7fe658004d40 2024-09-15T21:46:24.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.539+0000 7fe669ffb640 1 -- 172.21.15.100:0/564212164 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe66402f070 con 0x7fe658004d40 2024-09-15T21:46:24.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.539+0000 7fe669ffb640 1 -- 172.21.15.100:0/564212164 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66402bb60 con 0x7fe658004d40 2024-09-15T21:46:24.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.540+0000 7fe67141e640 1 -- 172.21.15.100:0/564212164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe658004d40 msgr2=0x7fe6580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:24.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.540+0000 7fe67141e640 1 --2- 172.21.15.100:0/564212164 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe658004d40 0x7fe6580051c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe66400a710 tx=0x7fe658005830 comp rx=0 tx=0).stop 2024-09-15T21:46:24.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.540+0000 7fe66b7fe640 1 -- 172.21.15.100:0/564212164 reap_dead start 2024-09-15T21:46:24.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.540+0000 7fe67141e640 1 -- 172.21.15.100:0/564212164 shutdown_connections 2024-09-15T21:46:24.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.540+0000 7fe67141e640 1 -- 172.21.15.100:0/564212164 >> 172.21.15.100:0/564212164 conn(0x7fe66c06d9a0 msgr2=0x7fe66c06ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:24.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.540+0000 7fe67141e640 1 -- 172.21.15.100:0/564212164 shutdown_connections 2024-09-15T21:46:24.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.540+0000 7fe67141e640 1 -- 172.21.15.100:0/564212164 wait complete. 2024-09-15T21:46:24.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.541+0000 7fe67141e640 1 Processor -- start 2024-09-15T21:46:24.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.541+0000 7fe67141e640 1 -- start start 2024-09-15T21:46:24.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.542+0000 7fe67141e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe658004d40 0x7fe66c1f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.542+0000 7fe67141e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe66c0cf3c0 0x7fe66c1536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.542+0000 7fe67141e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe66c0d0530 0x7fe66c14b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.542+0000 7fe67141e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c0c61f0 con 0x7fe66c0d0530 2024-09-15T21:46:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.542+0000 7fe67141e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c0c6820 con 0x7fe658004d40 2024-09-15T21:46:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.542+0000 7fe67141e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe66c0c7450 con 0x7fe66c0cf3c0 2024-09-15T21:46:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.542+0000 7fe66b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe66c0d0530 0x7fe66c14b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.542+0000 7fe66b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe66c0d0530 0x7fe66c14b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47770/0 (socket says 172.21.15.100:47770) 2024-09-15T21:46:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.542+0000 7fe66b7fe640 1 -- 172.21.15.100:0/450910602 learned_addr learned my addr 172.21.15.100:0/450910602 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.542+0000 7fe66a7fc640 1 --2- 172.21.15.100:0/450910602 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe66c0cf3c0 0x7fe66c1536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.542+0000 7fe66affd640 1 --2- 172.21.15.100:0/450910602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe658004d40 0x7fe66c1f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.542+0000 7fe66b7fe640 1 -- 172.21.15.100:0/450910602 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe66c0cf3c0 msgr2=0x7fe66c1536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.542+0000 7fe66b7fe640 1 --2- 172.21.15.100:0/450910602 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe66c0cf3c0 0x7fe66c1536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:24.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.542+0000 7fe66b7fe640 1 -- 172.21.15.100:0/450910602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe658004d40 msgr2=0x7fe66c1f4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.542+0000 7fe66b7fe640 1 --2- 172.21.15.100:0/450910602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe658004d40 0x7fe66c1f4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.542+0000 7fe66b7fe640 1 -- 172.21.15.100:0/450910602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c14fa10 con 0x7fe66c0d0530 2024-09-15T21:46:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.542+0000 7fe66a7fc640 1 --2- 172.21.15.100:0/450910602 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe66c0cf3c0 0x7fe66c1536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.542+0000 7fe66b7fe640 1 --2- 172.21.15.100:0/450910602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe66c0d0530 0x7fe66c14b590 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe66400a830 tx=0x7fe66c15bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.543+0000 7fe6637fe640 1 -- 172.21.15.100:0/450910602 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664044070 con 0x7fe66c0d0530 2024-09-15T21:46:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.543+0000 7fe6637fe640 1 -- 172.21.15.100:0/450910602 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe66402ce50 con 0x7fe66c0d0530 2024-09-15T21:46:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.543+0000 7fe67141e640 1 -- 172.21.15.100:0/450910602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe66c14fcd0 con 0x7fe66c0d0530 2024-09-15T21:46:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.543+0000 7fe6637fe640 1 -- 172.21.15.100:0/450910602 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66400a330 con 0x7fe66c0d0530 2024-09-15T21:46:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.543+0000 7fe67141e640 1 -- 172.21.15.100:0/450910602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe66c0cfd20 con 0x7fe66c0d0530 2024-09-15T21:46:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.544+0000 7fe67141e640 1 -- 172.21.15.100:0/450910602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe66c15b0f0 con 0x7fe66c0d0530 2024-09-15T21:46:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.544+0000 7fe6637fe640 1 -- 172.21.15.100:0/450910602 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe664047070 con 0x7fe66c0d0530 2024-09-15T21:46:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.545+0000 7fe6637fe640 1 --2- 172.21.15.100:0/450910602 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe64c06c2b0 0x7fe64c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.545+0000 7fe66affd640 1 --2- 172.21.15.100:0/450910602 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe64c06c2b0 0x7fe64c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:24.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.545+0000 7fe6637fe640 1 -- 172.21.15.100:0/450910602 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe6640ab2a0 con 0x7fe66c0d0530 2024-09-15T21:46:24.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.545+0000 7fe66affd640 1 --2- 172.21.15.100:0/450910602 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe64c06c2b0 0x7fe64c06e770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe6540055b0 tx=0x7fe654009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:24.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.549+0000 7fe6637fe640 1 -- 172.21.15.100:0/450910602 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe6640793c0 con 0x7fe66c0d0530 2024-09-15T21:46:24.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:24.750+0000 7fe67141e640 1 -- 172.21.15.100:0/450910602 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fe66c0c7980 con 0x7fe66c0d0530 2024-09-15T21:46:25.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.428+0000 7fe6637fe640 1 -- 172.21.15.100:0/450910602 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v6) ==== 105+0+0 (secure 0 0 0) 0x7fe66407d6c0 con 0x7fe66c0d0530 2024-09-15T21:46:25.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.430+0000 7fe67141e640 1 -- 172.21.15.100:0/450910602 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe64c06c2b0 msgr2=0x7fe64c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:25.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.430+0000 7fe67141e640 1 --2- 172.21.15.100:0/450910602 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe64c06c2b0 0x7fe64c06e770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe6540055b0 tx=0x7fe654009040 comp rx=0 tx=0).stop 2024-09-15T21:46:25.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.430+0000 7fe67141e640 1 -- 172.21.15.100:0/450910602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe66c0d0530 msgr2=0x7fe66c14b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:25.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.430+0000 7fe67141e640 1 --2- 172.21.15.100:0/450910602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe66c0d0530 0x7fe66c14b590 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe66400a830 tx=0x7fe66c15bc80 comp rx=0 tx=0).stop 2024-09-15T21:46:25.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.430+0000 7fe67141e640 1 -- 172.21.15.100:0/450910602 shutdown_connections 2024-09-15T21:46:25.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.430+0000 7fe67141e640 1 --2- 172.21.15.100:0/450910602 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe64c06c2b0 0x7fe64c06e770 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:25.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.430+0000 7fe67141e640 1 --2- 172.21.15.100:0/450910602 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe66c0d0530 0x7fe66c14b590 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:25.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.430+0000 7fe67141e640 1 --2- 172.21.15.100:0/450910602 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe66c0cf3c0 0x7fe66c1536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:25.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.430+0000 7fe67141e640 1 --2- 172.21.15.100:0/450910602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe658004d40 0x7fe66c1f4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:25.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.430+0000 7fe67141e640 1 -- 172.21.15.100:0/450910602 >> 172.21.15.100:0/450910602 conn(0x7fe66c06d9a0 msgr2=0x7fe66c0d0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:25.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.430+0000 7fe67141e640 1 -- 172.21.15.100:0/450910602 shutdown_connections 2024-09-15T21:46:25.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.430+0000 7fe67141e640 1 -- 172.21.15.100:0/450910602 wait complete. 2024-09-15T21:46:25.444 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-15T21:46:25.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.548+0000 7f8f66186640 1 Processor -- start 2024-09-15T21:46:25.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.548+0000 7f8f66186640 1 -- start start 2024-09-15T21:46:25.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.549+0000 7f8f66186640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8f600ced30 0x7f8f6015be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:25.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.549+0000 7f8f66186640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f6015eb40 con 0x7f8f600cf110 2024-09-15T21:46:25.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.549+0000 7f8f66186640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f6015ee20 con 0x7f8f60144250 2024-09-15T21:46:25.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.549+0000 7f8f66186640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f8f6015c340 con 0x7f8f600ced30 2024-09-15T21:46:25.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.549+0000 7f8f5f7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8f600ced30 0x7f8f6015be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:25.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.549+0000 7f8f5f7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8f600ced30 0x7f8f6015be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47786/0 (socket says 172.21.15.100:47786) 2024-09-15T21:46:25.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.549+0000 7f8f5f7fe640 1 -- 172.21.15.100:0/2821486419 learned_addr learned my addr 172.21.15.100:0/2821486419 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:25.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.549+0000 7f8f5f7fe640 1 -- 172.21.15.100:0/2821486419 >> v1:172.21.15.122:6789/0 conn(0x7f8f60144250 legacy=0x7f8f60144690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:25.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.549+0000 7f8f5f7fe640 1 -- 172.21.15.100:0/2821486419 >> v1:172.21.15.100:6789/0 conn(0x7f8f600cf110 legacy=0x7f8f600739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:25.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.549+0000 7f8f5f7fe640 1 -- 172.21.15.100:0/2821486419 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6015f100 con 0x7f8f600ced30 2024-09-15T21:46:25.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.550+0000 7f8f5f7fe640 1 --2- 172.21.15.100:0/2821486419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f600ced30 0x7f8f6015be00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009870 tx=0x7f8f5002a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a04f61a6f98706 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:25.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.550+0000 7f8f5e7fc640 1 -- 172.21.15.100:0/2821486419 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50037070 con 0x7f8f600ced30 2024-09-15T21:46:25.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.550+0000 7f8f5e7fc640 1 -- 172.21.15.100:0/2821486419 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f50032640 con 0x7f8f600ced30 2024-09-15T21:46:25.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.550+0000 7f8f5e7fc640 1 -- 172.21.15.100:0/2821486419 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f50032a10 con 0x7f8f600ced30 2024-09-15T21:46:25.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.550+0000 7f8f66186640 1 -- 172.21.15.100:0/2821486419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f600ced30 msgr2=0x7f8f6015be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:25.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.550+0000 7f8f66186640 1 --2- 172.21.15.100:0/2821486419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f600ced30 0x7f8f6015be00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009870 tx=0x7f8f5002a330 comp rx=0 tx=0).stop 2024-09-15T21:46:25.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.550+0000 7f8f66186640 1 -- 172.21.15.100:0/2821486419 shutdown_connections 2024-09-15T21:46:25.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.550+0000 7f8f66186640 1 --2- 172.21.15.100:0/2821486419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f600ced30 0x7f8f6015be00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:25.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.550+0000 7f8f66186640 1 -- 172.21.15.100:0/2821486419 >> 172.21.15.100:0/2821486419 conn(0x7f8f6006f990 msgr2=0x7f8f60070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:25.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.551+0000 7f8f66186640 1 -- 172.21.15.100:0/2821486419 shutdown_connections 2024-09-15T21:46:25.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.551+0000 7f8f66186640 1 -- 172.21.15.100:0/2821486419 wait complete. 2024-09-15T21:46:25.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.552+0000 7f8f66186640 1 Processor -- start 2024-09-15T21:46:25.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.552+0000 7f8f66186640 1 -- start start 2024-09-15T21:46:25.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.552+0000 7f8f66186640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f600ced30 0x7f8f601f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:25.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.552+0000 7f8f66186640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f600cf110 0x7f8f600c4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:25.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.552+0000 7f8f66186640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f60144250 0x7f8f60155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:25.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.552+0000 7f8f66186640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6015f750 con 0x7f8f600cf110 2024-09-15T21:46:25.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.552+0000 7f8f66186640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8f6015e8f0 con 0x7f8f60144250 2024-09-15T21:46:25.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.553+0000 7f8f66186640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8f600740c0 con 0x7f8f600ced30 2024-09-15T21:46:25.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.553+0000 7f8f5effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f600cf110 0x7f8f600c4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:25.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.553+0000 7f8f5effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f600cf110 0x7f8f600c4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:47802/0 (socket says 172.21.15.100:47802) 2024-09-15T21:46:25.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.553+0000 7f8f5effd640 1 -- 172.21.15.100:0/2314160955 learned_addr learned my addr 172.21.15.100:0/2314160955 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:25.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.553+0000 7f8f5f7fe640 1 --2- 172.21.15.100:0/2314160955 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f600ced30 0x7f8f601f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:25.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.553+0000 7f8f5ffff640 1 --2- 172.21.15.100:0/2314160955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f60144250 0x7f8f60155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:25.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.553+0000 7f8f5effd640 1 -- 172.21.15.100:0/2314160955 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f600ced30 msgr2=0x7f8f601f0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:25.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.553+0000 7f8f5effd640 1 --2- 172.21.15.100:0/2314160955 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f600ced30 0x7f8f601f0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:25.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.553+0000 7f8f5effd640 1 -- 172.21.15.100:0/2314160955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f60144250 msgr2=0x7f8f60155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:25.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.553+0000 7f8f5effd640 1 --2- 172.21.15.100:0/2314160955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f60144250 0x7f8f60155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:25.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.553+0000 7f8f5effd640 1 -- 172.21.15.100:0/2314160955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f601f4610 con 0x7f8f600cf110 2024-09-15T21:46:25.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.553+0000 7f8f5f7fe640 1 --2- 172.21.15.100:0/2314160955 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f600ced30 0x7f8f601f0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:25.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.553+0000 7f8f5ffff640 1 --2- 172.21.15.100:0/2314160955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f60144250 0x7f8f60155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:25.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.553+0000 7f8f5effd640 1 --2- 172.21.15.100:0/2314160955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f600cf110 0x7f8f600c4330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400c440 tx=0x7f8f54001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:25.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.553+0000 7f8f47fff640 1 -- 172.21.15.100:0/2314160955 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f540052d0 con 0x7f8f600cf110 2024-09-15T21:46:25.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.553+0000 7f8f47fff640 1 -- 172.21.15.100:0/2314160955 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f54005470 con 0x7f8f600cf110 2024-09-15T21:46:25.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.553+0000 7f8f66186640 1 -- 172.21.15.100:0/2314160955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f60156c20 con 0x7f8f600cf110 2024-09-15T21:46:25.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.553+0000 7f8f47fff640 1 -- 172.21.15.100:0/2314160955 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f54007260 con 0x7f8f600cf110 2024-09-15T21:46:25.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.553+0000 7f8f66186640 1 -- 172.21.15.100:0/2314160955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f60156940 con 0x7f8f600cf110 2024-09-15T21:46:25.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.554+0000 7f8f66186640 1 -- 172.21.15.100:0/2314160955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f6015be00 con 0x7f8f600cf110 2024-09-15T21:46:25.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.555+0000 7f8f47fff640 1 -- 172.21.15.100:0/2314160955 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8f54007400 con 0x7f8f600cf110 2024-09-15T21:46:25.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.555+0000 7f8f47fff640 1 --2- 172.21.15.100:0/2314160955 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8f4c06c2b0 0x7f8f4c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:25.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.555+0000 7f8f47fff640 1 -- 172.21.15.100:0/2314160955 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8f5408a8c0 con 0x7f8f600cf110 2024-09-15T21:46:25.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.555+0000 7f8f5f7fe640 1 --2- 172.21.15.100:0/2314160955 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8f4c06c2b0 0x7f8f4c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:25.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.555+0000 7f8f5f7fe640 1 --2- 172.21.15.100:0/2314160955 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8f4c06c2b0 0x7f8f4c06e770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8f50034000 tx=0x7f8f5002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:25.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.559+0000 7f8f47fff640 1 -- 172.21.15.100:0/2314160955 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8f540590c0 con 0x7f8f600cf110 2024-09-15T21:46:25.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:25.760+0000 7f8f66186640 1 -- 172.21.15.100:0/2314160955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f8f60159cd0 con 0x7f8f600cf110 2024-09-15T21:46:26.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.431+0000 7f8f47fff640 1 -- 172.21.15.100:0/2314160955 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f8f540589c0 con 0x7f8f600cf110 2024-09-15T21:46:26.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.433+0000 7f8f66186640 1 -- 172.21.15.100:0/2314160955 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8f4c06c2b0 msgr2=0x7f8f4c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:26.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.433+0000 7f8f66186640 1 --2- 172.21.15.100:0/2314160955 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8f4c06c2b0 0x7f8f4c06e770 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8f50034000 tx=0x7f8f5002e000 comp rx=0 tx=0).stop 2024-09-15T21:46:26.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.433+0000 7f8f66186640 1 -- 172.21.15.100:0/2314160955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f600cf110 msgr2=0x7f8f600c4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:26.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.433+0000 7f8f66186640 1 --2- 172.21.15.100:0/2314160955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f600cf110 0x7f8f600c4330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8f5400c440 tx=0x7f8f54001e70 comp rx=0 tx=0).stop 2024-09-15T21:46:26.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.433+0000 7f8f66186640 1 -- 172.21.15.100:0/2314160955 shutdown_connections 2024-09-15T21:46:26.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.433+0000 7f8f66186640 1 --2- 172.21.15.100:0/2314160955 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8f4c06c2b0 0x7f8f4c06e770 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:26.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.433+0000 7f8f66186640 1 --2- 172.21.15.100:0/2314160955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f60144250 0x7f8f60155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:26.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.433+0000 7f8f66186640 1 --2- 172.21.15.100:0/2314160955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8f600cf110 0x7f8f600c4330 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:26.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.433+0000 7f8f66186640 1 --2- 172.21.15.100:0/2314160955 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8f600ced30 0x7f8f601f0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:26.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.433+0000 7f8f66186640 1 -- 172.21.15.100:0/2314160955 >> 172.21.15.100:0/2314160955 conn(0x7f8f6006f990 msgr2=0x7f8f6006dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:26.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.433+0000 7f8f66186640 1 -- 172.21.15.100:0/2314160955 shutdown_connections 2024-09-15T21:46:26.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.433+0000 7f8f66186640 1 -- 172.21.15.100:0/2314160955 wait complete. 2024-09-15T21:46:26.446 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-09-15T21:46:26.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.546+0000 7fcc629e4640 1 Processor -- start 2024-09-15T21:46:26.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.547+0000 7fcc629e4640 1 -- start start 2024-09-15T21:46:26.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.547+0000 7fcc629e4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fcc5c06d1f0 0x7fcc5c0cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:26.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.547+0000 7fcc629e4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc5c15e850 con 0x7fcc5c0cc390 2024-09-15T21:46:26.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.547+0000 7fcc629e4640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc5c15ead0 con 0x7fcc5c0cbfb0 2024-09-15T21:46:26.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.547+0000 7fcc629e4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fcc5c15f400 con 0x7fcc5c06d1f0 2024-09-15T21:46:26.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.547+0000 7fcc60f81640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fcc5c0cc390 0x7fcc5c154f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:51768/0 (socket says 172.21.15.100:51768) 2024-09-15T21:46:26.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.547+0000 7fcc60f81640 1 -- 172.21.15.100:0/1419450143 learned_addr learned my addr 172.21.15.100:0/1419450143 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:26.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc53fff640 1 --2- 172.21.15.100:0/1419450143 >> v2:172.21.15.122:3300/0 conn(0x7fcc5c06d1f0 0x7fcc5c0cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:26.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5b7fe640 1 -- 172.21.15.100:0/1419450143 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 568339822 0 0) 0x7fcc54001490 con 0x7fcc5c0cc390 2024-09-15T21:46:26.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5b7fe640 1 -- 172.21.15.100:0/1419450143 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc48003520 con 0x7fcc5c0cc390 2024-09-15T21:46:26.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5b7fe640 1 -- 172.21.15.100:0/1419450143 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 383807865 0 0) 0x7fcc48003520 con 0x7fcc5c0cc390 2024-09-15T21:46:26.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5b7fe640 1 -- 172.21.15.100:0/1419450143 >> v2:172.21.15.122:3300/0 conn(0x7fcc5c06d1f0 msgr2=0x7fcc5c0cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:26.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5b7fe640 1 --2- 172.21.15.100:0/1419450143 >> v2:172.21.15.122:3300/0 conn(0x7fcc5c06d1f0 0x7fcc5c0cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:26.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5b7fe640 1 -- 172.21.15.100:0/1419450143 >> v1:172.21.15.122:6789/0 conn(0x7fcc5c0cbfb0 legacy=0x7fcc5c0c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:26.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5b7fe640 1 -- 172.21.15.100:0/1419450143 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5c15edb0 con 0x7fcc5c0cc390 2024-09-15T21:46:26.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5b7fe640 1 -- 172.21.15.100:0/1419450143 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7fcc54001bc0 con 0x7fcc5c0cc390 2024-09-15T21:46:26.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5b7fe640 1 -- 172.21.15.100:0/1419450143 >> v1:172.21.15.100:6789/0 conn(0x7fcc5c0cc390 legacy=0x7fcc5c154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:26.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5b7fe640 1 --2- 172.21.15.100:0/1419450143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc480039b0 0x7fcc48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:26.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5b7fe640 1 --2- 172.21.15.100:0/1419450143 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcc48004800 0x7fcc48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:26.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5b7fe640 1 --2- 172.21.15.100:0/1419450143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc480051c0 0x7fcc4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:26.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5b7fe640 1 -- 172.21.15.100:0/1419450143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4800c040 con 0x7fcc480039b0 2024-09-15T21:46:26.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5b7fe640 1 -- 172.21.15.100:0/1419450143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4800c1f0 con 0x7fcc480051c0 2024-09-15T21:46:26.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5b7fe640 1 -- 172.21.15.100:0/1419450143 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fcc4800c3f0 con 0x7fcc48004800 2024-09-15T21:46:26.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5bfff640 1 --2- 172.21.15.100:0/1419450143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc480039b0 0x7fcc48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:26.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc60f81640 1 --2- 172.21.15.100:0/1419450143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc480051c0 0x7fcc4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:26.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc53fff640 1 --2- 172.21.15.100:0/1419450143 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcc48004800 0x7fcc48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:26.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5bfff640 1 -- 172.21.15.100:0/1419450143 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcc48004800 msgr2=0x7fcc48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:26.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5bfff640 1 --2- 172.21.15.100:0/1419450143 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcc48004800 0x7fcc48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:26.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5bfff640 1 -- 172.21.15.100:0/1419450143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc480051c0 msgr2=0x7fcc4800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:26.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5bfff640 1 --2- 172.21.15.100:0/1419450143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc480051c0 0x7fcc4800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:26.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc5bfff640 1 -- 172.21.15.100:0/1419450143 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc48003520 con 0x7fcc480039b0 2024-09-15T21:46:26.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc60f81640 1 -- 172.21.15.100:0/1419450143 reap_dead start 2024-09-15T21:46:26.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.548+0000 7fcc53fff640 1 --2- 172.21.15.100:0/1419450143 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcc48004800 0x7fcc48004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:26.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.549+0000 7fcc5bfff640 1 --2- 172.21.15.100:0/1419450143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc480039b0 0x7fcc48003dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c001750 tx=0x7fcc48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab75dcde601a70e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:26.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.549+0000 7fcc5b7fe640 1 -- 172.21.15.100:0/1419450143 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c001370 con 0x7fcc480039b0 2024-09-15T21:46:26.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.549+0000 7fcc5b7fe640 1 -- 172.21.15.100:0/1419450143 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc4c0052d0 con 0x7fcc480039b0 2024-09-15T21:46:26.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.549+0000 7fcc5b7fe640 1 -- 172.21.15.100:0/1419450143 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c02abc0 con 0x7fcc480039b0 2024-09-15T21:46:26.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.550+0000 7fcc629e4640 1 -- 172.21.15.100:0/1419450143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc480039b0 msgr2=0x7fcc48003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:26.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.550+0000 7fcc629e4640 1 --2- 172.21.15.100:0/1419450143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc480039b0 0x7fcc48003dd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c001750 tx=0x7fcc48005830 comp rx=0 tx=0).stop 2024-09-15T21:46:26.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.550+0000 7fcc629e4640 1 -- 172.21.15.100:0/1419450143 shutdown_connections 2024-09-15T21:46:26.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.550+0000 7fcc629e4640 1 --2- 172.21.15.100:0/1419450143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc480039b0 0x7fcc48003dd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:26.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.550+0000 7fcc629e4640 1 -- 172.21.15.100:0/1419450143 >> 172.21.15.100:0/1419450143 conn(0x7fcc5c06e260 msgr2=0x7fcc5c067980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:26.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.550+0000 7fcc629e4640 1 -- 172.21.15.100:0/1419450143 shutdown_connections 2024-09-15T21:46:26.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.550+0000 7fcc629e4640 1 -- 172.21.15.100:0/1419450143 wait complete. 2024-09-15T21:46:26.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.551+0000 7fcc629e4640 1 Processor -- start 2024-09-15T21:46:26.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.551+0000 7fcc629e4640 1 -- start start 2024-09-15T21:46:26.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc629e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc480039b0 0x7fcc5c148c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:26.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc629e4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc5c1f5350 0x7fcc5c0ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:26.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc629e4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcc5c153a20 0x7fcc5c153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:26.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc629e4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c0cb4c0 con 0x7fcc5c1f5350 2024-09-15T21:46:26.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc629e4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c0cbae0 con 0x7fcc480039b0 2024-09-15T21:46:26.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc629e4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fcc5c0cb640 con 0x7fcc5c153a20 2024-09-15T21:46:26.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc53fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc5c1f5350 0x7fcc5c0ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:26.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc53fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc5c1f5350 0x7fcc5c0ca790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43614/0 (socket says 172.21.15.100:43614) 2024-09-15T21:46:26.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc53fff640 1 -- 172.21.15.100:0/1757568921 learned_addr learned my addr 172.21.15.100:0/1757568921 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:26.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc5bfff640 1 --2- 172.21.15.100:0/1757568921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc480039b0 0x7fcc5c148c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:26.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc60f81640 1 --2- 172.21.15.100:0/1757568921 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcc5c153a20 0x7fcc5c153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:26.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc53fff640 1 -- 172.21.15.100:0/1757568921 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcc5c153a20 msgr2=0x7fcc5c153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:26.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc53fff640 1 --2- 172.21.15.100:0/1757568921 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcc5c153a20 0x7fcc5c153ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:26.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc53fff640 1 -- 172.21.15.100:0/1757568921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc480039b0 msgr2=0x7fcc5c148c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:26.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc53fff640 1 --2- 172.21.15.100:0/1757568921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc480039b0 0x7fcc5c148c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:26.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc53fff640 1 -- 172.21.15.100:0/1757568921 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5c1492a0 con 0x7fcc5c1f5350 2024-09-15T21:46:26.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc5bfff640 1 --2- 172.21.15.100:0/1757568921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc480039b0 0x7fcc5c148c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:26.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc60f81640 1 --2- 172.21.15.100:0/1757568921 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcc5c153a20 0x7fcc5c153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:26.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc53fff640 1 --2- 172.21.15.100:0/1757568921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc5c1f5350 0x7fcc5c0ca790 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcc44011590 tx=0x7fcc44011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:26.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc597fa640 1 -- 172.21.15.100:0/1757568921 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44019070 con 0x7fcc5c1f5350 2024-09-15T21:46:26.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc597fa640 1 -- 172.21.15.100:0/1757568921 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc44005da0 con 0x7fcc5c1f5350 2024-09-15T21:46:26.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc629e4640 1 -- 172.21.15.100:0/1757568921 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5c149560 con 0x7fcc5c1f5350 2024-09-15T21:46:26.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc597fa640 1 -- 172.21.15.100:0/1757568921 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44004240 con 0x7fcc5c1f5350 2024-09-15T21:46:26.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.552+0000 7fcc629e4640 1 -- 172.21.15.100:0/1757568921 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc5c1f1a00 con 0x7fcc5c1f5350 2024-09-15T21:46:26.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.554+0000 7fcc629e4640 1 -- 172.21.15.100:0/1757568921 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc5c14b2d0 con 0x7fcc5c1f5350 2024-09-15T21:46:26.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.554+0000 7fcc597fa640 1 -- 172.21.15.100:0/1757568921 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcc44003610 con 0x7fcc5c1f5350 2024-09-15T21:46:26.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.554+0000 7fcc597fa640 1 --2- 172.21.15.100:0/1757568921 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fcc3c06c2b0 0x7fcc3c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:26.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.554+0000 7fcc597fa640 1 -- 172.21.15.100:0/1757568921 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fcc4408b780 con 0x7fcc5c1f5350 2024-09-15T21:46:26.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.554+0000 7fcc5bfff640 1 --2- 172.21.15.100:0/1757568921 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fcc3c06c2b0 0x7fcc3c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:26.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.555+0000 7fcc5bfff640 1 --2- 172.21.15.100:0/1757568921 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fcc3c06c2b0 0x7fcc3c06e770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c032000 tx=0x7fcc4c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:26.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.559+0000 7fcc597fa640 1 -- 172.21.15.100:0/1757568921 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcc5c14b2d0 con 0x7fcc5c1f5350 2024-09-15T21:46:26.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:26.758+0000 7fcc629e4640 1 -- 172.21.15.100:0/1757568921 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fcc5c15bc40 con 0x7fcc5c1f5350 2024-09-15T21:46:27.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.432+0000 7fcc597fa640 1 -- 172.21.15.100:0/1757568921 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v8) ==== 115+0+0 (secure 0 0 0) 0x7fcc44059e60 con 0x7fcc5c1f5350 2024-09-15T21:46:27.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.435+0000 7fcc629e4640 1 -- 172.21.15.100:0/1757568921 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fcc3c06c2b0 msgr2=0x7fcc3c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:27.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.435+0000 7fcc629e4640 1 --2- 172.21.15.100:0/1757568921 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fcc3c06c2b0 0x7fcc3c06e770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c032000 tx=0x7fcc4c02e000 comp rx=0 tx=0).stop 2024-09-15T21:46:27.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.435+0000 7fcc629e4640 1 -- 172.21.15.100:0/1757568921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc5c1f5350 msgr2=0x7fcc5c0ca790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:27.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.435+0000 7fcc629e4640 1 --2- 172.21.15.100:0/1757568921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc5c1f5350 0x7fcc5c0ca790 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcc44011590 tx=0x7fcc44011a50 comp rx=0 tx=0).stop 2024-09-15T21:46:27.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.435+0000 7fcc629e4640 1 -- 172.21.15.100:0/1757568921 shutdown_connections 2024-09-15T21:46:27.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.435+0000 7fcc629e4640 1 --2- 172.21.15.100:0/1757568921 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fcc3c06c2b0 0x7fcc3c06e770 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.435+0000 7fcc629e4640 1 --2- 172.21.15.100:0/1757568921 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcc5c153a20 0x7fcc5c153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.435+0000 7fcc629e4640 1 --2- 172.21.15.100:0/1757568921 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc5c1f5350 0x7fcc5c0ca790 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.435+0000 7fcc629e4640 1 --2- 172.21.15.100:0/1757568921 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcc480039b0 0x7fcc5c148c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.435+0000 7fcc629e4640 1 -- 172.21.15.100:0/1757568921 >> 172.21.15.100:0/1757568921 conn(0x7fcc5c06e260 msgr2=0x7fcc5c06bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:27.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.436+0000 7fcc629e4640 1 -- 172.21.15.100:0/1757568921 shutdown_connections 2024-09-15T21:46:27.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.436+0000 7fcc629e4640 1 -- 172.21.15.100:0/1757568921 wait complete. 2024-09-15T21:46:27.449 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-15T21:46:27.450 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T21:46:27.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.549+0000 7fa225331640 1 Processor -- start 2024-09-15T21:46:27.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.549+0000 7fa225331640 1 -- start start 2024-09-15T21:46:27.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.550+0000 7fa225331640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa220155440 0x7fa2200cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:27.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.550+0000 7fa225331640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa2200cc8f0 0x7fa2200cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:27.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.550+0000 7fa225331640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa22015e7a0 con 0x7fa2200cb7d0 2024-09-15T21:46:27.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.550+0000 7fa225331640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa22015b470 con 0x7fa2200cc8f0 2024-09-15T21:46:27.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.550+0000 7fa225331640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa220155820 con 0x7fa220155440 2024-09-15T21:46:27.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.550+0000 7fa21f7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa2200cc8f0 0x7fa2200cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:27.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.550+0000 7fa21f7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa2200cc8f0 0x7fa2200cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43616/0 (socket says 172.21.15.100:43616) 2024-09-15T21:46:27.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.550+0000 7fa21f7fe640 1 -- 172.21.15.100:0/3399770837 learned_addr learned my addr 172.21.15.100:0/3399770837 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:27.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.550+0000 7fa21effd640 1 --2- 172.21.15.100:0/3399770837 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa220155440 0x7fa2200cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:27.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.550+0000 7fa21f7fe640 1 -- 172.21.15.100:0/3399770837 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa220155440 msgr2=0x7fa2200cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:27.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.550+0000 7fa21f7fe640 1 --2- 172.21.15.100:0/3399770837 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa220155440 0x7fa2200cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.550+0000 7fa21f7fe640 1 -- 172.21.15.100:0/3399770837 >> v1:172.21.15.122:6789/0 conn(0x7fa2200cb7d0 legacy=0x7fa2200cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:27.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.550+0000 7fa21f7fe640 1 -- 172.21.15.100:0/3399770837 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22015ea40 con 0x7fa2200cc8f0 2024-09-15T21:46:27.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.551+0000 7fa21f7fe640 1 --2- 172.21.15.100:0/3399770837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2200cc8f0 0x7fa2200cd960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa218009e50 tx=0x7fa21802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c26e9e86d0668c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:27.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.551+0000 7fa21dffb640 1 -- 172.21.15.100:0/3399770837 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218036070 con 0x7fa2200cc8f0 2024-09-15T21:46:27.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.551+0000 7fa21dffb640 1 -- 172.21.15.100:0/3399770837 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa218031600 con 0x7fa2200cc8f0 2024-09-15T21:46:27.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.551+0000 7fa21dffb640 1 -- 172.21.15.100:0/3399770837 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2180319d0 con 0x7fa2200cc8f0 2024-09-15T21:46:27.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.551+0000 7fa225331640 1 -- 172.21.15.100:0/3399770837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2200cc8f0 msgr2=0x7fa2200cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:27.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.551+0000 7fa225331640 1 --2- 172.21.15.100:0/3399770837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2200cc8f0 0x7fa2200cd960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa218009e50 tx=0x7fa21802a710 comp rx=0 tx=0).stop 2024-09-15T21:46:27.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.552+0000 7fa225331640 1 -- 172.21.15.100:0/3399770837 shutdown_connections 2024-09-15T21:46:27.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.552+0000 7fa225331640 1 --2- 172.21.15.100:0/3399770837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2200cc8f0 0x7fa2200cd960 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.552+0000 7fa225331640 1 --2- 172.21.15.100:0/3399770837 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa220155440 0x7fa2200cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.552+0000 7fa225331640 1 -- 172.21.15.100:0/3399770837 >> 172.21.15.100:0/3399770837 conn(0x7fa2200680d0 msgr2=0x7fa22006d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:27.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.552+0000 7fa225331640 1 -- 172.21.15.100:0/3399770837 shutdown_connections 2024-09-15T21:46:27.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.552+0000 7fa225331640 1 -- 172.21.15.100:0/3399770837 wait complete. 2024-09-15T21:46:27.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.553+0000 7fa225331640 1 Processor -- start 2024-09-15T21:46:27.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.553+0000 7fa225331640 1 -- start start 2024-09-15T21:46:27.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.553+0000 7fa225331640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2200cb7d0 0x7fa220149500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:27.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.553+0000 7fa225331640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2200cc8f0 0x7fa2200cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:27.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.553+0000 7fa225331640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa220155440 0x7fa2201442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:27.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.554+0000 7fa225331640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa22015f2e0 con 0x7fa2200cc8f0 2024-09-15T21:46:27.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.554+0000 7fa225331640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa22015f000 con 0x7fa2200cb7d0 2024-09-15T21:46:27.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.554+0000 7fa225331640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa22015b070 con 0x7fa220155440 2024-09-15T21:46:27.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.554+0000 7fa21e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2200cc8f0 0x7fa2200cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:27.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.554+0000 7fa21e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2200cc8f0 0x7fa2200cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43626/0 (socket says 172.21.15.100:43626) 2024-09-15T21:46:27.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.554+0000 7fa21e7fc640 1 -- 172.21.15.100:0/2891764896 learned_addr learned my addr 172.21.15.100:0/2891764896 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:27.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.554+0000 7fa21f7fe640 1 --2- 172.21.15.100:0/2891764896 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa220155440 0x7fa2201442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:27.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.554+0000 7fa21effd640 1 --2- 172.21.15.100:0/2891764896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2200cb7d0 0x7fa220149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:27.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.554+0000 7fa21e7fc640 1 -- 172.21.15.100:0/2891764896 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa220155440 msgr2=0x7fa2201442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:27.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.554+0000 7fa21e7fc640 1 --2- 172.21.15.100:0/2891764896 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa220155440 0x7fa2201442e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.554+0000 7fa21e7fc640 1 -- 172.21.15.100:0/2891764896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2200cb7d0 msgr2=0x7fa220149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:27.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.554+0000 7fa21e7fc640 1 --2- 172.21.15.100:0/2891764896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2200cb7d0 0x7fa220149500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.554+0000 7fa21e7fc640 1 -- 172.21.15.100:0/2891764896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2200c83a0 con 0x7fa2200cc8f0 2024-09-15T21:46:27.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.554+0000 7fa21e7fc640 1 --2- 172.21.15.100:0/2891764896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2200cc8f0 0x7fa2200cd670 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa21400c470 tx=0x7fa214001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:27.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.554+0000 7fa2077fe640 1 -- 172.21.15.100:0/2891764896 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2140052d0 con 0x7fa2200cc8f0 2024-09-15T21:46:27.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.554+0000 7fa2077fe640 1 -- 172.21.15.100:0/2891764896 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa214005470 con 0x7fa2200cc8f0 2024-09-15T21:46:27.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.554+0000 7fa225331640 1 -- 172.21.15.100:0/2891764896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2200c7de0 con 0x7fa2200cc8f0 2024-09-15T21:46:27.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.554+0000 7fa2077fe640 1 -- 172.21.15.100:0/2891764896 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa214007260 con 0x7fa2200cc8f0 2024-09-15T21:46:27.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.554+0000 7fa225331640 1 -- 172.21.15.100:0/2891764896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2201f4bd0 con 0x7fa2200cc8f0 2024-09-15T21:46:27.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.555+0000 7fa225331640 1 -- 172.21.15.100:0/2891764896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2200b3940 con 0x7fa2200cc8f0 2024-09-15T21:46:27.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.556+0000 7fa2077fe640 1 -- 172.21.15.100:0/2891764896 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa214007400 con 0x7fa2200cc8f0 2024-09-15T21:46:27.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.556+0000 7fa2077fe640 1 --2- 172.21.15.100:0/2891764896 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa20c06c2b0 0x7fa20c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:27.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.556+0000 7fa2077fe640 1 -- 172.21.15.100:0/2891764896 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa21408a7c0 con 0x7fa2200cc8f0 2024-09-15T21:46:27.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.556+0000 7fa21effd640 1 --2- 172.21.15.100:0/2891764896 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa20c06c2b0 0x7fa20c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:27.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.557+0000 7fa21effd640 1 --2- 172.21.15.100:0/2891764896 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa20c06c2b0 0x7fa20c06e770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa2100079a0 tx=0x7fa210008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:27.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.560+0000 7fa2077fe640 1 -- 172.21.15.100:0/2891764896 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa214058970 con 0x7fa2200cc8f0 2024-09-15T21:46:27.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.763+0000 7fa225331640 1 -- 172.21.15.100:0/2891764896 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa22014f4c0 con 0x7fa2200cc8f0 2024-09-15T21:46:27.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.764+0000 7fa2077fe640 1 -- 172.21.15.100:0/2891764896 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6870 (secure 0 0 0) 0x7fa21405cc70 con 0x7fa2200cc8f0 2024-09-15T21:46:27.766 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:27.766 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":8,"btime":"2024-09-15T21:46:27:432116+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2992633397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2992633397},{"type":"v1","addr":"172.21.15.122:6833","nonce":2992633397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/1809406468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1809406468},{"type":"v1","addr":"172.21.15.100:6835","nonce":1809406468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4288,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/371886422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":371886422},{"type":"v1","addr":"172.21.15.122:6837","nonce":371886422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6838/276075809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":276075809},{"type":"v1","addr":"172.21.15.100:6838","nonce":276075809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4291,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6836/2947743626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6835","nonce":2947743626},{"type":"v1","addr":"172.21.15.122:6836","nonce":2947743626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T21:46:22.753977+0000","modified":"2024-09-15T21:46:26.760117+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"e","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.100:6839/602064326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6837","nonce":602064326},{"type":"v1","addr":"172.21.15.100:6839","nonce":602064326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-15T21:46:27.766 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 8 2024-09-15T21:46:27.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.766+0000 7fa225331640 1 -- 172.21.15.100:0/2891764896 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa20c06c2b0 msgr2=0x7fa20c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:27.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.766+0000 7fa225331640 1 --2- 172.21.15.100:0/2891764896 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa20c06c2b0 0x7fa20c06e770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa2100079a0 tx=0x7fa210008040 comp rx=0 tx=0).stop 2024-09-15T21:46:27.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.766+0000 7fa225331640 1 -- 172.21.15.100:0/2891764896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2200cc8f0 msgr2=0x7fa2200cd670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:27.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.766+0000 7fa225331640 1 --2- 172.21.15.100:0/2891764896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2200cc8f0 0x7fa2200cd670 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa21400c470 tx=0x7fa214001e70 comp rx=0 tx=0).stop 2024-09-15T21:46:27.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.766+0000 7fa225331640 1 -- 172.21.15.100:0/2891764896 shutdown_connections 2024-09-15T21:46:27.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.766+0000 7fa225331640 1 --2- 172.21.15.100:0/2891764896 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa20c06c2b0 0x7fa20c06e770 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.766+0000 7fa225331640 1 --2- 172.21.15.100:0/2891764896 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa220155440 0x7fa2201442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.766+0000 7fa225331640 1 --2- 172.21.15.100:0/2891764896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2200cc8f0 0x7fa2200cd670 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.766+0000 7fa225331640 1 --2- 172.21.15.100:0/2891764896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2200cb7d0 0x7fa220149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.766+0000 7fa225331640 1 -- 172.21.15.100:0/2891764896 >> 172.21.15.100:0/2891764896 conn(0x7fa2200680d0 msgr2=0x7fa220159880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:27.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.766+0000 7fa225331640 1 -- 172.21.15.100:0/2891764896 shutdown_connections 2024-09-15T21:46:27.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.766+0000 7fa225331640 1 -- 172.21.15.100:0/2891764896 wait complete. 2024-09-15T21:46:27.779 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:46:27.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.876+0000 7f852daa4640 1 Processor -- start 2024-09-15T21:46:27.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.876+0000 7f852daa4640 1 -- start start 2024-09-15T21:46:27.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.877+0000 7f852daa4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8528155440 0x7f85280cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:27.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.877+0000 7f852daa4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85280cb7d0 0x7f85280cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:27.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.877+0000 7f852daa4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f852806d0e0 0x7f85280cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:27.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.877+0000 7f852daa4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f852815b470 con 0x7f8528155440 2024-09-15T21:46:27.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.877+0000 7f852daa4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f8528155820 con 0x7f852806d0e0 2024-09-15T21:46:27.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.877+0000 7f852daa4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f852815e7a0 con 0x7f85280cb7d0 2024-09-15T21:46:27.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.877+0000 7f85277fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8528155440 0x7f85280cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:27.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.877+0000 7f85277fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8528155440 0x7f85280cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43628/0 (socket says 172.21.15.100:43628) 2024-09-15T21:46:27.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.877+0000 7f85277fe640 1 -- 172.21.15.100:0/2811983779 learned_addr learned my addr 172.21.15.100:0/2811983779 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:27.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.877+0000 7f8527fff640 1 --2- 172.21.15.100:0/2811983779 >> v2:172.21.15.122:3300/0 conn(0x7f852806d0e0 0x7f85280cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:27.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.877+0000 7f8526ffd640 1 --2- 172.21.15.100:0/2811983779 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85280cb7d0 0x7f85280cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:27.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.877+0000 7f85277fe640 1 -- 172.21.15.100:0/2811983779 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85280cb7d0 msgr2=0x7f85280cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:27.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.877+0000 7f85277fe640 1 --2- 172.21.15.100:0/2811983779 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85280cb7d0 0x7f85280cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.877+0000 7f85277fe640 1 -- 172.21.15.100:0/2811983779 >> v2:172.21.15.122:3300/0 conn(0x7f852806d0e0 msgr2=0x7f85280cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:27.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.877+0000 7f85277fe640 1 --2- 172.21.15.100:0/2811983779 >> v2:172.21.15.122:3300/0 conn(0x7f852806d0e0 0x7f85280cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.877+0000 7f85277fe640 1 -- 172.21.15.100:0/2811983779 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f852815ea40 con 0x7f8528155440 2024-09-15T21:46:27.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.878+0000 7f85277fe640 1 --2- 172.21.15.100:0/2811983779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8528155440 0x7f85280cd420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f85280ce930 tx=0x7f851802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412b8a5b9e641a65 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:27.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.878+0000 7f85267fc640 1 -- 172.21.15.100:0/2811983779 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518037070 con 0x7f8528155440 2024-09-15T21:46:27.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.878+0000 7f85267fc640 1 -- 172.21.15.100:0/2811983779 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8518032440 con 0x7f8528155440 2024-09-15T21:46:27.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.878+0000 7f85267fc640 1 -- 172.21.15.100:0/2811983779 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518032810 con 0x7f8528155440 2024-09-15T21:46:27.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.878+0000 7f852daa4640 1 -- 172.21.15.100:0/2811983779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8528155440 msgr2=0x7f85280cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:27.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.878+0000 7f852daa4640 1 --2- 172.21.15.100:0/2811983779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8528155440 0x7f85280cd420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f85280ce930 tx=0x7f851802a180 comp rx=0 tx=0).stop 2024-09-15T21:46:27.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.878+0000 7f852daa4640 1 -- 172.21.15.100:0/2811983779 shutdown_connections 2024-09-15T21:46:27.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.878+0000 7f852daa4640 1 --2- 172.21.15.100:0/2811983779 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85280cb7d0 0x7f85280cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.878+0000 7f852daa4640 1 --2- 172.21.15.100:0/2811983779 >> v2:172.21.15.122:3300/0 conn(0x7f852806d0e0 0x7f85280cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.878+0000 7f852daa4640 1 --2- 172.21.15.100:0/2811983779 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8528155440 0x7f85280cd420 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.878+0000 7f852daa4640 1 -- 172.21.15.100:0/2811983779 >> 172.21.15.100:0/2811983779 conn(0x7f85280680d0 msgr2=0x7f852806d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:27.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.878+0000 7f852daa4640 1 -- 172.21.15.100:0/2811983779 shutdown_connections 2024-09-15T21:46:27.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.878+0000 7f852daa4640 1 -- 172.21.15.100:0/2811983779 wait complete. 2024-09-15T21:46:27.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.879+0000 7f852daa4640 1 Processor -- start 2024-09-15T21:46:27.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.880+0000 7f852daa4640 1 -- start start 2024-09-15T21:46:27.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.880+0000 7f852daa4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f852806d0e0 0x7f85281e4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:27.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.880+0000 7f852daa4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85280cb7d0 0x7f85280cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:27.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.880+0000 7f852daa4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8528155440 0x7f8528150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:27.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.880+0000 7f852daa4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f852815f2e0 con 0x7f85280cb7d0 2024-09-15T21:46:27.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.880+0000 7f852daa4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f852815f000 con 0x7f852806d0e0 2024-09-15T21:46:27.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.880+0000 7f852daa4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f852815b070 con 0x7f8528155440 2024-09-15T21:46:27.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.880+0000 7f8526ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85280cb7d0 0x7f85280cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:27.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.880+0000 7f8526ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85280cb7d0 0x7f85280cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43642/0 (socket says 172.21.15.100:43642) 2024-09-15T21:46:27.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.880+0000 7f8526ffd640 1 -- 172.21.15.100:0/3924078555 learned_addr learned my addr 172.21.15.100:0/3924078555 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:27.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.881+0000 7f8527fff640 1 --2- 172.21.15.100:0/3924078555 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8528155440 0x7f8528150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:27.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.881+0000 7f85277fe640 1 --2- 172.21.15.100:0/3924078555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f852806d0e0 0x7f85281e4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:27.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.881+0000 7f8526ffd640 1 -- 172.21.15.100:0/3924078555 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8528155440 msgr2=0x7f8528150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:27.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.881+0000 7f8526ffd640 1 --2- 172.21.15.100:0/3924078555 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8528155440 0x7f8528150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.881+0000 7f8526ffd640 1 -- 172.21.15.100:0/3924078555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f852806d0e0 msgr2=0x7f85281e4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:27.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.881+0000 7f8526ffd640 1 --2- 172.21.15.100:0/3924078555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f852806d0e0 0x7f85281e4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:27.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.881+0000 7f8526ffd640 1 -- 172.21.15.100:0/3924078555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85281f1350 con 0x7f85280cb7d0 2024-09-15T21:46:27.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.881+0000 7f8526ffd640 1 --2- 172.21.15.100:0/3924078555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85280cb7d0 0x7f85280cfb70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f851c00d5f0 tx=0x7f851c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:27.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.881+0000 7f850ffff640 1 -- 172.21.15.100:0/3924078555 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c0125f0 con 0x7f85280cb7d0 2024-09-15T21:46:27.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.881+0000 7f850ffff640 1 -- 172.21.15.100:0/3924078555 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f851c019070 con 0x7f85280cb7d0 2024-09-15T21:46:27.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.881+0000 7f852daa4640 1 -- 172.21.15.100:0/3924078555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f852814f6d0 con 0x7f85280cb7d0 2024-09-15T21:46:27.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.881+0000 7f850ffff640 1 -- 172.21.15.100:0/3924078555 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c0129d0 con 0x7f85280cb7d0 2024-09-15T21:46:27.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.881+0000 7f852daa4640 1 -- 172.21.15.100:0/3924078555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f852814fc90 con 0x7f85280cb7d0 2024-09-15T21:46:27.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.882+0000 7f852daa4640 1 -- 172.21.15.100:0/3924078555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85281f20d0 con 0x7f85280cb7d0 2024-09-15T21:46:27.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.883+0000 7f850ffff640 1 -- 172.21.15.100:0/3924078555 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f851c012b70 con 0x7f85280cb7d0 2024-09-15T21:46:27.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.883+0000 7f850ffff640 1 --2- 172.21.15.100:0/3924078555 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f851406c2b0 0x7f851406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:27.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.883+0000 7f85277fe640 1 --2- 172.21.15.100:0/3924078555 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f851406c2b0 0x7f851406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:27.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.883+0000 7f850ffff640 1 -- 172.21.15.100:0/3924078555 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f851c08b610 con 0x7f85280cb7d0 2024-09-15T21:46:27.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.883+0000 7f85277fe640 1 --2- 172.21.15.100:0/3924078555 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f851406c2b0 0x7f851406e770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8518034000 tx=0x7f851802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:27.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:27.887+0000 7f850ffff640 1 -- 172.21.15.100:0/3924078555 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f851c0597c0 con 0x7f85280cb7d0 2024-09-15T21:46:28.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.054+0000 7f852daa4640 1 -- 172.21.15.100:0/3924078555 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85280b3940 con 0x7f85280cb7d0 2024-09-15T21:46:28.057 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.055+0000 7f850ffff640 1 -- 172.21.15.100:0/3924078555 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16737 (secure 0 0 0) 0x7f851c05dac0 con 0x7f85280cb7d0 2024-09-15T21:46:28.058 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:28.058 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"30e2fff4-2171-4531-8806-0e03b63f494f","created":"2024-09-15T21:45:59.746897+0000","modified":"2024-09-15T21:46:24.409383+0000","last_up_change":"2024-09-15T21:46:13.224709+0000","last_in_change":"2024-09-15T21:46:01.812129+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:46:11.837603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T21:46:20.577982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T21:46:21.729231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5887900b-522d-476f-9704-cc2c5ee87a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6819","nonce":3920435546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6821","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6822","nonce":3920435546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6829","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6831","nonce":3920435546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6825","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6827","nonce":3920435546}]},"public_addr":"172.21.15.100:6819/3920435546","cluster_addr":"172.21.15.100:6822/3920435546","heartbeat_back_addr":"172.21.15.100:6831/3920435546","heartbeat_front_addr":"172.21.15.100:6827/3920435546","state":["exists","up"]},{"osd":1,"uuid":"adc17993-5c50-4251-85e1-4c7ac62e25e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6801","nonce":3802679972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6803","nonce":3802679972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6807","nonce":3802679972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6805","nonce":3802679972}]},"public_addr":"172.21.15.100:6801/3802679972","cluster_addr":"172.21.15.100:6803/3802679972","heartbeat_back_addr":"172.21.15.100:6807/3802679972","heartbeat_front_addr":"172.21.15.100:6805/3802679972","state":["exists","up"]},{"osd":2,"uuid":"8de3b7d5-1980-4a3b-a934-264f867c7c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6823","nonce":2316787974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6826","nonce":2316787974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6833","nonce":2316787974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6830","nonce":2316787974}]},"public_addr":"172.21.15.100:6823/2316787974","cluster_addr":"172.21.15.100:6826/2316787974","heartbeat_back_addr":"172.21.15.100:6833/2316787974","heartbeat_front_addr":"172.21.15.100:6830/2316787974","state":["exists","up"]},{"osd":3,"uuid":"93dbcd1c-6a6f-49cf-bc94-e64b883de5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6811","nonce":1287562741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6813","nonce":1287562741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6817","nonce":1287562741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6815","nonce":1287562741}]},"public_addr":"172.21.15.100:6811/1287562741","cluster_addr":"172.21.15.100:6813/1287562741","heartbeat_back_addr":"172.21.15.100:6817/1287562741","heartbeat_front_addr":"172.21.15.100:6815/1287562741","state":["exists","up"]},{"osd":4,"uuid":"7caee2a0-bab4-45a9-8109-724948c863c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6801","nonce":4045445512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6803","nonce":4045445512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6807","nonce":4045445512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6805","nonce":4045445512}]},"public_addr":"172.21.15.122:6801/4045445512","cluster_addr":"172.21.15.122:6803/4045445512","heartbeat_back_addr":"172.21.15.122:6807/4045445512","heartbeat_front_addr":"172.21.15.122:6805/4045445512","state":["exists","up"]},{"osd":5,"uuid":"6e64070a-671a-444b-a061-b3051a29b121","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6825","nonce":2400668532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6827","nonce":2400668532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6831","nonce":2400668532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6829","nonce":2400668532}]},"public_addr":"172.21.15.122:6825/2400668532","cluster_addr":"172.21.15.122:6827/2400668532","heartbeat_back_addr":"172.21.15.122:6831/2400668532","heartbeat_front_addr":"172.21.15.122:6829/2400668532","state":["exists","up"]},{"osd":6,"uuid":"47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6809","nonce":1437167311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6811","nonce":1437167311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6815","nonce":1437167311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6813","nonce":1437167311}]},"public_addr":"172.21.15.122:6809/1437167311","cluster_addr":"172.21.15.122:6811/1437167311","heartbeat_back_addr":"172.21.15.122:6815/1437167311","heartbeat_front_addr":"172.21.15.122:6813/1437167311","state":["exists","up"]},{"osd":7,"uuid":"9a916b6d-af82-486f-af40-4a7291c900fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6817","nonce":2431377162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6819","nonce":2431377162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6823","nonce":2431377162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6821","nonce":2431377162}]},"public_addr":"172.21.15.122:6817/2431377162","cluster_addr":"172.21.15.122:6819/2431377162","heartbeat_back_addr":"172.21.15.122:6823/2431377162","heartbeat_front_addr":"172.21.15.122:6821/2431377162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.168771+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.886614+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.154129+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.570531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.926773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.146320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.409137+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.373359+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:46:28.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.057+0000 7f852daa4640 1 -- 172.21.15.100:0/3924078555 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f851406c2b0 msgr2=0x7f851406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:28.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.057+0000 7f852daa4640 1 --2- 172.21.15.100:0/3924078555 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f851406c2b0 0x7f851406e770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8518034000 tx=0x7f851802e000 comp rx=0 tx=0).stop 2024-09-15T21:46:28.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.057+0000 7f852daa4640 1 -- 172.21.15.100:0/3924078555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85280cb7d0 msgr2=0x7f85280cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:28.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.057+0000 7f852daa4640 1 --2- 172.21.15.100:0/3924078555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85280cb7d0 0x7f85280cfb70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f851c00d5f0 tx=0x7f851c00dab0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.057+0000 7f852daa4640 1 -- 172.21.15.100:0/3924078555 shutdown_connections 2024-09-15T21:46:28.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.057+0000 7f852daa4640 1 --2- 172.21.15.100:0/3924078555 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f851406c2b0 0x7f851406e770 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.057+0000 7f852daa4640 1 --2- 172.21.15.100:0/3924078555 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8528155440 0x7f8528150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.057+0000 7f852daa4640 1 --2- 172.21.15.100:0/3924078555 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f85280cb7d0 0x7f85280cfb70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.057+0000 7f852daa4640 1 --2- 172.21.15.100:0/3924078555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f852806d0e0 0x7f85281e4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.057+0000 7f852daa4640 1 -- 172.21.15.100:0/3924078555 >> 172.21.15.100:0/3924078555 conn(0x7f85280680d0 msgr2=0x7f852806d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:28.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.057+0000 7f852daa4640 1 -- 172.21.15.100:0/3924078555 shutdown_connections 2024-09-15T21:46:28.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.057+0000 7f852daa4640 1 -- 172.21.15.100:0/3924078555 wait complete. 2024-09-15T21:46:28.070 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:46:28.070 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:46:28.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.165+0000 7fbcd105c640 1 Processor -- start 2024-09-15T21:46:28.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.166+0000 7fbcd105c640 1 -- start start 2024-09-15T21:46:28.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.166+0000 7fbcd105c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbccc155210 0x7fbccc0ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.166+0000 7fbcd105c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fbccc1555f0 0x7fbccc0cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.166+0000 7fbcd105c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbccc15e7e0 con 0x7fbccc0c48e0 2024-09-15T21:46:28.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.166+0000 7fbcd105c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fbccc15f640 con 0x7fbccc155210 2024-09-15T21:46:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.166+0000 7fbcd105c640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fbccc15eac0 con 0x7fbccc1555f0 2024-09-15T21:46:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.166+0000 7fbccad9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbccc155210 0x7fbccc0ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.166+0000 7fbccad9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbccc155210 0x7fbccc0ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43644/0 (socket says 172.21.15.100:43644) 2024-09-15T21:46:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.166+0000 7fbccad9d640 1 -- 172.21.15.100:0/2208444681 learned_addr learned my addr 172.21.15.100:0/2208444681 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.167+0000 7fbcca59c640 1 --2- 172.21.15.100:0/2208444681 >> v2:172.21.15.122:3300/0 conn(0x7fbccc1555f0 0x7fbccc0cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.167+0000 7fbccad9d640 1 -- 172.21.15.100:0/2208444681 >> v2:172.21.15.122:3300/0 conn(0x7fbccc1555f0 msgr2=0x7fbccc0cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.167+0000 7fbccad9d640 1 --2- 172.21.15.100:0/2208444681 >> v2:172.21.15.122:3300/0 conn(0x7fbccc1555f0 0x7fbccc0cefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.167+0000 7fbccad9d640 1 -- 172.21.15.100:0/2208444681 >> v1:172.21.15.122:6789/0 conn(0x7fbccc0c48e0 legacy=0x7fbccc073840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.167+0000 7fbccad9d640 1 -- 172.21.15.100:0/2208444681 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbccc15eda0 con 0x7fbccc155210 2024-09-15T21:46:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.167+0000 7fbccad9d640 1 --2- 172.21.15.100:0/2208444681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbccc155210 0x7fbccc0ceaa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbccc070530 tx=0x7fbcbc02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d32df6fd93792c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.168+0000 7fbcc9d9b640 1 -- 172.21.15.100:0/2208444681 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc037070 con 0x7fbccc155210 2024-09-15T21:46:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.168+0000 7fbcc9d9b640 1 -- 172.21.15.100:0/2208444681 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcbc032440 con 0x7fbccc155210 2024-09-15T21:46:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.168+0000 7fbcc9d9b640 1 -- 172.21.15.100:0/2208444681 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc032810 con 0x7fbccc155210 2024-09-15T21:46:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.168+0000 7fbcd105c640 1 -- 172.21.15.100:0/2208444681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbccc155210 msgr2=0x7fbccc0ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.168+0000 7fbcd105c640 1 --2- 172.21.15.100:0/2208444681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbccc155210 0x7fbccc0ceaa0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbccc070530 tx=0x7fbcbc02a2c0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.168+0000 7fbcd105c640 1 -- 172.21.15.100:0/2208444681 shutdown_connections 2024-09-15T21:46:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.168+0000 7fbcd105c640 1 --2- 172.21.15.100:0/2208444681 >> v2:172.21.15.122:3300/0 conn(0x7fbccc1555f0 0x7fbccc0cefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.168+0000 7fbcd105c640 1 --2- 172.21.15.100:0/2208444681 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbccc155210 0x7fbccc0ceaa0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.168+0000 7fbcd105c640 1 -- 172.21.15.100:0/2208444681 >> 172.21.15.100:0/2208444681 conn(0x7fbccc06c210 msgr2=0x7fbccc067980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.168+0000 7fbcd105c640 1 -- 172.21.15.100:0/2208444681 shutdown_connections 2024-09-15T21:46:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.168+0000 7fbcd105c640 1 -- 172.21.15.100:0/2208444681 wait complete. 2024-09-15T21:46:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.170+0000 7fbcd105c640 1 Processor -- start 2024-09-15T21:46:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.170+0000 7fbcd105c640 1 -- start start 2024-09-15T21:46:28.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.170+0000 7fbcd105c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbccc0c48e0 0x7fbccc1504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.170+0000 7fbcd105c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc155210 0x7fbccc14baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.170+0000 7fbcd105c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbccc1555f0 0x7fbccc15c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.170+0000 7fbcd105c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc073f50 con 0x7fbccc1555f0 2024-09-15T21:46:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.170+0000 7fbcd105c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc15f360 con 0x7fbccc155210 2024-09-15T21:46:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.170+0000 7fbcd105c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbccc0740d0 con 0x7fbccc0c48e0 2024-09-15T21:46:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.170+0000 7fbccb59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbccc1555f0 0x7fbccc15c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.170+0000 7fbccb59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbccc1555f0 0x7fbccc15c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43660/0 (socket says 172.21.15.100:43660) 2024-09-15T21:46:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.170+0000 7fbccb59e640 1 -- 172.21.15.100:0/484690783 learned_addr learned my addr 172.21.15.100:0/484690783 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.170+0000 7fbcca59c640 1 --2- 172.21.15.100:0/484690783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc155210 0x7fbccc14baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.170+0000 7fbccad9d640 1 --2- 172.21.15.100:0/484690783 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbccc0c48e0 0x7fbccc1504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.171+0000 7fbccb59e640 1 -- 172.21.15.100:0/484690783 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbccc0c48e0 msgr2=0x7fbccc1504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.171+0000 7fbccb59e640 1 --2- 172.21.15.100:0/484690783 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbccc0c48e0 0x7fbccc1504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.171+0000 7fbccb59e640 1 -- 172.21.15.100:0/484690783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc155210 msgr2=0x7fbccc14baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.171+0000 7fbccb59e640 1 --2- 172.21.15.100:0/484690783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc155210 0x7fbccc14baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.171+0000 7fbccb59e640 1 -- 172.21.15.100:0/484690783 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbccc15c790 con 0x7fbccc1555f0 2024-09-15T21:46:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.171+0000 7fbccb59e640 1 --2- 172.21.15.100:0/484690783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbccc1555f0 0x7fbccc15c250 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400c780 tx=0x7fbcc400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.171+0000 7fbcc37fe640 1 -- 172.21.15.100:0/484690783 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4004b60 con 0x7fbccc1555f0 2024-09-15T21:46:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.171+0000 7fbcc37fe640 1 -- 172.21.15.100:0/484690783 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc4004d00 con 0x7fbccc1555f0 2024-09-15T21:46:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.171+0000 7fbcd105c640 1 -- 172.21.15.100:0/484690783 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbccc1f1250 con 0x7fbccc1555f0 2024-09-15T21:46:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.171+0000 7fbcc37fe640 1 -- 172.21.15.100:0/484690783 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc400aa90 con 0x7fbccc1555f0 2024-09-15T21:46:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.171+0000 7fbcd105c640 1 -- 172.21.15.100:0/484690783 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbccc14fcd0 con 0x7fbccc1555f0 2024-09-15T21:46:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.172+0000 7fbcd105c640 1 -- 172.21.15.100:0/484690783 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbccc073840 con 0x7fbccc1555f0 2024-09-15T21:46:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.173+0000 7fbcc37fe640 1 -- 172.21.15.100:0/484690783 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbcc400ac30 con 0x7fbccc1555f0 2024-09-15T21:46:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.173+0000 7fbcc37fe640 1 --2- 172.21.15.100:0/484690783 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbcac06c2b0 0x7fbcac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.173+0000 7fbccad9d640 1 --2- 172.21.15.100:0/484690783 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbcac06c2b0 0x7fbcac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.173+0000 7fbcc37fe640 1 -- 172.21.15.100:0/484690783 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbcc408b7e0 con 0x7fbccc1555f0 2024-09-15T21:46:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.173+0000 7fbccad9d640 1 --2- 172.21.15.100:0/484690783 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbcac06c2b0 0x7fbcac06e770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc034000 tx=0x7fbcbc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:28.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.177+0000 7fbcc37fe640 1 -- 172.21.15.100:0/484690783 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbcc4059990 con 0x7fbccc1555f0 2024-09-15T21:46:28.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.339+0000 7fbcd105c640 1 -- 172.21.15.100:0/484690783 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbccc1592f0 con 0x7fbcac06c2b0 2024-09-15T21:46:28.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.345+0000 7fbcc37fe640 1 -- 172.21.15.100:0/484690783 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248805 (secure 0 0 0) 0x7fbccc1592f0 con 0x7fbcac06c2b0 2024-09-15T21:46:28.351 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:28.356 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-15T21:46:28.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.349+0000 7fbcd105c640 1 -- 172.21.15.100:0/484690783 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbcac06c2b0 msgr2=0x7fbcac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:28.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.349+0000 7fbcd105c640 1 --2- 172.21.15.100:0/484690783 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbcac06c2b0 0x7fbcac06e770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc034000 tx=0x7fbcbc02e000 comp rx=0 tx=0).stop 2024-09-15T21:46:28.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.349+0000 7fbcd105c640 1 -- 172.21.15.100:0/484690783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbccc1555f0 msgr2=0x7fbccc15c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:28.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.349+0000 7fbcd105c640 1 --2- 172.21.15.100:0/484690783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbccc1555f0 0x7fbccc15c250 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400c780 tx=0x7fbcc400cc40 comp rx=0 tx=0).stop 2024-09-15T21:46:28.362 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.349+0000 7fbcd105c640 1 -- 172.21.15.100:0/484690783 shutdown_connections 2024-09-15T21:46:28.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.349+0000 7fbcd105c640 1 --2- 172.21.15.100:0/484690783 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbcac06c2b0 0x7fbcac06e770 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.349+0000 7fbcd105c640 1 --2- 172.21.15.100:0/484690783 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbccc155210 0x7fbccc14baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.349+0000 7fbcd105c640 1 --2- 172.21.15.100:0/484690783 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbccc1555f0 0x7fbccc15c250 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.349+0000 7fbcd105c640 1 --2- 172.21.15.100:0/484690783 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbccc0c48e0 0x7fbccc1504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.349+0000 7fbcd105c640 1 -- 172.21.15.100:0/484690783 >> 172.21.15.100:0/484690783 conn(0x7fbccc06c210 msgr2=0x7fbccc15b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:28.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.349+0000 7fbcd105c640 1 -- 172.21.15.100:0/484690783 shutdown_connections 2024-09-15T21:46:28.363 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.349+0000 7fbcd105c640 1 -- 172.21.15.100:0/484690783 wait complete. 2024-09-15T21:46:28.363 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-15T21:46:27.804526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218908,"kb_used_data":2684,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755812,"statfs":{"total":773094113280,"available":772869951488,"internally_reserved":0,"allocated":2748416,"data_stored":1686550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000848"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.421348+0000","last_change":"2024-09-15T21:46:22.421348+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.421348+0000","last_undegraded":"2024-09-15T21:46:22.421348+0000","last_fullsized":"2024-09-15T21:46:22.421348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808513+0000","last_change":"2024-09-15T21:46:23.418716+0000","last_active":"2024-09-15T21:46:24.808513+0000","last_peered":"2024-09-15T21:46:24.808513+0000","last_clean":"2024-09-15T21:46:24.808513+0000","last_became_active":"2024-09-15T21:46:23.418471+0000","last_became_peered":"2024-09-15T21:46:23.418471+0000","last_unstale":"2024-09-15T21:46:24.808513+0000","last_undegraded":"2024-09-15T21:46:24.808513+0000","last_fullsized":"2024-09-15T21:46:24.808513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:49.599801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431027+0000","last_change":"2024-09-15T21:46:23.426404+0000","last_active":"2024-09-15T21:46:23.431027+0000","last_peered":"2024-09-15T21:46:23.431027+0000","last_clean":"2024-09-15T21:46:23.431027+0000","last_became_active":"2024-09-15T21:46:23.426146+0000","last_became_peered":"2024-09-15T21:46:23.426146+0000","last_unstale":"2024-09-15T21:46:23.431027+0000","last_undegraded":"2024-09-15T21:46:23.431027+0000","last_fullsized":"2024-09-15T21:46:23.431027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:40.061508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419990+0000","last_change":"2024-09-15T21:46:22.419990+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419990+0000","last_undegraded":"2024-09-15T21:46:22.419990+0000","last_fullsized":"2024-09-15T21:46:22.419990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418695+0000","last_change":"2024-09-15T21:46:22.418695+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418695+0000","last_undegraded":"2024-09-15T21:46:22.418695+0000","last_fullsized":"2024-09-15T21:46:22.418695+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418843+0000","last_change":"2024-09-15T21:46:22.418843+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418843+0000","last_undegraded":"2024-09-15T21:46:22.418843+0000","last_fullsized":"2024-09-15T21:46:22.418843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419832+0000","last_change":"2024-09-15T21:46:22.419832+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419832+0000","last_undegraded":"2024-09-15T21:46:22.419832+0000","last_fullsized":"2024-09-15T21:46:22.419832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.438915+0000","last_change":"2024-09-15T21:46:23.437143+0000","last_active":"2024-09-15T21:46:23.438915+0000","last_peered":"2024-09-15T21:46:23.438915+0000","last_clean":"2024-09-15T21:46:23.438915+0000","last_became_active":"2024-09-15T21:46:23.436864+0000","last_became_peered":"2024-09-15T21:46:23.436864+0000","last_unstale":"2024-09-15T21:46:23.438915+0000","last_undegraded":"2024-09-15T21:46:23.438915+0000","last_fullsized":"2024-09-15T21:46:23.438915+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:12:37.128024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.420920+0000","last_change":"2024-09-15T21:46:22.420920+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.420920+0000","last_undegraded":"2024-09-15T21:46:22.420920+0000","last_fullsized":"2024-09-15T21:46:22.420920+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.422403+0000","last_change":"2024-09-15T21:46:22.422403+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.422403+0000","last_undegraded":"2024-09-15T21:46:22.422403+0000","last_fullsized":"2024-09-15T21:46:22.422403+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418976+0000","last_change":"2024-09-15T21:46:22.418976+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418976+0000","last_undegraded":"2024-09-15T21:46:22.418976+0000","last_fullsized":"2024-09-15T21:46:22.418976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415544+0000","last_change":"2024-09-15T21:46:22.415544+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415544+0000","last_undegraded":"2024-09-15T21:46:22.415544+0000","last_fullsized":"2024-09-15T21:46:22.415544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431804+0000","last_change":"2024-09-15T21:46:23.427841+0000","last_active":"2024-09-15T21:46:23.431804+0000","last_peered":"2024-09-15T21:46:23.431804+0000","last_clean":"2024-09-15T21:46:23.431804+0000","last_became_active":"2024-09-15T21:46:23.427621+0000","last_became_peered":"2024-09-15T21:46:23.427621+0000","last_unstale":"2024-09-15T21:46:23.431804+0000","last_undegraded":"2024-09-15T21:46:23.431804+0000","last_fullsized":"2024-09-15T21:46:23.431804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:17:35.795757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.428658+0000","last_change":"2024-09-15T21:46:23.424707+0000","last_active":"2024-09-15T21:46:23.428658+0000","last_peered":"2024-09-15T21:46:23.428658+0000","last_clean":"2024-09-15T21:46:23.428658+0000","last_became_active":"2024-09-15T21:46:23.424447+0000","last_became_peered":"2024-09-15T21:46:23.424447+0000","last_unstale":"2024-09-15T21:46:23.428658+0000","last_undegraded":"2024-09-15T21:46:23.428658+0000","last_fullsized":"2024-09-15T21:46:23.428658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:28:21.800777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808321+0000","last_change":"2024-09-15T21:46:23.430297+0000","last_active":"2024-09-15T21:46:24.808321+0000","last_peered":"2024-09-15T21:46:24.808321+0000","last_clean":"2024-09-15T21:46:24.808321+0000","last_became_active":"2024-09-15T21:46:23.430070+0000","last_became_peered":"2024-09-15T21:46:23.430070+0000","last_unstale":"2024-09-15T21:46:24.808321+0000","last_undegraded":"2024-09-15T21:46:24.808321+0000","last_fullsized":"2024-09-15T21:46:24.808321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:03:19.615022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415614+0000","last_change":"2024-09-15T21:46:22.415614+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415614+0000","last_undegraded":"2024-09-15T21:46:22.415614+0000","last_fullsized":"2024-09-15T21:46:22.415614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415415+0000","last_change":"2024-09-15T21:46:22.415415+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415415+0000","last_undegraded":"2024-09-15T21:46:22.415415+0000","last_fullsized":"2024-09-15T21:46:22.415415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415507+0000","last_change":"2024-09-15T21:46:22.415507+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415507+0000","last_undegraded":"2024-09-15T21:46:22.415507+0000","last_fullsized":"2024-09-15T21:46:22.415507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418352+0000","last_change":"2024-09-15T21:46:22.418352+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418352+0000","last_undegraded":"2024-09-15T21:46:22.418352+0000","last_fullsized":"2024-09-15T21:46:22.418352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.420228+0000","last_change":"2024-09-15T21:46:22.420228+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.420228+0000","last_undegraded":"2024-09-15T21:46:22.420228+0000","last_fullsized":"2024-09-15T21:46:22.420228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433725+0000","last_change":"2024-09-15T21:46:23.428323+0000","last_active":"2024-09-15T21:46:23.433725+0000","last_peered":"2024-09-15T21:46:23.433725+0000","last_clean":"2024-09-15T21:46:23.433725+0000","last_became_active":"2024-09-15T21:46:23.428016+0000","last_became_peered":"2024-09-15T21:46:23.428016+0000","last_unstale":"2024-09-15T21:46:23.433725+0000","last_undegraded":"2024-09-15T21:46:23.433725+0000","last_fullsized":"2024-09-15T21:46:23.433725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:24:06.673523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418399+0000","last_change":"2024-09-15T21:46:22.418399+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418399+0000","last_undegraded":"2024-09-15T21:46:22.418399+0000","last_fullsized":"2024-09-15T21:46:22.418399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433661+0000","last_change":"2024-09-15T21:46:23.431527+0000","last_active":"2024-09-15T21:46:23.433661+0000","last_peered":"2024-09-15T21:46:23.433661+0000","last_clean":"2024-09-15T21:46:23.433661+0000","last_became_active":"2024-09-15T21:46:23.430719+0000","last_became_peered":"2024-09-15T21:46:23.430719+0000","last_unstale":"2024-09-15T21:46:23.433661+0000","last_undegraded":"2024-09-15T21:46:23.433661+0000","last_fullsized":"2024-09-15T21:46:23.433661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:25.362126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415180+0000","last_change":"2024-09-15T21:46:22.415180+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415180+0000","last_undegraded":"2024-09-15T21:46:22.415180+0000","last_fullsized":"2024-09-15T21:46:22.415180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.436122+0000","last_change":"2024-09-15T21:46:23.432646+0000","last_active":"2024-09-15T21:46:23.436122+0000","last_peered":"2024-09-15T21:46:23.436122+0000","last_clean":"2024-09-15T21:46:23.436122+0000","last_became_active":"2024-09-15T21:46:23.432317+0000","last_became_peered":"2024-09-15T21:46:23.432317+0000","last_unstale":"2024-09-15T21:46:23.436122+0000","last_undegraded":"2024-09-15T21:46:23.436122+0000","last_fullsized":"2024-09-15T21:46:23.436122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:34.359059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.421958+0000","last_change":"2024-09-15T21:46:22.421958+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.421958+0000","last_undegraded":"2024-09-15T21:46:22.421958+0000","last_fullsized":"2024-09-15T21:46:22.421958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808369+0000","last_change":"2024-09-15T21:46:23.421749+0000","last_active":"2024-09-15T21:46:24.808369+0000","last_peered":"2024-09-15T21:46:24.808369+0000","last_clean":"2024-09-15T21:46:24.808369+0000","last_became_active":"2024-09-15T21:46:23.421501+0000","last_became_peered":"2024-09-15T21:46:23.421501+0000","last_unstale":"2024-09-15T21:46:24.808369+0000","last_undegraded":"2024-09-15T21:46:24.808369+0000","last_fullsized":"2024-09-15T21:46:24.808369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:55.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.436035+0000","last_change":"2024-09-15T21:46:23.432591+0000","last_active":"2024-09-15T21:46:23.436035+0000","last_peered":"2024-09-15T21:46:23.436035+0000","last_clean":"2024-09-15T21:46:23.436035+0000","last_became_active":"2024-09-15T21:46:23.432326+0000","last_became_peered":"2024-09-15T21:46:23.432326+0000","last_unstale":"2024-09-15T21:46:23.436035+0000","last_undegraded":"2024-09-15T21:46:23.436035+0000","last_fullsized":"2024-09-15T21:46:23.436035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:46:56.508701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.428496+0000","last_change":"2024-09-15T21:46:23.423428+0000","last_active":"2024-09-15T21:46:23.428496+0000","last_peered":"2024-09-15T21:46:23.428496+0000","last_clean":"2024-09-15T21:46:23.428496+0000","last_became_active":"2024-09-15T21:46:23.422734+0000","last_became_peered":"2024-09-15T21:46:23.422734+0000","last_unstale":"2024-09-15T21:46:23.428496+0000","last_undegraded":"2024-09-15T21:46:23.428496+0000","last_fullsized":"2024-09-15T21:46:23.428496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:02.082197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.417891+0000","last_change":"2024-09-15T21:46:22.417891+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.417891+0000","last_undegraded":"2024-09-15T21:46:22.417891+0000","last_fullsized":"2024-09-15T21:46:22.417891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414635+0000","last_change":"2024-09-15T21:46:22.414635+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414635+0000","last_undegraded":"2024-09-15T21:46:22.414635+0000","last_fullsized":"2024-09-15T21:46:22.414635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418469+0000","last_change":"2024-09-15T21:46:22.418469+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418469+0000","last_undegraded":"2024-09-15T21:46:22.418469+0000","last_fullsized":"2024-09-15T21:46:22.418469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414859+0000","last_change":"2024-09-15T21:46:22.414859+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414859+0000","last_undegraded":"2024-09-15T21:46:22.414859+0000","last_fullsized":"2024-09-15T21:46:22.414859+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.424245+0000","last_change":"2024-09-15T21:46:22.429407+0000","last_active":"2024-09-15T21:46:23.424245+0000","last_peered":"2024-09-15T21:46:23.424245+0000","last_clean":"2024-09-15T21:46:23.424245+0000","last_became_active":"2024-09-15T21:46:22.429107+0000","last_became_peered":"2024-09-15T21:46:22.429107+0000","last_unstale":"2024-09-15T21:46:23.424245+0000","last_undegraded":"2024-09-15T21:46:23.424245+0000","last_fullsized":"2024-09-15T21:46:23.424245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:14:54.680349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.438772+0000","last_change":"2024-09-15T21:46:23.434198+0000","last_active":"2024-09-15T21:46:23.438772+0000","last_peered":"2024-09-15T21:46:23.438772+0000","last_clean":"2024-09-15T21:46:23.438772+0000","last_became_active":"2024-09-15T21:46:23.433944+0000","last_became_peered":"2024-09-15T21:46:23.433944+0000","last_unstale":"2024-09-15T21:46:23.438772+0000","last_undegraded":"2024-09-15T21:46:23.438772+0000","last_fullsized":"2024-09-15T21:46:23.438772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:20:05.953148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.428922+0000","last_change":"2024-09-15T21:46:22.425571+0000","last_active":"2024-09-15T21:46:22.428922+0000","last_peered":"2024-09-15T21:46:22.428922+0000","last_clean":"2024-09-15T21:46:22.428922+0000","last_became_active":"2024-09-15T21:46:22.425315+0000","last_became_peered":"2024-09-15T21:46:22.425315+0000","last_unstale":"2024-09-15T21:46:22.428922+0000","last_undegraded":"2024-09-15T21:46:22.428922+0000","last_fullsized":"2024-09-15T21:46:22.428922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:37.405710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.423270+0000","last_change":"2024-09-15T21:46:22.425492+0000","last_active":"2024-09-15T21:46:23.423270+0000","last_peered":"2024-09-15T21:46:23.423270+0000","last_clean":"2024-09-15T21:46:23.423270+0000","last_became_active":"2024-09-15T21:46:22.425226+0000","last_became_peered":"2024-09-15T21:46:22.425226+0000","last_unstale":"2024-09-15T21:46:23.423270+0000","last_undegraded":"2024-09-15T21:46:23.423270+0000","last_fullsized":"2024-09-15T21:46:23.423270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:24.322384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433622+0000","last_change":"2024-09-15T21:46:23.429140+0000","last_active":"2024-09-15T21:46:23.433622+0000","last_peered":"2024-09-15T21:46:23.433622+0000","last_clean":"2024-09-15T21:46:23.433622+0000","last_became_active":"2024-09-15T21:46:23.428902+0000","last_became_peered":"2024-09-15T21:46:23.428902+0000","last_unstale":"2024-09-15T21:46:23.433622+0000","last_undegraded":"2024-09-15T21:46:23.433622+0000","last_fullsized":"2024-09-15T21:46:23.433622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:44:54.129937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.431357+0000","last_change":"2024-09-15T21:46:22.428319+0000","last_active":"2024-09-15T21:46:22.431357+0000","last_peered":"2024-09-15T21:46:22.431357+0000","last_clean":"2024-09-15T21:46:22.431357+0000","last_became_active":"2024-09-15T21:46:22.428061+0000","last_became_peered":"2024-09-15T21:46:22.428061+0000","last_unstale":"2024-09-15T21:46:22.431357+0000","last_undegraded":"2024-09-15T21:46:22.431357+0000","last_fullsized":"2024-09-15T21:46:22.431357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:34.432046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808315+0000","last_change":"2024-09-15T21:46:23.427442+0000","last_active":"2024-09-15T21:46:24.808315+0000","last_peered":"2024-09-15T21:46:24.808315+0000","last_clean":"2024-09-15T21:46:24.808315+0000","last_became_active":"2024-09-15T21:46:23.427200+0000","last_became_peered":"2024-09-15T21:46:23.427200+0000","last_unstale":"2024-09-15T21:46:24.808315+0000","last_undegraded":"2024-09-15T21:46:24.808315+0000","last_fullsized":"2024-09-15T21:46:24.808315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:27.248545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433993+0000","last_change":"2024-09-15T21:46:22.430599+0000","last_active":"2024-09-15T21:46:22.433993+0000","last_peered":"2024-09-15T21:46:22.433993+0000","last_clean":"2024-09-15T21:46:22.433993+0000","last_became_active":"2024-09-15T21:46:22.430327+0000","last_became_peered":"2024-09-15T21:46:22.430327+0000","last_unstale":"2024-09-15T21:46:22.433993+0000","last_undegraded":"2024-09-15T21:46:22.433993+0000","last_fullsized":"2024-09-15T21:46:22.433993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:42:31.882474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.416669+0000","last_change":"2024-09-15T21:46:22.416669+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.416669+0000","last_undegraded":"2024-09-15T21:46:22.416669+0000","last_fullsized":"2024-09-15T21:46:22.416669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.429815+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:22.429815+0000","last_peered":"2024-09-15T21:46:22.429815+0000","last_clean":"2024-09-15T21:46:22.429815+0000","last_became_active":"2024-09-15T21:46:22.425236+0000","last_became_peered":"2024-09-15T21:46:22.425236+0000","last_unstale":"2024-09-15T21:46:22.429815+0000","last_undegraded":"2024-09-15T21:46:22.429815+0000","last_fullsized":"2024-09-15T21:46:22.429815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:01.525976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433802+0000","last_change":"2024-09-15T21:46:23.429587+0000","last_active":"2024-09-15T21:46:23.433802+0000","last_peered":"2024-09-15T21:46:23.433802+0000","last_clean":"2024-09-15T21:46:23.433802+0000","last_became_active":"2024-09-15T21:46:23.429334+0000","last_became_peered":"2024-09-15T21:46:23.429334+0000","last_unstale":"2024-09-15T21:46:23.433802+0000","last_undegraded":"2024-09-15T21:46:23.433802+0000","last_fullsized":"2024-09-15T21:46:23.433802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:31.263309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808325+0000","last_change":"2024-09-15T21:46:22.421964+0000","last_active":"2024-09-15T21:46:24.808325+0000","last_peered":"2024-09-15T21:46:24.808325+0000","last_clean":"2024-09-15T21:46:24.808325+0000","last_became_active":"2024-09-15T21:46:22.421661+0000","last_became_peered":"2024-09-15T21:46:22.421661+0000","last_unstale":"2024-09-15T21:46:24.808325+0000","last_undegraded":"2024-09-15T21:46:24.808325+0000","last_fullsized":"2024-09-15T21:46:24.808325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:07.395070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414271+0000","last_change":"2024-09-15T21:46:22.414271+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414271+0000","last_undegraded":"2024-09-15T21:46:22.414271+0000","last_fullsized":"2024-09-15T21:46:22.414271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433356+0000","last_change":"2024-09-15T21:46:22.426151+0000","last_active":"2024-09-15T21:46:22.433356+0000","last_peered":"2024-09-15T21:46:22.433356+0000","last_clean":"2024-09-15T21:46:22.433356+0000","last_became_active":"2024-09-15T21:46:22.425866+0000","last_became_peered":"2024-09-15T21:46:22.425866+0000","last_unstale":"2024-09-15T21:46:22.433356+0000","last_undegraded":"2024-09-15T21:46:22.433356+0000","last_fullsized":"2024-09-15T21:46:22.433356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:18.129618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419893+0000","last_change":"2024-09-15T21:46:22.419893+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419893+0000","last_undegraded":"2024-09-15T21:46:22.419893+0000","last_fullsized":"2024-09-15T21:46:22.419893+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.417870+0000","last_change":"2024-09-15T21:46:22.427415+0000","last_active":"2024-09-15T21:46:23.417870+0000","last_peered":"2024-09-15T21:46:23.417870+0000","last_clean":"2024-09-15T21:46:23.417870+0000","last_became_active":"2024-09-15T21:46:22.427087+0000","last_became_peered":"2024-09-15T21:46:22.427087+0000","last_unstale":"2024-09-15T21:46:23.417870+0000","last_undegraded":"2024-09-15T21:46:23.417870+0000","last_fullsized":"2024-09-15T21:46:23.417870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:09.365202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431262+0000","last_change":"2024-09-15T21:46:23.427238+0000","last_active":"2024-09-15T21:46:23.431262+0000","last_peered":"2024-09-15T21:46:23.431262+0000","last_clean":"2024-09-15T21:46:23.431262+0000","last_became_active":"2024-09-15T21:46:23.426993+0000","last_became_peered":"2024-09-15T21:46:23.426993+0000","last_unstale":"2024-09-15T21:46:23.431262+0000","last_undegraded":"2024-09-15T21:46:23.431262+0000","last_fullsized":"2024-09-15T21:46:23.431262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:36.125958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.431315+0000","last_change":"2024-09-15T21:46:22.427164+0000","last_active":"2024-09-15T21:46:22.431315+0000","last_peered":"2024-09-15T21:46:22.431315+0000","last_clean":"2024-09-15T21:46:22.431315+0000","last_became_active":"2024-09-15T21:46:22.426917+0000","last_became_peered":"2024-09-15T21:46:22.426917+0000","last_unstale":"2024-09-15T21:46:22.431315+0000","last_undegraded":"2024-09-15T21:46:22.431315+0000","last_fullsized":"2024-09-15T21:46:22.431315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:12:47.575708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418200+0000","last_change":"2024-09-15T21:46:22.418200+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418200+0000","last_undegraded":"2024-09-15T21:46:22.418200+0000","last_fullsized":"2024-09-15T21:46:22.418200+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.428957+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:22.428957+0000","last_peered":"2024-09-15T21:46:22.428957+0000","last_clean":"2024-09-15T21:46:22.428957+0000","last_became_active":"2024-09-15T21:46:22.425237+0000","last_became_peered":"2024-09-15T21:46:22.425237+0000","last_unstale":"2024-09-15T21:46:22.428957+0000","last_undegraded":"2024-09-15T21:46:22.428957+0000","last_fullsized":"2024-09-15T21:46:22.428957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:46.301372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.417788+0000","last_change":"2024-09-15T21:46:22.417788+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.417788+0000","last_undegraded":"2024-09-15T21:46:22.417788+0000","last_fullsized":"2024-09-15T21:46:22.417788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.423016+0000","last_change":"2024-09-15T21:46:22.419905+0000","last_active":"2024-09-15T21:46:22.423016+0000","last_peered":"2024-09-15T21:46:22.423016+0000","last_clean":"2024-09-15T21:46:22.423016+0000","last_became_active":"2024-09-15T21:46:22.419620+0000","last_became_peered":"2024-09-15T21:46:22.419620+0000","last_unstale":"2024-09-15T21:46:22.423016+0000","last_undegraded":"2024-09-15T21:46:22.423016+0000","last_fullsized":"2024-09-15T21:46:22.423016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:41.513967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419332+0000","last_change":"2024-09-15T21:46:22.419332+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419332+0000","last_undegraded":"2024-09-15T21:46:22.419332+0000","last_fullsized":"2024-09-15T21:46:22.419332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433527+0000","last_change":"2024-09-15T21:46:22.429661+0000","last_active":"2024-09-15T21:46:22.433527+0000","last_peered":"2024-09-15T21:46:22.433527+0000","last_clean":"2024-09-15T21:46:22.433527+0000","last_became_active":"2024-09-15T21:46:22.429250+0000","last_became_peered":"2024-09-15T21:46:22.429250+0000","last_unstale":"2024-09-15T21:46:22.433527+0000","last_undegraded":"2024-09-15T21:46:22.433527+0000","last_fullsized":"2024-09-15T21:46:22.433527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:52:16.852861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418349+0000","last_change":"2024-09-15T21:46:22.418349+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418349+0000","last_undegraded":"2024-09-15T21:46:22.418349+0000","last_fullsized":"2024-09-15T21:46:22.418349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.430022+0000","last_change":"2024-09-15T21:46:22.425870+0000","last_active":"2024-09-15T21:46:22.430022+0000","last_peered":"2024-09-15T21:46:22.430022+0000","last_clean":"2024-09-15T21:46:22.430022+0000","last_became_active":"2024-09-15T21:46:22.425588+0000","last_became_peered":"2024-09-15T21:46:22.425588+0000","last_unstale":"2024-09-15T21:46:22.430022+0000","last_undegraded":"2024-09-15T21:46:22.430022+0000","last_fullsized":"2024-09-15T21:46:22.430022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:53.982102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.420429+0000","last_change":"2024-09-15T21:46:22.420429+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.420429+0000","last_undegraded":"2024-09-15T21:46:22.420429+0000","last_fullsized":"2024-09-15T21:46:22.420429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808420+0000","last_change":"2024-09-15T21:46:22.425072+0000","last_active":"2024-09-15T21:46:24.808420+0000","last_peered":"2024-09-15T21:46:24.808420+0000","last_clean":"2024-09-15T21:46:24.808420+0000","last_became_active":"2024-09-15T21:46:22.424771+0000","last_became_peered":"2024-09-15T21:46:22.424771+0000","last_unstale":"2024-09-15T21:46:24.808420+0000","last_undegraded":"2024-09-15T21:46:24.808420+0000","last_fullsized":"2024-09-15T21:46:24.808420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:26.256785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415969+0000","last_change":"2024-09-15T21:46:22.415969+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415969+0000","last_undegraded":"2024-09-15T21:46:22.415969+0000","last_fullsized":"2024-09-15T21:46:22.415969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.432304+0000","last_change":"2024-09-15T21:46:22.428668+0000","last_active":"2024-09-15T21:46:22.432304+0000","last_peered":"2024-09-15T21:46:22.432304+0000","last_clean":"2024-09-15T21:46:22.432304+0000","last_became_active":"2024-09-15T21:46:22.428341+0000","last_became_peered":"2024-09-15T21:46:22.428341+0000","last_unstale":"2024-09-15T21:46:22.432304+0000","last_undegraded":"2024-09-15T21:46:22.432304+0000","last_fullsized":"2024-09-15T21:46:22.432304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:25.844475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418811+0000","last_change":"2024-09-15T21:46:22.418811+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418811+0000","last_undegraded":"2024-09-15T21:46:22.418811+0000","last_fullsized":"2024-09-15T21:46:22.418811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":167,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.422877+0000","last_change":"2024-09-15T21:46:14.328094+0000","last_active":"2024-09-15T21:46:22.422877+0000","last_peered":"2024-09-15T21:46:22.422877+0000","last_clean":"2024-09-15T21:46:22.422877+0000","last_became_active":"2024-09-15T21:46:14.326991+0000","last_became_peered":"2024-09-15T21:46:14.326991+0000","last_unstale":"2024-09-15T21:46:22.422877+0000","last_undegraded":"2024-09-15T21:46:22.422877+0000","last_fullsized":"2024-09-15T21:46:22.422877+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_clean_scrub_stamp":"2024-09-15T21:46:12.223400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:54:27.704130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.431406+0000","last_change":"2024-09-15T21:46:22.428887+0000","last_active":"2024-09-15T21:46:22.431406+0000","last_peered":"2024-09-15T21:46:22.431406+0000","last_clean":"2024-09-15T21:46:22.431406+0000","last_became_active":"2024-09-15T21:46:22.428619+0000","last_became_peered":"2024-09-15T21:46:22.428619+0000","last_unstale":"2024-09-15T21:46:22.431406+0000","last_undegraded":"2024-09-15T21:46:22.431406+0000","last_fullsized":"2024-09-15T21:46:22.431406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:43.510542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.438679+0000","last_change":"2024-09-15T21:46:23.435459+0000","last_active":"2024-09-15T21:46:23.438679+0000","last_peered":"2024-09-15T21:46:23.438679+0000","last_clean":"2024-09-15T21:46:23.438679+0000","last_became_active":"2024-09-15T21:46:23.435141+0000","last_became_peered":"2024-09-15T21:46:23.435141+0000","last_unstale":"2024-09-15T21:46:23.438679+0000","last_undegraded":"2024-09-15T21:46:23.438679+0000","last_fullsized":"2024-09-15T21:46:23.438679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:38.434075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433564+0000","last_change":"2024-09-15T21:46:22.431113+0000","last_active":"2024-09-15T21:46:22.433564+0000","last_peered":"2024-09-15T21:46:22.433564+0000","last_clean":"2024-09-15T21:46:22.433564+0000","last_became_active":"2024-09-15T21:46:22.430832+0000","last_became_peered":"2024-09-15T21:46:22.430832+0000","last_unstale":"2024-09-15T21:46:22.433564+0000","last_undegraded":"2024-09-15T21:46:22.433564+0000","last_fullsized":"2024-09-15T21:46:22.433564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:08.068054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433942+0000","last_change":"2024-09-15T21:46:23.430396+0000","last_active":"2024-09-15T21:46:23.433942+0000","last_peered":"2024-09-15T21:46:23.433942+0000","last_clean":"2024-09-15T21:46:23.433942+0000","last_became_active":"2024-09-15T21:46:23.430138+0000","last_became_peered":"2024-09-15T21:46:23.430138+0000","last_unstale":"2024-09-15T21:46:23.433942+0000","last_undegraded":"2024-09-15T21:46:23.433942+0000","last_fullsized":"2024-09-15T21:46:23.433942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:04.056726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.430035+0000","last_change":"2024-09-15T21:46:22.425808+0000","last_active":"2024-09-15T21:46:22.430035+0000","last_peered":"2024-09-15T21:46:22.430035+0000","last_clean":"2024-09-15T21:46:22.430035+0000","last_became_active":"2024-09-15T21:46:22.425510+0000","last_became_peered":"2024-09-15T21:46:22.425510+0000","last_unstale":"2024-09-15T21:46:22.430035+0000","last_undegraded":"2024-09-15T21:46:22.430035+0000","last_fullsized":"2024-09-15T21:46:22.430035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:13.502626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808338+0000","last_change":"2024-09-15T21:46:23.427051+0000","last_active":"2024-09-15T21:46:24.808338+0000","last_peered":"2024-09-15T21:46:24.808338+0000","last_clean":"2024-09-15T21:46:24.808338+0000","last_became_active":"2024-09-15T21:46:23.426185+0000","last_became_peered":"2024-09-15T21:46:23.426185+0000","last_unstale":"2024-09-15T21:46:24.808338+0000","last_undegraded":"2024-09-15T21:46:24.808338+0000","last_fullsized":"2024-09-15T21:46:24.808338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:48.318405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.429782+0000","last_change":"2024-09-15T21:46:22.424645+0000","last_active":"2024-09-15T21:46:22.429782+0000","last_peered":"2024-09-15T21:46:22.429782+0000","last_clean":"2024-09-15T21:46:22.429782+0000","last_became_active":"2024-09-15T21:46:22.421229+0000","last_became_peered":"2024-09-15T21:46:22.421229+0000","last_unstale":"2024-09-15T21:46:22.429782+0000","last_undegraded":"2024-09-15T21:46:22.429782+0000","last_fullsized":"2024-09-15T21:46:22.429782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:02.298935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.413899+0000","last_change":"2024-09-15T21:46:22.413899+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.413899+0000","last_undegraded":"2024-09-15T21:46:22.413899+0000","last_fullsized":"2024-09-15T21:46:22.413899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808438+0000","last_change":"2024-09-15T21:46:22.426492+0000","last_active":"2024-09-15T21:46:24.808438+0000","last_peered":"2024-09-15T21:46:24.808438+0000","last_clean":"2024-09-15T21:46:24.808438+0000","last_became_active":"2024-09-15T21:46:22.426210+0000","last_became_peered":"2024-09-15T21:46:22.426210+0000","last_unstale":"2024-09-15T21:46:24.808438+0000","last_undegraded":"2024-09-15T21:46:24.808438+0000","last_fullsized":"2024-09-15T21:46:24.808438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:05.248091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414998+0000","last_change":"2024-09-15T21:46:22.414998+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414998+0000","last_undegraded":"2024-09-15T21:46:22.414998+0000","last_fullsized":"2024-09-15T21:46:22.414998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433499+0000","last_change":"2024-09-15T21:46:22.430710+0000","last_active":"2024-09-15T21:46:22.433499+0000","last_peered":"2024-09-15T21:46:22.433499+0000","last_clean":"2024-09-15T21:46:22.433499+0000","last_became_active":"2024-09-15T21:46:22.430375+0000","last_became_peered":"2024-09-15T21:46:22.430375+0000","last_unstale":"2024-09-15T21:46:22.433499+0000","last_undegraded":"2024-09-15T21:46:22.433499+0000","last_fullsized":"2024-09-15T21:46:22.433499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:17.852485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808447+0000","last_change":"2024-09-15T21:46:23.426729+0000","last_active":"2024-09-15T21:46:24.808447+0000","last_peered":"2024-09-15T21:46:24.808447+0000","last_clean":"2024-09-15T21:46:24.808447+0000","last_became_active":"2024-09-15T21:46:23.426495+0000","last_became_peered":"2024-09-15T21:46:23.426495+0000","last_unstale":"2024-09-15T21:46:24.808447+0000","last_undegraded":"2024-09-15T21:46:24.808447+0000","last_fullsized":"2024-09-15T21:46:24.808447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:03.277997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808436+0000","last_change":"2024-09-15T21:46:22.427137+0000","last_active":"2024-09-15T21:46:24.808436+0000","last_peered":"2024-09-15T21:46:24.808436+0000","last_clean":"2024-09-15T21:46:24.808436+0000","last_became_active":"2024-09-15T21:46:22.426855+0000","last_became_peered":"2024-09-15T21:46:22.426855+0000","last_unstale":"2024-09-15T21:46:24.808436+0000","last_undegraded":"2024-09-15T21:46:24.808436+0000","last_fullsized":"2024-09-15T21:46:24.808436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:40:14.277940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431767+0000","last_change":"2024-09-15T21:46:23.427327+0000","last_active":"2024-09-15T21:46:23.431767+0000","last_peered":"2024-09-15T21:46:23.431767+0000","last_clean":"2024-09-15T21:46:23.431767+0000","last_became_active":"2024-09-15T21:46:23.427096+0000","last_became_peered":"2024-09-15T21:46:23.427096+0000","last_unstale":"2024-09-15T21:46:23.431767+0000","last_undegraded":"2024-09-15T21:46:23.431767+0000","last_fullsized":"2024-09-15T21:46:23.431767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:43.650053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433399+0000","last_change":"2024-09-15T21:46:22.427798+0000","last_active":"2024-09-15T21:46:22.433399+0000","last_peered":"2024-09-15T21:46:22.433399+0000","last_clean":"2024-09-15T21:46:22.433399+0000","last_became_active":"2024-09-15T21:46:22.427537+0000","last_became_peered":"2024-09-15T21:46:22.427537+0000","last_unstale":"2024-09-15T21:46:22.433399+0000","last_undegraded":"2024-09-15T21:46:22.433399+0000","last_fullsized":"2024-09-15T21:46:22.433399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:36.227177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.428576+0000","last_change":"2024-09-15T21:46:23.424266+0000","last_active":"2024-09-15T21:46:23.428576+0000","last_peered":"2024-09-15T21:46:23.428576+0000","last_clean":"2024-09-15T21:46:23.428576+0000","last_became_active":"2024-09-15T21:46:23.424007+0000","last_became_peered":"2024-09-15T21:46:23.424007+0000","last_unstale":"2024-09-15T21:46:23.428576+0000","last_undegraded":"2024-09-15T21:46:23.428576+0000","last_fullsized":"2024-09-15T21:46:23.428576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:04:11.974067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433546+0000","last_change":"2024-09-15T21:46:22.430901+0000","last_active":"2024-09-15T21:46:22.433546+0000","last_peered":"2024-09-15T21:46:22.433546+0000","last_clean":"2024-09-15T21:46:22.433546+0000","last_became_active":"2024-09-15T21:46:22.430452+0000","last_became_peered":"2024-09-15T21:46:22.430452+0000","last_unstale":"2024-09-15T21:46:22.433546+0000","last_undegraded":"2024-09-15T21:46:22.433546+0000","last_fullsized":"2024-09-15T21:46:22.433546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:02.951631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414149+0000","last_change":"2024-09-15T21:46:22.414149+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414149+0000","last_undegraded":"2024-09-15T21:46:22.414149+0000","last_fullsized":"2024-09-15T21:46:22.414149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808380+0000","last_change":"2024-09-15T21:46:22.423795+0000","last_active":"2024-09-15T21:46:24.808380+0000","last_peered":"2024-09-15T21:46:24.808380+0000","last_clean":"2024-09-15T21:46:24.808380+0000","last_became_active":"2024-09-15T21:46:22.422327+0000","last_became_peered":"2024-09-15T21:46:22.422327+0000","last_unstale":"2024-09-15T21:46:24.808380+0000","last_undegraded":"2024-09-15T21:46:24.808380+0000","last_fullsized":"2024-09-15T21:46:24.808380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:16.670599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433600+0000","last_change":"2024-09-15T21:46:23.428634+0000","last_active":"2024-09-15T21:46:23.433600+0000","last_peered":"2024-09-15T21:46:23.433600+0000","last_clean":"2024-09-15T21:46:23.433600+0000","last_became_active":"2024-09-15T21:46:23.428301+0000","last_became_peered":"2024-09-15T21:46:23.428301+0000","last_unstale":"2024-09-15T21:46:23.433600+0000","last_undegraded":"2024-09-15T21:46:23.433600+0000","last_fullsized":"2024-09-15T21:46:23.433600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:17.178940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808304+0000","last_change":"2024-09-15T21:46:22.425810+0000","last_active":"2024-09-15T21:46:24.808304+0000","last_peered":"2024-09-15T21:46:24.808304+0000","last_clean":"2024-09-15T21:46:24.808304+0000","last_became_active":"2024-09-15T21:46:22.425478+0000","last_became_peered":"2024-09-15T21:46:22.425478+0000","last_unstale":"2024-09-15T21:46:24.808304+0000","last_undegraded":"2024-09-15T21:46:24.808304+0000","last_fullsized":"2024-09-15T21:46:24.808304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:53.296599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431785+0000","last_change":"2024-09-15T21:46:23.426054+0000","last_active":"2024-09-15T21:46:23.431785+0000","last_peered":"2024-09-15T21:46:23.431785+0000","last_clean":"2024-09-15T21:46:23.431785+0000","last_became_active":"2024-09-15T21:46:23.425547+0000","last_became_peered":"2024-09-15T21:46:23.425547+0000","last_unstale":"2024-09-15T21:46:23.431785+0000","last_undegraded":"2024-09-15T21:46:23.431785+0000","last_fullsized":"2024-09-15T21:46:23.431785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:25.738831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433353+0000","last_change":"2024-09-15T21:46:22.428524+0000","last_active":"2024-09-15T21:46:22.433353+0000","last_peered":"2024-09-15T21:46:22.433353+0000","last_clean":"2024-09-15T21:46:22.433353+0000","last_became_active":"2024-09-15T21:46:22.428043+0000","last_became_peered":"2024-09-15T21:46:22.428043+0000","last_unstale":"2024-09-15T21:46:22.433353+0000","last_undegraded":"2024-09-15T21:46:22.433353+0000","last_fullsized":"2024-09-15T21:46:22.433353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:53.400491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419544+0000","last_change":"2024-09-15T21:46:22.419544+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419544+0000","last_undegraded":"2024-09-15T21:46:22.419544+0000","last_fullsized":"2024-09-15T21:46:22.419544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415492+0000","last_change":"2024-09-15T21:46:22.415492+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415492+0000","last_undegraded":"2024-09-15T21:46:22.415492+0000","last_fullsized":"2024-09-15T21:46:22.415492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433558+0000","last_change":"2024-09-15T21:46:22.431172+0000","last_active":"2024-09-15T21:46:22.433558+0000","last_peered":"2024-09-15T21:46:22.433558+0000","last_clean":"2024-09-15T21:46:22.433558+0000","last_became_active":"2024-09-15T21:46:22.430800+0000","last_became_peered":"2024-09-15T21:46:22.430800+0000","last_unstale":"2024-09-15T21:46:22.433558+0000","last_undegraded":"2024-09-15T21:46:22.433558+0000","last_fullsized":"2024-09-15T21:46:22.433558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:52:07.231504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.417557+0000","last_change":"2024-09-15T21:46:22.417557+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.417557+0000","last_undegraded":"2024-09-15T21:46:22.417557+0000","last_fullsized":"2024-09-15T21:46:22.417557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433476+0000","last_change":"2024-09-15T21:46:22.429870+0000","last_active":"2024-09-15T21:46:22.433476+0000","last_peered":"2024-09-15T21:46:22.433476+0000","last_clean":"2024-09-15T21:46:22.433476+0000","last_became_active":"2024-09-15T21:46:22.429578+0000","last_became_peered":"2024-09-15T21:46:22.429578+0000","last_unstale":"2024-09-15T21:46:22.433476+0000","last_undegraded":"2024-09-15T21:46:22.433476+0000","last_fullsized":"2024-09-15T21:46:22.433476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:59.880081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433864+0000","last_change":"2024-09-15T21:46:23.429604+0000","last_active":"2024-09-15T21:46:23.433864+0000","last_peered":"2024-09-15T21:46:23.433864+0000","last_clean":"2024-09-15T21:46:23.433864+0000","last_became_active":"2024-09-15T21:46:23.429355+0000","last_became_peered":"2024-09-15T21:46:23.429355+0000","last_unstale":"2024-09-15T21:46:23.433864+0000","last_undegraded":"2024-09-15T21:46:23.433864+0000","last_fullsized":"2024-09-15T21:46:23.433864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:23:10.533918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.424314+0000","last_change":"2024-09-15T21:46:22.428529+0000","last_active":"2024-09-15T21:46:23.424314+0000","last_peered":"2024-09-15T21:46:23.424314+0000","last_clean":"2024-09-15T21:46:23.424314+0000","last_became_active":"2024-09-15T21:46:22.428263+0000","last_became_peered":"2024-09-15T21:46:22.428263+0000","last_unstale":"2024-09-15T21:46:23.424314+0000","last_undegraded":"2024-09-15T21:46:23.424314+0000","last_fullsized":"2024-09-15T21:46:23.424314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:22:40.160062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433487+0000","last_change":"2024-09-15T21:46:23.427968+0000","last_active":"2024-09-15T21:46:23.433487+0000","last_peered":"2024-09-15T21:46:23.433487+0000","last_clean":"2024-09-15T21:46:23.433487+0000","last_became_active":"2024-09-15T21:46:23.427546+0000","last_became_peered":"2024-09-15T21:46:23.427546+0000","last_unstale":"2024-09-15T21:46:23.433487+0000","last_undegraded":"2024-09-15T21:46:23.433487+0000","last_fullsized":"2024-09-15T21:46:23.433487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:45.544297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.417176+0000","last_change":"2024-09-15T21:46:22.428258+0000","last_active":"2024-09-15T21:46:23.417176+0000","last_peered":"2024-09-15T21:46:23.417176+0000","last_clean":"2024-09-15T21:46:23.417176+0000","last_became_active":"2024-09-15T21:46:22.427954+0000","last_became_peered":"2024-09-15T21:46:22.427954+0000","last_unstale":"2024-09-15T21:46:23.417176+0000","last_undegraded":"2024-09-15T21:46:23.417176+0000","last_fullsized":"2024-09-15T21:46:23.417176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:22.440456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476740,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":654251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":654251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":57608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":57608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":57608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":57608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":76958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":70658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:46:28.369 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:46:28.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.503+0000 7f8d3123c640 1 Processor -- start 2024-09-15T21:46:28.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.503+0000 7f8d3123c640 1 -- start start 2024-09-15T21:46:28.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.504+0000 7f8d3123c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8d2c073320 0x7f8d2c158e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.504+0000 7f8d3123c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f8d2c0c8a10 0x7f8d2c0c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.504+0000 7f8d3123c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d2c0c91a0 con 0x7f8d2c073700 2024-09-15T21:46:28.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.504+0000 7f8d3123c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f8d2c0c9440 con 0x7f8d2c073320 2024-09-15T21:46:28.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.504+0000 7f8d3123c640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f8d2c0c9cc0 con 0x7f8d2c0c8a10 2024-09-15T21:46:28.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.504+0000 7f8d2ad9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8d2c073320 0x7f8d2c158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.504+0000 7f8d2ad9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8d2c073320 0x7f8d2c158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43674/0 (socket says 172.21.15.100:43674) 2024-09-15T21:46:28.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.504+0000 7f8d2ad9d640 1 -- 172.21.15.100:0/3846784119 learned_addr learned my addr 172.21.15.100:0/3846784119 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:28.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.504+0000 7f8d2b59e640 1 --2- 172.21.15.100:0/3846784119 >> v2:172.21.15.122:3300/0 conn(0x7f8d2c0c8a10 0x7f8d2c0c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.504+0000 7f8d2ad9d640 1 -- 172.21.15.100:0/3846784119 >> v2:172.21.15.122:3300/0 conn(0x7f8d2c0c8a10 msgr2=0x7f8d2c0c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:28.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.504+0000 7f8d2ad9d640 1 --2- 172.21.15.100:0/3846784119 >> v2:172.21.15.122:3300/0 conn(0x7f8d2c0c8a10 0x7f8d2c0c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.504+0000 7f8d2ad9d640 1 -- 172.21.15.100:0/3846784119 >> v1:172.21.15.122:6789/0 conn(0x7f8d2c073700 legacy=0x7f8d2c152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:28.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.504+0000 7f8d2ad9d640 1 -- 172.21.15.100:0/3846784119 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2c0c9700 con 0x7f8d2c073320 2024-09-15T21:46:28.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.505+0000 7f8d2ad9d640 1 --2- 172.21.15.100:0/3846784119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c073320 0x7f8d2c158e70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0c5500 tx=0x7f8d1c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36ef2dca45cd513 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:28.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.505+0000 7f8d29d9b640 1 -- 172.21.15.100:0/3846784119 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c036070 con 0x7f8d2c073320 2024-09-15T21:46:28.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.505+0000 7f8d29d9b640 1 -- 172.21.15.100:0/3846784119 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d1c031600 con 0x7f8d2c073320 2024-09-15T21:46:28.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.505+0000 7f8d29d9b640 1 -- 172.21.15.100:0/3846784119 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c0319d0 con 0x7f8d2c073320 2024-09-15T21:46:28.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.506+0000 7f8d3123c640 1 -- 172.21.15.100:0/3846784119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c073320 msgr2=0x7f8d2c158e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:28.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.506+0000 7f8d3123c640 1 --2- 172.21.15.100:0/3846784119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c073320 0x7f8d2c158e70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0c5500 tx=0x7f8d1c02a730 comp rx=0 tx=0).stop 2024-09-15T21:46:28.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.506+0000 7f8d3123c640 1 -- 172.21.15.100:0/3846784119 shutdown_connections 2024-09-15T21:46:28.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.506+0000 7f8d3123c640 1 --2- 172.21.15.100:0/3846784119 >> v2:172.21.15.122:3300/0 conn(0x7f8d2c0c8a10 0x7f8d2c0c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.506+0000 7f8d3123c640 1 --2- 172.21.15.100:0/3846784119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c073320 0x7f8d2c158e70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.506+0000 7f8d3123c640 1 -- 172.21.15.100:0/3846784119 >> 172.21.15.100:0/3846784119 conn(0x7f8d2c070ca0 msgr2=0x7f8d2c071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:28.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.506+0000 7f8d3123c640 1 -- 172.21.15.100:0/3846784119 shutdown_connections 2024-09-15T21:46:28.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.506+0000 7f8d3123c640 1 -- 172.21.15.100:0/3846784119 wait complete. 2024-09-15T21:46:28.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.507+0000 7f8d3123c640 1 Processor -- start 2024-09-15T21:46:28.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.507+0000 7f8d3123c640 1 -- start start 2024-09-15T21:46:28.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.508+0000 7f8d3123c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8d2c073320 0x7f8d2c0cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.508+0000 7f8d3123c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c073700 0x7f8d2c14bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.508+0000 7f8d3123c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d2c0c8a10 0x7f8d2c14ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.508+0000 7f8d3123c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c0c6020 con 0x7f8d2c073700 2024-09-15T21:46:28.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.508+0000 7f8d3123c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c0c5d40 con 0x7f8d2c0c8a10 2024-09-15T21:46:28.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.508+0000 7f8d3123c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8d2c0c78c0 con 0x7f8d2c073320 2024-09-15T21:46:28.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.508+0000 7f8d2a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c073700 0x7f8d2c14bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.508+0000 7f8d2a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c073700 0x7f8d2c14bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43678/0 (socket says 172.21.15.100:43678) 2024-09-15T21:46:28.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.508+0000 7f8d2a59c640 1 -- 172.21.15.100:0/682152920 learned_addr learned my addr 172.21.15.100:0/682152920 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:28.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.508+0000 7f8d2b59e640 1 --2- 172.21.15.100:0/682152920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d2c0c8a10 0x7f8d2c14ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.508+0000 7f8d2ad9d640 1 --2- 172.21.15.100:0/682152920 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8d2c073320 0x7f8d2c0cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.508+0000 7f8d2a59c640 1 -- 172.21.15.100:0/682152920 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8d2c073320 msgr2=0x7f8d2c0cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:28.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.508+0000 7f8d2a59c640 1 --2- 172.21.15.100:0/682152920 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8d2c073320 0x7f8d2c0cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.508+0000 7f8d2a59c640 1 -- 172.21.15.100:0/682152920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d2c0c8a10 msgr2=0x7f8d2c14ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.508+0000 7f8d2a59c640 1 --2- 172.21.15.100:0/682152920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d2c0c8a10 0x7f8d2c14ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.508+0000 7f8d2a59c640 1 -- 172.21.15.100:0/682152920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2c1eec60 con 0x7f8d2c073700 2024-09-15T21:46:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.508+0000 7f8d2a59c640 1 --2- 172.21.15.100:0/682152920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c073700 0x7f8d2c14bc20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8d2000c3d0 tx=0x7f8d2000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.509+0000 7f8d137fe640 1 -- 172.21.15.100:0/682152920 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d20017070 con 0x7f8d2c073700 2024-09-15T21:46:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.509+0000 7f8d137fe640 1 -- 172.21.15.100:0/682152920 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d200052d0 con 0x7f8d2c073700 2024-09-15T21:46:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.509+0000 7f8d3123c640 1 -- 172.21.15.100:0/682152920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2c0cd140 con 0x7f8d2c073700 2024-09-15T21:46:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.509+0000 7f8d137fe640 1 -- 172.21.15.100:0/682152920 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d20003660 con 0x7f8d2c073700 2024-09-15T21:46:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.509+0000 7f8d3123c640 1 -- 172.21.15.100:0/682152920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d2c1e1200 con 0x7f8d2c073700 2024-09-15T21:46:28.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.510+0000 7f8d3123c640 1 -- 172.21.15.100:0/682152920 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d2c0ced90 con 0x7f8d2c073700 2024-09-15T21:46:28.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.510+0000 7f8d137fe640 1 -- 172.21.15.100:0/682152920 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8d2001a070 con 0x7f8d2c073700 2024-09-15T21:46:28.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.511+0000 7f8d137fe640 1 --2- 172.21.15.100:0/682152920 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8d1806c2e0 0x7f8d1806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.511+0000 7f8d137fe640 1 -- 172.21.15.100:0/682152920 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8d2008a780 con 0x7f8d2c073700 2024-09-15T21:46:28.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.511+0000 7f8d2ad9d640 1 --2- 172.21.15.100:0/682152920 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8d1806c2e0 0x7f8d1806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.511+0000 7f8d2ad9d640 1 --2- 172.21.15.100:0/682152920 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8d1806c2e0 0x7f8d1806e7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c033000 tx=0x7f8d1c009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:28.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.515+0000 7f8d137fe640 1 -- 172.21.15.100:0/682152920 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8d200588a0 con 0x7f8d2c073700 2024-09-15T21:46:28.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.679+0000 7f8d3123c640 1 -- 172.21.15.100:0/682152920 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8d2c1e0b40 con 0x7f8d1806c2e0 2024-09-15T21:46:28.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.685+0000 7f8d137fe640 1 -- 172.21.15.100:0/682152920 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248805 (secure 0 0 0) 0x7f8d1c037040 con 0x7f8d1806c2e0 2024-09-15T21:46:28.692 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:28.695 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-15T21:46:28.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.688+0000 7f8d3123c640 1 -- 172.21.15.100:0/682152920 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8d1806c2e0 msgr2=0x7f8d1806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:28.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.688+0000 7f8d3123c640 1 --2- 172.21.15.100:0/682152920 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8d1806c2e0 0x7f8d1806e7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c033000 tx=0x7f8d1c009000 comp rx=0 tx=0).stop 2024-09-15T21:46:28.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.688+0000 7f8d3123c640 1 -- 172.21.15.100:0/682152920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c073700 msgr2=0x7f8d2c14bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:28.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.688+0000 7f8d3123c640 1 --2- 172.21.15.100:0/682152920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c073700 0x7f8d2c14bc20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8d2000c3d0 tx=0x7f8d2000c890 comp rx=0 tx=0).stop 2024-09-15T21:46:28.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.688+0000 7f8d3123c640 1 -- 172.21.15.100:0/682152920 shutdown_connections 2024-09-15T21:46:28.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.688+0000 7f8d3123c640 1 --2- 172.21.15.100:0/682152920 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8d1806c2e0 0x7f8d1806e7a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.688+0000 7f8d3123c640 1 --2- 172.21.15.100:0/682152920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d2c0c8a10 0x7f8d2c14ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.688+0000 7f8d3123c640 1 --2- 172.21.15.100:0/682152920 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d2c073700 0x7f8d2c14bc20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.688+0000 7f8d3123c640 1 --2- 172.21.15.100:0/682152920 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8d2c073320 0x7f8d2c0cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.688+0000 7f8d3123c640 1 -- 172.21.15.100:0/682152920 >> 172.21.15.100:0/682152920 conn(0x7f8d2c070ca0 msgr2=0x7f8d2c0c8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:28.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.689+0000 7f8d3123c640 1 -- 172.21.15.100:0/682152920 shutdown_connections 2024-09-15T21:46:28.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.689+0000 7f8d3123c640 1 -- 172.21.15.100:0/682152920 wait complete. 2024-09-15T21:46:28.701 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-15T21:46:27.804526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218908,"kb_used_data":2684,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755812,"statfs":{"total":773094113280,"available":772869951488,"internally_reserved":0,"allocated":2748416,"data_stored":1686550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000848"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.421348+0000","last_change":"2024-09-15T21:46:22.421348+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.421348+0000","last_undegraded":"2024-09-15T21:46:22.421348+0000","last_fullsized":"2024-09-15T21:46:22.421348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808513+0000","last_change":"2024-09-15T21:46:23.418716+0000","last_active":"2024-09-15T21:46:24.808513+0000","last_peered":"2024-09-15T21:46:24.808513+0000","last_clean":"2024-09-15T21:46:24.808513+0000","last_became_active":"2024-09-15T21:46:23.418471+0000","last_became_peered":"2024-09-15T21:46:23.418471+0000","last_unstale":"2024-09-15T21:46:24.808513+0000","last_undegraded":"2024-09-15T21:46:24.808513+0000","last_fullsized":"2024-09-15T21:46:24.808513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:49.599801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431027+0000","last_change":"2024-09-15T21:46:23.426404+0000","last_active":"2024-09-15T21:46:23.431027+0000","last_peered":"2024-09-15T21:46:23.431027+0000","last_clean":"2024-09-15T21:46:23.431027+0000","last_became_active":"2024-09-15T21:46:23.426146+0000","last_became_peered":"2024-09-15T21:46:23.426146+0000","last_unstale":"2024-09-15T21:46:23.431027+0000","last_undegraded":"2024-09-15T21:46:23.431027+0000","last_fullsized":"2024-09-15T21:46:23.431027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:40.061508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419990+0000","last_change":"2024-09-15T21:46:22.419990+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419990+0000","last_undegraded":"2024-09-15T21:46:22.419990+0000","last_fullsized":"2024-09-15T21:46:22.419990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418695+0000","last_change":"2024-09-15T21:46:22.418695+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418695+0000","last_undegraded":"2024-09-15T21:46:22.418695+0000","last_fullsized":"2024-09-15T21:46:22.418695+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418843+0000","last_change":"2024-09-15T21:46:22.418843+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418843+0000","last_undegraded":"2024-09-15T21:46:22.418843+0000","last_fullsized":"2024-09-15T21:46:22.418843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419832+0000","last_change":"2024-09-15T21:46:22.419832+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419832+0000","last_undegraded":"2024-09-15T21:46:22.419832+0000","last_fullsized":"2024-09-15T21:46:22.419832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.438915+0000","last_change":"2024-09-15T21:46:23.437143+0000","last_active":"2024-09-15T21:46:23.438915+0000","last_peered":"2024-09-15T21:46:23.438915+0000","last_clean":"2024-09-15T21:46:23.438915+0000","last_became_active":"2024-09-15T21:46:23.436864+0000","last_became_peered":"2024-09-15T21:46:23.436864+0000","last_unstale":"2024-09-15T21:46:23.438915+0000","last_undegraded":"2024-09-15T21:46:23.438915+0000","last_fullsized":"2024-09-15T21:46:23.438915+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:12:37.128024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.420920+0000","last_change":"2024-09-15T21:46:22.420920+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.420920+0000","last_undegraded":"2024-09-15T21:46:22.420920+0000","last_fullsized":"2024-09-15T21:46:22.420920+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.422403+0000","last_change":"2024-09-15T21:46:22.422403+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.422403+0000","last_undegraded":"2024-09-15T21:46:22.422403+0000","last_fullsized":"2024-09-15T21:46:22.422403+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418976+0000","last_change":"2024-09-15T21:46:22.418976+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418976+0000","last_undegraded":"2024-09-15T21:46:22.418976+0000","last_fullsized":"2024-09-15T21:46:22.418976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415544+0000","last_change":"2024-09-15T21:46:22.415544+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415544+0000","last_undegraded":"2024-09-15T21:46:22.415544+0000","last_fullsized":"2024-09-15T21:46:22.415544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431804+0000","last_change":"2024-09-15T21:46:23.427841+0000","last_active":"2024-09-15T21:46:23.431804+0000","last_peered":"2024-09-15T21:46:23.431804+0000","last_clean":"2024-09-15T21:46:23.431804+0000","last_became_active":"2024-09-15T21:46:23.427621+0000","last_became_peered":"2024-09-15T21:46:23.427621+0000","last_unstale":"2024-09-15T21:46:23.431804+0000","last_undegraded":"2024-09-15T21:46:23.431804+0000","last_fullsized":"2024-09-15T21:46:23.431804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:17:35.795757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.428658+0000","last_change":"2024-09-15T21:46:23.424707+0000","last_active":"2024-09-15T21:46:23.428658+0000","last_peered":"2024-09-15T21:46:23.428658+0000","last_clean":"2024-09-15T21:46:23.428658+0000","last_became_active":"2024-09-15T21:46:23.424447+0000","last_became_peered":"2024-09-15T21:46:23.424447+0000","last_unstale":"2024-09-15T21:46:23.428658+0000","last_undegraded":"2024-09-15T21:46:23.428658+0000","last_fullsized":"2024-09-15T21:46:23.428658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:28:21.800777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808321+0000","last_change":"2024-09-15T21:46:23.430297+0000","last_active":"2024-09-15T21:46:24.808321+0000","last_peered":"2024-09-15T21:46:24.808321+0000","last_clean":"2024-09-15T21:46:24.808321+0000","last_became_active":"2024-09-15T21:46:23.430070+0000","last_became_peered":"2024-09-15T21:46:23.430070+0000","last_unstale":"2024-09-15T21:46:24.808321+0000","last_undegraded":"2024-09-15T21:46:24.808321+0000","last_fullsized":"2024-09-15T21:46:24.808321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:03:19.615022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415614+0000","last_change":"2024-09-15T21:46:22.415614+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415614+0000","last_undegraded":"2024-09-15T21:46:22.415614+0000","last_fullsized":"2024-09-15T21:46:22.415614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415415+0000","last_change":"2024-09-15T21:46:22.415415+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415415+0000","last_undegraded":"2024-09-15T21:46:22.415415+0000","last_fullsized":"2024-09-15T21:46:22.415415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415507+0000","last_change":"2024-09-15T21:46:22.415507+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415507+0000","last_undegraded":"2024-09-15T21:46:22.415507+0000","last_fullsized":"2024-09-15T21:46:22.415507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418352+0000","last_change":"2024-09-15T21:46:22.418352+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418352+0000","last_undegraded":"2024-09-15T21:46:22.418352+0000","last_fullsized":"2024-09-15T21:46:22.418352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.420228+0000","last_change":"2024-09-15T21:46:22.420228+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.420228+0000","last_undegraded":"2024-09-15T21:46:22.420228+0000","last_fullsized":"2024-09-15T21:46:22.420228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433725+0000","last_change":"2024-09-15T21:46:23.428323+0000","last_active":"2024-09-15T21:46:23.433725+0000","last_peered":"2024-09-15T21:46:23.433725+0000","last_clean":"2024-09-15T21:46:23.433725+0000","last_became_active":"2024-09-15T21:46:23.428016+0000","last_became_peered":"2024-09-15T21:46:23.428016+0000","last_unstale":"2024-09-15T21:46:23.433725+0000","last_undegraded":"2024-09-15T21:46:23.433725+0000","last_fullsized":"2024-09-15T21:46:23.433725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:24:06.673523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418399+0000","last_change":"2024-09-15T21:46:22.418399+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418399+0000","last_undegraded":"2024-09-15T21:46:22.418399+0000","last_fullsized":"2024-09-15T21:46:22.418399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433661+0000","last_change":"2024-09-15T21:46:23.431527+0000","last_active":"2024-09-15T21:46:23.433661+0000","last_peered":"2024-09-15T21:46:23.433661+0000","last_clean":"2024-09-15T21:46:23.433661+0000","last_became_active":"2024-09-15T21:46:23.430719+0000","last_became_peered":"2024-09-15T21:46:23.430719+0000","last_unstale":"2024-09-15T21:46:23.433661+0000","last_undegraded":"2024-09-15T21:46:23.433661+0000","last_fullsized":"2024-09-15T21:46:23.433661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:25.362126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415180+0000","last_change":"2024-09-15T21:46:22.415180+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415180+0000","last_undegraded":"2024-09-15T21:46:22.415180+0000","last_fullsized":"2024-09-15T21:46:22.415180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.436122+0000","last_change":"2024-09-15T21:46:23.432646+0000","last_active":"2024-09-15T21:46:23.436122+0000","last_peered":"2024-09-15T21:46:23.436122+0000","last_clean":"2024-09-15T21:46:23.436122+0000","last_became_active":"2024-09-15T21:46:23.432317+0000","last_became_peered":"2024-09-15T21:46:23.432317+0000","last_unstale":"2024-09-15T21:46:23.436122+0000","last_undegraded":"2024-09-15T21:46:23.436122+0000","last_fullsized":"2024-09-15T21:46:23.436122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:34.359059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.421958+0000","last_change":"2024-09-15T21:46:22.421958+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.421958+0000","last_undegraded":"2024-09-15T21:46:22.421958+0000","last_fullsized":"2024-09-15T21:46:22.421958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808369+0000","last_change":"2024-09-15T21:46:23.421749+0000","last_active":"2024-09-15T21:46:24.808369+0000","last_peered":"2024-09-15T21:46:24.808369+0000","last_clean":"2024-09-15T21:46:24.808369+0000","last_became_active":"2024-09-15T21:46:23.421501+0000","last_became_peered":"2024-09-15T21:46:23.421501+0000","last_unstale":"2024-09-15T21:46:24.808369+0000","last_undegraded":"2024-09-15T21:46:24.808369+0000","last_fullsized":"2024-09-15T21:46:24.808369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:55.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.436035+0000","last_change":"2024-09-15T21:46:23.432591+0000","last_active":"2024-09-15T21:46:23.436035+0000","last_peered":"2024-09-15T21:46:23.436035+0000","last_clean":"2024-09-15T21:46:23.436035+0000","last_became_active":"2024-09-15T21:46:23.432326+0000","last_became_peered":"2024-09-15T21:46:23.432326+0000","last_unstale":"2024-09-15T21:46:23.436035+0000","last_undegraded":"2024-09-15T21:46:23.436035+0000","last_fullsized":"2024-09-15T21:46:23.436035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:46:56.508701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.428496+0000","last_change":"2024-09-15T21:46:23.423428+0000","last_active":"2024-09-15T21:46:23.428496+0000","last_peered":"2024-09-15T21:46:23.428496+0000","last_clean":"2024-09-15T21:46:23.428496+0000","last_became_active":"2024-09-15T21:46:23.422734+0000","last_became_peered":"2024-09-15T21:46:23.422734+0000","last_unstale":"2024-09-15T21:46:23.428496+0000","last_undegraded":"2024-09-15T21:46:23.428496+0000","last_fullsized":"2024-09-15T21:46:23.428496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:02.082197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.417891+0000","last_change":"2024-09-15T21:46:22.417891+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.417891+0000","last_undegraded":"2024-09-15T21:46:22.417891+0000","last_fullsized":"2024-09-15T21:46:22.417891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414635+0000","last_change":"2024-09-15T21:46:22.414635+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414635+0000","last_undegraded":"2024-09-15T21:46:22.414635+0000","last_fullsized":"2024-09-15T21:46:22.414635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418469+0000","last_change":"2024-09-15T21:46:22.418469+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418469+0000","last_undegraded":"2024-09-15T21:46:22.418469+0000","last_fullsized":"2024-09-15T21:46:22.418469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414859+0000","last_change":"2024-09-15T21:46:22.414859+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414859+0000","last_undegraded":"2024-09-15T21:46:22.414859+0000","last_fullsized":"2024-09-15T21:46:22.414859+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.424245+0000","last_change":"2024-09-15T21:46:22.429407+0000","last_active":"2024-09-15T21:46:23.424245+0000","last_peered":"2024-09-15T21:46:23.424245+0000","last_clean":"2024-09-15T21:46:23.424245+0000","last_became_active":"2024-09-15T21:46:22.429107+0000","last_became_peered":"2024-09-15T21:46:22.429107+0000","last_unstale":"2024-09-15T21:46:23.424245+0000","last_undegraded":"2024-09-15T21:46:23.424245+0000","last_fullsized":"2024-09-15T21:46:23.424245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:14:54.680349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.438772+0000","last_change":"2024-09-15T21:46:23.434198+0000","last_active":"2024-09-15T21:46:23.438772+0000","last_peered":"2024-09-15T21:46:23.438772+0000","last_clean":"2024-09-15T21:46:23.438772+0000","last_became_active":"2024-09-15T21:46:23.433944+0000","last_became_peered":"2024-09-15T21:46:23.433944+0000","last_unstale":"2024-09-15T21:46:23.438772+0000","last_undegraded":"2024-09-15T21:46:23.438772+0000","last_fullsized":"2024-09-15T21:46:23.438772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:20:05.953148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.428922+0000","last_change":"2024-09-15T21:46:22.425571+0000","last_active":"2024-09-15T21:46:22.428922+0000","last_peered":"2024-09-15T21:46:22.428922+0000","last_clean":"2024-09-15T21:46:22.428922+0000","last_became_active":"2024-09-15T21:46:22.425315+0000","last_became_peered":"2024-09-15T21:46:22.425315+0000","last_unstale":"2024-09-15T21:46:22.428922+0000","last_undegraded":"2024-09-15T21:46:22.428922+0000","last_fullsized":"2024-09-15T21:46:22.428922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:37.405710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.423270+0000","last_change":"2024-09-15T21:46:22.425492+0000","last_active":"2024-09-15T21:46:23.423270+0000","last_peered":"2024-09-15T21:46:23.423270+0000","last_clean":"2024-09-15T21:46:23.423270+0000","last_became_active":"2024-09-15T21:46:22.425226+0000","last_became_peered":"2024-09-15T21:46:22.425226+0000","last_unstale":"2024-09-15T21:46:23.423270+0000","last_undegraded":"2024-09-15T21:46:23.423270+0000","last_fullsized":"2024-09-15T21:46:23.423270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:24.322384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433622+0000","last_change":"2024-09-15T21:46:23.429140+0000","last_active":"2024-09-15T21:46:23.433622+0000","last_peered":"2024-09-15T21:46:23.433622+0000","last_clean":"2024-09-15T21:46:23.433622+0000","last_became_active":"2024-09-15T21:46:23.428902+0000","last_became_peered":"2024-09-15T21:46:23.428902+0000","last_unstale":"2024-09-15T21:46:23.433622+0000","last_undegraded":"2024-09-15T21:46:23.433622+0000","last_fullsized":"2024-09-15T21:46:23.433622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:44:54.129937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.431357+0000","last_change":"2024-09-15T21:46:22.428319+0000","last_active":"2024-09-15T21:46:22.431357+0000","last_peered":"2024-09-15T21:46:22.431357+0000","last_clean":"2024-09-15T21:46:22.431357+0000","last_became_active":"2024-09-15T21:46:22.428061+0000","last_became_peered":"2024-09-15T21:46:22.428061+0000","last_unstale":"2024-09-15T21:46:22.431357+0000","last_undegraded":"2024-09-15T21:46:22.431357+0000","last_fullsized":"2024-09-15T21:46:22.431357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:34.432046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808315+0000","last_change":"2024-09-15T21:46:23.427442+0000","last_active":"2024-09-15T21:46:24.808315+0000","last_peered":"2024-09-15T21:46:24.808315+0000","last_clean":"2024-09-15T21:46:24.808315+0000","last_became_active":"2024-09-15T21:46:23.427200+0000","last_became_peered":"2024-09-15T21:46:23.427200+0000","last_unstale":"2024-09-15T21:46:24.808315+0000","last_undegraded":"2024-09-15T21:46:24.808315+0000","last_fullsized":"2024-09-15T21:46:24.808315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:27.248545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433993+0000","last_change":"2024-09-15T21:46:22.430599+0000","last_active":"2024-09-15T21:46:22.433993+0000","last_peered":"2024-09-15T21:46:22.433993+0000","last_clean":"2024-09-15T21:46:22.433993+0000","last_became_active":"2024-09-15T21:46:22.430327+0000","last_became_peered":"2024-09-15T21:46:22.430327+0000","last_unstale":"2024-09-15T21:46:22.433993+0000","last_undegraded":"2024-09-15T21:46:22.433993+0000","last_fullsized":"2024-09-15T21:46:22.433993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:42:31.882474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.416669+0000","last_change":"2024-09-15T21:46:22.416669+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.416669+0000","last_undegraded":"2024-09-15T21:46:22.416669+0000","last_fullsized":"2024-09-15T21:46:22.416669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.429815+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:22.429815+0000","last_peered":"2024-09-15T21:46:22.429815+0000","last_clean":"2024-09-15T21:46:22.429815+0000","last_became_active":"2024-09-15T21:46:22.425236+0000","last_became_peered":"2024-09-15T21:46:22.425236+0000","last_unstale":"2024-09-15T21:46:22.429815+0000","last_undegraded":"2024-09-15T21:46:22.429815+0000","last_fullsized":"2024-09-15T21:46:22.429815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:01.525976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433802+0000","last_change":"2024-09-15T21:46:23.429587+0000","last_active":"2024-09-15T21:46:23.433802+0000","last_peered":"2024-09-15T21:46:23.433802+0000","last_clean":"2024-09-15T21:46:23.433802+0000","last_became_active":"2024-09-15T21:46:23.429334+0000","last_became_peered":"2024-09-15T21:46:23.429334+0000","last_unstale":"2024-09-15T21:46:23.433802+0000","last_undegraded":"2024-09-15T21:46:23.433802+0000","last_fullsized":"2024-09-15T21:46:23.433802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:31.263309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808325+0000","last_change":"2024-09-15T21:46:22.421964+0000","last_active":"2024-09-15T21:46:24.808325+0000","last_peered":"2024-09-15T21:46:24.808325+0000","last_clean":"2024-09-15T21:46:24.808325+0000","last_became_active":"2024-09-15T21:46:22.421661+0000","last_became_peered":"2024-09-15T21:46:22.421661+0000","last_unstale":"2024-09-15T21:46:24.808325+0000","last_undegraded":"2024-09-15T21:46:24.808325+0000","last_fullsized":"2024-09-15T21:46:24.808325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:07.395070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414271+0000","last_change":"2024-09-15T21:46:22.414271+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414271+0000","last_undegraded":"2024-09-15T21:46:22.414271+0000","last_fullsized":"2024-09-15T21:46:22.414271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433356+0000","last_change":"2024-09-15T21:46:22.426151+0000","last_active":"2024-09-15T21:46:22.433356+0000","last_peered":"2024-09-15T21:46:22.433356+0000","last_clean":"2024-09-15T21:46:22.433356+0000","last_became_active":"2024-09-15T21:46:22.425866+0000","last_became_peered":"2024-09-15T21:46:22.425866+0000","last_unstale":"2024-09-15T21:46:22.433356+0000","last_undegraded":"2024-09-15T21:46:22.433356+0000","last_fullsized":"2024-09-15T21:46:22.433356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:18.129618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419893+0000","last_change":"2024-09-15T21:46:22.419893+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419893+0000","last_undegraded":"2024-09-15T21:46:22.419893+0000","last_fullsized":"2024-09-15T21:46:22.419893+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.417870+0000","last_change":"2024-09-15T21:46:22.427415+0000","last_active":"2024-09-15T21:46:23.417870+0000","last_peered":"2024-09-15T21:46:23.417870+0000","last_clean":"2024-09-15T21:46:23.417870+0000","last_became_active":"2024-09-15T21:46:22.427087+0000","last_became_peered":"2024-09-15T21:46:22.427087+0000","last_unstale":"2024-09-15T21:46:23.417870+0000","last_undegraded":"2024-09-15T21:46:23.417870+0000","last_fullsized":"2024-09-15T21:46:23.417870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:09.365202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431262+0000","last_change":"2024-09-15T21:46:23.427238+0000","last_active":"2024-09-15T21:46:23.431262+0000","last_peered":"2024-09-15T21:46:23.431262+0000","last_clean":"2024-09-15T21:46:23.431262+0000","last_became_active":"2024-09-15T21:46:23.426993+0000","last_became_peered":"2024-09-15T21:46:23.426993+0000","last_unstale":"2024-09-15T21:46:23.431262+0000","last_undegraded":"2024-09-15T21:46:23.431262+0000","last_fullsized":"2024-09-15T21:46:23.431262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:36.125958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.431315+0000","last_change":"2024-09-15T21:46:22.427164+0000","last_active":"2024-09-15T21:46:22.431315+0000","last_peered":"2024-09-15T21:46:22.431315+0000","last_clean":"2024-09-15T21:46:22.431315+0000","last_became_active":"2024-09-15T21:46:22.426917+0000","last_became_peered":"2024-09-15T21:46:22.426917+0000","last_unstale":"2024-09-15T21:46:22.431315+0000","last_undegraded":"2024-09-15T21:46:22.431315+0000","last_fullsized":"2024-09-15T21:46:22.431315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:12:47.575708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418200+0000","last_change":"2024-09-15T21:46:22.418200+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418200+0000","last_undegraded":"2024-09-15T21:46:22.418200+0000","last_fullsized":"2024-09-15T21:46:22.418200+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.428957+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:22.428957+0000","last_peered":"2024-09-15T21:46:22.428957+0000","last_clean":"2024-09-15T21:46:22.428957+0000","last_became_active":"2024-09-15T21:46:22.425237+0000","last_became_peered":"2024-09-15T21:46:22.425237+0000","last_unstale":"2024-09-15T21:46:22.428957+0000","last_undegraded":"2024-09-15T21:46:22.428957+0000","last_fullsized":"2024-09-15T21:46:22.428957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:46.301372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.417788+0000","last_change":"2024-09-15T21:46:22.417788+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.417788+0000","last_undegraded":"2024-09-15T21:46:22.417788+0000","last_fullsized":"2024-09-15T21:46:22.417788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.423016+0000","last_change":"2024-09-15T21:46:22.419905+0000","last_active":"2024-09-15T21:46:22.423016+0000","last_peered":"2024-09-15T21:46:22.423016+0000","last_clean":"2024-09-15T21:46:22.423016+0000","last_became_active":"2024-09-15T21:46:22.419620+0000","last_became_peered":"2024-09-15T21:46:22.419620+0000","last_unstale":"2024-09-15T21:46:22.423016+0000","last_undegraded":"2024-09-15T21:46:22.423016+0000","last_fullsized":"2024-09-15T21:46:22.423016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:41.513967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419332+0000","last_change":"2024-09-15T21:46:22.419332+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419332+0000","last_undegraded":"2024-09-15T21:46:22.419332+0000","last_fullsized":"2024-09-15T21:46:22.419332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433527+0000","last_change":"2024-09-15T21:46:22.429661+0000","last_active":"2024-09-15T21:46:22.433527+0000","last_peered":"2024-09-15T21:46:22.433527+0000","last_clean":"2024-09-15T21:46:22.433527+0000","last_became_active":"2024-09-15T21:46:22.429250+0000","last_became_peered":"2024-09-15T21:46:22.429250+0000","last_unstale":"2024-09-15T21:46:22.433527+0000","last_undegraded":"2024-09-15T21:46:22.433527+0000","last_fullsized":"2024-09-15T21:46:22.433527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:52:16.852861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418349+0000","last_change":"2024-09-15T21:46:22.418349+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418349+0000","last_undegraded":"2024-09-15T21:46:22.418349+0000","last_fullsized":"2024-09-15T21:46:22.418349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.430022+0000","last_change":"2024-09-15T21:46:22.425870+0000","last_active":"2024-09-15T21:46:22.430022+0000","last_peered":"2024-09-15T21:46:22.430022+0000","last_clean":"2024-09-15T21:46:22.430022+0000","last_became_active":"2024-09-15T21:46:22.425588+0000","last_became_peered":"2024-09-15T21:46:22.425588+0000","last_unstale":"2024-09-15T21:46:22.430022+0000","last_undegraded":"2024-09-15T21:46:22.430022+0000","last_fullsized":"2024-09-15T21:46:22.430022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:53.982102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.420429+0000","last_change":"2024-09-15T21:46:22.420429+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.420429+0000","last_undegraded":"2024-09-15T21:46:22.420429+0000","last_fullsized":"2024-09-15T21:46:22.420429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808420+0000","last_change":"2024-09-15T21:46:22.425072+0000","last_active":"2024-09-15T21:46:24.808420+0000","last_peered":"2024-09-15T21:46:24.808420+0000","last_clean":"2024-09-15T21:46:24.808420+0000","last_became_active":"2024-09-15T21:46:22.424771+0000","last_became_peered":"2024-09-15T21:46:22.424771+0000","last_unstale":"2024-09-15T21:46:24.808420+0000","last_undegraded":"2024-09-15T21:46:24.808420+0000","last_fullsized":"2024-09-15T21:46:24.808420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:26.256785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415969+0000","last_change":"2024-09-15T21:46:22.415969+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415969+0000","last_undegraded":"2024-09-15T21:46:22.415969+0000","last_fullsized":"2024-09-15T21:46:22.415969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.432304+0000","last_change":"2024-09-15T21:46:22.428668+0000","last_active":"2024-09-15T21:46:22.432304+0000","last_peered":"2024-09-15T21:46:22.432304+0000","last_clean":"2024-09-15T21:46:22.432304+0000","last_became_active":"2024-09-15T21:46:22.428341+0000","last_became_peered":"2024-09-15T21:46:22.428341+0000","last_unstale":"2024-09-15T21:46:22.432304+0000","last_undegraded":"2024-09-15T21:46:22.432304+0000","last_fullsized":"2024-09-15T21:46:22.432304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:25.844475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418811+0000","last_change":"2024-09-15T21:46:22.418811+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418811+0000","last_undegraded":"2024-09-15T21:46:22.418811+0000","last_fullsized":"2024-09-15T21:46:22.418811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":167,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.422877+0000","last_change":"2024-09-15T21:46:14.328094+0000","last_active":"2024-09-15T21:46:22.422877+0000","last_peered":"2024-09-15T21:46:22.422877+0000","last_clean":"2024-09-15T21:46:22.422877+0000","last_became_active":"2024-09-15T21:46:14.326991+0000","last_became_peered":"2024-09-15T21:46:14.326991+0000","last_unstale":"2024-09-15T21:46:22.422877+0000","last_undegraded":"2024-09-15T21:46:22.422877+0000","last_fullsized":"2024-09-15T21:46:22.422877+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_clean_scrub_stamp":"2024-09-15T21:46:12.223400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:54:27.704130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.431406+0000","last_change":"2024-09-15T21:46:22.428887+0000","last_active":"2024-09-15T21:46:22.431406+0000","last_peered":"2024-09-15T21:46:22.431406+0000","last_clean":"2024-09-15T21:46:22.431406+0000","last_became_active":"2024-09-15T21:46:22.428619+0000","last_became_peered":"2024-09-15T21:46:22.428619+0000","last_unstale":"2024-09-15T21:46:22.431406+0000","last_undegraded":"2024-09-15T21:46:22.431406+0000","last_fullsized":"2024-09-15T21:46:22.431406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:43.510542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.438679+0000","last_change":"2024-09-15T21:46:23.435459+0000","last_active":"2024-09-15T21:46:23.438679+0000","last_peered":"2024-09-15T21:46:23.438679+0000","last_clean":"2024-09-15T21:46:23.438679+0000","last_became_active":"2024-09-15T21:46:23.435141+0000","last_became_peered":"2024-09-15T21:46:23.435141+0000","last_unstale":"2024-09-15T21:46:23.438679+0000","last_undegraded":"2024-09-15T21:46:23.438679+0000","last_fullsized":"2024-09-15T21:46:23.438679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:38.434075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433564+0000","last_change":"2024-09-15T21:46:22.431113+0000","last_active":"2024-09-15T21:46:22.433564+0000","last_peered":"2024-09-15T21:46:22.433564+0000","last_clean":"2024-09-15T21:46:22.433564+0000","last_became_active":"2024-09-15T21:46:22.430832+0000","last_became_peered":"2024-09-15T21:46:22.430832+0000","last_unstale":"2024-09-15T21:46:22.433564+0000","last_undegraded":"2024-09-15T21:46:22.433564+0000","last_fullsized":"2024-09-15T21:46:22.433564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:08.068054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433942+0000","last_change":"2024-09-15T21:46:23.430396+0000","last_active":"2024-09-15T21:46:23.433942+0000","last_peered":"2024-09-15T21:46:23.433942+0000","last_clean":"2024-09-15T21:46:23.433942+0000","last_became_active":"2024-09-15T21:46:23.430138+0000","last_became_peered":"2024-09-15T21:46:23.430138+0000","last_unstale":"2024-09-15T21:46:23.433942+0000","last_undegraded":"2024-09-15T21:46:23.433942+0000","last_fullsized":"2024-09-15T21:46:23.433942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:04.056726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.430035+0000","last_change":"2024-09-15T21:46:22.425808+0000","last_active":"2024-09-15T21:46:22.430035+0000","last_peered":"2024-09-15T21:46:22.430035+0000","last_clean":"2024-09-15T21:46:22.430035+0000","last_became_active":"2024-09-15T21:46:22.425510+0000","last_became_peered":"2024-09-15T21:46:22.425510+0000","last_unstale":"2024-09-15T21:46:22.430035+0000","last_undegraded":"2024-09-15T21:46:22.430035+0000","last_fullsized":"2024-09-15T21:46:22.430035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:13.502626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808338+0000","last_change":"2024-09-15T21:46:23.427051+0000","last_active":"2024-09-15T21:46:24.808338+0000","last_peered":"2024-09-15T21:46:24.808338+0000","last_clean":"2024-09-15T21:46:24.808338+0000","last_became_active":"2024-09-15T21:46:23.426185+0000","last_became_peered":"2024-09-15T21:46:23.426185+0000","last_unstale":"2024-09-15T21:46:24.808338+0000","last_undegraded":"2024-09-15T21:46:24.808338+0000","last_fullsized":"2024-09-15T21:46:24.808338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:48.318405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.429782+0000","last_change":"2024-09-15T21:46:22.424645+0000","last_active":"2024-09-15T21:46:22.429782+0000","last_peered":"2024-09-15T21:46:22.429782+0000","last_clean":"2024-09-15T21:46:22.429782+0000","last_became_active":"2024-09-15T21:46:22.421229+0000","last_became_peered":"2024-09-15T21:46:22.421229+0000","last_unstale":"2024-09-15T21:46:22.429782+0000","last_undegraded":"2024-09-15T21:46:22.429782+0000","last_fullsized":"2024-09-15T21:46:22.429782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:02.298935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.413899+0000","last_change":"2024-09-15T21:46:22.413899+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.413899+0000","last_undegraded":"2024-09-15T21:46:22.413899+0000","last_fullsized":"2024-09-15T21:46:22.413899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808438+0000","last_change":"2024-09-15T21:46:22.426492+0000","last_active":"2024-09-15T21:46:24.808438+0000","last_peered":"2024-09-15T21:46:24.808438+0000","last_clean":"2024-09-15T21:46:24.808438+0000","last_became_active":"2024-09-15T21:46:22.426210+0000","last_became_peered":"2024-09-15T21:46:22.426210+0000","last_unstale":"2024-09-15T21:46:24.808438+0000","last_undegraded":"2024-09-15T21:46:24.808438+0000","last_fullsized":"2024-09-15T21:46:24.808438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:05.248091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414998+0000","last_change":"2024-09-15T21:46:22.414998+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414998+0000","last_undegraded":"2024-09-15T21:46:22.414998+0000","last_fullsized":"2024-09-15T21:46:22.414998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433499+0000","last_change":"2024-09-15T21:46:22.430710+0000","last_active":"2024-09-15T21:46:22.433499+0000","last_peered":"2024-09-15T21:46:22.433499+0000","last_clean":"2024-09-15T21:46:22.433499+0000","last_became_active":"2024-09-15T21:46:22.430375+0000","last_became_peered":"2024-09-15T21:46:22.430375+0000","last_unstale":"2024-09-15T21:46:22.433499+0000","last_undegraded":"2024-09-15T21:46:22.433499+0000","last_fullsized":"2024-09-15T21:46:22.433499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:17.852485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808447+0000","last_change":"2024-09-15T21:46:23.426729+0000","last_active":"2024-09-15T21:46:24.808447+0000","last_peered":"2024-09-15T21:46:24.808447+0000","last_clean":"2024-09-15T21:46:24.808447+0000","last_became_active":"2024-09-15T21:46:23.426495+0000","last_became_peered":"2024-09-15T21:46:23.426495+0000","last_unstale":"2024-09-15T21:46:24.808447+0000","last_undegraded":"2024-09-15T21:46:24.808447+0000","last_fullsized":"2024-09-15T21:46:24.808447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:03.277997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808436+0000","last_change":"2024-09-15T21:46:22.427137+0000","last_active":"2024-09-15T21:46:24.808436+0000","last_peered":"2024-09-15T21:46:24.808436+0000","last_clean":"2024-09-15T21:46:24.808436+0000","last_became_active":"2024-09-15T21:46:22.426855+0000","last_became_peered":"2024-09-15T21:46:22.426855+0000","last_unstale":"2024-09-15T21:46:24.808436+0000","last_undegraded":"2024-09-15T21:46:24.808436+0000","last_fullsized":"2024-09-15T21:46:24.808436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:40:14.277940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431767+0000","last_change":"2024-09-15T21:46:23.427327+0000","last_active":"2024-09-15T21:46:23.431767+0000","last_peered":"2024-09-15T21:46:23.431767+0000","last_clean":"2024-09-15T21:46:23.431767+0000","last_became_active":"2024-09-15T21:46:23.427096+0000","last_became_peered":"2024-09-15T21:46:23.427096+0000","last_unstale":"2024-09-15T21:46:23.431767+0000","last_undegraded":"2024-09-15T21:46:23.431767+0000","last_fullsized":"2024-09-15T21:46:23.431767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:43.650053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433399+0000","last_change":"2024-09-15T21:46:22.427798+0000","last_active":"2024-09-15T21:46:22.433399+0000","last_peered":"2024-09-15T21:46:22.433399+0000","last_clean":"2024-09-15T21:46:22.433399+0000","last_became_active":"2024-09-15T21:46:22.427537+0000","last_became_peered":"2024-09-15T21:46:22.427537+0000","last_unstale":"2024-09-15T21:46:22.433399+0000","last_undegraded":"2024-09-15T21:46:22.433399+0000","last_fullsized":"2024-09-15T21:46:22.433399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:36.227177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.428576+0000","last_change":"2024-09-15T21:46:23.424266+0000","last_active":"2024-09-15T21:46:23.428576+0000","last_peered":"2024-09-15T21:46:23.428576+0000","last_clean":"2024-09-15T21:46:23.428576+0000","last_became_active":"2024-09-15T21:46:23.424007+0000","last_became_peered":"2024-09-15T21:46:23.424007+0000","last_unstale":"2024-09-15T21:46:23.428576+0000","last_undegraded":"2024-09-15T21:46:23.428576+0000","last_fullsized":"2024-09-15T21:46:23.428576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:04:11.974067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433546+0000","last_change":"2024-09-15T21:46:22.430901+0000","last_active":"2024-09-15T21:46:22.433546+0000","last_peered":"2024-09-15T21:46:22.433546+0000","last_clean":"2024-09-15T21:46:22.433546+0000","last_became_active":"2024-09-15T21:46:22.430452+0000","last_became_peered":"2024-09-15T21:46:22.430452+0000","last_unstale":"2024-09-15T21:46:22.433546+0000","last_undegraded":"2024-09-15T21:46:22.433546+0000","last_fullsized":"2024-09-15T21:46:22.433546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:02.951631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414149+0000","last_change":"2024-09-15T21:46:22.414149+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414149+0000","last_undegraded":"2024-09-15T21:46:22.414149+0000","last_fullsized":"2024-09-15T21:46:22.414149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808380+0000","last_change":"2024-09-15T21:46:22.423795+0000","last_active":"2024-09-15T21:46:24.808380+0000","last_peered":"2024-09-15T21:46:24.808380+0000","last_clean":"2024-09-15T21:46:24.808380+0000","last_became_active":"2024-09-15T21:46:22.422327+0000","last_became_peered":"2024-09-15T21:46:22.422327+0000","last_unstale":"2024-09-15T21:46:24.808380+0000","last_undegraded":"2024-09-15T21:46:24.808380+0000","last_fullsized":"2024-09-15T21:46:24.808380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:16.670599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433600+0000","last_change":"2024-09-15T21:46:23.428634+0000","last_active":"2024-09-15T21:46:23.433600+0000","last_peered":"2024-09-15T21:46:23.433600+0000","last_clean":"2024-09-15T21:46:23.433600+0000","last_became_active":"2024-09-15T21:46:23.428301+0000","last_became_peered":"2024-09-15T21:46:23.428301+0000","last_unstale":"2024-09-15T21:46:23.433600+0000","last_undegraded":"2024-09-15T21:46:23.433600+0000","last_fullsized":"2024-09-15T21:46:23.433600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:17.178940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808304+0000","last_change":"2024-09-15T21:46:22.425810+0000","last_active":"2024-09-15T21:46:24.808304+0000","last_peered":"2024-09-15T21:46:24.808304+0000","last_clean":"2024-09-15T21:46:24.808304+0000","last_became_active":"2024-09-15T21:46:22.425478+0000","last_became_peered":"2024-09-15T21:46:22.425478+0000","last_unstale":"2024-09-15T21:46:24.808304+0000","last_undegraded":"2024-09-15T21:46:24.808304+0000","last_fullsized":"2024-09-15T21:46:24.808304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:53.296599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431785+0000","last_change":"2024-09-15T21:46:23.426054+0000","last_active":"2024-09-15T21:46:23.431785+0000","last_peered":"2024-09-15T21:46:23.431785+0000","last_clean":"2024-09-15T21:46:23.431785+0000","last_became_active":"2024-09-15T21:46:23.425547+0000","last_became_peered":"2024-09-15T21:46:23.425547+0000","last_unstale":"2024-09-15T21:46:23.431785+0000","last_undegraded":"2024-09-15T21:46:23.431785+0000","last_fullsized":"2024-09-15T21:46:23.431785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:25.738831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433353+0000","last_change":"2024-09-15T21:46:22.428524+0000","last_active":"2024-09-15T21:46:22.433353+0000","last_peered":"2024-09-15T21:46:22.433353+0000","last_clean":"2024-09-15T21:46:22.433353+0000","last_became_active":"2024-09-15T21:46:22.428043+0000","last_became_peered":"2024-09-15T21:46:22.428043+0000","last_unstale":"2024-09-15T21:46:22.433353+0000","last_undegraded":"2024-09-15T21:46:22.433353+0000","last_fullsized":"2024-09-15T21:46:22.433353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:53.400491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419544+0000","last_change":"2024-09-15T21:46:22.419544+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419544+0000","last_undegraded":"2024-09-15T21:46:22.419544+0000","last_fullsized":"2024-09-15T21:46:22.419544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415492+0000","last_change":"2024-09-15T21:46:22.415492+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415492+0000","last_undegraded":"2024-09-15T21:46:22.415492+0000","last_fullsized":"2024-09-15T21:46:22.415492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433558+0000","last_change":"2024-09-15T21:46:22.431172+0000","last_active":"2024-09-15T21:46:22.433558+0000","last_peered":"2024-09-15T21:46:22.433558+0000","last_clean":"2024-09-15T21:46:22.433558+0000","last_became_active":"2024-09-15T21:46:22.430800+0000","last_became_peered":"2024-09-15T21:46:22.430800+0000","last_unstale":"2024-09-15T21:46:22.433558+0000","last_undegraded":"2024-09-15T21:46:22.433558+0000","last_fullsized":"2024-09-15T21:46:22.433558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:52:07.231504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.417557+0000","last_change":"2024-09-15T21:46:22.417557+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.417557+0000","last_undegraded":"2024-09-15T21:46:22.417557+0000","last_fullsized":"2024-09-15T21:46:22.417557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433476+0000","last_change":"2024-09-15T21:46:22.429870+0000","last_active":"2024-09-15T21:46:22.433476+0000","last_peered":"2024-09-15T21:46:22.433476+0000","last_clean":"2024-09-15T21:46:22.433476+0000","last_became_active":"2024-09-15T21:46:22.429578+0000","last_became_peered":"2024-09-15T21:46:22.429578+0000","last_unstale":"2024-09-15T21:46:22.433476+0000","last_undegraded":"2024-09-15T21:46:22.433476+0000","last_fullsized":"2024-09-15T21:46:22.433476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:59.880081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433864+0000","last_change":"2024-09-15T21:46:23.429604+0000","last_active":"2024-09-15T21:46:23.433864+0000","last_peered":"2024-09-15T21:46:23.433864+0000","last_clean":"2024-09-15T21:46:23.433864+0000","last_became_active":"2024-09-15T21:46:23.429355+0000","last_became_peered":"2024-09-15T21:46:23.429355+0000","last_unstale":"2024-09-15T21:46:23.433864+0000","last_undegraded":"2024-09-15T21:46:23.433864+0000","last_fullsized":"2024-09-15T21:46:23.433864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:23:10.533918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.424314+0000","last_change":"2024-09-15T21:46:22.428529+0000","last_active":"2024-09-15T21:46:23.424314+0000","last_peered":"2024-09-15T21:46:23.424314+0000","last_clean":"2024-09-15T21:46:23.424314+0000","last_became_active":"2024-09-15T21:46:22.428263+0000","last_became_peered":"2024-09-15T21:46:22.428263+0000","last_unstale":"2024-09-15T21:46:23.424314+0000","last_undegraded":"2024-09-15T21:46:23.424314+0000","last_fullsized":"2024-09-15T21:46:23.424314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:22:40.160062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433487+0000","last_change":"2024-09-15T21:46:23.427968+0000","last_active":"2024-09-15T21:46:23.433487+0000","last_peered":"2024-09-15T21:46:23.433487+0000","last_clean":"2024-09-15T21:46:23.433487+0000","last_became_active":"2024-09-15T21:46:23.427546+0000","last_became_peered":"2024-09-15T21:46:23.427546+0000","last_unstale":"2024-09-15T21:46:23.433487+0000","last_undegraded":"2024-09-15T21:46:23.433487+0000","last_fullsized":"2024-09-15T21:46:23.433487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:45.544297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.417176+0000","last_change":"2024-09-15T21:46:22.428258+0000","last_active":"2024-09-15T21:46:23.417176+0000","last_peered":"2024-09-15T21:46:23.417176+0000","last_clean":"2024-09-15T21:46:23.417176+0000","last_became_active":"2024-09-15T21:46:22.427954+0000","last_became_peered":"2024-09-15T21:46:22.427954+0000","last_unstale":"2024-09-15T21:46:23.417176+0000","last_undegraded":"2024-09-15T21:46:23.417176+0000","last_fullsized":"2024-09-15T21:46:23.417176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:22.440456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476740,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":654251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":654251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":57608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":57608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":57608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":57608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":76958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":70658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:46:28.705 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:46:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.839+0000 7f6bab131640 1 Processor -- start 2024-09-15T21:46:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.839+0000 7f6bab131640 1 -- start start 2024-09-15T21:46:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6bab131640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6ba40cca30 0x7f6ba4071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6bab131640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6ba40cce10 0x7f6ba415af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6bab131640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ba40c91f0 con 0x7f6ba4070ee0 2024-09-15T21:46:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6bab131640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f6ba40c94d0 con 0x7f6ba40cce10 2024-09-15T21:46:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6bab131640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba40c97b0 con 0x7f6ba40cca30 2024-09-15T21:46:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba3fff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f6ba4070ee0 0x7f6ba4151450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:51784/0 (socket says 172.21.15.100:51784) 2024-09-15T21:46:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba3fff640 1 -- 172.21.15.100:0/3679125816 learned_addr learned my addr 172.21.15.100:0/3679125816 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba96ce640 1 --2- 172.21.15.100:0/3679125816 >> v2:172.21.15.122:3300/0 conn(0x7f6ba40cce10 0x7f6ba415af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba8ecd640 1 --2- 172.21.15.100:0/3679125816 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6ba40cca30 0x7f6ba4071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba37fe640 1 -- 172.21.15.100:0/3679125816 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1521333321 0 0) 0x7f6ba40c91f0 con 0x7f6ba4070ee0 2024-09-15T21:46:28.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba37fe640 1 -- 172.21.15.100:0/3679125816 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6b94003520 con 0x7f6ba4070ee0 2024-09-15T21:46:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba37fe640 1 -- 172.21.15.100:0/3679125816 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 447892694 0 0) 0x7f6b94003520 con 0x7f6ba4070ee0 2024-09-15T21:46:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba37fe640 1 -- 172.21.15.100:0/3679125816 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6ba40cca30 msgr2=0x7f6ba4071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba37fe640 1 --2- 172.21.15.100:0/3679125816 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6ba40cca30 0x7f6ba4071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba37fe640 1 -- 172.21.15.100:0/3679125816 >> v2:172.21.15.122:3300/0 conn(0x7f6ba40cce10 msgr2=0x7f6ba415af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba37fe640 1 --2- 172.21.15.100:0/3679125816 >> v2:172.21.15.122:3300/0 conn(0x7f6ba40cce10 0x7f6ba415af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba37fe640 1 -- 172.21.15.100:0/3679125816 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba40c9a90 con 0x7f6ba4070ee0 2024-09-15T21:46:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba37fe640 1 -- 172.21.15.100:0/3679125816 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7f6b90001670 con 0x7f6ba4070ee0 2024-09-15T21:46:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba37fe640 1 -- 172.21.15.100:0/3679125816 >> v1:172.21.15.100:6789/0 conn(0x7f6ba4070ee0 legacy=0x7f6ba4151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba37fe640 1 --2- 172.21.15.100:0/3679125816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b940039b0 0x7f6b94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba37fe640 1 --2- 172.21.15.100:0/3679125816 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6ba40cca30 0x7f6b94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba37fe640 1 --2- 172.21.15.100:0/3679125816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b94004d40 0x7f6b940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba37fe640 1 -- 172.21.15.100:0/3679125816 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba40c94d0 con 0x7f6b94004d40 2024-09-15T21:46:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba37fe640 1 -- 172.21.15.100:0/3679125816 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba40c97b0 con 0x7f6b940039b0 2024-09-15T21:46:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba37fe640 1 -- 172.21.15.100:0/3679125816 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6b9400b9c0 con 0x7f6ba40cca30 2024-09-15T21:46:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba96ce640 1 --2- 172.21.15.100:0/3679125816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b94004d40 0x7f6b940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.840+0000 7f6ba8ecd640 1 --2- 172.21.15.100:0/3679125816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b940039b0 0x7f6b94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.841+0000 7f6ba3fff640 1 --2- 172.21.15.100:0/3679125816 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6ba40cca30 0x7f6b94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.841+0000 7f6ba96ce640 1 -- 172.21.15.100:0/3679125816 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6ba40cca30 msgr2=0x7f6b94004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.841+0000 7f6ba96ce640 1 --2- 172.21.15.100:0/3679125816 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6ba40cca30 0x7f6b94004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.841+0000 7f6ba96ce640 1 -- 172.21.15.100:0/3679125816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b940039b0 msgr2=0x7f6b94003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.841+0000 7f6ba96ce640 1 --2- 172.21.15.100:0/3679125816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b940039b0 0x7f6b94003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.841+0000 7f6ba96ce640 1 -- 172.21.15.100:0/3679125816 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b94003520 con 0x7f6b94004d40 2024-09-15T21:46:28.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.841+0000 7f6ba3fff640 1 --2- 172.21.15.100:0/3679125816 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6ba40cca30 0x7f6b94004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.841+0000 7f6ba96ce640 1 --2- 172.21.15.100:0/3679125816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b94004d40 0x7f6b940051c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40c3350 tx=0x7f6b94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757a243875d92294 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.841+0000 7f6ba37fe640 1 -- 172.21.15.100:0/3679125816 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c037070 con 0x7f6b94004d40 2024-09-15T21:46:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.841+0000 7f6ba37fe640 1 -- 172.21.15.100:0/3679125816 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b9c02e050 con 0x7f6b94004d40 2024-09-15T21:46:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.841+0000 7f6ba37fe640 1 -- 172.21.15.100:0/3679125816 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c009c10 con 0x7f6b94004d40 2024-09-15T21:46:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.842+0000 7f6bab131640 1 -- 172.21.15.100:0/3679125816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b94004d40 msgr2=0x7f6b940051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.842+0000 7f6bab131640 1 --2- 172.21.15.100:0/3679125816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6b94004d40 0x7f6b940051c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40c3350 tx=0x7f6b94005830 comp rx=0 tx=0).stop 2024-09-15T21:46:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.842+0000 7f6ba96ce640 1 -- 172.21.15.100:0/3679125816 reap_dead start 2024-09-15T21:46:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.842+0000 7f6bab131640 1 -- 172.21.15.100:0/3679125816 shutdown_connections 2024-09-15T21:46:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.842+0000 7f6bab131640 1 -- 172.21.15.100:0/3679125816 >> 172.21.15.100:0/3679125816 conn(0x7f6ba4067dd0 msgr2=0x7f6ba406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.842+0000 7f6bab131640 1 -- 172.21.15.100:0/3679125816 shutdown_connections 2024-09-15T21:46:28.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.842+0000 7f6bab131640 1 -- 172.21.15.100:0/3679125816 wait complete. 2024-09-15T21:46:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.843+0000 7f6bab131640 1 Processor -- start 2024-09-15T21:46:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.843+0000 7f6bab131640 1 -- start start 2024-09-15T21:46:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.844+0000 7f6bab131640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6b94004d40 0x7f6ba4145e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.844+0000 7f6bab131640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ba4151f00 0x7f6ba415a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.844+0000 7f6bab131640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ba4158420 0x7f6ba41f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.844+0000 7f6bab131640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba4151ce0 con 0x7f6ba4158420 2024-09-15T21:46:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.844+0000 7f6bab131640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba4151b60 con 0x7f6ba4151f00 2024-09-15T21:46:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.844+0000 7f6bab131640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6ba4071990 con 0x7f6b94004d40 2024-09-15T21:46:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.844+0000 7f6ba96ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ba4158420 0x7f6ba41f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.844+0000 7f6ba96ce640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ba4158420 0x7f6ba41f21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43696/0 (socket says 172.21.15.100:43696) 2024-09-15T21:46:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.844+0000 7f6ba96ce640 1 -- 172.21.15.100:0/1084606757 learned_addr learned my addr 172.21.15.100:0/1084606757 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.844+0000 7f6ba8ecd640 1 --2- 172.21.15.100:0/1084606757 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6b94004d40 0x7f6ba4145e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.844+0000 7f6ba3fff640 1 --2- 172.21.15.100:0/1084606757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ba4151f00 0x7f6ba415a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.844+0000 7f6ba96ce640 1 -- 172.21.15.100:0/1084606757 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6b94004d40 msgr2=0x7f6ba4145e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:28.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.844+0000 7f6ba96ce640 1 --2- 172.21.15.100:0/1084606757 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6b94004d40 0x7f6ba4145e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.844+0000 7f6ba96ce640 1 -- 172.21.15.100:0/1084606757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ba4151f00 msgr2=0x7f6ba415a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:28.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.844+0000 7f6ba96ce640 1 --2- 172.21.15.100:0/1084606757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ba4151f00 0x7f6ba415a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:28.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.844+0000 7f6ba96ce640 1 -- 172.21.15.100:0/1084606757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba4155b20 con 0x7f6ba4158420 2024-09-15T21:46:28.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.844+0000 7f6ba3fff640 1 --2- 172.21.15.100:0/1084606757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ba4151f00 0x7f6ba415a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:28.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.845+0000 7f6ba96ce640 1 --2- 172.21.15.100:0/1084606757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ba4158420 0x7f6ba41f21d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c035040 tx=0x7f6b9c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:28.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.845+0000 7f6ba17fa640 1 -- 172.21.15.100:0/1084606757 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c037070 con 0x7f6ba4158420 2024-09-15T21:46:28.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.845+0000 7f6ba17fa640 1 -- 172.21.15.100:0/1084606757 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b9c038040 con 0x7f6ba4158420 2024-09-15T21:46:28.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.845+0000 7f6bab131640 1 -- 172.21.15.100:0/1084606757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba4146390 con 0x7f6ba4158420 2024-09-15T21:46:28.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.845+0000 7f6ba17fa640 1 -- 172.21.15.100:0/1084606757 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c009c10 con 0x7f6ba4158420 2024-09-15T21:46:28.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.845+0000 7f6bab131640 1 -- 172.21.15.100:0/1084606757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba4156880 con 0x7f6ba4158420 2024-09-15T21:46:28.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.846+0000 7f6bab131640 1 -- 172.21.15.100:0/1084606757 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba40c8310 con 0x7f6ba4158420 2024-09-15T21:46:28.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.846+0000 7f6ba17fa640 1 -- 172.21.15.100:0/1084606757 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6b9c044030 con 0x7f6ba4158420 2024-09-15T21:46:28.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.847+0000 7f6ba17fa640 1 --2- 172.21.15.100:0/1084606757 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6b8806c2e0 0x7f6b8806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:28.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.847+0000 7f6ba17fa640 1 -- 172.21.15.100:0/1084606757 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6b9c0ab790 con 0x7f6ba4158420 2024-09-15T21:46:28.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.847+0000 7f6ba8ecd640 1 --2- 172.21.15.100:0/1084606757 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6b8806c2e0 0x7f6b8806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:28.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.847+0000 7f6ba8ecd640 1 --2- 172.21.15.100:0/1084606757 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6b8806c2e0 0x7f6b8806e7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009a80 tx=0x7f6b98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:28.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:28.851+0000 7f6ba17fa640 1 -- 172.21.15.100:0/1084606757 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6b9c04b020 con 0x7f6ba4158420 2024-09-15T21:46:29.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.014+0000 7f6bab131640 1 -- 172.21.15.100:0/1084606757 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ba415b680 con 0x7f6b8806c2e0 2024-09-15T21:46:29.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.019+0000 7f6ba17fa640 1 -- 172.21.15.100:0/1084606757 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248805 (secure 0 0 0) 0x7f6ba415b680 con 0x7f6b8806c2e0 2024-09-15T21:46:29.027 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:29.029 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-15T21:46:29.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.022+0000 7f6bab131640 1 -- 172.21.15.100:0/1084606757 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6b8806c2e0 msgr2=0x7f6b8806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:29.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.022+0000 7f6bab131640 1 --2- 172.21.15.100:0/1084606757 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6b8806c2e0 0x7f6b8806e7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009a80 tx=0x7f6b98008040 comp rx=0 tx=0).stop 2024-09-15T21:46:29.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.022+0000 7f6bab131640 1 -- 172.21.15.100:0/1084606757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ba4158420 msgr2=0x7f6ba41f21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:29.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.022+0000 7f6bab131640 1 --2- 172.21.15.100:0/1084606757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ba4158420 0x7f6ba41f21d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c035040 tx=0x7f6b9c009750 comp rx=0 tx=0).stop 2024-09-15T21:46:29.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.023+0000 7f6bab131640 1 -- 172.21.15.100:0/1084606757 shutdown_connections 2024-09-15T21:46:29.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.023+0000 7f6bab131640 1 --2- 172.21.15.100:0/1084606757 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6b8806c2e0 0x7f6b8806e7a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.023+0000 7f6bab131640 1 --2- 172.21.15.100:0/1084606757 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6ba4151f00 0x7f6ba415a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.023+0000 7f6bab131640 1 --2- 172.21.15.100:0/1084606757 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6ba4158420 0x7f6ba41f21d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.023+0000 7f6bab131640 1 --2- 172.21.15.100:0/1084606757 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6b94004d40 0x7f6ba4145e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.023+0000 7f6bab131640 1 -- 172.21.15.100:0/1084606757 >> 172.21.15.100:0/1084606757 conn(0x7f6ba4067dd0 msgr2=0x7f6ba40c4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:29.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.023+0000 7f6bab131640 1 -- 172.21.15.100:0/1084606757 shutdown_connections 2024-09-15T21:46:29.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.023+0000 7f6bab131640 1 -- 172.21.15.100:0/1084606757 wait complete. 2024-09-15T21:46:29.036 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-15T21:46:27.804526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218908,"kb_used_data":2684,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755812,"statfs":{"total":773094113280,"available":772869951488,"internally_reserved":0,"allocated":2748416,"data_stored":1686550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000848"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.421348+0000","last_change":"2024-09-15T21:46:22.421348+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.421348+0000","last_undegraded":"2024-09-15T21:46:22.421348+0000","last_fullsized":"2024-09-15T21:46:22.421348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808513+0000","last_change":"2024-09-15T21:46:23.418716+0000","last_active":"2024-09-15T21:46:24.808513+0000","last_peered":"2024-09-15T21:46:24.808513+0000","last_clean":"2024-09-15T21:46:24.808513+0000","last_became_active":"2024-09-15T21:46:23.418471+0000","last_became_peered":"2024-09-15T21:46:23.418471+0000","last_unstale":"2024-09-15T21:46:24.808513+0000","last_undegraded":"2024-09-15T21:46:24.808513+0000","last_fullsized":"2024-09-15T21:46:24.808513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:49.599801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431027+0000","last_change":"2024-09-15T21:46:23.426404+0000","last_active":"2024-09-15T21:46:23.431027+0000","last_peered":"2024-09-15T21:46:23.431027+0000","last_clean":"2024-09-15T21:46:23.431027+0000","last_became_active":"2024-09-15T21:46:23.426146+0000","last_became_peered":"2024-09-15T21:46:23.426146+0000","last_unstale":"2024-09-15T21:46:23.431027+0000","last_undegraded":"2024-09-15T21:46:23.431027+0000","last_fullsized":"2024-09-15T21:46:23.431027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:40.061508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419990+0000","last_change":"2024-09-15T21:46:22.419990+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419990+0000","last_undegraded":"2024-09-15T21:46:22.419990+0000","last_fullsized":"2024-09-15T21:46:22.419990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418695+0000","last_change":"2024-09-15T21:46:22.418695+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418695+0000","last_undegraded":"2024-09-15T21:46:22.418695+0000","last_fullsized":"2024-09-15T21:46:22.418695+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418843+0000","last_change":"2024-09-15T21:46:22.418843+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418843+0000","last_undegraded":"2024-09-15T21:46:22.418843+0000","last_fullsized":"2024-09-15T21:46:22.418843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419832+0000","last_change":"2024-09-15T21:46:22.419832+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419832+0000","last_undegraded":"2024-09-15T21:46:22.419832+0000","last_fullsized":"2024-09-15T21:46:22.419832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.438915+0000","last_change":"2024-09-15T21:46:23.437143+0000","last_active":"2024-09-15T21:46:23.438915+0000","last_peered":"2024-09-15T21:46:23.438915+0000","last_clean":"2024-09-15T21:46:23.438915+0000","last_became_active":"2024-09-15T21:46:23.436864+0000","last_became_peered":"2024-09-15T21:46:23.436864+0000","last_unstale":"2024-09-15T21:46:23.438915+0000","last_undegraded":"2024-09-15T21:46:23.438915+0000","last_fullsized":"2024-09-15T21:46:23.438915+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:12:37.128024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.420920+0000","last_change":"2024-09-15T21:46:22.420920+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.420920+0000","last_undegraded":"2024-09-15T21:46:22.420920+0000","last_fullsized":"2024-09-15T21:46:22.420920+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.422403+0000","last_change":"2024-09-15T21:46:22.422403+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.422403+0000","last_undegraded":"2024-09-15T21:46:22.422403+0000","last_fullsized":"2024-09-15T21:46:22.422403+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418976+0000","last_change":"2024-09-15T21:46:22.418976+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418976+0000","last_undegraded":"2024-09-15T21:46:22.418976+0000","last_fullsized":"2024-09-15T21:46:22.418976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415544+0000","last_change":"2024-09-15T21:46:22.415544+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415544+0000","last_undegraded":"2024-09-15T21:46:22.415544+0000","last_fullsized":"2024-09-15T21:46:22.415544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431804+0000","last_change":"2024-09-15T21:46:23.427841+0000","last_active":"2024-09-15T21:46:23.431804+0000","last_peered":"2024-09-15T21:46:23.431804+0000","last_clean":"2024-09-15T21:46:23.431804+0000","last_became_active":"2024-09-15T21:46:23.427621+0000","last_became_peered":"2024-09-15T21:46:23.427621+0000","last_unstale":"2024-09-15T21:46:23.431804+0000","last_undegraded":"2024-09-15T21:46:23.431804+0000","last_fullsized":"2024-09-15T21:46:23.431804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:17:35.795757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.428658+0000","last_change":"2024-09-15T21:46:23.424707+0000","last_active":"2024-09-15T21:46:23.428658+0000","last_peered":"2024-09-15T21:46:23.428658+0000","last_clean":"2024-09-15T21:46:23.428658+0000","last_became_active":"2024-09-15T21:46:23.424447+0000","last_became_peered":"2024-09-15T21:46:23.424447+0000","last_unstale":"2024-09-15T21:46:23.428658+0000","last_undegraded":"2024-09-15T21:46:23.428658+0000","last_fullsized":"2024-09-15T21:46:23.428658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:28:21.800777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808321+0000","last_change":"2024-09-15T21:46:23.430297+0000","last_active":"2024-09-15T21:46:24.808321+0000","last_peered":"2024-09-15T21:46:24.808321+0000","last_clean":"2024-09-15T21:46:24.808321+0000","last_became_active":"2024-09-15T21:46:23.430070+0000","last_became_peered":"2024-09-15T21:46:23.430070+0000","last_unstale":"2024-09-15T21:46:24.808321+0000","last_undegraded":"2024-09-15T21:46:24.808321+0000","last_fullsized":"2024-09-15T21:46:24.808321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:03:19.615022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415614+0000","last_change":"2024-09-15T21:46:22.415614+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415614+0000","last_undegraded":"2024-09-15T21:46:22.415614+0000","last_fullsized":"2024-09-15T21:46:22.415614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415415+0000","last_change":"2024-09-15T21:46:22.415415+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415415+0000","last_undegraded":"2024-09-15T21:46:22.415415+0000","last_fullsized":"2024-09-15T21:46:22.415415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415507+0000","last_change":"2024-09-15T21:46:22.415507+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415507+0000","last_undegraded":"2024-09-15T21:46:22.415507+0000","last_fullsized":"2024-09-15T21:46:22.415507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418352+0000","last_change":"2024-09-15T21:46:22.418352+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418352+0000","last_undegraded":"2024-09-15T21:46:22.418352+0000","last_fullsized":"2024-09-15T21:46:22.418352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.420228+0000","last_change":"2024-09-15T21:46:22.420228+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.420228+0000","last_undegraded":"2024-09-15T21:46:22.420228+0000","last_fullsized":"2024-09-15T21:46:22.420228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433725+0000","last_change":"2024-09-15T21:46:23.428323+0000","last_active":"2024-09-15T21:46:23.433725+0000","last_peered":"2024-09-15T21:46:23.433725+0000","last_clean":"2024-09-15T21:46:23.433725+0000","last_became_active":"2024-09-15T21:46:23.428016+0000","last_became_peered":"2024-09-15T21:46:23.428016+0000","last_unstale":"2024-09-15T21:46:23.433725+0000","last_undegraded":"2024-09-15T21:46:23.433725+0000","last_fullsized":"2024-09-15T21:46:23.433725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:24:06.673523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418399+0000","last_change":"2024-09-15T21:46:22.418399+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418399+0000","last_undegraded":"2024-09-15T21:46:22.418399+0000","last_fullsized":"2024-09-15T21:46:22.418399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433661+0000","last_change":"2024-09-15T21:46:23.431527+0000","last_active":"2024-09-15T21:46:23.433661+0000","last_peered":"2024-09-15T21:46:23.433661+0000","last_clean":"2024-09-15T21:46:23.433661+0000","last_became_active":"2024-09-15T21:46:23.430719+0000","last_became_peered":"2024-09-15T21:46:23.430719+0000","last_unstale":"2024-09-15T21:46:23.433661+0000","last_undegraded":"2024-09-15T21:46:23.433661+0000","last_fullsized":"2024-09-15T21:46:23.433661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:25.362126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415180+0000","last_change":"2024-09-15T21:46:22.415180+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415180+0000","last_undegraded":"2024-09-15T21:46:22.415180+0000","last_fullsized":"2024-09-15T21:46:22.415180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.436122+0000","last_change":"2024-09-15T21:46:23.432646+0000","last_active":"2024-09-15T21:46:23.436122+0000","last_peered":"2024-09-15T21:46:23.436122+0000","last_clean":"2024-09-15T21:46:23.436122+0000","last_became_active":"2024-09-15T21:46:23.432317+0000","last_became_peered":"2024-09-15T21:46:23.432317+0000","last_unstale":"2024-09-15T21:46:23.436122+0000","last_undegraded":"2024-09-15T21:46:23.436122+0000","last_fullsized":"2024-09-15T21:46:23.436122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:34.359059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.421958+0000","last_change":"2024-09-15T21:46:22.421958+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.421958+0000","last_undegraded":"2024-09-15T21:46:22.421958+0000","last_fullsized":"2024-09-15T21:46:22.421958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808369+0000","last_change":"2024-09-15T21:46:23.421749+0000","last_active":"2024-09-15T21:46:24.808369+0000","last_peered":"2024-09-15T21:46:24.808369+0000","last_clean":"2024-09-15T21:46:24.808369+0000","last_became_active":"2024-09-15T21:46:23.421501+0000","last_became_peered":"2024-09-15T21:46:23.421501+0000","last_unstale":"2024-09-15T21:46:24.808369+0000","last_undegraded":"2024-09-15T21:46:24.808369+0000","last_fullsized":"2024-09-15T21:46:24.808369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:55.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.436035+0000","last_change":"2024-09-15T21:46:23.432591+0000","last_active":"2024-09-15T21:46:23.436035+0000","last_peered":"2024-09-15T21:46:23.436035+0000","last_clean":"2024-09-15T21:46:23.436035+0000","last_became_active":"2024-09-15T21:46:23.432326+0000","last_became_peered":"2024-09-15T21:46:23.432326+0000","last_unstale":"2024-09-15T21:46:23.436035+0000","last_undegraded":"2024-09-15T21:46:23.436035+0000","last_fullsized":"2024-09-15T21:46:23.436035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:46:56.508701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.428496+0000","last_change":"2024-09-15T21:46:23.423428+0000","last_active":"2024-09-15T21:46:23.428496+0000","last_peered":"2024-09-15T21:46:23.428496+0000","last_clean":"2024-09-15T21:46:23.428496+0000","last_became_active":"2024-09-15T21:46:23.422734+0000","last_became_peered":"2024-09-15T21:46:23.422734+0000","last_unstale":"2024-09-15T21:46:23.428496+0000","last_undegraded":"2024-09-15T21:46:23.428496+0000","last_fullsized":"2024-09-15T21:46:23.428496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:02.082197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.417891+0000","last_change":"2024-09-15T21:46:22.417891+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.417891+0000","last_undegraded":"2024-09-15T21:46:22.417891+0000","last_fullsized":"2024-09-15T21:46:22.417891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414635+0000","last_change":"2024-09-15T21:46:22.414635+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414635+0000","last_undegraded":"2024-09-15T21:46:22.414635+0000","last_fullsized":"2024-09-15T21:46:22.414635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418469+0000","last_change":"2024-09-15T21:46:22.418469+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418469+0000","last_undegraded":"2024-09-15T21:46:22.418469+0000","last_fullsized":"2024-09-15T21:46:22.418469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414859+0000","last_change":"2024-09-15T21:46:22.414859+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414859+0000","last_undegraded":"2024-09-15T21:46:22.414859+0000","last_fullsized":"2024-09-15T21:46:22.414859+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.424245+0000","last_change":"2024-09-15T21:46:22.429407+0000","last_active":"2024-09-15T21:46:23.424245+0000","last_peered":"2024-09-15T21:46:23.424245+0000","last_clean":"2024-09-15T21:46:23.424245+0000","last_became_active":"2024-09-15T21:46:22.429107+0000","last_became_peered":"2024-09-15T21:46:22.429107+0000","last_unstale":"2024-09-15T21:46:23.424245+0000","last_undegraded":"2024-09-15T21:46:23.424245+0000","last_fullsized":"2024-09-15T21:46:23.424245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:14:54.680349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.438772+0000","last_change":"2024-09-15T21:46:23.434198+0000","last_active":"2024-09-15T21:46:23.438772+0000","last_peered":"2024-09-15T21:46:23.438772+0000","last_clean":"2024-09-15T21:46:23.438772+0000","last_became_active":"2024-09-15T21:46:23.433944+0000","last_became_peered":"2024-09-15T21:46:23.433944+0000","last_unstale":"2024-09-15T21:46:23.438772+0000","last_undegraded":"2024-09-15T21:46:23.438772+0000","last_fullsized":"2024-09-15T21:46:23.438772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:20:05.953148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.428922+0000","last_change":"2024-09-15T21:46:22.425571+0000","last_active":"2024-09-15T21:46:22.428922+0000","last_peered":"2024-09-15T21:46:22.428922+0000","last_clean":"2024-09-15T21:46:22.428922+0000","last_became_active":"2024-09-15T21:46:22.425315+0000","last_became_peered":"2024-09-15T21:46:22.425315+0000","last_unstale":"2024-09-15T21:46:22.428922+0000","last_undegraded":"2024-09-15T21:46:22.428922+0000","last_fullsized":"2024-09-15T21:46:22.428922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:37.405710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.423270+0000","last_change":"2024-09-15T21:46:22.425492+0000","last_active":"2024-09-15T21:46:23.423270+0000","last_peered":"2024-09-15T21:46:23.423270+0000","last_clean":"2024-09-15T21:46:23.423270+0000","last_became_active":"2024-09-15T21:46:22.425226+0000","last_became_peered":"2024-09-15T21:46:22.425226+0000","last_unstale":"2024-09-15T21:46:23.423270+0000","last_undegraded":"2024-09-15T21:46:23.423270+0000","last_fullsized":"2024-09-15T21:46:23.423270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:24.322384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433622+0000","last_change":"2024-09-15T21:46:23.429140+0000","last_active":"2024-09-15T21:46:23.433622+0000","last_peered":"2024-09-15T21:46:23.433622+0000","last_clean":"2024-09-15T21:46:23.433622+0000","last_became_active":"2024-09-15T21:46:23.428902+0000","last_became_peered":"2024-09-15T21:46:23.428902+0000","last_unstale":"2024-09-15T21:46:23.433622+0000","last_undegraded":"2024-09-15T21:46:23.433622+0000","last_fullsized":"2024-09-15T21:46:23.433622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:44:54.129937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.431357+0000","last_change":"2024-09-15T21:46:22.428319+0000","last_active":"2024-09-15T21:46:22.431357+0000","last_peered":"2024-09-15T21:46:22.431357+0000","last_clean":"2024-09-15T21:46:22.431357+0000","last_became_active":"2024-09-15T21:46:22.428061+0000","last_became_peered":"2024-09-15T21:46:22.428061+0000","last_unstale":"2024-09-15T21:46:22.431357+0000","last_undegraded":"2024-09-15T21:46:22.431357+0000","last_fullsized":"2024-09-15T21:46:22.431357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:34.432046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808315+0000","last_change":"2024-09-15T21:46:23.427442+0000","last_active":"2024-09-15T21:46:24.808315+0000","last_peered":"2024-09-15T21:46:24.808315+0000","last_clean":"2024-09-15T21:46:24.808315+0000","last_became_active":"2024-09-15T21:46:23.427200+0000","last_became_peered":"2024-09-15T21:46:23.427200+0000","last_unstale":"2024-09-15T21:46:24.808315+0000","last_undegraded":"2024-09-15T21:46:24.808315+0000","last_fullsized":"2024-09-15T21:46:24.808315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:27.248545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433993+0000","last_change":"2024-09-15T21:46:22.430599+0000","last_active":"2024-09-15T21:46:22.433993+0000","last_peered":"2024-09-15T21:46:22.433993+0000","last_clean":"2024-09-15T21:46:22.433993+0000","last_became_active":"2024-09-15T21:46:22.430327+0000","last_became_peered":"2024-09-15T21:46:22.430327+0000","last_unstale":"2024-09-15T21:46:22.433993+0000","last_undegraded":"2024-09-15T21:46:22.433993+0000","last_fullsized":"2024-09-15T21:46:22.433993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:42:31.882474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.416669+0000","last_change":"2024-09-15T21:46:22.416669+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.416669+0000","last_undegraded":"2024-09-15T21:46:22.416669+0000","last_fullsized":"2024-09-15T21:46:22.416669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.429815+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:22.429815+0000","last_peered":"2024-09-15T21:46:22.429815+0000","last_clean":"2024-09-15T21:46:22.429815+0000","last_became_active":"2024-09-15T21:46:22.425236+0000","last_became_peered":"2024-09-15T21:46:22.425236+0000","last_unstale":"2024-09-15T21:46:22.429815+0000","last_undegraded":"2024-09-15T21:46:22.429815+0000","last_fullsized":"2024-09-15T21:46:22.429815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:01.525976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433802+0000","last_change":"2024-09-15T21:46:23.429587+0000","last_active":"2024-09-15T21:46:23.433802+0000","last_peered":"2024-09-15T21:46:23.433802+0000","last_clean":"2024-09-15T21:46:23.433802+0000","last_became_active":"2024-09-15T21:46:23.429334+0000","last_became_peered":"2024-09-15T21:46:23.429334+0000","last_unstale":"2024-09-15T21:46:23.433802+0000","last_undegraded":"2024-09-15T21:46:23.433802+0000","last_fullsized":"2024-09-15T21:46:23.433802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:31.263309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808325+0000","last_change":"2024-09-15T21:46:22.421964+0000","last_active":"2024-09-15T21:46:24.808325+0000","last_peered":"2024-09-15T21:46:24.808325+0000","last_clean":"2024-09-15T21:46:24.808325+0000","last_became_active":"2024-09-15T21:46:22.421661+0000","last_became_peered":"2024-09-15T21:46:22.421661+0000","last_unstale":"2024-09-15T21:46:24.808325+0000","last_undegraded":"2024-09-15T21:46:24.808325+0000","last_fullsized":"2024-09-15T21:46:24.808325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:07.395070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414271+0000","last_change":"2024-09-15T21:46:22.414271+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414271+0000","last_undegraded":"2024-09-15T21:46:22.414271+0000","last_fullsized":"2024-09-15T21:46:22.414271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433356+0000","last_change":"2024-09-15T21:46:22.426151+0000","last_active":"2024-09-15T21:46:22.433356+0000","last_peered":"2024-09-15T21:46:22.433356+0000","last_clean":"2024-09-15T21:46:22.433356+0000","last_became_active":"2024-09-15T21:46:22.425866+0000","last_became_peered":"2024-09-15T21:46:22.425866+0000","last_unstale":"2024-09-15T21:46:22.433356+0000","last_undegraded":"2024-09-15T21:46:22.433356+0000","last_fullsized":"2024-09-15T21:46:22.433356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:18.129618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419893+0000","last_change":"2024-09-15T21:46:22.419893+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419893+0000","last_undegraded":"2024-09-15T21:46:22.419893+0000","last_fullsized":"2024-09-15T21:46:22.419893+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.417870+0000","last_change":"2024-09-15T21:46:22.427415+0000","last_active":"2024-09-15T21:46:23.417870+0000","last_peered":"2024-09-15T21:46:23.417870+0000","last_clean":"2024-09-15T21:46:23.417870+0000","last_became_active":"2024-09-15T21:46:22.427087+0000","last_became_peered":"2024-09-15T21:46:22.427087+0000","last_unstale":"2024-09-15T21:46:23.417870+0000","last_undegraded":"2024-09-15T21:46:23.417870+0000","last_fullsized":"2024-09-15T21:46:23.417870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:09.365202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431262+0000","last_change":"2024-09-15T21:46:23.427238+0000","last_active":"2024-09-15T21:46:23.431262+0000","last_peered":"2024-09-15T21:46:23.431262+0000","last_clean":"2024-09-15T21:46:23.431262+0000","last_became_active":"2024-09-15T21:46:23.426993+0000","last_became_peered":"2024-09-15T21:46:23.426993+0000","last_unstale":"2024-09-15T21:46:23.431262+0000","last_undegraded":"2024-09-15T21:46:23.431262+0000","last_fullsized":"2024-09-15T21:46:23.431262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:36.125958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.431315+0000","last_change":"2024-09-15T21:46:22.427164+0000","last_active":"2024-09-15T21:46:22.431315+0000","last_peered":"2024-09-15T21:46:22.431315+0000","last_clean":"2024-09-15T21:46:22.431315+0000","last_became_active":"2024-09-15T21:46:22.426917+0000","last_became_peered":"2024-09-15T21:46:22.426917+0000","last_unstale":"2024-09-15T21:46:22.431315+0000","last_undegraded":"2024-09-15T21:46:22.431315+0000","last_fullsized":"2024-09-15T21:46:22.431315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:12:47.575708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418200+0000","last_change":"2024-09-15T21:46:22.418200+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418200+0000","last_undegraded":"2024-09-15T21:46:22.418200+0000","last_fullsized":"2024-09-15T21:46:22.418200+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.428957+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:22.428957+0000","last_peered":"2024-09-15T21:46:22.428957+0000","last_clean":"2024-09-15T21:46:22.428957+0000","last_became_active":"2024-09-15T21:46:22.425237+0000","last_became_peered":"2024-09-15T21:46:22.425237+0000","last_unstale":"2024-09-15T21:46:22.428957+0000","last_undegraded":"2024-09-15T21:46:22.428957+0000","last_fullsized":"2024-09-15T21:46:22.428957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:46.301372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.417788+0000","last_change":"2024-09-15T21:46:22.417788+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.417788+0000","last_undegraded":"2024-09-15T21:46:22.417788+0000","last_fullsized":"2024-09-15T21:46:22.417788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.423016+0000","last_change":"2024-09-15T21:46:22.419905+0000","last_active":"2024-09-15T21:46:22.423016+0000","last_peered":"2024-09-15T21:46:22.423016+0000","last_clean":"2024-09-15T21:46:22.423016+0000","last_became_active":"2024-09-15T21:46:22.419620+0000","last_became_peered":"2024-09-15T21:46:22.419620+0000","last_unstale":"2024-09-15T21:46:22.423016+0000","last_undegraded":"2024-09-15T21:46:22.423016+0000","last_fullsized":"2024-09-15T21:46:22.423016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:41.513967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419332+0000","last_change":"2024-09-15T21:46:22.419332+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419332+0000","last_undegraded":"2024-09-15T21:46:22.419332+0000","last_fullsized":"2024-09-15T21:46:22.419332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433527+0000","last_change":"2024-09-15T21:46:22.429661+0000","last_active":"2024-09-15T21:46:22.433527+0000","last_peered":"2024-09-15T21:46:22.433527+0000","last_clean":"2024-09-15T21:46:22.433527+0000","last_became_active":"2024-09-15T21:46:22.429250+0000","last_became_peered":"2024-09-15T21:46:22.429250+0000","last_unstale":"2024-09-15T21:46:22.433527+0000","last_undegraded":"2024-09-15T21:46:22.433527+0000","last_fullsized":"2024-09-15T21:46:22.433527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:52:16.852861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418349+0000","last_change":"2024-09-15T21:46:22.418349+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418349+0000","last_undegraded":"2024-09-15T21:46:22.418349+0000","last_fullsized":"2024-09-15T21:46:22.418349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.430022+0000","last_change":"2024-09-15T21:46:22.425870+0000","last_active":"2024-09-15T21:46:22.430022+0000","last_peered":"2024-09-15T21:46:22.430022+0000","last_clean":"2024-09-15T21:46:22.430022+0000","last_became_active":"2024-09-15T21:46:22.425588+0000","last_became_peered":"2024-09-15T21:46:22.425588+0000","last_unstale":"2024-09-15T21:46:22.430022+0000","last_undegraded":"2024-09-15T21:46:22.430022+0000","last_fullsized":"2024-09-15T21:46:22.430022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:53.982102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.420429+0000","last_change":"2024-09-15T21:46:22.420429+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.420429+0000","last_undegraded":"2024-09-15T21:46:22.420429+0000","last_fullsized":"2024-09-15T21:46:22.420429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808420+0000","last_change":"2024-09-15T21:46:22.425072+0000","last_active":"2024-09-15T21:46:24.808420+0000","last_peered":"2024-09-15T21:46:24.808420+0000","last_clean":"2024-09-15T21:46:24.808420+0000","last_became_active":"2024-09-15T21:46:22.424771+0000","last_became_peered":"2024-09-15T21:46:22.424771+0000","last_unstale":"2024-09-15T21:46:24.808420+0000","last_undegraded":"2024-09-15T21:46:24.808420+0000","last_fullsized":"2024-09-15T21:46:24.808420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:26.256785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415969+0000","last_change":"2024-09-15T21:46:22.415969+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415969+0000","last_undegraded":"2024-09-15T21:46:22.415969+0000","last_fullsized":"2024-09-15T21:46:22.415969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.432304+0000","last_change":"2024-09-15T21:46:22.428668+0000","last_active":"2024-09-15T21:46:22.432304+0000","last_peered":"2024-09-15T21:46:22.432304+0000","last_clean":"2024-09-15T21:46:22.432304+0000","last_became_active":"2024-09-15T21:46:22.428341+0000","last_became_peered":"2024-09-15T21:46:22.428341+0000","last_unstale":"2024-09-15T21:46:22.432304+0000","last_undegraded":"2024-09-15T21:46:22.432304+0000","last_fullsized":"2024-09-15T21:46:22.432304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:25.844475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418811+0000","last_change":"2024-09-15T21:46:22.418811+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418811+0000","last_undegraded":"2024-09-15T21:46:22.418811+0000","last_fullsized":"2024-09-15T21:46:22.418811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":167,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.422877+0000","last_change":"2024-09-15T21:46:14.328094+0000","last_active":"2024-09-15T21:46:22.422877+0000","last_peered":"2024-09-15T21:46:22.422877+0000","last_clean":"2024-09-15T21:46:22.422877+0000","last_became_active":"2024-09-15T21:46:14.326991+0000","last_became_peered":"2024-09-15T21:46:14.326991+0000","last_unstale":"2024-09-15T21:46:22.422877+0000","last_undegraded":"2024-09-15T21:46:22.422877+0000","last_fullsized":"2024-09-15T21:46:22.422877+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_clean_scrub_stamp":"2024-09-15T21:46:12.223400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:54:27.704130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.431406+0000","last_change":"2024-09-15T21:46:22.428887+0000","last_active":"2024-09-15T21:46:22.431406+0000","last_peered":"2024-09-15T21:46:22.431406+0000","last_clean":"2024-09-15T21:46:22.431406+0000","last_became_active":"2024-09-15T21:46:22.428619+0000","last_became_peered":"2024-09-15T21:46:22.428619+0000","last_unstale":"2024-09-15T21:46:22.431406+0000","last_undegraded":"2024-09-15T21:46:22.431406+0000","last_fullsized":"2024-09-15T21:46:22.431406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:43.510542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.438679+0000","last_change":"2024-09-15T21:46:23.435459+0000","last_active":"2024-09-15T21:46:23.438679+0000","last_peered":"2024-09-15T21:46:23.438679+0000","last_clean":"2024-09-15T21:46:23.438679+0000","last_became_active":"2024-09-15T21:46:23.435141+0000","last_became_peered":"2024-09-15T21:46:23.435141+0000","last_unstale":"2024-09-15T21:46:23.438679+0000","last_undegraded":"2024-09-15T21:46:23.438679+0000","last_fullsized":"2024-09-15T21:46:23.438679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:38.434075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433564+0000","last_change":"2024-09-15T21:46:22.431113+0000","last_active":"2024-09-15T21:46:22.433564+0000","last_peered":"2024-09-15T21:46:22.433564+0000","last_clean":"2024-09-15T21:46:22.433564+0000","last_became_active":"2024-09-15T21:46:22.430832+0000","last_became_peered":"2024-09-15T21:46:22.430832+0000","last_unstale":"2024-09-15T21:46:22.433564+0000","last_undegraded":"2024-09-15T21:46:22.433564+0000","last_fullsized":"2024-09-15T21:46:22.433564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:08.068054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433942+0000","last_change":"2024-09-15T21:46:23.430396+0000","last_active":"2024-09-15T21:46:23.433942+0000","last_peered":"2024-09-15T21:46:23.433942+0000","last_clean":"2024-09-15T21:46:23.433942+0000","last_became_active":"2024-09-15T21:46:23.430138+0000","last_became_peered":"2024-09-15T21:46:23.430138+0000","last_unstale":"2024-09-15T21:46:23.433942+0000","last_undegraded":"2024-09-15T21:46:23.433942+0000","last_fullsized":"2024-09-15T21:46:23.433942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:04.056726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.430035+0000","last_change":"2024-09-15T21:46:22.425808+0000","last_active":"2024-09-15T21:46:22.430035+0000","last_peered":"2024-09-15T21:46:22.430035+0000","last_clean":"2024-09-15T21:46:22.430035+0000","last_became_active":"2024-09-15T21:46:22.425510+0000","last_became_peered":"2024-09-15T21:46:22.425510+0000","last_unstale":"2024-09-15T21:46:22.430035+0000","last_undegraded":"2024-09-15T21:46:22.430035+0000","last_fullsized":"2024-09-15T21:46:22.430035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:13.502626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808338+0000","last_change":"2024-09-15T21:46:23.427051+0000","last_active":"2024-09-15T21:46:24.808338+0000","last_peered":"2024-09-15T21:46:24.808338+0000","last_clean":"2024-09-15T21:46:24.808338+0000","last_became_active":"2024-09-15T21:46:23.426185+0000","last_became_peered":"2024-09-15T21:46:23.426185+0000","last_unstale":"2024-09-15T21:46:24.808338+0000","last_undegraded":"2024-09-15T21:46:24.808338+0000","last_fullsized":"2024-09-15T21:46:24.808338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:48.318405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.429782+0000","last_change":"2024-09-15T21:46:22.424645+0000","last_active":"2024-09-15T21:46:22.429782+0000","last_peered":"2024-09-15T21:46:22.429782+0000","last_clean":"2024-09-15T21:46:22.429782+0000","last_became_active":"2024-09-15T21:46:22.421229+0000","last_became_peered":"2024-09-15T21:46:22.421229+0000","last_unstale":"2024-09-15T21:46:22.429782+0000","last_undegraded":"2024-09-15T21:46:22.429782+0000","last_fullsized":"2024-09-15T21:46:22.429782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:02.298935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.413899+0000","last_change":"2024-09-15T21:46:22.413899+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.413899+0000","last_undegraded":"2024-09-15T21:46:22.413899+0000","last_fullsized":"2024-09-15T21:46:22.413899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808438+0000","last_change":"2024-09-15T21:46:22.426492+0000","last_active":"2024-09-15T21:46:24.808438+0000","last_peered":"2024-09-15T21:46:24.808438+0000","last_clean":"2024-09-15T21:46:24.808438+0000","last_became_active":"2024-09-15T21:46:22.426210+0000","last_became_peered":"2024-09-15T21:46:22.426210+0000","last_unstale":"2024-09-15T21:46:24.808438+0000","last_undegraded":"2024-09-15T21:46:24.808438+0000","last_fullsized":"2024-09-15T21:46:24.808438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:05.248091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414998+0000","last_change":"2024-09-15T21:46:22.414998+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414998+0000","last_undegraded":"2024-09-15T21:46:22.414998+0000","last_fullsized":"2024-09-15T21:46:22.414998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433499+0000","last_change":"2024-09-15T21:46:22.430710+0000","last_active":"2024-09-15T21:46:22.433499+0000","last_peered":"2024-09-15T21:46:22.433499+0000","last_clean":"2024-09-15T21:46:22.433499+0000","last_became_active":"2024-09-15T21:46:22.430375+0000","last_became_peered":"2024-09-15T21:46:22.430375+0000","last_unstale":"2024-09-15T21:46:22.433499+0000","last_undegraded":"2024-09-15T21:46:22.433499+0000","last_fullsized":"2024-09-15T21:46:22.433499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:17.852485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808447+0000","last_change":"2024-09-15T21:46:23.426729+0000","last_active":"2024-09-15T21:46:24.808447+0000","last_peered":"2024-09-15T21:46:24.808447+0000","last_clean":"2024-09-15T21:46:24.808447+0000","last_became_active":"2024-09-15T21:46:23.426495+0000","last_became_peered":"2024-09-15T21:46:23.426495+0000","last_unstale":"2024-09-15T21:46:24.808447+0000","last_undegraded":"2024-09-15T21:46:24.808447+0000","last_fullsized":"2024-09-15T21:46:24.808447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:03.277997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808436+0000","last_change":"2024-09-15T21:46:22.427137+0000","last_active":"2024-09-15T21:46:24.808436+0000","last_peered":"2024-09-15T21:46:24.808436+0000","last_clean":"2024-09-15T21:46:24.808436+0000","last_became_active":"2024-09-15T21:46:22.426855+0000","last_became_peered":"2024-09-15T21:46:22.426855+0000","last_unstale":"2024-09-15T21:46:24.808436+0000","last_undegraded":"2024-09-15T21:46:24.808436+0000","last_fullsized":"2024-09-15T21:46:24.808436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:40:14.277940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431767+0000","last_change":"2024-09-15T21:46:23.427327+0000","last_active":"2024-09-15T21:46:23.431767+0000","last_peered":"2024-09-15T21:46:23.431767+0000","last_clean":"2024-09-15T21:46:23.431767+0000","last_became_active":"2024-09-15T21:46:23.427096+0000","last_became_peered":"2024-09-15T21:46:23.427096+0000","last_unstale":"2024-09-15T21:46:23.431767+0000","last_undegraded":"2024-09-15T21:46:23.431767+0000","last_fullsized":"2024-09-15T21:46:23.431767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:43.650053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433399+0000","last_change":"2024-09-15T21:46:22.427798+0000","last_active":"2024-09-15T21:46:22.433399+0000","last_peered":"2024-09-15T21:46:22.433399+0000","last_clean":"2024-09-15T21:46:22.433399+0000","last_became_active":"2024-09-15T21:46:22.427537+0000","last_became_peered":"2024-09-15T21:46:22.427537+0000","last_unstale":"2024-09-15T21:46:22.433399+0000","last_undegraded":"2024-09-15T21:46:22.433399+0000","last_fullsized":"2024-09-15T21:46:22.433399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:36.227177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.428576+0000","last_change":"2024-09-15T21:46:23.424266+0000","last_active":"2024-09-15T21:46:23.428576+0000","last_peered":"2024-09-15T21:46:23.428576+0000","last_clean":"2024-09-15T21:46:23.428576+0000","last_became_active":"2024-09-15T21:46:23.424007+0000","last_became_peered":"2024-09-15T21:46:23.424007+0000","last_unstale":"2024-09-15T21:46:23.428576+0000","last_undegraded":"2024-09-15T21:46:23.428576+0000","last_fullsized":"2024-09-15T21:46:23.428576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:04:11.974067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433546+0000","last_change":"2024-09-15T21:46:22.430901+0000","last_active":"2024-09-15T21:46:22.433546+0000","last_peered":"2024-09-15T21:46:22.433546+0000","last_clean":"2024-09-15T21:46:22.433546+0000","last_became_active":"2024-09-15T21:46:22.430452+0000","last_became_peered":"2024-09-15T21:46:22.430452+0000","last_unstale":"2024-09-15T21:46:22.433546+0000","last_undegraded":"2024-09-15T21:46:22.433546+0000","last_fullsized":"2024-09-15T21:46:22.433546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:02.951631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414149+0000","last_change":"2024-09-15T21:46:22.414149+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414149+0000","last_undegraded":"2024-09-15T21:46:22.414149+0000","last_fullsized":"2024-09-15T21:46:22.414149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808380+0000","last_change":"2024-09-15T21:46:22.423795+0000","last_active":"2024-09-15T21:46:24.808380+0000","last_peered":"2024-09-15T21:46:24.808380+0000","last_clean":"2024-09-15T21:46:24.808380+0000","last_became_active":"2024-09-15T21:46:22.422327+0000","last_became_peered":"2024-09-15T21:46:22.422327+0000","last_unstale":"2024-09-15T21:46:24.808380+0000","last_undegraded":"2024-09-15T21:46:24.808380+0000","last_fullsized":"2024-09-15T21:46:24.808380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:16.670599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433600+0000","last_change":"2024-09-15T21:46:23.428634+0000","last_active":"2024-09-15T21:46:23.433600+0000","last_peered":"2024-09-15T21:46:23.433600+0000","last_clean":"2024-09-15T21:46:23.433600+0000","last_became_active":"2024-09-15T21:46:23.428301+0000","last_became_peered":"2024-09-15T21:46:23.428301+0000","last_unstale":"2024-09-15T21:46:23.433600+0000","last_undegraded":"2024-09-15T21:46:23.433600+0000","last_fullsized":"2024-09-15T21:46:23.433600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:17.178940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808304+0000","last_change":"2024-09-15T21:46:22.425810+0000","last_active":"2024-09-15T21:46:24.808304+0000","last_peered":"2024-09-15T21:46:24.808304+0000","last_clean":"2024-09-15T21:46:24.808304+0000","last_became_active":"2024-09-15T21:46:22.425478+0000","last_became_peered":"2024-09-15T21:46:22.425478+0000","last_unstale":"2024-09-15T21:46:24.808304+0000","last_undegraded":"2024-09-15T21:46:24.808304+0000","last_fullsized":"2024-09-15T21:46:24.808304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:53.296599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431785+0000","last_change":"2024-09-15T21:46:23.426054+0000","last_active":"2024-09-15T21:46:23.431785+0000","last_peered":"2024-09-15T21:46:23.431785+0000","last_clean":"2024-09-15T21:46:23.431785+0000","last_became_active":"2024-09-15T21:46:23.425547+0000","last_became_peered":"2024-09-15T21:46:23.425547+0000","last_unstale":"2024-09-15T21:46:23.431785+0000","last_undegraded":"2024-09-15T21:46:23.431785+0000","last_fullsized":"2024-09-15T21:46:23.431785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:25.738831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433353+0000","last_change":"2024-09-15T21:46:22.428524+0000","last_active":"2024-09-15T21:46:22.433353+0000","last_peered":"2024-09-15T21:46:22.433353+0000","last_clean":"2024-09-15T21:46:22.433353+0000","last_became_active":"2024-09-15T21:46:22.428043+0000","last_became_peered":"2024-09-15T21:46:22.428043+0000","last_unstale":"2024-09-15T21:46:22.433353+0000","last_undegraded":"2024-09-15T21:46:22.433353+0000","last_fullsized":"2024-09-15T21:46:22.433353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:53.400491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419544+0000","last_change":"2024-09-15T21:46:22.419544+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419544+0000","last_undegraded":"2024-09-15T21:46:22.419544+0000","last_fullsized":"2024-09-15T21:46:22.419544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415492+0000","last_change":"2024-09-15T21:46:22.415492+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415492+0000","last_undegraded":"2024-09-15T21:46:22.415492+0000","last_fullsized":"2024-09-15T21:46:22.415492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433558+0000","last_change":"2024-09-15T21:46:22.431172+0000","last_active":"2024-09-15T21:46:22.433558+0000","last_peered":"2024-09-15T21:46:22.433558+0000","last_clean":"2024-09-15T21:46:22.433558+0000","last_became_active":"2024-09-15T21:46:22.430800+0000","last_became_peered":"2024-09-15T21:46:22.430800+0000","last_unstale":"2024-09-15T21:46:22.433558+0000","last_undegraded":"2024-09-15T21:46:22.433558+0000","last_fullsized":"2024-09-15T21:46:22.433558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:52:07.231504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.417557+0000","last_change":"2024-09-15T21:46:22.417557+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.417557+0000","last_undegraded":"2024-09-15T21:46:22.417557+0000","last_fullsized":"2024-09-15T21:46:22.417557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433476+0000","last_change":"2024-09-15T21:46:22.429870+0000","last_active":"2024-09-15T21:46:22.433476+0000","last_peered":"2024-09-15T21:46:22.433476+0000","last_clean":"2024-09-15T21:46:22.433476+0000","last_became_active":"2024-09-15T21:46:22.429578+0000","last_became_peered":"2024-09-15T21:46:22.429578+0000","last_unstale":"2024-09-15T21:46:22.433476+0000","last_undegraded":"2024-09-15T21:46:22.433476+0000","last_fullsized":"2024-09-15T21:46:22.433476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:59.880081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433864+0000","last_change":"2024-09-15T21:46:23.429604+0000","last_active":"2024-09-15T21:46:23.433864+0000","last_peered":"2024-09-15T21:46:23.433864+0000","last_clean":"2024-09-15T21:46:23.433864+0000","last_became_active":"2024-09-15T21:46:23.429355+0000","last_became_peered":"2024-09-15T21:46:23.429355+0000","last_unstale":"2024-09-15T21:46:23.433864+0000","last_undegraded":"2024-09-15T21:46:23.433864+0000","last_fullsized":"2024-09-15T21:46:23.433864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:23:10.533918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.424314+0000","last_change":"2024-09-15T21:46:22.428529+0000","last_active":"2024-09-15T21:46:23.424314+0000","last_peered":"2024-09-15T21:46:23.424314+0000","last_clean":"2024-09-15T21:46:23.424314+0000","last_became_active":"2024-09-15T21:46:22.428263+0000","last_became_peered":"2024-09-15T21:46:22.428263+0000","last_unstale":"2024-09-15T21:46:23.424314+0000","last_undegraded":"2024-09-15T21:46:23.424314+0000","last_fullsized":"2024-09-15T21:46:23.424314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:22:40.160062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433487+0000","last_change":"2024-09-15T21:46:23.427968+0000","last_active":"2024-09-15T21:46:23.433487+0000","last_peered":"2024-09-15T21:46:23.433487+0000","last_clean":"2024-09-15T21:46:23.433487+0000","last_became_active":"2024-09-15T21:46:23.427546+0000","last_became_peered":"2024-09-15T21:46:23.427546+0000","last_unstale":"2024-09-15T21:46:23.433487+0000","last_undegraded":"2024-09-15T21:46:23.433487+0000","last_fullsized":"2024-09-15T21:46:23.433487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:45.544297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.417176+0000","last_change":"2024-09-15T21:46:22.428258+0000","last_active":"2024-09-15T21:46:23.417176+0000","last_peered":"2024-09-15T21:46:23.417176+0000","last_clean":"2024-09-15T21:46:23.417176+0000","last_became_active":"2024-09-15T21:46:22.427954+0000","last_became_peered":"2024-09-15T21:46:22.427954+0000","last_unstale":"2024-09-15T21:46:23.417176+0000","last_undegraded":"2024-09-15T21:46:23.417176+0000","last_fullsized":"2024-09-15T21:46:23.417176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:22.440456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476740,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":654251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":654251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":57608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":57608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":57608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":57608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":76958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":70658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:46:29.043 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-09-15T21:46:29.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.421348+0000', 'last_change': '2024-09-15T21:46:22.421348+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.421348+0000', 'last_undegraded': '2024-09-15T21:46:22.421348+0000', 'last_fullsized': '2024-09-15T21:46:22.421348+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-15T21:46:29.043 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3c is not active+clean 2024-09-15T21:46:29.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.419990+0000', 'last_change': '2024-09-15T21:46:22.419990+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.419990+0000', 'last_undegraded': '2024-09-15T21:46:22.419990+0000', 'last_fullsized': '2024-09-15T21:46:22.419990+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-15T21:46:29.043 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-09-15T21:46:29.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.418695+0000', 'last_change': '2024-09-15T21:46:22.418695+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.418695+0000', 'last_undegraded': '2024-09-15T21:46:22.418695+0000', 'last_fullsized': '2024-09-15T21:46:22.418695+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-15T21:46:29.043 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3a is not active+clean 2024-09-15T21:46:29.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.418843+0000', 'last_change': '2024-09-15T21:46:22.418843+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.418843+0000', 'last_undegraded': '2024-09-15T21:46:22.418843+0000', 'last_fullsized': '2024-09-15T21:46:22.418843+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-15T21:46:29.044 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-09-15T21:46:29.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.419832+0000', 'last_change': '2024-09-15T21:46:22.419832+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.419832+0000', 'last_undegraded': '2024-09-15T21:46:22.419832+0000', 'last_fullsized': '2024-09-15T21:46:22.419832+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-15T21:46:29.044 INFO:tasks.ceph.ceph_manager.ceph:PG 3.37 is not active+clean 2024-09-15T21:46:29.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.420920+0000', 'last_change': '2024-09-15T21:46:22.420920+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.420920+0000', 'last_undegraded': '2024-09-15T21:46:22.420920+0000', 'last_fullsized': '2024-09-15T21:46:22.420920+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-15T21:46:29.044 INFO:tasks.ceph.ceph_manager.ceph:PG 3.36 is not active+clean 2024-09-15T21:46:29.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.422403+0000', 'last_change': '2024-09-15T21:46:22.422403+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.422403+0000', 'last_undegraded': '2024-09-15T21:46:22.422403+0000', 'last_fullsized': '2024-09-15T21:46:22.422403+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-15T21:46:29.044 INFO:tasks.ceph.ceph_manager.ceph:PG 3.35 is not active+clean 2024-09-15T21:46:29.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.418976+0000', 'last_change': '2024-09-15T21:46:22.418976+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.418976+0000', 'last_undegraded': '2024-09-15T21:46:22.418976+0000', 'last_fullsized': '2024-09-15T21:46:22.418976+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-15T21:46:29.044 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-15T21:46:29.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.415544+0000', 'last_change': '2024-09-15T21:46:22.415544+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.415544+0000', 'last_undegraded': '2024-09-15T21:46:22.415544+0000', 'last_fullsized': '2024-09-15T21:46:22.415544+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-15T21:46:29.044 INFO:tasks.ceph.ceph_manager.ceph:PG 3.30 is not active+clean 2024-09-15T21:46:29.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.415614+0000', 'last_change': '2024-09-15T21:46:22.415614+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.415614+0000', 'last_undegraded': '2024-09-15T21:46:22.415614+0000', 'last_fullsized': '2024-09-15T21:46:22.415614+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-15T21:46:29.045 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2f is not active+clean 2024-09-15T21:46:29.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.415415+0000', 'last_change': '2024-09-15T21:46:22.415415+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.415415+0000', 'last_undegraded': '2024-09-15T21:46:22.415415+0000', 'last_fullsized': '2024-09-15T21:46:22.415415+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-15T21:46:29.045 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2e is not active+clean 2024-09-15T21:46:29.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.415507+0000', 'last_change': '2024-09-15T21:46:22.415507+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.415507+0000', 'last_undegraded': '2024-09-15T21:46:22.415507+0000', 'last_fullsized': '2024-09-15T21:46:22.415507+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:46:29.045 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-09-15T21:46:29.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.418352+0000', 'last_change': '2024-09-15T21:46:22.418352+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.418352+0000', 'last_undegraded': '2024-09-15T21:46:22.418352+0000', 'last_fullsized': '2024-09-15T21:46:22.418352+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-15T21:46:29.046 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2c is not active+clean 2024-09-15T21:46:29.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.420228+0000', 'last_change': '2024-09-15T21:46:22.420228+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.420228+0000', 'last_undegraded': '2024-09-15T21:46:22.420228+0000', 'last_fullsized': '2024-09-15T21:46:22.420228+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-15T21:46:29.046 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2a is not active+clean 2024-09-15T21:46:29.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.418399+0000', 'last_change': '2024-09-15T21:46:22.418399+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.418399+0000', 'last_undegraded': '2024-09-15T21:46:22.418399+0000', 'last_fullsized': '2024-09-15T21:46:22.418399+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-15T21:46:29.046 INFO:tasks.ceph.ceph_manager.ceph:PG 3.28 is not active+clean 2024-09-15T21:46:29.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.415180+0000', 'last_change': '2024-09-15T21:46:22.415180+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.415180+0000', 'last_undegraded': '2024-09-15T21:46:22.415180+0000', 'last_fullsized': '2024-09-15T21:46:22.415180+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-15T21:46:29.047 INFO:tasks.ceph.ceph_manager.ceph:PG 3.26 is not active+clean 2024-09-15T21:46:29.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.421958+0000', 'last_change': '2024-09-15T21:46:22.421958+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.421958+0000', 'last_undegraded': '2024-09-15T21:46:22.421958+0000', 'last_fullsized': '2024-09-15T21:46:22.421958+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-15T21:46:29.047 INFO:tasks.ceph.ceph_manager.ceph:PG 3.22 is not active+clean 2024-09-15T21:46:29.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.417891+0000', 'last_change': '2024-09-15T21:46:22.417891+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.417891+0000', 'last_undegraded': '2024-09-15T21:46:22.417891+0000', 'last_fullsized': '2024-09-15T21:46:22.417891+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-15T21:46:29.047 INFO:tasks.ceph.ceph_manager.ceph:PG 3.21 is not active+clean 2024-09-15T21:46:29.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.414635+0000', 'last_change': '2024-09-15T21:46:22.414635+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.414635+0000', 'last_undegraded': '2024-09-15T21:46:22.414635+0000', 'last_fullsized': '2024-09-15T21:46:22.414635+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:46:29.047 INFO:tasks.ceph.ceph_manager.ceph:PG 3.20 is not active+clean 2024-09-15T21:46:29.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.418469+0000', 'last_change': '2024-09-15T21:46:22.418469+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.418469+0000', 'last_undegraded': '2024-09-15T21:46:22.418469+0000', 'last_fullsized': '2024-09-15T21:46:22.418469+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-15T21:46:29.047 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1a is not active+clean 2024-09-15T21:46:29.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.414859+0000', 'last_change': '2024-09-15T21:46:22.414859+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.414859+0000', 'last_undegraded': '2024-09-15T21:46:22.414859+0000', 'last_fullsized': '2024-09-15T21:46:22.414859+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-15T21:46:29.047 INFO:tasks.ceph.ceph_manager.ceph:PG 3.16 is not active+clean 2024-09-15T21:46:29.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.416669+0000', 'last_change': '2024-09-15T21:46:22.416669+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.416669+0000', 'last_undegraded': '2024-09-15T21:46:22.416669+0000', 'last_fullsized': '2024-09-15T21:46:22.416669+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-15T21:46:29.048 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-09-15T21:46:29.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.414271+0000', 'last_change': '2024-09-15T21:46:22.414271+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.414271+0000', 'last_undegraded': '2024-09-15T21:46:22.414271+0000', 'last_fullsized': '2024-09-15T21:46:22.414271+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-15T21:46:29.048 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-09-15T21:46:29.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.419893+0000', 'last_change': '2024-09-15T21:46:22.419893+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.419893+0000', 'last_undegraded': '2024-09-15T21:46:22.419893+0000', 'last_fullsized': '2024-09-15T21:46:22.419893+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-15T21:46:29.048 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-09-15T21:46:29.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.418200+0000', 'last_change': '2024-09-15T21:46:22.418200+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.418200+0000', 'last_undegraded': '2024-09-15T21:46:22.418200+0000', 'last_fullsized': '2024-09-15T21:46:22.418200+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-15T21:46:29.048 INFO:tasks.ceph.ceph_manager.ceph:PG 3.10 is not active+clean 2024-09-15T21:46:29.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.417788+0000', 'last_change': '2024-09-15T21:46:22.417788+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.417788+0000', 'last_undegraded': '2024-09-15T21:46:22.417788+0000', 'last_fullsized': '2024-09-15T21:46:22.417788+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-15T21:46:29.049 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-15T21:46:29.049 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.419332+0000', 'last_change': '2024-09-15T21:46:22.419332+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.419332+0000', 'last_undegraded': '2024-09-15T21:46:22.419332+0000', 'last_fullsized': '2024-09-15T21:46:22.419332+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-15T21:46:29.049 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-09-15T21:46:29.049 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.418349+0000', 'last_change': '2024-09-15T21:46:22.418349+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.418349+0000', 'last_undegraded': '2024-09-15T21:46:22.418349+0000', 'last_fullsized': '2024-09-15T21:46:22.418349+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-15T21:46:29.049 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-09-15T21:46:29.051 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.420429+0000', 'last_change': '2024-09-15T21:46:22.420429+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.420429+0000', 'last_undegraded': '2024-09-15T21:46:22.420429+0000', 'last_fullsized': '2024-09-15T21:46:22.420429+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-15T21:46:29.051 INFO:tasks.ceph.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-15T21:46:29.051 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.415969+0000', 'last_change': '2024-09-15T21:46:22.415969+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.415969+0000', 'last_undegraded': '2024-09-15T21:46:22.415969+0000', 'last_fullsized': '2024-09-15T21:46:22.415969+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-15T21:46:29.051 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-15T21:46:29.051 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.418811+0000', 'last_change': '2024-09-15T21:46:22.418811+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.418811+0000', 'last_undegraded': '2024-09-15T21:46:22.418811+0000', 'last_fullsized': '2024-09-15T21:46:22.418811+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-15T21:46:29.053 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-15T21:46:29.055 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.413899+0000', 'last_change': '2024-09-15T21:46:22.413899+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.413899+0000', 'last_undegraded': '2024-09-15T21:46:22.413899+0000', 'last_fullsized': '2024-09-15T21:46:22.413899+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-15T21:46:29.055 INFO:tasks.ceph.ceph_manager.ceph:PG 3.5 is not active+clean 2024-09-15T21:46:29.055 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.414998+0000', 'last_change': '2024-09-15T21:46:22.414998+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.414998+0000', 'last_undegraded': '2024-09-15T21:46:22.414998+0000', 'last_fullsized': '2024-09-15T21:46:22.414998+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-15T21:46:29.055 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-09-15T21:46:29.055 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.414149+0000', 'last_change': '2024-09-15T21:46:22.414149+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.414149+0000', 'last_undegraded': '2024-09-15T21:46:22.414149+0000', 'last_fullsized': '2024-09-15T21:46:22.414149+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-15T21:46:29.056 INFO:tasks.ceph.ceph_manager.ceph:PG 3.a is not active+clean 2024-09-15T21:46:29.056 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.419544+0000', 'last_change': '2024-09-15T21:46:22.419544+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.419544+0000', 'last_undegraded': '2024-09-15T21:46:22.419544+0000', 'last_fullsized': '2024-09-15T21:46:22.419544+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-15T21:46:29.056 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1d is not active+clean 2024-09-15T21:46:29.056 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.415492+0000', 'last_change': '2024-09-15T21:46:22.415492+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.415492+0000', 'last_undegraded': '2024-09-15T21:46:22.415492+0000', 'last_fullsized': '2024-09-15T21:46:22.415492+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-15T21:46:29.056 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1c is not active+clean 2024-09-15T21:46:29.056 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-15T21:46:22.417557+0000', 'last_change': '2024-09-15T21:46:22.417557+0000', 'last_active': '2024-09-15T21:46:22.402110+0000', 'last_peered': '2024-09-15T21:46:22.402110+0000', 'last_clean': '2024-09-15T21:46:22.402110+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-15T21:46:22.417557+0000', 'last_undegraded': '2024-09-15T21:46:22.417557+0000', 'last_fullsized': '2024-09-15T21:46:22.417557+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'last_clean_scrub_stamp': '2024-09-15T21:46:22.402110+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-15T21:46:29.057 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:46:29.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.180+0000 7fbc1afcf640 1 Processor -- start 2024-09-15T21:46:29.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.180+0000 7fbc1afcf640 1 -- start start 2024-09-15T21:46:29.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.181+0000 7fbc1afcf640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fbc14155330 0x7fbc14071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:29.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.181+0000 7fbc1afcf640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc140cc4c0 0x7fbc140733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:29.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.181+0000 7fbc1afcf640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc140b3e40 con 0x7fbc14070bd0 2024-09-15T21:46:29.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.181+0000 7fbc1afcf640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fbc140ca060 con 0x7fbc14155330 2024-09-15T21:46:29.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.181+0000 7fbc1afcf640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbc140c9bd0 con 0x7fbc140cc4c0 2024-09-15T21:46:29.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.181+0000 7fbc18d6b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fbc14155330 0x7fbc14071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:29.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.181+0000 7fbc18d6b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fbc14155330 0x7fbc14071500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:40806/0 (socket says 172.21.15.100:40806) 2024-09-15T21:46:29.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.181+0000 7fbc1956c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc140cc4c0 0x7fbc140733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:29.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.181+0000 7fbc18d6b640 1 -- 172.21.15.100:0/4063196847 learned_addr learned my addr 172.21.15.100:0/4063196847 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:29.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.181+0000 7fbc13fff640 1 --1- 172.21.15.100:0/4063196847 >> v1:172.21.15.122:6789/0 conn(0x7fbc14070bd0 0x7fbc1415e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.100:46790/0 (socket says 172.21.15.100:46790) 2024-09-15T21:46:29.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.182+0000 7fbc18d6b640 1 -- 172.21.15.100:0/4063196847 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc140cc4c0 msgr2=0x7fbc140733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:29.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.182+0000 7fbc18d6b640 1 --2- 172.21.15.100:0/4063196847 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc140cc4c0 0x7fbc140733f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.182+0000 7fbc18d6b640 1 -- 172.21.15.100:0/4063196847 >> v1:172.21.15.122:6789/0 conn(0x7fbc14070bd0 legacy=0x7fbc1415e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:29.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.182+0000 7fbc18d6b640 1 -- 172.21.15.100:0/4063196847 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc140c98a0 con 0x7fbc14155330 2024-09-15T21:46:29.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.182+0000 7fbc18d6b640 1 --2- 172.21.15.100:0/4063196847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc14155330 0x7fbc14071500 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbc04009870 tx=0x7fbc0402a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e18decb05328345f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:29.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.183+0000 7fbc137fe640 1 -- 172.21.15.100:0/4063196847 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04037070 con 0x7fbc14155330 2024-09-15T21:46:29.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.183+0000 7fbc137fe640 1 -- 172.21.15.100:0/4063196847 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc0402ae50 con 0x7fbc14155330 2024-09-15T21:46:29.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.183+0000 7fbc137fe640 1 -- 172.21.15.100:0/4063196847 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04032860 con 0x7fbc14155330 2024-09-15T21:46:29.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.183+0000 7fbc1afcf640 1 -- 172.21.15.100:0/4063196847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc14155330 msgr2=0x7fbc14071500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:29.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.183+0000 7fbc1afcf640 1 --2- 172.21.15.100:0/4063196847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc14155330 0x7fbc14071500 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbc04009870 tx=0x7fbc0402a300 comp rx=0 tx=0).stop 2024-09-15T21:46:29.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.183+0000 7fbc1afcf640 1 -- 172.21.15.100:0/4063196847 shutdown_connections 2024-09-15T21:46:29.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.183+0000 7fbc1afcf640 1 --2- 172.21.15.100:0/4063196847 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc140cc4c0 0x7fbc140733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.183+0000 7fbc1afcf640 1 --2- 172.21.15.100:0/4063196847 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc14155330 0x7fbc14071500 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.183+0000 7fbc1afcf640 1 -- 172.21.15.100:0/4063196847 >> 172.21.15.100:0/4063196847 conn(0x7fbc1406c210 msgr2=0x7fbc1406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:29.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.183+0000 7fbc1afcf640 1 -- 172.21.15.100:0/4063196847 shutdown_connections 2024-09-15T21:46:29.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.183+0000 7fbc1afcf640 1 -- 172.21.15.100:0/4063196847 wait complete. 2024-09-15T21:46:29.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.184+0000 7fbc1afcf640 1 Processor -- start 2024-09-15T21:46:29.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.184+0000 7fbc1afcf640 1 -- start start 2024-09-15T21:46:29.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.185+0000 7fbc1afcf640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc14070bd0 0x7fbc141540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:29.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.185+0000 7fbc1afcf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc140cc4c0 0x7fbc14155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:29.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.185+0000 7fbc1afcf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc14155330 0x7fbc141564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:29.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.185+0000 7fbc1afcf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbc140b40e0 con 0x7fbc140cc4c0 2024-09-15T21:46:29.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.185+0000 7fbc1afcf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbc140710a0 con 0x7fbc14155330 2024-09-15T21:46:29.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.185+0000 7fbc1afcf640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbc140b43c0 con 0x7fbc14070bd0 2024-09-15T21:46:29.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.185+0000 7fbc13fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc140cc4c0 0x7fbc14155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:29.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.185+0000 7fbc13fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc140cc4c0 0x7fbc14155f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43704/0 (socket says 172.21.15.100:43704) 2024-09-15T21:46:29.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.185+0000 7fbc13fff640 1 -- 172.21.15.100:0/3070989391 learned_addr learned my addr 172.21.15.100:0/3070989391 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:29.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.186+0000 7fbc18d6b640 1 --2- 172.21.15.100:0/3070989391 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc14070bd0 0x7fbc141540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:29.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.186+0000 7fbc1956c640 1 --2- 172.21.15.100:0/3070989391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc14155330 0x7fbc141564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:29.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.186+0000 7fbc13fff640 1 -- 172.21.15.100:0/3070989391 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc14070bd0 msgr2=0x7fbc141540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:29.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.186+0000 7fbc13fff640 1 --2- 172.21.15.100:0/3070989391 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc14070bd0 0x7fbc141540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.186+0000 7fbc13fff640 1 -- 172.21.15.100:0/3070989391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc14155330 msgr2=0x7fbc141564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:29.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.186+0000 7fbc13fff640 1 --2- 172.21.15.100:0/3070989391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc14155330 0x7fbc141564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.186+0000 7fbc13fff640 1 -- 172.21.15.100:0/3070989391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc141f11f0 con 0x7fbc140cc4c0 2024-09-15T21:46:29.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.186+0000 7fbc13fff640 1 --2- 172.21.15.100:0/3070989391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc140cc4c0 0x7fbc14155f80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800c7d0 tx=0x7fbc0800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:29.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.186+0000 7fbc117fa640 1 -- 172.21.15.100:0/3070989391 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc08017070 con 0x7fbc140cc4c0 2024-09-15T21:46:29.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.186+0000 7fbc117fa640 1 -- 172.21.15.100:0/3070989391 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc0800a640 con 0x7fbc140cc4c0 2024-09-15T21:46:29.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.186+0000 7fbc1afcf640 1 -- 172.21.15.100:0/3070989391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc141f14d0 con 0x7fbc140cc4c0 2024-09-15T21:46:29.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.186+0000 7fbc117fa640 1 -- 172.21.15.100:0/3070989391 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0800aa10 con 0x7fbc140cc4c0 2024-09-15T21:46:29.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.186+0000 7fbc1afcf640 1 -- 172.21.15.100:0/3070989391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc141f1a90 con 0x7fbc140cc4c0 2024-09-15T21:46:29.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.187+0000 7fbc1afcf640 1 -- 172.21.15.100:0/3070989391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc14153950 con 0x7fbc140cc4c0 2024-09-15T21:46:29.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.187+0000 7fbc117fa640 1 -- 172.21.15.100:0/3070989391 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbc0801a070 con 0x7fbc140cc4c0 2024-09-15T21:46:29.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.188+0000 7fbc117fa640 1 --2- 172.21.15.100:0/3070989391 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbc0006c2b0 0x7fbc0006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:29.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.188+0000 7fbc18d6b640 1 --2- 172.21.15.100:0/3070989391 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbc0006c2b0 0x7fbc0006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:29.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.188+0000 7fbc117fa640 1 -- 172.21.15.100:0/3070989391 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbc0808ab50 con 0x7fbc140cc4c0 2024-09-15T21:46:29.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.188+0000 7fbc18d6b640 1 --2- 172.21.15.100:0/3070989391 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbc0006c2b0 0x7fbc0006e770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbc04034000 tx=0x7fbc0402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:29.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.192+0000 7fbc117fa640 1 -- 172.21.15.100:0/3070989391 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbc08059240 con 0x7fbc140cc4c0 2024-09-15T21:46:29.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.425+0000 7fbc1afcf640 1 -- 172.21.15.100:0/3070989391 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fbc1414f0e0 con 0x7fbc140cc4c0 2024-09-15T21:46:29.427 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.425+0000 7fbc117fa640 1 -- 172.21.15.100:0/3070989391 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1226 (secure 0 0 0) 0x7fbc08058b40 con 0x7fbc140cc4c0 2024-09-15T21:46:29.427 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:29.427 INFO:teuthology.orchestra.run.smithi100.stdout:{"fsid":"30e2fff4-2171-4531-8806-0e03b63f494f","health":{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":29,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":22,"num_osds":8,"num_up_osds":8,"osd_up_since":1726436773,"num_in_osds":8,"osd_in_since":1726436761,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":60},{"state_name":"creating+peering","count":37}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":224161792,"bytes_avail":772869951488,"bytes_total":773094113280,"inactive_pgs_ratio":0.38144329190254211},"fsmap":{"epoch":8,"btime":"2024-09-15T21:46:27:432116+0000","id":1,"up":1,"in":1,"max":5,"by_rank":[{"filesystem_id":1,"rank":0,"name":"e","status":"up:creating","gid":4293}],"up:standby":5},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-15T21:45:59.747901+0000","services":{}},"progress_events":{}} 2024-09-15T21:46:29.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.427+0000 7fbc1afcf640 1 -- 172.21.15.100:0/3070989391 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbc0006c2b0 msgr2=0x7fbc0006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:29.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.427+0000 7fbc1afcf640 1 --2- 172.21.15.100:0/3070989391 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbc0006c2b0 0x7fbc0006e770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbc04034000 tx=0x7fbc0402e000 comp rx=0 tx=0).stop 2024-09-15T21:46:29.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.427+0000 7fbc1afcf640 1 -- 172.21.15.100:0/3070989391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc140cc4c0 msgr2=0x7fbc14155f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:29.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.427+0000 7fbc1afcf640 1 --2- 172.21.15.100:0/3070989391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc140cc4c0 0x7fbc14155f80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800c7d0 tx=0x7fbc0800cc90 comp rx=0 tx=0).stop 2024-09-15T21:46:29.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.427+0000 7fbc1afcf640 1 -- 172.21.15.100:0/3070989391 shutdown_connections 2024-09-15T21:46:29.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.427+0000 7fbc1afcf640 1 --2- 172.21.15.100:0/3070989391 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbc0006c2b0 0x7fbc0006e770 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.427+0000 7fbc1afcf640 1 --2- 172.21.15.100:0/3070989391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc14155330 0x7fbc141564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.427+0000 7fbc1afcf640 1 --2- 172.21.15.100:0/3070989391 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc140cc4c0 0x7fbc14155f80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.427+0000 7fbc1afcf640 1 --2- 172.21.15.100:0/3070989391 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc14070bd0 0x7fbc141540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.427+0000 7fbc1afcf640 1 -- 172.21.15.100:0/3070989391 >> 172.21.15.100:0/3070989391 conn(0x7fbc1406c210 msgr2=0x7fbc14152380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:29.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.427+0000 7fbc1afcf640 1 -- 172.21.15.100:0/3070989391 shutdown_connections 2024-09-15T21:46:29.430 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.427+0000 7fbc1afcf640 1 -- 172.21.15.100:0/3070989391 wait complete. 2024-09-15T21:46:29.440 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-15T21:46:29.440 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:46:29.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.536+0000 7f779e911640 1 Processor -- start 2024-09-15T21:46:29.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.537+0000 7f779e911640 1 -- start start 2024-09-15T21:46:29.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.537+0000 7f779e911640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7798072f20 0x7f77980733b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:29.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.537+0000 7f779e911640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77980738f0 con 0x7f779814e9c0 2024-09-15T21:46:29.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.537+0000 7f779e911640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7798157e30 con 0x7f77980c4600 2024-09-15T21:46:29.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.537+0000 7f779e911640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f77981580d0 con 0x7f7798072f20 2024-09-15T21:46:29.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.537+0000 7f7797fff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f779814e9c0 0x7f779806df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:51792/0 (socket says 172.21.15.100:51792) 2024-09-15T21:46:29.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.537+0000 7f7797fff640 1 -- 172.21.15.100:0/2828438155 learned_addr learned my addr 172.21.15.100:0/2828438155 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:29.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.537+0000 7f779ceae640 1 --2- 172.21.15.100:0/2828438155 >> v2:172.21.15.100:3300/0 conn(0x7f7798072f20 0x7f77980733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:29.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.538+0000 7f779ceae640 1 -- 172.21.15.100:0/2828438155 >> v1:172.21.15.122:6789/0 conn(0x7f77980c4600 legacy=0x7f779806ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:29.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.538+0000 7f779ceae640 1 -- 172.21.15.100:0/2828438155 >> v1:172.21.15.100:6789/0 conn(0x7f779814e9c0 legacy=0x7f779806df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:29.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.538+0000 7f779ceae640 1 -- 172.21.15.100:0/2828438155 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77981583b0 con 0x7f7798072f20 2024-09-15T21:46:29.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.538+0000 7f779ceae640 1 --2- 172.21.15.100:0/2828438155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7798072f20 0x7f77980733b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7790009d90 tx=0x7f779002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8607d0c0c1a60e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:29.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.538+0000 7f77977fe640 1 -- 172.21.15.100:0/2828438155 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790007740 con 0x7f7798072f20 2024-09-15T21:46:29.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.539+0000 7f77977fe640 1 -- 172.21.15.100:0/2828438155 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77900078e0 con 0x7f7798072f20 2024-09-15T21:46:29.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.539+0000 7f77977fe640 1 -- 172.21.15.100:0/2828438155 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790007cb0 con 0x7f7798072f20 2024-09-15T21:46:29.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.539+0000 7f779e911640 1 -- 172.21.15.100:0/2828438155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7798072f20 msgr2=0x7f77980733b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:29.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.539+0000 7f779e911640 1 --2- 172.21.15.100:0/2828438155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7798072f20 0x7f77980733b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7790009d90 tx=0x7f779002a730 comp rx=0 tx=0).stop 2024-09-15T21:46:29.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.539+0000 7f779e911640 1 -- 172.21.15.100:0/2828438155 shutdown_connections 2024-09-15T21:46:29.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.539+0000 7f779e911640 1 --2- 172.21.15.100:0/2828438155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7798072f20 0x7f77980733b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.539+0000 7f779e911640 1 -- 172.21.15.100:0/2828438155 >> 172.21.15.100:0/2828438155 conn(0x7f77980ca430 msgr2=0x7f77980ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:29.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.539+0000 7f779e911640 1 -- 172.21.15.100:0/2828438155 shutdown_connections 2024-09-15T21:46:29.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.539+0000 7f779e911640 1 -- 172.21.15.100:0/2828438155 wait complete. 2024-09-15T21:46:29.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.540+0000 7f779e911640 1 Processor -- start 2024-09-15T21:46:29.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.540+0000 7f779e911640 1 -- start start 2024-09-15T21:46:29.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.541+0000 7f779e911640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7798072f20 0x7f77981e3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:29.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.541+0000 7f779e911640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77980c4600 0x7f77980ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:29.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.541+0000 7f779e911640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f779814e9c0 0x7f77981e2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:29.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.541+0000 7f779e911640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f77980caf20 con 0x7f7798072f20 2024-09-15T21:46:29.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.541+0000 7f779e911640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7798158970 con 0x7f77980c4600 2024-09-15T21:46:29.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.541+0000 7f779e911640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f779814eda0 con 0x7f779814e9c0 2024-09-15T21:46:29.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.541+0000 7f7797fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7798072f20 0x7f77981e3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:29.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.541+0000 7f7797fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7798072f20 0x7f77981e3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43726/0 (socket says 172.21.15.100:43726) 2024-09-15T21:46:29.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.541+0000 7f7797fff640 1 -- 172.21.15.100:0/227293521 learned_addr learned my addr 172.21.15.100:0/227293521 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:29.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.541+0000 7f779ceae640 1 --2- 172.21.15.100:0/227293521 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f779814e9c0 0x7f77981e2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:29.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.541+0000 7f778ffff640 1 --2- 172.21.15.100:0/227293521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77980c4600 0x7f77980ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:29.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.541+0000 7f7797fff640 1 -- 172.21.15.100:0/227293521 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f779814e9c0 msgr2=0x7f77981e2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:29.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.541+0000 7f7797fff640 1 --2- 172.21.15.100:0/227293521 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f779814e9c0 0x7f77981e2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.541+0000 7f7797fff640 1 -- 172.21.15.100:0/227293521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77980c4600 msgr2=0x7f77980ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:29.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.541+0000 7f7797fff640 1 --2- 172.21.15.100:0/227293521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77980c4600 0x7f77980ccb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.541+0000 7f7797fff640 1 -- 172.21.15.100:0/227293521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77981e4100 con 0x7f7798072f20 2024-09-15T21:46:29.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.541+0000 7f7797fff640 1 --2- 172.21.15.100:0/227293521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7798072f20 0x7f77981e3ac0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f778800c3f0 tx=0x7f778800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:29.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.542+0000 7f77957fa640 1 -- 172.21.15.100:0/227293521 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7788017070 con 0x7f7798072f20 2024-09-15T21:46:29.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.542+0000 7f77957fa640 1 -- 172.21.15.100:0/227293521 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7788005ce0 con 0x7f7798072f20 2024-09-15T21:46:29.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.542+0000 7f77957fa640 1 -- 172.21.15.100:0/227293521 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77880072f0 con 0x7f7798072f20 2024-09-15T21:46:29.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.542+0000 7f779e911640 1 -- 172.21.15.100:0/227293521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f779815b2f0 con 0x7f7798072f20 2024-09-15T21:46:29.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.542+0000 7f779e911640 1 -- 172.21.15.100:0/227293521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f779815b8b0 con 0x7f7798072f20 2024-09-15T21:46:29.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.543+0000 7f779e911640 1 -- 172.21.15.100:0/227293521 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77980733b0 con 0x7f7798072f20 2024-09-15T21:46:29.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.543+0000 7f77957fa640 1 -- 172.21.15.100:0/227293521 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f778801a070 con 0x7f7798072f20 2024-09-15T21:46:29.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.543+0000 7f77957fa640 1 --2- 172.21.15.100:0/227293521 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f777806c2b0 0x7f777806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:29.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.544+0000 7f77957fa640 1 -- 172.21.15.100:0/227293521 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f778808a7e0 con 0x7f7798072f20 2024-09-15T21:46:29.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.544+0000 7f778ffff640 1 --2- 172.21.15.100:0/227293521 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f777806c2b0 0x7f777806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:29.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.544+0000 7f778ffff640 1 --2- 172.21.15.100:0/227293521 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f777806c2b0 0x7f777806e770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f77980c95d0 tx=0x7f7780009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:29.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.548+0000 7f77957fa640 1 -- 172.21.15.100:0/227293521 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7788058ec0 con 0x7f7798072f20 2024-09-15T21:46:29.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.712+0000 7f779e911640 1 -- 172.21.15.100:0/227293521 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7798072b10 con 0x7f777806c2b0 2024-09-15T21:46:29.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.717+0000 7f77957fa640 1 -- 172.21.15.100:0/227293521 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248805 (secure 0 0 0) 0x7f7798072b10 con 0x7f777806c2b0 2024-09-15T21:46:29.722 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:29.723 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-15T21:46:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.720+0000 7f779e911640 1 -- 172.21.15.100:0/227293521 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f777806c2b0 msgr2=0x7f777806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.720+0000 7f779e911640 1 --2- 172.21.15.100:0/227293521 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f777806c2b0 0x7f777806e770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f77980c95d0 tx=0x7f7780009040 comp rx=0 tx=0).stop 2024-09-15T21:46:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.720+0000 7f779e911640 1 -- 172.21.15.100:0/227293521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7798072f20 msgr2=0x7f77981e3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.720+0000 7f779e911640 1 --2- 172.21.15.100:0/227293521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7798072f20 0x7f77981e3ac0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f778800c3f0 tx=0x7f778800c8b0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.721+0000 7f779e911640 1 -- 172.21.15.100:0/227293521 shutdown_connections 2024-09-15T21:46:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.721+0000 7f779e911640 1 --2- 172.21.15.100:0/227293521 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f777806c2b0 0x7f777806e770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.721+0000 7f779e911640 1 --2- 172.21.15.100:0/227293521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77980c4600 0x7f77980ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.721+0000 7f779e911640 1 --2- 172.21.15.100:0/227293521 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f779814e9c0 0x7f77981e2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.721+0000 7f779e911640 1 --2- 172.21.15.100:0/227293521 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7798072f20 0x7f77981e3ac0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:29.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.721+0000 7f779e911640 1 -- 172.21.15.100:0/227293521 >> 172.21.15.100:0/227293521 conn(0x7f77980ca430 msgr2=0x7f779814b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:29.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.721+0000 7f779e911640 1 -- 172.21.15.100:0/227293521 shutdown_connections 2024-09-15T21:46:29.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:29.721+0000 7f779e911640 1 -- 172.21.15.100:0/227293521 wait complete. 2024-09-15T21:46:29.733 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-15T21:46:27.804526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":59,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218908,"kb_used_data":2684,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755812,"statfs":{"total":773094113280,"available":772869951488,"internally_reserved":0,"allocated":2748416,"data_stored":1686550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000848"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.421348+0000","last_change":"2024-09-15T21:46:22.421348+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.421348+0000","last_undegraded":"2024-09-15T21:46:22.421348+0000","last_fullsized":"2024-09-15T21:46:22.421348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808513+0000","last_change":"2024-09-15T21:46:23.418716+0000","last_active":"2024-09-15T21:46:24.808513+0000","last_peered":"2024-09-15T21:46:24.808513+0000","last_clean":"2024-09-15T21:46:24.808513+0000","last_became_active":"2024-09-15T21:46:23.418471+0000","last_became_peered":"2024-09-15T21:46:23.418471+0000","last_unstale":"2024-09-15T21:46:24.808513+0000","last_undegraded":"2024-09-15T21:46:24.808513+0000","last_fullsized":"2024-09-15T21:46:24.808513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:49.599801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431027+0000","last_change":"2024-09-15T21:46:23.426404+0000","last_active":"2024-09-15T21:46:23.431027+0000","last_peered":"2024-09-15T21:46:23.431027+0000","last_clean":"2024-09-15T21:46:23.431027+0000","last_became_active":"2024-09-15T21:46:23.426146+0000","last_became_peered":"2024-09-15T21:46:23.426146+0000","last_unstale":"2024-09-15T21:46:23.431027+0000","last_undegraded":"2024-09-15T21:46:23.431027+0000","last_fullsized":"2024-09-15T21:46:23.431027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:40.061508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419990+0000","last_change":"2024-09-15T21:46:22.419990+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419990+0000","last_undegraded":"2024-09-15T21:46:22.419990+0000","last_fullsized":"2024-09-15T21:46:22.419990+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418695+0000","last_change":"2024-09-15T21:46:22.418695+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418695+0000","last_undegraded":"2024-09-15T21:46:22.418695+0000","last_fullsized":"2024-09-15T21:46:22.418695+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418843+0000","last_change":"2024-09-15T21:46:22.418843+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418843+0000","last_undegraded":"2024-09-15T21:46:22.418843+0000","last_fullsized":"2024-09-15T21:46:22.418843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419832+0000","last_change":"2024-09-15T21:46:22.419832+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419832+0000","last_undegraded":"2024-09-15T21:46:22.419832+0000","last_fullsized":"2024-09-15T21:46:22.419832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.438915+0000","last_change":"2024-09-15T21:46:23.437143+0000","last_active":"2024-09-15T21:46:23.438915+0000","last_peered":"2024-09-15T21:46:23.438915+0000","last_clean":"2024-09-15T21:46:23.438915+0000","last_became_active":"2024-09-15T21:46:23.436864+0000","last_became_peered":"2024-09-15T21:46:23.436864+0000","last_unstale":"2024-09-15T21:46:23.438915+0000","last_undegraded":"2024-09-15T21:46:23.438915+0000","last_fullsized":"2024-09-15T21:46:23.438915+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:12:37.128024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.420920+0000","last_change":"2024-09-15T21:46:22.420920+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.420920+0000","last_undegraded":"2024-09-15T21:46:22.420920+0000","last_fullsized":"2024-09-15T21:46:22.420920+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.422403+0000","last_change":"2024-09-15T21:46:22.422403+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.422403+0000","last_undegraded":"2024-09-15T21:46:22.422403+0000","last_fullsized":"2024-09-15T21:46:22.422403+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418976+0000","last_change":"2024-09-15T21:46:22.418976+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418976+0000","last_undegraded":"2024-09-15T21:46:22.418976+0000","last_fullsized":"2024-09-15T21:46:22.418976+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415544+0000","last_change":"2024-09-15T21:46:22.415544+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415544+0000","last_undegraded":"2024-09-15T21:46:22.415544+0000","last_fullsized":"2024-09-15T21:46:22.415544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431804+0000","last_change":"2024-09-15T21:46:23.427841+0000","last_active":"2024-09-15T21:46:23.431804+0000","last_peered":"2024-09-15T21:46:23.431804+0000","last_clean":"2024-09-15T21:46:23.431804+0000","last_became_active":"2024-09-15T21:46:23.427621+0000","last_became_peered":"2024-09-15T21:46:23.427621+0000","last_unstale":"2024-09-15T21:46:23.431804+0000","last_undegraded":"2024-09-15T21:46:23.431804+0000","last_fullsized":"2024-09-15T21:46:23.431804+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:17:35.795757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.428658+0000","last_change":"2024-09-15T21:46:23.424707+0000","last_active":"2024-09-15T21:46:23.428658+0000","last_peered":"2024-09-15T21:46:23.428658+0000","last_clean":"2024-09-15T21:46:23.428658+0000","last_became_active":"2024-09-15T21:46:23.424447+0000","last_became_peered":"2024-09-15T21:46:23.424447+0000","last_unstale":"2024-09-15T21:46:23.428658+0000","last_undegraded":"2024-09-15T21:46:23.428658+0000","last_fullsized":"2024-09-15T21:46:23.428658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:28:21.800777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808321+0000","last_change":"2024-09-15T21:46:23.430297+0000","last_active":"2024-09-15T21:46:24.808321+0000","last_peered":"2024-09-15T21:46:24.808321+0000","last_clean":"2024-09-15T21:46:24.808321+0000","last_became_active":"2024-09-15T21:46:23.430070+0000","last_became_peered":"2024-09-15T21:46:23.430070+0000","last_unstale":"2024-09-15T21:46:24.808321+0000","last_undegraded":"2024-09-15T21:46:24.808321+0000","last_fullsized":"2024-09-15T21:46:24.808321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:03:19.615022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415614+0000","last_change":"2024-09-15T21:46:22.415614+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415614+0000","last_undegraded":"2024-09-15T21:46:22.415614+0000","last_fullsized":"2024-09-15T21:46:22.415614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415415+0000","last_change":"2024-09-15T21:46:22.415415+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415415+0000","last_undegraded":"2024-09-15T21:46:22.415415+0000","last_fullsized":"2024-09-15T21:46:22.415415+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415507+0000","last_change":"2024-09-15T21:46:22.415507+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415507+0000","last_undegraded":"2024-09-15T21:46:22.415507+0000","last_fullsized":"2024-09-15T21:46:22.415507+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418352+0000","last_change":"2024-09-15T21:46:22.418352+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418352+0000","last_undegraded":"2024-09-15T21:46:22.418352+0000","last_fullsized":"2024-09-15T21:46:22.418352+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.420228+0000","last_change":"2024-09-15T21:46:22.420228+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.420228+0000","last_undegraded":"2024-09-15T21:46:22.420228+0000","last_fullsized":"2024-09-15T21:46:22.420228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433725+0000","last_change":"2024-09-15T21:46:23.428323+0000","last_active":"2024-09-15T21:46:23.433725+0000","last_peered":"2024-09-15T21:46:23.433725+0000","last_clean":"2024-09-15T21:46:23.433725+0000","last_became_active":"2024-09-15T21:46:23.428016+0000","last_became_peered":"2024-09-15T21:46:23.428016+0000","last_unstale":"2024-09-15T21:46:23.433725+0000","last_undegraded":"2024-09-15T21:46:23.433725+0000","last_fullsized":"2024-09-15T21:46:23.433725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:24:06.673523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418399+0000","last_change":"2024-09-15T21:46:22.418399+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418399+0000","last_undegraded":"2024-09-15T21:46:22.418399+0000","last_fullsized":"2024-09-15T21:46:22.418399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433661+0000","last_change":"2024-09-15T21:46:23.431527+0000","last_active":"2024-09-15T21:46:23.433661+0000","last_peered":"2024-09-15T21:46:23.433661+0000","last_clean":"2024-09-15T21:46:23.433661+0000","last_became_active":"2024-09-15T21:46:23.430719+0000","last_became_peered":"2024-09-15T21:46:23.430719+0000","last_unstale":"2024-09-15T21:46:23.433661+0000","last_undegraded":"2024-09-15T21:46:23.433661+0000","last_fullsized":"2024-09-15T21:46:23.433661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:25.362126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415180+0000","last_change":"2024-09-15T21:46:22.415180+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415180+0000","last_undegraded":"2024-09-15T21:46:22.415180+0000","last_fullsized":"2024-09-15T21:46:22.415180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.436122+0000","last_change":"2024-09-15T21:46:23.432646+0000","last_active":"2024-09-15T21:46:23.436122+0000","last_peered":"2024-09-15T21:46:23.436122+0000","last_clean":"2024-09-15T21:46:23.436122+0000","last_became_active":"2024-09-15T21:46:23.432317+0000","last_became_peered":"2024-09-15T21:46:23.432317+0000","last_unstale":"2024-09-15T21:46:23.436122+0000","last_undegraded":"2024-09-15T21:46:23.436122+0000","last_fullsized":"2024-09-15T21:46:23.436122+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:34.359059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.421958+0000","last_change":"2024-09-15T21:46:22.421958+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.421958+0000","last_undegraded":"2024-09-15T21:46:22.421958+0000","last_fullsized":"2024-09-15T21:46:22.421958+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808369+0000","last_change":"2024-09-15T21:46:23.421749+0000","last_active":"2024-09-15T21:46:24.808369+0000","last_peered":"2024-09-15T21:46:24.808369+0000","last_clean":"2024-09-15T21:46:24.808369+0000","last_became_active":"2024-09-15T21:46:23.421501+0000","last_became_peered":"2024-09-15T21:46:23.421501+0000","last_unstale":"2024-09-15T21:46:24.808369+0000","last_undegraded":"2024-09-15T21:46:24.808369+0000","last_fullsized":"2024-09-15T21:46:24.808369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:55.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.436035+0000","last_change":"2024-09-15T21:46:23.432591+0000","last_active":"2024-09-15T21:46:23.436035+0000","last_peered":"2024-09-15T21:46:23.436035+0000","last_clean":"2024-09-15T21:46:23.436035+0000","last_became_active":"2024-09-15T21:46:23.432326+0000","last_became_peered":"2024-09-15T21:46:23.432326+0000","last_unstale":"2024-09-15T21:46:23.436035+0000","last_undegraded":"2024-09-15T21:46:23.436035+0000","last_fullsized":"2024-09-15T21:46:23.436035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:46:56.508701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.428496+0000","last_change":"2024-09-15T21:46:23.423428+0000","last_active":"2024-09-15T21:46:23.428496+0000","last_peered":"2024-09-15T21:46:23.428496+0000","last_clean":"2024-09-15T21:46:23.428496+0000","last_became_active":"2024-09-15T21:46:23.422734+0000","last_became_peered":"2024-09-15T21:46:23.422734+0000","last_unstale":"2024-09-15T21:46:23.428496+0000","last_undegraded":"2024-09-15T21:46:23.428496+0000","last_fullsized":"2024-09-15T21:46:23.428496+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:21:02.082197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.417891+0000","last_change":"2024-09-15T21:46:22.417891+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.417891+0000","last_undegraded":"2024-09-15T21:46:22.417891+0000","last_fullsized":"2024-09-15T21:46:22.417891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414635+0000","last_change":"2024-09-15T21:46:22.414635+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414635+0000","last_undegraded":"2024-09-15T21:46:22.414635+0000","last_fullsized":"2024-09-15T21:46:22.414635+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418469+0000","last_change":"2024-09-15T21:46:22.418469+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418469+0000","last_undegraded":"2024-09-15T21:46:22.418469+0000","last_fullsized":"2024-09-15T21:46:22.418469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414859+0000","last_change":"2024-09-15T21:46:22.414859+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414859+0000","last_undegraded":"2024-09-15T21:46:22.414859+0000","last_fullsized":"2024-09-15T21:46:22.414859+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.424245+0000","last_change":"2024-09-15T21:46:22.429407+0000","last_active":"2024-09-15T21:46:23.424245+0000","last_peered":"2024-09-15T21:46:23.424245+0000","last_clean":"2024-09-15T21:46:23.424245+0000","last_became_active":"2024-09-15T21:46:22.429107+0000","last_became_peered":"2024-09-15T21:46:22.429107+0000","last_unstale":"2024-09-15T21:46:23.424245+0000","last_undegraded":"2024-09-15T21:46:23.424245+0000","last_fullsized":"2024-09-15T21:46:23.424245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:14:54.680349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.438772+0000","last_change":"2024-09-15T21:46:23.434198+0000","last_active":"2024-09-15T21:46:23.438772+0000","last_peered":"2024-09-15T21:46:23.438772+0000","last_clean":"2024-09-15T21:46:23.438772+0000","last_became_active":"2024-09-15T21:46:23.433944+0000","last_became_peered":"2024-09-15T21:46:23.433944+0000","last_unstale":"2024-09-15T21:46:23.438772+0000","last_undegraded":"2024-09-15T21:46:23.438772+0000","last_fullsized":"2024-09-15T21:46:23.438772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:20:05.953148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.428922+0000","last_change":"2024-09-15T21:46:22.425571+0000","last_active":"2024-09-15T21:46:22.428922+0000","last_peered":"2024-09-15T21:46:22.428922+0000","last_clean":"2024-09-15T21:46:22.428922+0000","last_became_active":"2024-09-15T21:46:22.425315+0000","last_became_peered":"2024-09-15T21:46:22.425315+0000","last_unstale":"2024-09-15T21:46:22.428922+0000","last_undegraded":"2024-09-15T21:46:22.428922+0000","last_fullsized":"2024-09-15T21:46:22.428922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:37.405710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.423270+0000","last_change":"2024-09-15T21:46:22.425492+0000","last_active":"2024-09-15T21:46:23.423270+0000","last_peered":"2024-09-15T21:46:23.423270+0000","last_clean":"2024-09-15T21:46:23.423270+0000","last_became_active":"2024-09-15T21:46:22.425226+0000","last_became_peered":"2024-09-15T21:46:22.425226+0000","last_unstale":"2024-09-15T21:46:23.423270+0000","last_undegraded":"2024-09-15T21:46:23.423270+0000","last_fullsized":"2024-09-15T21:46:23.423270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:24.322384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433622+0000","last_change":"2024-09-15T21:46:23.429140+0000","last_active":"2024-09-15T21:46:23.433622+0000","last_peered":"2024-09-15T21:46:23.433622+0000","last_clean":"2024-09-15T21:46:23.433622+0000","last_became_active":"2024-09-15T21:46:23.428902+0000","last_became_peered":"2024-09-15T21:46:23.428902+0000","last_unstale":"2024-09-15T21:46:23.433622+0000","last_undegraded":"2024-09-15T21:46:23.433622+0000","last_fullsized":"2024-09-15T21:46:23.433622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:44:54.129937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.431357+0000","last_change":"2024-09-15T21:46:22.428319+0000","last_active":"2024-09-15T21:46:22.431357+0000","last_peered":"2024-09-15T21:46:22.431357+0000","last_clean":"2024-09-15T21:46:22.431357+0000","last_became_active":"2024-09-15T21:46:22.428061+0000","last_became_peered":"2024-09-15T21:46:22.428061+0000","last_unstale":"2024-09-15T21:46:22.431357+0000","last_undegraded":"2024-09-15T21:46:22.431357+0000","last_fullsized":"2024-09-15T21:46:22.431357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:34.432046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808315+0000","last_change":"2024-09-15T21:46:23.427442+0000","last_active":"2024-09-15T21:46:24.808315+0000","last_peered":"2024-09-15T21:46:24.808315+0000","last_clean":"2024-09-15T21:46:24.808315+0000","last_became_active":"2024-09-15T21:46:23.427200+0000","last_became_peered":"2024-09-15T21:46:23.427200+0000","last_unstale":"2024-09-15T21:46:24.808315+0000","last_undegraded":"2024-09-15T21:46:24.808315+0000","last_fullsized":"2024-09-15T21:46:24.808315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:27.248545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433993+0000","last_change":"2024-09-15T21:46:22.430599+0000","last_active":"2024-09-15T21:46:22.433993+0000","last_peered":"2024-09-15T21:46:22.433993+0000","last_clean":"2024-09-15T21:46:22.433993+0000","last_became_active":"2024-09-15T21:46:22.430327+0000","last_became_peered":"2024-09-15T21:46:22.430327+0000","last_unstale":"2024-09-15T21:46:22.433993+0000","last_undegraded":"2024-09-15T21:46:22.433993+0000","last_fullsized":"2024-09-15T21:46:22.433993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:42:31.882474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.416669+0000","last_change":"2024-09-15T21:46:22.416669+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.416669+0000","last_undegraded":"2024-09-15T21:46:22.416669+0000","last_fullsized":"2024-09-15T21:46:22.416669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.429815+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:22.429815+0000","last_peered":"2024-09-15T21:46:22.429815+0000","last_clean":"2024-09-15T21:46:22.429815+0000","last_became_active":"2024-09-15T21:46:22.425236+0000","last_became_peered":"2024-09-15T21:46:22.425236+0000","last_unstale":"2024-09-15T21:46:22.429815+0000","last_undegraded":"2024-09-15T21:46:22.429815+0000","last_fullsized":"2024-09-15T21:46:22.429815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:01.525976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433802+0000","last_change":"2024-09-15T21:46:23.429587+0000","last_active":"2024-09-15T21:46:23.433802+0000","last_peered":"2024-09-15T21:46:23.433802+0000","last_clean":"2024-09-15T21:46:23.433802+0000","last_became_active":"2024-09-15T21:46:23.429334+0000","last_became_peered":"2024-09-15T21:46:23.429334+0000","last_unstale":"2024-09-15T21:46:23.433802+0000","last_undegraded":"2024-09-15T21:46:23.433802+0000","last_fullsized":"2024-09-15T21:46:23.433802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:31.263309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808325+0000","last_change":"2024-09-15T21:46:22.421964+0000","last_active":"2024-09-15T21:46:24.808325+0000","last_peered":"2024-09-15T21:46:24.808325+0000","last_clean":"2024-09-15T21:46:24.808325+0000","last_became_active":"2024-09-15T21:46:22.421661+0000","last_became_peered":"2024-09-15T21:46:22.421661+0000","last_unstale":"2024-09-15T21:46:24.808325+0000","last_undegraded":"2024-09-15T21:46:24.808325+0000","last_fullsized":"2024-09-15T21:46:24.808325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:07.395070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414271+0000","last_change":"2024-09-15T21:46:22.414271+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414271+0000","last_undegraded":"2024-09-15T21:46:22.414271+0000","last_fullsized":"2024-09-15T21:46:22.414271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433356+0000","last_change":"2024-09-15T21:46:22.426151+0000","last_active":"2024-09-15T21:46:22.433356+0000","last_peered":"2024-09-15T21:46:22.433356+0000","last_clean":"2024-09-15T21:46:22.433356+0000","last_became_active":"2024-09-15T21:46:22.425866+0000","last_became_peered":"2024-09-15T21:46:22.425866+0000","last_unstale":"2024-09-15T21:46:22.433356+0000","last_undegraded":"2024-09-15T21:46:22.433356+0000","last_fullsized":"2024-09-15T21:46:22.433356+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:20:18.129618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419893+0000","last_change":"2024-09-15T21:46:22.419893+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419893+0000","last_undegraded":"2024-09-15T21:46:22.419893+0000","last_fullsized":"2024-09-15T21:46:22.419893+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.417870+0000","last_change":"2024-09-15T21:46:22.427415+0000","last_active":"2024-09-15T21:46:23.417870+0000","last_peered":"2024-09-15T21:46:23.417870+0000","last_clean":"2024-09-15T21:46:23.417870+0000","last_became_active":"2024-09-15T21:46:22.427087+0000","last_became_peered":"2024-09-15T21:46:22.427087+0000","last_unstale":"2024-09-15T21:46:23.417870+0000","last_undegraded":"2024-09-15T21:46:23.417870+0000","last_fullsized":"2024-09-15T21:46:23.417870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:09.365202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431262+0000","last_change":"2024-09-15T21:46:23.427238+0000","last_active":"2024-09-15T21:46:23.431262+0000","last_peered":"2024-09-15T21:46:23.431262+0000","last_clean":"2024-09-15T21:46:23.431262+0000","last_became_active":"2024-09-15T21:46:23.426993+0000","last_became_peered":"2024-09-15T21:46:23.426993+0000","last_unstale":"2024-09-15T21:46:23.431262+0000","last_undegraded":"2024-09-15T21:46:23.431262+0000","last_fullsized":"2024-09-15T21:46:23.431262+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:45:36.125958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.431315+0000","last_change":"2024-09-15T21:46:22.427164+0000","last_active":"2024-09-15T21:46:22.431315+0000","last_peered":"2024-09-15T21:46:22.431315+0000","last_clean":"2024-09-15T21:46:22.431315+0000","last_became_active":"2024-09-15T21:46:22.426917+0000","last_became_peered":"2024-09-15T21:46:22.426917+0000","last_unstale":"2024-09-15T21:46:22.431315+0000","last_undegraded":"2024-09-15T21:46:22.431315+0000","last_fullsized":"2024-09-15T21:46:22.431315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:12:47.575708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418200+0000","last_change":"2024-09-15T21:46:22.418200+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418200+0000","last_undegraded":"2024-09-15T21:46:22.418200+0000","last_fullsized":"2024-09-15T21:46:22.418200+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.428957+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:22.428957+0000","last_peered":"2024-09-15T21:46:22.428957+0000","last_clean":"2024-09-15T21:46:22.428957+0000","last_became_active":"2024-09-15T21:46:22.425237+0000","last_became_peered":"2024-09-15T21:46:22.425237+0000","last_unstale":"2024-09-15T21:46:22.428957+0000","last_undegraded":"2024-09-15T21:46:22.428957+0000","last_fullsized":"2024-09-15T21:46:22.428957+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:46.301372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.417788+0000","last_change":"2024-09-15T21:46:22.417788+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.417788+0000","last_undegraded":"2024-09-15T21:46:22.417788+0000","last_fullsized":"2024-09-15T21:46:22.417788+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.423016+0000","last_change":"2024-09-15T21:46:22.419905+0000","last_active":"2024-09-15T21:46:22.423016+0000","last_peered":"2024-09-15T21:46:22.423016+0000","last_clean":"2024-09-15T21:46:22.423016+0000","last_became_active":"2024-09-15T21:46:22.419620+0000","last_became_peered":"2024-09-15T21:46:22.419620+0000","last_unstale":"2024-09-15T21:46:22.423016+0000","last_undegraded":"2024-09-15T21:46:22.423016+0000","last_fullsized":"2024-09-15T21:46:22.423016+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:41.513967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419332+0000","last_change":"2024-09-15T21:46:22.419332+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419332+0000","last_undegraded":"2024-09-15T21:46:22.419332+0000","last_fullsized":"2024-09-15T21:46:22.419332+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433527+0000","last_change":"2024-09-15T21:46:22.429661+0000","last_active":"2024-09-15T21:46:22.433527+0000","last_peered":"2024-09-15T21:46:22.433527+0000","last_clean":"2024-09-15T21:46:22.433527+0000","last_became_active":"2024-09-15T21:46:22.429250+0000","last_became_peered":"2024-09-15T21:46:22.429250+0000","last_unstale":"2024-09-15T21:46:22.433527+0000","last_undegraded":"2024-09-15T21:46:22.433527+0000","last_fullsized":"2024-09-15T21:46:22.433527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:52:16.852861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418349+0000","last_change":"2024-09-15T21:46:22.418349+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418349+0000","last_undegraded":"2024-09-15T21:46:22.418349+0000","last_fullsized":"2024-09-15T21:46:22.418349+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.430022+0000","last_change":"2024-09-15T21:46:22.425870+0000","last_active":"2024-09-15T21:46:22.430022+0000","last_peered":"2024-09-15T21:46:22.430022+0000","last_clean":"2024-09-15T21:46:22.430022+0000","last_became_active":"2024-09-15T21:46:22.425588+0000","last_became_peered":"2024-09-15T21:46:22.425588+0000","last_unstale":"2024-09-15T21:46:22.430022+0000","last_undegraded":"2024-09-15T21:46:22.430022+0000","last_fullsized":"2024-09-15T21:46:22.430022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:53.982102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.420429+0000","last_change":"2024-09-15T21:46:22.420429+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.420429+0000","last_undegraded":"2024-09-15T21:46:22.420429+0000","last_fullsized":"2024-09-15T21:46:22.420429+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808420+0000","last_change":"2024-09-15T21:46:22.425072+0000","last_active":"2024-09-15T21:46:24.808420+0000","last_peered":"2024-09-15T21:46:24.808420+0000","last_clean":"2024-09-15T21:46:24.808420+0000","last_became_active":"2024-09-15T21:46:22.424771+0000","last_became_peered":"2024-09-15T21:46:22.424771+0000","last_unstale":"2024-09-15T21:46:24.808420+0000","last_undegraded":"2024-09-15T21:46:24.808420+0000","last_fullsized":"2024-09-15T21:46:24.808420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:26.256785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415969+0000","last_change":"2024-09-15T21:46:22.415969+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415969+0000","last_undegraded":"2024-09-15T21:46:22.415969+0000","last_fullsized":"2024-09-15T21:46:22.415969+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.432304+0000","last_change":"2024-09-15T21:46:22.428668+0000","last_active":"2024-09-15T21:46:22.432304+0000","last_peered":"2024-09-15T21:46:22.432304+0000","last_clean":"2024-09-15T21:46:22.432304+0000","last_became_active":"2024-09-15T21:46:22.428341+0000","last_became_peered":"2024-09-15T21:46:22.428341+0000","last_unstale":"2024-09-15T21:46:22.432304+0000","last_undegraded":"2024-09-15T21:46:22.432304+0000","last_fullsized":"2024-09-15T21:46:22.432304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:25.844475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.418811+0000","last_change":"2024-09-15T21:46:22.418811+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.418811+0000","last_undegraded":"2024-09-15T21:46:22.418811+0000","last_fullsized":"2024-09-15T21:46:22.418811+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":167,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.422877+0000","last_change":"2024-09-15T21:46:14.328094+0000","last_active":"2024-09-15T21:46:22.422877+0000","last_peered":"2024-09-15T21:46:22.422877+0000","last_clean":"2024-09-15T21:46:22.422877+0000","last_became_active":"2024-09-15T21:46:14.326991+0000","last_became_peered":"2024-09-15T21:46:14.326991+0000","last_unstale":"2024-09-15T21:46:22.422877+0000","last_undegraded":"2024-09-15T21:46:22.422877+0000","last_fullsized":"2024-09-15T21:46:22.422877+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_clean_scrub_stamp":"2024-09-15T21:46:12.223400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:54:27.704130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.431406+0000","last_change":"2024-09-15T21:46:22.428887+0000","last_active":"2024-09-15T21:46:22.431406+0000","last_peered":"2024-09-15T21:46:22.431406+0000","last_clean":"2024-09-15T21:46:22.431406+0000","last_became_active":"2024-09-15T21:46:22.428619+0000","last_became_peered":"2024-09-15T21:46:22.428619+0000","last_unstale":"2024-09-15T21:46:22.431406+0000","last_undegraded":"2024-09-15T21:46:22.431406+0000","last_fullsized":"2024-09-15T21:46:22.431406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:43.510542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.438679+0000","last_change":"2024-09-15T21:46:23.435459+0000","last_active":"2024-09-15T21:46:23.438679+0000","last_peered":"2024-09-15T21:46:23.438679+0000","last_clean":"2024-09-15T21:46:23.438679+0000","last_became_active":"2024-09-15T21:46:23.435141+0000","last_became_peered":"2024-09-15T21:46:23.435141+0000","last_unstale":"2024-09-15T21:46:23.438679+0000","last_undegraded":"2024-09-15T21:46:23.438679+0000","last_fullsized":"2024-09-15T21:46:23.438679+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:14:38.434075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433564+0000","last_change":"2024-09-15T21:46:22.431113+0000","last_active":"2024-09-15T21:46:22.433564+0000","last_peered":"2024-09-15T21:46:22.433564+0000","last_clean":"2024-09-15T21:46:22.433564+0000","last_became_active":"2024-09-15T21:46:22.430832+0000","last_became_peered":"2024-09-15T21:46:22.430832+0000","last_unstale":"2024-09-15T21:46:22.433564+0000","last_undegraded":"2024-09-15T21:46:22.433564+0000","last_fullsized":"2024-09-15T21:46:22.433564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:08.068054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433942+0000","last_change":"2024-09-15T21:46:23.430396+0000","last_active":"2024-09-15T21:46:23.433942+0000","last_peered":"2024-09-15T21:46:23.433942+0000","last_clean":"2024-09-15T21:46:23.433942+0000","last_became_active":"2024-09-15T21:46:23.430138+0000","last_became_peered":"2024-09-15T21:46:23.430138+0000","last_unstale":"2024-09-15T21:46:23.433942+0000","last_undegraded":"2024-09-15T21:46:23.433942+0000","last_fullsized":"2024-09-15T21:46:23.433942+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:25:04.056726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.430035+0000","last_change":"2024-09-15T21:46:22.425808+0000","last_active":"2024-09-15T21:46:22.430035+0000","last_peered":"2024-09-15T21:46:22.430035+0000","last_clean":"2024-09-15T21:46:22.430035+0000","last_became_active":"2024-09-15T21:46:22.425510+0000","last_became_peered":"2024-09-15T21:46:22.425510+0000","last_unstale":"2024-09-15T21:46:22.430035+0000","last_undegraded":"2024-09-15T21:46:22.430035+0000","last_fullsized":"2024-09-15T21:46:22.430035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:13.502626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808338+0000","last_change":"2024-09-15T21:46:23.427051+0000","last_active":"2024-09-15T21:46:24.808338+0000","last_peered":"2024-09-15T21:46:24.808338+0000","last_clean":"2024-09-15T21:46:24.808338+0000","last_became_active":"2024-09-15T21:46:23.426185+0000","last_became_peered":"2024-09-15T21:46:23.426185+0000","last_unstale":"2024-09-15T21:46:24.808338+0000","last_undegraded":"2024-09-15T21:46:24.808338+0000","last_fullsized":"2024-09-15T21:46:24.808338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:48.318405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.429782+0000","last_change":"2024-09-15T21:46:22.424645+0000","last_active":"2024-09-15T21:46:22.429782+0000","last_peered":"2024-09-15T21:46:22.429782+0000","last_clean":"2024-09-15T21:46:22.429782+0000","last_became_active":"2024-09-15T21:46:22.421229+0000","last_became_peered":"2024-09-15T21:46:22.421229+0000","last_unstale":"2024-09-15T21:46:22.429782+0000","last_undegraded":"2024-09-15T21:46:22.429782+0000","last_fullsized":"2024-09-15T21:46:22.429782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:02.298935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.413899+0000","last_change":"2024-09-15T21:46:22.413899+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.413899+0000","last_undegraded":"2024-09-15T21:46:22.413899+0000","last_fullsized":"2024-09-15T21:46:22.413899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808438+0000","last_change":"2024-09-15T21:46:22.426492+0000","last_active":"2024-09-15T21:46:24.808438+0000","last_peered":"2024-09-15T21:46:24.808438+0000","last_clean":"2024-09-15T21:46:24.808438+0000","last_became_active":"2024-09-15T21:46:22.426210+0000","last_became_peered":"2024-09-15T21:46:22.426210+0000","last_unstale":"2024-09-15T21:46:24.808438+0000","last_undegraded":"2024-09-15T21:46:24.808438+0000","last_fullsized":"2024-09-15T21:46:24.808438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:05.248091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414998+0000","last_change":"2024-09-15T21:46:22.414998+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414998+0000","last_undegraded":"2024-09-15T21:46:22.414998+0000","last_fullsized":"2024-09-15T21:46:22.414998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433499+0000","last_change":"2024-09-15T21:46:22.430710+0000","last_active":"2024-09-15T21:46:22.433499+0000","last_peered":"2024-09-15T21:46:22.433499+0000","last_clean":"2024-09-15T21:46:22.433499+0000","last_became_active":"2024-09-15T21:46:22.430375+0000","last_became_peered":"2024-09-15T21:46:22.430375+0000","last_unstale":"2024-09-15T21:46:22.433499+0000","last_undegraded":"2024-09-15T21:46:22.433499+0000","last_fullsized":"2024-09-15T21:46:22.433499+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:30:17.852485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808447+0000","last_change":"2024-09-15T21:46:23.426729+0000","last_active":"2024-09-15T21:46:24.808447+0000","last_peered":"2024-09-15T21:46:24.808447+0000","last_clean":"2024-09-15T21:46:24.808447+0000","last_became_active":"2024-09-15T21:46:23.426495+0000","last_became_peered":"2024-09-15T21:46:23.426495+0000","last_unstale":"2024-09-15T21:46:24.808447+0000","last_undegraded":"2024-09-15T21:46:24.808447+0000","last_fullsized":"2024-09-15T21:46:24.808447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:03.277997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808436+0000","last_change":"2024-09-15T21:46:22.427137+0000","last_active":"2024-09-15T21:46:24.808436+0000","last_peered":"2024-09-15T21:46:24.808436+0000","last_clean":"2024-09-15T21:46:24.808436+0000","last_became_active":"2024-09-15T21:46:22.426855+0000","last_became_peered":"2024-09-15T21:46:22.426855+0000","last_unstale":"2024-09-15T21:46:24.808436+0000","last_undegraded":"2024-09-15T21:46:24.808436+0000","last_fullsized":"2024-09-15T21:46:24.808436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:40:14.277940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431767+0000","last_change":"2024-09-15T21:46:23.427327+0000","last_active":"2024-09-15T21:46:23.431767+0000","last_peered":"2024-09-15T21:46:23.431767+0000","last_clean":"2024-09-15T21:46:23.431767+0000","last_became_active":"2024-09-15T21:46:23.427096+0000","last_became_peered":"2024-09-15T21:46:23.427096+0000","last_unstale":"2024-09-15T21:46:23.431767+0000","last_undegraded":"2024-09-15T21:46:23.431767+0000","last_fullsized":"2024-09-15T21:46:23.431767+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:43.650053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433399+0000","last_change":"2024-09-15T21:46:22.427798+0000","last_active":"2024-09-15T21:46:22.433399+0000","last_peered":"2024-09-15T21:46:22.433399+0000","last_clean":"2024-09-15T21:46:22.433399+0000","last_became_active":"2024-09-15T21:46:22.427537+0000","last_became_peered":"2024-09-15T21:46:22.427537+0000","last_unstale":"2024-09-15T21:46:22.433399+0000","last_undegraded":"2024-09-15T21:46:22.433399+0000","last_fullsized":"2024-09-15T21:46:22.433399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:07:36.227177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.428576+0000","last_change":"2024-09-15T21:46:23.424266+0000","last_active":"2024-09-15T21:46:23.428576+0000","last_peered":"2024-09-15T21:46:23.428576+0000","last_clean":"2024-09-15T21:46:23.428576+0000","last_became_active":"2024-09-15T21:46:23.424007+0000","last_became_peered":"2024-09-15T21:46:23.424007+0000","last_unstale":"2024-09-15T21:46:23.428576+0000","last_undegraded":"2024-09-15T21:46:23.428576+0000","last_fullsized":"2024-09-15T21:46:23.428576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:04:11.974067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433546+0000","last_change":"2024-09-15T21:46:22.430901+0000","last_active":"2024-09-15T21:46:22.433546+0000","last_peered":"2024-09-15T21:46:22.433546+0000","last_clean":"2024-09-15T21:46:22.433546+0000","last_became_active":"2024-09-15T21:46:22.430452+0000","last_became_peered":"2024-09-15T21:46:22.430452+0000","last_unstale":"2024-09-15T21:46:22.433546+0000","last_undegraded":"2024-09-15T21:46:22.433546+0000","last_fullsized":"2024-09-15T21:46:22.433546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:02.951631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.414149+0000","last_change":"2024-09-15T21:46:22.414149+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.414149+0000","last_undegraded":"2024-09-15T21:46:22.414149+0000","last_fullsized":"2024-09-15T21:46:22.414149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808380+0000","last_change":"2024-09-15T21:46:22.423795+0000","last_active":"2024-09-15T21:46:24.808380+0000","last_peered":"2024-09-15T21:46:24.808380+0000","last_clean":"2024-09-15T21:46:24.808380+0000","last_became_active":"2024-09-15T21:46:22.422327+0000","last_became_peered":"2024-09-15T21:46:22.422327+0000","last_unstale":"2024-09-15T21:46:24.808380+0000","last_undegraded":"2024-09-15T21:46:24.808380+0000","last_fullsized":"2024-09-15T21:46:24.808380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:16.670599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433600+0000","last_change":"2024-09-15T21:46:23.428634+0000","last_active":"2024-09-15T21:46:23.433600+0000","last_peered":"2024-09-15T21:46:23.433600+0000","last_clean":"2024-09-15T21:46:23.433600+0000","last_became_active":"2024-09-15T21:46:23.428301+0000","last_became_peered":"2024-09-15T21:46:23.428301+0000","last_unstale":"2024-09-15T21:46:23.433600+0000","last_undegraded":"2024-09-15T21:46:23.433600+0000","last_fullsized":"2024-09-15T21:46:23.433600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:17.178940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808304+0000","last_change":"2024-09-15T21:46:22.425810+0000","last_active":"2024-09-15T21:46:24.808304+0000","last_peered":"2024-09-15T21:46:24.808304+0000","last_clean":"2024-09-15T21:46:24.808304+0000","last_became_active":"2024-09-15T21:46:22.425478+0000","last_became_peered":"2024-09-15T21:46:22.425478+0000","last_unstale":"2024-09-15T21:46:24.808304+0000","last_undegraded":"2024-09-15T21:46:24.808304+0000","last_fullsized":"2024-09-15T21:46:24.808304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:53.296599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.431785+0000","last_change":"2024-09-15T21:46:23.426054+0000","last_active":"2024-09-15T21:46:23.431785+0000","last_peered":"2024-09-15T21:46:23.431785+0000","last_clean":"2024-09-15T21:46:23.431785+0000","last_became_active":"2024-09-15T21:46:23.425547+0000","last_became_peered":"2024-09-15T21:46:23.425547+0000","last_unstale":"2024-09-15T21:46:23.431785+0000","last_undegraded":"2024-09-15T21:46:23.431785+0000","last_fullsized":"2024-09-15T21:46:23.431785+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:25.738831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433353+0000","last_change":"2024-09-15T21:46:22.428524+0000","last_active":"2024-09-15T21:46:22.433353+0000","last_peered":"2024-09-15T21:46:22.433353+0000","last_clean":"2024-09-15T21:46:22.433353+0000","last_became_active":"2024-09-15T21:46:22.428043+0000","last_became_peered":"2024-09-15T21:46:22.428043+0000","last_unstale":"2024-09-15T21:46:22.433353+0000","last_undegraded":"2024-09-15T21:46:22.433353+0000","last_fullsized":"2024-09-15T21:46:22.433353+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:53.400491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.419544+0000","last_change":"2024-09-15T21:46:22.419544+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.419544+0000","last_undegraded":"2024-09-15T21:46:22.419544+0000","last_fullsized":"2024-09-15T21:46:22.419544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.415492+0000","last_change":"2024-09-15T21:46:22.415492+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.415492+0000","last_undegraded":"2024-09-15T21:46:22.415492+0000","last_fullsized":"2024-09-15T21:46:22.415492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433558+0000","last_change":"2024-09-15T21:46:22.431172+0000","last_active":"2024-09-15T21:46:22.433558+0000","last_peered":"2024-09-15T21:46:22.433558+0000","last_clean":"2024-09-15T21:46:22.433558+0000","last_became_active":"2024-09-15T21:46:22.430800+0000","last_became_peered":"2024-09-15T21:46:22.430800+0000","last_unstale":"2024-09-15T21:46:22.433558+0000","last_undegraded":"2024-09-15T21:46:22.433558+0000","last_fullsized":"2024-09-15T21:46:22.433558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:52:07.231504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-15T21:46:22.417557+0000","last_change":"2024-09-15T21:46:22.417557+0000","last_active":"2024-09-15T21:46:22.402110+0000","last_peered":"2024-09-15T21:46:22.402110+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:46:22.417557+0000","last_undegraded":"2024-09-15T21:46:22.417557+0000","last_fullsized":"2024-09-15T21:46:22.417557+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:46:22.433476+0000","last_change":"2024-09-15T21:46:22.429870+0000","last_active":"2024-09-15T21:46:22.433476+0000","last_peered":"2024-09-15T21:46:22.433476+0000","last_clean":"2024-09-15T21:46:22.433476+0000","last_became_active":"2024-09-15T21:46:22.429578+0000","last_became_peered":"2024-09-15T21:46:22.429578+0000","last_unstale":"2024-09-15T21:46:22.433476+0000","last_undegraded":"2024-09-15T21:46:22.433476+0000","last_fullsized":"2024-09-15T21:46:22.433476+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:59.880081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433864+0000","last_change":"2024-09-15T21:46:23.429604+0000","last_active":"2024-09-15T21:46:23.433864+0000","last_peered":"2024-09-15T21:46:23.433864+0000","last_clean":"2024-09-15T21:46:23.433864+0000","last_became_active":"2024-09-15T21:46:23.429355+0000","last_became_peered":"2024-09-15T21:46:23.429355+0000","last_unstale":"2024-09-15T21:46:23.433864+0000","last_undegraded":"2024-09-15T21:46:23.433864+0000","last_fullsized":"2024-09-15T21:46:23.433864+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:23:10.533918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.424314+0000","last_change":"2024-09-15T21:46:22.428529+0000","last_active":"2024-09-15T21:46:23.424314+0000","last_peered":"2024-09-15T21:46:23.424314+0000","last_clean":"2024-09-15T21:46:23.424314+0000","last_became_active":"2024-09-15T21:46:22.428263+0000","last_became_peered":"2024-09-15T21:46:22.428263+0000","last_unstale":"2024-09-15T21:46:23.424314+0000","last_undegraded":"2024-09-15T21:46:23.424314+0000","last_fullsized":"2024-09-15T21:46:23.424314+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:22:40.160062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.433487+0000","last_change":"2024-09-15T21:46:23.427968+0000","last_active":"2024-09-15T21:46:23.433487+0000","last_peered":"2024-09-15T21:46:23.433487+0000","last_clean":"2024-09-15T21:46:23.433487+0000","last_became_active":"2024-09-15T21:46:23.427546+0000","last_became_peered":"2024-09-15T21:46:23.427546+0000","last_unstale":"2024-09-15T21:46:23.433487+0000","last_undegraded":"2024-09-15T21:46:23.433487+0000","last_fullsized":"2024-09-15T21:46:23.433487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:45.544297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:46:23.417176+0000","last_change":"2024-09-15T21:46:22.428258+0000","last_active":"2024-09-15T21:46:23.417176+0000","last_peered":"2024-09-15T21:46:23.417176+0000","last_clean":"2024-09-15T21:46:23.417176+0000","last_became_active":"2024-09-15T21:46:22.427954+0000","last_became_peered":"2024-09-15T21:46:22.427954+0000","last_unstale":"2024-09-15T21:46:23.417176+0000","last_undegraded":"2024-09-15T21:46:23.417176+0000","last_fullsized":"2024-09-15T21:46:23.417176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:22.440456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476740,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":654251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":790528,"data_stored":654251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":57608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":57608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":57608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":184320,"data_stored":57608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":76958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":208896,"data_stored":70658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:46:32.738 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:46:32.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.847+0000 7f0b264ed640 1 Processor -- start 2024-09-15T21:46:32.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.848+0000 7f0b264ed640 1 -- start start 2024-09-15T21:46:32.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.848+0000 7f0b264ed640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f0b2006d700 0x7f0b200730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:32.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.848+0000 7f0b264ed640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0b200735f0 0x7f0b2015c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:32.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.848+0000 7f0b264ed640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b2006db30 con 0x7f0b201552e0 2024-09-15T21:46:32.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.848+0000 7f0b264ed640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f0b2006ec60 con 0x7f0b200735f0 2024-09-15T21:46:32.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.848+0000 7f0b264ed640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f0b2015af40 con 0x7f0b2006d700 2024-09-15T21:46:32.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.849+0000 7f0b1ffff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f0b201552e0 0x7f0b200cee00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:51806/0 (socket says 172.21.15.100:51806) 2024-09-15T21:46:32.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.849+0000 7f0b1ffff640 1 -- 172.21.15.100:0/4099097575 learned_addr learned my addr 172.21.15.100:0/4099097575 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:32.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.849+0000 7f0b24a8a640 1 --2- 172.21.15.100:0/4099097575 >> v2:172.21.15.100:3300/0 conn(0x7f0b200735f0 0x7f0b2015c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:32.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.849+0000 7f0b1f7fe640 1 --2- 172.21.15.100:0/4099097575 >> v2:172.21.15.122:3300/0 conn(0x7f0b2006d700 0x7f0b200730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:32.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.849+0000 7f0b24a8a640 1 -- 172.21.15.100:0/4099097575 >> v2:172.21.15.122:3300/0 conn(0x7f0b2006d700 msgr2=0x7f0b200730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:32.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.849+0000 7f0b24a8a640 1 --2- 172.21.15.100:0/4099097575 >> v2:172.21.15.122:3300/0 conn(0x7f0b2006d700 0x7f0b200730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:32.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.849+0000 7f0b24a8a640 1 -- 172.21.15.100:0/4099097575 >> v1:172.21.15.100:6789/0 conn(0x7f0b201552e0 legacy=0x7f0b200cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:32.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.849+0000 7f0b24a8a640 1 -- 172.21.15.100:0/4099097575 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2015b200 con 0x7f0b200735f0 2024-09-15T21:46:32.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.849+0000 7f0b1f7fe640 1 --2- 172.21.15.100:0/4099097575 >> v2:172.21.15.122:3300/0 conn(0x7f0b2006d700 0x7f0b200730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:32.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.850+0000 7f0b24a8a640 1 --2- 172.21.15.100:0/4099097575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b200735f0 0x7f0b2015c5e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0b2006e730 tx=0x7f0b1802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c1d7ea1b87dedb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:32.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.850+0000 7f0b1effd640 1 -- 172.21.15.100:0/4099097575 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18007740 con 0x7f0b200735f0 2024-09-15T21:46:32.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.850+0000 7f0b1effd640 1 -- 172.21.15.100:0/4099097575 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b180078e0 con 0x7f0b200735f0 2024-09-15T21:46:32.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.850+0000 7f0b1effd640 1 -- 172.21.15.100:0/4099097575 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18007cb0 con 0x7f0b200735f0 2024-09-15T21:46:32.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.850+0000 7f0b264ed640 1 -- 172.21.15.100:0/4099097575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b200735f0 msgr2=0x7f0b2015c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:32.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.850+0000 7f0b264ed640 1 --2- 172.21.15.100:0/4099097575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b200735f0 0x7f0b2015c5e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0b2006e730 tx=0x7f0b1802a680 comp rx=0 tx=0).stop 2024-09-15T21:46:32.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.850+0000 7f0b264ed640 1 -- 172.21.15.100:0/4099097575 shutdown_connections 2024-09-15T21:46:32.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.850+0000 7f0b264ed640 1 --2- 172.21.15.100:0/4099097575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b200735f0 0x7f0b2015c5e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:32.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.850+0000 7f0b264ed640 1 --2- 172.21.15.100:0/4099097575 >> v2:172.21.15.122:3300/0 conn(0x7f0b2006d700 0x7f0b200730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:32.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.850+0000 7f0b264ed640 1 -- 172.21.15.100:0/4099097575 >> 172.21.15.100:0/4099097575 conn(0x7f0b20070ca0 msgr2=0x7f0b20071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:32.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.850+0000 7f0b264ed640 1 -- 172.21.15.100:0/4099097575 shutdown_connections 2024-09-15T21:46:32.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.850+0000 7f0b264ed640 1 -- 172.21.15.100:0/4099097575 wait complete. 2024-09-15T21:46:32.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.852+0000 7f0b264ed640 1 Processor -- start 2024-09-15T21:46:32.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.852+0000 7f0b264ed640 1 -- start start 2024-09-15T21:46:32.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b264ed640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2006d700 0x7f0b200cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:32.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b264ed640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0b200735f0 0x7f0b200c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:32.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b264ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b201552e0 0x7f0b200c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:32.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b264ed640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2015baa0 con 0x7f0b2006d700 2024-09-15T21:46:32.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b264ed640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0b2015b7c0 con 0x7f0b201552e0 2024-09-15T21:46:32.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b264ed640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0b200c91a0 con 0x7f0b200735f0 2024-09-15T21:46:32.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b1ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2006d700 0x7f0b200cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:32.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b1ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2006d700 0x7f0b200cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43750/0 (socket says 172.21.15.100:43750) 2024-09-15T21:46:32.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b1ffff640 1 -- 172.21.15.100:0/1813191517 learned_addr learned my addr 172.21.15.100:0/1813191517 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:32.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b1f7fe640 1 --2- 172.21.15.100:0/1813191517 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0b200735f0 0x7f0b200c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:32.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b24a8a640 1 --2- 172.21.15.100:0/1813191517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b201552e0 0x7f0b200c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:32.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b1ffff640 1 -- 172.21.15.100:0/1813191517 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0b200735f0 msgr2=0x7f0b200c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:32.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b1ffff640 1 --2- 172.21.15.100:0/1813191517 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0b200735f0 0x7f0b200c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:32.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b1ffff640 1 -- 172.21.15.100:0/1813191517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b201552e0 msgr2=0x7f0b200c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:32.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b1ffff640 1 --2- 172.21.15.100:0/1813191517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b201552e0 0x7f0b200c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:32.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b1ffff640 1 -- 172.21.15.100:0/1813191517 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b201f2970 con 0x7f0b2006d700 2024-09-15T21:46:32.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b1ffff640 1 --2- 172.21.15.100:0/1813191517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2006d700 0x7f0b200cb280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c430 tx=0x7f0b1000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:32.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b1cff9640 1 -- 172.21.15.100:0/1813191517 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b10017070 con 0x7f0b2006d700 2024-09-15T21:46:32.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b1cff9640 1 -- 172.21.15.100:0/1813191517 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b10005ce0 con 0x7f0b2006d700 2024-09-15T21:46:32.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b264ed640 1 -- 172.21.15.100:0/1813191517 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b200c7990 con 0x7f0b2006d700 2024-09-15T21:46:32.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b1cff9640 1 -- 172.21.15.100:0/1813191517 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b100072f0 con 0x7f0b2006d700 2024-09-15T21:46:32.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.853+0000 7f0b264ed640 1 -- 172.21.15.100:0/1813191517 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b201f2c10 con 0x7f0b2006d700 2024-09-15T21:46:32.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.854+0000 7f0b264ed640 1 -- 172.21.15.100:0/1813191517 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b2006dd00 con 0x7f0b2006d700 2024-09-15T21:46:32.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.855+0000 7f0b1cff9640 1 -- 172.21.15.100:0/1813191517 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0b1001a070 con 0x7f0b2006d700 2024-09-15T21:46:32.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.855+0000 7f0b1cff9640 1 --2- 172.21.15.100:0/1813191517 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0b0c06c2b0 0x7f0b0c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:32.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.855+0000 7f0b1cff9640 1 -- 172.21.15.100:0/1813191517 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0b1008a780 con 0x7f0b2006d700 2024-09-15T21:46:32.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.855+0000 7f0b1f7fe640 1 --2- 172.21.15.100:0/1813191517 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0b0c06c2b0 0x7f0b0c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:32.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.856+0000 7f0b1f7fe640 1 --2- 172.21.15.100:0/1813191517 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0b0c06c2b0 0x7f0b0c06e770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0b201522e0 tx=0x7f0b14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:32.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:32.859+0000 7f0b1cff9640 1 -- 172.21.15.100:0/1813191517 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0b2006dd00 con 0x7f0b2006d700 2024-09-15T21:46:33.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.022+0000 7f0b264ed640 1 -- 172.21.15.100:0/1813191517 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0b200730b0 con 0x7f0b0c06c2b0 2024-09-15T21:46:33.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.028+0000 7f0b1cff9640 1 -- 172.21.15.100:0/1813191517 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251883 (secure 0 0 0) 0x7f0b200730b0 con 0x7f0b0c06c2b0 2024-09-15T21:46:33.032 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:33.034 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-15T21:46:33.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.031+0000 7f0b264ed640 1 -- 172.21.15.100:0/1813191517 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0b0c06c2b0 msgr2=0x7f0b0c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:33.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.031+0000 7f0b264ed640 1 --2- 172.21.15.100:0/1813191517 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0b0c06c2b0 0x7f0b0c06e770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0b201522e0 tx=0x7f0b14008040 comp rx=0 tx=0).stop 2024-09-15T21:46:33.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.031+0000 7f0b264ed640 1 -- 172.21.15.100:0/1813191517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2006d700 msgr2=0x7f0b200cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:33.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.031+0000 7f0b264ed640 1 --2- 172.21.15.100:0/1813191517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2006d700 0x7f0b200cb280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0b1000c430 tx=0x7f0b1000c8f0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.031+0000 7f0b264ed640 1 -- 172.21.15.100:0/1813191517 shutdown_connections 2024-09-15T21:46:33.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.031+0000 7f0b264ed640 1 --2- 172.21.15.100:0/1813191517 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0b0c06c2b0 0x7f0b0c06e770 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.031+0000 7f0b264ed640 1 --2- 172.21.15.100:0/1813191517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b201552e0 0x7f0b200c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.031+0000 7f0b264ed640 1 --2- 172.21.15.100:0/1813191517 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0b200735f0 0x7f0b200c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.031+0000 7f0b264ed640 1 --2- 172.21.15.100:0/1813191517 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b2006d700 0x7f0b200cb280 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.031+0000 7f0b264ed640 1 -- 172.21.15.100:0/1813191517 >> 172.21.15.100:0/1813191517 conn(0x7f0b20070ca0 msgr2=0x7f0b20071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:33.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.031+0000 7f0b264ed640 1 -- 172.21.15.100:0/1813191517 shutdown_connections 2024-09-15T21:46:33.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.031+0000 7f0b264ed640 1 -- 172.21.15.100:0/1813191517 wait complete. 2024-09-15T21:46:33.043 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-15T21:46:31.805266+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590984,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":166,"num_write_kb":2424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125,"ondisk_log_size":125,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":180,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219148,"kb_used_data":2900,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755572,"statfs":{"total":773094113280,"available":772869705728,"internally_reserved":0,"allocated":2969600,"data_stored":1798012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.390787"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835668+0000","last_change":"2024-09-15T21:46:23.428662+0000","last_active":"2024-09-15T21:46:24.835668+0000","last_peered":"2024-09-15T21:46:24.835668+0000","last_clean":"2024-09-15T21:46:24.835668+0000","last_became_active":"2024-09-15T21:46:23.427735+0000","last_became_peered":"2024-09-15T21:46:23.427735+0000","last_unstale":"2024-09-15T21:46:24.835668+0000","last_undegraded":"2024-09-15T21:46:24.835668+0000","last_fullsized":"2024-09-15T21:46:24.835668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:29.702757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808513+0000","last_change":"2024-09-15T21:46:23.418716+0000","last_active":"2024-09-15T21:46:24.808513+0000","last_peered":"2024-09-15T21:46:24.808513+0000","last_clean":"2024-09-15T21:46:24.808513+0000","last_became_active":"2024-09-15T21:46:23.418471+0000","last_became_peered":"2024-09-15T21:46:23.418471+0000","last_unstale":"2024-09-15T21:46:24.808513+0000","last_undegraded":"2024-09-15T21:46:24.808513+0000","last_fullsized":"2024-09-15T21:46:24.808513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:49.599801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727016+0000","last_change":"2024-09-15T21:46:23.426404+0000","last_active":"2024-09-15T21:46:24.727016+0000","last_peered":"2024-09-15T21:46:24.727016+0000","last_clean":"2024-09-15T21:46:24.727016+0000","last_became_active":"2024-09-15T21:46:23.426146+0000","last_became_peered":"2024-09-15T21:46:23.426146+0000","last_unstale":"2024-09-15T21:46:24.727016+0000","last_undegraded":"2024-09-15T21:46:24.727016+0000","last_fullsized":"2024-09-15T21:46:24.727016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:23.626109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414109+0000","last_change":"2024-09-15T21:46:23.432515+0000","last_active":"2024-09-15T21:46:24.414109+0000","last_peered":"2024-09-15T21:46:24.414109+0000","last_clean":"2024-09-15T21:46:24.414109+0000","last_became_active":"2024-09-15T21:46:23.432283+0000","last_became_peered":"2024-09-15T21:46:23.432283+0000","last_unstale":"2024-09-15T21:46:24.414109+0000","last_undegraded":"2024-09-15T21:46:24.414109+0000","last_fullsized":"2024-09-15T21:46:24.414109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:53.288080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835464+0000","last_change":"2024-09-15T21:46:23.427120+0000","last_active":"2024-09-15T21:46:24.835464+0000","last_peered":"2024-09-15T21:46:24.835464+0000","last_clean":"2024-09-15T21:46:24.835464+0000","last_became_active":"2024-09-15T21:46:23.426706+0000","last_became_peered":"2024-09-15T21:46:23.426706+0000","last_unstale":"2024-09-15T21:46:24.835464+0000","last_undegraded":"2024-09-15T21:46:24.835464+0000","last_fullsized":"2024-09-15T21:46:24.835464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:29.513446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414665+0000","last_change":"2024-09-15T21:46:23.434690+0000","last_active":"2024-09-15T21:46:24.414665+0000","last_peered":"2024-09-15T21:46:24.414665+0000","last_clean":"2024-09-15T21:46:24.414665+0000","last_became_active":"2024-09-15T21:46:23.434146+0000","last_became_peered":"2024-09-15T21:46:23.434146+0000","last_unstale":"2024-09-15T21:46:24.414665+0000","last_undegraded":"2024-09-15T21:46:24.414665+0000","last_fullsized":"2024-09-15T21:46:24.414665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:26:59.905178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835273+0000","last_change":"2024-09-15T21:46:23.429541+0000","last_active":"2024-09-15T21:46:24.835273+0000","last_peered":"2024-09-15T21:46:24.835273+0000","last_clean":"2024-09-15T21:46:24.835273+0000","last_became_active":"2024-09-15T21:46:23.429263+0000","last_became_peered":"2024-09-15T21:46:23.429263+0000","last_unstale":"2024-09-15T21:46:24.835273+0000","last_undegraded":"2024-09-15T21:46:24.835273+0000","last_fullsized":"2024-09-15T21:46:24.835273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:34:51.530981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834382+0000","last_change":"2024-09-15T21:46:23.437143+0000","last_active":"2024-09-15T21:46:24.834382+0000","last_peered":"2024-09-15T21:46:24.834382+0000","last_clean":"2024-09-15T21:46:24.834382+0000","last_became_active":"2024-09-15T21:46:23.436864+0000","last_became_peered":"2024-09-15T21:46:23.436864+0000","last_unstale":"2024-09-15T21:46:24.834382+0000","last_undegraded":"2024-09-15T21:46:24.834382+0000","last_fullsized":"2024-09-15T21:46:24.834382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:45.870403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835817+0000","last_change":"2024-09-15T21:46:23.438642+0000","last_active":"2024-09-15T21:46:24.835817+0000","last_peered":"2024-09-15T21:46:24.835817+0000","last_clean":"2024-09-15T21:46:24.835817+0000","last_became_active":"2024-09-15T21:46:23.437521+0000","last_became_peered":"2024-09-15T21:46:23.437521+0000","last_unstale":"2024-09-15T21:46:24.835817+0000","last_undegraded":"2024-09-15T21:46:24.835817+0000","last_fullsized":"2024-09-15T21:46:24.835817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:04.648853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835520+0000","last_change":"2024-09-15T21:46:23.436106+0000","last_active":"2024-09-15T21:46:24.835520+0000","last_peered":"2024-09-15T21:46:24.835520+0000","last_clean":"2024-09-15T21:46:24.835520+0000","last_became_active":"2024-09-15T21:46:23.435821+0000","last_became_peered":"2024-09-15T21:46:23.435821+0000","last_unstale":"2024-09-15T21:46:24.835520+0000","last_undegraded":"2024-09-15T21:46:24.835520+0000","last_fullsized":"2024-09-15T21:46:24.835520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835070+0000","last_change":"2024-09-15T21:46:23.435442+0000","last_active":"2024-09-15T21:46:24.835070+0000","last_peered":"2024-09-15T21:46:24.835070+0000","last_clean":"2024-09-15T21:46:24.835070+0000","last_became_active":"2024-09-15T21:46:23.435165+0000","last_became_peered":"2024-09-15T21:46:23.435165+0000","last_unstale":"2024-09-15T21:46:24.835070+0000","last_undegraded":"2024-09-15T21:46:24.835070+0000","last_fullsized":"2024-09-15T21:46:24.835070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:02.361682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414099+0000","last_change":"2024-09-15T21:46:23.432534+0000","last_active":"2024-09-15T21:46:24.414099+0000","last_peered":"2024-09-15T21:46:24.414099+0000","last_clean":"2024-09-15T21:46:24.414099+0000","last_became_active":"2024-09-15T21:46:23.430170+0000","last_became_peered":"2024-09-15T21:46:23.430170+0000","last_unstale":"2024-09-15T21:46:24.414099+0000","last_undegraded":"2024-09-15T21:46:24.414099+0000","last_fullsized":"2024-09-15T21:46:24.414099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:11.463937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834691+0000","last_change":"2024-09-15T21:46:23.427841+0000","last_active":"2024-09-15T21:46:24.834691+0000","last_peered":"2024-09-15T21:46:24.834691+0000","last_clean":"2024-09-15T21:46:24.834691+0000","last_became_active":"2024-09-15T21:46:23.427621+0000","last_became_peered":"2024-09-15T21:46:23.427621+0000","last_unstale":"2024-09-15T21:46:24.834691+0000","last_undegraded":"2024-09-15T21:46:24.834691+0000","last_fullsized":"2024-09-15T21:46:24.834691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:18.331943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.726852+0000","last_change":"2024-09-15T21:46:23.424707+0000","last_active":"2024-09-15T21:46:24.726852+0000","last_peered":"2024-09-15T21:46:24.726852+0000","last_clean":"2024-09-15T21:46:24.726852+0000","last_became_active":"2024-09-15T21:46:23.424447+0000","last_became_peered":"2024-09-15T21:46:23.424447+0000","last_unstale":"2024-09-15T21:46:24.726852+0000","last_undegraded":"2024-09-15T21:46:24.726852+0000","last_fullsized":"2024-09-15T21:46:24.726852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:23.617214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808321+0000","last_change":"2024-09-15T21:46:23.430297+0000","last_active":"2024-09-15T21:46:24.808321+0000","last_peered":"2024-09-15T21:46:24.808321+0000","last_clean":"2024-09-15T21:46:24.808321+0000","last_became_active":"2024-09-15T21:46:23.430070+0000","last_became_peered":"2024-09-15T21:46:23.430070+0000","last_unstale":"2024-09-15T21:46:24.808321+0000","last_undegraded":"2024-09-15T21:46:24.808321+0000","last_fullsized":"2024-09-15T21:46:24.808321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:03:19.615022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414542+0000","last_change":"2024-09-15T21:46:23.430540+0000","last_active":"2024-09-15T21:46:24.414542+0000","last_peered":"2024-09-15T21:46:24.414542+0000","last_clean":"2024-09-15T21:46:24.414542+0000","last_became_active":"2024-09-15T21:46:23.430256+0000","last_became_peered":"2024-09-15T21:46:23.430256+0000","last_unstale":"2024-09-15T21:46:24.414542+0000","last_undegraded":"2024-09-15T21:46:24.414542+0000","last_fullsized":"2024-09-15T21:46:24.414542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:36.539762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.415216+0000","last_change":"2024-09-15T21:46:23.427802+0000","last_active":"2024-09-15T21:46:24.415216+0000","last_peered":"2024-09-15T21:46:24.415216+0000","last_clean":"2024-09-15T21:46:24.415216+0000","last_became_active":"2024-09-15T21:46:23.427502+0000","last_became_peered":"2024-09-15T21:46:23.427502+0000","last_unstale":"2024-09-15T21:46:24.415216+0000","last_undegraded":"2024-09-15T21:46:24.415216+0000","last_fullsized":"2024-09-15T21:46:24.415216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:31.062624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412600+0000","last_change":"2024-09-15T21:46:23.427570+0000","last_active":"2024-09-15T21:46:24.412600+0000","last_peered":"2024-09-15T21:46:24.412600+0000","last_clean":"2024-09-15T21:46:24.412600+0000","last_became_active":"2024-09-15T21:46:23.427304+0000","last_became_peered":"2024-09-15T21:46:23.427304+0000","last_unstale":"2024-09-15T21:46:24.412600+0000","last_undegraded":"2024-09-15T21:46:24.412600+0000","last_fullsized":"2024-09-15T21:46:24.412600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:37.345106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835899+0000","last_change":"2024-09-15T21:46:23.431434+0000","last_active":"2024-09-15T21:46:24.835899+0000","last_peered":"2024-09-15T21:46:24.835899+0000","last_clean":"2024-09-15T21:46:24.835899+0000","last_became_active":"2024-09-15T21:46:23.430987+0000","last_became_peered":"2024-09-15T21:46:23.430987+0000","last_unstale":"2024-09-15T21:46:24.835899+0000","last_undegraded":"2024-09-15T21:46:24.835899+0000","last_fullsized":"2024-09-15T21:46:24.835899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:48:30.532378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835198+0000","last_change":"2024-09-15T21:46:23.434558+0000","last_active":"2024-09-15T21:46:24.835198+0000","last_peered":"2024-09-15T21:46:24.835198+0000","last_clean":"2024-09-15T21:46:24.835198+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:46:24.835198+0000","last_undegraded":"2024-09-15T21:46:24.835198+0000","last_fullsized":"2024-09-15T21:46:24.835198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:03:39.562977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727140+0000","last_change":"2024-09-15T21:46:23.428323+0000","last_active":"2024-09-15T21:46:24.727140+0000","last_peered":"2024-09-15T21:46:24.727140+0000","last_clean":"2024-09-15T21:46:24.727140+0000","last_became_active":"2024-09-15T21:46:23.428016+0000","last_became_peered":"2024-09-15T21:46:23.428016+0000","last_unstale":"2024-09-15T21:46:24.727140+0000","last_undegraded":"2024-09-15T21:46:24.727140+0000","last_fullsized":"2024-09-15T21:46:24.727140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:28:48.678550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414612+0000","last_change":"2024-09-15T21:46:23.434465+0000","last_active":"2024-09-15T21:46:24.414612+0000","last_peered":"2024-09-15T21:46:24.414612+0000","last_clean":"2024-09-15T21:46:24.414612+0000","last_became_active":"2024-09-15T21:46:23.433849+0000","last_became_peered":"2024-09-15T21:46:23.433849+0000","last_unstale":"2024-09-15T21:46:24.414612+0000","last_undegraded":"2024-09-15T21:46:24.414612+0000","last_fullsized":"2024-09-15T21:46:24.414612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:46.093198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834522+0000","last_change":"2024-09-15T21:46:23.431527+0000","last_active":"2024-09-15T21:46:24.834522+0000","last_peered":"2024-09-15T21:46:24.834522+0000","last_clean":"2024-09-15T21:46:24.834522+0000","last_became_active":"2024-09-15T21:46:23.430719+0000","last_became_peered":"2024-09-15T21:46:23.430719+0000","last_unstale":"2024-09-15T21:46:24.834522+0000","last_undegraded":"2024-09-15T21:46:24.834522+0000","last_fullsized":"2024-09-15T21:46:24.834522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:16.326529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414532+0000","last_change":"2024-09-15T21:46:23.432282+0000","last_active":"2024-09-15T21:46:24.414532+0000","last_peered":"2024-09-15T21:46:24.414532+0000","last_clean":"2024-09-15T21:46:24.414532+0000","last_became_active":"2024-09-15T21:46:23.432035+0000","last_became_peered":"2024-09-15T21:46:23.432035+0000","last_unstale":"2024-09-15T21:46:24.414532+0000","last_undegraded":"2024-09-15T21:46:24.414532+0000","last_fullsized":"2024-09-15T21:46:24.414532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:22.163940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834776+0000","last_change":"2024-09-15T21:46:23.432646+0000","last_active":"2024-09-15T21:46:24.834776+0000","last_peered":"2024-09-15T21:46:24.834776+0000","last_clean":"2024-09-15T21:46:24.834776+0000","last_became_active":"2024-09-15T21:46:23.432317+0000","last_became_peered":"2024-09-15T21:46:23.432317+0000","last_unstale":"2024-09-15T21:46:24.834776+0000","last_undegraded":"2024-09-15T21:46:24.834776+0000","last_fullsized":"2024-09-15T21:46:24.834776+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:49:07.706450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835541+0000","last_change":"2024-09-15T21:46:23.428183+0000","last_active":"2024-09-15T21:46:24.835541+0000","last_peered":"2024-09-15T21:46:24.835541+0000","last_clean":"2024-09-15T21:46:24.835541+0000","last_became_active":"2024-09-15T21:46:23.427886+0000","last_became_peered":"2024-09-15T21:46:23.427886+0000","last_unstale":"2024-09-15T21:46:24.835541+0000","last_undegraded":"2024-09-15T21:46:24.835541+0000","last_fullsized":"2024-09-15T21:46:24.835541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:15.584377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808369+0000","last_change":"2024-09-15T21:46:23.421749+0000","last_active":"2024-09-15T21:46:24.808369+0000","last_peered":"2024-09-15T21:46:24.808369+0000","last_clean":"2024-09-15T21:46:24.808369+0000","last_became_active":"2024-09-15T21:46:23.421501+0000","last_became_peered":"2024-09-15T21:46:23.421501+0000","last_unstale":"2024-09-15T21:46:24.808369+0000","last_undegraded":"2024-09-15T21:46:24.808369+0000","last_fullsized":"2024-09-15T21:46:24.808369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:55.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834843+0000","last_change":"2024-09-15T21:46:23.432591+0000","last_active":"2024-09-15T21:46:24.834843+0000","last_peered":"2024-09-15T21:46:24.834843+0000","last_clean":"2024-09-15T21:46:24.834843+0000","last_became_active":"2024-09-15T21:46:23.432326+0000","last_became_peered":"2024-09-15T21:46:23.432326+0000","last_unstale":"2024-09-15T21:46:24.834843+0000","last_undegraded":"2024-09-15T21:46:24.834843+0000","last_fullsized":"2024-09-15T21:46:24.834843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:48.565579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727016+0000","last_change":"2024-09-15T21:46:23.423428+0000","last_active":"2024-09-15T21:46:24.727016+0000","last_peered":"2024-09-15T21:46:24.727016+0000","last_clean":"2024-09-15T21:46:24.727016+0000","last_became_active":"2024-09-15T21:46:23.422734+0000","last_became_peered":"2024-09-15T21:46:23.422734+0000","last_unstale":"2024-09-15T21:46:24.727016+0000","last_undegraded":"2024-09-15T21:46:24.727016+0000","last_fullsized":"2024-09-15T21:46:24.727016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:06.592891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414770+0000","last_change":"2024-09-15T21:46:23.435244+0000","last_active":"2024-09-15T21:46:24.414770+0000","last_peered":"2024-09-15T21:46:24.414770+0000","last_clean":"2024-09-15T21:46:24.414770+0000","last_became_active":"2024-09-15T21:46:23.434932+0000","last_became_peered":"2024-09-15T21:46:23.434932+0000","last_unstale":"2024-09-15T21:46:24.414770+0000","last_undegraded":"2024-09-15T21:46:24.414770+0000","last_fullsized":"2024-09-15T21:46:24.414770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:55.810146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412500+0000","last_change":"2024-09-15T21:46:23.437276+0000","last_active":"2024-09-15T21:46:24.412500+0000","last_peered":"2024-09-15T21:46:24.412500+0000","last_clean":"2024-09-15T21:46:24.412500+0000","last_became_active":"2024-09-15T21:46:23.436962+0000","last_became_peered":"2024-09-15T21:46:23.436962+0000","last_unstale":"2024-09-15T21:46:24.412500+0000","last_undegraded":"2024-09-15T21:46:24.412500+0000","last_fullsized":"2024-09-15T21:46:24.412500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:13.043999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834825+0000","last_change":"2024-09-15T21:46:23.429648+0000","last_active":"2024-09-15T21:46:24.834825+0000","last_peered":"2024-09-15T21:46:24.834825+0000","last_clean":"2024-09-15T21:46:24.834825+0000","last_became_active":"2024-09-15T21:46:23.429376+0000","last_became_peered":"2024-09-15T21:46:23.429376+0000","last_unstale":"2024-09-15T21:46:24.834825+0000","last_undegraded":"2024-09-15T21:46:24.834825+0000","last_fullsized":"2024-09-15T21:46:24.834825+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:31.278283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.413663+0000","last_change":"2024-09-15T21:46:23.429847+0000","last_active":"2024-09-15T21:46:24.413663+0000","last_peered":"2024-09-15T21:46:24.413663+0000","last_clean":"2024-09-15T21:46:24.413663+0000","last_became_active":"2024-09-15T21:46:23.429442+0000","last_became_peered":"2024-09-15T21:46:23.429442+0000","last_unstale":"2024-09-15T21:46:24.413663+0000","last_undegraded":"2024-09-15T21:46:24.413663+0000","last_fullsized":"2024-09-15T21:46:24.413663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:31.232512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834855+0000","last_change":"2024-09-15T21:46:22.429407+0000","last_active":"2024-09-15T21:46:24.834855+0000","last_peered":"2024-09-15T21:46:24.834855+0000","last_clean":"2024-09-15T21:46:24.834855+0000","last_became_active":"2024-09-15T21:46:22.429107+0000","last_became_peered":"2024-09-15T21:46:22.429107+0000","last_unstale":"2024-09-15T21:46:24.834855+0000","last_undegraded":"2024-09-15T21:46:24.834855+0000","last_fullsized":"2024-09-15T21:46:24.834855+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:14:54.680349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727169+0000","last_change":"2024-09-15T21:46:23.434198+0000","last_active":"2024-09-15T21:46:24.727169+0000","last_peered":"2024-09-15T21:46:24.727169+0000","last_clean":"2024-09-15T21:46:24.727169+0000","last_became_active":"2024-09-15T21:46:23.433944+0000","last_became_peered":"2024-09-15T21:46:23.433944+0000","last_unstale":"2024-09-15T21:46:24.727169+0000","last_undegraded":"2024-09-15T21:46:24.727169+0000","last_fullsized":"2024-09-15T21:46:24.727169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:25.412167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834840+0000","last_change":"2024-09-15T21:46:22.425571+0000","last_active":"2024-09-15T21:46:24.834840+0000","last_peered":"2024-09-15T21:46:24.834840+0000","last_clean":"2024-09-15T21:46:24.834840+0000","last_became_active":"2024-09-15T21:46:22.425315+0000","last_became_peered":"2024-09-15T21:46:22.425315+0000","last_unstale":"2024-09-15T21:46:24.834840+0000","last_undegraded":"2024-09-15T21:46:24.834840+0000","last_fullsized":"2024-09-15T21:46:24.834840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:31.983301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834481+0000","last_change":"2024-09-15T21:46:22.425492+0000","last_active":"2024-09-15T21:46:24.834481+0000","last_peered":"2024-09-15T21:46:24.834481+0000","last_clean":"2024-09-15T21:46:24.834481+0000","last_became_active":"2024-09-15T21:46:22.425226+0000","last_became_peered":"2024-09-15T21:46:22.425226+0000","last_unstale":"2024-09-15T21:46:24.834481+0000","last_undegraded":"2024-09-15T21:46:24.834481+0000","last_fullsized":"2024-09-15T21:46:24.834481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:24.322384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834460+0000","last_change":"2024-09-15T21:46:23.429140+0000","last_active":"2024-09-15T21:46:24.834460+0000","last_peered":"2024-09-15T21:46:24.834460+0000","last_clean":"2024-09-15T21:46:24.834460+0000","last_became_active":"2024-09-15T21:46:23.428902+0000","last_became_peered":"2024-09-15T21:46:23.428902+0000","last_unstale":"2024-09-15T21:46:24.834460+0000","last_undegraded":"2024-09-15T21:46:24.834460+0000","last_fullsized":"2024-09-15T21:46:24.834460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:11.256924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414663+0000","last_change":"2024-09-15T21:46:22.428319+0000","last_active":"2024-09-15T21:46:24.414663+0000","last_peered":"2024-09-15T21:46:24.414663+0000","last_clean":"2024-09-15T21:46:24.414663+0000","last_became_active":"2024-09-15T21:46:22.428061+0000","last_became_peered":"2024-09-15T21:46:22.428061+0000","last_unstale":"2024-09-15T21:46:24.414663+0000","last_undegraded":"2024-09-15T21:46:24.414663+0000","last_fullsized":"2024-09-15T21:46:24.414663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:14.238988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808315+0000","last_change":"2024-09-15T21:46:23.427442+0000","last_active":"2024-09-15T21:46:24.808315+0000","last_peered":"2024-09-15T21:46:24.808315+0000","last_clean":"2024-09-15T21:46:24.808315+0000","last_became_active":"2024-09-15T21:46:23.427200+0000","last_became_peered":"2024-09-15T21:46:23.427200+0000","last_unstale":"2024-09-15T21:46:24.808315+0000","last_undegraded":"2024-09-15T21:46:24.808315+0000","last_fullsized":"2024-09-15T21:46:24.808315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:27.248545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835085+0000","last_change":"2024-09-15T21:46:22.430599+0000","last_active":"2024-09-15T21:46:24.835085+0000","last_peered":"2024-09-15T21:46:24.835085+0000","last_clean":"2024-09-15T21:46:24.835085+0000","last_became_active":"2024-09-15T21:46:22.430327+0000","last_became_peered":"2024-09-15T21:46:22.430327+0000","last_unstale":"2024-09-15T21:46:24.835085+0000","last_undegraded":"2024-09-15T21:46:24.835085+0000","last_fullsized":"2024-09-15T21:46:24.835085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:51.904693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414860+0000","last_change":"2024-09-15T21:46:23.434006+0000","last_active":"2024-09-15T21:46:24.414860+0000","last_peered":"2024-09-15T21:46:24.414860+0000","last_clean":"2024-09-15T21:46:24.414860+0000","last_became_active":"2024-09-15T21:46:23.433634+0000","last_became_peered":"2024-09-15T21:46:23.433634+0000","last_unstale":"2024-09-15T21:46:24.414860+0000","last_undegraded":"2024-09-15T21:46:24.414860+0000","last_fullsized":"2024-09-15T21:46:24.414860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:22.288806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414976+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:24.414976+0000","last_peered":"2024-09-15T21:46:24.414976+0000","last_clean":"2024-09-15T21:46:24.414976+0000","last_became_active":"2024-09-15T21:46:22.425236+0000","last_became_peered":"2024-09-15T21:46:22.425236+0000","last_unstale":"2024-09-15T21:46:24.414976+0000","last_undegraded":"2024-09-15T21:46:24.414976+0000","last_fullsized":"2024-09-15T21:46:24.414976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:45:21.332492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727237+0000","last_change":"2024-09-15T21:46:23.429587+0000","last_active":"2024-09-15T21:46:24.727237+0000","last_peered":"2024-09-15T21:46:24.727237+0000","last_clean":"2024-09-15T21:46:24.727237+0000","last_became_active":"2024-09-15T21:46:23.429334+0000","last_became_peered":"2024-09-15T21:46:23.429334+0000","last_unstale":"2024-09-15T21:46:24.727237+0000","last_undegraded":"2024-09-15T21:46:24.727237+0000","last_fullsized":"2024-09-15T21:46:24.727237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:48:56.361550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808325+0000","last_change":"2024-09-15T21:46:22.421964+0000","last_active":"2024-09-15T21:46:24.808325+0000","last_peered":"2024-09-15T21:46:24.808325+0000","last_clean":"2024-09-15T21:46:24.808325+0000","last_became_active":"2024-09-15T21:46:22.421661+0000","last_became_peered":"2024-09-15T21:46:22.421661+0000","last_unstale":"2024-09-15T21:46:24.808325+0000","last_undegraded":"2024-09-15T21:46:24.808325+0000","last_fullsized":"2024-09-15T21:46:24.808325+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:07.395070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414253+0000","last_change":"2024-09-15T21:46:23.427416+0000","last_active":"2024-09-15T21:46:24.414253+0000","last_peered":"2024-09-15T21:46:24.414253+0000","last_clean":"2024-09-15T21:46:24.414253+0000","last_became_active":"2024-09-15T21:46:23.427057+0000","last_became_peered":"2024-09-15T21:46:23.427057+0000","last_unstale":"2024-09-15T21:46:24.414253+0000","last_undegraded":"2024-09-15T21:46:24.414253+0000","last_fullsized":"2024-09-15T21:46:24.414253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:42.671501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835128+0000","last_change":"2024-09-15T21:46:22.426151+0000","last_active":"2024-09-15T21:46:24.835128+0000","last_peered":"2024-09-15T21:46:24.835128+0000","last_clean":"2024-09-15T21:46:24.835128+0000","last_became_active":"2024-09-15T21:46:22.425866+0000","last_became_peered":"2024-09-15T21:46:22.425866+0000","last_unstale":"2024-09-15T21:46:24.835128+0000","last_undegraded":"2024-09-15T21:46:24.835128+0000","last_fullsized":"2024-09-15T21:46:24.835128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:10.684265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835569+0000","last_change":"2024-09-15T21:46:23.432292+0000","last_active":"2024-09-15T21:46:24.835569+0000","last_peered":"2024-09-15T21:46:24.835569+0000","last_clean":"2024-09-15T21:46:24.835569+0000","last_became_active":"2024-09-15T21:46:23.431877+0000","last_became_peered":"2024-09-15T21:46:23.431877+0000","last_unstale":"2024-09-15T21:46:24.835569+0000","last_undegraded":"2024-09-15T21:46:24.835569+0000","last_fullsized":"2024-09-15T21:46:24.835569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:16.857279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.417259+0000","last_change":"2024-09-15T21:46:22.427415+0000","last_active":"2024-09-15T21:46:28.417259+0000","last_peered":"2024-09-15T21:46:28.417259+0000","last_clean":"2024-09-15T21:46:28.417259+0000","last_became_active":"2024-09-15T21:46:22.427087+0000","last_became_peered":"2024-09-15T21:46:22.427087+0000","last_unstale":"2024-09-15T21:46:28.417259+0000","last_undegraded":"2024-09-15T21:46:28.417259+0000","last_fullsized":"2024-09-15T21:46:28.417259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:09.365202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.726943+0000","last_change":"2024-09-15T21:46:23.427238+0000","last_active":"2024-09-15T21:46:24.726943+0000","last_peered":"2024-09-15T21:46:24.726943+0000","last_clean":"2024-09-15T21:46:24.726943+0000","last_became_active":"2024-09-15T21:46:23.426993+0000","last_became_peered":"2024-09-15T21:46:23.426993+0000","last_unstale":"2024-09-15T21:46:24.726943+0000","last_undegraded":"2024-09-15T21:46:24.726943+0000","last_fullsized":"2024-09-15T21:46:24.726943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:52:24.175554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414781+0000","last_change":"2024-09-15T21:46:22.427164+0000","last_active":"2024-09-15T21:46:24.414781+0000","last_peered":"2024-09-15T21:46:24.414781+0000","last_clean":"2024-09-15T21:46:24.414781+0000","last_became_active":"2024-09-15T21:46:22.426917+0000","last_became_peered":"2024-09-15T21:46:22.426917+0000","last_unstale":"2024-09-15T21:46:24.414781+0000","last_undegraded":"2024-09-15T21:46:24.414781+0000","last_fullsized":"2024-09-15T21:46:24.414781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:46.471935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835504+0000","last_change":"2024-09-15T21:46:23.433924+0000","last_active":"2024-09-15T21:46:24.835504+0000","last_peered":"2024-09-15T21:46:24.835504+0000","last_clean":"2024-09-15T21:46:24.835504+0000","last_became_active":"2024-09-15T21:46:23.433606+0000","last_became_peered":"2024-09-15T21:46:23.433606+0000","last_unstale":"2024-09-15T21:46:24.835504+0000","last_undegraded":"2024-09-15T21:46:24.835504+0000","last_fullsized":"2024-09-15T21:46:24.835504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:09:35.744149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834887+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:24.834887+0000","last_peered":"2024-09-15T21:46:24.834887+0000","last_clean":"2024-09-15T21:46:24.834887+0000","last_became_active":"2024-09-15T21:46:22.425237+0000","last_became_peered":"2024-09-15T21:46:22.425237+0000","last_unstale":"2024-09-15T21:46:24.834887+0000","last_undegraded":"2024-09-15T21:46:24.834887+0000","last_fullsized":"2024-09-15T21:46:24.834887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:59.460420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834853+0000","last_change":"2024-09-15T21:46:23.427515+0000","last_active":"2024-09-15T21:46:24.834853+0000","last_peered":"2024-09-15T21:46:24.834853+0000","last_clean":"2024-09-15T21:46:24.834853+0000","last_became_active":"2024-09-15T21:46:23.427116+0000","last_became_peered":"2024-09-15T21:46:23.427116+0000","last_unstale":"2024-09-15T21:46:24.834853+0000","last_undegraded":"2024-09-15T21:46:24.834853+0000","last_fullsized":"2024-09-15T21:46:24.834853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:35.499970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412425+0000","last_change":"2024-09-15T21:46:22.419905+0000","last_active":"2024-09-15T21:46:24.412425+0000","last_peered":"2024-09-15T21:46:24.412425+0000","last_clean":"2024-09-15T21:46:24.412425+0000","last_became_active":"2024-09-15T21:46:22.419620+0000","last_became_peered":"2024-09-15T21:46:22.419620+0000","last_unstale":"2024-09-15T21:46:24.412425+0000","last_undegraded":"2024-09-15T21:46:24.412425+0000","last_fullsized":"2024-09-15T21:46:24.412425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:26:51.215156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835459+0000","last_change":"2024-09-15T21:46:23.432975+0000","last_active":"2024-09-15T21:46:24.835459+0000","last_peered":"2024-09-15T21:46:24.835459+0000","last_clean":"2024-09-15T21:46:24.835459+0000","last_became_active":"2024-09-15T21:46:23.432718+0000","last_became_peered":"2024-09-15T21:46:23.432718+0000","last_unstale":"2024-09-15T21:46:24.835459+0000","last_undegraded":"2024-09-15T21:46:24.835459+0000","last_fullsized":"2024-09-15T21:46:24.835459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:59.789691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414318+0000","last_change":"2024-09-15T21:46:22.429661+0000","last_active":"2024-09-15T21:46:24.414318+0000","last_peered":"2024-09-15T21:46:24.414318+0000","last_clean":"2024-09-15T21:46:24.414318+0000","last_became_active":"2024-09-15T21:46:22.429250+0000","last_became_peered":"2024-09-15T21:46:22.429250+0000","last_unstale":"2024-09-15T21:46:24.414318+0000","last_undegraded":"2024-09-15T21:46:24.414318+0000","last_fullsized":"2024-09-15T21:46:24.414318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:29.341739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835590+0000","last_change":"2024-09-15T21:46:23.435003+0000","last_active":"2024-09-15T21:46:24.835590+0000","last_peered":"2024-09-15T21:46:24.835590+0000","last_clean":"2024-09-15T21:46:24.835590+0000","last_became_active":"2024-09-15T21:46:23.434738+0000","last_became_peered":"2024-09-15T21:46:23.434738+0000","last_unstale":"2024-09-15T21:46:24.835590+0000","last_undegraded":"2024-09-15T21:46:24.835590+0000","last_fullsized":"2024-09-15T21:46:24.835590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:53:15.814759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412551+0000","last_change":"2024-09-15T21:46:22.425870+0000","last_active":"2024-09-15T21:46:24.412551+0000","last_peered":"2024-09-15T21:46:24.412551+0000","last_clean":"2024-09-15T21:46:24.412551+0000","last_became_active":"2024-09-15T21:46:22.425588+0000","last_became_peered":"2024-09-15T21:46:22.425588+0000","last_unstale":"2024-09-15T21:46:24.412551+0000","last_undegraded":"2024-09-15T21:46:24.412551+0000","last_fullsized":"2024-09-15T21:46:24.412551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:40:07.586608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835837+0000","last_change":"2024-09-15T21:46:23.434657+0000","last_active":"2024-09-15T21:46:24.835837+0000","last_peered":"2024-09-15T21:46:24.835837+0000","last_clean":"2024-09-15T21:46:24.835837+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:46:24.835837+0000","last_undegraded":"2024-09-15T21:46:24.835837+0000","last_fullsized":"2024-09-15T21:46:24.835837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:02.545810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808420+0000","last_change":"2024-09-15T21:46:22.425072+0000","last_active":"2024-09-15T21:46:24.808420+0000","last_peered":"2024-09-15T21:46:24.808420+0000","last_clean":"2024-09-15T21:46:24.808420+0000","last_became_active":"2024-09-15T21:46:22.424771+0000","last_became_peered":"2024-09-15T21:46:22.424771+0000","last_unstale":"2024-09-15T21:46:24.808420+0000","last_undegraded":"2024-09-15T21:46:24.808420+0000","last_fullsized":"2024-09-15T21:46:24.808420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:26.256785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414457+0000","last_change":"2024-09-15T21:46:23.432497+0000","last_active":"2024-09-15T21:46:24.414457+0000","last_peered":"2024-09-15T21:46:24.414457+0000","last_clean":"2024-09-15T21:46:24.414457+0000","last_became_active":"2024-09-15T21:46:23.431969+0000","last_became_peered":"2024-09-15T21:46:23.431969+0000","last_unstale":"2024-09-15T21:46:24.414457+0000","last_undegraded":"2024-09-15T21:46:24.414457+0000","last_fullsized":"2024-09-15T21:46:24.414457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:32:56.359477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412577+0000","last_change":"2024-09-15T21:46:22.428668+0000","last_active":"2024-09-15T21:46:24.412577+0000","last_peered":"2024-09-15T21:46:24.412577+0000","last_clean":"2024-09-15T21:46:24.412577+0000","last_became_active":"2024-09-15T21:46:22.428341+0000","last_became_peered":"2024-09-15T21:46:22.428341+0000","last_unstale":"2024-09-15T21:46:24.412577+0000","last_undegraded":"2024-09-15T21:46:24.412577+0000","last_fullsized":"2024-09-15T21:46:24.412577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:29:45.521090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835717+0000","last_change":"2024-09-15T21:46:23.438665+0000","last_active":"2024-09-15T21:46:24.835717+0000","last_peered":"2024-09-15T21:46:24.835717+0000","last_clean":"2024-09-15T21:46:24.835717+0000","last_became_active":"2024-09-15T21:46:23.437606+0000","last_became_peered":"2024-09-15T21:46:23.437606+0000","last_unstale":"2024-09-15T21:46:24.835717+0000","last_undegraded":"2024-09-15T21:46:24.835717+0000","last_fullsized":"2024-09-15T21:46:24.835717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:14:53.751884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":171,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835126+0000","last_change":"2024-09-15T21:46:14.328094+0000","last_active":"2024-09-15T21:46:24.835126+0000","last_peered":"2024-09-15T21:46:24.835126+0000","last_clean":"2024-09-15T21:46:24.835126+0000","last_became_active":"2024-09-15T21:46:14.326991+0000","last_became_peered":"2024-09-15T21:46:14.326991+0000","last_unstale":"2024-09-15T21:46:24.835126+0000","last_undegraded":"2024-09-15T21:46:24.835126+0000","last_fullsized":"2024-09-15T21:46:24.835126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_clean_scrub_stamp":"2024-09-15T21:46:12.223400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:54:27.704130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.413799+0000","last_change":"2024-09-15T21:46:22.428887+0000","last_active":"2024-09-15T21:46:24.413799+0000","last_peered":"2024-09-15T21:46:24.413799+0000","last_clean":"2024-09-15T21:46:24.413799+0000","last_became_active":"2024-09-15T21:46:22.428619+0000","last_became_peered":"2024-09-15T21:46:22.428619+0000","last_unstale":"2024-09-15T21:46:24.413799+0000","last_undegraded":"2024-09-15T21:46:24.413799+0000","last_fullsized":"2024-09-15T21:46:24.413799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:48.434763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834644+0000","last_change":"2024-09-15T21:46:23.435459+0000","last_active":"2024-09-15T21:46:24.834644+0000","last_peered":"2024-09-15T21:46:24.834644+0000","last_clean":"2024-09-15T21:46:24.834644+0000","last_became_active":"2024-09-15T21:46:23.435141+0000","last_became_peered":"2024-09-15T21:46:23.435141+0000","last_unstale":"2024-09-15T21:46:24.834644+0000","last_undegraded":"2024-09-15T21:46:24.834644+0000","last_fullsized":"2024-09-15T21:46:24.834644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:40:50.707124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834903+0000","last_change":"2024-09-15T21:46:22.431113+0000","last_active":"2024-09-15T21:46:24.834903+0000","last_peered":"2024-09-15T21:46:24.834903+0000","last_clean":"2024-09-15T21:46:24.834903+0000","last_became_active":"2024-09-15T21:46:22.430832+0000","last_became_peered":"2024-09-15T21:46:22.430832+0000","last_unstale":"2024-09-15T21:46:24.834903+0000","last_undegraded":"2024-09-15T21:46:24.834903+0000","last_fullsized":"2024-09-15T21:46:24.834903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:28.849259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.726839+0000","last_change":"2024-09-15T21:46:23.430396+0000","last_active":"2024-09-15T21:46:24.726839+0000","last_peered":"2024-09-15T21:46:24.726839+0000","last_clean":"2024-09-15T21:46:24.726839+0000","last_became_active":"2024-09-15T21:46:23.430138+0000","last_became_peered":"2024-09-15T21:46:23.430138+0000","last_unstale":"2024-09-15T21:46:24.726839+0000","last_undegraded":"2024-09-15T21:46:24.726839+0000","last_fullsized":"2024-09-15T21:46:24.726839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:50.622949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412620+0000","last_change":"2024-09-15T21:46:22.425808+0000","last_active":"2024-09-15T21:46:24.412620+0000","last_peered":"2024-09-15T21:46:24.412620+0000","last_clean":"2024-09-15T21:46:24.412620+0000","last_became_active":"2024-09-15T21:46:22.425510+0000","last_became_peered":"2024-09-15T21:46:22.425510+0000","last_unstale":"2024-09-15T21:46:24.412620+0000","last_undegraded":"2024-09-15T21:46:24.412620+0000","last_fullsized":"2024-09-15T21:46:24.412620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:11:00.338076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808338+0000","last_change":"2024-09-15T21:46:23.427051+0000","last_active":"2024-09-15T21:46:24.808338+0000","last_peered":"2024-09-15T21:46:24.808338+0000","last_clean":"2024-09-15T21:46:24.808338+0000","last_became_active":"2024-09-15T21:46:23.426185+0000","last_became_peered":"2024-09-15T21:46:23.426185+0000","last_unstale":"2024-09-15T21:46:24.808338+0000","last_undegraded":"2024-09-15T21:46:24.808338+0000","last_fullsized":"2024-09-15T21:46:24.808338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:48.318405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414866+0000","last_change":"2024-09-15T21:46:22.424645+0000","last_active":"2024-09-15T21:46:24.414866+0000","last_peered":"2024-09-15T21:46:24.414866+0000","last_clean":"2024-09-15T21:46:24.414866+0000","last_became_active":"2024-09-15T21:46:22.421229+0000","last_became_peered":"2024-09-15T21:46:22.421229+0000","last_unstale":"2024-09-15T21:46:24.414866+0000","last_undegraded":"2024-09-15T21:46:24.414866+0000","last_fullsized":"2024-09-15T21:46:24.414866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:49.165670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414000+0000","last_change":"2024-09-15T21:46:23.425699+0000","last_active":"2024-09-15T21:46:24.414000+0000","last_peered":"2024-09-15T21:46:24.414000+0000","last_clean":"2024-09-15T21:46:24.414000+0000","last_became_active":"2024-09-15T21:46:23.425416+0000","last_became_peered":"2024-09-15T21:46:23.425416+0000","last_unstale":"2024-09-15T21:46:24.414000+0000","last_undegraded":"2024-09-15T21:46:24.414000+0000","last_fullsized":"2024-09-15T21:46:24.414000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:00.325179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808438+0000","last_change":"2024-09-15T21:46:22.426492+0000","last_active":"2024-09-15T21:46:24.808438+0000","last_peered":"2024-09-15T21:46:24.808438+0000","last_clean":"2024-09-15T21:46:24.808438+0000","last_became_active":"2024-09-15T21:46:22.426210+0000","last_became_peered":"2024-09-15T21:46:22.426210+0000","last_unstale":"2024-09-15T21:46:24.808438+0000","last_undegraded":"2024-09-15T21:46:24.808438+0000","last_fullsized":"2024-09-15T21:46:24.808438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:05.248091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414861+0000","last_change":"2024-09-15T21:46:23.429837+0000","last_active":"2024-09-15T21:46:24.414861+0000","last_peered":"2024-09-15T21:46:24.414861+0000","last_clean":"2024-09-15T21:46:24.414861+0000","last_became_active":"2024-09-15T21:46:23.429254+0000","last_became_peered":"2024-09-15T21:46:23.429254+0000","last_unstale":"2024-09-15T21:46:24.414861+0000","last_undegraded":"2024-09-15T21:46:24.414861+0000","last_fullsized":"2024-09-15T21:46:24.414861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:58.465583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835937+0000","last_change":"2024-09-15T21:46:22.430710+0000","last_active":"2024-09-15T21:46:24.835937+0000","last_peered":"2024-09-15T21:46:24.835937+0000","last_clean":"2024-09-15T21:46:24.835937+0000","last_became_active":"2024-09-15T21:46:22.430375+0000","last_became_peered":"2024-09-15T21:46:22.430375+0000","last_unstale":"2024-09-15T21:46:24.835937+0000","last_undegraded":"2024-09-15T21:46:24.835937+0000","last_fullsized":"2024-09-15T21:46:24.835937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:04.081179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808447+0000","last_change":"2024-09-15T21:46:23.426729+0000","last_active":"2024-09-15T21:46:24.808447+0000","last_peered":"2024-09-15T21:46:24.808447+0000","last_clean":"2024-09-15T21:46:24.808447+0000","last_became_active":"2024-09-15T21:46:23.426495+0000","last_became_peered":"2024-09-15T21:46:23.426495+0000","last_unstale":"2024-09-15T21:46:24.808447+0000","last_undegraded":"2024-09-15T21:46:24.808447+0000","last_fullsized":"2024-09-15T21:46:24.808447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:03.277997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808436+0000","last_change":"2024-09-15T21:46:22.427137+0000","last_active":"2024-09-15T21:46:24.808436+0000","last_peered":"2024-09-15T21:46:24.808436+0000","last_clean":"2024-09-15T21:46:24.808436+0000","last_became_active":"2024-09-15T21:46:22.426855+0000","last_became_peered":"2024-09-15T21:46:22.426855+0000","last_unstale":"2024-09-15T21:46:24.808436+0000","last_undegraded":"2024-09-15T21:46:24.808436+0000","last_fullsized":"2024-09-15T21:46:24.808436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:40:14.277940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834714+0000","last_change":"2024-09-15T21:46:23.427327+0000","last_active":"2024-09-15T21:46:24.834714+0000","last_peered":"2024-09-15T21:46:24.834714+0000","last_clean":"2024-09-15T21:46:24.834714+0000","last_became_active":"2024-09-15T21:46:23.427096+0000","last_became_peered":"2024-09-15T21:46:23.427096+0000","last_unstale":"2024-09-15T21:46:24.834714+0000","last_undegraded":"2024-09-15T21:46:24.834714+0000","last_fullsized":"2024-09-15T21:46:24.834714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:32.370390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835297+0000","last_change":"2024-09-15T21:46:22.427798+0000","last_active":"2024-09-15T21:46:24.835297+0000","last_peered":"2024-09-15T21:46:24.835297+0000","last_clean":"2024-09-15T21:46:24.835297+0000","last_became_active":"2024-09-15T21:46:22.427537+0000","last_became_peered":"2024-09-15T21:46:22.427537+0000","last_unstale":"2024-09-15T21:46:24.835297+0000","last_undegraded":"2024-09-15T21:46:24.835297+0000","last_fullsized":"2024-09-15T21:46:24.835297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:18:54.963834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727157+0000","last_change":"2024-09-15T21:46:23.424266+0000","last_active":"2024-09-15T21:46:24.727157+0000","last_peered":"2024-09-15T21:46:24.727157+0000","last_clean":"2024-09-15T21:46:24.727157+0000","last_became_active":"2024-09-15T21:46:23.424007+0000","last_became_peered":"2024-09-15T21:46:23.424007+0000","last_unstale":"2024-09-15T21:46:24.727157+0000","last_undegraded":"2024-09-15T21:46:24.727157+0000","last_fullsized":"2024-09-15T21:46:24.727157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:23.492253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835005+0000","last_change":"2024-09-15T21:46:22.430901+0000","last_active":"2024-09-15T21:46:24.835005+0000","last_peered":"2024-09-15T21:46:24.835005+0000","last_clean":"2024-09-15T21:46:24.835005+0000","last_became_active":"2024-09-15T21:46:22.430452+0000","last_became_peered":"2024-09-15T21:46:22.430452+0000","last_unstale":"2024-09-15T21:46:24.835005+0000","last_undegraded":"2024-09-15T21:46:24.835005+0000","last_fullsized":"2024-09-15T21:46:24.835005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:59:45.663976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.413661+0000","last_change":"2024-09-15T21:46:23.430836+0000","last_active":"2024-09-15T21:46:24.413661+0000","last_peered":"2024-09-15T21:46:24.413661+0000","last_clean":"2024-09-15T21:46:24.413661+0000","last_became_active":"2024-09-15T21:46:23.429283+0000","last_became_peered":"2024-09-15T21:46:23.429283+0000","last_unstale":"2024-09-15T21:46:24.413661+0000","last_undegraded":"2024-09-15T21:46:24.413661+0000","last_fullsized":"2024-09-15T21:46:24.413661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:30.571376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808380+0000","last_change":"2024-09-15T21:46:22.423795+0000","last_active":"2024-09-15T21:46:24.808380+0000","last_peered":"2024-09-15T21:46:24.808380+0000","last_clean":"2024-09-15T21:46:24.808380+0000","last_became_active":"2024-09-15T21:46:22.422327+0000","last_became_peered":"2024-09-15T21:46:22.422327+0000","last_unstale":"2024-09-15T21:46:24.808380+0000","last_undegraded":"2024-09-15T21:46:24.808380+0000","last_fullsized":"2024-09-15T21:46:24.808380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:16.670599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834522+0000","last_change":"2024-09-15T21:46:23.428634+0000","last_active":"2024-09-15T21:46:24.834522+0000","last_peered":"2024-09-15T21:46:24.834522+0000","last_clean":"2024-09-15T21:46:24.834522+0000","last_became_active":"2024-09-15T21:46:23.428301+0000","last_became_peered":"2024-09-15T21:46:23.428301+0000","last_unstale":"2024-09-15T21:46:24.834522+0000","last_undegraded":"2024-09-15T21:46:24.834522+0000","last_fullsized":"2024-09-15T21:46:24.834522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:59.492942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808304+0000","last_change":"2024-09-15T21:46:22.425810+0000","last_active":"2024-09-15T21:46:24.808304+0000","last_peered":"2024-09-15T21:46:24.808304+0000","last_clean":"2024-09-15T21:46:24.808304+0000","last_became_active":"2024-09-15T21:46:22.425478+0000","last_became_peered":"2024-09-15T21:46:22.425478+0000","last_unstale":"2024-09-15T21:46:24.808304+0000","last_undegraded":"2024-09-15T21:46:24.808304+0000","last_fullsized":"2024-09-15T21:46:24.808304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:53.296599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834743+0000","last_change":"2024-09-15T21:46:23.426054+0000","last_active":"2024-09-15T21:46:24.834743+0000","last_peered":"2024-09-15T21:46:24.834743+0000","last_clean":"2024-09-15T21:46:24.834743+0000","last_became_active":"2024-09-15T21:46:23.425547+0000","last_became_peered":"2024-09-15T21:46:23.425547+0000","last_unstale":"2024-09-15T21:46:24.834743+0000","last_undegraded":"2024-09-15T21:46:24.834743+0000","last_fullsized":"2024-09-15T21:46:24.834743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:56.838503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835609+0000","last_change":"2024-09-15T21:46:22.428524+0000","last_active":"2024-09-15T21:46:24.835609+0000","last_peered":"2024-09-15T21:46:24.835609+0000","last_clean":"2024-09-15T21:46:24.835609+0000","last_became_active":"2024-09-15T21:46:22.428043+0000","last_became_peered":"2024-09-15T21:46:22.428043+0000","last_unstale":"2024-09-15T21:46:24.835609+0000","last_undegraded":"2024-09-15T21:46:24.835609+0000","last_fullsized":"2024-09-15T21:46:24.835609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:08.393381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834850+0000","last_change":"2024-09-15T21:46:23.434719+0000","last_active":"2024-09-15T21:46:24.834850+0000","last_peered":"2024-09-15T21:46:24.834850+0000","last_clean":"2024-09-15T21:46:24.834850+0000","last_became_active":"2024-09-15T21:46:23.434448+0000","last_became_peered":"2024-09-15T21:46:23.434448+0000","last_unstale":"2024-09-15T21:46:24.834850+0000","last_undegraded":"2024-09-15T21:46:24.834850+0000","last_fullsized":"2024-09-15T21:46:24.834850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:14.901671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414691+0000","last_change":"2024-09-15T21:46:23.430058+0000","last_active":"2024-09-15T21:46:24.414691+0000","last_peered":"2024-09-15T21:46:24.414691+0000","last_clean":"2024-09-15T21:46:24.414691+0000","last_became_active":"2024-09-15T21:46:23.429598+0000","last_became_peered":"2024-09-15T21:46:23.429598+0000","last_unstale":"2024-09-15T21:46:24.414691+0000","last_undegraded":"2024-09-15T21:46:24.414691+0000","last_fullsized":"2024-09-15T21:46:24.414691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:44.142207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835254+0000","last_change":"2024-09-15T21:46:22.431172+0000","last_active":"2024-09-15T21:46:24.835254+0000","last_peered":"2024-09-15T21:46:24.835254+0000","last_clean":"2024-09-15T21:46:24.835254+0000","last_became_active":"2024-09-15T21:46:22.430800+0000","last_became_peered":"2024-09-15T21:46:22.430800+0000","last_unstale":"2024-09-15T21:46:24.835254+0000","last_undegraded":"2024-09-15T21:46:24.835254+0000","last_fullsized":"2024-09-15T21:46:24.835254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:55.072466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414264+0000","last_change":"2024-09-15T21:46:23.434353+0000","last_active":"2024-09-15T21:46:24.414264+0000","last_peered":"2024-09-15T21:46:24.414264+0000","last_clean":"2024-09-15T21:46:24.414264+0000","last_became_active":"2024-09-15T21:46:23.434097+0000","last_became_peered":"2024-09-15T21:46:23.434097+0000","last_unstale":"2024-09-15T21:46:24.414264+0000","last_undegraded":"2024-09-15T21:46:24.414264+0000","last_fullsized":"2024-09-15T21:46:24.414264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:46.007227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835783+0000","last_change":"2024-09-15T21:46:22.429870+0000","last_active":"2024-09-15T21:46:24.835783+0000","last_peered":"2024-09-15T21:46:24.835783+0000","last_clean":"2024-09-15T21:46:24.835783+0000","last_became_active":"2024-09-15T21:46:22.429578+0000","last_became_peered":"2024-09-15T21:46:22.429578+0000","last_unstale":"2024-09-15T21:46:24.835783+0000","last_undegraded":"2024-09-15T21:46:24.835783+0000","last_fullsized":"2024-09-15T21:46:24.835783+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:41.678078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727142+0000","last_change":"2024-09-15T21:46:23.429604+0000","last_active":"2024-09-15T21:46:24.727142+0000","last_peered":"2024-09-15T21:46:24.727142+0000","last_clean":"2024-09-15T21:46:24.727142+0000","last_became_active":"2024-09-15T21:46:23.429355+0000","last_became_peered":"2024-09-15T21:46:23.429355+0000","last_unstale":"2024-09-15T21:46:24.727142+0000","last_undegraded":"2024-09-15T21:46:24.727142+0000","last_fullsized":"2024-09-15T21:46:24.727142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:45.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834679+0000","last_change":"2024-09-15T21:46:22.428529+0000","last_active":"2024-09-15T21:46:24.834679+0000","last_peered":"2024-09-15T21:46:24.834679+0000","last_clean":"2024-09-15T21:46:24.834679+0000","last_became_active":"2024-09-15T21:46:22.428263+0000","last_became_peered":"2024-09-15T21:46:22.428263+0000","last_unstale":"2024-09-15T21:46:24.834679+0000","last_undegraded":"2024-09-15T21:46:24.834679+0000","last_fullsized":"2024-09-15T21:46:24.834679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:22:40.160062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834701+0000","last_change":"2024-09-15T21:46:23.427968+0000","last_active":"2024-09-15T21:46:24.834701+0000","last_peered":"2024-09-15T21:46:24.834701+0000","last_clean":"2024-09-15T21:46:24.834701+0000","last_became_active":"2024-09-15T21:46:23.427546+0000","last_became_peered":"2024-09-15T21:46:23.427546+0000","last_unstale":"2024-09-15T21:46:24.834701+0000","last_undegraded":"2024-09-15T21:46:24.834701+0000","last_fullsized":"2024-09-15T21:46:24.834701+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:47.226097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.419172+0000","last_change":"2024-09-15T21:46:22.428258+0000","last_active":"2024-09-15T21:46:28.419172+0000","last_peered":"2024-09-15T21:46:28.419172+0000","last_clean":"2024-09-15T21:46:28.419172+0000","last_became_active":"2024-09-15T21:46:22.427954+0000","last_became_peered":"2024-09-15T21:46:22.427954+0000","last_unstale":"2024-09-15T21:46:28.419172+0000","last_undegraded":"2024-09-15T21:46:28.419172+0000","last_fullsized":"2024-09-15T21:46:28.419172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:22.440456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":616,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476741,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":667326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":815104,"data_stored":667326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":76958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":76958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509446,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":76958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":221184,"data_stored":76958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":76958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":78570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:46:33.047 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:46:33.048 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T21:46:33.048 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-15T21:46:33.048 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-15T21:46:33.048 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-15T21:46:33.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.181+0000 7f4f10e7b640 1 Processor -- start 2024-09-15T21:46:33.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.181+0000 7f4f10e7b640 1 -- start start 2024-09-15T21:46:33.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.182+0000 7f4f10e7b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f4f0c0c4600 0x7f4f0c06d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.182+0000 7f4f10e7b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f0c0c4a10 con 0x7f4f0c152160 2024-09-15T21:46:33.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.182+0000 7f4f10e7b640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f0c157ce0 con 0x7f4f0c073790 2024-09-15T21:46:33.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.182+0000 7f4f10e7b640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f4f0c072bf0 con 0x7f4f0c0c4600 2024-09-15T21:46:33.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.182+0000 7f4f0a59c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f4f0c152160 0x7f4f0c073080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:51812/0 (socket says 172.21.15.100:51812) 2024-09-15T21:46:33.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.182+0000 7f4f0a59c640 1 -- 172.21.15.100:0/149495023 learned_addr learned my addr 172.21.15.100:0/149495023 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:33.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.182+0000 7f4f0ad9d640 1 --2- 172.21.15.100:0/149495023 >> v2:172.21.15.122:3300/0 conn(0x7f4f0c0c4600 0x7f4f0c06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.182+0000 7f4f0959a640 1 -- 172.21.15.100:0/149495023 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2852951858 0 0) 0x7f4f04001110 con 0x7f4f0c152160 2024-09-15T21:46:33.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.182+0000 7f4f0959a640 1 -- 172.21.15.100:0/149495023 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ef4003520 con 0x7f4f0c152160 2024-09-15T21:46:33.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.182+0000 7f4f0959a640 1 -- 172.21.15.100:0/149495023 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 171747307 0 0) 0x7f4ef4003520 con 0x7f4f0c152160 2024-09-15T21:46:33.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.182+0000 7f4f0959a640 1 -- 172.21.15.100:0/149495023 >> v2:172.21.15.122:3300/0 conn(0x7f4f0c0c4600 msgr2=0x7f4f0c06d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:33.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.182+0000 7f4f0959a640 1 --2- 172.21.15.100:0/149495023 >> v2:172.21.15.122:3300/0 conn(0x7f4f0c0c4600 0x7f4f0c06d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.182+0000 7f4f0959a640 1 -- 172.21.15.100:0/149495023 >> v1:172.21.15.122:6789/0 conn(0x7f4f0c073790 legacy=0x7f4f0c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:33.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.182+0000 7f4f0959a640 1 -- 172.21.15.100:0/149495023 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0c157fa0 con 0x7f4f0c152160 2024-09-15T21:46:33.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.182+0000 7f4f0959a640 1 -- 172.21.15.100:0/149495023 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7f4f04001840 con 0x7f4f0c152160 2024-09-15T21:46:33.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.182+0000 7f4f0ad9d640 1 --2- 172.21.15.100:0/149495023 >> v2:172.21.15.122:3300/0 conn(0x7f4f0c0c4600 0x7f4f0c06d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:33.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.183+0000 7f4f0959a640 1 -- 172.21.15.100:0/149495023 >> v1:172.21.15.100:6789/0 conn(0x7f4f0c152160 legacy=0x7f4f0c073080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:33.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.183+0000 7f4f0959a640 1 --2- 172.21.15.100:0/149495023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef40039b0 0x7f4ef4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.183+0000 7f4f0959a640 1 --2- 172.21.15.100:0/149495023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ef4004800 0x7f4ef4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.183+0000 7f4f0959a640 1 --2- 172.21.15.100:0/149495023 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4ef40051c0 0x7f4ef400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.183+0000 7f4f0959a640 1 -- 172.21.15.100:0/149495023 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef400c0e0 con 0x7f4ef40039b0 2024-09-15T21:46:33.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.183+0000 7f4f0959a640 1 -- 172.21.15.100:0/149495023 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef400c290 con 0x7f4ef4004800 2024-09-15T21:46:33.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.183+0000 7f4f0959a640 1 -- 172.21.15.100:0/149495023 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4ef400e6a0 con 0x7f4ef40051c0 2024-09-15T21:46:33.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.183+0000 7f4f0a59c640 1 --2- 172.21.15.100:0/149495023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef40039b0 0x7f4ef4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.183+0000 7f4f09d9b640 1 --2- 172.21.15.100:0/149495023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ef4004800 0x7f4ef4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.183+0000 7f4f0ad9d640 1 --2- 172.21.15.100:0/149495023 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4ef40051c0 0x7f4ef400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.183+0000 7f4f0a59c640 1 -- 172.21.15.100:0/149495023 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4ef40051c0 msgr2=0x7f4ef400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:33.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.183+0000 7f4f0a59c640 1 --2- 172.21.15.100:0/149495023 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4ef40051c0 0x7f4ef400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.183+0000 7f4f0a59c640 1 -- 172.21.15.100:0/149495023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ef4004800 msgr2=0x7f4ef4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:33.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.183+0000 7f4f0a59c640 1 --2- 172.21.15.100:0/149495023 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ef4004800 0x7f4ef4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.183+0000 7f4f0a59c640 1 -- 172.21.15.100:0/149495023 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef4003520 con 0x7f4ef40039b0 2024-09-15T21:46:33.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.183+0000 7f4f0ad9d640 1 -- 172.21.15.100:0/149495023 reap_dead start 2024-09-15T21:46:33.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.184+0000 7f4f0a59c640 1 --2- 172.21.15.100:0/149495023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef40039b0 0x7f4ef4003dd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400c4c0 tx=0x7f4ef4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6c5a39a981355b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:33.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.184+0000 7f4f0959a640 1 -- 172.21.15.100:0/149495023 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0400c0e0 con 0x7f4ef40039b0 2024-09-15T21:46:33.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.184+0000 7f4f0959a640 1 -- 172.21.15.100:0/149495023 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f04034070 con 0x7f4ef40039b0 2024-09-15T21:46:33.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.184+0000 7f4f0959a640 1 -- 172.21.15.100:0/149495023 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f04005d70 con 0x7f4ef40039b0 2024-09-15T21:46:33.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.184+0000 7f4f10e7b640 1 -- 172.21.15.100:0/149495023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef40039b0 msgr2=0x7f4ef4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:33.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.184+0000 7f4f10e7b640 1 --2- 172.21.15.100:0/149495023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef40039b0 0x7f4ef4003dd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400c4c0 tx=0x7f4ef4005830 comp rx=0 tx=0).stop 2024-09-15T21:46:33.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.184+0000 7f4f10e7b640 1 -- 172.21.15.100:0/149495023 shutdown_connections 2024-09-15T21:46:33.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.184+0000 7f4f10e7b640 1 --2- 172.21.15.100:0/149495023 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4ef40039b0 0x7f4ef4003dd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.184+0000 7f4f10e7b640 1 -- 172.21.15.100:0/149495023 >> 172.21.15.100:0/149495023 conn(0x7f4f0c0ca390 msgr2=0x7f4f0c0ca770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:33.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.184+0000 7f4f10e7b640 1 -- 172.21.15.100:0/149495023 shutdown_connections 2024-09-15T21:46:33.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.184+0000 7f4f10e7b640 1 -- 172.21.15.100:0/149495023 wait complete. 2024-09-15T21:46:33.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.186+0000 7f4f10e7b640 1 Processor -- start 2024-09-15T21:46:33.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.186+0000 7f4f10e7b640 1 -- start start 2024-09-15T21:46:33.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.186+0000 7f4f10e7b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ef40039b0 0x7f4f0c157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.186+0000 7f4f10e7b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f0c147740 0x7f4f0c14dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.186+0000 7f4f10e7b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4f0c147b80 0x7f4f0c14e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.186+0000 7f4f10e7b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c158840 con 0x7f4f0c147740 2024-09-15T21:46:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.186+0000 7f4f10e7b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c158560 con 0x7f4ef40039b0 2024-09-15T21:46:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.186+0000 7f4f10e7b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4f0c158b20 con 0x7f4f0c147b80 2024-09-15T21:46:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.187+0000 7f4f09d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f0c147740 0x7f4f0c14dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.187+0000 7f4f09d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f0c147740 0x7f4f0c14dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43780/0 (socket says 172.21.15.100:43780) 2024-09-15T21:46:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.187+0000 7f4f09d9b640 1 -- 172.21.15.100:0/520640866 learned_addr learned my addr 172.21.15.100:0/520640866 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.187+0000 7f4f0ad9d640 1 --2- 172.21.15.100:0/520640866 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4f0c147b80 0x7f4f0c14e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.187+0000 7f4f0a59c640 1 --2- 172.21.15.100:0/520640866 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ef40039b0 0x7f4f0c157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.187+0000 7f4f09d9b640 1 -- 172.21.15.100:0/520640866 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4f0c147b80 msgr2=0x7f4f0c14e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.187+0000 7f4f09d9b640 1 --2- 172.21.15.100:0/520640866 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4f0c147b80 0x7f4f0c14e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.187+0000 7f4f09d9b640 1 -- 172.21.15.100:0/520640866 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ef40039b0 msgr2=0x7f4f0c157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.187+0000 7f4f09d9b640 1 --2- 172.21.15.100:0/520640866 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ef40039b0 0x7f4f0c157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.187+0000 7f4f09d9b640 1 -- 172.21.15.100:0/520640866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0c0cc9e0 con 0x7f4f0c147740 2024-09-15T21:46:33.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.187+0000 7f4f09d9b640 1 --2- 172.21.15.100:0/520640866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f0c147740 0x7f4f0c14dc20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000f8f0 tx=0x7f4f0000fdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:33.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.187+0000 7f4efaffd640 1 -- 172.21.15.100:0/520640866 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00003b80 con 0x7f4f0c147740 2024-09-15T21:46:33.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.187+0000 7f4efaffd640 1 -- 172.21.15.100:0/520640866 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f00018070 con 0x7f4f0c147740 2024-09-15T21:46:33.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.187+0000 7f4efaffd640 1 -- 172.21.15.100:0/520640866 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f000050d0 con 0x7f4f0c147740 2024-09-15T21:46:33.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.187+0000 7f4f10e7b640 1 -- 172.21.15.100:0/520640866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0c14d260 con 0x7f4f0c147740 2024-09-15T21:46:33.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.187+0000 7f4f10e7b640 1 -- 172.21.15.100:0/520640866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f0c155b50 con 0x7f4f0c147740 2024-09-15T21:46:33.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.188+0000 7f4f10e7b640 1 -- 172.21.15.100:0/520640866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f0c06efd0 con 0x7f4f0c147740 2024-09-15T21:46:33.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.189+0000 7f4efaffd640 1 -- 172.21.15.100:0/520640866 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4f00005270 con 0x7f4f0c147740 2024-09-15T21:46:33.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.189+0000 7f4efaffd640 1 --2- 172.21.15.100:0/520640866 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4eec06c2b0 0x7f4eec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.189+0000 7f4f0a59c640 1 --2- 172.21.15.100:0/520640866 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4eec06c2b0 0x7f4eec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.189+0000 7f4efaffd640 1 -- 172.21.15.100:0/520640866 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f4f0008b2c0 con 0x7f4f0c147740 2024-09-15T21:46:33.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.189+0000 7f4f0a59c640 1 --2- 172.21.15.100:0/520640866 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4eec06c2b0 0x7f4eec06e770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4f04033000 tx=0x7f4f04009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:33.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.193+0000 7f4efaffd640 1 -- 172.21.15.100:0/520640866 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4f00059470 con 0x7f4f0c147740 2024-09-15T21:46:33.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.404+0000 7f4f10e7b640 1 -- 172.21.15.100:0/520640866 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f4f0c06dd40 con 0x7f4f0c147740 2024-09-15T21:46:33.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.406+0000 7f4efaffd640 1 -- 172.21.15.100:0/520640866 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+173512 (secure 0 0 0) 0x7f4f0005d770 con 0x7f4f0c147740 2024-09-15T21:46:33.411 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:33.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.409+0000 7f4f10e7b640 1 -- 172.21.15.100:0/520640866 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4eec06c2b0 msgr2=0x7f4eec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:33.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.409+0000 7f4f10e7b640 1 --2- 172.21.15.100:0/520640866 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4eec06c2b0 0x7f4eec06e770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4f04033000 tx=0x7f4f04009030 comp rx=0 tx=0).stop 2024-09-15T21:46:33.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.409+0000 7f4f10e7b640 1 -- 172.21.15.100:0/520640866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f0c147740 msgr2=0x7f4f0c14dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:33.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.409+0000 7f4f10e7b640 1 --2- 172.21.15.100:0/520640866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f0c147740 0x7f4f0c14dc20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4f0000f8f0 tx=0x7f4f0000fdb0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.409+0000 7f4f10e7b640 1 -- 172.21.15.100:0/520640866 shutdown_connections 2024-09-15T21:46:33.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.409+0000 7f4f10e7b640 1 --2- 172.21.15.100:0/520640866 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4eec06c2b0 0x7f4eec06e770 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.409+0000 7f4f10e7b640 1 --2- 172.21.15.100:0/520640866 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4f0c147b80 0x7f4f0c14e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.409+0000 7f4f10e7b640 1 --2- 172.21.15.100:0/520640866 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f0c147740 0x7f4f0c14dc20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.409+0000 7f4f10e7b640 1 --2- 172.21.15.100:0/520640866 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4ef40039b0 0x7f4f0c157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.409+0000 7f4f10e7b640 1 -- 172.21.15.100:0/520640866 >> 172.21.15.100:0/520640866 conn(0x7f4f0c0ca390 msgr2=0x7f4f0c0c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:33.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.409+0000 7f4f10e7b640 1 -- 172.21.15.100:0/520640866 shutdown_connections 2024-09-15T21:46:33.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.409+0000 7f4f10e7b640 1 -- 172.21.15.100:0/520640866 wait complete. 2024-09-15T21:46:33.422 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3516908621},{"type":"v1","addr":"172.21.15.100:6809","nonce":3516908621}]},"active_addr":"172.21.15.100:6809/3516908621","active_change":"2024-09-15T21:46:03.797642+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":4235397466}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":1752340873}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":2756546806}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.100:0","nonce":3953988934}]}]} 2024-09-15T21:46:33.425 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-15T21:46:33.425 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T21:46:33.426 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:46:33.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.522+0000 7f33fb303640 1 Processor -- start 2024-09-15T21:46:33.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.522+0000 7f33fb303640 1 -- start start 2024-09-15T21:46:33.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33fb303640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33f40cb7d0 0x7f33f40cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33fb303640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f33f406d0e0 0x7f33f40cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33fb303640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33f415e7a0 con 0x7f33f4155440 2024-09-15T21:46:33.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33fb303640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f33f415b470 con 0x7f33f406d0e0 2024-09-15T21:46:33.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33fb303640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f33f4155820 con 0x7f33f40cb7d0 2024-09-15T21:46:33.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33f909f640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f33f4155440 0x7f33f40cd420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:51822/0 (socket says 172.21.15.100:51822) 2024-09-15T21:46:33.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33f909f640 1 -- 172.21.15.100:0/2184430043 learned_addr learned my addr 172.21.15.100:0/2184430043 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:33.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33ebfff640 1 -- 172.21.15.100:0/2184430043 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3681668813 0 0) 0x7f33f415e7a0 con 0x7f33f4155440 2024-09-15T21:46:33.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33ebfff640 1 -- 172.21.15.100:0/2184430043 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33e0003520 con 0x7f33f4155440 2024-09-15T21:46:33.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33f889e640 1 --2- 172.21.15.100:0/2184430043 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33f40cb7d0 0x7f33f40cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33f98a0640 1 --2- 172.21.15.100:0/2184430043 >> v2:172.21.15.122:3300/0 conn(0x7f33f406d0e0 0x7f33f40cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33ebfff640 1 -- 172.21.15.100:0/2184430043 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2525590834 0 0) 0x7f33e0003520 con 0x7f33f4155440 2024-09-15T21:46:33.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33ebfff640 1 -- 172.21.15.100:0/2184430043 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33f40cb7d0 msgr2=0x7f33f40cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:33.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33ebfff640 1 --2- 172.21.15.100:0/2184430043 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33f40cb7d0 0x7f33f40cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33ebfff640 1 -- 172.21.15.100:0/2184430043 >> v2:172.21.15.122:3300/0 conn(0x7f33f406d0e0 msgr2=0x7f33f40cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:33.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33ebfff640 1 --2- 172.21.15.100:0/2184430043 >> v2:172.21.15.122:3300/0 conn(0x7f33f406d0e0 0x7f33f40cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33ebfff640 1 -- 172.21.15.100:0/2184430043 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f415ea40 con 0x7f33f4155440 2024-09-15T21:46:33.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33ebfff640 1 -- 172.21.15.100:0/2184430043 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7f33f0001670 con 0x7f33f4155440 2024-09-15T21:46:33.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33f98a0640 1 --2- 172.21.15.100:0/2184430043 >> v2:172.21.15.122:3300/0 conn(0x7f33f406d0e0 0x7f33f40cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:33.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33ebfff640 1 -- 172.21.15.100:0/2184430043 >> v1:172.21.15.100:6789/0 conn(0x7f33f4155440 legacy=0x7f33f40cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:33.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33ebfff640 1 --2- 172.21.15.100:0/2184430043 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33f40cb7d0 0x7f33e00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33ebfff640 1 --2- 172.21.15.100:0/2184430043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e0004030 0x7f33e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33ebfff640 1 --2- 172.21.15.100:0/2184430043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33e0004d40 0x7f33e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33ebfff640 1 -- 172.21.15.100:0/2184430043 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f33f415b470 con 0x7f33e0004030 2024-09-15T21:46:33.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33ebfff640 1 -- 172.21.15.100:0/2184430043 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f33f4155820 con 0x7f33e0004d40 2024-09-15T21:46:33.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33ebfff640 1 -- 172.21.15.100:0/2184430043 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f33e000bbd0 con 0x7f33f40cb7d0 2024-09-15T21:46:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33f889e640 1 --2- 172.21.15.100:0/2184430043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e0004030 0x7f33e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33f909f640 1 --2- 172.21.15.100:0/2184430043 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33f40cb7d0 0x7f33e00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.523+0000 7f33f98a0640 1 --2- 172.21.15.100:0/2184430043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33e0004d40 0x7f33e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.524+0000 7f33f889e640 1 -- 172.21.15.100:0/2184430043 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33f40cb7d0 msgr2=0x7f33e00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.524+0000 7f33f889e640 1 --2- 172.21.15.100:0/2184430043 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33f40cb7d0 0x7f33e00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.524+0000 7f33f889e640 1 -- 172.21.15.100:0/2184430043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33e0004d40 msgr2=0x7f33e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.524+0000 7f33f889e640 1 --2- 172.21.15.100:0/2184430043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33e0004d40 0x7f33e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.524+0000 7f33f889e640 1 -- 172.21.15.100:0/2184430043 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e0003520 con 0x7f33e0004030 2024-09-15T21:46:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.524+0000 7f33f98a0640 1 --2- 172.21.15.100:0/2184430043 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33e0004d40 0x7f33e00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.524+0000 7f33f889e640 1 --2- 172.21.15.100:0/2184430043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e0004030 0x7f33e0004800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f33e400ba30 tx=0x7f33e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3208cd9ef85733 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.524+0000 7f33ebfff640 1 -- 172.21.15.100:0/2184430043 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4030070 con 0x7f33e0004030 2024-09-15T21:46:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.524+0000 7f33ebfff640 1 -- 172.21.15.100:0/2184430043 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33e402ade0 con 0x7f33e0004030 2024-09-15T21:46:33.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.524+0000 7f33ebfff640 1 -- 172.21.15.100:0/2184430043 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e400b600 con 0x7f33e0004030 2024-09-15T21:46:33.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.525+0000 7f33fb303640 1 -- 172.21.15.100:0/2184430043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e0004030 msgr2=0x7f33e0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:33.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.525+0000 7f33fb303640 1 --2- 172.21.15.100:0/2184430043 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e0004030 0x7f33e0004800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f33e400ba30 tx=0x7f33e0005830 comp rx=0 tx=0).stop 2024-09-15T21:46:33.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.525+0000 7f33f98a0640 1 -- 172.21.15.100:0/2184430043 reap_dead start 2024-09-15T21:46:33.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.525+0000 7f33fb303640 1 -- 172.21.15.100:0/2184430043 shutdown_connections 2024-09-15T21:46:33.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.525+0000 7f33fb303640 1 -- 172.21.15.100:0/2184430043 >> 172.21.15.100:0/2184430043 conn(0x7f33f40680d0 msgr2=0x7f33f406d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:33.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.525+0000 7f33fb303640 1 -- 172.21.15.100:0/2184430043 shutdown_connections 2024-09-15T21:46:33.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.525+0000 7f33fb303640 1 -- 172.21.15.100:0/2184430043 wait complete. 2024-09-15T21:46:33.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.526+0000 7f33fb303640 1 Processor -- start 2024-09-15T21:46:33.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.526+0000 7f33fb303640 1 -- start start 2024-09-15T21:46:33.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33fb303640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33e0004030 0x7f33f4155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33fb303640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33f41f46c0 0x7f33f41f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33fb303640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33f40c7f70 0x7f33f40c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33fb303640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f33f415f2e0 con 0x7f33f41f46c0 2024-09-15T21:46:33.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33fb303640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f33f415f000 con 0x7f33f40c7f70 2024-09-15T21:46:33.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33fb303640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f33f415b070 con 0x7f33e0004030 2024-09-15T21:46:33.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33f889e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33f41f46c0 0x7f33f41f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33f889e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33f41f46c0 0x7f33f41f4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43792/0 (socket says 172.21.15.100:43792) 2024-09-15T21:46:33.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33f889e640 1 -- 172.21.15.100:0/2150884506 learned_addr learned my addr 172.21.15.100:0/2150884506 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:33.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33f909f640 1 --2- 172.21.15.100:0/2150884506 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33e0004030 0x7f33f4155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33f98a0640 1 --2- 172.21.15.100:0/2150884506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33f40c7f70 0x7f33f40c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33f889e640 1 -- 172.21.15.100:0/2150884506 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33e0004030 msgr2=0x7f33f4155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:33.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33f889e640 1 --2- 172.21.15.100:0/2150884506 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33e0004030 0x7f33f4155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33f889e640 1 -- 172.21.15.100:0/2150884506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33f40c7f70 msgr2=0x7f33f40c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:33.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33f889e640 1 --2- 172.21.15.100:0/2150884506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33f40c7f70 0x7f33f40c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33f889e640 1 -- 172.21.15.100:0/2150884506 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33f415df60 con 0x7f33f41f46c0 2024-09-15T21:46:33.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33f909f640 1 --2- 172.21.15.100:0/2150884506 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33e0004030 0x7f33f4155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:33.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33f98a0640 1 --2- 172.21.15.100:0/2150884506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33f40c7f70 0x7f33f40c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:33.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33f889e640 1 --2- 172.21.15.100:0/2150884506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33f41f46c0 0x7f33f41f4b00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f33e402a8d0 tx=0x7f33e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33e9ffb640 1 -- 172.21.15.100:0/2150884506 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e4043030 con 0x7f33f41f46c0 2024-09-15T21:46:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33e9ffb640 1 -- 172.21.15.100:0/2150884506 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33e402abe0 con 0x7f33f41f46c0 2024-09-15T21:46:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33fb303640 1 -- 172.21.15.100:0/2150884506 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33f415e4e0 con 0x7f33f41f46c0 2024-09-15T21:46:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.527+0000 7f33e9ffb640 1 -- 172.21.15.100:0/2150884506 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33e400b600 con 0x7f33f41f46c0 2024-09-15T21:46:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.528+0000 7f33fb303640 1 -- 172.21.15.100:0/2150884506 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33f415e200 con 0x7f33f41f46c0 2024-09-15T21:46:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.529+0000 7f33fb303640 1 -- 172.21.15.100:0/2150884506 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33f40b3940 con 0x7f33f41f46c0 2024-09-15T21:46:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.529+0000 7f33e9ffb640 1 -- 172.21.15.100:0/2150884506 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f33e4046070 con 0x7f33f41f46c0 2024-09-15T21:46:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.529+0000 7f33e9ffb640 1 --2- 172.21.15.100:0/2150884506 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f33d806c2b0 0x7f33d806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.529+0000 7f33f909f640 1 --2- 172.21.15.100:0/2150884506 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f33d806c2b0 0x7f33d806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.529+0000 7f33e9ffb640 1 -- 172.21.15.100:0/2150884506 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f33e40abb00 con 0x7f33f41f46c0 2024-09-15T21:46:33.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.530+0000 7f33f909f640 1 --2- 172.21.15.100:0/2150884506 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f33d806c2b0 0x7f33d806e770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f33f0000fb0 tx=0x7f33f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:33.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.534+0000 7f33e9ffb640 1 -- 172.21.15.100:0/2150884506 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f33e407a1e0 con 0x7f33f41f46c0 2024-09-15T21:46:33.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.702+0000 7f33fb303640 1 -- 172.21.15.100:0/2150884506 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33f40caed0 con 0x7f33f41f46c0 2024-09-15T21:46:33.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.703+0000 7f33e9ffb640 1 -- 172.21.15.100:0/2150884506 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16737 (secure 0 0 0) 0x7f33e402adb0 con 0x7f33f41f46c0 2024-09-15T21:46:33.705 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:33.706 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"30e2fff4-2171-4531-8806-0e03b63f494f","created":"2024-09-15T21:45:59.746897+0000","modified":"2024-09-15T21:46:24.409383+0000","last_up_change":"2024-09-15T21:46:13.224709+0000","last_in_change":"2024-09-15T21:46:01.812129+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:46:11.837603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T21:46:20.577982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T21:46:21.729231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5887900b-522d-476f-9704-cc2c5ee87a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6819","nonce":3920435546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6821","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6822","nonce":3920435546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6829","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6831","nonce":3920435546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6825","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6827","nonce":3920435546}]},"public_addr":"172.21.15.100:6819/3920435546","cluster_addr":"172.21.15.100:6822/3920435546","heartbeat_back_addr":"172.21.15.100:6831/3920435546","heartbeat_front_addr":"172.21.15.100:6827/3920435546","state":["exists","up"]},{"osd":1,"uuid":"adc17993-5c50-4251-85e1-4c7ac62e25e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6801","nonce":3802679972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6803","nonce":3802679972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6807","nonce":3802679972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6805","nonce":3802679972}]},"public_addr":"172.21.15.100:6801/3802679972","cluster_addr":"172.21.15.100:6803/3802679972","heartbeat_back_addr":"172.21.15.100:6807/3802679972","heartbeat_front_addr":"172.21.15.100:6805/3802679972","state":["exists","up"]},{"osd":2,"uuid":"8de3b7d5-1980-4a3b-a934-264f867c7c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6823","nonce":2316787974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6826","nonce":2316787974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6833","nonce":2316787974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6830","nonce":2316787974}]},"public_addr":"172.21.15.100:6823/2316787974","cluster_addr":"172.21.15.100:6826/2316787974","heartbeat_back_addr":"172.21.15.100:6833/2316787974","heartbeat_front_addr":"172.21.15.100:6830/2316787974","state":["exists","up"]},{"osd":3,"uuid":"93dbcd1c-6a6f-49cf-bc94-e64b883de5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6811","nonce":1287562741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6813","nonce":1287562741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6817","nonce":1287562741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6815","nonce":1287562741}]},"public_addr":"172.21.15.100:6811/1287562741","cluster_addr":"172.21.15.100:6813/1287562741","heartbeat_back_addr":"172.21.15.100:6817/1287562741","heartbeat_front_addr":"172.21.15.100:6815/1287562741","state":["exists","up"]},{"osd":4,"uuid":"7caee2a0-bab4-45a9-8109-724948c863c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6801","nonce":4045445512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6803","nonce":4045445512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6807","nonce":4045445512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6805","nonce":4045445512}]},"public_addr":"172.21.15.122:6801/4045445512","cluster_addr":"172.21.15.122:6803/4045445512","heartbeat_back_addr":"172.21.15.122:6807/4045445512","heartbeat_front_addr":"172.21.15.122:6805/4045445512","state":["exists","up"]},{"osd":5,"uuid":"6e64070a-671a-444b-a061-b3051a29b121","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6825","nonce":2400668532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6827","nonce":2400668532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6831","nonce":2400668532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6829","nonce":2400668532}]},"public_addr":"172.21.15.122:6825/2400668532","cluster_addr":"172.21.15.122:6827/2400668532","heartbeat_back_addr":"172.21.15.122:6831/2400668532","heartbeat_front_addr":"172.21.15.122:6829/2400668532","state":["exists","up"]},{"osd":6,"uuid":"47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6809","nonce":1437167311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6811","nonce":1437167311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6815","nonce":1437167311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6813","nonce":1437167311}]},"public_addr":"172.21.15.122:6809/1437167311","cluster_addr":"172.21.15.122:6811/1437167311","heartbeat_back_addr":"172.21.15.122:6815/1437167311","heartbeat_front_addr":"172.21.15.122:6813/1437167311","state":["exists","up"]},{"osd":7,"uuid":"9a916b6d-af82-486f-af40-4a7291c900fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6817","nonce":2431377162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6819","nonce":2431377162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6823","nonce":2431377162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6821","nonce":2431377162}]},"public_addr":"172.21.15.122:6817/2431377162","cluster_addr":"172.21.15.122:6819/2431377162","heartbeat_back_addr":"172.21.15.122:6823/2431377162","heartbeat_front_addr":"172.21.15.122:6821/2431377162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.168771+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.886614+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.154129+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.570531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.926773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.146320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.409137+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.373359+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:46:33.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.705+0000 7f33fb303640 1 -- 172.21.15.100:0/2150884506 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f33d806c2b0 msgr2=0x7f33d806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:33.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.705+0000 7f33fb303640 1 --2- 172.21.15.100:0/2150884506 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f33d806c2b0 0x7f33d806e770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f33f0000fb0 tx=0x7f33f0009040 comp rx=0 tx=0).stop 2024-09-15T21:46:33.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.705+0000 7f33fb303640 1 -- 172.21.15.100:0/2150884506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33f41f46c0 msgr2=0x7f33f41f4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:33.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.705+0000 7f33fb303640 1 --2- 172.21.15.100:0/2150884506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33f41f46c0 0x7f33f41f4b00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f33e402a8d0 tx=0x7f33e40046c0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.705+0000 7f33fb303640 1 -- 172.21.15.100:0/2150884506 shutdown_connections 2024-09-15T21:46:33.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.705+0000 7f33fb303640 1 --2- 172.21.15.100:0/2150884506 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f33d806c2b0 0x7f33d806e770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.705+0000 7f33fb303640 1 --2- 172.21.15.100:0/2150884506 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33f40c7f70 0x7f33f40c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.705+0000 7f33fb303640 1 --2- 172.21.15.100:0/2150884506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33f41f46c0 0x7f33f41f4b00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.705+0000 7f33fb303640 1 --2- 172.21.15.100:0/2150884506 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f33e0004030 0x7f33f4155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.705+0000 7f33fb303640 1 -- 172.21.15.100:0/2150884506 >> 172.21.15.100:0/2150884506 conn(0x7f33f40680d0 msgr2=0x7f33f406d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:33.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.706+0000 7f33fb303640 1 -- 172.21.15.100:0/2150884506 shutdown_connections 2024-09-15T21:46:33.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.706+0000 7f33fb303640 1 -- 172.21.15.100:0/2150884506 wait complete. 2024-09-15T21:46:33.718 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T21:46:33.719 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:46:33.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.816+0000 7f51bdf86640 1 Processor -- start 2024-09-15T21:46:33.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.816+0000 7f51bdf86640 1 -- start start 2024-09-15T21:46:33.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.816+0000 7f51bdf86640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f51b80cb7d0 0x7f51b80cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.816+0000 7f51bdf86640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f51b806d0e0 0x7f51b80cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.816+0000 7f51bdf86640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51b815e7a0 con 0x7f51b8155440 2024-09-15T21:46:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.816+0000 7f51bdf86640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f51b815b470 con 0x7f51b80cb7d0 2024-09-15T21:46:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.816+0000 7f51bdf86640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f51b8155820 con 0x7f51b806d0e0 2024-09-15T21:46:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.817+0000 7f51b6ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f51b80cb7d0 0x7f51b80cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.817+0000 7f51b6ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f51b80cb7d0 0x7f51b80cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43794/0 (socket says 172.21.15.100:43794) 2024-09-15T21:46:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.817+0000 7f51b6ffd640 1 -- 172.21.15.100:0/1919455811 learned_addr learned my addr 172.21.15.100:0/1919455811 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.817+0000 7f51b7fff640 1 --2- 172.21.15.100:0/1919455811 >> v2:172.21.15.122:3300/0 conn(0x7f51b806d0e0 0x7f51b80cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.817+0000 7f51b6ffd640 1 -- 172.21.15.100:0/1919455811 >> v2:172.21.15.122:3300/0 conn(0x7f51b806d0e0 msgr2=0x7f51b80cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.817+0000 7f51b6ffd640 1 --2- 172.21.15.100:0/1919455811 >> v2:172.21.15.122:3300/0 conn(0x7f51b806d0e0 0x7f51b80cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.817+0000 7f51b6ffd640 1 -- 172.21.15.100:0/1919455811 >> v1:172.21.15.100:6789/0 conn(0x7f51b8155440 legacy=0x7f51b80cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.817+0000 7f51b6ffd640 1 -- 172.21.15.100:0/1919455811 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b815ea40 con 0x7f51b80cb7d0 2024-09-15T21:46:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.818+0000 7f51b6ffd640 1 --2- 172.21.15.100:0/1919455811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51b80cb7d0 0x7f51b80cc1e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f51b80ce140 tx=0x7f51a002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75599ce94734989d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.818+0000 7f51b67fc640 1 -- 172.21.15.100:0/1919455811 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0037070 con 0x7f51b80cb7d0 2024-09-15T21:46:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.818+0000 7f51b67fc640 1 -- 172.21.15.100:0/1919455811 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a0032440 con 0x7f51b80cb7d0 2024-09-15T21:46:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.818+0000 7f51b67fc640 1 -- 172.21.15.100:0/1919455811 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0032810 con 0x7f51b80cb7d0 2024-09-15T21:46:33.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.818+0000 7f51bdf86640 1 -- 172.21.15.100:0/1919455811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51b80cb7d0 msgr2=0x7f51b80cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:33.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.818+0000 7f51bdf86640 1 --2- 172.21.15.100:0/1919455811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51b80cb7d0 0x7f51b80cc1e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f51b80ce140 tx=0x7f51a002a1e0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.818+0000 7f51bdf86640 1 -- 172.21.15.100:0/1919455811 shutdown_connections 2024-09-15T21:46:33.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.818+0000 7f51bdf86640 1 --2- 172.21.15.100:0/1919455811 >> v2:172.21.15.122:3300/0 conn(0x7f51b806d0e0 0x7f51b80cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.818+0000 7f51bdf86640 1 --2- 172.21.15.100:0/1919455811 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51b80cb7d0 0x7f51b80cc1e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.818+0000 7f51bdf86640 1 -- 172.21.15.100:0/1919455811 >> 172.21.15.100:0/1919455811 conn(0x7f51b80680d0 msgr2=0x7f51b806d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:33.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.818+0000 7f51bdf86640 1 -- 172.21.15.100:0/1919455811 shutdown_connections 2024-09-15T21:46:33.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.818+0000 7f51bdf86640 1 -- 172.21.15.100:0/1919455811 wait complete. 2024-09-15T21:46:33.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.819+0000 7f51bdf86640 1 Processor -- start 2024-09-15T21:46:33.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.820+0000 7f51bdf86640 1 -- start start 2024-09-15T21:46:33.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.820+0000 7f51bdf86640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f51b806d0e0 0x7f51b815e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.820+0000 7f51bdf86640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51b80cb7d0 0x7f51b8158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.820+0000 7f51bdf86640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51b8155440 0x7f51b8153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.820+0000 7f51bdf86640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f51b815f2e0 con 0x7f51b80cb7d0 2024-09-15T21:46:33.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.820+0000 7f51bdf86640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f51b815f000 con 0x7f51b8155440 2024-09-15T21:46:33.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.820+0000 7f51bdf86640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f51b815b070 con 0x7f51b806d0e0 2024-09-15T21:46:33.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.820+0000 7f51b6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51b80cb7d0 0x7f51b8158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.820+0000 7f51b6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51b80cb7d0 0x7f51b8158cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43810/0 (socket says 172.21.15.100:43810) 2024-09-15T21:46:33.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.820+0000 7f51b6ffd640 1 -- 172.21.15.100:0/521769466 learned_addr learned my addr 172.21.15.100:0/521769466 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:33.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.820+0000 7f51b7fff640 1 --2- 172.21.15.100:0/521769466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51b8155440 0x7f51b8153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.820+0000 7f51b6ffd640 1 -- 172.21.15.100:0/521769466 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f51b806d0e0 msgr2=0x7f51b815e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:33.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.821+0000 7f51b6ffd640 1 --2- 172.21.15.100:0/521769466 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f51b806d0e0 0x7f51b815e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.821+0000 7f51b6ffd640 1 -- 172.21.15.100:0/521769466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51b8155440 msgr2=0x7f51b8153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:33.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.821+0000 7f51b6ffd640 1 --2- 172.21.15.100:0/521769466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51b8155440 0x7f51b8153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:33.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.821+0000 7f51b6ffd640 1 -- 172.21.15.100:0/521769466 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b8159f80 con 0x7f51b80cb7d0 2024-09-15T21:46:33.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.821+0000 7f51b7fff640 1 --2- 172.21.15.100:0/521769466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51b8155440 0x7f51b8153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:33.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.821+0000 7f51b6ffd640 1 --2- 172.21.15.100:0/521769466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51b80cb7d0 0x7f51b8158cd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f51a0000c00 tx=0x7f51a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:33.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.821+0000 7f51affff640 1 -- 172.21.15.100:0/521769466 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0037040 con 0x7f51b80cb7d0 2024-09-15T21:46:33.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.821+0000 7f51affff640 1 -- 172.21.15.100:0/521769466 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51a0042070 con 0x7f51b80cb7d0 2024-09-15T21:46:33.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.821+0000 7f51bdf86640 1 -- 172.21.15.100:0/521769466 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b815a7c0 con 0x7f51b80cb7d0 2024-09-15T21:46:33.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.821+0000 7f51affff640 1 -- 172.21.15.100:0/521769466 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0030880 con 0x7f51b80cb7d0 2024-09-15T21:46:33.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.821+0000 7f51bdf86640 1 -- 172.21.15.100:0/521769466 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b815ad60 con 0x7f51b80cb7d0 2024-09-15T21:46:33.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.822+0000 7f51bdf86640 1 -- 172.21.15.100:0/521769466 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b80769a0 con 0x7f51b80cb7d0 2024-09-15T21:46:33.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.822+0000 7f51affff640 1 -- 172.21.15.100:0/521769466 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f51a0030a20 con 0x7f51b80cb7d0 2024-09-15T21:46:33.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.823+0000 7f51affff640 1 --2- 172.21.15.100:0/521769466 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f519806c2e0 0x7f519806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:33.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.823+0000 7f51affff640 1 -- 172.21.15.100:0/521769466 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f51a00abc40 con 0x7f51b80cb7d0 2024-09-15T21:46:33.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.823+0000 7f51b77fe640 1 --2- 172.21.15.100:0/521769466 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f519806c2e0 0x7f519806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:33.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.823+0000 7f51b77fe640 1 --2- 172.21.15.100:0/521769466 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f519806c2e0 0x7f519806e7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f51a8000f60 tx=0x7f51a80079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:33.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.827+0000 7f51affff640 1 -- 172.21.15.100:0/521769466 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f51b80769a0 con 0x7f51b80cb7d0 2024-09-15T21:46:33.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.992+0000 7f51bdf86640 1 -- 172.21.15.100:0/521769466 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51b80b3940 con 0x7f51b80cb7d0 2024-09-15T21:46:34.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.994+0000 7f51affff640 1 -- 172.21.15.100:0/521769466 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16737 (secure 0 0 0) 0x7f51a007a440 con 0x7f51b80cb7d0 2024-09-15T21:46:34.042 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:34.042 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"30e2fff4-2171-4531-8806-0e03b63f494f","created":"2024-09-15T21:45:59.746897+0000","modified":"2024-09-15T21:46:24.409383+0000","last_up_change":"2024-09-15T21:46:13.224709+0000","last_in_change":"2024-09-15T21:46:01.812129+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:46:11.837603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T21:46:20.577982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T21:46:21.729231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5887900b-522d-476f-9704-cc2c5ee87a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6819","nonce":3920435546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6821","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6822","nonce":3920435546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6829","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6831","nonce":3920435546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6825","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6827","nonce":3920435546}]},"public_addr":"172.21.15.100:6819/3920435546","cluster_addr":"172.21.15.100:6822/3920435546","heartbeat_back_addr":"172.21.15.100:6831/3920435546","heartbeat_front_addr":"172.21.15.100:6827/3920435546","state":["exists","up"]},{"osd":1,"uuid":"adc17993-5c50-4251-85e1-4c7ac62e25e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6801","nonce":3802679972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6803","nonce":3802679972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6807","nonce":3802679972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6805","nonce":3802679972}]},"public_addr":"172.21.15.100:6801/3802679972","cluster_addr":"172.21.15.100:6803/3802679972","heartbeat_back_addr":"172.21.15.100:6807/3802679972","heartbeat_front_addr":"172.21.15.100:6805/3802679972","state":["exists","up"]},{"osd":2,"uuid":"8de3b7d5-1980-4a3b-a934-264f867c7c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6823","nonce":2316787974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6826","nonce":2316787974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6833","nonce":2316787974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6830","nonce":2316787974}]},"public_addr":"172.21.15.100:6823/2316787974","cluster_addr":"172.21.15.100:6826/2316787974","heartbeat_back_addr":"172.21.15.100:6833/2316787974","heartbeat_front_addr":"172.21.15.100:6830/2316787974","state":["exists","up"]},{"osd":3,"uuid":"93dbcd1c-6a6f-49cf-bc94-e64b883de5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6811","nonce":1287562741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6813","nonce":1287562741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6817","nonce":1287562741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6815","nonce":1287562741}]},"public_addr":"172.21.15.100:6811/1287562741","cluster_addr":"172.21.15.100:6813/1287562741","heartbeat_back_addr":"172.21.15.100:6817/1287562741","heartbeat_front_addr":"172.21.15.100:6815/1287562741","state":["exists","up"]},{"osd":4,"uuid":"7caee2a0-bab4-45a9-8109-724948c863c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6801","nonce":4045445512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6803","nonce":4045445512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6807","nonce":4045445512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6805","nonce":4045445512}]},"public_addr":"172.21.15.122:6801/4045445512","cluster_addr":"172.21.15.122:6803/4045445512","heartbeat_back_addr":"172.21.15.122:6807/4045445512","heartbeat_front_addr":"172.21.15.122:6805/4045445512","state":["exists","up"]},{"osd":5,"uuid":"6e64070a-671a-444b-a061-b3051a29b121","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6825","nonce":2400668532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6827","nonce":2400668532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6831","nonce":2400668532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6829","nonce":2400668532}]},"public_addr":"172.21.15.122:6825/2400668532","cluster_addr":"172.21.15.122:6827/2400668532","heartbeat_back_addr":"172.21.15.122:6831/2400668532","heartbeat_front_addr":"172.21.15.122:6829/2400668532","state":["exists","up"]},{"osd":6,"uuid":"47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6809","nonce":1437167311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6811","nonce":1437167311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6815","nonce":1437167311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6813","nonce":1437167311}]},"public_addr":"172.21.15.122:6809/1437167311","cluster_addr":"172.21.15.122:6811/1437167311","heartbeat_back_addr":"172.21.15.122:6815/1437167311","heartbeat_front_addr":"172.21.15.122:6813/1437167311","state":["exists","up"]},{"osd":7,"uuid":"9a916b6d-af82-486f-af40-4a7291c900fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6817","nonce":2431377162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6819","nonce":2431377162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6823","nonce":2431377162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6821","nonce":2431377162}]},"public_addr":"172.21.15.122:6817/2431377162","cluster_addr":"172.21.15.122:6819/2431377162","heartbeat_back_addr":"172.21.15.122:6823/2431377162","heartbeat_front_addr":"172.21.15.122:6821/2431377162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.168771+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.886614+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.154129+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.570531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.926773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.146320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.409137+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.373359+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:46:34.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.996+0000 7f51bdf86640 1 -- 172.21.15.100:0/521769466 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f519806c2e0 msgr2=0x7f519806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.996+0000 7f51bdf86640 1 --2- 172.21.15.100:0/521769466 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f519806c2e0 0x7f519806e7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f51a8000f60 tx=0x7f51a80079d0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.996+0000 7f51bdf86640 1 -- 172.21.15.100:0/521769466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51b80cb7d0 msgr2=0x7f51b8158cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.996+0000 7f51bdf86640 1 --2- 172.21.15.100:0/521769466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51b80cb7d0 0x7f51b8158cd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f51a0000c00 tx=0x7f51a00046c0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.996+0000 7f51bdf86640 1 -- 172.21.15.100:0/521769466 shutdown_connections 2024-09-15T21:46:34.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.996+0000 7f51bdf86640 1 --2- 172.21.15.100:0/521769466 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f519806c2e0 0x7f519806e7a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.996+0000 7f51bdf86640 1 --2- 172.21.15.100:0/521769466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51b8155440 0x7f51b8153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.996+0000 7f51bdf86640 1 --2- 172.21.15.100:0/521769466 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f51b80cb7d0 0x7f51b8158cd0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.996+0000 7f51bdf86640 1 --2- 172.21.15.100:0/521769466 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f51b806d0e0 0x7f51b815e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.996+0000 7f51bdf86640 1 -- 172.21.15.100:0/521769466 >> 172.21.15.100:0/521769466 conn(0x7f51b80680d0 msgr2=0x7f51b80c66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.996+0000 7f51bdf86640 1 -- 172.21.15.100:0/521769466 shutdown_connections 2024-09-15T21:46:34.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:33.996+0000 7f51bdf86640 1 -- 172.21.15.100:0/521769466 wait complete. 2024-09-15T21:46:34.050 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:46:34.050 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T21:46:34.051 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T21:46:34.051 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T21:46:34.051 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-15T21:46:34.052 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-15T21:46:34.053 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-15T21:46:34.053 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-15T21:46:34.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.216+0000 7feb0db24640 1 Processor -- start 2024-09-15T21:46:34.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.217+0000 7feb0db24640 1 -- start start 2024-09-15T21:46:34.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.218+0000 7feb0db24640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7feb080c9c30 0x7feb0806e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.218+0000 7feb0db24640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb080c32f0 0x7feb0806e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.218+0000 7feb0db24640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb08157f40 con 0x7feb080c3b10 2024-09-15T21:46:34.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.218+0000 7feb0db24640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7feb080c6f10 con 0x7feb080c9c30 2024-09-15T21:46:34.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.218+0000 7feb0db24640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7feb080c7160 con 0x7feb080c32f0 2024-09-15T21:46:34.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.218+0000 7f593e52c640 1 Processor -- start 2024-09-15T21:46:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.218+0000 7f212d731640 1 Processor -- start 2024-09-15T21:46:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.218+0000 7feb07fff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7feb080c3b10 0x7feb08154310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.100:46816/0 (socket says 172.21.15.100:46816) 2024-09-15T21:46:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.219+0000 7f212d731640 1 -- start start 2024-09-15T21:46:34.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.218+0000 7feb07fff640 1 -- 172.21.15.100:0/2539437434 learned_addr learned my addr 172.21.15.100:0/2539437434 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:34.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.219+0000 7f52ab59e640 1 Processor -- start 2024-09-15T21:46:34.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.218+0000 7feb0d323640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb080c32f0 0x7feb0806e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.218+0000 7f593e52c640 1 -- start start 2024-09-15T21:46:34.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f593e52c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f59380c8a60 0x7f5938160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f593e52c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f59380b27a0 0x7f5938159420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f593e52c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59380c36a0 con 0x7f59380c8e40 2024-09-15T21:46:34.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f593e52c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f59380c2b00 con 0x7f59380b27a0 2024-09-15T21:46:34.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f593e52c640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f59380c1f30 con 0x7f59380c8a60 2024-09-15T21:46:34.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f5937fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f59380c8a60 0x7f5938160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f5937fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f59380c8a60 0x7f5938160110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:40944/0 (socket says 172.21.15.100:40944) 2024-09-15T21:46:34.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f5937fff640 1 -- 172.21.15.100:0/2138927287 learned_addr learned my addr 172.21.15.100:0/2138927287 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.219+0000 7feb0d323640 1 -- 172.21.15.100:0/2539437434 >> v2:172.21.15.122:3300/0 conn(0x7feb080c9c30 msgr2=0x7feb0806e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:34.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.219+0000 7feb0d323640 1 --2- 172.21.15.100:0/2539437434 >> v2:172.21.15.122:3300/0 conn(0x7feb080c9c30 0x7feb0806e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.219+0000 7feb0d323640 1 -- 172.21.15.100:0/2539437434 >> v1:172.21.15.122:6789/0 conn(0x7feb080c3b10 legacy=0x7feb08154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.219+0000 7feb0d323640 1 -- 172.21.15.100:0/2539437434 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb080c7440 con 0x7feb080c32f0 2024-09-15T21:46:34.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7faa28b74640 1 Processor -- start 2024-09-15T21:46:34.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7faa28b74640 1 -- start start 2024-09-15T21:46:34.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.219+0000 7f52ab59e640 1 -- start start 2024-09-15T21:46:34.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f212d731640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f21280c8380 0x7f212815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f212d731640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21280ca570 0x7f2128158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f212d731640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21280c8870 con 0x7f21281518d0 2024-09-15T21:46:34.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f212d731640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f21280c8b50 con 0x7f21280c8380 2024-09-15T21:46:34.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f212d731640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f21280c8e30 con 0x7f21280ca570 2024-09-15T21:46:34.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f2126ffd640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f21281518d0 0x7f212806f330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.100:46828/0 (socket says 172.21.15.100:46828) 2024-09-15T21:46:34.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f2126ffd640 1 -- 172.21.15.100:0/934870200 learned_addr learned my addr 172.21.15.100:0/934870200 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:34.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f21267fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f21280c8380 0x7f212815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7f52ab59e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f52ac06e5c0 0x7f52ac154310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7f2125ffb640 1 -- 172.21.15.100:0/934870200 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1651317350 0 0) 0x7f21280c8870 con 0x7f21281518d0 2024-09-15T21:46:34.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7f2125ffb640 1 -- 172.21.15.100:0/934870200 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2110005730 con 0x7f21281518d0 2024-09-15T21:46:34.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7f21267fc640 1 -- 172.21.15.100:0/934870200 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21280ca570 msgr2=0x7f2128158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:34.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7f21277fe640 1 --2- 172.21.15.100:0/934870200 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21280ca570 0x7f2128158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7faa28b74640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7faa1c0d5580 0x7faa1c0059f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7f21267fc640 1 --2- 172.21.15.100:0/934870200 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21280ca570 0x7f2128158150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7f21267fc640 1 -- 172.21.15.100:0/934870200 >> v1:172.21.15.122:6789/0 conn(0x7f21281518d0 legacy=0x7f212806f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7feb0d323640 1 --2- 172.21.15.100:0/2539437434 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb080c32f0 0x7feb0806e890 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feb00009dc0 tx=0x7feb0002a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77eb2740ff115f0a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7faa28b74640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faa1c044fc0 0x7faa1c0461e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f593cac9640 1 --2- 172.21.15.100:0/2138927287 >> v2:172.21.15.100:3300/0 conn(0x7f59380b27a0 0x7f5938159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f5937fff640 1 -- 172.21.15.100:0/2138927287 >> v2:172.21.15.100:3300/0 conn(0x7f59380b27a0 msgr2=0x7f5938159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:34.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f5937fff640 1 --2- 172.21.15.100:0/2138927287 >> v2:172.21.15.100:3300/0 conn(0x7f59380b27a0 0x7f5938159420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f5937fff640 1 -- 172.21.15.100:0/2138927287 >> v1:172.21.15.122:6789/0 conn(0x7f59380c8e40 legacy=0x7f59380c2f90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T21:46:34.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.220+0000 7f5937fff640 1 -- 172.21.15.100:0/2138927287 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59380c21d0 con 0x7f59380c8a60 2024-09-15T21:46:34.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7faa28b74640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa1c034c30 con 0x7faa1c046720 2024-09-15T21:46:34.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7faa28b74640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7faa1c0d5960 con 0x7faa1c044fc0 2024-09-15T21:46:34.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7faa28b74640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7faa1c0357b0 con 0x7faa1c0d5580 2024-09-15T21:46:34.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7faa237fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7faa1c0d5580 0x7faa1c0059f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7faa22ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faa1c044fc0 0x7faa1c0461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7faa22ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faa1c044fc0 0x7faa1c0461e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43850/0 (socket says 172.21.15.100:43850) 2024-09-15T21:46:34.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7faa22ffd640 1 -- 172.21.15.100:0/2002170682 learned_addr learned my addr 172.21.15.100:0/2002170682 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7faa22ffd640 1 -- 172.21.15.100:0/2002170682 >> v2:172.21.15.122:3300/0 conn(0x7faa1c0d5580 msgr2=0x7faa1c0059f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:34.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7faa22ffd640 1 --2- 172.21.15.100:0/2002170682 >> v2:172.21.15.122:3300/0 conn(0x7faa1c0d5580 0x7faa1c0059f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7faa22ffd640 1 -- 172.21.15.100:0/2002170682 >> v1:172.21.15.100:6789/0 conn(0x7faa1c046720 legacy=0x7faa1c0db640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:34.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7faa22ffd640 1 -- 172.21.15.100:0/2002170682 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1c034f10 con 0x7faa1c044fc0 2024-09-15T21:46:34.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7f52aa59c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f52ac06e5c0 0x7f52ac154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7f52aa59c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f52ac06e5c0 0x7f52ac154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:40958/0 (socket says 172.21.15.100:40958) 2024-09-15T21:46:34.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7f52a9d9b640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f52ac06e9a0 0x7f52ac0c7c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.100:46840/0 (socket says 172.21.15.100:46840) 2024-09-15T21:46:34.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7f52ab59e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52ac0c3b10 0x7f52ac154850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7f52ab59e640 1 -- 172.21.15.100:0/2494718436 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52ac157ca0 con 0x7f52ac06e9a0 2024-09-15T21:46:34.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7f52ab59e640 1 -- 172.21.15.100:0/2494718436 --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f52ac0c6f10 con 0x7f52ac06e5c0 2024-09-15T21:46:34.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7f52ab59e640 1 -- 172.21.15.100:0/2494718436 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f52ac157f40 con 0x7f52ac0c3b10 2024-09-15T21:46:34.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7f52a9d9b640 1 -- 172.21.15.100:0/2494718436 learned_addr learned my addr 172.21.15.100:0/2494718436 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:34.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7f52aad9d640 1 --2- 172.21.15.100:0/2494718436 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52ac0c3b10 0x7f52ac154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.221+0000 7f21267fc640 1 -- 172.21.15.100:0/934870200 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2128157950 con 0x7f21280c8380 2024-09-15T21:46:34.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7f5937fff640 1 --2- 172.21.15.100:0/2138927287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59380c8a60 0x7f5938160110 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f59380c2c80 tx=0x7f593002a2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88da3d1f3c5edfe1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7feb077fe640 1 -- 172.21.15.100:0/2539437434 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00007740 con 0x7feb080c32f0 2024-09-15T21:46:34.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7feb077fe640 1 -- 172.21.15.100:0/2539437434 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb000078e0 con 0x7feb080c32f0 2024-09-15T21:46:34.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7feb077fe640 1 -- 172.21.15.100:0/2539437434 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00007cf0 con 0x7feb080c32f0 2024-09-15T21:46:34.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7f5936ffd640 1 -- 172.21.15.100:0/2138927287 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5930037070 con 0x7f59380c8a60 2024-09-15T21:46:34.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7f5936ffd640 1 -- 172.21.15.100:0/2138927287 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5930032440 con 0x7f59380c8a60 2024-09-15T21:46:34.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7f5936ffd640 1 -- 172.21.15.100:0/2138927287 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59300327f0 con 0x7f59380c8a60 2024-09-15T21:46:34.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7f52a959a640 1 -- 172.21.15.100:0/2494718436 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3869939220 0 0) 0x7f52ac157ca0 con 0x7f52ac06e9a0 2024-09-15T21:46:34.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7f52a959a640 1 -- 172.21.15.100:0/2494718436 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5294003520 con 0x7f52ac06e9a0 2024-09-15T21:46:34.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7f52a959a640 1 -- 172.21.15.100:0/2494718436 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2364902149 0 0) 0x7f5294003520 con 0x7f52ac06e9a0 2024-09-15T21:46:34.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7f52a959a640 1 -- 172.21.15.100:0/2494718436 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52ac0c3b10 msgr2=0x7f52ac154850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:34.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7f52a959a640 1 --2- 172.21.15.100:0/2494718436 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52ac0c3b10 0x7f52ac154850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7f52a959a640 1 -- 172.21.15.100:0/2494718436 >> v2:172.21.15.122:3300/0 conn(0x7f52ac06e5c0 msgr2=0x7f52ac154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:34.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7f52a959a640 1 --2- 172.21.15.100:0/2494718436 >> v2:172.21.15.122:3300/0 conn(0x7f52ac06e5c0 0x7f52ac154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7f52a959a640 1 -- 172.21.15.100:0/2494718436 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ac0c7160 con 0x7f52ac06e9a0 2024-09-15T21:46:34.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7f52a959a640 1 -- 172.21.15.100:0/2494718436 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7f52a0001670 con 0x7f52ac06e9a0 2024-09-15T21:46:34.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7f52a959a640 1 -- 172.21.15.100:0/2494718436 >> v1:172.21.15.122:6789/0 conn(0x7f52ac06e9a0 legacy=0x7f52ac0c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7f21267fc640 1 --2- 172.21.15.100:0/934870200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21280c8380 0x7f212815abd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f211c009a00 tx=0x7f211c02a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc02e6fa550bc1f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.222+0000 7faa22ffd640 1 --2- 172.21.15.100:0/2002170682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa1c044fc0 0x7faa1c0461e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0073f0 tx=0x7faa1002c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840250bb57fd5a01 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f52a959a640 1 --2- 172.21.15.100:0/2494718436 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5294003a20 0x7f5294003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f52a959a640 1 --2- 172.21.15.100:0/2494718436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52940043d0 0x7f52940048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7feb0db24640 1 -- 172.21.15.100:0/2539437434 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb080c32f0 msgr2=0x7feb0806e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7feb0db24640 1 --2- 172.21.15.100:0/2539437434 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb080c32f0 0x7feb0806e890 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7feb00009dc0 tx=0x7feb0002a6c0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f593e52c640 1 -- 172.21.15.100:0/2138927287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59380c8a60 msgr2=0x7f5938160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f593e52c640 1 --2- 172.21.15.100:0/2138927287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59380c8a60 0x7f5938160110 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f59380c2c80 tx=0x7f593002a2e0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f593e52c640 1 -- 172.21.15.100:0/2138927287 shutdown_connections 2024-09-15T21:46:34.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f593e52c640 1 --2- 172.21.15.100:0/2138927287 >> v2:172.21.15.100:3300/0 conn(0x7f59380b27a0 0x7f5938159420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f593e52c640 1 --2- 172.21.15.100:0/2138927287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59380c8a60 0x7f5938160110 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f593e52c640 1 -- 172.21.15.100:0/2138927287 >> 172.21.15.100:0/2138927287 conn(0x7f593806ed20 msgr2=0x7f593806f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f593e52c640 1 -- 172.21.15.100:0/2138927287 shutdown_connections 2024-09-15T21:46:34.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f593e52c640 1 -- 172.21.15.100:0/2138927287 wait complete. 2024-09-15T21:46:34.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f5ad5d98640 1 Processor -- start 2024-09-15T21:46:34.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7feb0db24640 1 -- 172.21.15.100:0/2539437434 shutdown_connections 2024-09-15T21:46:34.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7feb0db24640 1 --2- 172.21.15.100:0/2539437434 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb080c32f0 0x7feb0806e890 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7feb0db24640 1 --2- 172.21.15.100:0/2539437434 >> v2:172.21.15.122:3300/0 conn(0x7feb080c9c30 0x7feb0806e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7feb0db24640 1 -- 172.21.15.100:0/2539437434 >> 172.21.15.100:0/2539437434 conn(0x7feb080c9110 msgr2=0x7feb080c9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7feb0db24640 1 -- 172.21.15.100:0/2539437434 shutdown_connections 2024-09-15T21:46:34.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7feb0db24640 1 -- 172.21.15.100:0/2539437434 wait complete. 2024-09-15T21:46:34.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7faa227fc640 1 -- 172.21.15.100:0/2002170682 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10038070 con 0x7faa1c044fc0 2024-09-15T21:46:34.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7faa227fc640 1 -- 172.21.15.100:0/2002170682 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa10031710 con 0x7faa1c044fc0 2024-09-15T21:46:34.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7faa227fc640 1 -- 172.21.15.100:0/2002170682 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa10031ae0 con 0x7faa1c044fc0 2024-09-15T21:46:34.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f52a959a640 1 --2- 172.21.15.100:0/2494718436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5294004fb0 0x7f529400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f52a959a640 1 -- 172.21.15.100:0/2494718436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac0c6f10 con 0x7f5294004fb0 2024-09-15T21:46:34.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f52a959a640 1 -- 172.21.15.100:0/2494718436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac157f40 con 0x7f52940043d0 2024-09-15T21:46:34.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f52a959a640 1 -- 172.21.15.100:0/2494718436 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f529400bf50 con 0x7f5294003a20 2024-09-15T21:46:34.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f52aa59c640 1 --2- 172.21.15.100:0/2494718436 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5294003a20 0x7f5294003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f52a9d9b640 1 --2- 172.21.15.100:0/2494718436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52940043d0 0x7f52940048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f2125ffb640 1 -- 172.21.15.100:0/934870200 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c036070 con 0x7f21280c8380 2024-09-15T21:46:34.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f3ddafff640 1 Processor -- start 2024-09-15T21:46:34.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f52aad9d640 1 --2- 172.21.15.100:0/2494718436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5294004fb0 0x7f529400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f2125ffb640 1 -- 172.21.15.100:0/934870200 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f211c031600 con 0x7f21280c8380 2024-09-15T21:46:34.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.224+0000 7faa28b74640 1 -- 172.21.15.100:0/2002170682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa1c044fc0 msgr2=0x7faa1c0461e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.224+0000 7faa28b74640 1 --2- 172.21.15.100:0/2002170682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa1c044fc0 0x7faa1c0461e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0073f0 tx=0x7faa1002c4d0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.224+0000 7faa28b74640 1 -- 172.21.15.100:0/2002170682 shutdown_connections 2024-09-15T21:46:34.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.224+0000 7faa28b74640 1 --2- 172.21.15.100:0/2002170682 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa1c044fc0 0x7faa1c0461e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.224+0000 7faa28b74640 1 --2- 172.21.15.100:0/2002170682 >> v2:172.21.15.122:3300/0 conn(0x7faa1c0d5580 0x7faa1c0059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.224+0000 7faa28b74640 1 -- 172.21.15.100:0/2002170682 >> 172.21.15.100:0/2002170682 conn(0x7faa1c04fc40 msgr2=0x7faa1c050050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.224+0000 7faa28b74640 1 -- 172.21.15.100:0/2002170682 shutdown_connections 2024-09-15T21:46:34.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.224+0000 7faa28b74640 1 -- 172.21.15.100:0/2002170682 wait complete. 2024-09-15T21:46:34.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f593e52c640 1 Processor -- start 2024-09-15T21:46:34.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f593e52c640 1 -- start start 2024-09-15T21:46:34.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f5ad5d98640 1 -- start start 2024-09-15T21:46:34.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.224+0000 7f5ad5d98640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5ad00b2af0 0x7f5ad00c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.224+0000 7f5ad5d98640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ad00c3d00 0x7f5ad00c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.224+0000 7f5ad5d98640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ad0151860 con 0x7f5ad0151480 2024-09-15T21:46:34.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.224+0000 7f5ad5d98640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f5ad006e750 con 0x7f5ad00b2af0 2024-09-15T21:46:34.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f5ad5d98640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad0157a50 con 0x7f5ad00c3d00 2024-09-15T21:46:34.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f5acffff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ad00c3d00 0x7f5ad00c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f5acffff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ad00c3d00 0x7f5ad00c8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:55856/0 (socket says 172.21.15.100:55856) 2024-09-15T21:46:34.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f5acffff640 1 -- 172.21.15.100:0/4262132444 learned_addr learned my addr 172.21.15.100:0/4262132444 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f5acffff640 1 -- 172.21.15.100:0/4262132444 >> v2:172.21.15.100:3300/0 conn(0x7f5ad00b2af0 msgr2=0x7f5ad00c87f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:46:34.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f5acffff640 1 --2- 172.21.15.100:0/4262132444 >> v2:172.21.15.100:3300/0 conn(0x7f5ad00b2af0 0x7f5ad00c87f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f5acffff640 1 -- 172.21.15.100:0/4262132444 >> v1:172.21.15.122:6789/0 conn(0x7f5ad0151480 legacy=0x7f5ad015a640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:46:34.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f5acffff640 1 -- 172.21.15.100:0/4262132444 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad006c750 con 0x7f5ad00c3d00 2024-09-15T21:46:34.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7feb0db24640 1 Processor -- start 2024-09-15T21:46:34.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7feb0db24640 1 -- start start 2024-09-15T21:46:34.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.224+0000 7f52aad9d640 1 -- 172.21.15.100:0/2494718436 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5294003a20 msgr2=0x7f5294003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:34.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.224+0000 7f52aad9d640 1 --2- 172.21.15.100:0/2494718436 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5294003a20 0x7f5294003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.224+0000 7f52aad9d640 1 -- 172.21.15.100:0/2494718436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52940043d0 msgr2=0x7f52940048a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:34.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.224+0000 7f52aad9d640 1 --2- 172.21.15.100:0/2494718436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52940043d0 0x7f52940048a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.224+0000 7f52aad9d640 1 -- 172.21.15.100:0/2494718436 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5294003520 con 0x7f5294004fb0 2024-09-15T21:46:34.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f52aad9d640 1 --2- 172.21.15.100:0/2494718436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5294004fb0 0x7f529400b9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f52a400a3f0 tx=0x7f5294005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c45415c9c78e121 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.223+0000 7f3ddafff640 1 -- start start 2024-09-15T21:46:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f3ddafff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3dd41597b0 0x7f3dd40cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f3ddafff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f3dd406bbe0 0x7f3dd406c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f3ddafff640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3dd415fe80 con 0x7f3dd41593d0 2024-09-15T21:46:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f3ddafff640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f3dd4072740 con 0x7f3dd41597b0 2024-09-15T21:46:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f3ddafff640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f3dd4160160 con 0x7f3dd406bbe0 2024-09-15T21:46:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f3dd3fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3dd41597b0 0x7f3dd40cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f3dd3fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3dd41597b0 0x7f3dd40cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43866/0 (socket says 172.21.15.100:43866) 2024-09-15T21:46:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f3dd3fff640 1 -- 172.21.15.100:0/1217890171 learned_addr learned my addr 172.21.15.100:0/1217890171 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.225+0000 7f3dd959c640 1 --2- 172.21.15.100:0/1217890171 >> v2:172.21.15.122:3300/0 conn(0x7f3dd406bbe0 0x7f3dd406c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.224+0000 7f2125ffb640 1 -- 172.21.15.100:0/934870200 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f211c0319d0 con 0x7f21280c8380 2024-09-15T21:46:34.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.226+0000 7f212d731640 1 -- 172.21.15.100:0/934870200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21280c8380 msgr2=0x7f212815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.226+0000 7f52a959a640 1 -- 172.21.15.100:0/2494718436 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a403d070 con 0x7f5294004fb0 2024-09-15T21:46:34.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.226+0000 7f212d731640 1 --2- 172.21.15.100:0/934870200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21280c8380 0x7f212815abd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f211c009a00 tx=0x7f211c02a400 comp rx=0 tx=0).stop 2024-09-15T21:46:34.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.226+0000 7f52a959a640 1 -- 172.21.15.100:0/2494718436 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a4038410 con 0x7f5294004fb0 2024-09-15T21:46:34.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.226+0000 7f52a959a640 1 -- 172.21.15.100:0/2494718436 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a400a090 con 0x7f5294004fb0 2024-09-15T21:46:34.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.226+0000 7f3dd3fff640 1 -- 172.21.15.100:0/1217890171 >> v2:172.21.15.122:3300/0 conn(0x7f3dd406bbe0 msgr2=0x7f3dd406c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:34.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.226+0000 7f3dd3fff640 1 --2- 172.21.15.100:0/1217890171 >> v2:172.21.15.122:3300/0 conn(0x7f3dd406bbe0 0x7f3dd406c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.226+0000 7f3dd3fff640 1 -- 172.21.15.100:0/1217890171 >> v1:172.21.15.100:6789/0 conn(0x7f3dd41593d0 legacy=0x7f3dd40c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.226+0000 7f3dd37fe640 1 -- 172.21.15.100:0/1217890171 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1316155441 0 0) 0x7f3dd415fe80 con 0x7f3dd41593d0 2024-09-15T21:46:34.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.226+0000 7f3dd3fff640 1 -- 172.21.15.100:0/1217890171 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd4071e10 con 0x7f3dd41597b0 2024-09-15T21:46:34.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.226+0000 7faa28b74640 1 Processor -- start 2024-09-15T21:46:34.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.226+0000 7f5acffff640 1 --2- 172.21.15.100:0/4262132444 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ad00c3d00 0x7f5ad00c8d30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009e50 tx=0x7f5ac002a780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e60e1d39b928d29 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.226+0000 7faa28b74640 1 -- start start 2024-09-15T21:46:34.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.226+0000 7f593e52c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f59380b27a0 0x7f59380734f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f593e52c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59380c8a60 0x7f5938157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7feb0db24640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb080c32f0 0x7feb080c2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f5937fff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f59380b27a0 0x7f59380734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f5937fff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f59380b27a0 0x7f59380734f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:55864/0 (socket says 172.21.15.100:55864) 2024-09-15T21:46:34.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f5ace7fc640 1 -- 172.21.15.100:0/4262132444 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0036070 con 0x7f5ad00c3d00 2024-09-15T21:46:34.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f5ace7fc640 1 -- 172.21.15.100:0/4262132444 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac0031600 con 0x7f5ad00c3d00 2024-09-15T21:46:34.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f5ace7fc640 1 -- 172.21.15.100:0/4262132444 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac00319d0 con 0x7f5ad00c3d00 2024-09-15T21:46:34.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7feb0db24640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb080c3b10 0x7feb080c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7feb0db24640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb080c9c30 0x7feb08155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7feb0db24640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7feb08154ba0 con 0x7feb080c3b10 2024-09-15T21:46:34.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7feb0db24640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feb08154a20 con 0x7feb080c32f0 2024-09-15T21:46:34.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7feb0db24640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7feb080c7c40 con 0x7feb080c9c30 2024-09-15T21:46:34.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f593e52c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59380c8e40 0x7f5938158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f5937fff640 1 -- 172.21.15.100:0/1921581609 learned_addr learned my addr 172.21.15.100:0/1921581609 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f593e52c640 1 -- 172.21.15.100:0/1921581609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f59380b31c0 con 0x7f59380c8a60 2024-09-15T21:46:34.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f593e52c640 1 -- 172.21.15.100:0/1921581609 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f59380c2790 con 0x7f59380c8e40 2024-09-15T21:46:34.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f593e52c640 1 -- 172.21.15.100:0/1921581609 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5938160a90 con 0x7f59380b27a0 2024-09-15T21:46:34.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f212d731640 1 -- 172.21.15.100:0/934870200 shutdown_connections 2024-09-15T21:46:34.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f212d731640 1 --2- 172.21.15.100:0/934870200 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21280ca570 0x7f2128158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f212d731640 1 --2- 172.21.15.100:0/934870200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21280c8380 0x7f212815abd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f52ab59e640 1 -- 172.21.15.100:0/2494718436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5294004fb0 msgr2=0x7f529400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f52ab59e640 1 --2- 172.21.15.100:0/2494718436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5294004fb0 0x7f529400b9c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f52a400a3f0 tx=0x7f5294005830 comp rx=0 tx=0).stop 2024-09-15T21:46:34.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f212d731640 1 -- 172.21.15.100:0/934870200 >> 172.21.15.100:0/934870200 conn(0x7f2128073820 msgr2=0x7f2128073c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7feb0cb22640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb080c32f0 0x7feb080c2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7feb07fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb080c3b10 0x7feb080c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7feb0cb22640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb080c32f0 0x7feb080c2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:40980/0 (socket says 172.21.15.100:40980) 2024-09-15T21:46:34.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7feb0cb22640 1 -- 172.21.15.100:0/477731727 learned_addr learned my addr 172.21.15.100:0/477731727 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f59377fe640 1 --2- 172.21.15.100:0/1921581609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59380c8a60 0x7f5938157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7feb0d323640 1 --2- 172.21.15.100:0/477731727 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb080c9c30 0x7feb08155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f3dd3fff640 1 --2- 172.21.15.100:0/1217890171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dd41597b0 0x7f3dd40cdfb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40728c0 tx=0x7f3dc802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d1b691df0e43e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f52aad9d640 1 -- 172.21.15.100:0/2494718436 reap_dead start 2024-09-15T21:46:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f593cac9640 1 --2- 172.21.15.100:0/1921581609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59380c8e40 0x7f5938158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f59377fe640 1 -- 172.21.15.100:0/1921581609 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f59380b27a0 msgr2=0x7f59380734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f59377fe640 1 --2- 172.21.15.100:0/1921581609 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f59380b27a0 0x7f59380734f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f59377fe640 1 -- 172.21.15.100:0/1921581609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59380c8e40 msgr2=0x7f5938158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f59377fe640 1 --2- 172.21.15.100:0/1921581609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59380c8e40 0x7f5938158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f59377fe640 1 -- 172.21.15.100:0/1921581609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59380740f0 con 0x7f59380c8a60 2024-09-15T21:46:34.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7feb0cb22640 1 -- 172.21.15.100:0/477731727 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb080c9c30 msgr2=0x7feb08155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7feb0cb22640 1 --2- 172.21.15.100:0/477731727 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb080c9c30 0x7feb08155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7feb0cb22640 1 -- 172.21.15.100:0/477731727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb080c3b10 msgr2=0x7feb080c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7feb0cb22640 1 --2- 172.21.15.100:0/477731727 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb080c3b10 0x7feb080c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7feb0cb22640 1 -- 172.21.15.100:0/477731727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb081572b0 con 0x7feb080c32f0 2024-09-15T21:46:34.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7faa28b74640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa1c044fc0 0x7faa1c0cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7faa28b74640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa1c046720 0x7faa1c0cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f52ab59e640 1 -- 172.21.15.100:0/2494718436 shutdown_connections 2024-09-15T21:46:34.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.227+0000 7f52ab59e640 1 -- 172.21.15.100:0/2494718436 >> 172.21.15.100:0/2494718436 conn(0x7f52ac0c9110 msgr2=0x7f52ac0c9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f3dd37fe640 1 -- 172.21.15.100:0/1217890171 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8037070 con 0x7f3dd41597b0 2024-09-15T21:46:34.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f3dd37fe640 1 -- 172.21.15.100:0/1217890171 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dc802ae50 con 0x7f3dd41597b0 2024-09-15T21:46:34.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7faa82123640 1 Processor -- start 2024-09-15T21:46:34.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7faa82123640 1 -- start start 2024-09-15T21:46:34.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7faa28b74640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa1c0d5580 0x7faa1c0cb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7faa28b74640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c0354d0 con 0x7faa1c0d5580 2024-09-15T21:46:34.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7faa28b74640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c0060a0 con 0x7faa1c044fc0 2024-09-15T21:46:34.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7faa28b74640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7faa1c050980 con 0x7faa1c046720 2024-09-15T21:46:34.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f212d731640 1 -- 172.21.15.100:0/934870200 shutdown_connections 2024-09-15T21:46:34.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f212d731640 1 -- 172.21.15.100:0/934870200 wait complete. 2024-09-15T21:46:34.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f5ad5d98640 1 -- 172.21.15.100:0/4262132444 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ad00c3d00 msgr2=0x7f5ad00c8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f5ad5d98640 1 --2- 172.21.15.100:0/4262132444 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ad00c3d00 0x7f5ad00c8d30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009e50 tx=0x7f5ac002a780 comp rx=0 tx=0).stop 2024-09-15T21:46:34.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7feb0cb22640 1 --2- 172.21.15.100:0/477731727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb080c32f0 0x7feb080c2a10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feaf800b440 tx=0x7feaf800b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7faa237fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa1c044fc0 0x7faa1c0cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f52ab59e640 1 -- 172.21.15.100:0/2494718436 shutdown_connections 2024-09-15T21:46:34.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f3dd37fe640 1 -- 172.21.15.100:0/1217890171 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8037070 con 0x7f3dd41597b0 2024-09-15T21:46:34.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f52ab59e640 1 -- 172.21.15.100:0/2494718436 wait complete. 2024-09-15T21:46:34.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f5ad5d98640 1 -- 172.21.15.100:0/4262132444 shutdown_connections 2024-09-15T21:46:34.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7faa237fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa1c044fc0 0x7faa1c0cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:41002/0 (socket says 172.21.15.100:41002) 2024-09-15T21:46:34.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7faa237fe640 1 -- 172.21.15.100:0/1516136761 learned_addr learned my addr 172.21.15.100:0/1516136761 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7faa23fff640 1 --2- 172.21.15.100:0/1516136761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa1c0d5580 0x7faa1c0cb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f5ad5d98640 1 --2- 172.21.15.100:0/4262132444 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ad00c3d00 0x7f5ad00c8d30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f5ad5d98640 1 --2- 172.21.15.100:0/4262132444 >> v2:172.21.15.100:3300/0 conn(0x7f5ad00b2af0 0x7f5ad00c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f5ad5d98640 1 -- 172.21.15.100:0/4262132444 >> 172.21.15.100:0/4262132444 conn(0x7f5ad006ed20 msgr2=0x7f5ad006f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f5ad5d98640 1 -- 172.21.15.100:0/4262132444 shutdown_connections 2024-09-15T21:46:34.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f5ad5d98640 1 -- 172.21.15.100:0/4262132444 wait complete. 2024-09-15T21:46:34.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.228+0000 7f59377fe640 1 --2- 172.21.15.100:0/1921581609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59380c8a60 0x7f5938157b30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f592c00b3e0 tx=0x7f592c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.229+0000 7faa23fff640 1 -- 172.21.15.100:0/1516136761 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa1c046720 msgr2=0x7faa1c0cc550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T21:46:34.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.229+0000 7faa23fff640 1 --2- 172.21.15.100:0/1516136761 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa1c046720 0x7faa1c0cc550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.229+0000 7faa23fff640 1 -- 172.21.15.100:0/1516136761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa1c044fc0 msgr2=0x7faa1c0cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.229+0000 7faa23fff640 1 --2- 172.21.15.100:0/1516136761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa1c044fc0 0x7faa1c0cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.229+0000 7faa23fff640 1 -- 172.21.15.100:0/1516136761 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1c0da8b0 con 0x7faa1c0d5580 2024-09-15T21:46:34.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.229+0000 7faa237fe640 1 --2- 172.21.15.100:0/1516136761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa1c044fc0 0x7faa1c0cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:46:34.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.229+0000 7feb057fa640 1 -- 172.21.15.100:0/477731727 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8015020 con 0x7feb080c32f0 2024-09-15T21:46:34.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.229+0000 7feb0db24640 1 -- 172.21.15.100:0/477731727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb08155f00 con 0x7feb080c32f0 2024-09-15T21:46:34.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.229+0000 7faa82123640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7faa7c153040 0x7faa7c0c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.229+0000 7feb0db24640 1 -- 172.21.15.100:0/477731727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb081567d0 con 0x7feb080c32f0 2024-09-15T21:46:34.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.229+0000 7feb057fa640 1 -- 172.21.15.100:0/477731727 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaf8011890 con 0x7feb080c32f0 2024-09-15T21:46:34.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.229+0000 7faa23fff640 1 --2- 172.21.15.100:0/1516136761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa1c0d5580 0x7faa1c0cb040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faa24051790 tx=0x7faa24069310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.229+0000 7faa82123640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa7c0c4820 0x7faa7c0c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.229+0000 7faa82123640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faa7c0caec0 0x7faa7c0720c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.229+0000 7faa82123640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7faa7c073ca0 con 0x7faa7c0caec0 2024-09-15T21:46:34.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7faa82123640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7faa7c06f040 con 0x7faa7c153040 2024-09-15T21:46:34.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7faa82123640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7faa7c0c9c40 con 0x7faa7c0c4820 2024-09-15T21:46:34.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7feb057fa640 1 -- 172.21.15.100:0/477731727 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8011c80 con 0x7feb080c32f0 2024-09-15T21:46:34.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7faa0ffff640 1 -- 172.21.15.100:0/1516136761 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa240776c0 con 0x7faa1c0d5580 2024-09-15T21:46:34.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7faa0ffff640 1 -- 172.21.15.100:0/1516136761 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa24077860 con 0x7faa1c0d5580 2024-09-15T21:46:34.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7faa0ffff640 1 -- 172.21.15.100:0/1516136761 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa24095640 con 0x7faa1c0d5580 2024-09-15T21:46:34.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7f3ddafff640 1 -- 172.21.15.100:0/1217890171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dd41597b0 msgr2=0x7f3dd40cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7f3ddafff640 1 --2- 172.21.15.100:0/1217890171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dd41597b0 0x7f3dd40cdfb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3dd40728c0 tx=0x7f3dc802a300 comp rx=0 tx=0).stop 2024-09-15T21:46:34.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7f5ad5d98640 1 Processor -- start 2024-09-15T21:46:34.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7faa7bfff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faa7c0caec0 0x7faa7c0720c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7faa7bfff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7faa7c0caec0 0x7faa7c0720c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43900/0 (socket says 172.21.15.100:43900) 2024-09-15T21:46:34.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7faa7bfff640 1 -- 172.21.15.100:0/1440977218 learned_addr learned my addr 172.21.15.100:0/1440977218 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7f5934ff9640 1 -- 172.21.15.100:0/1921581609 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c015020 con 0x7f59380c8a60 2024-09-15T21:46:34.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7f5934ff9640 1 -- 172.21.15.100:0/1921581609 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f592c011830 con 0x7f59380c8a60 2024-09-15T21:46:34.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7feb0db24640 1 -- 172.21.15.100:0/477731727 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feb08154310 con 0x7feb080c32f0 2024-09-15T21:46:34.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7faa7b7fe640 1 --2- 172.21.15.100:0/1440977218 >> v2:172.21.15.122:3300/0 conn(0x7faa7c153040 0x7faa7c0c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7faa7affd640 1 --2- 172.21.15.100:0/1440977218 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa7c0c4820 0x7faa7c0c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7faa7bfff640 1 -- 172.21.15.100:0/1440977218 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa7c0c4820 msgr2=0x7faa7c0c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:34.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7faa28b74640 1 -- 172.21.15.100:0/1516136761 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1c0da280 con 0x7faa1c0d5580 2024-09-15T21:46:34.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7f52ab59e640 1 Processor -- start 2024-09-15T21:46:34.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7f212d731640 1 Processor -- start 2024-09-15T21:46:34.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7faa7bfff640 1 --2- 172.21.15.100:0/1440977218 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa7c0c4820 0x7faa7c0c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7faa7bfff640 1 -- 172.21.15.100:0/1440977218 >> v2:172.21.15.122:3300/0 conn(0x7faa7c153040 msgr2=0x7faa7c0c6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:34.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7faa7bfff640 1 --2- 172.21.15.100:0/1440977218 >> v2:172.21.15.122:3300/0 conn(0x7faa7c153040 0x7faa7c0c6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7faa7bfff640 1 -- 172.21.15.100:0/1440977218 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7c0c9f20 con 0x7faa7c0caec0 2024-09-15T21:46:34.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7f52ab59e640 1 -- start start 2024-09-15T21:46:34.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.231+0000 7f212d731640 1 -- start start 2024-09-15T21:46:34.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.231+0000 7faa28b74640 1 -- 172.21.15.100:0/1516136761 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa1c0d9fa0 con 0x7faa1c0d5580 2024-09-15T21:46:34.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.231+0000 7f5934ff9640 1 -- 172.21.15.100:0/1921581609 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c011c00 con 0x7f59380c8a60 2024-09-15T21:46:34.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.231+0000 7f593e52c640 1 -- 172.21.15.100:0/1921581609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5938073b30 con 0x7f59380c8a60 2024-09-15T21:46:34.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.230+0000 7f5ad5d98640 1 -- start start 2024-09-15T21:46:34.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.231+0000 7f5ad5d98640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ad00b2af0 0x7f5ad01f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.231+0000 7f5ad5d98640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ad00c3d00 0x7f5ad0150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.231+0000 7f5ad5d98640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ad0151480 0x7f5ad0164c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.231+0000 7faa7b7fe640 1 --2- 172.21.15.100:0/1440977218 >> v2:172.21.15.122:3300/0 conn(0x7faa7c153040 0x7faa7c0c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:34.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.231+0000 7faa7affd640 1 --2- 172.21.15.100:0/1440977218 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa7c0c4820 0x7faa7c0c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:34.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f5ad5d98640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad01576e0 con 0x7f5ad00c3d00 2024-09-15T21:46:34.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f3ddafff640 1 -- 172.21.15.100:0/1217890171 shutdown_connections 2024-09-15T21:46:34.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f3ddafff640 1 --2- 172.21.15.100:0/1217890171 >> v2:172.21.15.122:3300/0 conn(0x7f3dd406bbe0 0x7f3dd406c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f3ddafff640 1 --2- 172.21.15.100:0/1217890171 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dd41597b0 0x7f3dd40cdfb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f3ddafff640 1 -- 172.21.15.100:0/1217890171 >> 172.21.15.100:0/1217890171 conn(0x7f3dd406ccf0 msgr2=0x7f3dd406d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f5ad5d98640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad0157400 con 0x7f5ad00b2af0 2024-09-15T21:46:34.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f5ad5d98640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad01570d0 con 0x7f5ad0151480 2024-09-15T21:46:34.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7faa28b74640 1 -- 172.21.15.100:0/1516136761 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faa1c0d9a70 con 0x7faa1c0d5580 2024-09-15T21:46:34.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f593e52c640 1 -- 172.21.15.100:0/1921581609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59380cacf0 con 0x7f59380c8a60 2024-09-15T21:46:34.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f5acffff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ad0151480 0x7f5ad0164c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f5acffff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ad0151480 0x7f5ad0164c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:55902/0 (socket says 172.21.15.100:55902) 2024-09-15T21:46:34.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f5acffff640 1 -- 172.21.15.100:0/3262587184 learned_addr learned my addr 172.21.15.100:0/3262587184 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f5aceffd640 1 --2- 172.21.15.100:0/3262587184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ad00c3d00 0x7f5ad0150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f212d731640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21280c8380 0x7f21281f0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f5acf7fe640 1 --2- 172.21.15.100:0/3262587184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ad00b2af0 0x7f5ad01f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f212d731640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21280ca570 0x7f2128156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f3ddafff640 1 -- 172.21.15.100:0/1217890171 shutdown_connections 2024-09-15T21:46:34.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f212d731640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21281518d0 0x7f21280cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f2126ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21280c8380 0x7f21281f0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f2126ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21280c8380 0x7f21281f0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:41034/0 (socket says 172.21.15.100:41034) 2024-09-15T21:46:34.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.231+0000 7f52ab59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52ac159330 0x7f52ac155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.231+0000 7f52ab59e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52ac159710 0x7f52ac14b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.231+0000 7f52ab59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52ac14bd80 0x7f52ac1eff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.231+0000 7f52ab59e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac1585f0 con 0x7f52ac159330 2024-09-15T21:46:34.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.231+0000 7f52ab59e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac0c9c30 con 0x7f52ac14bd80 2024-09-15T21:46:34.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f52ab59e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f52ac155ca0 con 0x7f52ac159710 2024-09-15T21:46:34.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f52aa59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52ac159330 0x7f52ac155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f52aa59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52ac159330 0x7f52ac155fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43908/0 (socket says 172.21.15.100:43908) 2024-09-15T21:46:34.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f52aa59c640 1 -- 172.21.15.100:0/661448237 learned_addr learned my addr 172.21.15.100:0/661448237 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f52a9d9b640 1 --2- 172.21.15.100:0/661448237 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52ac159710 0x7f52ac14b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f52aad9d640 1 --2- 172.21.15.100:0/661448237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52ac14bd80 0x7f52ac1eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f52aad9d640 1 -- 172.21.15.100:0/661448237 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52ac159710 msgr2=0x7f52ac14b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f52aad9d640 1 --2- 172.21.15.100:0/661448237 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f52ac159710 0x7f52ac14b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f52aad9d640 1 -- 172.21.15.100:0/661448237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52ac159330 msgr2=0x7f52ac155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f52aad9d640 1 --2- 172.21.15.100:0/661448237 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f52ac159330 0x7f52ac155fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f52aad9d640 1 -- 172.21.15.100:0/661448237 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ac156c00 con 0x7f52ac14bd80 2024-09-15T21:46:34.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7feb057fa640 1 -- 172.21.15.100:0/477731727 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7feaf8018020 con 0x7feb080c32f0 2024-09-15T21:46:34.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f21277fe640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21281518d0 0x7f21280cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f2126ffd640 1 -- 172.21.15.100:0/2284045277 learned_addr learned my addr 172.21.15.100:0/2284045277 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f212d731640 1 -- 172.21.15.100:0/2284045277 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2128151cb0 con 0x7f21280ca570 2024-09-15T21:46:34.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f3ddafff640 1 -- 172.21.15.100:0/1217890171 wait complete. 2024-09-15T21:46:34.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f52aad9d640 1 --2- 172.21.15.100:0/661448237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52ac14bd80 0x7f52ac1eff00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f52940057a0 tx=0x7f52a4004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f212d731640 1 -- 172.21.15.100:0/2284045277 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2128157600 con 0x7f21280c8380 2024-09-15T21:46:34.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f212d731640 1 -- 172.21.15.100:0/2284045277 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f212806fa40 con 0x7f21281518d0 2024-09-15T21:46:34.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f593e52c640 1 -- 172.21.15.100:0/1921581609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f59380cded0 con 0x7f59380c8a60 2024-09-15T21:46:34.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7faa7bfff640 1 --2- 172.21.15.100:0/1440977218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa7c0caec0 0x7faa7c0720c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faa7c075a70 tx=0x7faa7402a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6533966ba6e42da0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f21267fc640 1 --2- 172.21.15.100:0/2284045277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21280ca570 0x7f2128156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7feb057fa640 1 --2- 172.21.15.100:0/477731727 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7feaf406c300 0x7feaf406e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f529affd640 1 -- 172.21.15.100:0/661448237 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a403d040 con 0x7f52ac14bd80 2024-09-15T21:46:34.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f529affd640 1 -- 172.21.15.100:0/661448237 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a4044660 con 0x7f52ac14bd80 2024-09-15T21:46:34.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f529affd640 1 -- 172.21.15.100:0/661448237 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a400a090 con 0x7f52ac14bd80 2024-09-15T21:46:34.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7feb057fa640 1 -- 172.21.15.100:0/477731727 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7feaf808b5d0 con 0x7feb080c32f0 2024-09-15T21:46:34.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f21277fe640 1 -- 172.21.15.100:0/2284045277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21280c8380 msgr2=0x7f21281f0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f21277fe640 1 --2- 172.21.15.100:0/2284045277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21280c8380 0x7f21281f0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f21277fe640 1 -- 172.21.15.100:0/2284045277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21280ca570 msgr2=0x7f2128156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f21277fe640 1 --2- 172.21.15.100:0/2284045277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21280ca570 0x7f2128156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f21277fe640 1 -- 172.21.15.100:0/2284045277 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21281f0ff0 con 0x7f21281518d0 2024-09-15T21:46:34.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7feb057fa640 1 --2- 172.21.15.100:0/477731727 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7feaf4074580 0x7feaf40769a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7feb057fa640 1 -- 172.21.15.100:0/477731727 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feaf8021e70 con 0x7feaf4074580 2024-09-15T21:46:34.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7feb057fa640 1 -- 172.21.15.100:0/477731727 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7feaf808b980 con 0x7feb080c32f0 2024-09-15T21:46:34.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f52ab59e640 1 -- 172.21.15.100:0/661448237 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52ac156ee0 con 0x7f52ac14bd80 2024-09-15T21:46:34.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.234+0000 7feb07fff640 1 --2- 172.21.15.100:0/477731727 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7feaf406c300 0x7feaf406e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f52ab59e640 1 -- 172.21.15.100:0/661448237 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52ac1ed330 con 0x7f52ac14bd80 2024-09-15T21:46:34.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.234+0000 7f21277fe640 1 --2- 172.21.15.100:0/2284045277 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21281518d0 0x7f21280cc780 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f212000cce0 tx=0x7f212000a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f5acf7fe640 1 -- 172.21.15.100:0/3262587184 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ad0151480 msgr2=0x7f5ad0164c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f5acf7fe640 1 --2- 172.21.15.100:0/3262587184 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ad0151480 0x7f5ad0164c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f5acf7fe640 1 -- 172.21.15.100:0/3262587184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ad00c3d00 msgr2=0x7f5ad0150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f5acf7fe640 1 --2- 172.21.15.100:0/3262587184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ad00c3d00 0x7f5ad0150200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.232+0000 7f5acf7fe640 1 -- 172.21.15.100:0/3262587184 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad0159ff0 con 0x7f5ad00b2af0 2024-09-15T21:46:34.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f5acf7fe640 1 --2- 172.21.15.100:0/3262587184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ad00b2af0 0x7f5ad01f6040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800b410 tx=0x7f5ac800b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f5ab7fff640 1 -- 172.21.15.100:0/3262587184 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8015020 con 0x7f5ad00b2af0 2024-09-15T21:46:34.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f5ab7fff640 1 -- 172.21.15.100:0/3262587184 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac8011850 con 0x7f5ad00b2af0 2024-09-15T21:46:34.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.233+0000 7f5ab7fff640 1 -- 172.21.15.100:0/3262587184 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac8011c40 con 0x7f5ad00b2af0 2024-09-15T21:46:34.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.234+0000 7f210f7fe640 1 -- 172.21.15.100:0/2284045277 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120003b90 con 0x7f21281518d0 2024-09-15T21:46:34.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.234+0000 7f210f7fe640 1 -- 172.21.15.100:0/2284045277 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2120003d30 con 0x7f21281518d0 2024-09-15T21:46:34.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.234+0000 7feb07fff640 1 --2- 172.21.15.100:0/477731727 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7feaf406c300 0x7feaf406e7c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feb08146ea0 tx=0x7feafc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.234+0000 7f212d731640 1 -- 172.21.15.100:0/2284045277 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21280cbcc0 con 0x7f21281518d0 2024-09-15T21:46:34.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.234+0000 7f52ab59e640 1 -- 172.21.15.100:0/661448237 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f52ac1ed570 con 0x7f52ac14bd80 2024-09-15T21:46:34.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.234+0000 7f210f7fe640 1 -- 172.21.15.100:0/2284045277 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120004240 con 0x7f21281518d0 2024-09-15T21:46:34.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.234+0000 7f212d731640 1 -- 172.21.15.100:0/2284045277 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f21281e2700 con 0x7f21281518d0 2024-09-15T21:46:34.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.234+0000 7faa7a7fc640 1 -- 172.21.15.100:0/1440977218 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa74036070 con 0x7faa7c0caec0 2024-09-15T21:46:34.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.234+0000 7faa7a7fc640 1 -- 172.21.15.100:0/1440977218 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa74031600 con 0x7faa7c0caec0 2024-09-15T21:46:34.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.234+0000 7faa7a7fc640 1 -- 172.21.15.100:0/1440977218 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa74031a10 con 0x7faa7c0caec0 2024-09-15T21:46:34.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.235+0000 7f5934ff9640 1 -- 172.21.15.100:0/1921581609 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f592c018020 con 0x7f59380c8a60 2024-09-15T21:46:34.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.235+0000 7f5ad5d98640 1 -- 172.21.15.100:0/3262587184 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad0154540 con 0x7f5ad00b2af0 2024-09-15T21:46:34.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.235+0000 7f5ad5d98640 1 -- 172.21.15.100:0/3262587184 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad0164140 con 0x7f5ad00b2af0 2024-09-15T21:46:34.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.235+0000 7feb0d323640 1 --2- 172.21.15.100:0/477731727 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7feaf4074580 0x7feaf40769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.235+0000 7faa0ffff640 1 -- 172.21.15.100:0/1516136761 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7faa240957e0 con 0x7faa1c0d5580 2024-09-15T21:46:34.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.235+0000 7faa82123640 1 -- 172.21.15.100:0/1440977218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa7c0caec0 msgr2=0x7faa7c0720c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.235+0000 7faa82123640 1 --2- 172.21.15.100:0/1440977218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa7c0caec0 0x7faa7c0720c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7faa7c075a70 tx=0x7faa7402a760 comp rx=0 tx=0).stop 2024-09-15T21:46:34.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.236+0000 7faa82123640 1 -- 172.21.15.100:0/1440977218 shutdown_connections 2024-09-15T21:46:34.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.236+0000 7feb0d323640 1 --2- 172.21.15.100:0/477731727 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7feaf4074580 0x7feaf40769a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.236+0000 7faa82123640 1 --2- 172.21.15.100:0/1440977218 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa7c0c4820 0x7faa7c0c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.236+0000 7faa82123640 1 --2- 172.21.15.100:0/1440977218 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa7c0caec0 0x7faa7c0720c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.236+0000 7faa82123640 1 --2- 172.21.15.100:0/1440977218 >> v2:172.21.15.122:3300/0 conn(0x7faa7c153040 0x7faa7c0c6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.236+0000 7faa82123640 1 -- 172.21.15.100:0/1440977218 >> 172.21.15.100:0/1440977218 conn(0x7faa7c0ca390 msgr2=0x7faa7c0ca770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.236+0000 7faa82123640 1 -- 172.21.15.100:0/1440977218 shutdown_connections 2024-09-15T21:46:34.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.236+0000 7faa0ffff640 1 --2- 172.21.15.100:0/1516136761 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7faa0806c380 0x7faa0806e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.236+0000 7faa82123640 1 -- 172.21.15.100:0/1440977218 wait complete. 2024-09-15T21:46:34.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.236+0000 7faa237fe640 1 --2- 172.21.15.100:0/1516136761 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7faa0806c380 0x7faa0806e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.236+0000 7faa0ffff640 1 -- 172.21.15.100:0/1516136761 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7faa24108160 con 0x7faa1c0d5580 2024-09-15T21:46:34.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.236+0000 7faa0ffff640 1 --2- 172.21.15.100:0/1516136761 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7faa08074600 0x7faa08076a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.236+0000 7f212d731640 1 -- 172.21.15.100:0/2284045277 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2128155fb0 con 0x7f21281518d0 2024-09-15T21:46:34.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.236+0000 7f5934ff9640 1 --2- 172.21.15.100:0/1921581609 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f592406c380 0x7f592406e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.236+0000 7faa0ffff640 1 -- 172.21.15.100:0/1516136761 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faa2408ee50 con 0x7faa08074600 2024-09-15T21:46:34.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.236+0000 7faa0ffff640 1 -- 172.21.15.100:0/1516136761 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7faa24108550 con 0x7faa1c0d5580 2024-09-15T21:46:34.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.236+0000 7faa22ffd640 1 --2- 172.21.15.100:0/1516136761 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7faa08074600 0x7faa08076a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.236+0000 7f529affd640 1 -- 172.21.15.100:0/661448237 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f52a4040040 con 0x7f52ac14bd80 2024-09-15T21:46:34.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.236+0000 7f529affd640 1 --2- 172.21.15.100:0/661448237 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f528c06c380 0x7f528c06e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.237+0000 7f529affd640 1 -- 172.21.15.100:0/661448237 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f52a40af970 con 0x7f52ac14bd80 2024-09-15T21:46:34.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.237+0000 7f529affd640 1 --2- 172.21.15.100:0/661448237 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f528c074600 0x7f528c076a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.237+0000 7faa237fe640 1 --2- 172.21.15.100:0/1516136761 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7faa0806c380 0x7faa0806e840 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa18007e50 tx=0x7faa18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.237+0000 7f529affd640 1 -- 172.21.15.100:0/661448237 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f52a403fe50 con 0x7f528c074600 2024-09-15T21:46:34.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.237+0000 7f529affd640 1 -- 172.21.15.100:0/661448237 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f52a40afd60 con 0x7f52ac14bd80 2024-09-15T21:46:34.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.237+0000 7feb057fa640 1 -- 172.21.15.100:0/477731727 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feaf8021e70 con 0x7feaf4074580 2024-09-15T21:46:34.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.237+0000 7f5ad5d98640 1 -- 172.21.15.100:0/3262587184 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ad0155e60 con 0x7f5ad00b2af0 2024-09-15T21:46:34.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.237+0000 7f210f7fe640 1 -- 172.21.15.100:0/2284045277 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f21200043e0 con 0x7f21281518d0 2024-09-15T21:46:34.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.237+0000 7faa22ffd640 1 --2- 172.21.15.100:0/1516136761 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7faa08074600 0x7faa08076a20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.237+0000 7f52aa59c640 1 --2- 172.21.15.100:0/661448237 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f528c06c380 0x7f528c06e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.237+0000 7f5937fff640 1 --2- 172.21.15.100:0/1921581609 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f592406c380 0x7f592406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.237+0000 7f5934ff9640 1 -- 172.21.15.100:0/1921581609 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f592c08b9a0 con 0x7f59380c8a60 2024-09-15T21:46:34.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.237+0000 7f5ab7fff640 1 -- 172.21.15.100:0/3262587184 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5ac8018020 con 0x7f5ad00b2af0 2024-09-15T21:46:34.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7f210f7fe640 1 --2- 172.21.15.100:0/2284045277 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f211406c2b0 0x7f211406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.237+0000 7f5934ff9640 1 --2- 172.21.15.100:0/1921581609 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f5924074600 0x7f5924076a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.237+0000 7f5934ff9640 1 -- 172.21.15.100:0/1921581609 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f592c006730 con 0x7f5924074600 2024-09-15T21:46:34.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.237+0000 7f5934ff9640 1 -- 172.21.15.100:0/1921581609 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f592c08bd90 con 0x7f59380c8a60 2024-09-15T21:46:34.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7faa82123640 1 Processor -- start 2024-09-15T21:46:34.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7f210f7fe640 1 -- 172.21.15.100:0/2284045277 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f212008be30 con 0x7f21281518d0 2024-09-15T21:46:34.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7f52aa59c640 1 --2- 172.21.15.100:0/661448237 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f528c06c380 0x7f528c06e840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f529c009cc0 tx=0x7f529c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7f5ab7fff640 1 --2- 172.21.15.100:0/3262587184 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5abc06c300 0x7f5abc06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7faa82123640 1 -- start start 2024-09-15T21:46:34.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7f210f7fe640 1 --2- 172.21.15.100:0/2284045277 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f21140744b0 0x7f21140768d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7f210f7fe640 1 -- 172.21.15.100:0/2284045277 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f212001be50 con 0x7f21140744b0 2024-09-15T21:46:34.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7f210f7fe640 1 -- 172.21.15.100:0/2284045277 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f212008e490 con 0x7f21281518d0 2024-09-15T21:46:34.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7f593cac9640 1 --2- 172.21.15.100:0/1921581609 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f5924074600 0x7f5924076a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7faa0ffff640 1 -- 172.21.15.100:0/1516136761 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faa2408ee50 con 0x7faa08074600 2024-09-15T21:46:34.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7f3ddafff640 1 Processor -- start 2024-09-15T21:46:34.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7f3ddafff640 1 -- start start 2024-09-15T21:46:34.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7f5ab7fff640 1 -- 172.21.15.100:0/3262587184 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5ac8011de0 con 0x7f5ad00b2af0 2024-09-15T21:46:34.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7f5ab7fff640 1 --2- 172.21.15.100:0/3262587184 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f5abc075120 0x7f5abc0775c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7f5ab7fff640 1 -- 172.21.15.100:0/3262587184 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5abc077c90 con 0x7f5abc075120 2024-09-15T21:46:34.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7f52a9d9b640 1 --2- 172.21.15.100:0/661448237 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f528c074600 0x7f528c076a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7f21267fc640 1 --2- 172.21.15.100:0/2284045277 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f21140744b0 0x7f21140768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7f5acffff640 1 --2- 172.21.15.100:0/3262587184 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f5abc075120 0x7f5abc0775c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.238+0000 7f593cac9640 1 --2- 172.21.15.100:0/1921581609 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f5924074600 0x7f5924076a20 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.239+0000 7f5ab7fff640 1 -- 172.21.15.100:0/3262587184 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f5ac8055a20 con 0x7f5ad00b2af0 2024-09-15T21:46:34.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.239+0000 7f5937fff640 1 --2- 172.21.15.100:0/1921581609 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f592406c380 0x7f592406e840 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5930009630 tx=0x7f593002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.239+0000 7f5aceffd640 1 --2- 172.21.15.100:0/3262587184 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5abc06c300 0x7f5abc06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.239+0000 7faa82123640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa7c0c4820 0x7faa7c1f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.239+0000 7faa82123640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa7c0caec0 0x7faa7c0c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.239+0000 7f3ddafff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dd406bbe0 0x7f3dd40cccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.239+0000 7f3ddafff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dd41593d0 0x7f3dd4148630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.239+0000 7f3ddafff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3dd40c9680 0x7f3dd4148b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.239+0000 7f3ddafff640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd40c8120 con 0x7f3dd406bbe0 2024-09-15T21:46:34.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.239+0000 7f3ddafff640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd40723d0 con 0x7f3dd41593d0 2024-09-15T21:46:34.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.239+0000 7f3ddafff640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3dd4072bd0 con 0x7f3dd40c9680 2024-09-15T21:46:34.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.239+0000 7f3dd8d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dd406bbe0 0x7f3dd40cccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7f3dd959c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3dd40c9680 0x7f3dd4148b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7f3dd959c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3dd40c9680 0x7f3dd4148b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:55932/0 (socket says 172.21.15.100:55932) 2024-09-15T21:46:34.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7f3dd959c640 1 -- 172.21.15.100:0/2649457792 learned_addr learned my addr 172.21.15.100:0/2649457792 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7faa7affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa7c0caec0 0x7faa7c0c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7f52a9d9b640 1 --2- 172.21.15.100:0/661448237 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f528c074600 0x7f528c076a20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7f3dd3fff640 1 --2- 172.21.15.100:0/2649457792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dd41593d0 0x7f3dd4148630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7f5934ff9640 1 -- 172.21.15.100:0/1921581609 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f592c006730 con 0x7f5924074600 2024-09-15T21:46:34.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.239+0000 7f2126ffd640 1 --2- 172.21.15.100:0/2284045277 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f211406c2b0 0x7f211406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7f2126ffd640 1 --2- 172.21.15.100:0/2284045277 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f211406c2b0 0x7f211406e770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2118005880 tx=0x7f2118009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7f21267fc640 1 --2- 172.21.15.100:0/2284045277 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f21140744b0 0x7f21140768d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7f3dd959c640 1 -- 172.21.15.100:0/2649457792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dd41593d0 msgr2=0x7f3dd4148630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.239+0000 7f5acffff640 1 --2- 172.21.15.100:0/3262587184 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f5abc075120 0x7f5abc0775c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7f3dd959c640 1 --2- 172.21.15.100:0/2649457792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3dd41593d0 0x7f3dd4148630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7f3dd959c640 1 -- 172.21.15.100:0/2649457792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dd406bbe0 msgr2=0x7f3dd40cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7f5aceffd640 1 --2- 172.21.15.100:0/3262587184 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5abc06c300 0x7f5abc06e7c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0073570 tx=0x7f5ac40090e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7f5ab7fff640 1 -- 172.21.15.100:0/3262587184 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5abc077c90 con 0x7f5abc075120 2024-09-15T21:46:34.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7faa7affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa7c0caec0 0x7faa7c0c76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43948/0 (socket says 172.21.15.100:43948) 2024-09-15T21:46:34.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7faa7b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa7c0c4820 0x7faa7c1f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7faa82123640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa7c153040 0x7faa7c1f0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7faa7b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa7c0c4820 0x7faa7c1f4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:41040/0 (socket says 172.21.15.100:41040) 2024-09-15T21:46:34.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7faa82123640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c073930 con 0x7faa7c0caec0 2024-09-15T21:46:34.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7faa82123640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faa7c073650 con 0x7faa7c0c4820 2024-09-15T21:46:34.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7faa82123640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7faa7c153420 con 0x7faa7c153040 2024-09-15T21:46:34.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7faa7affd640 1 -- 172.21.15.100:0/2705235507 learned_addr learned my addr 172.21.15.100:0/2705235507 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7faa7bfff640 1 --2- 172.21.15.100:0/2705235507 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa7c153040 0x7faa7c1f0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7faa7bfff640 1 -- 172.21.15.100:0/2705235507 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa7c0c4820 msgr2=0x7faa7c1f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7faa7bfff640 1 --2- 172.21.15.100:0/2705235507 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faa7c0c4820 0x7faa7c1f4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7faa7bfff640 1 -- 172.21.15.100:0/2705235507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa7c0caec0 msgr2=0x7faa7c0c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7f3dd959c640 1 --2- 172.21.15.100:0/2649457792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dd406bbe0 0x7f3dd40cccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.240+0000 7f3dd959c640 1 -- 172.21.15.100:0/2649457792 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd40c9db0 con 0x7f3dd40c9680 2024-09-15T21:46:34.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.241+0000 7f3dd8d9b640 1 --2- 172.21.15.100:0/2649457792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3dd406bbe0 0x7f3dd40cccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:34.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.241+0000 7faa7bfff640 1 --2- 172.21.15.100:0/2705235507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa7c0caec0 0x7faa7c0c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.241+0000 7faa7bfff640 1 -- 172.21.15.100:0/2705235507 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7c156460 con 0x7faa7c153040 2024-09-15T21:46:34.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.241+0000 7f529affd640 1 -- 172.21.15.100:0/661448237 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f52a403fe50 con 0x7f528c074600 2024-09-15T21:46:34.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.241+0000 7f3dd959c640 1 --2- 172.21.15.100:0/2649457792 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3dd40c9680 0x7f3dd4148b70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00efc0 tx=0x7f3dcc00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.241+0000 7f3dd17fa640 1 -- 172.21.15.100:0/2649457792 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc018020 con 0x7f3dd40c9680 2024-09-15T21:46:34.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.241+0000 7f3ddafff640 1 -- 172.21.15.100:0/2649457792 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd40ca090 con 0x7f3dd40c9680 2024-09-15T21:46:34.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.242+0000 7f3ddafff640 1 -- 172.21.15.100:0/2649457792 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd40ca650 con 0x7f3dd40c9680 2024-09-15T21:46:34.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.242+0000 7f3dd17fa640 1 -- 172.21.15.100:0/2649457792 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dcc014480 con 0x7f3dd40c9680 2024-09-15T21:46:34.326 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.242+0000 7f3dd17fa640 1 -- 172.21.15.100:0/2649457792 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc014870 con 0x7f3dd40c9680 2024-09-15T21:46:34.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.241+0000 7faa7affd640 1 --2- 172.21.15.100:0/2705235507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa7c0caec0 0x7faa7c0c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:46:34.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.241+0000 7faa7bfff640 1 --2- 172.21.15.100:0/2705235507 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa7c153040 0x7faa7c1f0bb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faa74030000 tx=0x7faa74004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.241+0000 7faa63fff640 1 -- 172.21.15.100:0/2705235507 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa74052070 con 0x7faa7c153040 2024-09-15T21:46:34.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.241+0000 7faa63fff640 1 -- 172.21.15.100:0/2705235507 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa74008920 con 0x7faa7c153040 2024-09-15T21:46:34.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.241+0000 7faa82123640 1 -- 172.21.15.100:0/2705235507 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa7c156740 con 0x7faa7c153040 2024-09-15T21:46:34.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.241+0000 7faa63fff640 1 -- 172.21.15.100:0/2705235507 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa74036040 con 0x7faa7c153040 2024-09-15T21:46:34.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.241+0000 7faa82123640 1 -- 172.21.15.100:0/2705235507 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faa7c1f44a0 con 0x7faa7c153040 2024-09-15T21:46:34.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.243+0000 7faa82123640 1 -- 172.21.15.100:0/2705235507 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faa7c06dcd0 con 0x7faa7c153040 2024-09-15T21:46:34.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.244+0000 7f3ddafff640 1 -- 172.21.15.100:0/2649457792 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3dd40cdfb0 con 0x7f3dd40c9680 2024-09-15T21:46:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.242+0000 7f210f7fe640 1 -- 172.21.15.100:0/2284045277 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f212001be50 con 0x7f21140744b0 2024-09-15T21:46:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.244+0000 7faa63fff640 1 -- 172.21.15.100:0/2705235507 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7faa74008cf0 con 0x7faa7c153040 2024-09-15T21:46:34.328 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.244+0000 7faa63fff640 1 --2- 172.21.15.100:0/2705235507 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7faa6806c2b0 0x7faa6806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.245+0000 7faa63fff640 1 -- 172.21.15.100:0/2705235507 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7faa740bce10 con 0x7faa7c153040 2024-09-15T21:46:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.245+0000 7faa63fff640 1 --2- 172.21.15.100:0/2705235507 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7faa680744b0 0x7faa680768d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.245+0000 7faa7b7fe640 1 --2- 172.21.15.100:0/2705235507 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7faa6806c2b0 0x7faa6806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.245+0000 7faa63fff640 1 -- 172.21.15.100:0/2705235507 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faa74031600 con 0x7faa680744b0 2024-09-15T21:46:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.245+0000 7faa63fff640 1 -- 172.21.15.100:0/2705235507 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7faa740bf510 con 0x7faa7c153040 2024-09-15T21:46:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.245+0000 7faa7affd640 1 --2- 172.21.15.100:0/2705235507 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7faa680744b0 0x7faa680768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.245+0000 7faa7b7fe640 1 --2- 172.21.15.100:0/2705235507 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7faa6806c2b0 0x7faa6806e770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00ab60 tx=0x7faa6c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.246+0000 7faa7affd640 1 --2- 172.21.15.100:0/2705235507 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7faa680744b0 0x7faa680768d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.329 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.246+0000 7f3dd17fa640 1 -- 172.21.15.100:0/2649457792 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3dcc01b020 con 0x7f3dd40c9680 2024-09-15T21:46:34.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.246+0000 7f3dd17fa640 1 --2- 172.21.15.100:0/2649457792 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3dc006c380 0x7f3dc006e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.330 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.247+0000 7faa63fff640 1 -- 172.21.15.100:0/2705235507 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faa74031600 con 0x7faa680744b0 2024-09-15T21:46:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.247+0000 7f3dd17fa640 1 -- 172.21.15.100:0/2649457792 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f3dcc08bf80 con 0x7f3dd40c9680 2024-09-15T21:46:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.247+0000 7f3dd17fa640 1 --2- 172.21.15.100:0/2649457792 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f3dc0074600 0x7f3dc0076a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.247+0000 7f3dd17fa640 1 -- 172.21.15.100:0/2649457792 --> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3dc00770d0 con 0x7f3dc0074600 2024-09-15T21:46:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.247+0000 7f3dd17fa640 1 -- 172.21.15.100:0/2649457792 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7f3dcc08e5e0 con 0x7f3dd40c9680 2024-09-15T21:46:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.247+0000 7f3dd8d9b640 1 --2- 172.21.15.100:0/2649457792 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3dc006c380 0x7f3dc006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.247+0000 7f3dd3fff640 1 --2- 172.21.15.100:0/2649457792 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f3dc0074600 0x7f3dc0076a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.248+0000 7f3dd3fff640 1 --2- 172.21.15.100:0/2649457792 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f3dc0074600 0x7f3dc0076a20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.331 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.248+0000 7f3dd8d9b640 1 --2- 172.21.15.100:0/2649457792 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3dc006c380 0x7f3dc006e840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4000f60 tx=0x7f3dc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.249+0000 7f3dd17fa640 1 -- 172.21.15.100:0/2649457792 <== osd.5 v2:172.21.15.122:6824/2400668532 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3dc00770d0 con 0x7f3dc0074600 2024-09-15T21:46:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.260+0000 7f5ad5d98640 1 -- 172.21.15.100:0/3262587184 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ad00c3ae0 con 0x7f5abc075120 2024-09-15T21:46:34.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.260+0000 7f5ab7fff640 1 -- 172.21.15.100:0/3262587184 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5ad00c3ae0 con 0x7f5abc075120 2024-09-15T21:46:34.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.261+0000 7f5ad5d98640 1 -- 172.21.15.100:0/3262587184 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f5abc075120 msgr2=0x7f5abc0775c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.261+0000 7f5ad5d98640 1 --2- 172.21.15.100:0/3262587184 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f5abc075120 0x7f5abc0775c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.261+0000 7f5ad5d98640 1 -- 172.21.15.100:0/3262587184 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5abc06c300 msgr2=0x7f5abc06e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.261+0000 7f5ad5d98640 1 --2- 172.21.15.100:0/3262587184 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5abc06c300 0x7f5abc06e7c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0073570 tx=0x7f5ac40090e0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.262+0000 7f5ad5d98640 1 -- 172.21.15.100:0/3262587184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ad00b2af0 msgr2=0x7f5ad01f6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.262+0000 7f5ad5d98640 1 --2- 172.21.15.100:0/3262587184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ad00b2af0 0x7f5ad01f6040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800b410 tx=0x7f5ac800b8d0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.262+0000 7f5acffff640 1 -- 172.21.15.100:0/3262587184 reap_dead start 2024-09-15T21:46:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.262+0000 7f5ad5d98640 1 -- 172.21.15.100:0/3262587184 shutdown_connections 2024-09-15T21:46:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.262+0000 7f5ad5d98640 1 -- 172.21.15.100:0/3262587184 >> 172.21.15.100:0/3262587184 conn(0x7f5ad006ed20 msgr2=0x7f5ad0155650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.262+0000 7f5ad5d98640 1 -- 172.21.15.100:0/3262587184 shutdown_connections 2024-09-15T21:46:34.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.262+0000 7f5ad5d98640 1 -- 172.21.15.100:0/3262587184 wait complete. 2024-09-15T21:46:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.267+0000 7faa28b74640 1 -- 172.21.15.100:0/1516136761 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faa1c0079e0 con 0x7faa08074600 2024-09-15T21:46:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.268+0000 7faa0ffff640 1 -- 172.21.15.100:0/1516136761 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faa1c0079e0 con 0x7faa08074600 2024-09-15T21:46:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.269+0000 7faa0dffb640 1 -- 172.21.15.100:0/1516136761 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7faa08074600 msgr2=0x7faa08076a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.269+0000 7faa0dffb640 1 --2- 172.21.15.100:0/1516136761 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7faa08074600 0x7faa08076a20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.269+0000 7faa0dffb640 1 -- 172.21.15.100:0/1516136761 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7faa0806c380 msgr2=0x7faa0806e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.269+0000 7faa0dffb640 1 --2- 172.21.15.100:0/1516136761 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7faa0806c380 0x7faa0806e840 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faa18007e50 tx=0x7faa18008040 comp rx=0 tx=0).stop 2024-09-15T21:46:34.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.269+0000 7feb0db24640 1 -- 172.21.15.100:0/477731727 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feb0806cb20 con 0x7feaf4074580 2024-09-15T21:46:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.269+0000 7faa0dffb640 1 -- 172.21.15.100:0/1516136761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa1c0d5580 msgr2=0x7faa1c0cb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.269+0000 7faa0dffb640 1 --2- 172.21.15.100:0/1516136761 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faa1c0d5580 0x7faa1c0cb040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7faa24051790 tx=0x7faa24069310 comp rx=0 tx=0).stop 2024-09-15T21:46:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.269+0000 7faa23fff640 1 -- 172.21.15.100:0/1516136761 reap_dead start 2024-09-15T21:46:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.269+0000 7faa0dffb640 1 -- 172.21.15.100:0/1516136761 shutdown_connections 2024-09-15T21:46:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.269+0000 7faa0dffb640 1 -- 172.21.15.100:0/1516136761 >> 172.21.15.100:0/1516136761 conn(0x7faa1c04fc40 msgr2=0x7faa1c04e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.269+0000 7feb057fa640 1 -- 172.21.15.100:0/477731727 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7feb0806cb20 con 0x7feaf4074580 2024-09-15T21:46:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.269+0000 7faa0dffb640 1 -- 172.21.15.100:0/1516136761 shutdown_connections 2024-09-15T21:46:34.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.270+0000 7faa0dffb640 1 -- 172.21.15.100:0/1516136761 wait complete. 2024-09-15T21:46:34.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.270+0000 7feaeeffd640 1 -- 172.21.15.100:0/477731727 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7feaf4074580 msgr2=0x7feaf40769a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.270+0000 7feaeeffd640 1 --2- 172.21.15.100:0/477731727 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7feaf4074580 0x7feaf40769a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.270+0000 7feaeeffd640 1 -- 172.21.15.100:0/477731727 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7feaf406c300 msgr2=0x7feaf406e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.271+0000 7feaeeffd640 1 --2- 172.21.15.100:0/477731727 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7feaf406c300 0x7feaf406e7c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feb08146ea0 tx=0x7feafc009040 comp rx=0 tx=0).stop 2024-09-15T21:46:34.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.271+0000 7feaeeffd640 1 -- 172.21.15.100:0/477731727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb080c32f0 msgr2=0x7feb080c2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.271+0000 7feaeeffd640 1 --2- 172.21.15.100:0/477731727 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb080c32f0 0x7feb080c2a10 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7feaf800b440 tx=0x7feaf800b900 comp rx=0 tx=0).stop 2024-09-15T21:46:34.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.271+0000 7feb0d323640 1 -- 172.21.15.100:0/477731727 reap_dead start 2024-09-15T21:46:34.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.271+0000 7feaeeffd640 1 -- 172.21.15.100:0/477731727 shutdown_connections 2024-09-15T21:46:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.271+0000 7feaeeffd640 1 -- 172.21.15.100:0/477731727 >> 172.21.15.100:0/477731727 conn(0x7feb080c9110 msgr2=0x7feb080ca570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.271+0000 7feaeeffd640 1 -- 172.21.15.100:0/477731727 shutdown_connections 2024-09-15T21:46:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.271+0000 7f52ab59e640 1 -- 172.21.15.100:0/661448237 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f52ac1ed570 con 0x7f528c074600 2024-09-15T21:46:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.271+0000 7feaeeffd640 1 -- 172.21.15.100:0/477731727 wait complete. 2024-09-15T21:46:34.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.272+0000 7f529affd640 1 -- 172.21.15.100:0/661448237 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f52a00018d0 con 0x7f528c074600 2024-09-15T21:46:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.272+0000 7f52ab59e640 1 -- 172.21.15.100:0/661448237 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f528c074600 msgr2=0x7f528c076a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.272+0000 7f52ab59e640 1 --2- 172.21.15.100:0/661448237 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f528c074600 0x7f528c076a20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.273+0000 7f52ab59e640 1 -- 172.21.15.100:0/661448237 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f528c06c380 msgr2=0x7f528c06e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.273+0000 7f52ab59e640 1 --2- 172.21.15.100:0/661448237 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f528c06c380 0x7f528c06e840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f529c009cc0 tx=0x7f529c009190 comp rx=0 tx=0).stop 2024-09-15T21:46:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.273+0000 7f52ab59e640 1 -- 172.21.15.100:0/661448237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52ac14bd80 msgr2=0x7f52ac1eff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.273+0000 7f52ab59e640 1 --2- 172.21.15.100:0/661448237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52ac14bd80 0x7f52ac1eff00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f52940057a0 tx=0x7f52a4004ab0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.273+0000 7f52aad9d640 1 -- 172.21.15.100:0/661448237 reap_dead start 2024-09-15T21:46:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.273+0000 7f52ab59e640 1 -- 172.21.15.100:0/661448237 shutdown_connections 2024-09-15T21:46:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.273+0000 7f52ab59e640 1 -- 172.21.15.100:0/661448237 >> 172.21.15.100:0/661448237 conn(0x7f52ac0c9110 msgr2=0x7f52ac152a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.273+0000 7f593e52c640 1 -- 172.21.15.100:0/1921581609 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f59380cded0 con 0x7f5924074600 2024-09-15T21:46:34.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.273+0000 7f5934ff9640 1 -- 172.21.15.100:0/1921581609 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f59380cded0 con 0x7f5924074600 2024-09-15T21:46:34.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.274+0000 7f593e52c640 1 -- 172.21.15.100:0/1921581609 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f5924074600 msgr2=0x7f5924076a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.274+0000 7f593e52c640 1 --2- 172.21.15.100:0/1921581609 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f5924074600 0x7f5924076a20 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.274+0000 7f593e52c640 1 -- 172.21.15.100:0/1921581609 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f592406c380 msgr2=0x7f592406e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.274+0000 7f593e52c640 1 --2- 172.21.15.100:0/1921581609 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f592406c380 0x7f592406e840 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5930009630 tx=0x7f593002e000 comp rx=0 tx=0).stop 2024-09-15T21:46:34.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.274+0000 7f593e52c640 1 -- 172.21.15.100:0/1921581609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59380c8a60 msgr2=0x7f5938157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.274+0000 7f593e52c640 1 --2- 172.21.15.100:0/1921581609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59380c8a60 0x7f5938157b30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f592c00b3e0 tx=0x7f592c00b8a0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.274+0000 7f593cac9640 1 -- 172.21.15.100:0/1921581609 reap_dead start 2024-09-15T21:46:34.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.274+0000 7f593e52c640 1 -- 172.21.15.100:0/1921581609 shutdown_connections 2024-09-15T21:46:34.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.274+0000 7f593e52c640 1 -- 172.21.15.100:0/1921581609 >> 172.21.15.100:0/1921581609 conn(0x7f593806ed20 msgr2=0x7f5938161b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.274+0000 7f212d731640 1 -- 172.21.15.100:0/2284045277 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f21280cc5d0 con 0x7f21140744b0 2024-09-15T21:46:34.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.275+0000 7f593e52c640 1 -- 172.21.15.100:0/1921581609 shutdown_connections 2024-09-15T21:46:34.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.275+0000 7f52ab59e640 1 -- 172.21.15.100:0/661448237 shutdown_connections 2024-09-15T21:46:34.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.275+0000 7f52ab59e640 1 -- 172.21.15.100:0/661448237 wait complete. 2024-09-15T21:46:34.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.275+0000 7f593e52c640 1 -- 172.21.15.100:0/1921581609 wait complete. 2024-09-15T21:46:34.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.275+0000 7f210f7fe640 1 -- 172.21.15.100:0/2284045277 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f21280cc5d0 con 0x7f21140744b0 2024-09-15T21:46:34.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.276+0000 7f212d731640 1 -- 172.21.15.100:0/2284045277 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f21140744b0 msgr2=0x7f21140768d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.276+0000 7f212d731640 1 --2- 172.21.15.100:0/2284045277 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f21140744b0 0x7f21140768d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.276+0000 7f212d731640 1 -- 172.21.15.100:0/2284045277 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f211406c2b0 msgr2=0x7f211406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.276+0000 7f212d731640 1 --2- 172.21.15.100:0/2284045277 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f211406c2b0 0x7f211406e770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2118005880 tx=0x7f2118009040 comp rx=0 tx=0).stop 2024-09-15T21:46:34.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.276+0000 7f212d731640 1 -- 172.21.15.100:0/2284045277 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21281518d0 msgr2=0x7f21280cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.276+0000 7f212d731640 1 --2- 172.21.15.100:0/2284045277 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21281518d0 0x7f21280cc780 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f212000cce0 tx=0x7f212000a660 comp rx=0 tx=0).stop 2024-09-15T21:46:34.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.276+0000 7f21277fe640 1 -- 172.21.15.100:0/2284045277 reap_dead start 2024-09-15T21:46:34.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.276+0000 7f212d731640 1 -- 172.21.15.100:0/2284045277 shutdown_connections 2024-09-15T21:46:34.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.276+0000 7f212d731640 1 -- 172.21.15.100:0/2284045277 >> 172.21.15.100:0/2284045277 conn(0x7f2128073820 msgr2=0x7f212806f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.277+0000 7f212d731640 1 -- 172.21.15.100:0/2284045277 shutdown_connections 2024-09-15T21:46:34.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.277+0000 7f212d731640 1 -- 172.21.15.100:0/2284045277 wait complete. 2024-09-15T21:46:34.347 INFO:teuthology.orchestra.run.smithi100.stdout:60129542151 2024-09-15T21:46:34.348 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:46:34.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.278+0000 7f3ddafff640 1 -- 172.21.15.100:0/2649457792 --> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3dd41f55a0 con 0x7f3dc0074600 2024-09-15T21:46:34.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.280+0000 7f3dd17fa640 1 -- 172.21.15.100:0/2649457792 <== osd.5 v2:172.21.15.122:6824/2400668532 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3dd41f55a0 con 0x7f3dc0074600 2024-09-15T21:46:34.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.280+0000 7faa82123640 1 -- 172.21.15.100:0/2705235507 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faa7c153f70 con 0x7faa680744b0 2024-09-15T21:46:34.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.281+0000 7faa63fff640 1 -- 172.21.15.100:0/2705235507 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faa7c153f70 con 0x7faa680744b0 2024-09-15T21:46:34.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.281+0000 7f3ddafff640 1 -- 172.21.15.100:0/2649457792 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f3dc0074600 msgr2=0x7f3dc0076a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.281+0000 7f3ddafff640 1 --2- 172.21.15.100:0/2649457792 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f3dc0074600 0x7f3dc0076a20 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.281+0000 7f3ddafff640 1 -- 172.21.15.100:0/2649457792 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3dc006c380 msgr2=0x7f3dc006e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.281+0000 7f3ddafff640 1 --2- 172.21.15.100:0/2649457792 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3dc006c380 0x7f3dc006e840 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4000f60 tx=0x7f3dc4009040 comp rx=0 tx=0).stop 2024-09-15T21:46:34.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.281+0000 7f3ddafff640 1 -- 172.21.15.100:0/2649457792 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3dd40c9680 msgr2=0x7f3dd4148b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.281+0000 7f3ddafff640 1 --2- 172.21.15.100:0/2649457792 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3dd40c9680 0x7f3dd4148b70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00efc0 tx=0x7f3dcc00c490 comp rx=0 tx=0).stop 2024-09-15T21:46:34.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.281+0000 7f3dd959c640 1 -- 172.21.15.100:0/2649457792 reap_dead start 2024-09-15T21:46:34.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.281+0000 7faa82123640 1 -- 172.21.15.100:0/2705235507 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7faa680744b0 msgr2=0x7faa680768d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.281+0000 7faa82123640 1 --2- 172.21.15.100:0/2705235507 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7faa680744b0 0x7faa680768d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.281+0000 7f3ddafff640 1 -- 172.21.15.100:0/2649457792 shutdown_connections 2024-09-15T21:46:34.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.281+0000 7f3ddafff640 1 -- 172.21.15.100:0/2649457792 >> 172.21.15.100:0/2649457792 conn(0x7f3dd406ccf0 msgr2=0x7f3dd4156bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.281+0000 7faa82123640 1 -- 172.21.15.100:0/2705235507 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7faa6806c2b0 msgr2=0x7faa6806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.281+0000 7faa82123640 1 --2- 172.21.15.100:0/2705235507 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7faa6806c2b0 0x7faa6806e770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00ab60 tx=0x7faa6c008040 comp rx=0 tx=0).stop 2024-09-15T21:46:34.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.281+0000 7faa82123640 1 -- 172.21.15.100:0/2705235507 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa7c153040 msgr2=0x7faa7c1f0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.282+0000 7faa82123640 1 --2- 172.21.15.100:0/2705235507 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7faa7c153040 0x7faa7c1f0bb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7faa74030000 tx=0x7faa74004a40 comp rx=0 tx=0).stop 2024-09-15T21:46:34.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.282+0000 7faa7bfff640 1 -- 172.21.15.100:0/2705235507 reap_dead start 2024-09-15T21:46:34.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.282+0000 7f3ddafff640 1 -- 172.21.15.100:0/2649457792 shutdown_connections 2024-09-15T21:46:34.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.282+0000 7f3ddafff640 1 -- 172.21.15.100:0/2649457792 wait complete. 2024-09-15T21:46:34.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.282+0000 7faa82123640 1 -- 172.21.15.100:0/2705235507 shutdown_connections 2024-09-15T21:46:34.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.282+0000 7faa82123640 1 -- 172.21.15.100:0/2705235507 >> 172.21.15.100:0/2705235507 conn(0x7faa7c0ca390 msgr2=0x7faa7c14bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.282+0000 7faa82123640 1 -- 172.21.15.100:0/2705235507 shutdown_connections 2024-09-15T21:46:34.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.282+0000 7faa82123640 1 -- 172.21.15.100:0/2705235507 wait complete. 2024-09-15T21:46:34.354 INFO:teuthology.orchestra.run.smithi100.stdout:55834574855 2024-09-15T21:46:34.354 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:46:34.355 INFO:teuthology.orchestra.run.smithi100.stdout:64424509447 2024-09-15T21:46:34.355 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:46:34.356 INFO:teuthology.orchestra.run.smithi100.stdout:68719476743 2024-09-15T21:46:34.356 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:46:34.357 INFO:teuthology.orchestra.run.smithi100.stdout:64424509448 2024-09-15T21:46:34.357 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T21:46:34.358 INFO:teuthology.orchestra.run.smithi100.stdout:64424509447 2024-09-15T21:46:34.358 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T21:46:34.359 INFO:teuthology.orchestra.run.smithi100.stdout:64424509449 2024-09-15T21:46:34.359 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T21:46:34.360 INFO:teuthology.orchestra.run.smithi100.stdout:64424509446 2024-09-15T21:46:34.360 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T21:46:34.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.556+0000 7f35b19fd640 1 Processor -- start 2024-09-15T21:46:34.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.557+0000 7f35b19fd640 1 -- start start 2024-09-15T21:46:34.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.558+0000 7f35b19fd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f35ac06c2f0 0x7f35ac1511e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.558+0000 7f35b19fd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35ac0c8420 con 0x7f35ac158490 2024-09-15T21:46:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.558+0000 7f35b19fd640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35ac0c9200 con 0x7f35ac157bb0 2024-09-15T21:46:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.558+0000 7f35b19fd640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f35ac0c94e0 con 0x7f35ac06c2f0 2024-09-15T21:46:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.558+0000 7f35ab7fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f35ac158490 0x7f35ac0cc620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:51876/0 (socket says 172.21.15.100:51876) 2024-09-15T21:46:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.558+0000 7f35ab7fe640 1 -- 172.21.15.100:0/2442784165 learned_addr learned my addr 172.21.15.100:0/2442784165 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.558+0000 7f35aaffd640 1 --2- 172.21.15.100:0/2442784165 >> v2:172.21.15.122:3300/0 conn(0x7f35ac06c2f0 0x7f35ac1511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.559+0000 7f35aaffd640 1 -- 172.21.15.100:0/2442784165 >> v1:172.21.15.122:6789/0 conn(0x7f35ac157bb0 legacy=0x7f35ac0c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.559+0000 7f35aaffd640 1 -- 172.21.15.100:0/2442784165 >> v1:172.21.15.100:6789/0 conn(0x7f35ac158490 legacy=0x7f35ac0cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.559+0000 7f35aaffd640 1 -- 172.21.15.100:0/2442784165 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac0c97c0 con 0x7f35ac06c2f0 2024-09-15T21:46:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.560+0000 7f35aaffd640 1 --2- 172.21.15.100:0/2442784165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35ac06c2f0 0x7f35ac1511e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f35a00097a0 tx=0x7f35a002a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bdb42cfc09324f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.560+0000 7f35a9ffb640 1 -- 172.21.15.100:0/2442784165 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a0037070 con 0x7f35ac06c2f0 2024-09-15T21:46:34.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.560+0000 7f35a9ffb640 1 -- 172.21.15.100:0/2442784165 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35a002ad60 con 0x7f35ac06c2f0 2024-09-15T21:46:34.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.561+0000 7f0efcd98640 1 Processor -- start 2024-09-15T21:46:34.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.561+0000 7f0efcd98640 1 -- start start 2024-09-15T21:46:34.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.561+0000 7f35b19fd640 1 -- 172.21.15.100:0/2442784165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35ac06c2f0 msgr2=0x7f35ac1511e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.561+0000 7f35b19fd640 1 --2- 172.21.15.100:0/2442784165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35ac06c2f0 0x7f35ac1511e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f35a00097a0 tx=0x7f35a002a0d0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.562+0000 7f35b19fd640 1 -- 172.21.15.100:0/2442784165 shutdown_connections 2024-09-15T21:46:34.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.562+0000 7f35b19fd640 1 --2- 172.21.15.100:0/2442784165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35ac06c2f0 0x7f35ac1511e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.562+0000 7f35b19fd640 1 -- 172.21.15.100:0/2442784165 >> 172.21.15.100:0/2442784165 conn(0x7f35ac067dd0 msgr2=0x7f35ac06bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.562+0000 7f35b19fd640 1 -- 172.21.15.100:0/2442784165 shutdown_connections 2024-09-15T21:46:34.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.562+0000 7f35b19fd640 1 -- 172.21.15.100:0/2442784165 wait complete. 2024-09-15T21:46:34.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.562+0000 7f0efcd98640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0ef80ca2e0 0x7f0ef8162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.562+0000 7f0efcd98640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0ef80c7a00 0x7f0ef815ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.562+0000 7f0efcd98640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ef815f4e0 con 0x7f0ef80c9f00 2024-09-15T21:46:34.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.562+0000 7f0efcd98640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f0ef815fb10 con 0x7f0ef80ca2e0 2024-09-15T21:46:34.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.562+0000 7f0efcd98640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0ef80c7f60 con 0x7f0ef80c7a00 2024-09-15T21:46:34.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.563+0000 7f0ef5d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0ef80ca2e0 0x7f0ef8162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.563+0000 7f0ef5d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0ef80ca2e0 0x7f0ef8162a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:43962/0 (socket says 172.21.15.100:43962) 2024-09-15T21:46:34.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.563+0000 7f0ef5d9b640 1 -- 172.21.15.100:0/3870395462 learned_addr learned my addr 172.21.15.100:0/3870395462 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.563+0000 7fac53d11640 1 Processor -- start 2024-09-15T21:46:34.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.563+0000 7f35b19fd640 1 Processor -- start 2024-09-15T21:46:34.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.563+0000 7fb3da574640 1 Processor -- start 2024-09-15T21:46:34.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.563+0000 7fb3da574640 1 -- start start 2024-09-15T21:46:34.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.563+0000 7f0ef5d9b640 1 -- 172.21.15.100:0/3870395462 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0ef80c7a00 msgr2=0x7f0ef815ffa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T21:46:34.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.563+0000 7f0ef5d9b640 1 --2- 172.21.15.100:0/3870395462 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0ef80c7a00 0x7f0ef815ffa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.563+0000 7f0ef5d9b640 1 -- 172.21.15.100:0/3870395462 >> v1:172.21.15.100:6789/0 conn(0x7f0ef80c9f00 legacy=0x7f0ef8148220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.563+0000 7f0ef5d9b640 1 -- 172.21.15.100:0/3870395462 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef80c8240 con 0x7f0ef80ca2e0 2024-09-15T21:46:34.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.564+0000 7fb3da574640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb3cc0d8870 0x7fb3cc049950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.564+0000 7fb3da574640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb3cc0502d0 0x7fb3cc049e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.564+0000 7fb3da574640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3cc050850 con 0x7fb3cc0491c0 2024-09-15T21:46:34.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.564+0000 7fb3da574640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fb3cc0e1c60 con 0x7fb3cc0d8870 2024-09-15T21:46:34.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.564+0000 7fb3da574640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fb3cc0e1f40 con 0x7fb3cc0502d0 2024-09-15T21:46:34.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.565+0000 7fb3d9d73640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb3cc0491c0 0x7fb3cc0de730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:51900/0 (socket says 172.21.15.100:51900) 2024-09-15T21:46:34.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.565+0000 7fb3d9d73640 1 -- 172.21.15.100:0/4015042798 learned_addr learned my addr 172.21.15.100:0/4015042798 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:34.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.565+0000 7fb3d9572640 1 --2- 172.21.15.100:0/4015042798 >> v2:172.21.15.100:3300/0 conn(0x7fb3cc0d8870 0x7fb3cc049950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.565+0000 7f0ef5d9b640 1 --2- 172.21.15.100:0/3870395462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ef80ca2e0 0x7f0ef8162a70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0ef815fc90 tx=0x7f0eec02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85342cde340ce659 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.564+0000 7f35b19fd640 1 -- start start 2024-09-15T21:46:34.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.565+0000 7f35b19fd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f35ac06c2f0 0x7f35ac155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.565+0000 7f35b19fd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35ac157bb0 0x7f35ac153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.563+0000 7fac53d11640 1 -- start start 2024-09-15T21:46:34.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.564+0000 7fac53d11640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fac4400d230 0x7fac4404d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.564+0000 7fac53d11640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fac4400bd90 0x7fac4400c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.565+0000 7fac51aad640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fac440dcdc0 0x7fac440e60b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:51894/0 (socket says 172.21.15.100:51894) 2024-09-15T21:46:34.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.565+0000 7fac51aad640 1 -- 172.21.15.100:0/3491670969 learned_addr learned my addr 172.21.15.100:0/3491670969 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:34.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.565+0000 7fb3d8d71640 1 --2- 172.21.15.100:0/4015042798 >> v2:172.21.15.122:3300/0 conn(0x7fb3cc0502d0 0x7fb3cc049e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.565+0000 7fb3cbfff640 1 -- 172.21.15.100:0/4015042798 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1152669209 0 0) 0x7fb3cc050850 con 0x7fb3cc0491c0 2024-09-15T21:46:34.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7fb3cbfff640 1 -- 172.21.15.100:0/4015042798 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb3c0003520 con 0x7fb3cc0491c0 2024-09-15T21:46:34.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.565+0000 7fac53d11640 1 -- 172.21.15.100:0/3491670969 --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac44049040 con 0x7fac440dcdc0 2024-09-15T21:46:34.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.565+0000 7fac53d11640 1 -- 172.21.15.100:0/3491670969 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fac4404d850 con 0x7fac4400d230 2024-09-15T21:46:34.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.565+0000 7fac53d11640 1 -- 172.21.15.100:0/3491670969 --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fac4404e530 con 0x7fac4400bd90 2024-09-15T21:46:34.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7fac522ae640 1 --2- 172.21.15.100:0/3491670969 >> v2:172.21.15.122:3300/0 conn(0x7fac4400bd90 0x7fac4400c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7fac512ac640 1 --2- 172.21.15.100:0/3491670969 >> v2:172.21.15.100:3300/0 conn(0x7fac4400d230 0x7fac4404d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.565+0000 7f0ef559a640 1 -- 172.21.15.100:0/3870395462 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec037070 con 0x7f0ef80ca2e0 2024-09-15T21:46:34.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7f0ef559a640 1 -- 172.21.15.100:0/3870395462 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0eec032440 con 0x7f0ef80ca2e0 2024-09-15T21:46:34.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7f0ef559a640 1 -- 172.21.15.100:0/3870395462 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec032830 con 0x7f0ef80ca2e0 2024-09-15T21:46:34.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7f35aaffd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f35ac06c2f0 0x7f35ac155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7f35aaffd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f35ac06c2f0 0x7f35ac155e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:55960/0 (socket says 172.21.15.100:55960) 2024-09-15T21:46:34.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7fac522ae640 1 -- 172.21.15.100:0/3491670969 >> v2:172.21.15.100:3300/0 conn(0x7fac4400d230 msgr2=0x7fac4404d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:34.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7fac522ae640 1 --2- 172.21.15.100:0/3491670969 >> v2:172.21.15.100:3300/0 conn(0x7fac4400d230 0x7fac4404d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7fac522ae640 1 -- 172.21.15.100:0/3491670969 >> v1:172.21.15.100:6789/0 conn(0x7fac440dcdc0 legacy=0x7fac440e60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7fac522ae640 1 -- 172.21.15.100:0/3491670969 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4400d640 con 0x7fac4400bd90 2024-09-15T21:46:34.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7f35aa7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35ac157bb0 0x7f35ac153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7f35b19fd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35ac158490 0x7f35ac153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7f35b19fd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac0cceb0 con 0x7f35ac157bb0 2024-09-15T21:46:34.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7f35aaffd640 1 -- 172.21.15.100:0/2153727740 learned_addr learned my addr 172.21.15.100:0/2153727740 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7f35ab7fe640 1 --2- 172.21.15.100:0/2153727740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35ac158490 0x7f35ac153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7f35b19fd640 1 -- 172.21.15.100:0/2153727740 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac0ccd30 con 0x7f35ac158490 2024-09-15T21:46:34.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7f35b19fd640 1 -- 172.21.15.100:0/2153727740 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f35ac0c7fe0 con 0x7f35ac06c2f0 2024-09-15T21:46:34.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7fb3d8d71640 1 -- 172.21.15.100:0/4015042798 >> v2:172.21.15.100:3300/0 conn(0x7fb3cc0d8870 msgr2=0x7fb3cc049950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:34.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7fb3d8d71640 1 --2- 172.21.15.100:0/4015042798 >> v2:172.21.15.100:3300/0 conn(0x7fb3cc0d8870 0x7fb3cc049950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7fb3d8d71640 1 -- 172.21.15.100:0/4015042798 >> v1:172.21.15.100:6789/0 conn(0x7fb3cc0491c0 legacy=0x7fb3cc0de730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.566+0000 7fb3d8d71640 1 -- 172.21.15.100:0/4015042798 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3cc0e2220 con 0x7fb3cc0502d0 2024-09-15T21:46:34.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.567+0000 7f35aa7fc640 1 -- 172.21.15.100:0/2153727740 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f35ac06c2f0 msgr2=0x7f35ac155e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.567+0000 7f35aa7fc640 1 --2- 172.21.15.100:0/2153727740 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f35ac06c2f0 0x7f35ac155e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.567+0000 7f35aa7fc640 1 -- 172.21.15.100:0/2153727740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35ac158490 msgr2=0x7f35ac153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.567+0000 7f35aa7fc640 1 --2- 172.21.15.100:0/2153727740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35ac158490 0x7f35ac153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.567+0000 7f35aa7fc640 1 -- 172.21.15.100:0/2153727740 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac154700 con 0x7f35ac157bb0 2024-09-15T21:46:34.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.567+0000 7fb3d8d71640 1 --2- 172.21.15.100:0/4015042798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3cc0502d0 0x7fb3cc049e90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0115e0 tx=0x7fb3c402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=941043a29f5217e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.567+0000 7f35aaffd640 1 --2- 172.21.15.100:0/2153727740 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f35ac06c2f0 0x7f35ac155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:34.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.567+0000 7fb3cbfff640 1 -- 172.21.15.100:0/4015042798 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4037070 con 0x7fb3cc0502d0 2024-09-15T21:46:34.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.567+0000 7f35aa7fc640 1 --2- 172.21.15.100:0/2153727740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35ac157bb0 0x7f35ac153850 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f35a40021d0 tx=0x7f35a400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.567+0000 7f3593fff640 1 -- 172.21.15.100:0/2153727740 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4015c80 con 0x7f35ac157bb0 2024-09-15T21:46:34.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.568+0000 7f35b19fd640 1 -- 172.21.15.100:0/2153727740 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ac1527b0 con 0x7f35ac157bb0 2024-09-15T21:46:34.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.568+0000 7f35b19fd640 1 -- 172.21.15.100:0/2153727740 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ac152d70 con 0x7f35ac157bb0 2024-09-15T21:46:34.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.568+0000 7f3593fff640 1 -- 172.21.15.100:0/2153727740 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35a4019070 con 0x7f35ac157bb0 2024-09-15T21:46:34.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.567+0000 7fac522ae640 1 --2- 172.21.15.100:0/3491670969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac4400bd90 0x7fac4400c1c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fac4404cdc0 tx=0x7fac4802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9833d33f8416f685 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.567+0000 7fac50aab640 1 -- 172.21.15.100:0/3491670969 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48037070 con 0x7fac4400bd90 2024-09-15T21:46:34.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.567+0000 7fac50aab640 1 -- 172.21.15.100:0/3491670969 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac48032440 con 0x7fac4400bd90 2024-09-15T21:46:34.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.567+0000 7fac50aab640 1 -- 172.21.15.100:0/3491670969 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48032810 con 0x7fac4400bd90 2024-09-15T21:46:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.567+0000 7f0efcd98640 1 -- 172.21.15.100:0/3870395462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ef80ca2e0 msgr2=0x7f0ef8162a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.567+0000 7f0efcd98640 1 --2- 172.21.15.100:0/3870395462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ef80ca2e0 0x7f0ef8162a70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0ef815fc90 tx=0x7f0eec02a290 comp rx=0 tx=0).stop 2024-09-15T21:46:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.567+0000 7f0efcd98640 1 -- 172.21.15.100:0/3870395462 shutdown_connections 2024-09-15T21:46:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.567+0000 7f0efcd98640 1 --2- 172.21.15.100:0/3870395462 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0ef80c7a00 0x7f0ef815ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.568+0000 7f0efcd98640 1 --2- 172.21.15.100:0/3870395462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ef80ca2e0 0x7f0ef8162a70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.568+0000 7f0efcd98640 1 -- 172.21.15.100:0/3870395462 >> 172.21.15.100:0/3870395462 conn(0x7f0ef8073820 msgr2=0x7f0ef8073c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.568+0000 7f0efcd98640 1 -- 172.21.15.100:0/3870395462 shutdown_connections 2024-09-15T21:46:34.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.568+0000 7f0efcd98640 1 -- 172.21.15.100:0/3870395462 wait complete. 2024-09-15T21:46:34.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.568+0000 7fb3cbfff640 1 -- 172.21.15.100:0/4015042798 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c4032440 con 0x7fb3cc0502d0 2024-09-15T21:46:34.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.569+0000 7fb3cbfff640 1 -- 172.21.15.100:0/4015042798 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4032850 con 0x7fb3cc0502d0 2024-09-15T21:46:34.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.569+0000 7fb3da574640 1 -- 172.21.15.100:0/4015042798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3cc0502d0 msgr2=0x7fb3cc049e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.569+0000 7fb3da574640 1 --2- 172.21.15.100:0/4015042798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3cc0502d0 0x7fb3cc049e90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0115e0 tx=0x7fb3c402a1c0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.569+0000 7f3593fff640 1 -- 172.21.15.100:0/2153727740 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4003bb0 con 0x7f35ac157bb0 2024-09-15T21:46:34.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.568+0000 7fac53d11640 1 -- 172.21.15.100:0/3491670969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac4400bd90 msgr2=0x7fac4400c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.568+0000 7fac53d11640 1 --2- 172.21.15.100:0/3491670969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac4400bd90 0x7fac4400c1c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fac4404cdc0 tx=0x7fac4802a1a0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.568+0000 7fac53d11640 1 -- 172.21.15.100:0/3491670969 shutdown_connections 2024-09-15T21:46:34.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.568+0000 7fac53d11640 1 --2- 172.21.15.100:0/3491670969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac4400bd90 0x7fac4400c1c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.568+0000 7fac53d11640 1 --2- 172.21.15.100:0/3491670969 >> v2:172.21.15.100:3300/0 conn(0x7fac4400d230 0x7fac4404d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.568+0000 7fac53d11640 1 -- 172.21.15.100:0/3491670969 >> 172.21.15.100:0/3491670969 conn(0x7fac4400a160 msgr2=0x7fac4400db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.569+0000 7fb3da574640 1 -- 172.21.15.100:0/4015042798 shutdown_connections 2024-09-15T21:46:34.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.569+0000 7fb3da574640 1 --2- 172.21.15.100:0/4015042798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3cc0502d0 0x7fb3cc049e90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.569+0000 7fb3da574640 1 --2- 172.21.15.100:0/4015042798 >> v2:172.21.15.100:3300/0 conn(0x7fb3cc0d8870 0x7fb3cc049950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.569+0000 7fb3da574640 1 -- 172.21.15.100:0/4015042798 >> 172.21.15.100:0/4015042798 conn(0x7fb3cc054450 msgr2=0x7fb3cc054860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.569+0000 7f0efcd98640 1 Processor -- start 2024-09-15T21:46:34.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.569+0000 7fac53d11640 1 -- 172.21.15.100:0/3491670969 shutdown_connections 2024-09-15T21:46:34.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.569+0000 7f0efcd98640 1 -- start start 2024-09-15T21:46:34.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.570+0000 7fb3da574640 1 -- 172.21.15.100:0/4015042798 shutdown_connections 2024-09-15T21:46:34.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.571+0000 7f35b19fd640 1 -- 172.21.15.100:0/2153727740 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f358c005290 con 0x7f35ac157bb0 2024-09-15T21:46:34.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.569+0000 7fac53d11640 1 -- 172.21.15.100:0/3491670969 wait complete. 2024-09-15T21:46:34.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.571+0000 7f0efcd98640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ef80c7a00 0x7f0ef815b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.571+0000 7f0efcd98640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0ef80c9f00 0x7f0ef81f8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.571+0000 7f2f1056d640 1 Processor -- start 2024-09-15T21:46:34.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.571+0000 7f0ef659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ef80c7a00 0x7f0ef815b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.571+0000 7fac53d11640 1 Processor -- start 2024-09-15T21:46:34.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.571+0000 7f0ef659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ef80c7a00 0x7f0ef815b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:41090/0 (socket says 172.21.15.100:41090) 2024-09-15T21:46:34.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.571+0000 7f17e759e640 1 Processor -- start 2024-09-15T21:46:34.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.571+0000 7f0ef659c640 1 -- 172.21.15.100:0/2237381404 learned_addr learned my addr 172.21.15.100:0/2237381404 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.571+0000 7f0ef5d9b640 1 --2- 172.21.15.100:0/2237381404 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0ef80c9f00 0x7f0ef81f8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.572+0000 7f0efcd98640 1 --2- 172.21.15.100:0/2237381404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ef80ca2e0 0x7f0ef815cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.572+0000 7f0ef6d9d640 1 --2- 172.21.15.100:0/2237381404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ef80ca2e0 0x7f0ef815cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.572+0000 7f0efcd98640 1 -- 172.21.15.100:0/2237381404 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef8148930 con 0x7f0ef80ca2e0 2024-09-15T21:46:34.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.572+0000 7f0efcd98640 1 -- 172.21.15.100:0/2237381404 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef815f7a0 con 0x7f0ef80c7a00 2024-09-15T21:46:34.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.572+0000 7f0efcd98640 1 -- 172.21.15.100:0/2237381404 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0ef8148ab0 con 0x7f0ef80c9f00 2024-09-15T21:46:34.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.571+0000 7fac53d11640 1 -- start start 2024-09-15T21:46:34.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.572+0000 7fac53d11640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac4400bd90 0x7fac44036b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.571+0000 7f17e759e640 1 -- start start 2024-09-15T21:46:34.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7fac53d11640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fac4400d230 0x7fac440dd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7fac51aad640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac4400bd90 0x7fac44036b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7fac51aad640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac4400bd90 0x7fac44036b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44008/0 (socket says 172.21.15.100:44008) 2024-09-15T21:46:34.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.571+0000 7f2f1056d640 1 -- start start 2024-09-15T21:46:34.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f2f1056d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2f080c9580 0x7f2f08158760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f2f1056d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f2f08072500 0x7f2f0806d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f2f1056d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f0806de40 con 0x7f2f08151f30 2024-09-15T21:46:34.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f2f1056d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f2f080c8290 con 0x7f2f080c9580 2024-09-15T21:46:34.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f2f1056d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f2f08072910 con 0x7f2f08072500 2024-09-15T21:46:34.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7fac51aad640 1 -- 172.21.15.100:0/40970190 learned_addr learned my addr 172.21.15.100:0/40970190 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f0ef659c640 1 -- 172.21.15.100:0/2237381404 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0ef80c9f00 msgr2=0x7f0ef81f8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f0ef659c640 1 --2- 172.21.15.100:0/2237381404 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0ef80c9f00 0x7f0ef81f8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f17e759e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f17e004a680 0x7f17e004aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f17e759e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f17e00e5f30 0x7f17e00e6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f17e759e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17e00e6d30 con 0x7f17e00dcd90 2024-09-15T21:46:34.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f0ef659c640 1 -- 172.21.15.100:0/2237381404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ef80ca2e0 msgr2=0x7f0ef815cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f0ef659c640 1 --2- 172.21.15.100:0/2237381404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ef80ca2e0 0x7f0ef815cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f0ef659c640 1 -- 172.21.15.100:0/2237381404 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef815c1e0 con 0x7f0ef80c7a00 2024-09-15T21:46:34.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f0ef6d9d640 1 --2- 172.21.15.100:0/2237381404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ef80ca2e0 0x7f0ef815cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:46:34.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f17e759e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f17e0044d40 con 0x7f17e004a680 2024-09-15T21:46:34.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f17e759e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f17e00dd170 con 0x7f17e00e5f30 2024-09-15T21:46:34.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7fac512ac640 1 --2- 172.21.15.100:0/40970190 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fac4400d230 0x7fac440dd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f17e6d9d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f17e00e5f30 0x7f17e00e6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f17e6d9d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f17e00e5f30 0x7f17e00e6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:41120/0 (socket says 172.21.15.100:41120) 2024-09-15T21:46:34.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f17e6d9d640 1 -- 172.21.15.100:0/252884810 learned_addr learned my addr 172.21.15.100:0/252884810 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f0ef659c640 1 --2- 172.21.15.100:0/2237381404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ef80c7a00 0x7f0ef815b800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800c450 tx=0x7f0ee800c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f2f0e309640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f2f08151f30 0x7f2f0806ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:51910/0 (socket says 172.21.15.100:51910) 2024-09-15T21:46:34.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f2f0e309640 1 -- 172.21.15.100:0/3284339436 learned_addr learned my addr 172.21.15.100:0/3284339436 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:34.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7fac53d11640 1 --2- 172.21.15.100:0/40970190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac440dcdc0 0x7fac440e1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7fac53d11640 1 -- 172.21.15.100:0/40970190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fac4404ea00 con 0x7fac4400bd90 2024-09-15T21:46:34.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7fac53d11640 1 -- 172.21.15.100:0/40970190 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fac4404e760 con 0x7fac440dcdc0 2024-09-15T21:46:34.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7fac53d11640 1 -- 172.21.15.100:0/40970190 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fac4400c700 con 0x7fac4400d230 2024-09-15T21:46:34.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f17e5d9b640 1 --2- 172.21.15.100:0/252884810 >> v2:172.21.15.100:3300/0 conn(0x7f17e004a680 0x7f17e004aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f2f0db08640 1 --2- 172.21.15.100:0/3284339436 >> v2:172.21.15.100:3300/0 conn(0x7f2f080c9580 0x7f2f08158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f17e6d9d640 1 -- 172.21.15.100:0/252884810 >> v2:172.21.15.100:3300/0 conn(0x7f17e004a680 msgr2=0x7f17e004aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:34.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f17e6d9d640 1 --2- 172.21.15.100:0/252884810 >> v2:172.21.15.100:3300/0 conn(0x7f17e004a680 0x7f17e004aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f17e6d9d640 1 -- 172.21.15.100:0/252884810 >> v1:172.21.15.100:6789/0 conn(0x7f17e00dcd90 legacy=0x7f17e0045d50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:46:34.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7f0ee6ffd640 1 -- 172.21.15.100:0/2237381404 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee80052d0 con 0x7f0ef80c7a00 2024-09-15T21:46:34.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.574+0000 7f0efcd98640 1 -- 172.21.15.100:0/2237381404 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef8072a80 con 0x7f0ef80c7a00 2024-09-15T21:46:34.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.574+0000 7f0efcd98640 1 -- 172.21.15.100:0/2237381404 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef815c730 con 0x7f0ef80c7a00 2024-09-15T21:46:34.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.574+0000 7f17e6d9d640 1 -- 172.21.15.100:0/252884810 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e0044f90 con 0x7f17e00e5f30 2024-09-15T21:46:34.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.574+0000 7fac51aad640 1 -- 172.21.15.100:0/40970190 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fac4400d230 msgr2=0x7fac440dd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.574+0000 7fac51aad640 1 --2- 172.21.15.100:0/40970190 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fac4400d230 0x7fac440dd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.574+0000 7fac51aad640 1 -- 172.21.15.100:0/40970190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac440dcdc0 msgr2=0x7fac440e1b70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T21:46:34.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.575+0000 7f17e6d9d640 1 --2- 172.21.15.100:0/252884810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17e00e5f30 0x7f17e00e6360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f17e80695f0 tx=0x7f17e80a7130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e4ee4342197429b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.574+0000 7f2f0db08640 1 -- 172.21.15.100:0/3284339436 >> v2:172.21.15.122:3300/0 conn(0x7f2f08072500 msgr2=0x7f2f0806d400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T21:46:34.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.574+0000 7f2f0db08640 1 --2- 172.21.15.100:0/3284339436 >> v2:172.21.15.122:3300/0 conn(0x7f2f08072500 0x7f2f0806d400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.574+0000 7f2f0db08640 1 -- 172.21.15.100:0/3284339436 >> v1:172.21.15.100:6789/0 conn(0x7f2f08151f30 legacy=0x7f2f0806ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.574+0000 7f2f0db08640 1 -- 172.21.15.100:0/3284339436 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f080c84e0 con 0x7f2f080c9580 2024-09-15T21:46:34.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.575+0000 7f3593fff640 1 -- 172.21.15.100:0/2153727740 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f35a4005070 con 0x7f35ac157bb0 2024-09-15T21:46:34.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.575+0000 7f17e559a640 1 -- 172.21.15.100:0/252884810 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e80b4070 con 0x7f17e00e5f30 2024-09-15T21:46:34.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.575+0000 7f17e559a640 1 -- 172.21.15.100:0/252884810 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17e80af440 con 0x7f17e00e5f30 2024-09-15T21:46:34.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.575+0000 7f2f0db08640 1 --2- 172.21.15.100:0/3284339436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f080c9580 0x7f2f08158760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2f080cb740 tx=0x7f2ef802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1400afae2305da server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.575+0000 7f3593fff640 1 --2- 172.21.15.100:0/2153727740 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f359806c2e0 0x7f359806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.575+0000 7f0ee6ffd640 1 -- 172.21.15.100:0/2237381404 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ee8005470 con 0x7f0ef80c7a00 2024-09-15T21:46:34.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.575+0000 7f17e559a640 1 -- 172.21.15.100:0/252884810 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e80af850 con 0x7f17e00e5f30 2024-09-15T21:46:34.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.574+0000 7fac51aad640 1 --2- 172.21.15.100:0/40970190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac440dcdc0 0x7fac440e1b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.574+0000 7fac51aad640 1 -- 172.21.15.100:0/40970190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4404bcb0 con 0x7fac4400bd90 2024-09-15T21:46:34.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.574+0000 7fac51aad640 1 --2- 172.21.15.100:0/40970190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac4400bd90 0x7fac44036b60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fac4c06b960 tx=0x7fac4c069a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.576+0000 7f0ee6ffd640 1 -- 172.21.15.100:0/2237381404 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8012870 con 0x7f0ef80c7a00 2024-09-15T21:46:34.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.576+0000 7f17e759e640 1 -- 172.21.15.100:0/252884810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17e00e5f30 msgr2=0x7f17e00e6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.576+0000 7f17e759e640 1 --2- 172.21.15.100:0/252884810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17e00e5f30 0x7f17e00e6360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f17e80695f0 tx=0x7f17e80a7130 comp rx=0 tx=0).stop 2024-09-15T21:46:34.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.571+0000 7fb3da574640 1 -- 172.21.15.100:0/4015042798 wait complete. 2024-09-15T21:46:34.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7fb3da574640 1 Processor -- start 2024-09-15T21:46:34.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.573+0000 7fb3da574640 1 -- start start 2024-09-15T21:46:34.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.574+0000 7fb3da574640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb3cc0491c0 0x7fb3cc038610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.574+0000 7fb3da574640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3cc0502d0 0x7fb3cc038b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.574+0000 7fb3da574640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3cc0d8870 0x7fb3cc0d6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.574+0000 7fb3da574640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb3cc0d8c50 con 0x7fb3cc0d8870 2024-09-15T21:46:34.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.574+0000 7fb3da574640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb3cc00f720 con 0x7fb3cc0502d0 2024-09-15T21:46:34.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.574+0000 7fb3da574640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb3cc0d8dd0 con 0x7fb3cc0491c0 2024-09-15T21:46:34.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.575+0000 7fb3d9d73640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3cc0d8870 0x7fb3cc0d6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.575+0000 7fb3d9d73640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3cc0d8870 0x7fb3cc0d6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44028/0 (socket says 172.21.15.100:44028) 2024-09-15T21:46:34.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.575+0000 7fb3d9d73640 1 -- 172.21.15.100:0/1498898860 learned_addr learned my addr 172.21.15.100:0/1498898860 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.575+0000 7fb3d8d71640 1 --2- 172.21.15.100:0/1498898860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3cc0502d0 0x7fb3cc038b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.575+0000 7fb3d9d73640 1 -- 172.21.15.100:0/1498898860 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb3cc0491c0 msgr2=0x7fb3cc038610 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T21:46:34.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.575+0000 7fb3d9d73640 1 --2- 172.21.15.100:0/1498898860 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb3cc0491c0 0x7fb3cc038610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.575+0000 7fb3d9d73640 1 -- 172.21.15.100:0/1498898860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3cc0502d0 msgr2=0x7fb3cc038b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.575+0000 7fb3d9d73640 1 --2- 172.21.15.100:0/1498898860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3cc0502d0 0x7fb3cc038b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.575+0000 7fb3d9d73640 1 -- 172.21.15.100:0/1498898860 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3cc0d80e0 con 0x7fb3cc0d8870 2024-09-15T21:46:34.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.575+0000 7fb3d8d71640 1 --2- 172.21.15.100:0/1498898860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3cc0502d0 0x7fb3cc038b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:34.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.576+0000 7fb3d9d73640 1 --2- 172.21.15.100:0/1498898860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3cc0d8870 0x7fb3cc0d6b40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c3c0 tx=0x7fb3d0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.576+0000 7f17e759e640 1 -- 172.21.15.100:0/252884810 shutdown_connections 2024-09-15T21:46:34.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.576+0000 7f17e759e640 1 --2- 172.21.15.100:0/252884810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17e00e5f30 0x7f17e00e6360 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.576+0000 7f17e759e640 1 --2- 172.21.15.100:0/252884810 >> v2:172.21.15.100:3300/0 conn(0x7f17e004a680 0x7f17e004aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.576+0000 7f17e759e640 1 -- 172.21.15.100:0/252884810 >> 172.21.15.100:0/252884810 conn(0x7f17e004fc50 msgr2=0x7f17e0050060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.575+0000 7f3593fff640 1 -- 172.21.15.100:0/2153727740 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f35a4093680 con 0x7f35ac157bb0 2024-09-15T21:46:34.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.576+0000 7f0ee4ff9640 1 -- 172.21.15.100:0/2237381404 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0edc005290 con 0x7f0ef80c7a00 2024-09-15T21:46:34.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.576+0000 7f17e759e640 1 -- 172.21.15.100:0/252884810 shutdown_connections 2024-09-15T21:46:34.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.576+0000 7f0ee6ffd640 1 -- 172.21.15.100:0/2237381404 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0ee8012a90 con 0x7f0ef80c7a00 2024-09-15T21:46:34.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.576+0000 7f17e759e640 1 -- 172.21.15.100:0/252884810 wait complete. 2024-09-15T21:46:34.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.577+0000 7f35aaffd640 1 --2- 172.21.15.100:0/2153727740 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f359806c2e0 0x7f359806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.577+0000 7f2e4cc4c640 1 Processor -- start 2024-09-15T21:46:34.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.577+0000 7f2e4cc4c640 1 -- start start 2024-09-15T21:46:34.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.577+0000 7f0ee6ffd640 1 --2- 172.21.15.100:0/2237381404 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0ed806c330 0x7f0ed806e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.577+0000 7f35aaffd640 1 --2- 172.21.15.100:0/2153727740 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f359806c2e0 0x7f359806e7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f35a002a5f0 tx=0x7f35a002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.577+0000 7f0ef5d9b640 1 --2- 172.21.15.100:0/2237381404 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0ed806c330 0x7f0ed806e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.577+0000 7f0ee6ffd640 1 -- 172.21.15.100:0/2237381404 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f0ee808ae90 con 0x7f0ef80c7a00 2024-09-15T21:46:34.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.578+0000 7f0ef5d9b640 1 --2- 172.21.15.100:0/2237381404 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0ed806c330 0x7f0ed806e7f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0eec034000 tx=0x7f0eec02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.578+0000 7f2e4cc4c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2e48152fe0 0x7f2e4815ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.578+0000 7f2e4cc4c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e4815c530 con 0x7f2e4815d200 2024-09-15T21:46:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.578+0000 7f17e759e640 1 Processor -- start 2024-09-15T21:46:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.578+0000 7f2e4cc4c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e480c9e20 con 0x7f2e480730f0 2024-09-15T21:46:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.578+0000 7f2e4cc4c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f2e48159ec0 con 0x7f2e48152fe0 2024-09-15T21:46:34.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.578+0000 7f2e4659c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2e48152fe0 0x7f2e4815ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.578+0000 7f17e759e640 1 -- start start 2024-09-15T21:46:34.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.578+0000 7f2e3ffff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f2e4815d200 0x7f2e480c9710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:51920/0 (socket says 172.21.15.100:51920) 2024-09-15T21:46:34.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.578+0000 7f2e4659c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2e48152fe0 0x7f2e4815ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44040/0 (socket says 172.21.15.100:44040) 2024-09-15T21:46:34.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.578+0000 7f2e3ffff640 1 -- 172.21.15.100:0/2464307004 learned_addr learned my addr 172.21.15.100:0/2464307004 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:34.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.578+0000 7f2e4659c640 1 -- 172.21.15.100:0/2464307004 >> v1:172.21.15.122:6789/0 conn(0x7f2e480730f0 legacy=0x7f2e48163820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.578+0000 7f2e4659c640 1 -- 172.21.15.100:0/2464307004 >> v1:172.21.15.100:6789/0 conn(0x7f2e4815d200 legacy=0x7f2e480c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.578+0000 7f2e4659c640 1 -- 172.21.15.100:0/2464307004 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e480c3f30 con 0x7f2e48152fe0 2024-09-15T21:46:34.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.579+0000 7f17e759e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17e004a680 0x7f17e016cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.579+0000 7f17e759e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f17e00dcd90 0x7f17e00d63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.579+0000 7f17e759e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e00e5f30 0x7f17e00d68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.579+0000 7f17e759e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f17e0045830 con 0x7f17e00e5f30 2024-09-15T21:46:34.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.579+0000 7f17e759e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17e0045550 con 0x7f17e004a680 2024-09-15T21:46:34.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.579+0000 7f17e759e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f17e00e2be0 con 0x7f17e00dcd90 2024-09-15T21:46:34.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.579+0000 7f17e6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e00e5f30 0x7f17e00d68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.579+0000 7f17e6d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e00e5f30 0x7f17e00d68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44044/0 (socket says 172.21.15.100:44044) 2024-09-15T21:46:34.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.579+0000 7f17e6d9d640 1 -- 172.21.15.100:0/3302468781 learned_addr learned my addr 172.21.15.100:0/3302468781 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.579+0000 7f17e659c640 1 --2- 172.21.15.100:0/3302468781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17e004a680 0x7f17e016cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.579+0000 7f17e5d9b640 1 --2- 172.21.15.100:0/3302468781 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f17e00dcd90 0x7f17e00d63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.579+0000 7f17e6d9d640 1 -- 172.21.15.100:0/3302468781 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f17e00dcd90 msgr2=0x7f17e00d63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.579+0000 7f17e6d9d640 1 --2- 172.21.15.100:0/3302468781 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f17e00dcd90 0x7f17e00d63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.579+0000 7f17e6d9d640 1 -- 172.21.15.100:0/3302468781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17e004a680 msgr2=0x7f17e016cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.579+0000 7f17e6d9d640 1 --2- 172.21.15.100:0/3302468781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17e004a680 0x7f17e016cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.579+0000 7f17e6d9d640 1 -- 172.21.15.100:0/3302468781 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17e00e4de0 con 0x7f17e00e5f30 2024-09-15T21:46:34.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.579+0000 7f2e4659c640 1 --2- 172.21.15.100:0/2464307004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e48152fe0 0x7f2e4815ccc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2e380097a0 tx=0x7f2e3802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fd8e96c54283d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.579+0000 7f17e6d9d640 1 --2- 172.21.15.100:0/3302468781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e00e5f30 0x7f17e00d68f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f17e8000b60 tx=0x7f17e8069fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.580+0000 7fc960a4b640 1 Processor -- start 2024-09-15T21:46:34.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.580+0000 7fc960a4b640 1 -- start start 2024-09-15T21:46:34.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.580+0000 7fac427fc640 1 -- 172.21.15.100:0/40970190 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c077800 con 0x7fac4400bd90 2024-09-15T21:46:34.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.580+0000 7f2f0d307640 1 -- 172.21.15.100:0/3284339436 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8037070 con 0x7f2f080c9580 2024-09-15T21:46:34.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.580+0000 7fb3c9ffb640 1 -- 172.21.15.100:0/1498898860 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0017070 con 0x7fb3cc0d8870 2024-09-15T21:46:34.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.580+0000 7fac53d11640 1 -- 172.21.15.100:0/40970190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4404bf70 con 0x7fac4400bd90 2024-09-15T21:46:34.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.580+0000 7fb3da574640 1 -- 172.21.15.100:0/1498898860 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3cc0df7e0 con 0x7fb3cc0d8870 2024-09-15T21:46:34.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.580+0000 7fac53d11640 1 -- 172.21.15.100:0/40970190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac440e2cb0 con 0x7fac4400bd90 2024-09-15T21:46:34.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.580+0000 7fac427fc640 1 -- 172.21.15.100:0/40970190 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac4c092020 con 0x7fac4400bd90 2024-09-15T21:46:34.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.580+0000 7fac427fc640 1 -- 172.21.15.100:0/40970190 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c06bbe0 con 0x7fac4400bd90 2024-09-15T21:46:34.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.580+0000 7f17d6ffd640 1 -- 172.21.15.100:0/3302468781 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e80bd020 con 0x7f17e00e5f30 2024-09-15T21:46:34.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.581+0000 7f2e45d9b640 1 -- 172.21.15.100:0/2464307004 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38037070 con 0x7f2e48152fe0 2024-09-15T21:46:34.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.581+0000 7fb3da574640 1 -- 172.21.15.100:0/1498898860 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3cc0d7b20 con 0x7fb3cc0d8870 2024-09-15T21:46:34.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.581+0000 7f3593fff640 1 -- 172.21.15.100:0/2153727740 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f35a4061d50 con 0x7f35ac157bb0 2024-09-15T21:46:34.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.581+0000 7fb3c9ffb640 1 -- 172.21.15.100:0/1498898860 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d0005ce0 con 0x7fb3cc0d8870 2024-09-15T21:46:34.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.581+0000 7fb3c9ffb640 1 -- 172.21.15.100:0/1498898860 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0021060 con 0x7fb3cc0d8870 2024-09-15T21:46:34.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.581+0000 7fc960a4b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc954048570 0x7fc9540dd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.581+0000 7f2f0d307640 1 -- 172.21.15.100:0/3284339436 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef8032440 con 0x7f2f080c9580 2024-09-15T21:46:34.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.581+0000 7f17e759e640 1 -- 172.21.15.100:0/3302468781 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17e0179190 con 0x7f17e00e5f30 2024-09-15T21:46:34.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.581+0000 7f2f0d307640 1 -- 172.21.15.100:0/3284339436 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8032810 con 0x7f2f080c9580 2024-09-15T21:46:34.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.582+0000 7fc960a4b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc954048950 0x7fc9540d6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.581+0000 7f17e759e640 1 -- 172.21.15.100:0/3302468781 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17e00e57d0 con 0x7f17e00e5f30 2024-09-15T21:46:34.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.582+0000 7fc95a59c640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fc95404c5c0 0x7fc9540dda80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.100:46860/0 (socket says 172.21.15.100:46860) 2024-09-15T21:46:34.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.582+0000 7fc95a59c640 1 -- 172.21.15.100:0/134824338 learned_addr learned my addr 172.21.15.100:0/134824338 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:34.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.582+0000 7fac53d11640 1 -- 172.21.15.100:0/40970190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac34005290 con 0x7fac4400bd90 2024-09-15T21:46:34.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.582+0000 7f17e759e640 1 -- 172.21.15.100:0/3302468781 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17e0005a00 con 0x7f17e00e5f30 2024-09-15T21:46:34.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.582+0000 7fc95b59e640 1 --2- 172.21.15.100:0/134824338 >> v2:172.21.15.122:3300/0 conn(0x7fc954048950 0x7fc9540d6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.582+0000 7fc960a4b640 1 -- 172.21.15.100:0/134824338 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc95404b240 con 0x7fc95404c5c0 2024-09-15T21:46:34.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.582+0000 7f17d6ffd640 1 -- 172.21.15.100:0/3302468781 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17e80b5040 con 0x7f17e00e5f30 2024-09-15T21:46:34.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.582+0000 7fb3da574640 1 -- 172.21.15.100:0/1498898860 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3b4005290 con 0x7fb3cc0d8870 2024-09-15T21:46:34.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.582+0000 7fc960a4b640 1 -- 172.21.15.100:0/134824338 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc954051410 con 0x7fc954048570 2024-09-15T21:46:34.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.582+0000 7f17d6ffd640 1 -- 172.21.15.100:0/3302468781 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e80b4070 con 0x7f17e00e5f30 2024-09-15T21:46:34.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.582+0000 7fc960a4b640 1 -- 172.21.15.100:0/134824338 --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fc9540dd5d0 con 0x7fc954048950 2024-09-15T21:46:34.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.582+0000 7f2e45d9b640 1 -- 172.21.15.100:0/2464307004 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e38032640 con 0x7f2e48152fe0 2024-09-15T21:46:34.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.582+0000 7f2e45d9b640 1 -- 172.21.15.100:0/2464307004 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38032a10 con 0x7f2e48152fe0 2024-09-15T21:46:34.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.583+0000 7f0ee6ffd640 1 -- 172.21.15.100:0/2237381404 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0ee8059720 con 0x7f0ef80c7a00 2024-09-15T21:46:34.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.583+0000 7f2f1056d640 1 -- 172.21.15.100:0/3284339436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f080c9580 msgr2=0x7f2f08158760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.583+0000 7f2f1056d640 1 --2- 172.21.15.100:0/3284339436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f080c9580 0x7f2f08158760 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2f080cb740 tx=0x7f2ef802a250 comp rx=0 tx=0).stop 2024-09-15T21:46:34.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.582+0000 7fc95ad9d640 1 --2- 172.21.15.100:0/134824338 >> v2:172.21.15.100:3300/0 conn(0x7fc954048570 0x7fc9540dd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.583+0000 7f2f1056d640 1 -- 172.21.15.100:0/3284339436 shutdown_connections 2024-09-15T21:46:34.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.583+0000 7fc959d9b640 1 -- 172.21.15.100:0/134824338 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3209786260 0 0) 0x7fc95404b240 con 0x7fc95404c5c0 2024-09-15T21:46:34.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.583+0000 7fc959d9b640 1 -- 172.21.15.100:0/134824338 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc944003520 con 0x7fc95404c5c0 2024-09-15T21:46:34.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.583+0000 7f2f1056d640 1 --2- 172.21.15.100:0/3284339436 >> v2:172.21.15.122:3300/0 conn(0x7f2f08072500 0x7f2f0806d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.583+0000 7f2f1056d640 1 --2- 172.21.15.100:0/3284339436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f080c9580 0x7f2f08158760 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.583+0000 7f2f1056d640 1 -- 172.21.15.100:0/3284339436 >> 172.21.15.100:0/3284339436 conn(0x7f2f080ca390 msgr2=0x7f2f080ca770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.583+0000 7fc95b59e640 1 -- 172.21.15.100:0/134824338 >> v2:172.21.15.100:3300/0 conn(0x7fc954048570 msgr2=0x7fc9540dd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:34.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.583+0000 7f2f1056d640 1 -- 172.21.15.100:0/3284339436 shutdown_connections 2024-09-15T21:46:34.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.583+0000 7f2f1056d640 1 -- 172.21.15.100:0/3284339436 wait complete. 2024-09-15T21:46:34.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.583+0000 7f2e4cc4c640 1 -- 172.21.15.100:0/2464307004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e48152fe0 msgr2=0x7f2e4815ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.583+0000 7f2e4cc4c640 1 --2- 172.21.15.100:0/2464307004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e48152fe0 0x7f2e4815ccc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2e380097a0 tx=0x7f2e3802a2c0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.583+0000 7f2e4cc4c640 1 -- 172.21.15.100:0/2464307004 shutdown_connections 2024-09-15T21:46:34.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.584+0000 7f2e4cc4c640 1 --2- 172.21.15.100:0/2464307004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e48152fe0 0x7f2e4815ccc0 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2e380097a0 tx=0x7f2e3802a2c0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.584+0000 7f2e4cc4c640 1 -- 172.21.15.100:0/2464307004 >> 172.21.15.100:0/2464307004 conn(0x7f2e48067dd0 msgr2=0x7f2e4806bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.584+0000 7fc959d9b640 1 -- 172.21.15.100:0/134824338 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3749518245 0 0) 0x7fc944003520 con 0x7fc95404c5c0 2024-09-15T21:46:34.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.584+0000 7f2e4cc4c640 1 -- 172.21.15.100:0/2464307004 shutdown_connections 2024-09-15T21:46:34.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.584+0000 7fc95b59e640 1 --2- 172.21.15.100:0/134824338 >> v2:172.21.15.100:3300/0 conn(0x7fc954048570 0x7fc9540dd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.584+0000 7fc95b59e640 1 -- 172.21.15.100:0/134824338 >> v1:172.21.15.122:6789/0 conn(0x7fc95404c5c0 legacy=0x7fc9540dda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.584+0000 7fc95b59e640 1 -- 172.21.15.100:0/134824338 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95404b520 con 0x7fc954048950 2024-09-15T21:46:34.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.585+0000 7fc95ad9d640 1 --2- 172.21.15.100:0/134824338 >> v2:172.21.15.100:3300/0 conn(0x7fc954048570 0x7fc9540dd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:46:34.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.585+0000 7f2f1056d640 1 Processor -- start 2024-09-15T21:46:34.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.585+0000 7fb3c9ffb640 1 -- 172.21.15.100:0/1498898860 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb3d001a070 con 0x7fb3cc0d8870 2024-09-15T21:46:34.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.585+0000 7f2e4cc4c640 1 -- 172.21.15.100:0/2464307004 wait complete. 2024-09-15T21:46:34.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.586+0000 7f2f1056d640 1 -- start start 2024-09-15T21:46:34.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.586+0000 7fb3c9ffb640 1 --2- 172.21.15.100:0/1498898860 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb3c006c780 0x7fb3c006ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.586+0000 7fb3c9ffb640 1 -- 172.21.15.100:0/1498898860 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb3d008b0b0 con 0x7fb3cc0d8870 2024-09-15T21:46:34.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.585+0000 7f17d6ffd640 1 -- 172.21.15.100:0/3302468781 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f17e80c0020 con 0x7f17e00e5f30 2024-09-15T21:46:34.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.585+0000 7f17d6ffd640 1 --2- 172.21.15.100:0/3302468781 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f17c806c2b0 0x7f17c806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.585+0000 7f17d6ffd640 1 -- 172.21.15.100:0/3302468781 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f17e8128ac0 con 0x7f17e00e5f30 2024-09-15T21:46:34.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.586+0000 7f17e659c640 1 --2- 172.21.15.100:0/3302468781 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f17c806c2b0 0x7f17c806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.586+0000 7fb3d9572640 1 --2- 172.21.15.100:0/1498898860 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb3c006c780 0x7fb3c006ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7fb3d9572640 1 --2- 172.21.15.100:0/1498898860 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb3c006c780 0x7fb3c006ec40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb3d406aef0 tx=0x7fb3d406aff0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.586+0000 7fac427fc640 1 -- 172.21.15.100:0/40970190 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fac4c06d110 con 0x7fac4400bd90 2024-09-15T21:46:34.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.586+0000 7fac427fc640 1 --2- 172.21.15.100:0/40970190 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fac3006c2b0 0x7fac3006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7fac512ac640 1 --2- 172.21.15.100:0/40970190 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fac3006c2b0 0x7fac3006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7fac427fc640 1 -- 172.21.15.100:0/40970190 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fac4c107eb0 con 0x7fac4400bd90 2024-09-15T21:46:34.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7fc95b59e640 1 --2- 172.21.15.100:0/134824338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc954048950 0x7fc9540d6330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc95c06ac10 tx=0x7fc95c0a70d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae6e986d83fad844 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7f2f1056d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f08072500 0x7f2f0814bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7f2f1056d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f080c9580 0x7f2f0814f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7f2f1056d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f08151f30 0x7f2f081f40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7f2f1056d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2f080c8d80 con 0x7f2f08072500 2024-09-15T21:46:34.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7f2f1056d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2f080c8aa0 con 0x7f2f080c9580 2024-09-15T21:46:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7f2f1056d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2f080c9060 con 0x7f2f08151f30 2024-09-15T21:46:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7fc959d9b640 1 -- 172.21.15.100:0/134824338 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c0b4070 con 0x7fc954048950 2024-09-15T21:46:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7fc959d9b640 1 -- 172.21.15.100:0/134824338 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc95c0a7c20 con 0x7fc954048950 2024-09-15T21:46:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7fc959d9b640 1 -- 172.21.15.100:0/134824338 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950001670 con 0x7fc954048950 2024-09-15T21:46:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7f2f0eb0a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f08151f30 0x7f2f081f40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7f2f0e309640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f08072500 0x7f2f0814bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7f2f0e309640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f08072500 0x7f2f0814bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44056/0 (socket says 172.21.15.100:44056) 2024-09-15T21:46:34.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7f2f0e309640 1 -- 172.21.15.100:0/3633093806 learned_addr learned my addr 172.21.15.100:0/3633093806 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7f2f0db08640 1 --2- 172.21.15.100:0/3633093806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f080c9580 0x7f2f0814f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7f2e4cc4c640 1 Processor -- start 2024-09-15T21:46:34.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7f2e4cc4c640 1 -- start start 2024-09-15T21:46:34.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.587+0000 7f17e659c640 1 --2- 172.21.15.100:0/3302468781 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f17c806c2b0 0x7f17c806e770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0017f0 tx=0x7f17dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7fac512ac640 1 --2- 172.21.15.100:0/40970190 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fac3006c2b0 0x7fac3006e770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fac440e3c30 tx=0x7fac3c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7f2e4cc4c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e480730f0 0x7f2e4815b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7f2f0eb0a640 1 -- 172.21.15.100:0/3633093806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f080c9580 msgr2=0x7f2f0814f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7f2f0eb0a640 1 --2- 172.21.15.100:0/3633093806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f080c9580 0x7f2f0814f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7f2f0eb0a640 1 -- 172.21.15.100:0/3633093806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f08072500 msgr2=0x7f2f0814bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7f2f0eb0a640 1 --2- 172.21.15.100:0/3633093806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f08072500 0x7f2f0814bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7f2f0eb0a640 1 -- 172.21.15.100:0/3633093806 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0814e1e0 con 0x7f2f08151f30 2024-09-15T21:46:34.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7f2e4cc4c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2e4815d200 0x7f2e481f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7f2e4cc4c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e48155d90 0x7f2e481f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.636 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7f2f0eb0a640 1 --2- 172.21.15.100:0/3633093806 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f08151f30 0x7f2f081f40a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000be50 tx=0x7f2f0000bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7f2e4659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e480730f0 0x7f2e4815b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7f2e4659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e480730f0 0x7f2e4815b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:41192/0 (socket says 172.21.15.100:41192) 2024-09-15T21:46:34.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7fc960a4b640 1 -- 172.21.15.100:0/134824338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc954048950 msgr2=0x7fc9540d6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7fc960a4b640 1 --2- 172.21.15.100:0/134824338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc954048950 0x7fc9540d6330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc95c06ac10 tx=0x7fc95c0a70d0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7f2e46d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e48155d90 0x7f2e481f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7f2e46d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e48155d90 0x7f2e481f9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44062/0 (socket says 172.21.15.100:44062) 2024-09-15T21:46:34.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7f2e4cc4c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2e480c44f0 con 0x7f2e48155d90 2024-09-15T21:46:34.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7f2e4cc4c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4815c830 con 0x7f2e480730f0 2024-09-15T21:46:34.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7f2e4cc4c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2e48158ea0 con 0x7f2e4815d200 2024-09-15T21:46:34.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7f2e4659c640 1 -- 172.21.15.100:0/3174154928 learned_addr learned my addr 172.21.15.100:0/3174154928 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7fc960a4b640 1 -- 172.21.15.100:0/134824338 shutdown_connections 2024-09-15T21:46:34.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7fc960a4b640 1 --2- 172.21.15.100:0/134824338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc954048950 0x7fc9540d6330 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7fc960a4b640 1 --2- 172.21.15.100:0/134824338 >> v2:172.21.15.100:3300/0 conn(0x7fc954048570 0x7fc9540dd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7fc960a4b640 1 -- 172.21.15.100:0/134824338 >> 172.21.15.100:0/134824338 conn(0x7fc954050560 msgr2=0x7fc954050940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.588+0000 7f2e3ffff640 1 --2- 172.21.15.100:0/3174154928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2e4815d200 0x7f2e481f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.589+0000 7f2efeffd640 1 -- 172.21.15.100:0/3633093806 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f00019070 con 0x7f2f08151f30 2024-09-15T21:46:34.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.589+0000 7f2efeffd640 1 -- 172.21.15.100:0/3633093806 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f00014440 con 0x7f2f08151f30 2024-09-15T21:46:34.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.589+0000 7f2f1056d640 1 -- 172.21.15.100:0/3633093806 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f081f4960 con 0x7f2f08151f30 2024-09-15T21:46:34.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.589+0000 7f2f1056d640 1 -- 172.21.15.100:0/3633093806 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f081e67f0 con 0x7f2f08151f30 2024-09-15T21:46:34.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.589+0000 7f2e4659c640 1 -- 172.21.15.100:0/3174154928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2e4815d200 msgr2=0x7f2e481f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.589+0000 7f2e4659c640 1 --2- 172.21.15.100:0/3174154928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2e4815d200 0x7f2e481f8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.589+0000 7f2e4659c640 1 -- 172.21.15.100:0/3174154928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e48155d90 msgr2=0x7f2e481f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.589+0000 7f2e4659c640 1 --2- 172.21.15.100:0/3174154928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e48155d90 0x7f2e481f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.589+0000 7f2e4659c640 1 -- 172.21.15.100:0/3174154928 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e481e3880 con 0x7f2e480730f0 2024-09-15T21:46:34.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.589+0000 7f2e3ffff640 1 --2- 172.21.15.100:0/3174154928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2e4815d200 0x7f2e481f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:34.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.589+0000 7fc960a4b640 1 -- 172.21.15.100:0/134824338 shutdown_connections 2024-09-15T21:46:34.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.590+0000 7f2efeffd640 1 -- 172.21.15.100:0/3633093806 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f00014830 con 0x7f2f08151f30 2024-09-15T21:46:34.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.589+0000 7fc960a4b640 1 -- 172.21.15.100:0/134824338 wait complete. 2024-09-15T21:46:34.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.589+0000 7f2e4659c640 1 --2- 172.21.15.100:0/3174154928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e480730f0 0x7f2e4815b320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2e3802a830 tx=0x7f2e380046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.589+0000 7f2e3effd640 1 -- 172.21.15.100:0/3174154928 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38041020 con 0x7f2e480730f0 2024-09-15T21:46:34.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.589+0000 7f2e3effd640 1 -- 172.21.15.100:0/3174154928 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e38030970 con 0x7f2e480730f0 2024-09-15T21:46:34.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.589+0000 7f2e3effd640 1 -- 172.21.15.100:0/3174154928 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38037040 con 0x7f2e480730f0 2024-09-15T21:46:34.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.589+0000 7f2e4cc4c640 1 -- 172.21.15.100:0/3174154928 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e481e3b20 con 0x7f2e480730f0 2024-09-15T21:46:34.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.589+0000 7f2e4cc4c640 1 -- 172.21.15.100:0/3174154928 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e481e40e0 con 0x7f2e480730f0 2024-09-15T21:46:34.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.591+0000 7f17d6ffd640 1 -- 172.21.15.100:0/3302468781 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f17e80c7030 con 0x7f17e00e5f30 2024-09-15T21:46:34.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.591+0000 7f2e3effd640 1 -- 172.21.15.100:0/3174154928 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2e38044020 con 0x7f2e480730f0 2024-09-15T21:46:34.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.591+0000 7f2efeffd640 1 -- 172.21.15.100:0/3633093806 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2f0001c360 con 0x7f2f08151f30 2024-09-15T21:46:34.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.592+0000 7fc960a4b640 1 Processor -- start 2024-09-15T21:46:34.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.592+0000 7fc960a4b640 1 -- start start 2024-09-15T21:46:34.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.592+0000 7f2e3effd640 1 --2- 172.21.15.100:0/3174154928 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2e2806c2b0 0x7f2e2806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.592+0000 7f2e3effd640 1 -- 172.21.15.100:0/3174154928 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2e380abc90 con 0x7f2e480730f0 2024-09-15T21:46:34.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.592+0000 7f2e4cc4c640 1 -- 172.21.15.100:0/3174154928 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e2c005290 con 0x7f2e480730f0 2024-09-15T21:46:34.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.592+0000 7f2efeffd640 1 --2- 172.21.15.100:0/3633093806 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2eec06c3b0 0x7f2eec06e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.592+0000 7f2f1056d640 1 -- 172.21.15.100:0/3633093806 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef0005290 con 0x7f2f08151f30 2024-09-15T21:46:34.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.592+0000 7f2f0e309640 1 --2- 172.21.15.100:0/3633093806 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2eec06c3b0 0x7f2eec06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.593+0000 7f2e3ffff640 1 --2- 172.21.15.100:0/3174154928 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2e2806c2b0 0x7f2e2806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.593+0000 7f2efeffd640 1 -- 172.21.15.100:0/3633093806 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2f0008b100 con 0x7f2f08151f30 2024-09-15T21:46:34.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.593+0000 7f2f0e309640 1 --2- 172.21.15.100:0/3633093806 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2eec06c3b0 0x7f2eec06e870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2f04005880 tx=0x7f2f04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.593+0000 7f2e3ffff640 1 --2- 172.21.15.100:0/3174154928 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2e2806c2b0 0x7f2e2806e770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e480c79c0 tx=0x7f2e30009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.594+0000 7fc960a4b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc954048570 0x7fc9540da980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.594+0000 7fc960a4b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc954048950 0x7fc9540db930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.594+0000 7fc95ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc954048570 0x7fc9540da980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.594+0000 7fc95ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc954048570 0x7fc9540da980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44066/0 (socket says 172.21.15.100:44066) 2024-09-15T21:46:34.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.594+0000 7fc95ad9d640 1 -- 172.21.15.100:0/1285519661 learned_addr learned my addr 172.21.15.100:0/1285519661 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:34.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.594+0000 7fc960a4b640 1 --2- 172.21.15.100:0/1285519661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc95404c5c0 0x7fc9540d2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.594+0000 7fc960a4b640 1 -- 172.21.15.100:0/1285519661 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc95404bae0 con 0x7fc954048570 2024-09-15T21:46:34.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.594+0000 7fc960a4b640 1 -- 172.21.15.100:0/1285519661 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc95404c130 con 0x7fc95404c5c0 2024-09-15T21:46:34.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.594+0000 7fc960a4b640 1 -- 172.21.15.100:0/1285519661 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc95404bdc0 con 0x7fc954048950 2024-09-15T21:46:34.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.594+0000 7fc95a59c640 1 --2- 172.21.15.100:0/1285519661 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc954048950 0x7fc9540db930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.594+0000 7fc95b59e640 1 --2- 172.21.15.100:0/1285519661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc95404c5c0 0x7fc9540d2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.594+0000 7fc95a59c640 1 -- 172.21.15.100:0/1285519661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc95404c5c0 msgr2=0x7fc9540d2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.594+0000 7fc95a59c640 1 --2- 172.21.15.100:0/1285519661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc95404c5c0 0x7fc9540d2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.594+0000 7fc95a59c640 1 -- 172.21.15.100:0/1285519661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc954048570 msgr2=0x7fc9540da980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.594+0000 7fc95a59c640 1 --2- 172.21.15.100:0/1285519661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc954048570 0x7fc9540da980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.595+0000 7fc95a59c640 1 -- 172.21.15.100:0/1285519661 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9540dc2d0 con 0x7fc954048950 2024-09-15T21:46:34.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.595+0000 7fc95ad9d640 1 --2- 172.21.15.100:0/1285519661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc954048570 0x7fc9540da980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:46:34.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.595+0000 7fc95b59e640 1 --2- 172.21.15.100:0/1285519661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc95404c5c0 0x7fc9540d2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:46:34.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.595+0000 7fc95a59c640 1 --2- 172.21.15.100:0/1285519661 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc954048950 0x7fc9540db930 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc950000c00 tx=0x7fc95000c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.597+0000 7fc9437fe640 1 -- 172.21.15.100:0/1285519661 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc950017070 con 0x7fc954048950 2024-09-15T21:46:34.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.597+0000 7f2efeffd640 1 -- 172.21.15.100:0/3633093806 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2f000592d0 con 0x7f2f08151f30 2024-09-15T21:46:34.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.597+0000 7fc9437fe640 1 -- 172.21.15.100:0/1285519661 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc950005d00 con 0x7fc954048950 2024-09-15T21:46:34.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.597+0000 7fc9437fe640 1 -- 172.21.15.100:0/1285519661 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9500072b0 con 0x7fc954048950 2024-09-15T21:46:34.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.597+0000 7fc960a4b640 1 -- 172.21.15.100:0/1285519661 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc954163700 con 0x7fc954048950 2024-09-15T21:46:34.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.597+0000 7fc960a4b640 1 -- 172.21.15.100:0/1285519661 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc954166d60 con 0x7fc954048950 2024-09-15T21:46:34.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.598+0000 7f2e3effd640 1 -- 172.21.15.100:0/3174154928 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2e3804b030 con 0x7f2e480730f0 2024-09-15T21:46:34.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.598+0000 7fac427fc640 1 -- 172.21.15.100:0/40970190 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fac4c09c030 con 0x7fac4400bd90 2024-09-15T21:46:34.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.599+0000 7fc960a4b640 1 -- 172.21.15.100:0/1285519661 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc95404b240 con 0x7fc954048950 2024-09-15T21:46:34.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.600+0000 7fc9437fe640 1 -- 172.21.15.100:0/1285519661 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc95001a070 con 0x7fc954048950 2024-09-15T21:46:34.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.601+0000 7fc9437fe640 1 --2- 172.21.15.100:0/1285519661 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc94806c380 0x7fc94806e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:34.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.601+0000 7fc9437fe640 1 -- 172.21.15.100:0/1285519661 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc95008a6c0 con 0x7fc954048950 2024-09-15T21:46:34.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.602+0000 7fc95ad9d640 1 --2- 172.21.15.100:0/1285519661 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc94806c380 0x7fc94806e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:34.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.602+0000 7fc95ad9d640 1 --2- 172.21.15.100:0/1285519661 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc94806c380 0x7fc94806e840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009560 tx=0x7fc94c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:34.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.603+0000 7fb3c9ffb640 1 -- 172.21.15.100:0/1498898860 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb3d0059240 con 0x7fb3cc0d8870 2024-09-15T21:46:34.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.608+0000 7fc9437fe640 1 -- 172.21.15.100:0/1285519661 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc95404b240 con 0x7fc954048950 2024-09-15T21:46:34.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.976+0000 7f0ee4ff9640 1 -- 172.21.15.100:0/2237381404 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0edc005580 con 0x7f0ef80c7a00 2024-09-15T21:46:34.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.976+0000 7f0ee6ffd640 1 -- 172.21.15.100:0/2237381404 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0ee8059020 con 0x7f0ef80c7a00 2024-09-15T21:46:34.979 INFO:teuthology.orchestra.run.smithi100.stdout:64424509447 2024-09-15T21:46:34.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.978+0000 7f35b19fd640 1 -- 172.21.15.100:0/2153727740 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f358c005580 con 0x7f35ac157bb0 2024-09-15T21:46:34.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.979+0000 7f3593fff640 1 -- 172.21.15.100:0/2153727740 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f35a4015e20 con 0x7f35ac157bb0 2024-09-15T21:46:34.981 INFO:teuthology.orchestra.run.smithi100.stdout:60129542150 2024-09-15T21:46:34.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.980+0000 7f0efcd98640 1 -- 172.21.15.100:0/2237381404 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0ed806c330 msgr2=0x7f0ed806e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.980+0000 7f0efcd98640 1 --2- 172.21.15.100:0/2237381404 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0ed806c330 0x7f0ed806e7f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0eec034000 tx=0x7f0eec02e000 comp rx=0 tx=0).stop 2024-09-15T21:46:34.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.980+0000 7f0efcd98640 1 -- 172.21.15.100:0/2237381404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ef80c7a00 msgr2=0x7f0ef815b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.980+0000 7f0efcd98640 1 --2- 172.21.15.100:0/2237381404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ef80c7a00 0x7f0ef815b800 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800c450 tx=0x7f0ee800c910 comp rx=0 tx=0).stop 2024-09-15T21:46:34.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.980+0000 7f0efcd98640 1 -- 172.21.15.100:0/2237381404 shutdown_connections 2024-09-15T21:46:34.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.980+0000 7f0efcd98640 1 --2- 172.21.15.100:0/2237381404 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0ed806c330 0x7f0ed806e7f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.980+0000 7f0efcd98640 1 --2- 172.21.15.100:0/2237381404 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0ef80ca2e0 0x7f0ef815cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.981+0000 7f0efcd98640 1 --2- 172.21.15.100:0/2237381404 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0ef80c9f00 0x7f0ef81f8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.981+0000 7f0efcd98640 1 --2- 172.21.15.100:0/2237381404 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ef80c7a00 0x7f0ef815b800 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.981+0000 7f0efcd98640 1 -- 172.21.15.100:0/2237381404 >> 172.21.15.100:0/2237381404 conn(0x7f0ef8073820 msgr2=0x7f0ef8161860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.981+0000 7f0efcd98640 1 -- 172.21.15.100:0/2237381404 shutdown_connections 2024-09-15T21:46:34.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.981+0000 7f0efcd98640 1 -- 172.21.15.100:0/2237381404 wait complete. 2024-09-15T21:46:34.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.982+0000 7f3591ffb640 1 -- 172.21.15.100:0/2153727740 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f359806c2e0 msgr2=0x7f359806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.982+0000 7f3591ffb640 1 --2- 172.21.15.100:0/2153727740 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f359806c2e0 0x7f359806e7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f35a002a5f0 tx=0x7f35a002e000 comp rx=0 tx=0).stop 2024-09-15T21:46:34.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.983+0000 7f17e759e640 1 -- 172.21.15.100:0/3302468781 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f17e004b980 con 0x7f17e00e5f30 2024-09-15T21:46:34.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.983+0000 7f3591ffb640 1 -- 172.21.15.100:0/2153727740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35ac157bb0 msgr2=0x7f35ac153850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.983+0000 7f3591ffb640 1 --2- 172.21.15.100:0/2153727740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35ac157bb0 0x7f35ac153850 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f35a40021d0 tx=0x7f35a400ced0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.983+0000 7f3591ffb640 1 -- 172.21.15.100:0/2153727740 shutdown_connections 2024-09-15T21:46:34.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.983+0000 7f17d6ffd640 1 -- 172.21.15.100:0/3302468781 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f17e80f6ab0 con 0x7f17e00e5f30 2024-09-15T21:46:34.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.983+0000 7f3591ffb640 1 --2- 172.21.15.100:0/2153727740 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f359806c2e0 0x7f359806e7a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.983+0000 7f3591ffb640 1 --2- 172.21.15.100:0/2153727740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f35ac158490 0x7f35ac153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.988 INFO:teuthology.orchestra.run.smithi100.stdout:64424509446 2024-09-15T21:46:34.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.983+0000 7f3591ffb640 1 --2- 172.21.15.100:0/2153727740 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f35ac157bb0 0x7f35ac153850 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.983+0000 7f3591ffb640 1 --2- 172.21.15.100:0/2153727740 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f35ac06c2f0 0x7f35ac155e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.984+0000 7f3591ffb640 1 -- 172.21.15.100:0/2153727740 >> 172.21.15.100:0/2153727740 conn(0x7f35ac067dd0 msgr2=0x7f35ac158ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.984+0000 7f3591ffb640 1 -- 172.21.15.100:0/2153727740 shutdown_connections 2024-09-15T21:46:34.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.984+0000 7f3591ffb640 1 -- 172.21.15.100:0/2153727740 wait complete. 2024-09-15T21:46:34.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.987+0000 7f17d4ff9640 1 -- 172.21.15.100:0/3302468781 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f17c806c2b0 msgr2=0x7f17c806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.987+0000 7f17d4ff9640 1 --2- 172.21.15.100:0/3302468781 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f17c806c2b0 0x7f17c806e770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f17dc0017f0 tx=0x7f17dc009040 comp rx=0 tx=0).stop 2024-09-15T21:46:34.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.987+0000 7f17d4ff9640 1 -- 172.21.15.100:0/3302468781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e00e5f30 msgr2=0x7f17e00d68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:34.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.987+0000 7f17d4ff9640 1 --2- 172.21.15.100:0/3302468781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e00e5f30 0x7f17e00d68f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f17e8000b60 tx=0x7f17e8069fa0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.988+0000 7f17d4ff9640 1 -- 172.21.15.100:0/3302468781 shutdown_connections 2024-09-15T21:46:34.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.988+0000 7f17d4ff9640 1 --2- 172.21.15.100:0/3302468781 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f17c806c2b0 0x7f17c806e770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.988+0000 7f17d4ff9640 1 --2- 172.21.15.100:0/3302468781 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17e00e5f30 0x7f17e00d68f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.988+0000 7f17d4ff9640 1 --2- 172.21.15.100:0/3302468781 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f17e00dcd90 0x7f17e00d63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.988+0000 7f17d4ff9640 1 --2- 172.21.15.100:0/3302468781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17e004a680 0x7f17e016cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:34.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.988+0000 7f17d4ff9640 1 -- 172.21.15.100:0/3302468781 >> 172.21.15.100:0/3302468781 conn(0x7f17e004fc50 msgr2=0x7f17e00db630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:34.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.988+0000 7f17d4ff9640 1 -- 172.21.15.100:0/3302468781 shutdown_connections 2024-09-15T21:46:34.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.988+0000 7f17d4ff9640 1 -- 172.21.15.100:0/3302468781 wait complete. 2024-09-15T21:46:34.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.997+0000 7f2e4cc4c640 1 -- 172.21.15.100:0/3174154928 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f2e2c005580 con 0x7f2e480730f0 2024-09-15T21:46:35.000 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:34.998+0000 7f2e3effd640 1 -- 172.21.15.100:0/3174154928 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2e3807a550 con 0x7f2e480730f0 2024-09-15T21:46:35.000 INFO:teuthology.orchestra.run.smithi100.stdout:68719476741 2024-09-15T21:46:35.002 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.4 2024-09-15T21:46:35.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.001+0000 7f2e4cc4c640 1 -- 172.21.15.100:0/3174154928 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2e2806c2b0 msgr2=0x7f2e2806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:35.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.001+0000 7f2e4cc4c640 1 --2- 172.21.15.100:0/3174154928 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2e2806c2b0 0x7f2e2806e770 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2e480c79c0 tx=0x7f2e30009040 comp rx=0 tx=0).stop 2024-09-15T21:46:35.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.001+0000 7f2e4cc4c640 1 -- 172.21.15.100:0/3174154928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e480730f0 msgr2=0x7f2e4815b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:35.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.001+0000 7f2e4cc4c640 1 --2- 172.21.15.100:0/3174154928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e480730f0 0x7f2e4815b320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2e3802a830 tx=0x7f2e380046c0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.001+0000 7f2e4cc4c640 1 -- 172.21.15.100:0/3174154928 shutdown_connections 2024-09-15T21:46:35.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.001+0000 7f2e4cc4c640 1 --2- 172.21.15.100:0/3174154928 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2e2806c2b0 0x7f2e2806e770 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.001+0000 7f2e4cc4c640 1 --2- 172.21.15.100:0/3174154928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2e48155d90 0x7f2e481f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.001+0000 7f2e4cc4c640 1 --2- 172.21.15.100:0/3174154928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2e4815d200 0x7f2e481f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.001+0000 7f2e4cc4c640 1 --2- 172.21.15.100:0/3174154928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2e480730f0 0x7f2e4815b320 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.001+0000 7f2e4cc4c640 1 -- 172.21.15.100:0/3174154928 >> 172.21.15.100:0/3174154928 conn(0x7f2e48067dd0 msgr2=0x7f2e48158760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:35.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.001+0000 7f2e4cc4c640 1 -- 172.21.15.100:0/3174154928 shutdown_connections 2024-09-15T21:46:35.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.001+0000 7f2e4cc4c640 1 -- 172.21.15.100:0/3174154928 wait complete. 2024-09-15T21:46:35.006 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.1 2024-09-15T21:46:35.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.002+0000 7fb3da574640 1 -- 172.21.15.100:0/1498898860 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fb3b4005850 con 0x7fb3cc0d8870 2024-09-15T21:46:35.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.003+0000 7fb3c9ffb640 1 -- 172.21.15.100:0/1498898860 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb3d005d540 con 0x7fb3cc0d8870 2024-09-15T21:46:35.007 INFO:teuthology.orchestra.run.smithi100.stdout:55834574854 2024-09-15T21:46:35.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.005+0000 7fac53d11640 1 -- 172.21.15.100:0/40970190 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fac34005850 con 0x7fac4400bd90 2024-09-15T21:46:35.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.005+0000 7fac427fc640 1 -- 172.21.15.100:0/40970190 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fac4c0d6790 con 0x7fac4400bd90 2024-09-15T21:46:35.008 INFO:teuthology.orchestra.run.smithi100.stdout:64424509446 2024-09-15T21:46:35.009 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.7 2024-09-15T21:46:35.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.005+0000 7fc960a4b640 1 -- 172.21.15.100:0/1285519661 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc95404aee0 con 0x7fc954048950 2024-09-15T21:46:35.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.006+0000 7fb3da574640 1 -- 172.21.15.100:0/1498898860 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb3c006c780 msgr2=0x7fb3c006ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:35.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.006+0000 7fb3da574640 1 --2- 172.21.15.100:0/1498898860 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb3c006c780 0x7fb3c006ec40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb3d406aef0 tx=0x7fb3d406aff0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.006+0000 7fb3da574640 1 -- 172.21.15.100:0/1498898860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3cc0d8870 msgr2=0x7fb3cc0d6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:35.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.006+0000 7fb3da574640 1 --2- 172.21.15.100:0/1498898860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3cc0d8870 0x7fb3cc0d6b40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c3c0 tx=0x7fb3d0001e70 comp rx=0 tx=0).stop 2024-09-15T21:46:35.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.006+0000 7fb3da574640 1 -- 172.21.15.100:0/1498898860 shutdown_connections 2024-09-15T21:46:35.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.006+0000 7fb3da574640 1 --2- 172.21.15.100:0/1498898860 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb3c006c780 0x7fb3c006ec40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.006+0000 7fb3da574640 1 --2- 172.21.15.100:0/1498898860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb3cc0502d0 0x7fb3cc038b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.006+0000 7fb3da574640 1 --2- 172.21.15.100:0/1498898860 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb3cc0d8870 0x7fb3cc0d6b40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.006+0000 7fb3da574640 1 --2- 172.21.15.100:0/1498898860 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb3cc0491c0 0x7fb3cc038610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.006+0000 7fb3da574640 1 -- 172.21.15.100:0/1498898860 >> 172.21.15.100:0/1498898860 conn(0x7fb3cc054450 msgr2=0x7fb3cc0dca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:35.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.006+0000 7fc9437fe640 1 -- 172.21.15.100:0/1285519661 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc950058850 con 0x7fc954048950 2024-09-15T21:46:35.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.006+0000 7fb3da574640 1 -- 172.21.15.100:0/1498898860 shutdown_connections 2024-09-15T21:46:35.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.006+0000 7fb3da574640 1 -- 172.21.15.100:0/1498898860 wait complete. 2024-09-15T21:46:35.012 INFO:teuthology.orchestra.run.smithi100.stdout:64424509445 2024-09-15T21:46:35.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.007+0000 7f2f1056d640 1 -- 172.21.15.100:0/3633093806 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f2ef0005580 con 0x7f2f08151f30 2024-09-15T21:46:35.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.007+0000 7f2efeffd640 1 -- 172.21.15.100:0/3633093806 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2f0005d5d0 con 0x7f2f08151f30 2024-09-15T21:46:35.013 INFO:teuthology.orchestra.run.smithi100.stdout:64424509448 2024-09-15T21:46:35.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.008+0000 7fac53d11640 1 -- 172.21.15.100:0/40970190 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fac3006c2b0 msgr2=0x7fac3006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:35.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.008+0000 7fac53d11640 1 --2- 172.21.15.100:0/40970190 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fac3006c2b0 0x7fac3006e770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fac440e3c30 tx=0x7fac3c008040 comp rx=0 tx=0).stop 2024-09-15T21:46:35.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.008+0000 7fac53d11640 1 -- 172.21.15.100:0/40970190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac4400bd90 msgr2=0x7fac44036b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:35.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.008+0000 7fac53d11640 1 --2- 172.21.15.100:0/40970190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac4400bd90 0x7fac44036b60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fac4c06b960 tx=0x7fac4c069a20 comp rx=0 tx=0).stop 2024-09-15T21:46:35.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fc960a4b640 1 -- 172.21.15.100:0/1285519661 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc94806c380 msgr2=0x7fc94806e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:35.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fc960a4b640 1 --2- 172.21.15.100:0/1285519661 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc94806c380 0x7fc94806e840 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009560 tx=0x7fc94c009190 comp rx=0 tx=0).stop 2024-09-15T21:46:35.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fac53d11640 1 -- 172.21.15.100:0/40970190 shutdown_connections 2024-09-15T21:46:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fac53d11640 1 --2- 172.21.15.100:0/40970190 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fac3006c2b0 0x7fac3006e770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fac53d11640 1 --2- 172.21.15.100:0/40970190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac440dcdc0 0x7fac440e1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fac53d11640 1 --2- 172.21.15.100:0/40970190 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fac4400d230 0x7fac440dd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fac53d11640 1 --2- 172.21.15.100:0/40970190 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fac4400bd90 0x7fac44036b60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fac53d11640 1 -- 172.21.15.100:0/40970190 >> 172.21.15.100:0/40970190 conn(0x7fac4400a160 msgr2=0x7fac4404fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:35.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fc960a4b640 1 -- 172.21.15.100:0/1285519661 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc954048950 msgr2=0x7fc9540db930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fc960a4b640 1 --2- 172.21.15.100:0/1285519661 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc954048950 0x7fc9540db930 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc950000c00 tx=0x7fc95000c960 comp rx=0 tx=0).stop 2024-09-15T21:46:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fc960a4b640 1 -- 172.21.15.100:0/1285519661 shutdown_connections 2024-09-15T21:46:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fc960a4b640 1 --2- 172.21.15.100:0/1285519661 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc94806c380 0x7fc94806e840 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fc960a4b640 1 --2- 172.21.15.100:0/1285519661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc95404c5c0 0x7fc9540d2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fc960a4b640 1 --2- 172.21.15.100:0/1285519661 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc954048950 0x7fc9540db930 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fc960a4b640 1 --2- 172.21.15.100:0/1285519661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc954048570 0x7fc9540da980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fc960a4b640 1 -- 172.21.15.100:0/1285519661 >> 172.21.15.100:0/1285519661 conn(0x7fc954050560 msgr2=0x7fc954045e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:35.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fac53d11640 1 -- 172.21.15.100:0/40970190 shutdown_connections 2024-09-15T21:46:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fac53d11640 1 -- 172.21.15.100:0/40970190 wait complete. 2024-09-15T21:46:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fc960a4b640 1 -- 172.21.15.100:0/1285519661 shutdown_connections 2024-09-15T21:46:35.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.009+0000 7fc960a4b640 1 -- 172.21.15.100:0/1285519661 wait complete. 2024-09-15T21:46:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.010+0000 7f2f1056d640 1 -- 172.21.15.100:0/3633093806 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2eec06c3b0 msgr2=0x7f2eec06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.010+0000 7f2f1056d640 1 --2- 172.21.15.100:0/3633093806 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2eec06c3b0 0x7f2eec06e870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2f04005880 tx=0x7f2f04009040 comp rx=0 tx=0).stop 2024-09-15T21:46:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.010+0000 7f2f1056d640 1 -- 172.21.15.100:0/3633093806 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f08151f30 msgr2=0x7f2f081f40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.010+0000 7f2f1056d640 1 --2- 172.21.15.100:0/3633093806 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f08151f30 0x7f2f081f40a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000be50 tx=0x7f2f0000bf40 comp rx=0 tx=0).stop 2024-09-15T21:46:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.010+0000 7f2f1056d640 1 -- 172.21.15.100:0/3633093806 shutdown_connections 2024-09-15T21:46:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.010+0000 7f2f1056d640 1 --2- 172.21.15.100:0/3633093806 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2eec06c3b0 0x7f2eec06e870 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.010+0000 7f2f1056d640 1 --2- 172.21.15.100:0/3633093806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f080c9580 0x7f2f0814f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.010+0000 7f2f1056d640 1 --2- 172.21.15.100:0/3633093806 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f08151f30 0x7f2f081f40a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.010+0000 7f2f1056d640 1 --2- 172.21.15.100:0/3633093806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f08072500 0x7f2f0814bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.010+0000 7f2f1056d640 1 -- 172.21.15.100:0/3633093806 >> 172.21.15.100:0/3633093806 conn(0x7f2f080ca390 msgr2=0x7f2f0806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:35.019 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.010+0000 7f2f1056d640 1 -- 172.21.15.100:0/3633093806 shutdown_connections 2024-09-15T21:46:35.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:35.010+0000 7f2f1056d640 1 -- 172.21.15.100:0/3633093806 wait complete. 2024-09-15T21:46:35.020 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476743 got 68719476741 for osd.0 2024-09-15T21:46:35.028 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2024-09-15T21:46:35.029 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.2 2024-09-15T21:46:35.029 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.5 2024-09-15T21:46:35.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.6 2024-09-15T21:46:36.003 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T21:46:36.007 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:46:36.010 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T21:46:36.022 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:46:36.028 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:46:36.029 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:46:36.031 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T21:46:36.031 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T21:46:36.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.162+0000 7ff971952640 1 Processor -- start 2024-09-15T21:46:36.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.163+0000 7ff971952640 1 -- start start 2024-09-15T21:46:36.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.164+0000 7ff971952640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff96c151270 0x7ff96c0c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.164+0000 7ff971952640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff96c0cc620 0x7ff96c0cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.164+0000 7ff971952640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff96c151650 con 0x7ff96c157bb0 2024-09-15T21:46:36.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.164+0000 7ff971952640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff96c0c8420 con 0x7ff96c0cc620 2024-09-15T21:46:36.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.164+0000 7ff971952640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7ff96c0c9200 con 0x7ff96c151270 2024-09-15T21:46:36.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.165+0000 7ff96a7fc640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff96c157bb0 0x7ff96c158490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:51930/0 (socket says 172.21.15.100:51930) 2024-09-15T21:46:36.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.165+0000 7ff96b7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff96c0cc620 0x7ff96c0cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.165+0000 7ff96b7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff96c0cc620 0x7ff96c0cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44078/0 (socket says 172.21.15.100:44078) 2024-09-15T21:46:36.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.165+0000 7ff96b7fe640 1 -- 172.21.15.100:0/3955320841 learned_addr learned my addr 172.21.15.100:0/3955320841 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:36.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.166+0000 7ff96b7fe640 1 -- 172.21.15.100:0/3955320841 >> v2:172.21.15.122:3300/0 conn(0x7ff96c151270 msgr2=0x7ff96c0c77d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:46:36.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.166+0000 7ff96b7fe640 1 --2- 172.21.15.100:0/3955320841 >> v2:172.21.15.122:3300/0 conn(0x7ff96c151270 0x7ff96c0c77d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.166+0000 7ff96b7fe640 1 -- 172.21.15.100:0/3955320841 >> v1:172.21.15.100:6789/0 conn(0x7ff96c157bb0 legacy=0x7ff96c158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.166+0000 7ff96b7fe640 1 -- 172.21.15.100:0/3955320841 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c0c94e0 con 0x7ff96c0cc620 2024-09-15T21:46:36.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.167+0000 7ff96b7fe640 1 --2- 172.21.15.100:0/3955320841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff96c0cc620 0x7ff96c0cca50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0cd750 tx=0x7ff96402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29820c848c0019ad server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.169+0000 7ff969ffb640 1 -- 172.21.15.100:0/3955320841 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964036070 con 0x7ff96c0cc620 2024-09-15T21:46:36.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.169+0000 7ff969ffb640 1 -- 172.21.15.100:0/3955320841 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff96402ae50 con 0x7ff96c0cc620 2024-09-15T21:46:36.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.170+0000 7ff969ffb640 1 -- 172.21.15.100:0/3955320841 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff964031820 con 0x7ff96c0cc620 2024-09-15T21:46:36.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.170+0000 7ff971952640 1 -- 172.21.15.100:0/3955320841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff96c0cc620 msgr2=0x7ff96c0cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.170+0000 7ff971952640 1 --2- 172.21.15.100:0/3955320841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff96c0cc620 0x7ff96c0cca50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0cd750 tx=0x7ff96402a6f0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.170+0000 7ff971952640 1 -- 172.21.15.100:0/3955320841 shutdown_connections 2024-09-15T21:46:36.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.170+0000 7ff971952640 1 --2- 172.21.15.100:0/3955320841 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff96c0cc620 0x7ff96c0cca50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.170+0000 7ff971952640 1 --2- 172.21.15.100:0/3955320841 >> v2:172.21.15.122:3300/0 conn(0x7ff96c151270 0x7ff96c0c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.170+0000 7ff971952640 1 -- 172.21.15.100:0/3955320841 >> 172.21.15.100:0/3955320841 conn(0x7ff96c067dd0 msgr2=0x7ff96c06bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:36.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.171+0000 7ff971952640 1 -- 172.21.15.100:0/3955320841 shutdown_connections 2024-09-15T21:46:36.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.172+0000 7ff971952640 1 -- 172.21.15.100:0/3955320841 wait complete. 2024-09-15T21:46:36.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.173+0000 7ff971952640 1 Processor -- start 2024-09-15T21:46:36.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.174+0000 7ff971952640 1 -- start start 2024-09-15T21:46:36.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.175+0000 7ff971952640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff96c0cc620 0x7ff96c153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.175+0000 7ff971952640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff96c151270 0x7ff96c153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.175+0000 7ff971952640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff96c157bb0 0x7ff96c15a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.175+0000 7ff971952640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c0c7d10 con 0x7ff96c151270 2024-09-15T21:46:36.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.175+0000 7ff971952640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c0c9aa0 con 0x7ff96c0cc620 2024-09-15T21:46:36.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.175+0000 7ff971952640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff96c158ba0 con 0x7ff96c157bb0 2024-09-15T21:46:36.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.176+0000 7ff96a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff96c151270 0x7ff96c153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.176+0000 7ff96a7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff96c151270 0x7ff96c153960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44080/0 (socket says 172.21.15.100:44080) 2024-09-15T21:46:36.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.176+0000 7ff96a7fc640 1 -- 172.21.15.100:0/3603280464 learned_addr learned my addr 172.21.15.100:0/3603280464 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:36.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.176+0000 7ff96b7fe640 1 --2- 172.21.15.100:0/3603280464 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff96c157bb0 0x7ff96c15a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.176+0000 7ff96affd640 1 --2- 172.21.15.100:0/3603280464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff96c0cc620 0x7ff96c153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.176+0000 7ff96a7fc640 1 -- 172.21.15.100:0/3603280464 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff96c157bb0 msgr2=0x7ff96c15a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.177+0000 7ff96a7fc640 1 --2- 172.21.15.100:0/3603280464 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff96c157bb0 0x7ff96c15a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.177+0000 7ff96a7fc640 1 -- 172.21.15.100:0/3603280464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff96c0cc620 msgr2=0x7ff96c153420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.177+0000 7ff96a7fc640 1 --2- 172.21.15.100:0/3603280464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff96c0cc620 0x7ff96c153420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.177+0000 7ff96a7fc640 1 -- 172.21.15.100:0/3603280464 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c1fe8e0 con 0x7ff96c151270 2024-09-15T21:46:36.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.177+0000 7ff96affd640 1 --2- 172.21.15.100:0/3603280464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff96c0cc620 0x7ff96c153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:36.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.177+0000 7ff96a7fc640 1 --2- 172.21.15.100:0/3603280464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff96c151270 0x7ff96c153960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff96000c7b0 tx=0x7ff96000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.178+0000 7ff953fff640 1 -- 172.21.15.100:0/3603280464 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff960018070 con 0x7ff96c151270 2024-09-15T21:46:36.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.178+0000 7ff971952640 1 -- 172.21.15.100:0/3603280464 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96c159b30 con 0x7ff96c151270 2024-09-15T21:46:36.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.178+0000 7ff971952640 1 -- 172.21.15.100:0/3603280464 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96c159dd0 con 0x7ff96c151270 2024-09-15T21:46:36.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.179+0000 7ff953fff640 1 -- 172.21.15.100:0/3603280464 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9600074e0 con 0x7ff96c151270 2024-09-15T21:46:36.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.179+0000 7ff953fff640 1 -- 172.21.15.100:0/3603280464 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96000ce60 con 0x7ff96c151270 2024-09-15T21:46:36.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.181+0000 7ff953fff640 1 -- 172.21.15.100:0/3603280464 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff96001b070 con 0x7ff96c151270 2024-09-15T21:46:36.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.181+0000 7ff953fff640 1 --2- 172.21.15.100:0/3603280464 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff95806c2b0 0x7ff95806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.181+0000 7ff971952640 1 -- 172.21.15.100:0/3603280464 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff96c0c7fe0 con 0x7ff96c151270 2024-09-15T21:46:36.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.181+0000 7ff953fff640 1 -- 172.21.15.100:0/3603280464 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7ff96008a9c0 con 0x7ff96c151270 2024-09-15T21:46:36.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.182+0000 7ff96affd640 1 --2- 172.21.15.100:0/3603280464 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff95806c2b0 0x7ff95806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.182+0000 7ff96affd640 1 --2- 172.21.15.100:0/3603280464 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff95806c2b0 0x7ff95806e770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00a900 tx=0x7ff95c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.191+0000 7ff953fff640 1 -- 172.21.15.100:0/3603280464 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff960058a40 con 0x7ff96c151270 2024-09-15T21:46:36.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.217+0000 7f8e10082640 1 Processor -- start 2024-09-15T21:46:36.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.217+0000 7f8e10082640 1 -- start start 2024-09-15T21:46:36.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.218+0000 7f724d903640 1 Processor -- start 2024-09-15T21:46:36.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.218+0000 7f8e10082640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8e080c97c0 0x7f8e08159470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.218+0000 7f8e10082640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9ba0 0x7f8e081599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.218+0000 7f8e10082640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f8e080b4da0 0x7f8e08073840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.218+0000 7f724d903640 1 -- start start 2024-09-15T21:46:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.218+0000 7f8e10082640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f8e080c3f50 con 0x7f8e080c97c0 2024-09-15T21:46:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.218+0000 7f8e10082640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f8e0806ed00 con 0x7f8e080b4da0 2024-09-15T21:46:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.218+0000 7f8e10082640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8e080c41c0 con 0x7f8e080c9ba0 2024-09-15T21:46:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.219+0000 7f8e0de1e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8e080c97c0 0x7f8e08159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.219+0000 7f8e0de1e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8e080c97c0 0x7f8e08159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44082/0 (socket says 172.21.15.100:44082) 2024-09-15T21:46:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.219+0000 7f8e0de1e640 1 -- 172.21.15.100:0/575489333 learned_addr learned my addr 172.21.15.100:0/575489333 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.219+0000 7f8e0d61d640 1 --2- 172.21.15.100:0/575489333 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9ba0 0x7f8e081599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.219+0000 7f8e0e61f640 1 --2- 172.21.15.100:0/575489333 >> v2:172.21.15.122:3300/0 conn(0x7f8e080b4da0 0x7f8e08073840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.219+0000 7f8e0d61d640 1 -- 172.21.15.100:0/575489333 >> v2:172.21.15.122:3300/0 conn(0x7f8e080b4da0 msgr2=0x7f8e08073840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:36.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.219+0000 7f724d903640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7248071150 0x7f72480c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.219+0000 7f724d903640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7248071530 0x7f72480c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.219+0000 7f724d903640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f72480c8650 0x7f724815f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.219+0000 7f724d903640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f7248163a40 con 0x7f7248071150 2024-09-15T21:46:36.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.219+0000 7f724d903640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f72481635b0 con 0x7f7248071530 2024-09-15T21:46:36.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.219+0000 7f724d903640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f72480c8bb0 con 0x7f72480c8650 2024-09-15T21:46:36.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.219+0000 7f8e0d61d640 1 --2- 172.21.15.100:0/575489333 >> v2:172.21.15.122:3300/0 conn(0x7f8e080b4da0 0x7f8e08073840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.219+0000 7f8e0d61d640 1 -- 172.21.15.100:0/575489333 >> v2:172.21.15.100:3300/0 conn(0x7f8e080c97c0 msgr2=0x7f8e08159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:36.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.220+0000 7f7246ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7248071150 0x7f72480c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.219+0000 7f8e0d61d640 1 --2- 172.21.15.100:0/575489333 >> v2:172.21.15.100:3300/0 conn(0x7f8e080c97c0 0x7f8e08159470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.219+0000 7f8e0d61d640 1 -- 172.21.15.100:0/575489333 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e080c44a0 con 0x7f8e080c9ba0 2024-09-15T21:46:36.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.220+0000 7f910b79f640 1 Processor -- start 2024-09-15T21:46:36.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.220+0000 7f7246ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7248071150 0x7f72480c7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44090/0 (socket says 172.21.15.100:44090) 2024-09-15T21:46:36.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.221+0000 7f8e0d61d640 1 --2- 172.21.15.100:0/575489333 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9ba0 0x7f8e081599b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8e080c6190 tx=0x7f8e0402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2af4010d473fc64 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.221+0000 7f910b79f640 1 -- start start 2024-09-15T21:46:36.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.220+0000 7f7246ffd640 1 -- 172.21.15.100:0/2476194093 learned_addr learned my addr 172.21.15.100:0/2476194093 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:36.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.220+0000 7f72467fc640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7248071530 0x7f72480c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.220+0000 7f7246ffd640 1 -- 172.21.15.100:0/2476194093 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f72480c8650 msgr2=0x7f724815f570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T21:46:36.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.220+0000 7f7246ffd640 1 --2- 172.21.15.100:0/2476194093 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f72480c8650 0x7f724815f570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.220+0000 7f6d8df4b640 1 Processor -- start 2024-09-15T21:46:36.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.220+0000 7f6d8df4b640 1 -- start start 2024-09-15T21:46:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.220+0000 7f7246ffd640 1 -- 172.21.15.100:0/2476194093 >> v2:172.21.15.122:3300/0 conn(0x7f7248071530 msgr2=0x7f72480c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.220+0000 7f7246ffd640 1 --2- 172.21.15.100:0/2476194093 >> v2:172.21.15.122:3300/0 conn(0x7f7248071530 0x7f72480c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.220+0000 7f7246ffd640 1 -- 172.21.15.100:0/2476194093 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7248162c80 con 0x7f7248071150 2024-09-15T21:46:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.221+0000 7f85a1ac8640 1 Processor -- start 2024-09-15T21:46:36.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.221+0000 7f7246ffd640 1 --2- 172.21.15.100:0/2476194093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7248071150 0x7f72480c7680 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f72480c7630 tx=0x7f723802a280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc8a70de9803a17 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.221+0000 7f85a1ac8640 1 -- start start 2024-09-15T21:46:36.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f6d8df4b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6d88151270 0x7f6d880c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f6d8df4b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6d88157bb0 0x7f6d88158490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f6d8df4b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6d881589d0 0x7f6d880cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f6d8df4b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f6d88151650 con 0x7f6d88157bb0 2024-09-15T21:46:36.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f6d8df4b640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f6d880c8420 con 0x7f6d88151270 2024-09-15T21:46:36.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f6d8df4b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6d880c9200 con 0x7f6d881589d0 2024-09-15T21:46:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.221+0000 7f910b79f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90fc04bb60 0x7f90fc0e5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.221+0000 7f910b79f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90fc045fc0 con 0x7f90fc04d390 2024-09-15T21:46:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.221+0000 7f910b79f640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90fc0462a0 con 0x7f90fc04dc70 2024-09-15T21:46:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.221+0000 7f910b79f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f90fc04c7f0 con 0x7f90fc04bb60 2024-09-15T21:46:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f910a53d640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f90fc04d390 0x7f90fc0ed290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:51940/0 (socket says 172.21.15.100:51940) 2024-09-15T21:46:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f910a53d640 1 -- 172.21.15.100:0/1268690956 learned_addr learned my addr 172.21.15.100:0/1268690956 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f9108d3a640 1 --2- 172.21.15.100:0/1268690956 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90fc04bb60 0x7f90fc0e5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f9108d3a640 1 -- 172.21.15.100:0/1268690956 >> v1:172.21.15.122:6789/0 conn(0x7f90fc04dc70 legacy=0x7f90fc04cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f9108d3a640 1 -- 172.21.15.100:0/1268690956 >> v1:172.21.15.100:6789/0 conn(0x7f90fc04d390 legacy=0x7f90fc0ed290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f9108d3a640 1 -- 172.21.15.100:0/1268690956 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90fc046580 con 0x7f90fc04bb60 2024-09-15T21:46:36.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f7245ffb640 1 -- 172.21.15.100:0/2476194093 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238037070 con 0x7f7248071150 2024-09-15T21:46:36.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f6d87fff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6d881589d0 0x7f6d880cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f6d86ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6d88157bb0 0x7f6d88158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f7245ffb640 1 -- 172.21.15.100:0/2476194093 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7238032440 con 0x7f7248071150 2024-09-15T21:46:36.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f7245ffb640 1 -- 172.21.15.100:0/2476194093 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238032810 con 0x7f7248071150 2024-09-15T21:46:36.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f6d86ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6d88157bb0 0x7f6d88158490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44094/0 (socket says 172.21.15.100:44094) 2024-09-15T21:46:36.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f6d86ffd640 1 -- 172.21.15.100:0/986979451 learned_addr learned my addr 172.21.15.100:0/986979451 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:36.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f6d86ffd640 1 -- 172.21.15.100:0/986979451 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6d881589d0 msgr2=0x7f6d880cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:36.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f6d86ffd640 1 --2- 172.21.15.100:0/986979451 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6d881589d0 0x7f6d880cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f6d86ffd640 1 -- 172.21.15.100:0/986979451 >> v2:172.21.15.122:3300/0 conn(0x7f6d88151270 msgr2=0x7f6d880c77d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T21:46:36.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f6d86ffd640 1 --2- 172.21.15.100:0/986979451 >> v2:172.21.15.122:3300/0 conn(0x7f6d88151270 0x7f6d880c77d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.222+0000 7f6d86ffd640 1 -- 172.21.15.100:0/986979451 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d880c94e0 con 0x7f6d88157bb0 2024-09-15T21:46:36.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f130359e640 1 Processor -- start 2024-09-15T21:46:36.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f8e0ce1c640 1 -- 172.21.15.100:0/575489333 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04037070 con 0x7f8e080c9ba0 2024-09-15T21:46:36.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f130359e640 1 -- start start 2024-09-15T21:46:36.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f8e0ce1c640 1 -- 172.21.15.100:0/575489333 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e04032440 con 0x7f8e080c9ba0 2024-09-15T21:46:36.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f85a1ac8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f8594050d50 0x7f8594035320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f85a1ac8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f859404ac30 0x7f85940dc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f85a0ac6640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f85940d6550 0x7f8594044f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.100:46876/0 (socket says 172.21.15.100:46876) 2024-09-15T21:46:36.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f8e0ce1c640 1 -- 172.21.15.100:0/575489333 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04032810 con 0x7f8e080c9ba0 2024-09-15T21:46:36.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f85a12c7640 1 --2- 172.21.15.100:0/2893498166 >> v2:172.21.15.100:3300/0 conn(0x7f859404ac30 0x7f85940dc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f724d903640 1 -- 172.21.15.100:0/2476194093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7248071150 msgr2=0x7f72480c7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f724d903640 1 --2- 172.21.15.100:0/2476194093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7248071150 0x7f72480c7680 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f72480c7630 tx=0x7f723802a280 comp rx=0 tx=0).stop 2024-09-15T21:46:36.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f85a1ac8640 1 -- 172.21.15.100:0/2893498166 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8594035940 con 0x7f85940d6550 2024-09-15T21:46:36.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f85a1ac8640 1 -- 172.21.15.100:0/2893498166 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f859404be50 con 0x7f859404ac30 2024-09-15T21:46:36.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f724d903640 1 -- 172.21.15.100:0/2476194093 shutdown_connections 2024-09-15T21:46:36.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f724d903640 1 --2- 172.21.15.100:0/2476194093 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f72480c8650 0x7f724815f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f724d903640 1 --2- 172.21.15.100:0/2476194093 >> v2:172.21.15.122:3300/0 conn(0x7f7248071530 0x7f72480c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f724d903640 1 --2- 172.21.15.100:0/2476194093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7248071150 0x7f72480c7680 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f724d903640 1 -- 172.21.15.100:0/2476194093 >> 172.21.15.100:0/2476194093 conn(0x7f7248067dd0 msgr2=0x7f724806bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:36.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f85a0ac6640 1 -- 172.21.15.100:0/2893498166 learned_addr learned my addr 172.21.15.100:0/2893498166 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:36.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f85a1ac8640 1 -- 172.21.15.100:0/2893498166 --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f8594035c00 con 0x7f8594050d50 2024-09-15T21:46:36.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f724d903640 1 -- 172.21.15.100:0/2476194093 shutdown_connections 2024-09-15T21:46:36.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f724d903640 1 -- 172.21.15.100:0/2476194093 wait complete. 2024-09-15T21:46:36.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.223+0000 7f859bfff640 1 --2- 172.21.15.100:0/2893498166 >> v2:172.21.15.122:3300/0 conn(0x7f8594050d50 0x7f8594035320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f85a12c7640 1 -- 172.21.15.100:0/2893498166 >> v2:172.21.15.122:3300/0 conn(0x7f8594050d50 msgr2=0x7f8594035320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:36.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f85a12c7640 1 --2- 172.21.15.100:0/2893498166 >> v2:172.21.15.122:3300/0 conn(0x7f8594050d50 0x7f8594035320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f85a12c7640 1 -- 172.21.15.100:0/2893498166 >> v1:172.21.15.122:6789/0 conn(0x7f85940d6550 legacy=0x7f8594044f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f859b7fe640 1 -- 172.21.15.100:0/2893498166 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1896517719 0 0) 0x7f8594035940 con 0x7f85940d6550 2024-09-15T21:46:36.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f6d86ffd640 1 --2- 172.21.15.100:0/986979451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d88157bb0 0x7f6d88158490 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6d880cd750 tx=0x7f6d7802ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e159296dc56236 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.240 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f85a12c7640 1 -- 172.21.15.100:0/2893498166 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859404c0a0 con 0x7f859404ac30 2024-09-15T21:46:36.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f8e10082640 1 -- 172.21.15.100:0/575489333 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9ba0 msgr2=0x7f8e081599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f8e10082640 1 --2- 172.21.15.100:0/575489333 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9ba0 0x7f8e081599b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8e080c6190 tx=0x7f8e0402a1a0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f9108d3a640 1 --2- 172.21.15.100:0/1268690956 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90fc04bb60 0x7f90fc0e5f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f90f80097d0 tx=0x7f90f802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7771eb85d2ac8bf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f8e10082640 1 -- 172.21.15.100:0/575489333 shutdown_connections 2024-09-15T21:46:36.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f8e10082640 1 --2- 172.21.15.100:0/575489333 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9ba0 0x7f8e081599b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f8e10082640 1 --2- 172.21.15.100:0/575489333 >> v2:172.21.15.122:3300/0 conn(0x7f8e080b4da0 0x7f8e08073840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f8e10082640 1 --2- 172.21.15.100:0/575489333 >> v2:172.21.15.100:3300/0 conn(0x7f8e080c97c0 0x7f8e08159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f8e10082640 1 -- 172.21.15.100:0/575489333 >> 172.21.15.100:0/575489333 conn(0x7f8e0806c210 msgr2=0x7f8e08067980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:36.242 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f6d867fc640 1 -- 172.21.15.100:0/986979451 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7802c880 con 0x7f6d88157bb0 2024-09-15T21:46:36.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f130359e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f12fc04d310 0x7f12fc045d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f6d867fc640 1 -- 172.21.15.100:0/986979451 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d7802ca20 con 0x7f6d88157bb0 2024-09-15T21:46:36.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f6d867fc640 1 -- 172.21.15.100:0/986979451 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7802cdf0 con 0x7f6d88157bb0 2024-09-15T21:46:36.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f130359e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f12fc047a80 0x7f12fc046290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f8e10082640 1 -- 172.21.15.100:0/575489333 shutdown_connections 2024-09-15T21:46:36.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f130359e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12fc0dc3a0 con 0x7f12fc0d5850 2024-09-15T21:46:36.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.225+0000 7f130359e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f12fc0dbe90 con 0x7f12fc047a80 2024-09-15T21:46:36.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.224+0000 7f8e10082640 1 -- 172.21.15.100:0/575489333 wait complete. 2024-09-15T21:46:36.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.225+0000 7f1302d9d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f12fc047a80 0x7f12fc046290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.225+0000 7f1302d9d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f12fc047a80 0x7f12fc046290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:41226/0 (socket says 172.21.15.100:41226) 2024-09-15T21:46:36.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.225+0000 7f1302d9d640 1 -- 172.21.15.100:0/3789756522 learned_addr learned my addr 172.21.15.100:0/3789756522 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:36.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.225+0000 7f1301d9b640 1 --2- 172.21.15.100:0/3789756522 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f12fc04d310 0x7f12fc045d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.225+0000 7f130359e640 1 -- 172.21.15.100:0/3789756522 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f12fc0dc0c0 con 0x7f12fc04d310 2024-09-15T21:46:36.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.225+0000 7f90f7fff640 1 -- 172.21.15.100:0/1268690956 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8037070 con 0x7f90fc04bb60 2024-09-15T21:46:36.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.225+0000 7f90f7fff640 1 -- 172.21.15.100:0/1268690956 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90f8032440 con 0x7f90fc04bb60 2024-09-15T21:46:36.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.225+0000 7f90f7fff640 1 -- 172.21.15.100:0/1268690956 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f8032810 con 0x7f90fc04bb60 2024-09-15T21:46:36.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.225+0000 7f1302d9d640 1 -- 172.21.15.100:0/3789756522 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f12fc04d310 msgr2=0x7f12fc045d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:36.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.225+0000 7f1302d9d640 1 --2- 172.21.15.100:0/3789756522 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f12fc04d310 0x7f12fc045d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.225+0000 7f724d903640 1 Processor -- start 2024-09-15T21:46:36.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.225+0000 7f1302d9d640 1 -- 172.21.15.100:0/3789756522 >> v1:172.21.15.100:6789/0 conn(0x7f12fc0d5850 legacy=0x7f12fc007730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:36.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.225+0000 7f6d8df4b640 1 -- 172.21.15.100:0/986979451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d88157bb0 msgr2=0x7f6d88158490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.225+0000 7f6d8df4b640 1 --2- 172.21.15.100:0/986979451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d88157bb0 0x7f6d88158490 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6d880cd750 tx=0x7f6d7802ab20 comp rx=0 tx=0).stop 2024-09-15T21:46:36.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.225+0000 7f724d903640 1 -- start start 2024-09-15T21:46:36.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.225+0000 7f1302d9d640 1 -- 172.21.15.100:0/3789756522 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12fc046cc0 con 0x7f12fc047a80 2024-09-15T21:46:36.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f7e8cfbd640 1 Processor -- start 2024-09-15T21:46:36.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f6d8df4b640 1 -- 172.21.15.100:0/986979451 shutdown_connections 2024-09-15T21:46:36.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f6d8df4b640 1 --2- 172.21.15.100:0/986979451 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6d881589d0 0x7f6d880cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f6d8df4b640 1 --2- 172.21.15.100:0/986979451 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d88157bb0 0x7f6d88158490 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f6d8df4b640 1 --2- 172.21.15.100:0/986979451 >> v2:172.21.15.122:3300/0 conn(0x7f6d88151270 0x7f6d880c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f6d8df4b640 1 -- 172.21.15.100:0/986979451 >> 172.21.15.100:0/986979451 conn(0x7f6d88067dd0 msgr2=0x7f6d8806bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:36.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f8e10082640 1 Processor -- start 2024-09-15T21:46:36.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f7e8cfbd640 1 -- start start 2024-09-15T21:46:36.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f8e10082640 1 -- start start 2024-09-15T21:46:36.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f910b79f640 1 -- 172.21.15.100:0/1268690956 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90fc04bb60 msgr2=0x7f90fc0e5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f910b79f640 1 --2- 172.21.15.100:0/1268690956 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90fc04bb60 0x7f90fc0e5f20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f90f80097d0 tx=0x7f90f802a1c0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f6d8df4b640 1 -- 172.21.15.100:0/986979451 shutdown_connections 2024-09-15T21:46:36.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f6d8df4b640 1 -- 172.21.15.100:0/986979451 wait complete. 2024-09-15T21:46:36.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f85a12c7640 1 --2- 172.21.15.100:0/2893498166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f859404ac30 0x7f85940dc470 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f859c000b60 tx=0x7f859c0a7060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964820a4d9904464 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f724d903640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7248071530 0x7f72481ebe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f910b79f640 1 -- 172.21.15.100:0/1268690956 shutdown_connections 2024-09-15T21:46:36.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f910b79f640 1 --2- 172.21.15.100:0/1268690956 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90fc04bb60 0x7f90fc0e5f20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f910b79f640 1 -- 172.21.15.100:0/1268690956 >> 172.21.15.100:0/1268690956 conn(0x7f90fc007990 msgr2=0x7f90fc007da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:36.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f1302d9d640 1 --2- 172.21.15.100:0/3789756522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12fc047a80 0x7f12fc046290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f13040695f0 tx=0x7f13040a70b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3282e44514a6020e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f724d903640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72480c8650 0x7f72480cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.226+0000 7f859b7fe640 1 -- 172.21.15.100:0/2893498166 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c0b4070 con 0x7f859404ac30 2024-09-15T21:46:36.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f859b7fe640 1 -- 172.21.15.100:0/2893498166 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f859c0a7db0 con 0x7f859404ac30 2024-09-15T21:46:36.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f859b7fe640 1 -- 172.21.15.100:0/2893498166 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c0af640 con 0x7f859404ac30 2024-09-15T21:46:36.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f724d903640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7248160700 0x7f72480cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f130159a640 1 -- 172.21.15.100:0/3789756522 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13040b4070 con 0x7f12fc047a80 2024-09-15T21:46:36.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f130159a640 1 -- 172.21.15.100:0/3789756522 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13040af440 con 0x7f12fc047a80 2024-09-15T21:46:36.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f130159a640 1 -- 172.21.15.100:0/3789756522 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13040af850 con 0x7f12fc047a80 2024-09-15T21:46:36.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f910b79f640 1 -- 172.21.15.100:0/1268690956 shutdown_connections 2024-09-15T21:46:36.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f72467fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72480c8650 0x7f72480cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f8e10082640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e080b4da0 0x7f8e0814c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f8e10082640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e080c97c0 0x7f8e0814b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f72467fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72480c8650 0x7f72480cb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:41236/0 (socket says 172.21.15.100:41236) 2024-09-15T21:46:36.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f8e10082640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9ba0 0x7f8e080c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f8e10082640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8e08158ee0 con 0x7f8e080b4da0 2024-09-15T21:46:36.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f8e10082640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e080c4a60 con 0x7f8e080c97c0 2024-09-15T21:46:36.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f8e10082640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8e08073d80 con 0x7f8e080c9ba0 2024-09-15T21:46:36.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f72467fc640 1 -- 172.21.15.100:0/267626372 learned_addr learned my addr 172.21.15.100:0/267626372 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:36.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f8e0de1e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e080b4da0 0x7f8e0814c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f8e0de1e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e080b4da0 0x7f8e0814c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44108/0 (socket says 172.21.15.100:44108) 2024-09-15T21:46:36.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f8e0de1e640 1 -- 172.21.15.100:0/2942179567 learned_addr learned my addr 172.21.15.100:0/2942179567 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:36.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f724d903640 1 -- 172.21.15.100:0/267626372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f724815fab0 con 0x7f7248071530 2024-09-15T21:46:36.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f724d903640 1 -- 172.21.15.100:0/267626372 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7248163240 con 0x7f72480c8650 2024-09-15T21:46:36.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f724d903640 1 -- 172.21.15.100:0/267626372 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f724815fc30 con 0x7f7248160700 2024-09-15T21:46:36.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f8e0e61f640 1 --2- 172.21.15.100:0/2942179567 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9ba0 0x7f8e080c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f72477fe640 1 --2- 172.21.15.100:0/267626372 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7248160700 0x7f72480cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f8e0d61d640 1 --2- 172.21.15.100:0/2942179567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e080c97c0 0x7f8e0814b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f7246ffd640 1 --2- 172.21.15.100:0/267626372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7248071530 0x7f72481ebe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f8e0e61f640 1 -- 172.21.15.100:0/2942179567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e080c97c0 msgr2=0x7f8e0814b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f8e0e61f640 1 --2- 172.21.15.100:0/2942179567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e080c97c0 0x7f8e0814b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f8e0e61f640 1 -- 172.21.15.100:0/2942179567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e080b4da0 msgr2=0x7f8e0814c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f8e0e61f640 1 --2- 172.21.15.100:0/2942179567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e080b4da0 0x7f8e0814c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f8e0e61f640 1 -- 172.21.15.100:0/2942179567 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e080c7770 con 0x7f8e080c9ba0 2024-09-15T21:46:36.258 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f7e8659c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f7e8806e250 0x7f7e880c5250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:51958/0 (socket says 172.21.15.100:51958) 2024-09-15T21:46:36.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f85a1ac8640 1 -- 172.21.15.100:0/2893498166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f859404ac30 msgr2=0x7f85940dc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f85a1ac8640 1 --2- 172.21.15.100:0/2893498166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f859404ac30 0x7f85940dc470 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f859c000b60 tx=0x7f859c0a7060 comp rx=0 tx=0).stop 2024-09-15T21:46:36.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f7246ffd640 1 -- 172.21.15.100:0/267626372 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7248160700 msgr2=0x7f72480cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f7246ffd640 1 --2- 172.21.15.100:0/267626372 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7248160700 0x7f72480cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f7246ffd640 1 -- 172.21.15.100:0/267626372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72480c8650 msgr2=0x7f72480cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f7246ffd640 1 --2- 172.21.15.100:0/267626372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72480c8650 0x7f72480cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f7246ffd640 1 -- 172.21.15.100:0/267626372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724815ef80 con 0x7f7248071530 2024-09-15T21:46:36.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f72467fc640 1 --2- 172.21.15.100:0/267626372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72480c8650 0x7f72480cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:36.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.227+0000 7f910b79f640 1 -- 172.21.15.100:0/1268690956 wait complete. 2024-09-15T21:46:36.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f72477fe640 1 --2- 172.21.15.100:0/267626372 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7248160700 0x7f72480cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:36.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f7e8659c640 1 -- 172.21.15.100:0/2923587056 learned_addr learned my addr 172.21.15.100:0/2923587056 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:36.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f8e0e61f640 1 --2- 172.21.15.100:0/2942179567 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9ba0 0x7f8e080c7c20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000eed0 tx=0x7f8e0000b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f7246ffd640 1 --2- 172.21.15.100:0/267626372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7248071530 0x7f72481ebe90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7238031000 tx=0x7f72380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f6d8df4b640 1 Processor -- start 2024-09-15T21:46:36.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f7e8cfbd640 1 --2- 172.21.15.100:0/2923587056 >> v2:172.21.15.100:3300/0 conn(0x7f7e88151590 0x7f7e88162910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/2923587056 --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e8806ec30 con 0x7f7e8806e250 2024-09-15T21:46:36.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f130359e640 1 -- 172.21.15.100:0/3789756522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12fc047a80 msgr2=0x7f12fc046290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f130359e640 1 --2- 172.21.15.100:0/3789756522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12fc047a80 0x7f12fc046290 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f13040695f0 tx=0x7f13040a70b0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f85a1ac8640 1 -- 172.21.15.100:0/2893498166 shutdown_connections 2024-09-15T21:46:36.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.228+0000 7f85a1ac8640 1 --2- 172.21.15.100:0/2893498166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f859404ac30 0x7f85940dc470 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f85a1ac8640 1 --2- 172.21.15.100:0/2893498166 >> v2:172.21.15.122:3300/0 conn(0x7f8594050d50 0x7f8594035320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f6d8df4b640 1 -- start start 2024-09-15T21:46:36.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f85a1ac8640 1 -- 172.21.15.100:0/2893498166 >> 172.21.15.100:0/2893498166 conn(0x7f85940500a0 msgr2=0x7f85940504b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:36.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f130359e640 1 -- 172.21.15.100:0/3789756522 shutdown_connections 2024-09-15T21:46:36.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f130359e640 1 --2- 172.21.15.100:0/3789756522 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12fc047a80 0x7f12fc046290 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f722ffff640 1 -- 172.21.15.100:0/267626372 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238041020 con 0x7f7248071530 2024-09-15T21:46:36.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f722ffff640 1 -- 172.21.15.100:0/267626372 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7238030640 con 0x7f7248071530 2024-09-15T21:46:36.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f722ffff640 1 -- 172.21.15.100:0/267626372 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238037040 con 0x7f7248071530 2024-09-15T21:46:36.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f724d903640 1 -- 172.21.15.100:0/267626372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72480cc5e0 con 0x7f7248071530 2024-09-15T21:46:36.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f130359e640 1 --2- 172.21.15.100:0/3789756522 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f12fc04d310 0x7f12fc045d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f7e86d9d640 1 --2- 172.21.15.100:0/2923587056 >> v2:172.21.15.100:3300/0 conn(0x7f7e88151590 0x7f7e88162910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f130359e640 1 -- 172.21.15.100:0/3789756522 >> 172.21.15.100:0/3789756522 conn(0x7f12fc04d830 msgr2=0x7f12fc04dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:36.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f7e8559a640 1 -- 172.21.15.100:0/2923587056 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3598787395 0 0) 0x7f7e78001490 con 0x7f7e8806e250 2024-09-15T21:46:36.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/2923587056 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e8806f220 con 0x7f7e8806e630 2024-09-15T21:46:36.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/2923587056 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f7e880c4dc0 con 0x7f7e88151590 2024-09-15T21:46:36.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f910b79f640 1 Processor -- start 2024-09-15T21:46:36.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f910b79f640 1 -- start start 2024-09-15T21:46:36.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f8dfe7fc640 1 -- 172.21.15.100:0/2942179567 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00018020 con 0x7f8e080c9ba0 2024-09-15T21:46:36.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f7e8559a640 1 -- 172.21.15.100:0/2923587056 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e70003520 con 0x7f7e8806e250 2024-09-15T21:46:36.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f7e8559a640 1 -- 172.21.15.100:0/2923587056 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3930837209 0 0) 0x7f7e70003520 con 0x7f7e8806e250 2024-09-15T21:46:36.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f6d8df4b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6d88151270 0x7f6d88150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f724d903640 1 -- 172.21.15.100:0/267626372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72480ccba0 con 0x7f7248071530 2024-09-15T21:46:36.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f6d8df4b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d88157bb0 0x7f6d881528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f6d8df4b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d881589d0 0x7f6d88152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f6d877fe640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6d88151270 0x7f6d88150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f6d8df4b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6d880c7d10 con 0x7f6d881589d0 2024-09-15T21:46:36.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f8e10082640 1 -- 172.21.15.100:0/2942179567 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e08154400 con 0x7f8e080c9ba0 2024-09-15T21:46:36.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.229+0000 7f8e10082640 1 -- 172.21.15.100:0/2942179567 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e081549a0 con 0x7f8e080c9ba0 2024-09-15T21:46:36.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f8dfe7fc640 1 -- 172.21.15.100:0/2942179567 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e00014400 con 0x7f8e080c9ba0 2024-09-15T21:46:36.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f6d8df4b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6d880c9aa0 con 0x7f6d88157bb0 2024-09-15T21:46:36.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f6d8df4b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6d880c7fe0 con 0x7f6d88151270 2024-09-15T21:46:36.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f6d87fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d881589d0 0x7f6d88152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f6d87fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d881589d0 0x7f6d88152de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:44140/0 (socket says 172.21.15.100:44140) 2024-09-15T21:46:36.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f130359e640 1 -- 172.21.15.100:0/3789756522 shutdown_connections 2024-09-15T21:46:36.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f6d87fff640 1 -- 172.21.15.100:0/4066362791 learned_addr learned my addr 172.21.15.100:0/4066362791 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:36.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f6d86ffd640 1 --2- 172.21.15.100:0/4066362791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d88157bb0 0x7f6d881528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f85a1ac8640 1 -- 172.21.15.100:0/2893498166 shutdown_connections 2024-09-15T21:46:36.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f8dfe7fc640 1 -- 172.21.15.100:0/2942179567 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00009e60 con 0x7f8e080c9ba0 2024-09-15T21:46:36.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f7e86d9d640 1 -- 172.21.15.100:0/2923587056 >> v1:172.21.15.122:6789/0 conn(0x7f7e8806e630 legacy=0x7f7e88150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:36.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f7e86d9d640 1 -- 172.21.15.100:0/2923587056 >> v1:172.21.15.100:6789/0 conn(0x7f7e8806e250 legacy=0x7f7e880c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f7e86d9d640 1 -- 172.21.15.100:0/2923587056 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8806f500 con 0x7f7e88151590 2024-09-15T21:46:36.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f910b79f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90fc04bb60 0x7f90fc0cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f910b79f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90fc04d390 0x7f90fc185300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f910b79f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90fc04dc70 0x7f90fc0d8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f910b79f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc0076e0 con 0x7f90fc04bb60 2024-09-15T21:46:36.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f910b79f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f90fc046860 con 0x7f90fc04d390 2024-09-15T21:46:36.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f910b79f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f90fc0dcfe0 con 0x7f90fc04dc70 2024-09-15T21:46:36.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f910a53d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90fc04dc70 0x7f90fc0d8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f910a53d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90fc04dc70 0x7f90fc0d8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:56120/0 (socket says 172.21.15.100:56120) 2024-09-15T21:46:36.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f910a53d640 1 -- 172.21.15.100:0/87344166 learned_addr learned my addr 172.21.15.100:0/87344166 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:36.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.230+0000 7f910953b640 1 --2- 172.21.15.100:0/87344166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90fc04bb60 0x7f90fc0cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f85a1ac8640 1 -- 172.21.15.100:0/2893498166 wait complete. 2024-09-15T21:46:36.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f130359e640 1 -- 172.21.15.100:0/3789756522 wait complete. 2024-09-15T21:46:36.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f722e7fc640 1 -- 172.21.15.100:0/267626372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72480c7c00 con 0x7f7248071530 2024-09-15T21:46:36.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f9108d3a640 1 --2- 172.21.15.100:0/87344166 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90fc04d390 0x7f90fc185300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f6d87fff640 1 -- 172.21.15.100:0/4066362791 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6d88151270 msgr2=0x7f6d88150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f6d87fff640 1 --2- 172.21.15.100:0/4066362791 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6d88151270 0x7f6d88150260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f6d87fff640 1 -- 172.21.15.100:0/4066362791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d88157bb0 msgr2=0x7f6d881528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f6d87fff640 1 --2- 172.21.15.100:0/4066362791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d88157bb0 0x7f6d881528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f6d87fff640 1 -- 172.21.15.100:0/4066362791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d881efd00 con 0x7f6d881589d0 2024-09-15T21:46:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f910a53d640 1 -- 172.21.15.100:0/87344166 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90fc04d390 msgr2=0x7f90fc185300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f910a53d640 1 --2- 172.21.15.100:0/87344166 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90fc04d390 0x7f90fc185300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f910a53d640 1 -- 172.21.15.100:0/87344166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90fc04bb60 msgr2=0x7f90fc0cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f910a53d640 1 --2- 172.21.15.100:0/87344166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90fc04bb60 0x7f90fc0cdd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f910a53d640 1 -- 172.21.15.100:0/87344166 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90fc185e60 con 0x7f90fc04dc70 2024-09-15T21:46:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f910953b640 1 --2- 172.21.15.100:0/87344166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90fc04bb60 0x7f90fc0cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:46:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.231+0000 7f9108d3a640 1 --2- 172.21.15.100:0/87344166 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90fc04d390 0x7f90fc185300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:36.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.232+0000 7f6d87fff640 1 --2- 172.21.15.100:0/4066362791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d881589d0 0x7f6d88152de0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000efc0 tx=0x7f6d8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.232+0000 7f8e10082640 1 -- 172.21.15.100:0/2942179567 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8df0005290 con 0x7f8e080c9ba0 2024-09-15T21:46:36.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.232+0000 7f910a53d640 1 --2- 172.21.15.100:0/87344166 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90fc04dc70 0x7f90fc0d8620 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f910406b960 tx=0x7f9104069a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.232+0000 7f90f5ffb640 1 -- 172.21.15.100:0/87344166 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91040741c0 con 0x7f90fc04dc70 2024-09-15T21:46:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.232+0000 7f7e86d9d640 1 --2- 172.21.15.100:0/2923587056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e88151590 0x7f7e88162910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7e800097a0 tx=0x7f7e8002c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e17af2211a883 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.232+0000 7f90f5ffb640 1 -- 172.21.15.100:0/87344166 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9104092020 con 0x7f90fc04dc70 2024-09-15T21:46:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.232+0000 7f6d6ffff640 1 -- 172.21.15.100:0/4066362791 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80018020 con 0x7f6d881589d0 2024-09-15T21:46:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.232+0000 7f6d6ffff640 1 -- 172.21.15.100:0/4066362791 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d80014480 con 0x7f6d881589d0 2024-09-15T21:46:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.232+0000 7f6d8df4b640 1 -- 172.21.15.100:0/4066362791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d88151ae0 con 0x7f6d881589d0 2024-09-15T21:46:36.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.232+0000 7f6d8df4b640 1 -- 172.21.15.100:0/4066362791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d881520a0 con 0x7f6d881589d0 2024-09-15T21:46:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.232+0000 7f722ffff640 1 -- 172.21.15.100:0/267626372 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7238044020 con 0x7f7248071530 2024-09-15T21:46:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.233+0000 7f722ffff640 1 --2- 172.21.15.100:0/267626372 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f723406c300 0x7f723406e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.233+0000 7f6d6ffff640 1 -- 172.21.15.100:0/4066362791 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80014870 con 0x7f6d881589d0 2024-09-15T21:46:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.234+0000 7f7e8559a640 1 -- 172.21.15.100:0/2923587056 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80039070 con 0x7f7e88151590 2024-09-15T21:46:36.280 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.234+0000 7f7e8559a640 1 -- 172.21.15.100:0/2923587056 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e80034440 con 0x7f7e88151590 2024-09-15T21:46:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.233+0000 7f8dfe7fc640 1 -- 172.21.15.100:0/2942179567 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8e00005d10 con 0x7f8e080c9ba0 2024-09-15T21:46:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.233+0000 7f8dfe7fc640 1 --2- 172.21.15.100:0/2942179567 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8dec06c3b0 0x7f8dec06e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.233+0000 7f8e0de1e640 1 --2- 172.21.15.100:0/2942179567 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8dec06c3b0 0x7f8dec06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.233+0000 7f90f5ffb640 1 -- 172.21.15.100:0/87344166 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f910406bbe0 con 0x7f90fc04dc70 2024-09-15T21:46:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.233+0000 7f910b79f640 1 -- 172.21.15.100:0/87344166 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90fc185bc0 con 0x7f90fc04dc70 2024-09-15T21:46:36.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.233+0000 7f910b79f640 1 -- 172.21.15.100:0/87344166 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f90fc000db0 con 0x7f90fc04dc70 2024-09-15T21:46:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.233+0000 7f130359e640 1 Processor -- start 2024-09-15T21:46:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.233+0000 7f130359e640 1 -- start start 2024-09-15T21:46:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.233+0000 7f85a1ac8640 1 Processor -- start 2024-09-15T21:46:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.233+0000 7f85a1ac8640 1 -- start start 2024-09-15T21:46:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.233+0000 7f722ffff640 1 -- 172.21.15.100:0/267626372 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7238077ba0 con 0x7f7248071530 2024-09-15T21:46:36.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.233+0000 7f72467fc640 1 --2- 172.21.15.100:0/267626372 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f723406c300 0x7f723406e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.234+0000 7f7e8559a640 1 -- 172.21.15.100:0/2923587056 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e78001bc0 con 0x7f7e88151590 2024-09-15T21:46:36.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.234+0000 7f8dfe7fc640 1 -- 172.21.15.100:0/2942179567 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8e0008bd30 con 0x7f8e080c9ba0 2024-09-15T21:46:36.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.234+0000 7f72467fc640 1 --2- 172.21.15.100:0/267626372 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f723406c300 0x7f723406e7c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7240009a40 tx=0x7f724000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.234+0000 7f130359e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12fc047a80 0x7f12fc0d5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.234+0000 7f85a1ac8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f859404ac30 0x7f85940d4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.234+0000 7f85a1ac8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8594050d50 0x7f85940d7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.234+0000 7f85a1ac8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85940d6550 0x7f85941657a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.234+0000 7f8e0de1e640 1 --2- 172.21.15.100:0/2942179567 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8dec06c3b0 0x7f8dec06e870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8df80094a0 tx=0x7f8df8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.234+0000 7f85a1ac8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f85940d6930 con 0x7f8594050d50 2024-09-15T21:46:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.234+0000 7f6d6dffb640 1 -- 172.21.15.100:0/4066362791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d8806c2f0 con 0x7f6d881589d0 2024-09-15T21:46:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f130359e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f12fc04d310 0x7f12fc0483a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f85a12c7640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85940d6550 0x7f85941657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f130259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12fc047a80 0x7f12fc0d5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f130259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12fc047a80 0x7f12fc0d5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:41274/0 (socket says 172.21.15.100:41274) 2024-09-15T21:46:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f130259c640 1 -- 172.21.15.100:0/1201778584 learned_addr learned my addr 172.21.15.100:0/1201778584 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f130359e640 1 --2- 172.21.15.100:0/1201778584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12fc0d5850 0x7f12fc049c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.286 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/2923587056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e88151590 msgr2=0x7f7e88162910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f7e8cfbd640 1 --2- 172.21.15.100:0/2923587056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e88151590 0x7f7e88162910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7e800097a0 tx=0x7f7e8002c410 comp rx=0 tx=0).stop 2024-09-15T21:46:36.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f1301d9b640 1 --2- 172.21.15.100:0/1201778584 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f12fc04d310 0x7f12fc0483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f130359e640 1 -- 172.21.15.100:0/1201778584 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc047560 con 0x7f12fc0d5850 2024-09-15T21:46:36.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f130359e640 1 -- 172.21.15.100:0/1201778584 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f12fc047280 con 0x7f12fc047a80 2024-09-15T21:46:36.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f130359e640 1 -- 172.21.15.100:0/1201778584 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f12fc0468c0 con 0x7f12fc04d310 2024-09-15T21:46:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f1302d9d640 1 --2- 172.21.15.100:0/1201778584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12fc0d5850 0x7f12fc049c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f85a12c7640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85940d6550 0x7f85941657a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:56140/0 (socket says 172.21.15.100:56140) 2024-09-15T21:46:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f85a12c7640 1 -- 172.21.15.100:0/1507384242 learned_addr learned my addr 172.21.15.100:0/1507384242 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f85a1ac8640 1 -- 172.21.15.100:0/1507384242 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f859404c660 con 0x7f859404ac30 2024-09-15T21:46:36.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f85a1ac8640 1 -- 172.21.15.100:0/1507384242 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f85940d6ab0 con 0x7f85940d6550 2024-09-15T21:46:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f910b79f640 1 -- 172.21.15.100:0/87344166 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90fc047960 con 0x7f90fc04dc70 2024-09-15T21:46:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/2923587056 shutdown_connections 2024-09-15T21:46:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f7e8cfbd640 1 --2- 172.21.15.100:0/2923587056 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e88151590 0x7f7e88162910 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/2923587056 >> 172.21.15.100:0/2923587056 conn(0x7f7e880c8f20 msgr2=0x7f7e880c9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:36.289 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f6d6ffff640 1 -- 172.21.15.100:0/4066362791 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6d8001b020 con 0x7f6d881589d0 2024-09-15T21:46:36.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f85a0ac6640 1 --2- 172.21.15.100:0/1507384242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f859404ac30 0x7f85940d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/2923587056 shutdown_connections 2024-09-15T21:46:36.290 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.236+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/2923587056 wait complete. 2024-09-15T21:46:36.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f130259c640 1 -- 172.21.15.100:0/1201778584 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f12fc04d310 msgr2=0x7f12fc0483a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f130259c640 1 --2- 172.21.15.100:0/1201778584 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f12fc04d310 0x7f12fc0483a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f130259c640 1 -- 172.21.15.100:0/1201778584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12fc0d5850 msgr2=0x7f12fc049c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f130259c640 1 --2- 172.21.15.100:0/1201778584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12fc0d5850 0x7f12fc049c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.235+0000 7f130259c640 1 -- 172.21.15.100:0/1201778584 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12fc1791b0 con 0x7f12fc047a80 2024-09-15T21:46:36.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.236+0000 7f130259c640 1 --2- 172.21.15.100:0/1201778584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12fc047a80 0x7f12fc0d5ef0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f12f400b3f0 tx=0x7f12f400b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.236+0000 7f859bfff640 1 --2- 172.21.15.100:0/1507384242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8594050d50 0x7f85940d7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.236+0000 7f6d6ffff640 1 --2- 172.21.15.100:0/4066362791 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6d7406c380 0x7f6d7406e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.236+0000 7f6d877fe640 1 --2- 172.21.15.100:0/4066362791 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6d7406c380 0x7f6d7406e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.236+0000 7f85a12c7640 1 -- 172.21.15.100:0/1507384242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f859404ac30 msgr2=0x7f85940d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.236+0000 7f85a12c7640 1 --2- 172.21.15.100:0/1507384242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f859404ac30 0x7f85940d4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.236+0000 7f85a12c7640 1 -- 172.21.15.100:0/1507384242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8594050d50 msgr2=0x7f85940d7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.236+0000 7f12f2ffd640 1 -- 172.21.15.100:0/1201778584 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4015020 con 0x7f12fc047a80 2024-09-15T21:46:36.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.236+0000 7f85a12c7640 1 --2- 172.21.15.100:0/1507384242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8594050d50 0x7f85940d7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.236+0000 7f85a12c7640 1 -- 172.21.15.100:0/1507384242 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85940d9970 con 0x7f85940d6550 2024-09-15T21:46:36.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.236+0000 7f859bfff640 1 --2- 172.21.15.100:0/1507384242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8594050d50 0x7f85940d7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:36.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.236+0000 7f6d6ffff640 1 -- 172.21.15.100:0/4066362791 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6d8008be90 con 0x7f6d881589d0 2024-09-15T21:46:36.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.236+0000 7f12f2ffd640 1 -- 172.21.15.100:0/1201778584 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f12f4011830 con 0x7f12fc047a80 2024-09-15T21:46:36.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.236+0000 7f130359e640 1 -- 172.21.15.100:0/1201778584 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12fc179470 con 0x7f12fc047a80 2024-09-15T21:46:36.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.236+0000 7f85a12c7640 1 --2- 172.21.15.100:0/1507384242 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85940d6550 0x7f85941657a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f859c050b40 tx=0x7f859c069fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.236+0000 7f130359e640 1 -- 172.21.15.100:0/1201778584 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12fc179a30 con 0x7f12fc047a80 2024-09-15T21:46:36.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.236+0000 7f6d877fe640 1 --2- 172.21.15.100:0/4066362791 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6d7406c380 0x7f6d7406e840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c006fd0 tx=0x7f6d7c009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.237+0000 7f12f2ffd640 1 -- 172.21.15.100:0/1201778584 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12f4015020 con 0x7f12fc047a80 2024-09-15T21:46:36.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.238+0000 7f7e8cfbd640 1 Processor -- start 2024-09-15T21:46:36.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.238+0000 7f8dfe7fc640 1 -- 172.21.15.100:0/2942179567 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8e00022030 con 0x7f8e080c9ba0 2024-09-15T21:46:36.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.238+0000 7f7e8cfbd640 1 -- start start 2024-09-15T21:46:36.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.238+0000 7f85997fa640 1 -- 172.21.15.100:0/1507384242 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c0bc070 con 0x7f85940d6550 2024-09-15T21:46:36.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.238+0000 7f85a1ac8640 1 -- 172.21.15.100:0/1507384242 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8594175be0 con 0x7f85940d6550 2024-09-15T21:46:36.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.238+0000 7f85a1ac8640 1 -- 172.21.15.100:0/1507384242 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f85941761a0 con 0x7f85940d6550 2024-09-15T21:46:36.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.239+0000 7f130359e640 1 -- 172.21.15.100:0/1201778584 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12fc0d3de0 con 0x7f12fc047a80 2024-09-15T21:46:36.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.239+0000 7f85997fa640 1 -- 172.21.15.100:0/1507384242 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f859c0a7c70 con 0x7f85940d6550 2024-09-15T21:46:36.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.239+0000 7f7e8cfbd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e8806e250 0x7f7e8814d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.239+0000 7f85997fa640 1 -- 172.21.15.100:0/1507384242 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c0b4040 con 0x7f85940d6550 2024-09-15T21:46:36.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.240+0000 7f7e8cfbd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e8806e630 0x7f7e8814c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.240+0000 7f12f2ffd640 1 -- 172.21.15.100:0/1201778584 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f12f4018020 con 0x7f12fc047a80 2024-09-15T21:46:36.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.240+0000 7f85a1ac8640 1 -- 172.21.15.100:0/1507384242 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f857c005290 con 0x7f85940d6550 2024-09-15T21:46:36.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.240+0000 7f7e8659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e8806e250 0x7f7e8814d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.240+0000 7f7e8659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e8806e250 0x7f7e8814d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:41292/0 (socket says 172.21.15.100:41292) 2024-09-15T21:46:36.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.240+0000 7f7e8659c640 1 -- 172.21.15.100:0/171860583 learned_addr learned my addr 172.21.15.100:0/171860583 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:36.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.240+0000 7f7e8cfbd640 1 --2- 172.21.15.100:0/171860583 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e88151590 0x7f7e881ed300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.240+0000 7f7e85d9b640 1 --2- 172.21.15.100:0/171860583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e8806e630 0x7f7e8814c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.240+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/171860583 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7e880c6ea0 con 0x7f7e8806e630 2024-09-15T21:46:36.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.240+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/171860583 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e880c6d20 con 0x7f7e8806e250 2024-09-15T21:46:36.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.240+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/171860583 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7e88070d50 con 0x7f7e88151590 2024-09-15T21:46:36.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.240+0000 7f7e86d9d640 1 --2- 172.21.15.100:0/171860583 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e88151590 0x7f7e881ed300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.240+0000 7f90f5ffb640 1 -- 172.21.15.100:0/87344166 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f910406d110 con 0x7f90fc04dc70 2024-09-15T21:46:36.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.240+0000 7f12f2ffd640 1 --2- 172.21.15.100:0/1201778584 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f12e406c4b0 0x7f12e406e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.240+0000 7f12f2ffd640 1 -- 172.21.15.100:0/1201778584 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f12f408aeb0 con 0x7f12fc047a80 2024-09-15T21:46:36.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.241+0000 7f1301d9b640 1 --2- 172.21.15.100:0/1201778584 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f12e406c4b0 0x7f12e406e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.241+0000 7f7e86d9d640 1 -- 172.21.15.100:0/171860583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e8806e250 msgr2=0x7f7e8814d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.241+0000 7f7e86d9d640 1 --2- 172.21.15.100:0/171860583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e8806e250 0x7f7e8814d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.241+0000 7f7e86d9d640 1 -- 172.21.15.100:0/171860583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e8806e630 msgr2=0x7f7e8814c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.241+0000 7f7e86d9d640 1 --2- 172.21.15.100:0/171860583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e8806e630 0x7f7e8814c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.241+0000 7f7e86d9d640 1 -- 172.21.15.100:0/171860583 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8814c040 con 0x7f7e88151590 2024-09-15T21:46:36.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.241+0000 7f90f5ffb640 1 --2- 172.21.15.100:0/87344166 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f90e806c2b0 0x7f90e806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.241+0000 7f90f5ffb640 1 -- 172.21.15.100:0/87344166 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9104107d00 con 0x7f90fc04dc70 2024-09-15T21:46:36.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.241+0000 7f910953b640 1 --2- 172.21.15.100:0/87344166 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f90e806c2b0 0x7f90e806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.241+0000 7f7e86d9d640 1 --2- 172.21.15.100:0/171860583 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e88151590 0x7f7e881ed300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7e80033000 tx=0x7f7e800046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.242+0000 7f910953b640 1 --2- 172.21.15.100:0/87344166 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f90e806c2b0 0x7f90e806e770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9100006fd0 tx=0x7f9100009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.242+0000 7f1301d9b640 1 --2- 172.21.15.100:0/1201778584 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f12e406c4b0 0x7f12e406e970 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0d1460 tx=0x7f12f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.243+0000 7f722ffff640 1 -- 172.21.15.100:0/267626372 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f723804b030 con 0x7f7248071530 2024-09-15T21:46:36.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.243+0000 7f7e76ffd640 1 -- 172.21.15.100:0/171860583 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80039040 con 0x7f7e88151590 2024-09-15T21:46:36.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.243+0000 7f90f5ffb640 1 -- 172.21.15.100:0/87344166 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f90fc047960 con 0x7f90fc04dc70 2024-09-15T21:46:36.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.243+0000 7f7e76ffd640 1 -- 172.21.15.100:0/171860583 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e80004230 con 0x7f7e88151590 2024-09-15T21:46:36.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.243+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/171860583 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e88151dd0 con 0x7f7e88151590 2024-09-15T21:46:36.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.244+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/171860583 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e881520b0 con 0x7f7e88151590 2024-09-15T21:46:36.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.245+0000 7f7e76ffd640 1 -- 172.21.15.100:0/171860583 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e80039040 con 0x7f7e88151590 2024-09-15T21:46:36.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.245+0000 7f85997fa640 1 -- 172.21.15.100:0/1507384242 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f859c0bf070 con 0x7f85940d6550 2024-09-15T21:46:36.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.245+0000 7f12f2ffd640 1 -- 172.21.15.100:0/1201778584 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f12fc0d3de0 con 0x7f12fc047a80 2024-09-15T21:46:36.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.246+0000 7f85997fa640 1 --2- 172.21.15.100:0/1507384242 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f858006c2b0 0x7f858006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.246+0000 7f85997fa640 1 -- 172.21.15.100:0/1507384242 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f859c128780 con 0x7f85940d6550 2024-09-15T21:46:36.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.246+0000 7f85a0ac6640 1 --2- 172.21.15.100:0/1507384242 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f858006c2b0 0x7f858006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.246+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/171860583 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e880c3bf0 con 0x7f7e88151590 2024-09-15T21:46:36.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.247+0000 7f85a0ac6640 1 --2- 172.21.15.100:0/1507384242 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f858006c2b0 0x7f858006e770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8590000f60 tx=0x7f8590007b50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.248+0000 7f85997fa640 1 -- 172.21.15.100:0/1507384242 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f859c0f6f60 con 0x7f85940d6550 2024-09-15T21:46:36.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.250+0000 7f7e76ffd640 1 -- 172.21.15.100:0/171860583 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7e80003610 con 0x7f7e88151590 2024-09-15T21:46:36.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.250+0000 7f7e76ffd640 1 --2- 172.21.15.100:0/171860583 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7e6806c6b0 0x7f7e6806eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.251+0000 7f7e8659c640 1 --2- 172.21.15.100:0/171860583 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7e6806c6b0 0x7f7e6806eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.250+0000 7f6d6ffff640 1 -- 172.21.15.100:0/4066362791 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6d80022030 con 0x7f6d881589d0 2024-09-15T21:46:36.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.251+0000 7f7e76ffd640 1 -- 172.21.15.100:0/171860583 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f7e80004030 con 0x7f7e88151590 2024-09-15T21:46:36.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.251+0000 7f7e8659c640 1 --2- 172.21.15.100:0/171860583 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7e6806c6b0 0x7f7e6806eb70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7e78005c00 tx=0x7f7e78009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.257+0000 7f7e76ffd640 1 -- 172.21.15.100:0/171860583 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7e8004f020 con 0x7f7e88151590 2024-09-15T21:46:36.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.582+0000 7ff971952640 1 -- 172.21.15.100:0/3603280464 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff96c073e00 con 0x7ff96c151270 2024-09-15T21:46:36.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.582+0000 7ff953fff640 1 -- 172.21.15.100:0/3603280464 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff96005cd40 con 0x7ff96c151270 2024-09-15T21:46:36.584 INFO:teuthology.orchestra.run.smithi100.stdout:64424509448 2024-09-15T21:46:36.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.586+0000 7ff951ffb640 1 -- 172.21.15.100:0/3603280464 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff95806c2b0 msgr2=0x7ff95806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.586+0000 7ff951ffb640 1 --2- 172.21.15.100:0/3603280464 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff95806c2b0 0x7ff95806e770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff95c00a900 tx=0x7ff95c008040 comp rx=0 tx=0).stop 2024-09-15T21:46:36.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.586+0000 7ff951ffb640 1 -- 172.21.15.100:0/3603280464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff96c151270 msgr2=0x7ff96c153960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.586+0000 7ff951ffb640 1 --2- 172.21.15.100:0/3603280464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff96c151270 0x7ff96c153960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff96000c7b0 tx=0x7ff96000cc70 comp rx=0 tx=0).stop 2024-09-15T21:46:36.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.587+0000 7ff951ffb640 1 -- 172.21.15.100:0/3603280464 shutdown_connections 2024-09-15T21:46:36.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.587+0000 7ff951ffb640 1 --2- 172.21.15.100:0/3603280464 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff95806c2b0 0x7ff95806e770 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.587+0000 7ff951ffb640 1 --2- 172.21.15.100:0/3603280464 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff96c157bb0 0x7ff96c15a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.587+0000 7ff951ffb640 1 --2- 172.21.15.100:0/3603280464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff96c151270 0x7ff96c153960 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.587+0000 7ff951ffb640 1 --2- 172.21.15.100:0/3603280464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff96c0cc620 0x7ff96c153420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.587+0000 7ff951ffb640 1 -- 172.21.15.100:0/3603280464 >> 172.21.15.100:0/3603280464 conn(0x7ff96c067dd0 msgr2=0x7ff96c072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:36.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.587+0000 7ff951ffb640 1 -- 172.21.15.100:0/3603280464 shutdown_connections 2024-09-15T21:46:36.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.587+0000 7ff951ffb640 1 -- 172.21.15.100:0/3603280464 wait complete. 2024-09-15T21:46:36.605 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.4 2024-09-15T21:46:36.606 DEBUG:teuthology.parallel:result is None 2024-09-15T21:46:36.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.618+0000 7f130359e640 1 -- 172.21.15.100:0/1201778584 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f12fc04b420 con 0x7f12fc047a80 2024-09-15T21:46:36.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.618+0000 7f12f2ffd640 1 -- 172.21.15.100:0/1201778584 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f12fc04b420 con 0x7f12fc047a80 2024-09-15T21:46:36.621 INFO:teuthology.orchestra.run.smithi100.stdout:64424509447 2024-09-15T21:46:36.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.620+0000 7f130359e640 1 -- 172.21.15.100:0/1201778584 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f12e406c4b0 msgr2=0x7f12e406e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.620+0000 7f130359e640 1 --2- 172.21.15.100:0/1201778584 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f12e406c4b0 0x7f12e406e970 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0d1460 tx=0x7f12f8008040 comp rx=0 tx=0).stop 2024-09-15T21:46:36.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.621+0000 7f130359e640 1 -- 172.21.15.100:0/1201778584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12fc047a80 msgr2=0x7f12fc0d5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.621+0000 7f130359e640 1 --2- 172.21.15.100:0/1201778584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12fc047a80 0x7f12fc0d5ef0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f12f400b3f0 tx=0x7f12f400b8b0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.621+0000 7f130359e640 1 -- 172.21.15.100:0/1201778584 shutdown_connections 2024-09-15T21:46:36.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.621+0000 7f130359e640 1 --2- 172.21.15.100:0/1201778584 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f12e406c4b0 0x7f12e406e970 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.621+0000 7f130359e640 1 --2- 172.21.15.100:0/1201778584 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12fc0d5850 0x7f12fc049c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.621+0000 7f130359e640 1 --2- 172.21.15.100:0/1201778584 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f12fc04d310 0x7f12fc0483a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.621+0000 7f130359e640 1 --2- 172.21.15.100:0/1201778584 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12fc047a80 0x7f12fc0d5ef0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.621+0000 7f130359e640 1 -- 172.21.15.100:0/1201778584 >> 172.21.15.100:0/1201778584 conn(0x7f12fc04d830 msgr2=0x7f12fc050d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:36.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.621+0000 7f130359e640 1 -- 172.21.15.100:0/1201778584 shutdown_connections 2024-09-15T21:46:36.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.621+0000 7f130359e640 1 -- 172.21.15.100:0/1201778584 wait complete. 2024-09-15T21:46:36.633 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.2 2024-09-15T21:46:36.634 DEBUG:teuthology.parallel:result is None 2024-09-15T21:46:36.637 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.636+0000 7f8e10082640 1 -- 172.21.15.100:0/2942179567 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f8df0005580 con 0x7f8e080c9ba0 2024-09-15T21:46:36.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.636+0000 7f8dfe7fc640 1 -- 172.21.15.100:0/2942179567 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8e0005a480 con 0x7f8e080c9ba0 2024-09-15T21:46:36.638 INFO:teuthology.orchestra.run.smithi100.stdout:68719476743 2024-09-15T21:46:36.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.638+0000 7f8e10082640 1 -- 172.21.15.100:0/2942179567 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8dec06c3b0 msgr2=0x7f8dec06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.638+0000 7f8e10082640 1 --2- 172.21.15.100:0/2942179567 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8dec06c3b0 0x7f8dec06e870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8df80094a0 tx=0x7f8df8008040 comp rx=0 tx=0).stop 2024-09-15T21:46:36.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.639+0000 7f8e10082640 1 -- 172.21.15.100:0/2942179567 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9ba0 msgr2=0x7f8e080c7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.639+0000 7f8e10082640 1 --2- 172.21.15.100:0/2942179567 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9ba0 0x7f8e080c7c20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000eed0 tx=0x7f8e0000b480 comp rx=0 tx=0).stop 2024-09-15T21:46:36.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.639+0000 7f8e10082640 1 -- 172.21.15.100:0/2942179567 shutdown_connections 2024-09-15T21:46:36.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.639+0000 7f8e10082640 1 --2- 172.21.15.100:0/2942179567 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8dec06c3b0 0x7f8dec06e870 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.639+0000 7f8e10082640 1 --2- 172.21.15.100:0/2942179567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e080c97c0 0x7f8e0814b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.639+0000 7f8e10082640 1 --2- 172.21.15.100:0/2942179567 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9ba0 0x7f8e080c7c20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.639+0000 7f8e10082640 1 --2- 172.21.15.100:0/2942179567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e080b4da0 0x7f8e0814c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.639+0000 7f8e10082640 1 -- 172.21.15.100:0/2942179567 >> 172.21.15.100:0/2942179567 conn(0x7f8e0806c210 msgr2=0x7f8e0806df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:36.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.639+0000 7f8e10082640 1 -- 172.21.15.100:0/2942179567 shutdown_connections 2024-09-15T21:46:36.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.639+0000 7f8e10082640 1 -- 172.21.15.100:0/2942179567 wait complete. 2024-09-15T21:46:36.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.644+0000 7f910b79f640 1 -- 172.21.15.100:0/87344166 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f90fc0e5f20 con 0x7f90fc04dc70 2024-09-15T21:46:36.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.644+0000 7f90f5ffb640 1 -- 172.21.15.100:0/87344166 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f910409c030 con 0x7f90fc04dc70 2024-09-15T21:46:36.646 INFO:teuthology.orchestra.run.smithi100.stdout:55834574856 2024-09-15T21:46:36.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.646+0000 7f910b79f640 1 -- 172.21.15.100:0/87344166 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f90e806c2b0 msgr2=0x7f90e806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.646+0000 7f910b79f640 1 --2- 172.21.15.100:0/87344166 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f90e806c2b0 0x7f90e806e770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9100006fd0 tx=0x7f9100009060 comp rx=0 tx=0).stop 2024-09-15T21:46:36.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.647+0000 7f910b79f640 1 -- 172.21.15.100:0/87344166 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90fc04dc70 msgr2=0x7f90fc0d8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.647+0000 7f910b79f640 1 --2- 172.21.15.100:0/87344166 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90fc04dc70 0x7f90fc0d8620 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f910406b960 tx=0x7f9104069a20 comp rx=0 tx=0).stop 2024-09-15T21:46:36.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.647+0000 7f910b79f640 1 -- 172.21.15.100:0/87344166 shutdown_connections 2024-09-15T21:46:36.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.647+0000 7f910b79f640 1 --2- 172.21.15.100:0/87344166 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f90e806c2b0 0x7f90e806e770 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.647+0000 7f910b79f640 1 --2- 172.21.15.100:0/87344166 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90fc04d390 0x7f90fc185300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.647+0000 7f910b79f640 1 --2- 172.21.15.100:0/87344166 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90fc04dc70 0x7f90fc0d8620 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.647+0000 7f910b79f640 1 --2- 172.21.15.100:0/87344166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90fc04bb60 0x7f90fc0cdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.647+0000 7f910b79f640 1 -- 172.21.15.100:0/87344166 >> 172.21.15.100:0/87344166 conn(0x7f90fc007990 msgr2=0x7f90fc04ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:36.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.647+0000 7f910b79f640 1 -- 172.21.15.100:0/87344166 shutdown_connections 2024-09-15T21:46:36.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.647+0000 7f910b79f640 1 -- 172.21.15.100:0/87344166 wait complete. 2024-09-15T21:46:36.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.649+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/171860583 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7e880c8060 con 0x7f7e88151590 2024-09-15T21:46:36.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.649+0000 7f722e7fc640 1 -- 172.21.15.100:0/267626372 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f7248157250 con 0x7f7248071530 2024-09-15T21:46:36.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.650+0000 7f722ffff640 1 -- 172.21.15.100:0/267626372 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f723807a200 con 0x7f7248071530 2024-09-15T21:46:36.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.650+0000 7f7e76ffd640 1 -- 172.21.15.100:0/171860583 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7e8007c120 con 0x7f7e88151590 2024-09-15T21:46:36.652 INFO:teuthology.orchestra.run.smithi100.stdout:64424509449 2024-09-15T21:46:36.652 INFO:teuthology.orchestra.run.smithi100.stdout:60129542151 2024-09-15T21:46:36.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.650+0000 7f6d6dffb640 1 -- 172.21.15.100:0/4066362791 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6d88073e00 con 0x7f6d881589d0 2024-09-15T21:46:36.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.651+0000 7f6d6ffff640 1 -- 172.21.15.100:0/4066362791 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6d80010040 con 0x7f6d881589d0 2024-09-15T21:46:36.653 INFO:teuthology.orchestra.run.smithi100.stdout:64424509447 2024-09-15T21:46:36.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.652+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/171860583 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7e6806c6b0 msgr2=0x7f7e6806eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.652+0000 7f7e8cfbd640 1 --2- 172.21.15.100:0/171860583 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7e6806c6b0 0x7f7e6806eb70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7e78005c00 tx=0x7f7e78009040 comp rx=0 tx=0).stop 2024-09-15T21:46:36.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/171860583 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e88151590 msgr2=0x7f7e881ed300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f7e8cfbd640 1 --2- 172.21.15.100:0/171860583 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e88151590 0x7f7e881ed300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7e80033000 tx=0x7f7e800046c0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/171860583 shutdown_connections 2024-09-15T21:46:36.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f7e8cfbd640 1 --2- 172.21.15.100:0/171860583 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7e6806c6b0 0x7f7e6806eb70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f7e8cfbd640 1 --2- 172.21.15.100:0/171860583 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e88151590 0x7f7e881ed300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f7e8cfbd640 1 --2- 172.21.15.100:0/171860583 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e8806e630 0x7f7e8814c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f7e8cfbd640 1 --2- 172.21.15.100:0/171860583 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e8806e250 0x7f7e8814d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/171860583 >> 172.21.15.100:0/171860583 conn(0x7f7e880c8f20 msgr2=0x7f7e8814f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:36.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/171860583 shutdown_connections 2024-09-15T21:46:36.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f7e8cfbd640 1 -- 172.21.15.100:0/171860583 wait complete. 2024-09-15T21:46:36.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f85a1ac8640 1 -- 172.21.15.100:0/1507384242 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f857c005850 con 0x7f85940d6550 2024-09-15T21:46:36.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f722e7fc640 1 -- 172.21.15.100:0/267626372 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f723406c300 msgr2=0x7f723406e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f722e7fc640 1 --2- 172.21.15.100:0/267626372 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f723406c300 0x7f723406e7c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7240009a40 tx=0x7f724000b040 comp rx=0 tx=0).stop 2024-09-15T21:46:36.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f722e7fc640 1 -- 172.21.15.100:0/267626372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7248071530 msgr2=0x7f72481ebe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f722e7fc640 1 --2- 172.21.15.100:0/267626372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7248071530 0x7f72481ebe90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7238031000 tx=0x7f72380046c0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f722e7fc640 1 -- 172.21.15.100:0/267626372 shutdown_connections 2024-09-15T21:46:36.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f722e7fc640 1 --2- 172.21.15.100:0/267626372 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f723406c300 0x7f723406e7c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f722e7fc640 1 --2- 172.21.15.100:0/267626372 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72480c8650 0x7f72480cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f722e7fc640 1 --2- 172.21.15.100:0/267626372 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7248160700 0x7f72480cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f722e7fc640 1 --2- 172.21.15.100:0/267626372 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7248071530 0x7f72481ebe90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.653+0000 7f722e7fc640 1 -- 172.21.15.100:0/267626372 >> 172.21.15.100:0/267626372 conn(0x7f7248067dd0 msgr2=0x7f72481e88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:36.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.654+0000 7f722e7fc640 1 -- 172.21.15.100:0/267626372 shutdown_connections 2024-09-15T21:46:36.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.654+0000 7f722e7fc640 1 -- 172.21.15.100:0/267626372 wait complete. 2024-09-15T21:46:36.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.654+0000 7f6d6dffb640 1 -- 172.21.15.100:0/4066362791 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6d7406c380 msgr2=0x7f6d7406e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.654+0000 7f6d6dffb640 1 --2- 172.21.15.100:0/4066362791 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6d7406c380 0x7f6d7406e840 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c006fd0 tx=0x7f6d7c009060 comp rx=0 tx=0).stop 2024-09-15T21:46:36.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.654+0000 7f6d6dffb640 1 -- 172.21.15.100:0/4066362791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d881589d0 msgr2=0x7f6d88152de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.654+0000 7f6d6dffb640 1 --2- 172.21.15.100:0/4066362791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d881589d0 0x7f6d88152de0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000efc0 tx=0x7f6d8000c490 comp rx=0 tx=0).stop 2024-09-15T21:46:36.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.654+0000 7f85997fa640 1 -- 172.21.15.100:0/1507384242 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f859c0f6860 con 0x7f85940d6550 2024-09-15T21:46:36.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.654+0000 7f6d6dffb640 1 -- 172.21.15.100:0/4066362791 shutdown_connections 2024-09-15T21:46:36.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.654+0000 7f6d6dffb640 1 --2- 172.21.15.100:0/4066362791 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6d7406c380 0x7f6d7406e840 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.654+0000 7f6d6dffb640 1 --2- 172.21.15.100:0/4066362791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d88157bb0 0x7f6d881528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.654+0000 7f6d6dffb640 1 --2- 172.21.15.100:0/4066362791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d881589d0 0x7f6d88152de0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.654+0000 7f6d6dffb640 1 --2- 172.21.15.100:0/4066362791 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6d88151270 0x7f6d88150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.654+0000 7f6d6dffb640 1 -- 172.21.15.100:0/4066362791 >> 172.21.15.100:0/4066362791 conn(0x7f6d88067dd0 msgr2=0x7f6d880c4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:36.659 INFO:teuthology.orchestra.run.smithi100.stdout:64424509446 2024-09-15T21:46:36.660 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476743 got 68719476743 for osd.0 2024-09-15T21:46:36.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.654+0000 7f6d6dffb640 1 -- 172.21.15.100:0/4066362791 shutdown_connections 2024-09-15T21:46:36.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.654+0000 7f6d6dffb640 1 -- 172.21.15.100:0/4066362791 wait complete. 2024-09-15T21:46:36.661 DEBUG:teuthology.parallel:result is None 2024-09-15T21:46:36.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.657+0000 7f85a1ac8640 1 -- 172.21.15.100:0/1507384242 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f858006c2b0 msgr2=0x7f858006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.657+0000 7f85a1ac8640 1 --2- 172.21.15.100:0/1507384242 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f858006c2b0 0x7f858006e770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8590000f60 tx=0x7f8590007b50 comp rx=0 tx=0).stop 2024-09-15T21:46:36.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.657+0000 7f85a1ac8640 1 -- 172.21.15.100:0/1507384242 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85940d6550 msgr2=0x7f85941657a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.657+0000 7f85a1ac8640 1 --2- 172.21.15.100:0/1507384242 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85940d6550 0x7f85941657a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f859c050b40 tx=0x7f859c069fa0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.657+0000 7f85a1ac8640 1 -- 172.21.15.100:0/1507384242 shutdown_connections 2024-09-15T21:46:36.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.657+0000 7f85a1ac8640 1 --2- 172.21.15.100:0/1507384242 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f858006c2b0 0x7f858006e770 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.657+0000 7f85a1ac8640 1 --2- 172.21.15.100:0/1507384242 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85940d6550 0x7f85941657a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.657+0000 7f85a1ac8640 1 --2- 172.21.15.100:0/1507384242 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8594050d50 0x7f85940d7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.657+0000 7f85a1ac8640 1 --2- 172.21.15.100:0/1507384242 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f859404ac30 0x7f85940d4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.657+0000 7f85a1ac8640 1 -- 172.21.15.100:0/1507384242 >> 172.21.15.100:0/1507384242 conn(0x7f85940500a0 msgr2=0x7f85940487c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:36.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.657+0000 7f85a1ac8640 1 -- 172.21.15.100:0/1507384242 shutdown_connections 2024-09-15T21:46:36.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.657+0000 7f85a1ac8640 1 -- 172.21.15.100:0/1507384242 wait complete. 2024-09-15T21:46:36.665 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574856 for osd.3 2024-09-15T21:46:36.665 DEBUG:teuthology.parallel:result is None 2024-09-15T21:46:36.669 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.1 2024-09-15T21:46:36.670 DEBUG:teuthology.parallel:result is None 2024-09-15T21:46:36.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.7 2024-09-15T21:46:36.671 DEBUG:teuthology.parallel:result is None 2024-09-15T21:46:36.675 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.6 2024-09-15T21:46:36.675 DEBUG:teuthology.parallel:result is None 2024-09-15T21:46:36.677 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509446 for osd.5 2024-09-15T21:46:36.677 DEBUG:teuthology.parallel:result is None 2024-09-15T21:46:36.677 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:46:36.677 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:46:36.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.813+0000 7f04eb4a4640 1 Processor -- start 2024-09-15T21:46:36.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.813+0000 7f04eb4a4640 1 -- start start 2024-09-15T21:46:36.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.814+0000 7f04eb4a4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f04e41552e0 0x7f04e406d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.814+0000 7f04eb4a4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f04e40cee00 0x7f04e40730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.814+0000 7f04eb4a4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04e406ec60 con 0x7f04e40735f0 2024-09-15T21:46:36.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.814+0000 7f04eb4a4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f04e415af40 con 0x7f04e40cee00 2024-09-15T21:46:36.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.814+0000 7f04eb4a4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f04e415b200 con 0x7f04e41552e0 2024-09-15T21:46:36.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.814+0000 7f04e8a3f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f04e40cee00 0x7f04e40730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.814+0000 7f04e8a3f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f04e40cee00 0x7f04e40730b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45416/0 (socket says 172.21.15.100:45416) 2024-09-15T21:46:36.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.814+0000 7f04e8a3f640 1 -- 172.21.15.100:0/3377486365 learned_addr learned my addr 172.21.15.100:0/3377486365 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:36.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.814+0000 7f04e9a41640 1 --1- 172.21.15.100:0/3377486365 >> v1:172.21.15.100:6789/0 conn(0x7f04e40735f0 0x7f04e415c5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:54160/0 (socket says 172.21.15.100:54160) 2024-09-15T21:46:36.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.814+0000 7f04e9240640 1 --2- 172.21.15.100:0/3377486365 >> v2:172.21.15.122:3300/0 conn(0x7f04e41552e0 0x7f04e406d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.814+0000 7f04e8a3f640 1 -- 172.21.15.100:0/3377486365 >> v2:172.21.15.122:3300/0 conn(0x7f04e41552e0 msgr2=0x7f04e406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:36.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.815+0000 7f04e8a3f640 1 --2- 172.21.15.100:0/3377486365 >> v2:172.21.15.122:3300/0 conn(0x7f04e41552e0 0x7f04e406d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.815+0000 7f04e8a3f640 1 -- 172.21.15.100:0/3377486365 >> v1:172.21.15.100:6789/0 conn(0x7f04e40735f0 legacy=0x7f04e415c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.815+0000 7f04e8a3f640 1 -- 172.21.15.100:0/3377486365 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e415b4e0 con 0x7f04e40cee00 2024-09-15T21:46:36.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.815+0000 7f04e9240640 1 --2- 172.21.15.100:0/3377486365 >> v2:172.21.15.122:3300/0 conn(0x7f04e41552e0 0x7f04e406d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:36.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.815+0000 7f04e8a3f640 1 --2- 172.21.15.100:0/3377486365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04e40cee00 0x7f04e40730b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f04e406ffc0 tx=0x7f04d402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4e0408909e4416 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.815+0000 7f04dbfff640 1 -- 172.21.15.100:0/3377486365 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4004030 con 0x7f04e40cee00 2024-09-15T21:46:36.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.815+0000 7f04dbfff640 1 -- 172.21.15.100:0/3377486365 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04d40041d0 con 0x7f04e40cee00 2024-09-15T21:46:36.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.815+0000 7f04dbfff640 1 -- 172.21.15.100:0/3377486365 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4003610 con 0x7f04e40cee00 2024-09-15T21:46:36.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.816+0000 7f04eb4a4640 1 -- 172.21.15.100:0/3377486365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04e40cee00 msgr2=0x7f04e40730b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.816+0000 7f04eb4a4640 1 --2- 172.21.15.100:0/3377486365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04e40cee00 0x7f04e40730b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f04e406ffc0 tx=0x7f04d402a270 comp rx=0 tx=0).stop 2024-09-15T21:46:36.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.816+0000 7f04eb4a4640 1 -- 172.21.15.100:0/3377486365 shutdown_connections 2024-09-15T21:46:36.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.816+0000 7f04eb4a4640 1 --2- 172.21.15.100:0/3377486365 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04e40cee00 0x7f04e40730b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.816+0000 7f04eb4a4640 1 --2- 172.21.15.100:0/3377486365 >> v2:172.21.15.122:3300/0 conn(0x7f04e41552e0 0x7f04e406d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.816+0000 7f04eb4a4640 1 -- 172.21.15.100:0/3377486365 >> 172.21.15.100:0/3377486365 conn(0x7f04e4070ca0 msgr2=0x7f04e4071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:36.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.816+0000 7f04eb4a4640 1 -- 172.21.15.100:0/3377486365 shutdown_connections 2024-09-15T21:46:36.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.816+0000 7f04eb4a4640 1 -- 172.21.15.100:0/3377486365 wait complete. 2024-09-15T21:46:36.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.817+0000 7f04eb4a4640 1 Processor -- start 2024-09-15T21:46:36.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.817+0000 7f04eb4a4640 1 -- start start 2024-09-15T21:46:36.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.818+0000 7f04eb4a4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04e40735f0 0x7f04e40c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.818+0000 7f04eb4a4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f04e40cee00 0x7f04e415c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.818+0000 7f04eb4a4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04e41552e0 0x7f04e41f0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.818+0000 7f04eb4a4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f04e40c91a0 con 0x7f04e40735f0 2024-09-15T21:46:36.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.818+0000 7f04eb4a4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f04e415baa0 con 0x7f04e41552e0 2024-09-15T21:46:36.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.818+0000 7f04eb4a4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f04e40cf250 con 0x7f04e40cee00 2024-09-15T21:46:36.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.818+0000 7f04e9240640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04e40735f0 0x7f04e40c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.818+0000 7f04e9240640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04e40735f0 0x7f04e40c6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45430/0 (socket says 172.21.15.100:45430) 2024-09-15T21:46:36.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.818+0000 7f04e9240640 1 -- 172.21.15.100:0/3272649013 learned_addr learned my addr 172.21.15.100:0/3272649013 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.818+0000 7f04e8a3f640 1 --2- 172.21.15.100:0/3272649013 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f04e40cee00 0x7f04e415c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.818+0000 7f04e9a41640 1 --2- 172.21.15.100:0/3272649013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04e41552e0 0x7f04e41f0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.818+0000 7f04e9240640 1 -- 172.21.15.100:0/3272649013 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f04e40cee00 msgr2=0x7f04e415c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.818+0000 7f04e9240640 1 --2- 172.21.15.100:0/3272649013 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f04e40cee00 0x7f04e415c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.818+0000 7f04e9240640 1 -- 172.21.15.100:0/3272649013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04e41552e0 msgr2=0x7f04e41f0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.818+0000 7f04e9240640 1 --2- 172.21.15.100:0/3272649013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04e41552e0 0x7f04e41f0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.818+0000 7f04e9240640 1 -- 172.21.15.100:0/3272649013 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e4156310 con 0x7f04e40735f0 2024-09-15T21:46:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.818+0000 7f04e9240640 1 --2- 172.21.15.100:0/3272649013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04e40735f0 0x7f04e40c6100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f04e000c820 tx=0x7f04e000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.819+0000 7f04d9ffb640 1 -- 172.21.15.100:0/3272649013 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0015070 con 0x7f04e40735f0 2024-09-15T21:46:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.819+0000 7f04d9ffb640 1 -- 172.21.15.100:0/3272649013 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e0007a30 con 0x7f04e40735f0 2024-09-15T21:46:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.819+0000 7f04d9ffb640 1 -- 172.21.15.100:0/3272649013 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0007e00 con 0x7f04e40735f0 2024-09-15T21:46:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.819+0000 7f04eb4a4640 1 -- 172.21.15.100:0/3272649013 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e41584c0 con 0x7f04e40735f0 2024-09-15T21:46:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.819+0000 7f04eb4a4640 1 -- 172.21.15.100:0/3272649013 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e40c8a80 con 0x7f04e40735f0 2024-09-15T21:46:36.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.820+0000 7f04eb4a4640 1 -- 172.21.15.100:0/3272649013 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e406dd00 con 0x7f04e40735f0 2024-09-15T21:46:36.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.820+0000 7f04d9ffb640 1 -- 172.21.15.100:0/3272649013 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f04e0018070 con 0x7f04e40735f0 2024-09-15T21:46:36.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.820+0000 7f04d9ffb640 1 --2- 172.21.15.100:0/3272649013 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f04c806c2e0 0x7f04c806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:36.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.821+0000 7f04d9ffb640 1 -- 172.21.15.100:0/3272649013 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f04e008b460 con 0x7f04e40735f0 2024-09-15T21:46:36.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.821+0000 7f04e8a3f640 1 --2- 172.21.15.100:0/3272649013 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f04c806c2e0 0x7f04c806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:36.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.821+0000 7f04e8a3f640 1 --2- 172.21.15.100:0/3272649013 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f04c806c2e0 0x7f04c806e7a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f04d4033000 tx=0x7f04d402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:36.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.825+0000 7f04d9ffb640 1 -- 172.21.15.100:0/3272649013 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f04e406dd00 con 0x7f04e40735f0 2024-09-15T21:46:36.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.988+0000 7f04eb4a4640 1 -- 172.21.15.100:0/3272649013 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f04e40730b0 con 0x7f04c806c2e0 2024-09-15T21:46:36.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.994+0000 7f04d9ffb640 1 -- 172.21.15.100:0/3272649013 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251862 (secure 0 0 0) 0x7f04e40730b0 con 0x7f04c806c2e0 2024-09-15T21:46:36.998 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:37.000 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-15T21:46:37.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.997+0000 7f04eb4a4640 1 -- 172.21.15.100:0/3272649013 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f04c806c2e0 msgr2=0x7f04c806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:37.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.997+0000 7f04eb4a4640 1 --2- 172.21.15.100:0/3272649013 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f04c806c2e0 0x7f04c806e7a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f04d4033000 tx=0x7f04d402e000 comp rx=0 tx=0).stop 2024-09-15T21:46:37.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.997+0000 7f04eb4a4640 1 -- 172.21.15.100:0/3272649013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04e40735f0 msgr2=0x7f04e40c6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:37.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.997+0000 7f04eb4a4640 1 --2- 172.21.15.100:0/3272649013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04e40735f0 0x7f04e40c6100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f04e000c820 tx=0x7f04e000cc20 comp rx=0 tx=0).stop 2024-09-15T21:46:37.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.997+0000 7f04eb4a4640 1 -- 172.21.15.100:0/3272649013 shutdown_connections 2024-09-15T21:46:37.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.997+0000 7f04eb4a4640 1 --2- 172.21.15.100:0/3272649013 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f04c806c2e0 0x7f04c806e7a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.997+0000 7f04eb4a4640 1 --2- 172.21.15.100:0/3272649013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04e41552e0 0x7f04e41f0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.997+0000 7f04eb4a4640 1 --2- 172.21.15.100:0/3272649013 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f04e40cee00 0x7f04e415c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.997+0000 7f04eb4a4640 1 --2- 172.21.15.100:0/3272649013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f04e40735f0 0x7f04e40c6100 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.997+0000 7f04eb4a4640 1 -- 172.21.15.100:0/3272649013 >> 172.21.15.100:0/3272649013 conn(0x7f04e4070ca0 msgr2=0x7f04e41598d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:37.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.997+0000 7f04eb4a4640 1 -- 172.21.15.100:0/3272649013 shutdown_connections 2024-09-15T21:46:37.002 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:36.997+0000 7f04eb4a4640 1 -- 172.21.15.100:0/3272649013 wait complete. 2024-09-15T21:46:37.009 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-15T21:46:35.806088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593400,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":205,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219152,"kb_used_data":2960,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755568,"statfs":{"total":773094113280,"available":772869701632,"internally_reserved":0,"allocated":3031040,"data_stored":1801842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3032,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.394192"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835668+0000","last_change":"2024-09-15T21:46:23.428662+0000","last_active":"2024-09-15T21:46:24.835668+0000","last_peered":"2024-09-15T21:46:24.835668+0000","last_clean":"2024-09-15T21:46:24.835668+0000","last_became_active":"2024-09-15T21:46:23.427735+0000","last_became_peered":"2024-09-15T21:46:23.427735+0000","last_unstale":"2024-09-15T21:46:24.835668+0000","last_undegraded":"2024-09-15T21:46:24.835668+0000","last_fullsized":"2024-09-15T21:46:24.835668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:29.702757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808513+0000","last_change":"2024-09-15T21:46:23.418716+0000","last_active":"2024-09-15T21:46:24.808513+0000","last_peered":"2024-09-15T21:46:24.808513+0000","last_clean":"2024-09-15T21:46:24.808513+0000","last_became_active":"2024-09-15T21:46:23.418471+0000","last_became_peered":"2024-09-15T21:46:23.418471+0000","last_unstale":"2024-09-15T21:46:24.808513+0000","last_undegraded":"2024-09-15T21:46:24.808513+0000","last_fullsized":"2024-09-15T21:46:24.808513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:49.599801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727016+0000","last_change":"2024-09-15T21:46:23.426404+0000","last_active":"2024-09-15T21:46:24.727016+0000","last_peered":"2024-09-15T21:46:24.727016+0000","last_clean":"2024-09-15T21:46:24.727016+0000","last_became_active":"2024-09-15T21:46:23.426146+0000","last_became_peered":"2024-09-15T21:46:23.426146+0000","last_unstale":"2024-09-15T21:46:24.727016+0000","last_undegraded":"2024-09-15T21:46:24.727016+0000","last_fullsized":"2024-09-15T21:46:24.727016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:23.626109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414109+0000","last_change":"2024-09-15T21:46:23.432515+0000","last_active":"2024-09-15T21:46:24.414109+0000","last_peered":"2024-09-15T21:46:24.414109+0000","last_clean":"2024-09-15T21:46:24.414109+0000","last_became_active":"2024-09-15T21:46:23.432283+0000","last_became_peered":"2024-09-15T21:46:23.432283+0000","last_unstale":"2024-09-15T21:46:24.414109+0000","last_undegraded":"2024-09-15T21:46:24.414109+0000","last_fullsized":"2024-09-15T21:46:24.414109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:53.288080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835464+0000","last_change":"2024-09-15T21:46:23.427120+0000","last_active":"2024-09-15T21:46:24.835464+0000","last_peered":"2024-09-15T21:46:24.835464+0000","last_clean":"2024-09-15T21:46:24.835464+0000","last_became_active":"2024-09-15T21:46:23.426706+0000","last_became_peered":"2024-09-15T21:46:23.426706+0000","last_unstale":"2024-09-15T21:46:24.835464+0000","last_undegraded":"2024-09-15T21:46:24.835464+0000","last_fullsized":"2024-09-15T21:46:24.835464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:29.513446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414665+0000","last_change":"2024-09-15T21:46:23.434690+0000","last_active":"2024-09-15T21:46:24.414665+0000","last_peered":"2024-09-15T21:46:24.414665+0000","last_clean":"2024-09-15T21:46:24.414665+0000","last_became_active":"2024-09-15T21:46:23.434146+0000","last_became_peered":"2024-09-15T21:46:23.434146+0000","last_unstale":"2024-09-15T21:46:24.414665+0000","last_undegraded":"2024-09-15T21:46:24.414665+0000","last_fullsized":"2024-09-15T21:46:24.414665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:26:59.905178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835273+0000","last_change":"2024-09-15T21:46:23.429541+0000","last_active":"2024-09-15T21:46:24.835273+0000","last_peered":"2024-09-15T21:46:24.835273+0000","last_clean":"2024-09-15T21:46:24.835273+0000","last_became_active":"2024-09-15T21:46:23.429263+0000","last_became_peered":"2024-09-15T21:46:23.429263+0000","last_unstale":"2024-09-15T21:46:24.835273+0000","last_undegraded":"2024-09-15T21:46:24.835273+0000","last_fullsized":"2024-09-15T21:46:24.835273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:34:51.530981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834382+0000","last_change":"2024-09-15T21:46:23.437143+0000","last_active":"2024-09-15T21:46:24.834382+0000","last_peered":"2024-09-15T21:46:24.834382+0000","last_clean":"2024-09-15T21:46:24.834382+0000","last_became_active":"2024-09-15T21:46:23.436864+0000","last_became_peered":"2024-09-15T21:46:23.436864+0000","last_unstale":"2024-09-15T21:46:24.834382+0000","last_undegraded":"2024-09-15T21:46:24.834382+0000","last_fullsized":"2024-09-15T21:46:24.834382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:45.870403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835817+0000","last_change":"2024-09-15T21:46:23.438642+0000","last_active":"2024-09-15T21:46:24.835817+0000","last_peered":"2024-09-15T21:46:24.835817+0000","last_clean":"2024-09-15T21:46:24.835817+0000","last_became_active":"2024-09-15T21:46:23.437521+0000","last_became_peered":"2024-09-15T21:46:23.437521+0000","last_unstale":"2024-09-15T21:46:24.835817+0000","last_undegraded":"2024-09-15T21:46:24.835817+0000","last_fullsized":"2024-09-15T21:46:24.835817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:04.648853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835520+0000","last_change":"2024-09-15T21:46:23.436106+0000","last_active":"2024-09-15T21:46:24.835520+0000","last_peered":"2024-09-15T21:46:24.835520+0000","last_clean":"2024-09-15T21:46:24.835520+0000","last_became_active":"2024-09-15T21:46:23.435821+0000","last_became_peered":"2024-09-15T21:46:23.435821+0000","last_unstale":"2024-09-15T21:46:24.835520+0000","last_undegraded":"2024-09-15T21:46:24.835520+0000","last_fullsized":"2024-09-15T21:46:24.835520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835070+0000","last_change":"2024-09-15T21:46:23.435442+0000","last_active":"2024-09-15T21:46:24.835070+0000","last_peered":"2024-09-15T21:46:24.835070+0000","last_clean":"2024-09-15T21:46:24.835070+0000","last_became_active":"2024-09-15T21:46:23.435165+0000","last_became_peered":"2024-09-15T21:46:23.435165+0000","last_unstale":"2024-09-15T21:46:24.835070+0000","last_undegraded":"2024-09-15T21:46:24.835070+0000","last_fullsized":"2024-09-15T21:46:24.835070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:02.361682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414099+0000","last_change":"2024-09-15T21:46:23.432534+0000","last_active":"2024-09-15T21:46:24.414099+0000","last_peered":"2024-09-15T21:46:24.414099+0000","last_clean":"2024-09-15T21:46:24.414099+0000","last_became_active":"2024-09-15T21:46:23.430170+0000","last_became_peered":"2024-09-15T21:46:23.430170+0000","last_unstale":"2024-09-15T21:46:24.414099+0000","last_undegraded":"2024-09-15T21:46:24.414099+0000","last_fullsized":"2024-09-15T21:46:24.414099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:11.463937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834691+0000","last_change":"2024-09-15T21:46:23.427841+0000","last_active":"2024-09-15T21:46:24.834691+0000","last_peered":"2024-09-15T21:46:24.834691+0000","last_clean":"2024-09-15T21:46:24.834691+0000","last_became_active":"2024-09-15T21:46:23.427621+0000","last_became_peered":"2024-09-15T21:46:23.427621+0000","last_unstale":"2024-09-15T21:46:24.834691+0000","last_undegraded":"2024-09-15T21:46:24.834691+0000","last_fullsized":"2024-09-15T21:46:24.834691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:18.331943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.726852+0000","last_change":"2024-09-15T21:46:23.424707+0000","last_active":"2024-09-15T21:46:24.726852+0000","last_peered":"2024-09-15T21:46:24.726852+0000","last_clean":"2024-09-15T21:46:24.726852+0000","last_became_active":"2024-09-15T21:46:23.424447+0000","last_became_peered":"2024-09-15T21:46:23.424447+0000","last_unstale":"2024-09-15T21:46:24.726852+0000","last_undegraded":"2024-09-15T21:46:24.726852+0000","last_fullsized":"2024-09-15T21:46:24.726852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:23.617214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808321+0000","last_change":"2024-09-15T21:46:23.430297+0000","last_active":"2024-09-15T21:46:24.808321+0000","last_peered":"2024-09-15T21:46:24.808321+0000","last_clean":"2024-09-15T21:46:24.808321+0000","last_became_active":"2024-09-15T21:46:23.430070+0000","last_became_peered":"2024-09-15T21:46:23.430070+0000","last_unstale":"2024-09-15T21:46:24.808321+0000","last_undegraded":"2024-09-15T21:46:24.808321+0000","last_fullsized":"2024-09-15T21:46:24.808321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:03:19.615022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414542+0000","last_change":"2024-09-15T21:46:23.430540+0000","last_active":"2024-09-15T21:46:24.414542+0000","last_peered":"2024-09-15T21:46:24.414542+0000","last_clean":"2024-09-15T21:46:24.414542+0000","last_became_active":"2024-09-15T21:46:23.430256+0000","last_became_peered":"2024-09-15T21:46:23.430256+0000","last_unstale":"2024-09-15T21:46:24.414542+0000","last_undegraded":"2024-09-15T21:46:24.414542+0000","last_fullsized":"2024-09-15T21:46:24.414542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:36.539762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.415216+0000","last_change":"2024-09-15T21:46:23.427802+0000","last_active":"2024-09-15T21:46:24.415216+0000","last_peered":"2024-09-15T21:46:24.415216+0000","last_clean":"2024-09-15T21:46:24.415216+0000","last_became_active":"2024-09-15T21:46:23.427502+0000","last_became_peered":"2024-09-15T21:46:23.427502+0000","last_unstale":"2024-09-15T21:46:24.415216+0000","last_undegraded":"2024-09-15T21:46:24.415216+0000","last_fullsized":"2024-09-15T21:46:24.415216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:31.062624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412600+0000","last_change":"2024-09-15T21:46:23.427570+0000","last_active":"2024-09-15T21:46:24.412600+0000","last_peered":"2024-09-15T21:46:24.412600+0000","last_clean":"2024-09-15T21:46:24.412600+0000","last_became_active":"2024-09-15T21:46:23.427304+0000","last_became_peered":"2024-09-15T21:46:23.427304+0000","last_unstale":"2024-09-15T21:46:24.412600+0000","last_undegraded":"2024-09-15T21:46:24.412600+0000","last_fullsized":"2024-09-15T21:46:24.412600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:37.345106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835899+0000","last_change":"2024-09-15T21:46:23.431434+0000","last_active":"2024-09-15T21:46:24.835899+0000","last_peered":"2024-09-15T21:46:24.835899+0000","last_clean":"2024-09-15T21:46:24.835899+0000","last_became_active":"2024-09-15T21:46:23.430987+0000","last_became_peered":"2024-09-15T21:46:23.430987+0000","last_unstale":"2024-09-15T21:46:24.835899+0000","last_undegraded":"2024-09-15T21:46:24.835899+0000","last_fullsized":"2024-09-15T21:46:24.835899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:48:30.532378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835198+0000","last_change":"2024-09-15T21:46:23.434558+0000","last_active":"2024-09-15T21:46:24.835198+0000","last_peered":"2024-09-15T21:46:24.835198+0000","last_clean":"2024-09-15T21:46:24.835198+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:46:24.835198+0000","last_undegraded":"2024-09-15T21:46:24.835198+0000","last_fullsized":"2024-09-15T21:46:24.835198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:03:39.562977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727140+0000","last_change":"2024-09-15T21:46:23.428323+0000","last_active":"2024-09-15T21:46:24.727140+0000","last_peered":"2024-09-15T21:46:24.727140+0000","last_clean":"2024-09-15T21:46:24.727140+0000","last_became_active":"2024-09-15T21:46:23.428016+0000","last_became_peered":"2024-09-15T21:46:23.428016+0000","last_unstale":"2024-09-15T21:46:24.727140+0000","last_undegraded":"2024-09-15T21:46:24.727140+0000","last_fullsized":"2024-09-15T21:46:24.727140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:28:48.678550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414612+0000","last_change":"2024-09-15T21:46:23.434465+0000","last_active":"2024-09-15T21:46:24.414612+0000","last_peered":"2024-09-15T21:46:24.414612+0000","last_clean":"2024-09-15T21:46:24.414612+0000","last_became_active":"2024-09-15T21:46:23.433849+0000","last_became_peered":"2024-09-15T21:46:23.433849+0000","last_unstale":"2024-09-15T21:46:24.414612+0000","last_undegraded":"2024-09-15T21:46:24.414612+0000","last_fullsized":"2024-09-15T21:46:24.414612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:46.093198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834522+0000","last_change":"2024-09-15T21:46:23.431527+0000","last_active":"2024-09-15T21:46:24.834522+0000","last_peered":"2024-09-15T21:46:24.834522+0000","last_clean":"2024-09-15T21:46:24.834522+0000","last_became_active":"2024-09-15T21:46:23.430719+0000","last_became_peered":"2024-09-15T21:46:23.430719+0000","last_unstale":"2024-09-15T21:46:24.834522+0000","last_undegraded":"2024-09-15T21:46:24.834522+0000","last_fullsized":"2024-09-15T21:46:24.834522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:16.326529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414532+0000","last_change":"2024-09-15T21:46:23.432282+0000","last_active":"2024-09-15T21:46:24.414532+0000","last_peered":"2024-09-15T21:46:24.414532+0000","last_clean":"2024-09-15T21:46:24.414532+0000","last_became_active":"2024-09-15T21:46:23.432035+0000","last_became_peered":"2024-09-15T21:46:23.432035+0000","last_unstale":"2024-09-15T21:46:24.414532+0000","last_undegraded":"2024-09-15T21:46:24.414532+0000","last_fullsized":"2024-09-15T21:46:24.414532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:22.163940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834776+0000","last_change":"2024-09-15T21:46:23.432646+0000","last_active":"2024-09-15T21:46:24.834776+0000","last_peered":"2024-09-15T21:46:24.834776+0000","last_clean":"2024-09-15T21:46:24.834776+0000","last_became_active":"2024-09-15T21:46:23.432317+0000","last_became_peered":"2024-09-15T21:46:23.432317+0000","last_unstale":"2024-09-15T21:46:24.834776+0000","last_undegraded":"2024-09-15T21:46:24.834776+0000","last_fullsized":"2024-09-15T21:46:24.834776+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:49:07.706450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835541+0000","last_change":"2024-09-15T21:46:23.428183+0000","last_active":"2024-09-15T21:46:24.835541+0000","last_peered":"2024-09-15T21:46:24.835541+0000","last_clean":"2024-09-15T21:46:24.835541+0000","last_became_active":"2024-09-15T21:46:23.427886+0000","last_became_peered":"2024-09-15T21:46:23.427886+0000","last_unstale":"2024-09-15T21:46:24.835541+0000","last_undegraded":"2024-09-15T21:46:24.835541+0000","last_fullsized":"2024-09-15T21:46:24.835541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:15.584377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808369+0000","last_change":"2024-09-15T21:46:23.421749+0000","last_active":"2024-09-15T21:46:24.808369+0000","last_peered":"2024-09-15T21:46:24.808369+0000","last_clean":"2024-09-15T21:46:24.808369+0000","last_became_active":"2024-09-15T21:46:23.421501+0000","last_became_peered":"2024-09-15T21:46:23.421501+0000","last_unstale":"2024-09-15T21:46:24.808369+0000","last_undegraded":"2024-09-15T21:46:24.808369+0000","last_fullsized":"2024-09-15T21:46:24.808369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:55.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834843+0000","last_change":"2024-09-15T21:46:23.432591+0000","last_active":"2024-09-15T21:46:24.834843+0000","last_peered":"2024-09-15T21:46:24.834843+0000","last_clean":"2024-09-15T21:46:24.834843+0000","last_became_active":"2024-09-15T21:46:23.432326+0000","last_became_peered":"2024-09-15T21:46:23.432326+0000","last_unstale":"2024-09-15T21:46:24.834843+0000","last_undegraded":"2024-09-15T21:46:24.834843+0000","last_fullsized":"2024-09-15T21:46:24.834843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:48.565579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727016+0000","last_change":"2024-09-15T21:46:23.423428+0000","last_active":"2024-09-15T21:46:24.727016+0000","last_peered":"2024-09-15T21:46:24.727016+0000","last_clean":"2024-09-15T21:46:24.727016+0000","last_became_active":"2024-09-15T21:46:23.422734+0000","last_became_peered":"2024-09-15T21:46:23.422734+0000","last_unstale":"2024-09-15T21:46:24.727016+0000","last_undegraded":"2024-09-15T21:46:24.727016+0000","last_fullsized":"2024-09-15T21:46:24.727016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:06.592891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414770+0000","last_change":"2024-09-15T21:46:23.435244+0000","last_active":"2024-09-15T21:46:24.414770+0000","last_peered":"2024-09-15T21:46:24.414770+0000","last_clean":"2024-09-15T21:46:24.414770+0000","last_became_active":"2024-09-15T21:46:23.434932+0000","last_became_peered":"2024-09-15T21:46:23.434932+0000","last_unstale":"2024-09-15T21:46:24.414770+0000","last_undegraded":"2024-09-15T21:46:24.414770+0000","last_fullsized":"2024-09-15T21:46:24.414770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:55.810146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412500+0000","last_change":"2024-09-15T21:46:23.437276+0000","last_active":"2024-09-15T21:46:24.412500+0000","last_peered":"2024-09-15T21:46:24.412500+0000","last_clean":"2024-09-15T21:46:24.412500+0000","last_became_active":"2024-09-15T21:46:23.436962+0000","last_became_peered":"2024-09-15T21:46:23.436962+0000","last_unstale":"2024-09-15T21:46:24.412500+0000","last_undegraded":"2024-09-15T21:46:24.412500+0000","last_fullsized":"2024-09-15T21:46:24.412500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:13.043999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834825+0000","last_change":"2024-09-15T21:46:23.429648+0000","last_active":"2024-09-15T21:46:24.834825+0000","last_peered":"2024-09-15T21:46:24.834825+0000","last_clean":"2024-09-15T21:46:24.834825+0000","last_became_active":"2024-09-15T21:46:23.429376+0000","last_became_peered":"2024-09-15T21:46:23.429376+0000","last_unstale":"2024-09-15T21:46:24.834825+0000","last_undegraded":"2024-09-15T21:46:24.834825+0000","last_fullsized":"2024-09-15T21:46:24.834825+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:31.278283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.413663+0000","last_change":"2024-09-15T21:46:23.429847+0000","last_active":"2024-09-15T21:46:24.413663+0000","last_peered":"2024-09-15T21:46:24.413663+0000","last_clean":"2024-09-15T21:46:24.413663+0000","last_became_active":"2024-09-15T21:46:23.429442+0000","last_became_peered":"2024-09-15T21:46:23.429442+0000","last_unstale":"2024-09-15T21:46:24.413663+0000","last_undegraded":"2024-09-15T21:46:24.413663+0000","last_fullsized":"2024-09-15T21:46:24.413663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:31.232512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:34.464539+0000","last_change":"2024-09-15T21:46:22.429407+0000","last_active":"2024-09-15T21:46:34.464539+0000","last_peered":"2024-09-15T21:46:34.464539+0000","last_clean":"2024-09-15T21:46:34.464539+0000","last_became_active":"2024-09-15T21:46:22.429107+0000","last_became_peered":"2024-09-15T21:46:22.429107+0000","last_unstale":"2024-09-15T21:46:34.464539+0000","last_undegraded":"2024-09-15T21:46:34.464539+0000","last_fullsized":"2024-09-15T21:46:34.464539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:14:54.680349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727169+0000","last_change":"2024-09-15T21:46:23.434198+0000","last_active":"2024-09-15T21:46:24.727169+0000","last_peered":"2024-09-15T21:46:24.727169+0000","last_clean":"2024-09-15T21:46:24.727169+0000","last_became_active":"2024-09-15T21:46:23.433944+0000","last_became_peered":"2024-09-15T21:46:23.433944+0000","last_unstale":"2024-09-15T21:46:24.727169+0000","last_undegraded":"2024-09-15T21:46:24.727169+0000","last_fullsized":"2024-09-15T21:46:24.727169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:25.412167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834914+0000","last_change":"2024-09-15T21:46:22.425571+0000","last_active":"2024-09-15T21:46:24.834914+0000","last_peered":"2024-09-15T21:46:24.834914+0000","last_clean":"2024-09-15T21:46:24.834914+0000","last_became_active":"2024-09-15T21:46:22.425315+0000","last_became_peered":"2024-09-15T21:46:22.425315+0000","last_unstale":"2024-09-15T21:46:24.834914+0000","last_undegraded":"2024-09-15T21:46:24.834914+0000","last_fullsized":"2024-09-15T21:46:24.834914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:31.983301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834481+0000","last_change":"2024-09-15T21:46:22.425492+0000","last_active":"2024-09-15T21:46:24.834481+0000","last_peered":"2024-09-15T21:46:24.834481+0000","last_clean":"2024-09-15T21:46:24.834481+0000","last_became_active":"2024-09-15T21:46:22.425226+0000","last_became_peered":"2024-09-15T21:46:22.425226+0000","last_unstale":"2024-09-15T21:46:24.834481+0000","last_undegraded":"2024-09-15T21:46:24.834481+0000","last_fullsized":"2024-09-15T21:46:24.834481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:24.322384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834460+0000","last_change":"2024-09-15T21:46:23.429140+0000","last_active":"2024-09-15T21:46:24.834460+0000","last_peered":"2024-09-15T21:46:24.834460+0000","last_clean":"2024-09-15T21:46:24.834460+0000","last_became_active":"2024-09-15T21:46:23.428902+0000","last_became_peered":"2024-09-15T21:46:23.428902+0000","last_unstale":"2024-09-15T21:46:24.834460+0000","last_undegraded":"2024-09-15T21:46:24.834460+0000","last_fullsized":"2024-09-15T21:46:24.834460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:11.256924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414663+0000","last_change":"2024-09-15T21:46:22.428319+0000","last_active":"2024-09-15T21:46:24.414663+0000","last_peered":"2024-09-15T21:46:24.414663+0000","last_clean":"2024-09-15T21:46:24.414663+0000","last_became_active":"2024-09-15T21:46:22.428061+0000","last_became_peered":"2024-09-15T21:46:22.428061+0000","last_unstale":"2024-09-15T21:46:24.414663+0000","last_undegraded":"2024-09-15T21:46:24.414663+0000","last_fullsized":"2024-09-15T21:46:24.414663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:14.238988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808315+0000","last_change":"2024-09-15T21:46:23.427442+0000","last_active":"2024-09-15T21:46:24.808315+0000","last_peered":"2024-09-15T21:46:24.808315+0000","last_clean":"2024-09-15T21:46:24.808315+0000","last_became_active":"2024-09-15T21:46:23.427200+0000","last_became_peered":"2024-09-15T21:46:23.427200+0000","last_unstale":"2024-09-15T21:46:24.808315+0000","last_undegraded":"2024-09-15T21:46:24.808315+0000","last_fullsized":"2024-09-15T21:46:24.808315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:27.248545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835085+0000","last_change":"2024-09-15T21:46:22.430599+0000","last_active":"2024-09-15T21:46:24.835085+0000","last_peered":"2024-09-15T21:46:24.835085+0000","last_clean":"2024-09-15T21:46:24.835085+0000","last_became_active":"2024-09-15T21:46:22.430327+0000","last_became_peered":"2024-09-15T21:46:22.430327+0000","last_unstale":"2024-09-15T21:46:24.835085+0000","last_undegraded":"2024-09-15T21:46:24.835085+0000","last_fullsized":"2024-09-15T21:46:24.835085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:51.904693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414860+0000","last_change":"2024-09-15T21:46:23.434006+0000","last_active":"2024-09-15T21:46:24.414860+0000","last_peered":"2024-09-15T21:46:24.414860+0000","last_clean":"2024-09-15T21:46:24.414860+0000","last_became_active":"2024-09-15T21:46:23.433634+0000","last_became_peered":"2024-09-15T21:46:23.433634+0000","last_unstale":"2024-09-15T21:46:24.414860+0000","last_undegraded":"2024-09-15T21:46:24.414860+0000","last_fullsized":"2024-09-15T21:46:24.414860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:22.288806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414976+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:24.414976+0000","last_peered":"2024-09-15T21:46:24.414976+0000","last_clean":"2024-09-15T21:46:24.414976+0000","last_became_active":"2024-09-15T21:46:22.425236+0000","last_became_peered":"2024-09-15T21:46:22.425236+0000","last_unstale":"2024-09-15T21:46:24.414976+0000","last_undegraded":"2024-09-15T21:46:24.414976+0000","last_fullsized":"2024-09-15T21:46:24.414976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:45:21.332492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727237+0000","last_change":"2024-09-15T21:46:23.429587+0000","last_active":"2024-09-15T21:46:24.727237+0000","last_peered":"2024-09-15T21:46:24.727237+0000","last_clean":"2024-09-15T21:46:24.727237+0000","last_became_active":"2024-09-15T21:46:23.429334+0000","last_became_peered":"2024-09-15T21:46:23.429334+0000","last_unstale":"2024-09-15T21:46:24.727237+0000","last_undegraded":"2024-09-15T21:46:24.727237+0000","last_fullsized":"2024-09-15T21:46:24.727237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:48:56.361550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.417234+0000","last_change":"2024-09-15T21:46:22.421964+0000","last_active":"2024-09-15T21:46:28.417234+0000","last_peered":"2024-09-15T21:46:28.417234+0000","last_clean":"2024-09-15T21:46:28.417234+0000","last_became_active":"2024-09-15T21:46:22.421661+0000","last_became_peered":"2024-09-15T21:46:22.421661+0000","last_unstale":"2024-09-15T21:46:28.417234+0000","last_undegraded":"2024-09-15T21:46:28.417234+0000","last_fullsized":"2024-09-15T21:46:28.417234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:07.395070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414253+0000","last_change":"2024-09-15T21:46:23.427416+0000","last_active":"2024-09-15T21:46:24.414253+0000","last_peered":"2024-09-15T21:46:24.414253+0000","last_clean":"2024-09-15T21:46:24.414253+0000","last_became_active":"2024-09-15T21:46:23.427057+0000","last_became_peered":"2024-09-15T21:46:23.427057+0000","last_unstale":"2024-09-15T21:46:24.414253+0000","last_undegraded":"2024-09-15T21:46:24.414253+0000","last_fullsized":"2024-09-15T21:46:24.414253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:42.671501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.422249+0000","last_change":"2024-09-15T21:46:22.426151+0000","last_active":"2024-09-15T21:46:28.422249+0000","last_peered":"2024-09-15T21:46:28.422249+0000","last_clean":"2024-09-15T21:46:28.422249+0000","last_became_active":"2024-09-15T21:46:22.425866+0000","last_became_peered":"2024-09-15T21:46:22.425866+0000","last_unstale":"2024-09-15T21:46:28.422249+0000","last_undegraded":"2024-09-15T21:46:28.422249+0000","last_fullsized":"2024-09-15T21:46:28.422249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:10.684265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835569+0000","last_change":"2024-09-15T21:46:23.432292+0000","last_active":"2024-09-15T21:46:24.835569+0000","last_peered":"2024-09-15T21:46:24.835569+0000","last_clean":"2024-09-15T21:46:24.835569+0000","last_became_active":"2024-09-15T21:46:23.431877+0000","last_became_peered":"2024-09-15T21:46:23.431877+0000","last_unstale":"2024-09-15T21:46:24.835569+0000","last_undegraded":"2024-09-15T21:46:24.835569+0000","last_fullsized":"2024-09-15T21:46:24.835569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:16.857279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.417259+0000","last_change":"2024-09-15T21:46:22.427415+0000","last_active":"2024-09-15T21:46:28.417259+0000","last_peered":"2024-09-15T21:46:28.417259+0000","last_clean":"2024-09-15T21:46:28.417259+0000","last_became_active":"2024-09-15T21:46:22.427087+0000","last_became_peered":"2024-09-15T21:46:22.427087+0000","last_unstale":"2024-09-15T21:46:28.417259+0000","last_undegraded":"2024-09-15T21:46:28.417259+0000","last_fullsized":"2024-09-15T21:46:28.417259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:09.365202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.726943+0000","last_change":"2024-09-15T21:46:23.427238+0000","last_active":"2024-09-15T21:46:24.726943+0000","last_peered":"2024-09-15T21:46:24.726943+0000","last_clean":"2024-09-15T21:46:24.726943+0000","last_became_active":"2024-09-15T21:46:23.426993+0000","last_became_peered":"2024-09-15T21:46:23.426993+0000","last_unstale":"2024-09-15T21:46:24.726943+0000","last_undegraded":"2024-09-15T21:46:24.726943+0000","last_fullsized":"2024-09-15T21:46:24.726943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:52:24.175554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414781+0000","last_change":"2024-09-15T21:46:22.427164+0000","last_active":"2024-09-15T21:46:24.414781+0000","last_peered":"2024-09-15T21:46:24.414781+0000","last_clean":"2024-09-15T21:46:24.414781+0000","last_became_active":"2024-09-15T21:46:22.426917+0000","last_became_peered":"2024-09-15T21:46:22.426917+0000","last_unstale":"2024-09-15T21:46:24.414781+0000","last_undegraded":"2024-09-15T21:46:24.414781+0000","last_fullsized":"2024-09-15T21:46:24.414781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:46.471935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835504+0000","last_change":"2024-09-15T21:46:23.433924+0000","last_active":"2024-09-15T21:46:24.835504+0000","last_peered":"2024-09-15T21:46:24.835504+0000","last_clean":"2024-09-15T21:46:24.835504+0000","last_became_active":"2024-09-15T21:46:23.433606+0000","last_became_peered":"2024-09-15T21:46:23.433606+0000","last_unstale":"2024-09-15T21:46:24.835504+0000","last_undegraded":"2024-09-15T21:46:24.835504+0000","last_fullsized":"2024-09-15T21:46:24.835504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:09:35.744149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834887+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:24.834887+0000","last_peered":"2024-09-15T21:46:24.834887+0000","last_clean":"2024-09-15T21:46:24.834887+0000","last_became_active":"2024-09-15T21:46:22.425237+0000","last_became_peered":"2024-09-15T21:46:22.425237+0000","last_unstale":"2024-09-15T21:46:24.834887+0000","last_undegraded":"2024-09-15T21:46:24.834887+0000","last_fullsized":"2024-09-15T21:46:24.834887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:59.460420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834853+0000","last_change":"2024-09-15T21:46:23.427515+0000","last_active":"2024-09-15T21:46:24.834853+0000","last_peered":"2024-09-15T21:46:24.834853+0000","last_clean":"2024-09-15T21:46:24.834853+0000","last_became_active":"2024-09-15T21:46:23.427116+0000","last_became_peered":"2024-09-15T21:46:23.427116+0000","last_unstale":"2024-09-15T21:46:24.834853+0000","last_undegraded":"2024-09-15T21:46:24.834853+0000","last_fullsized":"2024-09-15T21:46:24.834853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:35.499970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.422679+0000","last_change":"2024-09-15T21:46:22.419905+0000","last_active":"2024-09-15T21:46:28.422679+0000","last_peered":"2024-09-15T21:46:28.422679+0000","last_clean":"2024-09-15T21:46:28.422679+0000","last_became_active":"2024-09-15T21:46:22.419620+0000","last_became_peered":"2024-09-15T21:46:22.419620+0000","last_unstale":"2024-09-15T21:46:28.422679+0000","last_undegraded":"2024-09-15T21:46:28.422679+0000","last_fullsized":"2024-09-15T21:46:28.422679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:26:51.215156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835459+0000","last_change":"2024-09-15T21:46:23.432975+0000","last_active":"2024-09-15T21:46:24.835459+0000","last_peered":"2024-09-15T21:46:24.835459+0000","last_clean":"2024-09-15T21:46:24.835459+0000","last_became_active":"2024-09-15T21:46:23.432718+0000","last_became_peered":"2024-09-15T21:46:23.432718+0000","last_unstale":"2024-09-15T21:46:24.835459+0000","last_undegraded":"2024-09-15T21:46:24.835459+0000","last_fullsized":"2024-09-15T21:46:24.835459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:59.789691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414365+0000","last_change":"2024-09-15T21:46:22.429661+0000","last_active":"2024-09-15T21:46:24.414365+0000","last_peered":"2024-09-15T21:46:24.414365+0000","last_clean":"2024-09-15T21:46:24.414365+0000","last_became_active":"2024-09-15T21:46:22.429250+0000","last_became_peered":"2024-09-15T21:46:22.429250+0000","last_unstale":"2024-09-15T21:46:24.414365+0000","last_undegraded":"2024-09-15T21:46:24.414365+0000","last_fullsized":"2024-09-15T21:46:24.414365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:29.341739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835590+0000","last_change":"2024-09-15T21:46:23.435003+0000","last_active":"2024-09-15T21:46:24.835590+0000","last_peered":"2024-09-15T21:46:24.835590+0000","last_clean":"2024-09-15T21:46:24.835590+0000","last_became_active":"2024-09-15T21:46:23.434738+0000","last_became_peered":"2024-09-15T21:46:23.434738+0000","last_unstale":"2024-09-15T21:46:24.835590+0000","last_undegraded":"2024-09-15T21:46:24.835590+0000","last_fullsized":"2024-09-15T21:46:24.835590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:53:15.814759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.417670+0000","last_change":"2024-09-15T21:46:22.425870+0000","last_active":"2024-09-15T21:46:28.417670+0000","last_peered":"2024-09-15T21:46:28.417670+0000","last_clean":"2024-09-15T21:46:28.417670+0000","last_became_active":"2024-09-15T21:46:22.425588+0000","last_became_peered":"2024-09-15T21:46:22.425588+0000","last_unstale":"2024-09-15T21:46:28.417670+0000","last_undegraded":"2024-09-15T21:46:28.417670+0000","last_fullsized":"2024-09-15T21:46:28.417670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:40:07.586608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835837+0000","last_change":"2024-09-15T21:46:23.434657+0000","last_active":"2024-09-15T21:46:24.835837+0000","last_peered":"2024-09-15T21:46:24.835837+0000","last_clean":"2024-09-15T21:46:24.835837+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:46:24.835837+0000","last_undegraded":"2024-09-15T21:46:24.835837+0000","last_fullsized":"2024-09-15T21:46:24.835837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:02.545810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808473+0000","last_change":"2024-09-15T21:46:22.425072+0000","last_active":"2024-09-15T21:46:24.808473+0000","last_peered":"2024-09-15T21:46:24.808473+0000","last_clean":"2024-09-15T21:46:24.808473+0000","last_became_active":"2024-09-15T21:46:22.424771+0000","last_became_peered":"2024-09-15T21:46:22.424771+0000","last_unstale":"2024-09-15T21:46:24.808473+0000","last_undegraded":"2024-09-15T21:46:24.808473+0000","last_fullsized":"2024-09-15T21:46:24.808473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:26.256785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414457+0000","last_change":"2024-09-15T21:46:23.432497+0000","last_active":"2024-09-15T21:46:24.414457+0000","last_peered":"2024-09-15T21:46:24.414457+0000","last_clean":"2024-09-15T21:46:24.414457+0000","last_became_active":"2024-09-15T21:46:23.431969+0000","last_became_peered":"2024-09-15T21:46:23.431969+0000","last_unstale":"2024-09-15T21:46:24.414457+0000","last_undegraded":"2024-09-15T21:46:24.414457+0000","last_fullsized":"2024-09-15T21:46:24.414457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:32:56.359477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412577+0000","last_change":"2024-09-15T21:46:22.428668+0000","last_active":"2024-09-15T21:46:24.412577+0000","last_peered":"2024-09-15T21:46:24.412577+0000","last_clean":"2024-09-15T21:46:24.412577+0000","last_became_active":"2024-09-15T21:46:22.428341+0000","last_became_peered":"2024-09-15T21:46:22.428341+0000","last_unstale":"2024-09-15T21:46:24.412577+0000","last_undegraded":"2024-09-15T21:46:24.412577+0000","last_fullsized":"2024-09-15T21:46:24.412577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:29:45.521090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835717+0000","last_change":"2024-09-15T21:46:23.438665+0000","last_active":"2024-09-15T21:46:24.835717+0000","last_peered":"2024-09-15T21:46:24.835717+0000","last_clean":"2024-09-15T21:46:24.835717+0000","last_became_active":"2024-09-15T21:46:23.437606+0000","last_became_peered":"2024-09-15T21:46:23.437606+0000","last_unstale":"2024-09-15T21:46:24.835717+0000","last_undegraded":"2024-09-15T21:46:24.835717+0000","last_fullsized":"2024-09-15T21:46:24.835717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:14:53.751884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":171,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835126+0000","last_change":"2024-09-15T21:46:14.328094+0000","last_active":"2024-09-15T21:46:24.835126+0000","last_peered":"2024-09-15T21:46:24.835126+0000","last_clean":"2024-09-15T21:46:24.835126+0000","last_became_active":"2024-09-15T21:46:14.326991+0000","last_became_peered":"2024-09-15T21:46:14.326991+0000","last_unstale":"2024-09-15T21:46:24.835126+0000","last_undegraded":"2024-09-15T21:46:24.835126+0000","last_fullsized":"2024-09-15T21:46:24.835126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_clean_scrub_stamp":"2024-09-15T21:46:12.223400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:54:27.704130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.413799+0000","last_change":"2024-09-15T21:46:22.428887+0000","last_active":"2024-09-15T21:46:24.413799+0000","last_peered":"2024-09-15T21:46:24.413799+0000","last_clean":"2024-09-15T21:46:24.413799+0000","last_became_active":"2024-09-15T21:46:22.428619+0000","last_became_peered":"2024-09-15T21:46:22.428619+0000","last_unstale":"2024-09-15T21:46:24.413799+0000","last_undegraded":"2024-09-15T21:46:24.413799+0000","last_fullsized":"2024-09-15T21:46:24.413799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:48.434763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834644+0000","last_change":"2024-09-15T21:46:23.435459+0000","last_active":"2024-09-15T21:46:24.834644+0000","last_peered":"2024-09-15T21:46:24.834644+0000","last_clean":"2024-09-15T21:46:24.834644+0000","last_became_active":"2024-09-15T21:46:23.435141+0000","last_became_peered":"2024-09-15T21:46:23.435141+0000","last_unstale":"2024-09-15T21:46:24.834644+0000","last_undegraded":"2024-09-15T21:46:24.834644+0000","last_fullsized":"2024-09-15T21:46:24.834644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:40:50.707124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834903+0000","last_change":"2024-09-15T21:46:22.431113+0000","last_active":"2024-09-15T21:46:24.834903+0000","last_peered":"2024-09-15T21:46:24.834903+0000","last_clean":"2024-09-15T21:46:24.834903+0000","last_became_active":"2024-09-15T21:46:22.430832+0000","last_became_peered":"2024-09-15T21:46:22.430832+0000","last_unstale":"2024-09-15T21:46:24.834903+0000","last_undegraded":"2024-09-15T21:46:24.834903+0000","last_fullsized":"2024-09-15T21:46:24.834903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:28.849259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.726839+0000","last_change":"2024-09-15T21:46:23.430396+0000","last_active":"2024-09-15T21:46:24.726839+0000","last_peered":"2024-09-15T21:46:24.726839+0000","last_clean":"2024-09-15T21:46:24.726839+0000","last_became_active":"2024-09-15T21:46:23.430138+0000","last_became_peered":"2024-09-15T21:46:23.430138+0000","last_unstale":"2024-09-15T21:46:24.726839+0000","last_undegraded":"2024-09-15T21:46:24.726839+0000","last_fullsized":"2024-09-15T21:46:24.726839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:50.622949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412620+0000","last_change":"2024-09-15T21:46:22.425808+0000","last_active":"2024-09-15T21:46:24.412620+0000","last_peered":"2024-09-15T21:46:24.412620+0000","last_clean":"2024-09-15T21:46:24.412620+0000","last_became_active":"2024-09-15T21:46:22.425510+0000","last_became_peered":"2024-09-15T21:46:22.425510+0000","last_unstale":"2024-09-15T21:46:24.412620+0000","last_undegraded":"2024-09-15T21:46:24.412620+0000","last_fullsized":"2024-09-15T21:46:24.412620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:11:00.338076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808338+0000","last_change":"2024-09-15T21:46:23.427051+0000","last_active":"2024-09-15T21:46:24.808338+0000","last_peered":"2024-09-15T21:46:24.808338+0000","last_clean":"2024-09-15T21:46:24.808338+0000","last_became_active":"2024-09-15T21:46:23.426185+0000","last_became_peered":"2024-09-15T21:46:23.426185+0000","last_unstale":"2024-09-15T21:46:24.808338+0000","last_undegraded":"2024-09-15T21:46:24.808338+0000","last_fullsized":"2024-09-15T21:46:24.808338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:48.318405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414866+0000","last_change":"2024-09-15T21:46:22.424645+0000","last_active":"2024-09-15T21:46:24.414866+0000","last_peered":"2024-09-15T21:46:24.414866+0000","last_clean":"2024-09-15T21:46:24.414866+0000","last_became_active":"2024-09-15T21:46:22.421229+0000","last_became_peered":"2024-09-15T21:46:22.421229+0000","last_unstale":"2024-09-15T21:46:24.414866+0000","last_undegraded":"2024-09-15T21:46:24.414866+0000","last_fullsized":"2024-09-15T21:46:24.414866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:49.165670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414000+0000","last_change":"2024-09-15T21:46:23.425699+0000","last_active":"2024-09-15T21:46:24.414000+0000","last_peered":"2024-09-15T21:46:24.414000+0000","last_clean":"2024-09-15T21:46:24.414000+0000","last_became_active":"2024-09-15T21:46:23.425416+0000","last_became_peered":"2024-09-15T21:46:23.425416+0000","last_unstale":"2024-09-15T21:46:24.414000+0000","last_undegraded":"2024-09-15T21:46:24.414000+0000","last_fullsized":"2024-09-15T21:46:24.414000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:00.325179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808438+0000","last_change":"2024-09-15T21:46:22.426492+0000","last_active":"2024-09-15T21:46:24.808438+0000","last_peered":"2024-09-15T21:46:24.808438+0000","last_clean":"2024-09-15T21:46:24.808438+0000","last_became_active":"2024-09-15T21:46:22.426210+0000","last_became_peered":"2024-09-15T21:46:22.426210+0000","last_unstale":"2024-09-15T21:46:24.808438+0000","last_undegraded":"2024-09-15T21:46:24.808438+0000","last_fullsized":"2024-09-15T21:46:24.808438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:05.248091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414861+0000","last_change":"2024-09-15T21:46:23.429837+0000","last_active":"2024-09-15T21:46:24.414861+0000","last_peered":"2024-09-15T21:46:24.414861+0000","last_clean":"2024-09-15T21:46:24.414861+0000","last_became_active":"2024-09-15T21:46:23.429254+0000","last_became_peered":"2024-09-15T21:46:23.429254+0000","last_unstale":"2024-09-15T21:46:24.414861+0000","last_undegraded":"2024-09-15T21:46:24.414861+0000","last_fullsized":"2024-09-15T21:46:24.414861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:58.465583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835937+0000","last_change":"2024-09-15T21:46:22.430710+0000","last_active":"2024-09-15T21:46:24.835937+0000","last_peered":"2024-09-15T21:46:24.835937+0000","last_clean":"2024-09-15T21:46:24.835937+0000","last_became_active":"2024-09-15T21:46:22.430375+0000","last_became_peered":"2024-09-15T21:46:22.430375+0000","last_unstale":"2024-09-15T21:46:24.835937+0000","last_undegraded":"2024-09-15T21:46:24.835937+0000","last_fullsized":"2024-09-15T21:46:24.835937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:04.081179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808447+0000","last_change":"2024-09-15T21:46:23.426729+0000","last_active":"2024-09-15T21:46:24.808447+0000","last_peered":"2024-09-15T21:46:24.808447+0000","last_clean":"2024-09-15T21:46:24.808447+0000","last_became_active":"2024-09-15T21:46:23.426495+0000","last_became_peered":"2024-09-15T21:46:23.426495+0000","last_unstale":"2024-09-15T21:46:24.808447+0000","last_undegraded":"2024-09-15T21:46:24.808447+0000","last_fullsized":"2024-09-15T21:46:24.808447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:03.277997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808436+0000","last_change":"2024-09-15T21:46:22.427137+0000","last_active":"2024-09-15T21:46:24.808436+0000","last_peered":"2024-09-15T21:46:24.808436+0000","last_clean":"2024-09-15T21:46:24.808436+0000","last_became_active":"2024-09-15T21:46:22.426855+0000","last_became_peered":"2024-09-15T21:46:22.426855+0000","last_unstale":"2024-09-15T21:46:24.808436+0000","last_undegraded":"2024-09-15T21:46:24.808436+0000","last_fullsized":"2024-09-15T21:46:24.808436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:40:14.277940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834714+0000","last_change":"2024-09-15T21:46:23.427327+0000","last_active":"2024-09-15T21:46:24.834714+0000","last_peered":"2024-09-15T21:46:24.834714+0000","last_clean":"2024-09-15T21:46:24.834714+0000","last_became_active":"2024-09-15T21:46:23.427096+0000","last_became_peered":"2024-09-15T21:46:23.427096+0000","last_unstale":"2024-09-15T21:46:24.834714+0000","last_undegraded":"2024-09-15T21:46:24.834714+0000","last_fullsized":"2024-09-15T21:46:24.834714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:32.370390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.418471+0000","last_change":"2024-09-15T21:46:22.427798+0000","last_active":"2024-09-15T21:46:28.418471+0000","last_peered":"2024-09-15T21:46:28.418471+0000","last_clean":"2024-09-15T21:46:28.418471+0000","last_became_active":"2024-09-15T21:46:22.427537+0000","last_became_peered":"2024-09-15T21:46:22.427537+0000","last_unstale":"2024-09-15T21:46:28.418471+0000","last_undegraded":"2024-09-15T21:46:28.418471+0000","last_fullsized":"2024-09-15T21:46:28.418471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:18:54.963834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727157+0000","last_change":"2024-09-15T21:46:23.424266+0000","last_active":"2024-09-15T21:46:24.727157+0000","last_peered":"2024-09-15T21:46:24.727157+0000","last_clean":"2024-09-15T21:46:24.727157+0000","last_became_active":"2024-09-15T21:46:23.424007+0000","last_became_peered":"2024-09-15T21:46:23.424007+0000","last_unstale":"2024-09-15T21:46:24.727157+0000","last_undegraded":"2024-09-15T21:46:24.727157+0000","last_fullsized":"2024-09-15T21:46:24.727157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:23.492253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835005+0000","last_change":"2024-09-15T21:46:22.430901+0000","last_active":"2024-09-15T21:46:24.835005+0000","last_peered":"2024-09-15T21:46:24.835005+0000","last_clean":"2024-09-15T21:46:24.835005+0000","last_became_active":"2024-09-15T21:46:22.430452+0000","last_became_peered":"2024-09-15T21:46:22.430452+0000","last_unstale":"2024-09-15T21:46:24.835005+0000","last_undegraded":"2024-09-15T21:46:24.835005+0000","last_fullsized":"2024-09-15T21:46:24.835005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:59:45.663976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.413661+0000","last_change":"2024-09-15T21:46:23.430836+0000","last_active":"2024-09-15T21:46:24.413661+0000","last_peered":"2024-09-15T21:46:24.413661+0000","last_clean":"2024-09-15T21:46:24.413661+0000","last_became_active":"2024-09-15T21:46:23.429283+0000","last_became_peered":"2024-09-15T21:46:23.429283+0000","last_unstale":"2024-09-15T21:46:24.413661+0000","last_undegraded":"2024-09-15T21:46:24.413661+0000","last_fullsized":"2024-09-15T21:46:24.413661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:30.571376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808380+0000","last_change":"2024-09-15T21:46:22.423795+0000","last_active":"2024-09-15T21:46:24.808380+0000","last_peered":"2024-09-15T21:46:24.808380+0000","last_clean":"2024-09-15T21:46:24.808380+0000","last_became_active":"2024-09-15T21:46:22.422327+0000","last_became_peered":"2024-09-15T21:46:22.422327+0000","last_unstale":"2024-09-15T21:46:24.808380+0000","last_undegraded":"2024-09-15T21:46:24.808380+0000","last_fullsized":"2024-09-15T21:46:24.808380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:16.670599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834522+0000","last_change":"2024-09-15T21:46:23.428634+0000","last_active":"2024-09-15T21:46:24.834522+0000","last_peered":"2024-09-15T21:46:24.834522+0000","last_clean":"2024-09-15T21:46:24.834522+0000","last_became_active":"2024-09-15T21:46:23.428301+0000","last_became_peered":"2024-09-15T21:46:23.428301+0000","last_unstale":"2024-09-15T21:46:24.834522+0000","last_undegraded":"2024-09-15T21:46:24.834522+0000","last_fullsized":"2024-09-15T21:46:24.834522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:59.492942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808304+0000","last_change":"2024-09-15T21:46:22.425810+0000","last_active":"2024-09-15T21:46:24.808304+0000","last_peered":"2024-09-15T21:46:24.808304+0000","last_clean":"2024-09-15T21:46:24.808304+0000","last_became_active":"2024-09-15T21:46:22.425478+0000","last_became_peered":"2024-09-15T21:46:22.425478+0000","last_unstale":"2024-09-15T21:46:24.808304+0000","last_undegraded":"2024-09-15T21:46:24.808304+0000","last_fullsized":"2024-09-15T21:46:24.808304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:53.296599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834743+0000","last_change":"2024-09-15T21:46:23.426054+0000","last_active":"2024-09-15T21:46:24.834743+0000","last_peered":"2024-09-15T21:46:24.834743+0000","last_clean":"2024-09-15T21:46:24.834743+0000","last_became_active":"2024-09-15T21:46:23.425547+0000","last_became_peered":"2024-09-15T21:46:23.425547+0000","last_unstale":"2024-09-15T21:46:24.834743+0000","last_undegraded":"2024-09-15T21:46:24.834743+0000","last_fullsized":"2024-09-15T21:46:24.834743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:56.838503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.418146+0000","last_change":"2024-09-15T21:46:22.428524+0000","last_active":"2024-09-15T21:46:28.418146+0000","last_peered":"2024-09-15T21:46:28.418146+0000","last_clean":"2024-09-15T21:46:28.418146+0000","last_became_active":"2024-09-15T21:46:22.428043+0000","last_became_peered":"2024-09-15T21:46:22.428043+0000","last_unstale":"2024-09-15T21:46:28.418146+0000","last_undegraded":"2024-09-15T21:46:28.418146+0000","last_fullsized":"2024-09-15T21:46:28.418146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:08.393381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834850+0000","last_change":"2024-09-15T21:46:23.434719+0000","last_active":"2024-09-15T21:46:24.834850+0000","last_peered":"2024-09-15T21:46:24.834850+0000","last_clean":"2024-09-15T21:46:24.834850+0000","last_became_active":"2024-09-15T21:46:23.434448+0000","last_became_peered":"2024-09-15T21:46:23.434448+0000","last_unstale":"2024-09-15T21:46:24.834850+0000","last_undegraded":"2024-09-15T21:46:24.834850+0000","last_fullsized":"2024-09-15T21:46:24.834850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:14.901671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414691+0000","last_change":"2024-09-15T21:46:23.430058+0000","last_active":"2024-09-15T21:46:24.414691+0000","last_peered":"2024-09-15T21:46:24.414691+0000","last_clean":"2024-09-15T21:46:24.414691+0000","last_became_active":"2024-09-15T21:46:23.429598+0000","last_became_peered":"2024-09-15T21:46:23.429598+0000","last_unstale":"2024-09-15T21:46:24.414691+0000","last_undegraded":"2024-09-15T21:46:24.414691+0000","last_fullsized":"2024-09-15T21:46:24.414691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:44.142207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.418133+0000","last_change":"2024-09-15T21:46:22.431172+0000","last_active":"2024-09-15T21:46:28.418133+0000","last_peered":"2024-09-15T21:46:28.418133+0000","last_clean":"2024-09-15T21:46:28.418133+0000","last_became_active":"2024-09-15T21:46:22.430800+0000","last_became_peered":"2024-09-15T21:46:22.430800+0000","last_unstale":"2024-09-15T21:46:28.418133+0000","last_undegraded":"2024-09-15T21:46:28.418133+0000","last_fullsized":"2024-09-15T21:46:28.418133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:55.072466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414264+0000","last_change":"2024-09-15T21:46:23.434353+0000","last_active":"2024-09-15T21:46:24.414264+0000","last_peered":"2024-09-15T21:46:24.414264+0000","last_clean":"2024-09-15T21:46:24.414264+0000","last_became_active":"2024-09-15T21:46:23.434097+0000","last_became_peered":"2024-09-15T21:46:23.434097+0000","last_unstale":"2024-09-15T21:46:24.414264+0000","last_undegraded":"2024-09-15T21:46:24.414264+0000","last_fullsized":"2024-09-15T21:46:24.414264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:46.007227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.418130+0000","last_change":"2024-09-15T21:46:22.429870+0000","last_active":"2024-09-15T21:46:28.418130+0000","last_peered":"2024-09-15T21:46:28.418130+0000","last_clean":"2024-09-15T21:46:28.418130+0000","last_became_active":"2024-09-15T21:46:22.429578+0000","last_became_peered":"2024-09-15T21:46:22.429578+0000","last_unstale":"2024-09-15T21:46:28.418130+0000","last_undegraded":"2024-09-15T21:46:28.418130+0000","last_fullsized":"2024-09-15T21:46:28.418130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:41.678078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727142+0000","last_change":"2024-09-15T21:46:23.429604+0000","last_active":"2024-09-15T21:46:24.727142+0000","last_peered":"2024-09-15T21:46:24.727142+0000","last_clean":"2024-09-15T21:46:24.727142+0000","last_became_active":"2024-09-15T21:46:23.429355+0000","last_became_peered":"2024-09-15T21:46:23.429355+0000","last_unstale":"2024-09-15T21:46:24.727142+0000","last_undegraded":"2024-09-15T21:46:24.727142+0000","last_fullsized":"2024-09-15T21:46:24.727142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:45.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834679+0000","last_change":"2024-09-15T21:46:22.428529+0000","last_active":"2024-09-15T21:46:24.834679+0000","last_peered":"2024-09-15T21:46:24.834679+0000","last_clean":"2024-09-15T21:46:24.834679+0000","last_became_active":"2024-09-15T21:46:22.428263+0000","last_became_peered":"2024-09-15T21:46:22.428263+0000","last_unstale":"2024-09-15T21:46:24.834679+0000","last_undegraded":"2024-09-15T21:46:24.834679+0000","last_fullsized":"2024-09-15T21:46:24.834679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:22:40.160062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834701+0000","last_change":"2024-09-15T21:46:23.427968+0000","last_active":"2024-09-15T21:46:24.834701+0000","last_peered":"2024-09-15T21:46:24.834701+0000","last_clean":"2024-09-15T21:46:24.834701+0000","last_became_active":"2024-09-15T21:46:23.427546+0000","last_became_peered":"2024-09-15T21:46:23.427546+0000","last_unstale":"2024-09-15T21:46:24.834701+0000","last_undegraded":"2024-09-15T21:46:24.834701+0000","last_fullsized":"2024-09-15T21:46:24.834701+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:47.226097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.419172+0000","last_change":"2024-09-15T21:46:22.428258+0000","last_active":"2024-09-15T21:46:28.419172+0000","last_peered":"2024-09-15T21:46:28.419172+0000","last_clean":"2024-09-15T21:46:28.419172+0000","last_became_active":"2024-09-15T21:46:22.427954+0000","last_became_peered":"2024-09-15T21:46:22.427954+0000","last_unstale":"2024-09-15T21:46:28.419172+0000","last_undegraded":"2024-09-15T21:46:28.419172+0000","last_fullsized":"2024-09-15T21:46:28.419172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:22.440456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3032,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":5442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476743,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":667964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":819200,"data_stored":667908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":78570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":77048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":77084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":76958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":79192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:46:37.013 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:46:37.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.145+0000 7f43d9ad0640 1 Processor -- start 2024-09-15T21:46:37.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.146+0000 7f43d9ad0640 1 -- start start 2024-09-15T21:46:37.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.146+0000 7f43d9ad0640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f43d4072260 0x7f43d415bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:37.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.146+0000 7f43d9ad0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d40c8f40 con 0x7f43d40c86f0 2024-09-15T21:46:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.146+0000 7f43d9ad0640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d4157d70 con 0x7f43d4151fd0 2024-09-15T21:46:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.146+0000 7f43d9ad0640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f43d4158030 con 0x7f43d4072260 2024-09-15T21:46:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.146+0000 7f43d2ffd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f43d40c86f0 0x7f43d4071b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:54164/0 (socket says 172.21.15.100:54164) 2024-09-15T21:46:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.146+0000 7f43d2ffd640 1 -- 172.21.15.100:0/846850085 learned_addr learned my addr 172.21.15.100:0/846850085 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d3fff640 1 --2- 172.21.15.100:0/846850085 >> v2:172.21.15.122:3300/0 conn(0x7f43d4072260 0x7f43d415bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d27fc640 1 -- 172.21.15.100:0/846850085 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1969352259 0 0) 0x7f43d40c8f40 con 0x7f43d40c86f0 2024-09-15T21:46:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d27fc640 1 -- 172.21.15.100:0/846850085 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43c0003520 con 0x7f43d40c86f0 2024-09-15T21:46:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d27fc640 1 -- 172.21.15.100:0/846850085 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2709738147 0 0) 0x7f43c0003520 con 0x7f43d40c86f0 2024-09-15T21:46:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d27fc640 1 -- 172.21.15.100:0/846850085 >> v2:172.21.15.122:3300/0 conn(0x7f43d4072260 msgr2=0x7f43d415bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d27fc640 1 --2- 172.21.15.100:0/846850085 >> v2:172.21.15.122:3300/0 conn(0x7f43d4072260 0x7f43d415bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d27fc640 1 -- 172.21.15.100:0/846850085 >> v1:172.21.15.122:6789/0 conn(0x7f43d4151fd0 legacy=0x7f43d40cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d27fc640 1 -- 172.21.15.100:0/846850085 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d4158310 con 0x7f43d40c86f0 2024-09-15T21:46:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d27fc640 1 -- 172.21.15.100:0/846850085 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7f43bc001670 con 0x7f43d40c86f0 2024-09-15T21:46:37.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d27fc640 1 -- 172.21.15.100:0/846850085 >> v1:172.21.15.100:6789/0 conn(0x7f43d40c86f0 legacy=0x7f43d4071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:37.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d27fc640 1 --2- 172.21.15.100:0/846850085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43c00039b0 0x7f43c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:37.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d27fc640 1 --2- 172.21.15.100:0/846850085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c00043b0 0x7f43c0004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:37.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d27fc640 1 --2- 172.21.15.100:0/846850085 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f43c0005080 0x7f43c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:37.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d27fc640 1 -- 172.21.15.100:0/846850085 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f43d4158030 con 0x7f43c00039b0 2024-09-15T21:46:37.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d27fc640 1 -- 172.21.15.100:0/846850085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43c000bfe0 con 0x7f43c00043b0 2024-09-15T21:46:37.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d27fc640 1 -- 172.21.15.100:0/846850085 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f43c000c1e0 con 0x7f43c0005080 2024-09-15T21:46:37.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d37fe640 1 --2- 172.21.15.100:0/846850085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43c00039b0 0x7f43c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:37.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d2ffd640 1 --2- 172.21.15.100:0/846850085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c00043b0 0x7f43c0004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:37.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d3fff640 1 --2- 172.21.15.100:0/846850085 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f43c0005080 0x7f43c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:37.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d37fe640 1 -- 172.21.15.100:0/846850085 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f43c0005080 msgr2=0x7f43c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:37.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d37fe640 1 --2- 172.21.15.100:0/846850085 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f43c0005080 0x7f43c000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d37fe640 1 -- 172.21.15.100:0/846850085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c00043b0 msgr2=0x7f43c0004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:37.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d37fe640 1 --2- 172.21.15.100:0/846850085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c00043b0 0x7f43c0004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d37fe640 1 -- 172.21.15.100:0/846850085 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c0003520 con 0x7f43c00039b0 2024-09-15T21:46:37.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d3fff640 1 -- 172.21.15.100:0/846850085 reap_dead start 2024-09-15T21:46:37.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.147+0000 7f43d2ffd640 1 --2- 172.21.15.100:0/846850085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43c00043b0 0x7f43c0004880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:37.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.148+0000 7f43d37fe640 1 --2- 172.21.15.100:0/846850085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43c00039b0 0x7f43c0003d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f43c4001f60 tx=0x7f43c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a63df79dc3ab60c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:37.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.148+0000 7f43d27fc640 1 -- 172.21.15.100:0/846850085 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4001b30 con 0x7f43c00039b0 2024-09-15T21:46:37.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.148+0000 7f43d27fc640 1 -- 172.21.15.100:0/846850085 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c4031620 con 0x7f43c00039b0 2024-09-15T21:46:37.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.148+0000 7f43d27fc640 1 -- 172.21.15.100:0/846850085 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4031a10 con 0x7f43c00039b0 2024-09-15T21:46:37.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.149+0000 7f43d9ad0640 1 -- 172.21.15.100:0/846850085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43c00039b0 msgr2=0x7f43c0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:37.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.149+0000 7f43d9ad0640 1 --2- 172.21.15.100:0/846850085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43c00039b0 0x7f43c0003d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f43c4001f60 tx=0x7f43c0005830 comp rx=0 tx=0).stop 2024-09-15T21:46:37.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.149+0000 7f43d9ad0640 1 -- 172.21.15.100:0/846850085 shutdown_connections 2024-09-15T21:46:37.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.149+0000 7f43d9ad0640 1 --2- 172.21.15.100:0/846850085 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43c00039b0 0x7f43c0003d90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.149+0000 7f43d9ad0640 1 -- 172.21.15.100:0/846850085 >> 172.21.15.100:0/846850085 conn(0x7f43d4067dd0 msgr2=0x7f43d406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:37.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.149+0000 7f43d9ad0640 1 -- 172.21.15.100:0/846850085 shutdown_connections 2024-09-15T21:46:37.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.149+0000 7f43d9ad0640 1 -- 172.21.15.100:0/846850085 wait complete. 2024-09-15T21:46:37.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.150+0000 7f43d9ad0640 1 Processor -- start 2024-09-15T21:46:37.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.150+0000 7f43d9ad0640 1 -- start start 2024-09-15T21:46:37.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43d9ad0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43c00039b0 0x7f43d4162fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:37.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43d9ad0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43d41552f0 0x7f43d4161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:37.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43d9ad0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f43d41623f0 0x7f43d40cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:37.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43d9ad0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f43d41523b0 con 0x7f43c00039b0 2024-09-15T21:46:37.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43d9ad0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f43d41588d0 con 0x7f43d41552f0 2024-09-15T21:46:37.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43d9ad0640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f43d4152530 con 0x7f43d41623f0 2024-09-15T21:46:37.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43d37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43c00039b0 0x7f43d4162fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:37.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43d37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43c00039b0 0x7f43d4162fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45450/0 (socket says 172.21.15.100:45450) 2024-09-15T21:46:37.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43d37fe640 1 -- 172.21.15.100:0/3803758452 learned_addr learned my addr 172.21.15.100:0/3803758452 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:37.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43d2ffd640 1 --2- 172.21.15.100:0/3803758452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43d41552f0 0x7f43d4161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:37.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43d3fff640 1 --2- 172.21.15.100:0/3803758452 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f43d41623f0 0x7f43d40cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:37.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43d37fe640 1 -- 172.21.15.100:0/3803758452 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f43d41623f0 msgr2=0x7f43d40cabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:37.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43d37fe640 1 --2- 172.21.15.100:0/3803758452 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f43d41623f0 0x7f43d40cabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43d37fe640 1 -- 172.21.15.100:0/3803758452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43d41552f0 msgr2=0x7f43d4161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:37.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43d37fe640 1 --2- 172.21.15.100:0/3803758452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43d41552f0 0x7f43d4161eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43d37fe640 1 -- 172.21.15.100:0/3803758452 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d41f3bd0 con 0x7f43c00039b0 2024-09-15T21:46:37.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43d37fe640 1 --2- 172.21.15.100:0/3803758452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43c00039b0 0x7f43d4162fa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f43c4002080 tx=0x7f43c40015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:37.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43cbfff640 1 -- 172.21.15.100:0/3803758452 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c403f070 con 0x7f43c00039b0 2024-09-15T21:46:37.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43cbfff640 1 -- 172.21.15.100:0/3803758452 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c4006b50 con 0x7f43c00039b0 2024-09-15T21:46:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43cbfff640 1 -- 172.21.15.100:0/3803758452 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c4003970 con 0x7f43c00039b0 2024-09-15T21:46:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43d9ad0640 1 -- 172.21.15.100:0/3803758452 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d41f3eb0 con 0x7f43c00039b0 2024-09-15T21:46:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.151+0000 7f43d9ad0640 1 -- 172.21.15.100:0/3803758452 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d41f4750 con 0x7f43c00039b0 2024-09-15T21:46:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.153+0000 7f43d9ad0640 1 -- 172.21.15.100:0/3803758452 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d406c920 con 0x7f43c00039b0 2024-09-15T21:46:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.153+0000 7f43cbfff640 1 -- 172.21.15.100:0/3803758452 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f43c4042070 con 0x7f43c00039b0 2024-09-15T21:46:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.153+0000 7f43cbfff640 1 --2- 172.21.15.100:0/3803758452 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f43b406c2b0 0x7f43b406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.153+0000 7f43cbfff640 1 -- 172.21.15.100:0/3803758452 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f43c40abbe0 con 0x7f43c00039b0 2024-09-15T21:46:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.153+0000 7f43d2ffd640 1 --2- 172.21.15.100:0/3803758452 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f43b406c2b0 0x7f43b406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.154+0000 7f43d2ffd640 1 --2- 172.21.15.100:0/3803758452 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f43b406c2b0 0x7f43b406e770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f43d40cb740 tx=0x7f43bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:37.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.157+0000 7f43cbfff640 1 -- 172.21.15.100:0/3803758452 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f43c407a3e0 con 0x7f43c00039b0 2024-09-15T21:46:37.322 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.320+0000 7f43d9ad0640 1 -- 172.21.15.100:0/3803758452 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f43d41634e0 con 0x7f43b406c2b0 2024-09-15T21:46:37.327 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.326+0000 7f43cbfff640 1 -- 172.21.15.100:0/3803758452 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251862 (secure 0 0 0) 0x7f43d41634e0 con 0x7f43b406c2b0 2024-09-15T21:46:37.330 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:37.331 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-15T21:46:37.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.329+0000 7f43d9ad0640 1 -- 172.21.15.100:0/3803758452 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f43b406c2b0 msgr2=0x7f43b406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:37.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.329+0000 7f43d9ad0640 1 --2- 172.21.15.100:0/3803758452 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f43b406c2b0 0x7f43b406e770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f43d40cb740 tx=0x7f43bc009040 comp rx=0 tx=0).stop 2024-09-15T21:46:37.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.329+0000 7f43d9ad0640 1 -- 172.21.15.100:0/3803758452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43c00039b0 msgr2=0x7f43d4162fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:37.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.329+0000 7f43d9ad0640 1 --2- 172.21.15.100:0/3803758452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43c00039b0 0x7f43d4162fa0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f43c4002080 tx=0x7f43c40015b0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.329+0000 7f43d9ad0640 1 -- 172.21.15.100:0/3803758452 shutdown_connections 2024-09-15T21:46:37.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.329+0000 7f43d9ad0640 1 --2- 172.21.15.100:0/3803758452 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f43b406c2b0 0x7f43b406e770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.329+0000 7f43d9ad0640 1 --2- 172.21.15.100:0/3803758452 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f43d41552f0 0x7f43d4161eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.329+0000 7f43d9ad0640 1 --2- 172.21.15.100:0/3803758452 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f43d41623f0 0x7f43d40cabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.329+0000 7f43d9ad0640 1 --2- 172.21.15.100:0/3803758452 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f43c00039b0 0x7f43d4162fa0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.329+0000 7f43d9ad0640 1 -- 172.21.15.100:0/3803758452 >> 172.21.15.100:0/3803758452 conn(0x7f43d4067dd0 msgr2=0x7f43d40c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:37.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.329+0000 7f43d9ad0640 1 -- 172.21.15.100:0/3803758452 shutdown_connections 2024-09-15T21:46:37.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.329+0000 7f43d9ad0640 1 -- 172.21.15.100:0/3803758452 wait complete. 2024-09-15T21:46:37.341 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-15T21:46:35.806088+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":593400,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":205,"num_write_kb":2432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219152,"kb_used_data":2960,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755568,"statfs":{"total":773094113280,"available":772869701632,"internally_reserved":0,"allocated":3031040,"data_stored":1801842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3032,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.394192"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835668+0000","last_change":"2024-09-15T21:46:23.428662+0000","last_active":"2024-09-15T21:46:24.835668+0000","last_peered":"2024-09-15T21:46:24.835668+0000","last_clean":"2024-09-15T21:46:24.835668+0000","last_became_active":"2024-09-15T21:46:23.427735+0000","last_became_peered":"2024-09-15T21:46:23.427735+0000","last_unstale":"2024-09-15T21:46:24.835668+0000","last_undegraded":"2024-09-15T21:46:24.835668+0000","last_fullsized":"2024-09-15T21:46:24.835668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:29.702757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808513+0000","last_change":"2024-09-15T21:46:23.418716+0000","last_active":"2024-09-15T21:46:24.808513+0000","last_peered":"2024-09-15T21:46:24.808513+0000","last_clean":"2024-09-15T21:46:24.808513+0000","last_became_active":"2024-09-15T21:46:23.418471+0000","last_became_peered":"2024-09-15T21:46:23.418471+0000","last_unstale":"2024-09-15T21:46:24.808513+0000","last_undegraded":"2024-09-15T21:46:24.808513+0000","last_fullsized":"2024-09-15T21:46:24.808513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:49.599801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727016+0000","last_change":"2024-09-15T21:46:23.426404+0000","last_active":"2024-09-15T21:46:24.727016+0000","last_peered":"2024-09-15T21:46:24.727016+0000","last_clean":"2024-09-15T21:46:24.727016+0000","last_became_active":"2024-09-15T21:46:23.426146+0000","last_became_peered":"2024-09-15T21:46:23.426146+0000","last_unstale":"2024-09-15T21:46:24.727016+0000","last_undegraded":"2024-09-15T21:46:24.727016+0000","last_fullsized":"2024-09-15T21:46:24.727016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:23.626109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414109+0000","last_change":"2024-09-15T21:46:23.432515+0000","last_active":"2024-09-15T21:46:24.414109+0000","last_peered":"2024-09-15T21:46:24.414109+0000","last_clean":"2024-09-15T21:46:24.414109+0000","last_became_active":"2024-09-15T21:46:23.432283+0000","last_became_peered":"2024-09-15T21:46:23.432283+0000","last_unstale":"2024-09-15T21:46:24.414109+0000","last_undegraded":"2024-09-15T21:46:24.414109+0000","last_fullsized":"2024-09-15T21:46:24.414109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:53.288080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835464+0000","last_change":"2024-09-15T21:46:23.427120+0000","last_active":"2024-09-15T21:46:24.835464+0000","last_peered":"2024-09-15T21:46:24.835464+0000","last_clean":"2024-09-15T21:46:24.835464+0000","last_became_active":"2024-09-15T21:46:23.426706+0000","last_became_peered":"2024-09-15T21:46:23.426706+0000","last_unstale":"2024-09-15T21:46:24.835464+0000","last_undegraded":"2024-09-15T21:46:24.835464+0000","last_fullsized":"2024-09-15T21:46:24.835464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:29.513446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414665+0000","last_change":"2024-09-15T21:46:23.434690+0000","last_active":"2024-09-15T21:46:24.414665+0000","last_peered":"2024-09-15T21:46:24.414665+0000","last_clean":"2024-09-15T21:46:24.414665+0000","last_became_active":"2024-09-15T21:46:23.434146+0000","last_became_peered":"2024-09-15T21:46:23.434146+0000","last_unstale":"2024-09-15T21:46:24.414665+0000","last_undegraded":"2024-09-15T21:46:24.414665+0000","last_fullsized":"2024-09-15T21:46:24.414665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:26:59.905178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835273+0000","last_change":"2024-09-15T21:46:23.429541+0000","last_active":"2024-09-15T21:46:24.835273+0000","last_peered":"2024-09-15T21:46:24.835273+0000","last_clean":"2024-09-15T21:46:24.835273+0000","last_became_active":"2024-09-15T21:46:23.429263+0000","last_became_peered":"2024-09-15T21:46:23.429263+0000","last_unstale":"2024-09-15T21:46:24.835273+0000","last_undegraded":"2024-09-15T21:46:24.835273+0000","last_fullsized":"2024-09-15T21:46:24.835273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:34:51.530981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834382+0000","last_change":"2024-09-15T21:46:23.437143+0000","last_active":"2024-09-15T21:46:24.834382+0000","last_peered":"2024-09-15T21:46:24.834382+0000","last_clean":"2024-09-15T21:46:24.834382+0000","last_became_active":"2024-09-15T21:46:23.436864+0000","last_became_peered":"2024-09-15T21:46:23.436864+0000","last_unstale":"2024-09-15T21:46:24.834382+0000","last_undegraded":"2024-09-15T21:46:24.834382+0000","last_fullsized":"2024-09-15T21:46:24.834382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:45.870403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835817+0000","last_change":"2024-09-15T21:46:23.438642+0000","last_active":"2024-09-15T21:46:24.835817+0000","last_peered":"2024-09-15T21:46:24.835817+0000","last_clean":"2024-09-15T21:46:24.835817+0000","last_became_active":"2024-09-15T21:46:23.437521+0000","last_became_peered":"2024-09-15T21:46:23.437521+0000","last_unstale":"2024-09-15T21:46:24.835817+0000","last_undegraded":"2024-09-15T21:46:24.835817+0000","last_fullsized":"2024-09-15T21:46:24.835817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:04.648853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835520+0000","last_change":"2024-09-15T21:46:23.436106+0000","last_active":"2024-09-15T21:46:24.835520+0000","last_peered":"2024-09-15T21:46:24.835520+0000","last_clean":"2024-09-15T21:46:24.835520+0000","last_became_active":"2024-09-15T21:46:23.435821+0000","last_became_peered":"2024-09-15T21:46:23.435821+0000","last_unstale":"2024-09-15T21:46:24.835520+0000","last_undegraded":"2024-09-15T21:46:24.835520+0000","last_fullsized":"2024-09-15T21:46:24.835520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835070+0000","last_change":"2024-09-15T21:46:23.435442+0000","last_active":"2024-09-15T21:46:24.835070+0000","last_peered":"2024-09-15T21:46:24.835070+0000","last_clean":"2024-09-15T21:46:24.835070+0000","last_became_active":"2024-09-15T21:46:23.435165+0000","last_became_peered":"2024-09-15T21:46:23.435165+0000","last_unstale":"2024-09-15T21:46:24.835070+0000","last_undegraded":"2024-09-15T21:46:24.835070+0000","last_fullsized":"2024-09-15T21:46:24.835070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:02.361682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414099+0000","last_change":"2024-09-15T21:46:23.432534+0000","last_active":"2024-09-15T21:46:24.414099+0000","last_peered":"2024-09-15T21:46:24.414099+0000","last_clean":"2024-09-15T21:46:24.414099+0000","last_became_active":"2024-09-15T21:46:23.430170+0000","last_became_peered":"2024-09-15T21:46:23.430170+0000","last_unstale":"2024-09-15T21:46:24.414099+0000","last_undegraded":"2024-09-15T21:46:24.414099+0000","last_fullsized":"2024-09-15T21:46:24.414099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:11.463937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834691+0000","last_change":"2024-09-15T21:46:23.427841+0000","last_active":"2024-09-15T21:46:24.834691+0000","last_peered":"2024-09-15T21:46:24.834691+0000","last_clean":"2024-09-15T21:46:24.834691+0000","last_became_active":"2024-09-15T21:46:23.427621+0000","last_became_peered":"2024-09-15T21:46:23.427621+0000","last_unstale":"2024-09-15T21:46:24.834691+0000","last_undegraded":"2024-09-15T21:46:24.834691+0000","last_fullsized":"2024-09-15T21:46:24.834691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:18.331943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.726852+0000","last_change":"2024-09-15T21:46:23.424707+0000","last_active":"2024-09-15T21:46:24.726852+0000","last_peered":"2024-09-15T21:46:24.726852+0000","last_clean":"2024-09-15T21:46:24.726852+0000","last_became_active":"2024-09-15T21:46:23.424447+0000","last_became_peered":"2024-09-15T21:46:23.424447+0000","last_unstale":"2024-09-15T21:46:24.726852+0000","last_undegraded":"2024-09-15T21:46:24.726852+0000","last_fullsized":"2024-09-15T21:46:24.726852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:23.617214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808321+0000","last_change":"2024-09-15T21:46:23.430297+0000","last_active":"2024-09-15T21:46:24.808321+0000","last_peered":"2024-09-15T21:46:24.808321+0000","last_clean":"2024-09-15T21:46:24.808321+0000","last_became_active":"2024-09-15T21:46:23.430070+0000","last_became_peered":"2024-09-15T21:46:23.430070+0000","last_unstale":"2024-09-15T21:46:24.808321+0000","last_undegraded":"2024-09-15T21:46:24.808321+0000","last_fullsized":"2024-09-15T21:46:24.808321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:03:19.615022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414542+0000","last_change":"2024-09-15T21:46:23.430540+0000","last_active":"2024-09-15T21:46:24.414542+0000","last_peered":"2024-09-15T21:46:24.414542+0000","last_clean":"2024-09-15T21:46:24.414542+0000","last_became_active":"2024-09-15T21:46:23.430256+0000","last_became_peered":"2024-09-15T21:46:23.430256+0000","last_unstale":"2024-09-15T21:46:24.414542+0000","last_undegraded":"2024-09-15T21:46:24.414542+0000","last_fullsized":"2024-09-15T21:46:24.414542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:36.539762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.415216+0000","last_change":"2024-09-15T21:46:23.427802+0000","last_active":"2024-09-15T21:46:24.415216+0000","last_peered":"2024-09-15T21:46:24.415216+0000","last_clean":"2024-09-15T21:46:24.415216+0000","last_became_active":"2024-09-15T21:46:23.427502+0000","last_became_peered":"2024-09-15T21:46:23.427502+0000","last_unstale":"2024-09-15T21:46:24.415216+0000","last_undegraded":"2024-09-15T21:46:24.415216+0000","last_fullsized":"2024-09-15T21:46:24.415216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:31.062624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412600+0000","last_change":"2024-09-15T21:46:23.427570+0000","last_active":"2024-09-15T21:46:24.412600+0000","last_peered":"2024-09-15T21:46:24.412600+0000","last_clean":"2024-09-15T21:46:24.412600+0000","last_became_active":"2024-09-15T21:46:23.427304+0000","last_became_peered":"2024-09-15T21:46:23.427304+0000","last_unstale":"2024-09-15T21:46:24.412600+0000","last_undegraded":"2024-09-15T21:46:24.412600+0000","last_fullsized":"2024-09-15T21:46:24.412600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:37.345106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835899+0000","last_change":"2024-09-15T21:46:23.431434+0000","last_active":"2024-09-15T21:46:24.835899+0000","last_peered":"2024-09-15T21:46:24.835899+0000","last_clean":"2024-09-15T21:46:24.835899+0000","last_became_active":"2024-09-15T21:46:23.430987+0000","last_became_peered":"2024-09-15T21:46:23.430987+0000","last_unstale":"2024-09-15T21:46:24.835899+0000","last_undegraded":"2024-09-15T21:46:24.835899+0000","last_fullsized":"2024-09-15T21:46:24.835899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:48:30.532378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835198+0000","last_change":"2024-09-15T21:46:23.434558+0000","last_active":"2024-09-15T21:46:24.835198+0000","last_peered":"2024-09-15T21:46:24.835198+0000","last_clean":"2024-09-15T21:46:24.835198+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:46:24.835198+0000","last_undegraded":"2024-09-15T21:46:24.835198+0000","last_fullsized":"2024-09-15T21:46:24.835198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:03:39.562977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727140+0000","last_change":"2024-09-15T21:46:23.428323+0000","last_active":"2024-09-15T21:46:24.727140+0000","last_peered":"2024-09-15T21:46:24.727140+0000","last_clean":"2024-09-15T21:46:24.727140+0000","last_became_active":"2024-09-15T21:46:23.428016+0000","last_became_peered":"2024-09-15T21:46:23.428016+0000","last_unstale":"2024-09-15T21:46:24.727140+0000","last_undegraded":"2024-09-15T21:46:24.727140+0000","last_fullsized":"2024-09-15T21:46:24.727140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:28:48.678550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414612+0000","last_change":"2024-09-15T21:46:23.434465+0000","last_active":"2024-09-15T21:46:24.414612+0000","last_peered":"2024-09-15T21:46:24.414612+0000","last_clean":"2024-09-15T21:46:24.414612+0000","last_became_active":"2024-09-15T21:46:23.433849+0000","last_became_peered":"2024-09-15T21:46:23.433849+0000","last_unstale":"2024-09-15T21:46:24.414612+0000","last_undegraded":"2024-09-15T21:46:24.414612+0000","last_fullsized":"2024-09-15T21:46:24.414612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:46.093198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834522+0000","last_change":"2024-09-15T21:46:23.431527+0000","last_active":"2024-09-15T21:46:24.834522+0000","last_peered":"2024-09-15T21:46:24.834522+0000","last_clean":"2024-09-15T21:46:24.834522+0000","last_became_active":"2024-09-15T21:46:23.430719+0000","last_became_peered":"2024-09-15T21:46:23.430719+0000","last_unstale":"2024-09-15T21:46:24.834522+0000","last_undegraded":"2024-09-15T21:46:24.834522+0000","last_fullsized":"2024-09-15T21:46:24.834522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:16.326529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414532+0000","last_change":"2024-09-15T21:46:23.432282+0000","last_active":"2024-09-15T21:46:24.414532+0000","last_peered":"2024-09-15T21:46:24.414532+0000","last_clean":"2024-09-15T21:46:24.414532+0000","last_became_active":"2024-09-15T21:46:23.432035+0000","last_became_peered":"2024-09-15T21:46:23.432035+0000","last_unstale":"2024-09-15T21:46:24.414532+0000","last_undegraded":"2024-09-15T21:46:24.414532+0000","last_fullsized":"2024-09-15T21:46:24.414532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:22.163940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834776+0000","last_change":"2024-09-15T21:46:23.432646+0000","last_active":"2024-09-15T21:46:24.834776+0000","last_peered":"2024-09-15T21:46:24.834776+0000","last_clean":"2024-09-15T21:46:24.834776+0000","last_became_active":"2024-09-15T21:46:23.432317+0000","last_became_peered":"2024-09-15T21:46:23.432317+0000","last_unstale":"2024-09-15T21:46:24.834776+0000","last_undegraded":"2024-09-15T21:46:24.834776+0000","last_fullsized":"2024-09-15T21:46:24.834776+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:49:07.706450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835541+0000","last_change":"2024-09-15T21:46:23.428183+0000","last_active":"2024-09-15T21:46:24.835541+0000","last_peered":"2024-09-15T21:46:24.835541+0000","last_clean":"2024-09-15T21:46:24.835541+0000","last_became_active":"2024-09-15T21:46:23.427886+0000","last_became_peered":"2024-09-15T21:46:23.427886+0000","last_unstale":"2024-09-15T21:46:24.835541+0000","last_undegraded":"2024-09-15T21:46:24.835541+0000","last_fullsized":"2024-09-15T21:46:24.835541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:15.584377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808369+0000","last_change":"2024-09-15T21:46:23.421749+0000","last_active":"2024-09-15T21:46:24.808369+0000","last_peered":"2024-09-15T21:46:24.808369+0000","last_clean":"2024-09-15T21:46:24.808369+0000","last_became_active":"2024-09-15T21:46:23.421501+0000","last_became_peered":"2024-09-15T21:46:23.421501+0000","last_unstale":"2024-09-15T21:46:24.808369+0000","last_undegraded":"2024-09-15T21:46:24.808369+0000","last_fullsized":"2024-09-15T21:46:24.808369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:55.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834843+0000","last_change":"2024-09-15T21:46:23.432591+0000","last_active":"2024-09-15T21:46:24.834843+0000","last_peered":"2024-09-15T21:46:24.834843+0000","last_clean":"2024-09-15T21:46:24.834843+0000","last_became_active":"2024-09-15T21:46:23.432326+0000","last_became_peered":"2024-09-15T21:46:23.432326+0000","last_unstale":"2024-09-15T21:46:24.834843+0000","last_undegraded":"2024-09-15T21:46:24.834843+0000","last_fullsized":"2024-09-15T21:46:24.834843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:48.565579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727016+0000","last_change":"2024-09-15T21:46:23.423428+0000","last_active":"2024-09-15T21:46:24.727016+0000","last_peered":"2024-09-15T21:46:24.727016+0000","last_clean":"2024-09-15T21:46:24.727016+0000","last_became_active":"2024-09-15T21:46:23.422734+0000","last_became_peered":"2024-09-15T21:46:23.422734+0000","last_unstale":"2024-09-15T21:46:24.727016+0000","last_undegraded":"2024-09-15T21:46:24.727016+0000","last_fullsized":"2024-09-15T21:46:24.727016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:06.592891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414770+0000","last_change":"2024-09-15T21:46:23.435244+0000","last_active":"2024-09-15T21:46:24.414770+0000","last_peered":"2024-09-15T21:46:24.414770+0000","last_clean":"2024-09-15T21:46:24.414770+0000","last_became_active":"2024-09-15T21:46:23.434932+0000","last_became_peered":"2024-09-15T21:46:23.434932+0000","last_unstale":"2024-09-15T21:46:24.414770+0000","last_undegraded":"2024-09-15T21:46:24.414770+0000","last_fullsized":"2024-09-15T21:46:24.414770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:55.810146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412500+0000","last_change":"2024-09-15T21:46:23.437276+0000","last_active":"2024-09-15T21:46:24.412500+0000","last_peered":"2024-09-15T21:46:24.412500+0000","last_clean":"2024-09-15T21:46:24.412500+0000","last_became_active":"2024-09-15T21:46:23.436962+0000","last_became_peered":"2024-09-15T21:46:23.436962+0000","last_unstale":"2024-09-15T21:46:24.412500+0000","last_undegraded":"2024-09-15T21:46:24.412500+0000","last_fullsized":"2024-09-15T21:46:24.412500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:13.043999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834825+0000","last_change":"2024-09-15T21:46:23.429648+0000","last_active":"2024-09-15T21:46:24.834825+0000","last_peered":"2024-09-15T21:46:24.834825+0000","last_clean":"2024-09-15T21:46:24.834825+0000","last_became_active":"2024-09-15T21:46:23.429376+0000","last_became_peered":"2024-09-15T21:46:23.429376+0000","last_unstale":"2024-09-15T21:46:24.834825+0000","last_undegraded":"2024-09-15T21:46:24.834825+0000","last_fullsized":"2024-09-15T21:46:24.834825+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:31.278283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.413663+0000","last_change":"2024-09-15T21:46:23.429847+0000","last_active":"2024-09-15T21:46:24.413663+0000","last_peered":"2024-09-15T21:46:24.413663+0000","last_clean":"2024-09-15T21:46:24.413663+0000","last_became_active":"2024-09-15T21:46:23.429442+0000","last_became_peered":"2024-09-15T21:46:23.429442+0000","last_unstale":"2024-09-15T21:46:24.413663+0000","last_undegraded":"2024-09-15T21:46:24.413663+0000","last_fullsized":"2024-09-15T21:46:24.413663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:31.232512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:34.464539+0000","last_change":"2024-09-15T21:46:22.429407+0000","last_active":"2024-09-15T21:46:34.464539+0000","last_peered":"2024-09-15T21:46:34.464539+0000","last_clean":"2024-09-15T21:46:34.464539+0000","last_became_active":"2024-09-15T21:46:22.429107+0000","last_became_peered":"2024-09-15T21:46:22.429107+0000","last_unstale":"2024-09-15T21:46:34.464539+0000","last_undegraded":"2024-09-15T21:46:34.464539+0000","last_fullsized":"2024-09-15T21:46:34.464539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:14:54.680349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727169+0000","last_change":"2024-09-15T21:46:23.434198+0000","last_active":"2024-09-15T21:46:24.727169+0000","last_peered":"2024-09-15T21:46:24.727169+0000","last_clean":"2024-09-15T21:46:24.727169+0000","last_became_active":"2024-09-15T21:46:23.433944+0000","last_became_peered":"2024-09-15T21:46:23.433944+0000","last_unstale":"2024-09-15T21:46:24.727169+0000","last_undegraded":"2024-09-15T21:46:24.727169+0000","last_fullsized":"2024-09-15T21:46:24.727169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:25.412167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834914+0000","last_change":"2024-09-15T21:46:22.425571+0000","last_active":"2024-09-15T21:46:24.834914+0000","last_peered":"2024-09-15T21:46:24.834914+0000","last_clean":"2024-09-15T21:46:24.834914+0000","last_became_active":"2024-09-15T21:46:22.425315+0000","last_became_peered":"2024-09-15T21:46:22.425315+0000","last_unstale":"2024-09-15T21:46:24.834914+0000","last_undegraded":"2024-09-15T21:46:24.834914+0000","last_fullsized":"2024-09-15T21:46:24.834914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:31.983301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834481+0000","last_change":"2024-09-15T21:46:22.425492+0000","last_active":"2024-09-15T21:46:24.834481+0000","last_peered":"2024-09-15T21:46:24.834481+0000","last_clean":"2024-09-15T21:46:24.834481+0000","last_became_active":"2024-09-15T21:46:22.425226+0000","last_became_peered":"2024-09-15T21:46:22.425226+0000","last_unstale":"2024-09-15T21:46:24.834481+0000","last_undegraded":"2024-09-15T21:46:24.834481+0000","last_fullsized":"2024-09-15T21:46:24.834481+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:24.322384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834460+0000","last_change":"2024-09-15T21:46:23.429140+0000","last_active":"2024-09-15T21:46:24.834460+0000","last_peered":"2024-09-15T21:46:24.834460+0000","last_clean":"2024-09-15T21:46:24.834460+0000","last_became_active":"2024-09-15T21:46:23.428902+0000","last_became_peered":"2024-09-15T21:46:23.428902+0000","last_unstale":"2024-09-15T21:46:24.834460+0000","last_undegraded":"2024-09-15T21:46:24.834460+0000","last_fullsized":"2024-09-15T21:46:24.834460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:11.256924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414663+0000","last_change":"2024-09-15T21:46:22.428319+0000","last_active":"2024-09-15T21:46:24.414663+0000","last_peered":"2024-09-15T21:46:24.414663+0000","last_clean":"2024-09-15T21:46:24.414663+0000","last_became_active":"2024-09-15T21:46:22.428061+0000","last_became_peered":"2024-09-15T21:46:22.428061+0000","last_unstale":"2024-09-15T21:46:24.414663+0000","last_undegraded":"2024-09-15T21:46:24.414663+0000","last_fullsized":"2024-09-15T21:46:24.414663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:14.238988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808315+0000","last_change":"2024-09-15T21:46:23.427442+0000","last_active":"2024-09-15T21:46:24.808315+0000","last_peered":"2024-09-15T21:46:24.808315+0000","last_clean":"2024-09-15T21:46:24.808315+0000","last_became_active":"2024-09-15T21:46:23.427200+0000","last_became_peered":"2024-09-15T21:46:23.427200+0000","last_unstale":"2024-09-15T21:46:24.808315+0000","last_undegraded":"2024-09-15T21:46:24.808315+0000","last_fullsized":"2024-09-15T21:46:24.808315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:27.248545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835085+0000","last_change":"2024-09-15T21:46:22.430599+0000","last_active":"2024-09-15T21:46:24.835085+0000","last_peered":"2024-09-15T21:46:24.835085+0000","last_clean":"2024-09-15T21:46:24.835085+0000","last_became_active":"2024-09-15T21:46:22.430327+0000","last_became_peered":"2024-09-15T21:46:22.430327+0000","last_unstale":"2024-09-15T21:46:24.835085+0000","last_undegraded":"2024-09-15T21:46:24.835085+0000","last_fullsized":"2024-09-15T21:46:24.835085+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:51.904693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414860+0000","last_change":"2024-09-15T21:46:23.434006+0000","last_active":"2024-09-15T21:46:24.414860+0000","last_peered":"2024-09-15T21:46:24.414860+0000","last_clean":"2024-09-15T21:46:24.414860+0000","last_became_active":"2024-09-15T21:46:23.433634+0000","last_became_peered":"2024-09-15T21:46:23.433634+0000","last_unstale":"2024-09-15T21:46:24.414860+0000","last_undegraded":"2024-09-15T21:46:24.414860+0000","last_fullsized":"2024-09-15T21:46:24.414860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:22.288806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414976+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:24.414976+0000","last_peered":"2024-09-15T21:46:24.414976+0000","last_clean":"2024-09-15T21:46:24.414976+0000","last_became_active":"2024-09-15T21:46:22.425236+0000","last_became_peered":"2024-09-15T21:46:22.425236+0000","last_unstale":"2024-09-15T21:46:24.414976+0000","last_undegraded":"2024-09-15T21:46:24.414976+0000","last_fullsized":"2024-09-15T21:46:24.414976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:45:21.332492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727237+0000","last_change":"2024-09-15T21:46:23.429587+0000","last_active":"2024-09-15T21:46:24.727237+0000","last_peered":"2024-09-15T21:46:24.727237+0000","last_clean":"2024-09-15T21:46:24.727237+0000","last_became_active":"2024-09-15T21:46:23.429334+0000","last_became_peered":"2024-09-15T21:46:23.429334+0000","last_unstale":"2024-09-15T21:46:24.727237+0000","last_undegraded":"2024-09-15T21:46:24.727237+0000","last_fullsized":"2024-09-15T21:46:24.727237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:48:56.361550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.417234+0000","last_change":"2024-09-15T21:46:22.421964+0000","last_active":"2024-09-15T21:46:28.417234+0000","last_peered":"2024-09-15T21:46:28.417234+0000","last_clean":"2024-09-15T21:46:28.417234+0000","last_became_active":"2024-09-15T21:46:22.421661+0000","last_became_peered":"2024-09-15T21:46:22.421661+0000","last_unstale":"2024-09-15T21:46:28.417234+0000","last_undegraded":"2024-09-15T21:46:28.417234+0000","last_fullsized":"2024-09-15T21:46:28.417234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:07.395070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414253+0000","last_change":"2024-09-15T21:46:23.427416+0000","last_active":"2024-09-15T21:46:24.414253+0000","last_peered":"2024-09-15T21:46:24.414253+0000","last_clean":"2024-09-15T21:46:24.414253+0000","last_became_active":"2024-09-15T21:46:23.427057+0000","last_became_peered":"2024-09-15T21:46:23.427057+0000","last_unstale":"2024-09-15T21:46:24.414253+0000","last_undegraded":"2024-09-15T21:46:24.414253+0000","last_fullsized":"2024-09-15T21:46:24.414253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:42.671501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.422249+0000","last_change":"2024-09-15T21:46:22.426151+0000","last_active":"2024-09-15T21:46:28.422249+0000","last_peered":"2024-09-15T21:46:28.422249+0000","last_clean":"2024-09-15T21:46:28.422249+0000","last_became_active":"2024-09-15T21:46:22.425866+0000","last_became_peered":"2024-09-15T21:46:22.425866+0000","last_unstale":"2024-09-15T21:46:28.422249+0000","last_undegraded":"2024-09-15T21:46:28.422249+0000","last_fullsized":"2024-09-15T21:46:28.422249+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:10.684265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835569+0000","last_change":"2024-09-15T21:46:23.432292+0000","last_active":"2024-09-15T21:46:24.835569+0000","last_peered":"2024-09-15T21:46:24.835569+0000","last_clean":"2024-09-15T21:46:24.835569+0000","last_became_active":"2024-09-15T21:46:23.431877+0000","last_became_peered":"2024-09-15T21:46:23.431877+0000","last_unstale":"2024-09-15T21:46:24.835569+0000","last_undegraded":"2024-09-15T21:46:24.835569+0000","last_fullsized":"2024-09-15T21:46:24.835569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:16.857279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.417259+0000","last_change":"2024-09-15T21:46:22.427415+0000","last_active":"2024-09-15T21:46:28.417259+0000","last_peered":"2024-09-15T21:46:28.417259+0000","last_clean":"2024-09-15T21:46:28.417259+0000","last_became_active":"2024-09-15T21:46:22.427087+0000","last_became_peered":"2024-09-15T21:46:22.427087+0000","last_unstale":"2024-09-15T21:46:28.417259+0000","last_undegraded":"2024-09-15T21:46:28.417259+0000","last_fullsized":"2024-09-15T21:46:28.417259+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:09.365202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.726943+0000","last_change":"2024-09-15T21:46:23.427238+0000","last_active":"2024-09-15T21:46:24.726943+0000","last_peered":"2024-09-15T21:46:24.726943+0000","last_clean":"2024-09-15T21:46:24.726943+0000","last_became_active":"2024-09-15T21:46:23.426993+0000","last_became_peered":"2024-09-15T21:46:23.426993+0000","last_unstale":"2024-09-15T21:46:24.726943+0000","last_undegraded":"2024-09-15T21:46:24.726943+0000","last_fullsized":"2024-09-15T21:46:24.726943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:52:24.175554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414781+0000","last_change":"2024-09-15T21:46:22.427164+0000","last_active":"2024-09-15T21:46:24.414781+0000","last_peered":"2024-09-15T21:46:24.414781+0000","last_clean":"2024-09-15T21:46:24.414781+0000","last_became_active":"2024-09-15T21:46:22.426917+0000","last_became_peered":"2024-09-15T21:46:22.426917+0000","last_unstale":"2024-09-15T21:46:24.414781+0000","last_undegraded":"2024-09-15T21:46:24.414781+0000","last_fullsized":"2024-09-15T21:46:24.414781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:46.471935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835504+0000","last_change":"2024-09-15T21:46:23.433924+0000","last_active":"2024-09-15T21:46:24.835504+0000","last_peered":"2024-09-15T21:46:24.835504+0000","last_clean":"2024-09-15T21:46:24.835504+0000","last_became_active":"2024-09-15T21:46:23.433606+0000","last_became_peered":"2024-09-15T21:46:23.433606+0000","last_unstale":"2024-09-15T21:46:24.835504+0000","last_undegraded":"2024-09-15T21:46:24.835504+0000","last_fullsized":"2024-09-15T21:46:24.835504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:09:35.744149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834887+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:24.834887+0000","last_peered":"2024-09-15T21:46:24.834887+0000","last_clean":"2024-09-15T21:46:24.834887+0000","last_became_active":"2024-09-15T21:46:22.425237+0000","last_became_peered":"2024-09-15T21:46:22.425237+0000","last_unstale":"2024-09-15T21:46:24.834887+0000","last_undegraded":"2024-09-15T21:46:24.834887+0000","last_fullsized":"2024-09-15T21:46:24.834887+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:59.460420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834853+0000","last_change":"2024-09-15T21:46:23.427515+0000","last_active":"2024-09-15T21:46:24.834853+0000","last_peered":"2024-09-15T21:46:24.834853+0000","last_clean":"2024-09-15T21:46:24.834853+0000","last_became_active":"2024-09-15T21:46:23.427116+0000","last_became_peered":"2024-09-15T21:46:23.427116+0000","last_unstale":"2024-09-15T21:46:24.834853+0000","last_undegraded":"2024-09-15T21:46:24.834853+0000","last_fullsized":"2024-09-15T21:46:24.834853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:35.499970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.422679+0000","last_change":"2024-09-15T21:46:22.419905+0000","last_active":"2024-09-15T21:46:28.422679+0000","last_peered":"2024-09-15T21:46:28.422679+0000","last_clean":"2024-09-15T21:46:28.422679+0000","last_became_active":"2024-09-15T21:46:22.419620+0000","last_became_peered":"2024-09-15T21:46:22.419620+0000","last_unstale":"2024-09-15T21:46:28.422679+0000","last_undegraded":"2024-09-15T21:46:28.422679+0000","last_fullsized":"2024-09-15T21:46:28.422679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:26:51.215156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835459+0000","last_change":"2024-09-15T21:46:23.432975+0000","last_active":"2024-09-15T21:46:24.835459+0000","last_peered":"2024-09-15T21:46:24.835459+0000","last_clean":"2024-09-15T21:46:24.835459+0000","last_became_active":"2024-09-15T21:46:23.432718+0000","last_became_peered":"2024-09-15T21:46:23.432718+0000","last_unstale":"2024-09-15T21:46:24.835459+0000","last_undegraded":"2024-09-15T21:46:24.835459+0000","last_fullsized":"2024-09-15T21:46:24.835459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:59.789691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414365+0000","last_change":"2024-09-15T21:46:22.429661+0000","last_active":"2024-09-15T21:46:24.414365+0000","last_peered":"2024-09-15T21:46:24.414365+0000","last_clean":"2024-09-15T21:46:24.414365+0000","last_became_active":"2024-09-15T21:46:22.429250+0000","last_became_peered":"2024-09-15T21:46:22.429250+0000","last_unstale":"2024-09-15T21:46:24.414365+0000","last_undegraded":"2024-09-15T21:46:24.414365+0000","last_fullsized":"2024-09-15T21:46:24.414365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:29.341739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835590+0000","last_change":"2024-09-15T21:46:23.435003+0000","last_active":"2024-09-15T21:46:24.835590+0000","last_peered":"2024-09-15T21:46:24.835590+0000","last_clean":"2024-09-15T21:46:24.835590+0000","last_became_active":"2024-09-15T21:46:23.434738+0000","last_became_peered":"2024-09-15T21:46:23.434738+0000","last_unstale":"2024-09-15T21:46:24.835590+0000","last_undegraded":"2024-09-15T21:46:24.835590+0000","last_fullsized":"2024-09-15T21:46:24.835590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:53:15.814759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.417670+0000","last_change":"2024-09-15T21:46:22.425870+0000","last_active":"2024-09-15T21:46:28.417670+0000","last_peered":"2024-09-15T21:46:28.417670+0000","last_clean":"2024-09-15T21:46:28.417670+0000","last_became_active":"2024-09-15T21:46:22.425588+0000","last_became_peered":"2024-09-15T21:46:22.425588+0000","last_unstale":"2024-09-15T21:46:28.417670+0000","last_undegraded":"2024-09-15T21:46:28.417670+0000","last_fullsized":"2024-09-15T21:46:28.417670+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:40:07.586608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835837+0000","last_change":"2024-09-15T21:46:23.434657+0000","last_active":"2024-09-15T21:46:24.835837+0000","last_peered":"2024-09-15T21:46:24.835837+0000","last_clean":"2024-09-15T21:46:24.835837+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:46:24.835837+0000","last_undegraded":"2024-09-15T21:46:24.835837+0000","last_fullsized":"2024-09-15T21:46:24.835837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:02.545810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808473+0000","last_change":"2024-09-15T21:46:22.425072+0000","last_active":"2024-09-15T21:46:24.808473+0000","last_peered":"2024-09-15T21:46:24.808473+0000","last_clean":"2024-09-15T21:46:24.808473+0000","last_became_active":"2024-09-15T21:46:22.424771+0000","last_became_peered":"2024-09-15T21:46:22.424771+0000","last_unstale":"2024-09-15T21:46:24.808473+0000","last_undegraded":"2024-09-15T21:46:24.808473+0000","last_fullsized":"2024-09-15T21:46:24.808473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:26.256785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414457+0000","last_change":"2024-09-15T21:46:23.432497+0000","last_active":"2024-09-15T21:46:24.414457+0000","last_peered":"2024-09-15T21:46:24.414457+0000","last_clean":"2024-09-15T21:46:24.414457+0000","last_became_active":"2024-09-15T21:46:23.431969+0000","last_became_peered":"2024-09-15T21:46:23.431969+0000","last_unstale":"2024-09-15T21:46:24.414457+0000","last_undegraded":"2024-09-15T21:46:24.414457+0000","last_fullsized":"2024-09-15T21:46:24.414457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:32:56.359477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412577+0000","last_change":"2024-09-15T21:46:22.428668+0000","last_active":"2024-09-15T21:46:24.412577+0000","last_peered":"2024-09-15T21:46:24.412577+0000","last_clean":"2024-09-15T21:46:24.412577+0000","last_became_active":"2024-09-15T21:46:22.428341+0000","last_became_peered":"2024-09-15T21:46:22.428341+0000","last_unstale":"2024-09-15T21:46:24.412577+0000","last_undegraded":"2024-09-15T21:46:24.412577+0000","last_fullsized":"2024-09-15T21:46:24.412577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:29:45.521090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835717+0000","last_change":"2024-09-15T21:46:23.438665+0000","last_active":"2024-09-15T21:46:24.835717+0000","last_peered":"2024-09-15T21:46:24.835717+0000","last_clean":"2024-09-15T21:46:24.835717+0000","last_became_active":"2024-09-15T21:46:23.437606+0000","last_became_peered":"2024-09-15T21:46:23.437606+0000","last_unstale":"2024-09-15T21:46:24.835717+0000","last_undegraded":"2024-09-15T21:46:24.835717+0000","last_fullsized":"2024-09-15T21:46:24.835717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:14:53.751884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":171,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835126+0000","last_change":"2024-09-15T21:46:14.328094+0000","last_active":"2024-09-15T21:46:24.835126+0000","last_peered":"2024-09-15T21:46:24.835126+0000","last_clean":"2024-09-15T21:46:24.835126+0000","last_became_active":"2024-09-15T21:46:14.326991+0000","last_became_peered":"2024-09-15T21:46:14.326991+0000","last_unstale":"2024-09-15T21:46:24.835126+0000","last_undegraded":"2024-09-15T21:46:24.835126+0000","last_fullsized":"2024-09-15T21:46:24.835126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_clean_scrub_stamp":"2024-09-15T21:46:12.223400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:54:27.704130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.413799+0000","last_change":"2024-09-15T21:46:22.428887+0000","last_active":"2024-09-15T21:46:24.413799+0000","last_peered":"2024-09-15T21:46:24.413799+0000","last_clean":"2024-09-15T21:46:24.413799+0000","last_became_active":"2024-09-15T21:46:22.428619+0000","last_became_peered":"2024-09-15T21:46:22.428619+0000","last_unstale":"2024-09-15T21:46:24.413799+0000","last_undegraded":"2024-09-15T21:46:24.413799+0000","last_fullsized":"2024-09-15T21:46:24.413799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:48.434763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834644+0000","last_change":"2024-09-15T21:46:23.435459+0000","last_active":"2024-09-15T21:46:24.834644+0000","last_peered":"2024-09-15T21:46:24.834644+0000","last_clean":"2024-09-15T21:46:24.834644+0000","last_became_active":"2024-09-15T21:46:23.435141+0000","last_became_peered":"2024-09-15T21:46:23.435141+0000","last_unstale":"2024-09-15T21:46:24.834644+0000","last_undegraded":"2024-09-15T21:46:24.834644+0000","last_fullsized":"2024-09-15T21:46:24.834644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:40:50.707124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834903+0000","last_change":"2024-09-15T21:46:22.431113+0000","last_active":"2024-09-15T21:46:24.834903+0000","last_peered":"2024-09-15T21:46:24.834903+0000","last_clean":"2024-09-15T21:46:24.834903+0000","last_became_active":"2024-09-15T21:46:22.430832+0000","last_became_peered":"2024-09-15T21:46:22.430832+0000","last_unstale":"2024-09-15T21:46:24.834903+0000","last_undegraded":"2024-09-15T21:46:24.834903+0000","last_fullsized":"2024-09-15T21:46:24.834903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:28.849259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.726839+0000","last_change":"2024-09-15T21:46:23.430396+0000","last_active":"2024-09-15T21:46:24.726839+0000","last_peered":"2024-09-15T21:46:24.726839+0000","last_clean":"2024-09-15T21:46:24.726839+0000","last_became_active":"2024-09-15T21:46:23.430138+0000","last_became_peered":"2024-09-15T21:46:23.430138+0000","last_unstale":"2024-09-15T21:46:24.726839+0000","last_undegraded":"2024-09-15T21:46:24.726839+0000","last_fullsized":"2024-09-15T21:46:24.726839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:50.622949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412620+0000","last_change":"2024-09-15T21:46:22.425808+0000","last_active":"2024-09-15T21:46:24.412620+0000","last_peered":"2024-09-15T21:46:24.412620+0000","last_clean":"2024-09-15T21:46:24.412620+0000","last_became_active":"2024-09-15T21:46:22.425510+0000","last_became_peered":"2024-09-15T21:46:22.425510+0000","last_unstale":"2024-09-15T21:46:24.412620+0000","last_undegraded":"2024-09-15T21:46:24.412620+0000","last_fullsized":"2024-09-15T21:46:24.412620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:11:00.338076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808338+0000","last_change":"2024-09-15T21:46:23.427051+0000","last_active":"2024-09-15T21:46:24.808338+0000","last_peered":"2024-09-15T21:46:24.808338+0000","last_clean":"2024-09-15T21:46:24.808338+0000","last_became_active":"2024-09-15T21:46:23.426185+0000","last_became_peered":"2024-09-15T21:46:23.426185+0000","last_unstale":"2024-09-15T21:46:24.808338+0000","last_undegraded":"2024-09-15T21:46:24.808338+0000","last_fullsized":"2024-09-15T21:46:24.808338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:48.318405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414866+0000","last_change":"2024-09-15T21:46:22.424645+0000","last_active":"2024-09-15T21:46:24.414866+0000","last_peered":"2024-09-15T21:46:24.414866+0000","last_clean":"2024-09-15T21:46:24.414866+0000","last_became_active":"2024-09-15T21:46:22.421229+0000","last_became_peered":"2024-09-15T21:46:22.421229+0000","last_unstale":"2024-09-15T21:46:24.414866+0000","last_undegraded":"2024-09-15T21:46:24.414866+0000","last_fullsized":"2024-09-15T21:46:24.414866+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:49.165670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414000+0000","last_change":"2024-09-15T21:46:23.425699+0000","last_active":"2024-09-15T21:46:24.414000+0000","last_peered":"2024-09-15T21:46:24.414000+0000","last_clean":"2024-09-15T21:46:24.414000+0000","last_became_active":"2024-09-15T21:46:23.425416+0000","last_became_peered":"2024-09-15T21:46:23.425416+0000","last_unstale":"2024-09-15T21:46:24.414000+0000","last_undegraded":"2024-09-15T21:46:24.414000+0000","last_fullsized":"2024-09-15T21:46:24.414000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:00.325179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808438+0000","last_change":"2024-09-15T21:46:22.426492+0000","last_active":"2024-09-15T21:46:24.808438+0000","last_peered":"2024-09-15T21:46:24.808438+0000","last_clean":"2024-09-15T21:46:24.808438+0000","last_became_active":"2024-09-15T21:46:22.426210+0000","last_became_peered":"2024-09-15T21:46:22.426210+0000","last_unstale":"2024-09-15T21:46:24.808438+0000","last_undegraded":"2024-09-15T21:46:24.808438+0000","last_fullsized":"2024-09-15T21:46:24.808438+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:05.248091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414861+0000","last_change":"2024-09-15T21:46:23.429837+0000","last_active":"2024-09-15T21:46:24.414861+0000","last_peered":"2024-09-15T21:46:24.414861+0000","last_clean":"2024-09-15T21:46:24.414861+0000","last_became_active":"2024-09-15T21:46:23.429254+0000","last_became_peered":"2024-09-15T21:46:23.429254+0000","last_unstale":"2024-09-15T21:46:24.414861+0000","last_undegraded":"2024-09-15T21:46:24.414861+0000","last_fullsized":"2024-09-15T21:46:24.414861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:58.465583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835937+0000","last_change":"2024-09-15T21:46:22.430710+0000","last_active":"2024-09-15T21:46:24.835937+0000","last_peered":"2024-09-15T21:46:24.835937+0000","last_clean":"2024-09-15T21:46:24.835937+0000","last_became_active":"2024-09-15T21:46:22.430375+0000","last_became_peered":"2024-09-15T21:46:22.430375+0000","last_unstale":"2024-09-15T21:46:24.835937+0000","last_undegraded":"2024-09-15T21:46:24.835937+0000","last_fullsized":"2024-09-15T21:46:24.835937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:04.081179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808447+0000","last_change":"2024-09-15T21:46:23.426729+0000","last_active":"2024-09-15T21:46:24.808447+0000","last_peered":"2024-09-15T21:46:24.808447+0000","last_clean":"2024-09-15T21:46:24.808447+0000","last_became_active":"2024-09-15T21:46:23.426495+0000","last_became_peered":"2024-09-15T21:46:23.426495+0000","last_unstale":"2024-09-15T21:46:24.808447+0000","last_undegraded":"2024-09-15T21:46:24.808447+0000","last_fullsized":"2024-09-15T21:46:24.808447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:03.277997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808436+0000","last_change":"2024-09-15T21:46:22.427137+0000","last_active":"2024-09-15T21:46:24.808436+0000","last_peered":"2024-09-15T21:46:24.808436+0000","last_clean":"2024-09-15T21:46:24.808436+0000","last_became_active":"2024-09-15T21:46:22.426855+0000","last_became_peered":"2024-09-15T21:46:22.426855+0000","last_unstale":"2024-09-15T21:46:24.808436+0000","last_undegraded":"2024-09-15T21:46:24.808436+0000","last_fullsized":"2024-09-15T21:46:24.808436+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:40:14.277940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834714+0000","last_change":"2024-09-15T21:46:23.427327+0000","last_active":"2024-09-15T21:46:24.834714+0000","last_peered":"2024-09-15T21:46:24.834714+0000","last_clean":"2024-09-15T21:46:24.834714+0000","last_became_active":"2024-09-15T21:46:23.427096+0000","last_became_peered":"2024-09-15T21:46:23.427096+0000","last_unstale":"2024-09-15T21:46:24.834714+0000","last_undegraded":"2024-09-15T21:46:24.834714+0000","last_fullsized":"2024-09-15T21:46:24.834714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:32.370390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.418471+0000","last_change":"2024-09-15T21:46:22.427798+0000","last_active":"2024-09-15T21:46:28.418471+0000","last_peered":"2024-09-15T21:46:28.418471+0000","last_clean":"2024-09-15T21:46:28.418471+0000","last_became_active":"2024-09-15T21:46:22.427537+0000","last_became_peered":"2024-09-15T21:46:22.427537+0000","last_unstale":"2024-09-15T21:46:28.418471+0000","last_undegraded":"2024-09-15T21:46:28.418471+0000","last_fullsized":"2024-09-15T21:46:28.418471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:18:54.963834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727157+0000","last_change":"2024-09-15T21:46:23.424266+0000","last_active":"2024-09-15T21:46:24.727157+0000","last_peered":"2024-09-15T21:46:24.727157+0000","last_clean":"2024-09-15T21:46:24.727157+0000","last_became_active":"2024-09-15T21:46:23.424007+0000","last_became_peered":"2024-09-15T21:46:23.424007+0000","last_unstale":"2024-09-15T21:46:24.727157+0000","last_undegraded":"2024-09-15T21:46:24.727157+0000","last_fullsized":"2024-09-15T21:46:24.727157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:23.492253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835005+0000","last_change":"2024-09-15T21:46:22.430901+0000","last_active":"2024-09-15T21:46:24.835005+0000","last_peered":"2024-09-15T21:46:24.835005+0000","last_clean":"2024-09-15T21:46:24.835005+0000","last_became_active":"2024-09-15T21:46:22.430452+0000","last_became_peered":"2024-09-15T21:46:22.430452+0000","last_unstale":"2024-09-15T21:46:24.835005+0000","last_undegraded":"2024-09-15T21:46:24.835005+0000","last_fullsized":"2024-09-15T21:46:24.835005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:59:45.663976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.413661+0000","last_change":"2024-09-15T21:46:23.430836+0000","last_active":"2024-09-15T21:46:24.413661+0000","last_peered":"2024-09-15T21:46:24.413661+0000","last_clean":"2024-09-15T21:46:24.413661+0000","last_became_active":"2024-09-15T21:46:23.429283+0000","last_became_peered":"2024-09-15T21:46:23.429283+0000","last_unstale":"2024-09-15T21:46:24.413661+0000","last_undegraded":"2024-09-15T21:46:24.413661+0000","last_fullsized":"2024-09-15T21:46:24.413661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:30.571376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808380+0000","last_change":"2024-09-15T21:46:22.423795+0000","last_active":"2024-09-15T21:46:24.808380+0000","last_peered":"2024-09-15T21:46:24.808380+0000","last_clean":"2024-09-15T21:46:24.808380+0000","last_became_active":"2024-09-15T21:46:22.422327+0000","last_became_peered":"2024-09-15T21:46:22.422327+0000","last_unstale":"2024-09-15T21:46:24.808380+0000","last_undegraded":"2024-09-15T21:46:24.808380+0000","last_fullsized":"2024-09-15T21:46:24.808380+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:16.670599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834522+0000","last_change":"2024-09-15T21:46:23.428634+0000","last_active":"2024-09-15T21:46:24.834522+0000","last_peered":"2024-09-15T21:46:24.834522+0000","last_clean":"2024-09-15T21:46:24.834522+0000","last_became_active":"2024-09-15T21:46:23.428301+0000","last_became_peered":"2024-09-15T21:46:23.428301+0000","last_unstale":"2024-09-15T21:46:24.834522+0000","last_undegraded":"2024-09-15T21:46:24.834522+0000","last_fullsized":"2024-09-15T21:46:24.834522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:59.492942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808304+0000","last_change":"2024-09-15T21:46:22.425810+0000","last_active":"2024-09-15T21:46:24.808304+0000","last_peered":"2024-09-15T21:46:24.808304+0000","last_clean":"2024-09-15T21:46:24.808304+0000","last_became_active":"2024-09-15T21:46:22.425478+0000","last_became_peered":"2024-09-15T21:46:22.425478+0000","last_unstale":"2024-09-15T21:46:24.808304+0000","last_undegraded":"2024-09-15T21:46:24.808304+0000","last_fullsized":"2024-09-15T21:46:24.808304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:53.296599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834743+0000","last_change":"2024-09-15T21:46:23.426054+0000","last_active":"2024-09-15T21:46:24.834743+0000","last_peered":"2024-09-15T21:46:24.834743+0000","last_clean":"2024-09-15T21:46:24.834743+0000","last_became_active":"2024-09-15T21:46:23.425547+0000","last_became_peered":"2024-09-15T21:46:23.425547+0000","last_unstale":"2024-09-15T21:46:24.834743+0000","last_undegraded":"2024-09-15T21:46:24.834743+0000","last_fullsized":"2024-09-15T21:46:24.834743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:56.838503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.418146+0000","last_change":"2024-09-15T21:46:22.428524+0000","last_active":"2024-09-15T21:46:28.418146+0000","last_peered":"2024-09-15T21:46:28.418146+0000","last_clean":"2024-09-15T21:46:28.418146+0000","last_became_active":"2024-09-15T21:46:22.428043+0000","last_became_peered":"2024-09-15T21:46:22.428043+0000","last_unstale":"2024-09-15T21:46:28.418146+0000","last_undegraded":"2024-09-15T21:46:28.418146+0000","last_fullsized":"2024-09-15T21:46:28.418146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:08.393381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834850+0000","last_change":"2024-09-15T21:46:23.434719+0000","last_active":"2024-09-15T21:46:24.834850+0000","last_peered":"2024-09-15T21:46:24.834850+0000","last_clean":"2024-09-15T21:46:24.834850+0000","last_became_active":"2024-09-15T21:46:23.434448+0000","last_became_peered":"2024-09-15T21:46:23.434448+0000","last_unstale":"2024-09-15T21:46:24.834850+0000","last_undegraded":"2024-09-15T21:46:24.834850+0000","last_fullsized":"2024-09-15T21:46:24.834850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:14.901671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414691+0000","last_change":"2024-09-15T21:46:23.430058+0000","last_active":"2024-09-15T21:46:24.414691+0000","last_peered":"2024-09-15T21:46:24.414691+0000","last_clean":"2024-09-15T21:46:24.414691+0000","last_became_active":"2024-09-15T21:46:23.429598+0000","last_became_peered":"2024-09-15T21:46:23.429598+0000","last_unstale":"2024-09-15T21:46:24.414691+0000","last_undegraded":"2024-09-15T21:46:24.414691+0000","last_fullsized":"2024-09-15T21:46:24.414691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:44.142207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.418133+0000","last_change":"2024-09-15T21:46:22.431172+0000","last_active":"2024-09-15T21:46:28.418133+0000","last_peered":"2024-09-15T21:46:28.418133+0000","last_clean":"2024-09-15T21:46:28.418133+0000","last_became_active":"2024-09-15T21:46:22.430800+0000","last_became_peered":"2024-09-15T21:46:22.430800+0000","last_unstale":"2024-09-15T21:46:28.418133+0000","last_undegraded":"2024-09-15T21:46:28.418133+0000","last_fullsized":"2024-09-15T21:46:28.418133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:55.072466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414264+0000","last_change":"2024-09-15T21:46:23.434353+0000","last_active":"2024-09-15T21:46:24.414264+0000","last_peered":"2024-09-15T21:46:24.414264+0000","last_clean":"2024-09-15T21:46:24.414264+0000","last_became_active":"2024-09-15T21:46:23.434097+0000","last_became_peered":"2024-09-15T21:46:23.434097+0000","last_unstale":"2024-09-15T21:46:24.414264+0000","last_undegraded":"2024-09-15T21:46:24.414264+0000","last_fullsized":"2024-09-15T21:46:24.414264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:46.007227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.418130+0000","last_change":"2024-09-15T21:46:22.429870+0000","last_active":"2024-09-15T21:46:28.418130+0000","last_peered":"2024-09-15T21:46:28.418130+0000","last_clean":"2024-09-15T21:46:28.418130+0000","last_became_active":"2024-09-15T21:46:22.429578+0000","last_became_peered":"2024-09-15T21:46:22.429578+0000","last_unstale":"2024-09-15T21:46:28.418130+0000","last_undegraded":"2024-09-15T21:46:28.418130+0000","last_fullsized":"2024-09-15T21:46:28.418130+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:41.678078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727142+0000","last_change":"2024-09-15T21:46:23.429604+0000","last_active":"2024-09-15T21:46:24.727142+0000","last_peered":"2024-09-15T21:46:24.727142+0000","last_clean":"2024-09-15T21:46:24.727142+0000","last_became_active":"2024-09-15T21:46:23.429355+0000","last_became_peered":"2024-09-15T21:46:23.429355+0000","last_unstale":"2024-09-15T21:46:24.727142+0000","last_undegraded":"2024-09-15T21:46:24.727142+0000","last_fullsized":"2024-09-15T21:46:24.727142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:45.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834679+0000","last_change":"2024-09-15T21:46:22.428529+0000","last_active":"2024-09-15T21:46:24.834679+0000","last_peered":"2024-09-15T21:46:24.834679+0000","last_clean":"2024-09-15T21:46:24.834679+0000","last_became_active":"2024-09-15T21:46:22.428263+0000","last_became_peered":"2024-09-15T21:46:22.428263+0000","last_unstale":"2024-09-15T21:46:24.834679+0000","last_undegraded":"2024-09-15T21:46:24.834679+0000","last_fullsized":"2024-09-15T21:46:24.834679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:22:40.160062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834701+0000","last_change":"2024-09-15T21:46:23.427968+0000","last_active":"2024-09-15T21:46:24.834701+0000","last_peered":"2024-09-15T21:46:24.834701+0000","last_clean":"2024-09-15T21:46:24.834701+0000","last_became_active":"2024-09-15T21:46:23.427546+0000","last_became_peered":"2024-09-15T21:46:23.427546+0000","last_unstale":"2024-09-15T21:46:24.834701+0000","last_undegraded":"2024-09-15T21:46:24.834701+0000","last_fullsized":"2024-09-15T21:46:24.834701+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:47.226097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.419172+0000","last_change":"2024-09-15T21:46:22.428258+0000","last_active":"2024-09-15T21:46:28.419172+0000","last_peered":"2024-09-15T21:46:28.419172+0000","last_clean":"2024-09-15T21:46:28.419172+0000","last_became_active":"2024-09-15T21:46:22.427954+0000","last_became_peered":"2024-09-15T21:46:22.427954+0000","last_unstale":"2024-09-15T21:46:28.419172+0000","last_undegraded":"2024-09-15T21:46:28.419172+0000","last_fullsized":"2024-09-15T21:46:28.419172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:22.440456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":3032,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":46,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":5442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476743,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":667964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":819200,"data_stored":667908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":78570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":77048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":77084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":76958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542151,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":79192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:46:37.345 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:46:37.345 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T21:46:37.345 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-15T21:46:37.345 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T21:46:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.478+0000 7fc5590af640 1 Processor -- start 2024-09-15T21:46:37.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.478+0000 7fc5590af640 1 -- start start 2024-09-15T21:46:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.479+0000 7fc5590af640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc5540c8600 0x7fc55415ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.479+0000 7fc5590af640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5540c9210 0x7fc554154e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.479+0000 7fc5590af640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc55415bfb0 con 0x7fc5540c89e0 2024-09-15T21:46:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.479+0000 7fc5590af640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc5540c9070 con 0x7fc5540c8600 2024-09-15T21:46:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.479+0000 7fc5590af640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc55406c920 con 0x7fc5540c9210 2024-09-15T21:46:37.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.479+0000 7fc552d9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc5540c8600 0x7fc55415ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.479+0000 7fc552d9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc5540c8600 0x7fc55415ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45464/0 (socket says 172.21.15.100:45464) 2024-09-15T21:46:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.479+0000 7fc552d9d640 1 -- 172.21.15.100:0/1788659601 learned_addr learned my addr 172.21.15.100:0/1788659601 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.479+0000 7fc55359e640 1 --2- 172.21.15.100:0/1788659601 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5540c9210 0x7fc554154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.479+0000 7fc552d9d640 1 -- 172.21.15.100:0/1788659601 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5540c9210 msgr2=0x7fc554154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.479+0000 7fc552d9d640 1 --2- 172.21.15.100:0/1788659601 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5540c9210 0x7fc554154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.479+0000 7fc552d9d640 1 -- 172.21.15.100:0/1788659601 >> v1:172.21.15.122:6789/0 conn(0x7fc5540c89e0 legacy=0x7fc55406cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.479+0000 7fc552d9d640 1 -- 172.21.15.100:0/1788659601 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc55406c010 con 0x7fc5540c8600 2024-09-15T21:46:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.480+0000 7fc552d9d640 1 --2- 172.21.15.100:0/1788659601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5540c8600 0x7fc55415ba70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc55406caa0 tx=0x7fc54402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48502789eba1cdf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.480+0000 7fc551d9b640 1 -- 172.21.15.100:0/1788659601 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544037070 con 0x7fc5540c8600 2024-09-15T21:46:37.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.480+0000 7fc551d9b640 1 -- 172.21.15.100:0/1788659601 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc544032440 con 0x7fc5540c8600 2024-09-15T21:46:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.480+0000 7fc551d9b640 1 -- 172.21.15.100:0/1788659601 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544032810 con 0x7fc5540c8600 2024-09-15T21:46:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.480+0000 7fc5590af640 1 -- 172.21.15.100:0/1788659601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5540c8600 msgr2=0x7fc55415ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.480+0000 7fc5590af640 1 --2- 172.21.15.100:0/1788659601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5540c8600 0x7fc55415ba70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc55406caa0 tx=0x7fc54402a250 comp rx=0 tx=0).stop 2024-09-15T21:46:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.480+0000 7fc5590af640 1 -- 172.21.15.100:0/1788659601 shutdown_connections 2024-09-15T21:46:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.480+0000 7fc5590af640 1 --2- 172.21.15.100:0/1788659601 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5540c9210 0x7fc554154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.480+0000 7fc5590af640 1 --2- 172.21.15.100:0/1788659601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5540c8600 0x7fc55415ba70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.480+0000 7fc5590af640 1 -- 172.21.15.100:0/1788659601 >> 172.21.15.100:0/1788659601 conn(0x7fc554070310 msgr2=0x7fc5540cd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.481+0000 7fc5590af640 1 -- 172.21.15.100:0/1788659601 shutdown_connections 2024-09-15T21:46:37.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.481+0000 7fc5590af640 1 -- 172.21.15.100:0/1788659601 wait complete. 2024-09-15T21:46:37.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.482+0000 7fc5590af640 1 Processor -- start 2024-09-15T21:46:37.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.482+0000 7fc5590af640 1 -- start start 2024-09-15T21:46:37.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc5590af640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5540c8600 0x7fc554153c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc5590af640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5540c89e0 0x7fc5541e55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc5590af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5540c9210 0x7fc5541e5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc5590af640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc55406c2d0 con 0x7fc5540c9210 2024-09-15T21:46:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc5590af640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5540c9e10 con 0x7fc5540c89e0 2024-09-15T21:46:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc5590af640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc55406c5b0 con 0x7fc5540c8600 2024-09-15T21:46:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc55359e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5540c9210 0x7fc5541e5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc55359e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5540c9210 0x7fc5541e5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45466/0 (socket says 172.21.15.100:45466) 2024-09-15T21:46:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc55359e640 1 -- 172.21.15.100:0/2800349419 learned_addr learned my addr 172.21.15.100:0/2800349419 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc552d9d640 1 --2- 172.21.15.100:0/2800349419 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5540c8600 0x7fc554153c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc55259c640 1 --2- 172.21.15.100:0/2800349419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5540c89e0 0x7fc5541e55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc55359e640 1 -- 172.21.15.100:0/2800349419 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5540c8600 msgr2=0x7fc554153c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:37.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc55359e640 1 --2- 172.21.15.100:0/2800349419 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5540c8600 0x7fc554153c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc55359e640 1 -- 172.21.15.100:0/2800349419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5540c89e0 msgr2=0x7fc5541e55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:37.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc55359e640 1 --2- 172.21.15.100:0/2800349419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5540c89e0 0x7fc5541e55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc55359e640 1 -- 172.21.15.100:0/2800349419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5541f0ef0 con 0x7fc5540c9210 2024-09-15T21:46:37.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc55359e640 1 --2- 172.21.15.100:0/2800349419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5540c9210 0x7fc5541e5ae0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc54c007e60 tx=0x7fc54c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:37.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc53b7fe640 1 -- 172.21.15.100:0/2800349419 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c014420 con 0x7fc5540c9210 2024-09-15T21:46:37.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc53b7fe640 1 -- 172.21.15.100:0/2800349419 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc54c019070 con 0x7fc5540c9210 2024-09-15T21:46:37.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc5590af640 1 -- 172.21.15.100:0/2800349419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5541f4480 con 0x7fc5540c9210 2024-09-15T21:46:37.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc53b7fe640 1 -- 172.21.15.100:0/2800349419 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54c014820 con 0x7fc5540c9210 2024-09-15T21:46:37.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.483+0000 7fc5590af640 1 -- 172.21.15.100:0/2800349419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5541f4a40 con 0x7fc5540c9210 2024-09-15T21:46:37.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.484+0000 7fc5590af640 1 -- 172.21.15.100:0/2800349419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc554072c30 con 0x7fc5540c9210 2024-09-15T21:46:37.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.485+0000 7fc53b7fe640 1 -- 172.21.15.100:0/2800349419 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc54c0149c0 con 0x7fc5540c9210 2024-09-15T21:46:37.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.485+0000 7fc53b7fe640 1 --2- 172.21.15.100:0/2800349419 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc54006c2b0 0x7fc54006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:37.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.485+0000 7fc53b7fe640 1 -- 172.21.15.100:0/2800349419 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fc54c08bb90 con 0x7fc5540c9210 2024-09-15T21:46:37.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.485+0000 7fc552d9d640 1 --2- 172.21.15.100:0/2800349419 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc54006c2b0 0x7fc54006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:37.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.486+0000 7fc552d9d640 1 --2- 172.21.15.100:0/2800349419 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc54006c2b0 0x7fc54006e770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc544034000 tx=0x7fc54402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:37.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.490+0000 7fc53b7fe640 1 -- 172.21.15.100:0/2800349419 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc554072c30 con 0x7fc5540c9210 2024-09-15T21:46:37.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.689+0000 7fc5590af640 1 -- 172.21.15.100:0/2800349419 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc5540c9760 con 0x7fc5540c9210 2024-09-15T21:46:37.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.689+0000 7fc53b7fe640 1 -- 172.21.15.100:0/2800349419 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7fc54c059c20 con 0x7fc5540c9210 2024-09-15T21:46:37.692 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:37.692 INFO:teuthology.orchestra.run.smithi100.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-09-15T21:46:37.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.691+0000 7fc5590af640 1 -- 172.21.15.100:0/2800349419 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc54006c2b0 msgr2=0x7fc54006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:37.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.691+0000 7fc5590af640 1 --2- 172.21.15.100:0/2800349419 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc54006c2b0 0x7fc54006e770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc544034000 tx=0x7fc54402e000 comp rx=0 tx=0).stop 2024-09-15T21:46:37.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.691+0000 7fc5590af640 1 -- 172.21.15.100:0/2800349419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5540c9210 msgr2=0x7fc5541e5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:37.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.691+0000 7fc5590af640 1 --2- 172.21.15.100:0/2800349419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5540c9210 0x7fc5541e5ae0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc54c007e60 tx=0x7fc54c00e4b0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.691+0000 7fc5590af640 1 -- 172.21.15.100:0/2800349419 shutdown_connections 2024-09-15T21:46:37.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.691+0000 7fc5590af640 1 --2- 172.21.15.100:0/2800349419 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc54006c2b0 0x7fc54006e770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.691+0000 7fc5590af640 1 --2- 172.21.15.100:0/2800349419 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5540c89e0 0x7fc5541e55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.691+0000 7fc5590af640 1 --2- 172.21.15.100:0/2800349419 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc5540c9210 0x7fc5541e5ae0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.691+0000 7fc5590af640 1 --2- 172.21.15.100:0/2800349419 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc5540c8600 0x7fc554153c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:37.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.691+0000 7fc5590af640 1 -- 172.21.15.100:0/2800349419 >> 172.21.15.100:0/2800349419 conn(0x7fc554070310 msgr2=0x7fc5540cc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:37.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.691+0000 7fc5590af640 1 -- 172.21.15.100:0/2800349419 shutdown_connections 2024-09-15T21:46:37.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:37.691+0000 7fc5590af640 1 -- 172.21.15.100:0/2800349419 wait complete. 2024-09-15T21:46:38.394 INFO:tasks.ceph.mds.e.smithi100.stderr:2024-09-15T21:46:38.393+0000 7fa09cc49640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-15T21:46:38.395 INFO:tasks.ceph.mds.e.smithi100.stderr:2024-09-15T21:46:38.393+0000 7fa09cc49640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-15T21:46:40.470 INFO:tasks.ceph.mds.b.smithi122.stderr:2024-09-15T21:46:40.469+0000 7ff533659640 -1 quiesce.mds.1 failed to decode message of type 1285 v1: End of buffer [buffer:2] 2024-09-15T21:46:40.705 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T21:46:40.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.811+0000 7f25e9bfd640 1 Processor -- start 2024-09-15T21:46:40.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.811+0000 7f25e9bfd640 1 -- start start 2024-09-15T21:46:40.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.812+0000 7f25e9bfd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f25e4070bd0 0x7f25e4071500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:40.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.812+0000 7f25e9bfd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25e40cc4c0 0x7f25e40733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:40.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.812+0000 7f25e9bfd640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25e40b3e40 con 0x7f25e4155330 2024-09-15T21:46:40.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.812+0000 7f25e9bfd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f25e40ca060 con 0x7f25e4070bd0 2024-09-15T21:46:40.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.812+0000 7f25e9bfd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f25e40c9bd0 con 0x7f25e40cc4c0 2024-09-15T21:46:40.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.812+0000 7f25e2ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f25e4070bd0 0x7f25e4071500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:40.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.812+0000 7f25e2ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f25e4070bd0 0x7f25e4071500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45476/0 (socket says 172.21.15.100:45476) 2024-09-15T21:46:40.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.812+0000 7f25e2ffd640 1 -- 172.21.15.100:0/3986223469 learned_addr learned my addr 172.21.15.100:0/3986223469 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:40.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.812+0000 7f25e3fff640 1 --2- 172.21.15.100:0/3986223469 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25e40cc4c0 0x7f25e40733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:40.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.812+0000 7f25e2ffd640 1 -- 172.21.15.100:0/3986223469 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25e40cc4c0 msgr2=0x7f25e40733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:40.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.812+0000 7f25e2ffd640 1 --2- 172.21.15.100:0/3986223469 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25e40cc4c0 0x7f25e40733f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:40.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.812+0000 7f25e2ffd640 1 -- 172.21.15.100:0/3986223469 >> v1:172.21.15.122:6789/0 conn(0x7f25e4155330 legacy=0x7f25e415e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:40.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.812+0000 7f25e2ffd640 1 -- 172.21.15.100:0/3986223469 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e40c98a0 con 0x7f25e4070bd0 2024-09-15T21:46:40.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.812+0000 7f25e3fff640 1 --2- 172.21.15.100:0/3986223469 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25e40cc4c0 0x7f25e40733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:40.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.813+0000 7f25e2ffd640 1 --2- 172.21.15.100:0/3986223469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e4070bd0 0x7f25e4071500 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f25d8009870 tx=0x7f25d802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677a2713eb503a2b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:40.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.813+0000 7f25e27fc640 1 -- 172.21.15.100:0/3986223469 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8037070 con 0x7f25e4070bd0 2024-09-15T21:46:40.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.813+0000 7f25e27fc640 1 -- 172.21.15.100:0/3986223469 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25d802ae50 con 0x7f25e4070bd0 2024-09-15T21:46:40.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.813+0000 7f25e27fc640 1 -- 172.21.15.100:0/3986223469 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25d8032860 con 0x7f25e4070bd0 2024-09-15T21:46:40.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.814+0000 7f25e9bfd640 1 -- 172.21.15.100:0/3986223469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e4070bd0 msgr2=0x7f25e4071500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:40.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.814+0000 7f25e9bfd640 1 --2- 172.21.15.100:0/3986223469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e4070bd0 0x7f25e4071500 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f25d8009870 tx=0x7f25d802a300 comp rx=0 tx=0).stop 2024-09-15T21:46:40.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.814+0000 7f25e9bfd640 1 -- 172.21.15.100:0/3986223469 shutdown_connections 2024-09-15T21:46:40.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.814+0000 7f25e9bfd640 1 --2- 172.21.15.100:0/3986223469 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25e40cc4c0 0x7f25e40733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:40.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.814+0000 7f25e9bfd640 1 --2- 172.21.15.100:0/3986223469 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e4070bd0 0x7f25e4071500 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:40.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.814+0000 7f25e9bfd640 1 -- 172.21.15.100:0/3986223469 >> 172.21.15.100:0/3986223469 conn(0x7f25e406c210 msgr2=0x7f25e406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:40.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.814+0000 7f25e9bfd640 1 -- 172.21.15.100:0/3986223469 shutdown_connections 2024-09-15T21:46:40.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.814+0000 7f25e9bfd640 1 -- 172.21.15.100:0/3986223469 wait complete. 2024-09-15T21:46:40.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.815+0000 7f25e9bfd640 1 Processor -- start 2024-09-15T21:46:40.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.815+0000 7f25e9bfd640 1 -- start start 2024-09-15T21:46:40.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.816+0000 7f25e9bfd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25e4070bd0 0x7f25e4155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:40.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.816+0000 7f25e9bfd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25e40cc4c0 0x7f25e414d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:40.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.816+0000 7f25e9bfd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e4155330 0x7f25e41e4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:40.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.816+0000 7f25e9bfd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f25e40b40e0 con 0x7f25e4155330 2024-09-15T21:46:40.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.816+0000 7f25e9bfd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f25e40710a0 con 0x7f25e40cc4c0 2024-09-15T21:46:40.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.816+0000 7f25e9bfd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f25e40b43c0 con 0x7f25e4070bd0 2024-09-15T21:46:40.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.816+0000 7f25e3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e4155330 0x7f25e41e4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:40.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.816+0000 7f25e3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e4155330 0x7f25e41e4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45480/0 (socket says 172.21.15.100:45480) 2024-09-15T21:46:40.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.816+0000 7f25e3fff640 1 -- 172.21.15.100:0/1219522001 learned_addr learned my addr 172.21.15.100:0/1219522001 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:40.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.816+0000 7f25e2ffd640 1 --2- 172.21.15.100:0/1219522001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25e40cc4c0 0x7f25e414d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:40.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.816+0000 7f25e37fe640 1 --2- 172.21.15.100:0/1219522001 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25e4070bd0 0x7f25e4155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:40.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.816+0000 7f25e3fff640 1 -- 172.21.15.100:0/1219522001 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25e4070bd0 msgr2=0x7f25e4155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:40.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.816+0000 7f25e3fff640 1 --2- 172.21.15.100:0/1219522001 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25e4070bd0 0x7f25e4155930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:40.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.816+0000 7f25e3fff640 1 -- 172.21.15.100:0/1219522001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25e40cc4c0 msgr2=0x7f25e414d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:40.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.816+0000 7f25e3fff640 1 --2- 172.21.15.100:0/1219522001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25e40cc4c0 0x7f25e414d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.816+0000 7f25e3fff640 1 -- 172.21.15.100:0/1219522001 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e41f2c60 con 0x7f25e4155330 2024-09-15T21:46:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.817+0000 7f25e3fff640 1 --2- 172.21.15.100:0/1219522001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e4155330 0x7f25e41e4470 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f25dc007ef0 tx=0x7f25dc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.817+0000 7f25cbfff640 1 -- 172.21.15.100:0/1219522001 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc01a070 con 0x7f25e4155330 2024-09-15T21:46:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.817+0000 7f25cbfff640 1 -- 172.21.15.100:0/1219522001 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25dc003b90 con 0x7f25e4155330 2024-09-15T21:46:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.817+0000 7f25e9bfd640 1 -- 172.21.15.100:0/1219522001 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e415ef60 con 0x7f25e4155330 2024-09-15T21:46:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.817+0000 7f25cbfff640 1 -- 172.21.15.100:0/1219522001 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25dc004030 con 0x7f25e4155330 2024-09-15T21:46:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.817+0000 7f25e9bfd640 1 -- 172.21.15.100:0/1219522001 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e41f2f00 con 0x7f25e4155330 2024-09-15T21:46:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.818+0000 7f25e9bfd640 1 -- 172.21.15.100:0/1219522001 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e4071940 con 0x7f25e4155330 2024-09-15T21:46:40.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.818+0000 7f25cbfff640 1 -- 172.21.15.100:0/1219522001 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f25dc01d070 con 0x7f25e4155330 2024-09-15T21:46:40.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.819+0000 7f25cbfff640 1 --2- 172.21.15.100:0/1219522001 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f25d006c2e0 0x7f25d006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:40.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.819+0000 7f25e37fe640 1 --2- 172.21.15.100:0/1219522001 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f25d006c2e0 0x7f25d006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:40.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.819+0000 7f25cbfff640 1 -- 172.21.15.100:0/1219522001 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f25dc08b520 con 0x7f25e4155330 2024-09-15T21:46:40.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.819+0000 7f25e37fe640 1 --2- 172.21.15.100:0/1219522001 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f25d006c2e0 0x7f25d006e7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f25d40016e0 tx=0x7f25d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:40.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:40.823+0000 7f25cbfff640 1 -- 172.21.15.100:0/1219522001 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f25e4071940 con 0x7f25e4155330 2024-09-15T21:46:41.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:41.024+0000 7f25e9bfd640 1 -- 172.21.15.100:0/1219522001 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f25e40cacc0 con 0x7f25e4155330 2024-09-15T21:46:41.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:41.024+0000 7f25cbfff640 1 -- 172.21.15.100:0/1219522001 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f25dc010070 con 0x7f25e4155330 2024-09-15T21:46:41.026 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:41.026 INFO:teuthology.orchestra.run.smithi100.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-09-15T21:46:41.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:41.026+0000 7f25e9bfd640 1 -- 172.21.15.100:0/1219522001 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f25d006c2e0 msgr2=0x7f25d006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:41.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:41.026+0000 7f25e9bfd640 1 --2- 172.21.15.100:0/1219522001 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f25d006c2e0 0x7f25d006e7a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f25d40016e0 tx=0x7f25d4009040 comp rx=0 tx=0).stop 2024-09-15T21:46:41.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:41.026+0000 7f25e9bfd640 1 -- 172.21.15.100:0/1219522001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e4155330 msgr2=0x7f25e41e4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:41.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:41.026+0000 7f25e9bfd640 1 --2- 172.21.15.100:0/1219522001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e4155330 0x7f25e41e4470 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f25dc007ef0 tx=0x7f25dc00d4a0 comp rx=0 tx=0).stop 2024-09-15T21:46:41.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:41.026+0000 7f25e9bfd640 1 -- 172.21.15.100:0/1219522001 shutdown_connections 2024-09-15T21:46:41.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:41.026+0000 7f25e9bfd640 1 --2- 172.21.15.100:0/1219522001 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f25d006c2e0 0x7f25d006e7a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:41.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:41.026+0000 7f25e9bfd640 1 --2- 172.21.15.100:0/1219522001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25e40cc4c0 0x7f25e414d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:41.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:41.026+0000 7f25e9bfd640 1 --2- 172.21.15.100:0/1219522001 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f25e4155330 0x7f25e41e4470 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:41.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:41.026+0000 7f25e9bfd640 1 --2- 172.21.15.100:0/1219522001 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f25e4070bd0 0x7f25e4155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:41.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:41.026+0000 7f25e9bfd640 1 -- 172.21.15.100:0/1219522001 >> 172.21.15.100:0/1219522001 conn(0x7f25e406c210 msgr2=0x7f25e40ca210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:41.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:41.026+0000 7f25e9bfd640 1 -- 172.21.15.100:0/1219522001 shutdown_connections 2024-09-15T21:46:41.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:41.026+0000 7f25e9bfd640 1 -- 172.21.15.100:0/1219522001 wait complete. 2024-09-15T21:46:43.395 INFO:tasks.ceph.mds.e.smithi100.stderr:2024-09-15T21:46:43.392+0000 7fa09cc49640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-15T21:46:44.040 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T21:46:44.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.148+0000 7f9745798640 1 Processor -- start 2024-09-15T21:46:44.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.148+0000 7f9745798640 1 -- start start 2024-09-15T21:46:44.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.149+0000 7f9745798640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f97400d0fe0 0x7f974015b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:44.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.149+0000 7f9745798640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f97400d13c0 0x7f97400cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:44.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.149+0000 7f9745798640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f974015b820 con 0x7f97400cdd90 2024-09-15T21:46:44.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.149+0000 7f9745798640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f974015bae0 con 0x7f97400d13c0 2024-09-15T21:46:44.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.149+0000 7f9745798640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f974015bdc0 con 0x7f97400d0fe0 2024-09-15T21:46:44.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.149+0000 7f97367fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f97400d13c0 0x7f97400cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:44.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.149+0000 7f97367fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f97400d13c0 0x7f97400cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45492/0 (socket says 172.21.15.100:45492) 2024-09-15T21:46:44.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.149+0000 7f97367fc640 1 -- 172.21.15.100:0/1011795890 learned_addr learned my addr 172.21.15.100:0/1011795890 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:44.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.149+0000 7f973effd640 1 --2- 172.21.15.100:0/1011795890 >> v2:172.21.15.122:3300/0 conn(0x7f97400d0fe0 0x7f974015b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:44.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.149+0000 7f97367fc640 1 -- 172.21.15.100:0/1011795890 >> v2:172.21.15.122:3300/0 conn(0x7f97400d0fe0 msgr2=0x7f974015b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:44.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.149+0000 7f97367fc640 1 --2- 172.21.15.100:0/1011795890 >> v2:172.21.15.122:3300/0 conn(0x7f97400d0fe0 0x7f974015b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:44.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.149+0000 7f97367fc640 1 -- 172.21.15.100:0/1011795890 >> v1:172.21.15.100:6789/0 conn(0x7f97400cdd90 legacy=0x7f974015dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:44.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.149+0000 7f97367fc640 1 -- 172.21.15.100:0/1011795890 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974015c0a0 con 0x7f97400d13c0 2024-09-15T21:46:44.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.150+0000 7f97367fc640 1 --2- 172.21.15.100:0/1011795890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97400d13c0 0x7f97400cbcb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f97400ce920 tx=0x7f972802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882bc20a40f38c1c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:44.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.150+0000 7f973e7fc640 1 -- 172.21.15.100:0/1011795890 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728037070 con 0x7f97400d13c0 2024-09-15T21:46:44.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.150+0000 7f973e7fc640 1 -- 172.21.15.100:0/1011795890 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9728032440 con 0x7f97400d13c0 2024-09-15T21:46:44.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.150+0000 7f973e7fc640 1 -- 172.21.15.100:0/1011795890 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9728032810 con 0x7f97400d13c0 2024-09-15T21:46:44.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.151+0000 7f9745798640 1 -- 172.21.15.100:0/1011795890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97400d13c0 msgr2=0x7f97400cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:44.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.151+0000 7f9745798640 1 --2- 172.21.15.100:0/1011795890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97400d13c0 0x7f97400cbcb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f97400ce920 tx=0x7f972802a270 comp rx=0 tx=0).stop 2024-09-15T21:46:44.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.151+0000 7f9745798640 1 -- 172.21.15.100:0/1011795890 shutdown_connections 2024-09-15T21:46:44.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.151+0000 7f9745798640 1 --2- 172.21.15.100:0/1011795890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97400d13c0 0x7f97400cbcb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:44.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.151+0000 7f9745798640 1 --2- 172.21.15.100:0/1011795890 >> v2:172.21.15.122:3300/0 conn(0x7f97400d0fe0 0x7f974015b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:44.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.151+0000 7f9745798640 1 -- 172.21.15.100:0/1011795890 >> 172.21.15.100:0/1011795890 conn(0x7f974006c610 msgr2=0x7f9740067980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:44.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.151+0000 7f9745798640 1 -- 172.21.15.100:0/1011795890 shutdown_connections 2024-09-15T21:46:44.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.151+0000 7f9745798640 1 -- 172.21.15.100:0/1011795890 wait complete. 2024-09-15T21:46:44.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.152+0000 7f9745798640 1 Processor -- start 2024-09-15T21:46:44.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.152+0000 7f9745798640 1 -- start start 2024-09-15T21:46:44.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.153+0000 7f9745798640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f97400cdd90 0x7f97400c7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:44.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.153+0000 7f9745798640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97400d0fe0 0x7f97401584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:44.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.153+0000 7f9745798640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97400d13c0 0x7f97401530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:44.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.153+0000 7f9745798640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f97400cc1f0 con 0x7f97400d13c0 2024-09-15T21:46:44.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.153+0000 7f9745798640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f97400cd590 con 0x7f97400d0fe0 2024-09-15T21:46:44.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.153+0000 7f9745798640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f974015e1f0 con 0x7f97400cdd90 2024-09-15T21:46:44.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.153+0000 7f973f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97400d13c0 0x7f97401530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:44.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.153+0000 7f973f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97400d13c0 0x7f97401530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45504/0 (socket says 172.21.15.100:45504) 2024-09-15T21:46:44.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.153+0000 7f973f7fe640 1 -- 172.21.15.100:0/1366210219 learned_addr learned my addr 172.21.15.100:0/1366210219 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:44.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.153+0000 7f97367fc640 1 --2- 172.21.15.100:0/1366210219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97400d0fe0 0x7f97401584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:44.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.153+0000 7f973effd640 1 --2- 172.21.15.100:0/1366210219 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f97400cdd90 0x7f97400c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:44.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.153+0000 7f973f7fe640 1 -- 172.21.15.100:0/1366210219 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f97400cdd90 msgr2=0x7f97400c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:44.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.153+0000 7f973f7fe640 1 --2- 172.21.15.100:0/1366210219 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f97400cdd90 0x7f97400c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:44.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.153+0000 7f973f7fe640 1 -- 172.21.15.100:0/1366210219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97400d0fe0 msgr2=0x7f97401584a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.153+0000 7f973f7fe640 1 --2- 172.21.15.100:0/1366210219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97400d0fe0 0x7f97401584a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.153+0000 7f973f7fe640 1 -- 172.21.15.100:0/1366210219 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97401f1b10 con 0x7f97400d13c0 2024-09-15T21:46:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.154+0000 7f973f7fe640 1 --2- 172.21.15.100:0/1366210219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97400d13c0 0x7f97401530a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f973800c7d0 tx=0x7f973800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.154+0000 7f9737fff640 1 -- 172.21.15.100:0/1366210219 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9738003b70 con 0x7f97400d13c0 2024-09-15T21:46:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.154+0000 7f9737fff640 1 -- 172.21.15.100:0/1366210219 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9738003d30 con 0x7f97400d13c0 2024-09-15T21:46:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.154+0000 7f9737fff640 1 -- 172.21.15.100:0/1366210219 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973800aa30 con 0x7f97400d13c0 2024-09-15T21:46:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.154+0000 7f9745798640 1 -- 172.21.15.100:0/1366210219 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97400c77b0 con 0x7f97400d13c0 2024-09-15T21:46:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.154+0000 7f9745798640 1 -- 172.21.15.100:0/1366210219 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97400c9ec0 con 0x7f97400d13c0 2024-09-15T21:46:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.155+0000 7f9745798640 1 -- 172.21.15.100:0/1366210219 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97400cbcb0 con 0x7f97400d13c0 2024-09-15T21:46:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.155+0000 7f9737fff640 1 -- 172.21.15.100:0/1366210219 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f973800abd0 con 0x7f97400d13c0 2024-09-15T21:46:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.156+0000 7f9737fff640 1 --2- 172.21.15.100:0/1366210219 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f972006c2b0 0x7f972006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:44.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.156+0000 7f973effd640 1 --2- 172.21.15.100:0/1366210219 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f972006c2b0 0x7f972006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:44.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.156+0000 7f9737fff640 1 -- 172.21.15.100:0/1366210219 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f973808b170 con 0x7f97400d13c0 2024-09-15T21:46:44.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.156+0000 7f973effd640 1 --2- 172.21.15.100:0/1366210219 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f972006c2b0 0x7f972006e770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9730006fd0 tx=0x7f9730008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:44.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.160+0000 7f9737fff640 1 -- 172.21.15.100:0/1366210219 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9738059320 con 0x7f97400d13c0 2024-09-15T21:46:44.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.361+0000 7f9745798640 1 -- 172.21.15.100:0/1366210219 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f97400ce310 con 0x7f97400d13c0 2024-09-15T21:46:44.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.361+0000 7f9737fff640 1 -- 172.21.15.100:0/1366210219 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+196 (secure 0 0 0) 0x7f973805d620 con 0x7f97400d13c0 2024-09-15T21:46:44.364 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:44.364 INFO:teuthology.orchestra.run.smithi100.stdout:{"status":"HEALTH_WARN","checks":{"MDS_UP_LESS_THAN_MAX":{"severity":"HEALTH_WARN","summary":{"message":"1 filesystem is online with fewer MDS than max_mds","count":1},"muted":false}},"mutes":[]} 2024-09-15T21:46:44.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.363+0000 7f9745798640 1 -- 172.21.15.100:0/1366210219 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f972006c2b0 msgr2=0x7f972006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:44.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.363+0000 7f9745798640 1 --2- 172.21.15.100:0/1366210219 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f972006c2b0 0x7f972006e770 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9730006fd0 tx=0x7f9730008040 comp rx=0 tx=0).stop 2024-09-15T21:46:44.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.363+0000 7f9745798640 1 -- 172.21.15.100:0/1366210219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97400d13c0 msgr2=0x7f97401530a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:44.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.363+0000 7f9745798640 1 --2- 172.21.15.100:0/1366210219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97400d13c0 0x7f97401530a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f973800c7d0 tx=0x7f973800cc90 comp rx=0 tx=0).stop 2024-09-15T21:46:44.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.363+0000 7f9745798640 1 -- 172.21.15.100:0/1366210219 shutdown_connections 2024-09-15T21:46:44.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.363+0000 7f9745798640 1 --2- 172.21.15.100:0/1366210219 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f972006c2b0 0x7f972006e770 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:44.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.363+0000 7f9745798640 1 --2- 172.21.15.100:0/1366210219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97400d0fe0 0x7f97401584a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:44.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.363+0000 7f9745798640 1 --2- 172.21.15.100:0/1366210219 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f97400d13c0 0x7f97401530a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:44.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.363+0000 7f9745798640 1 --2- 172.21.15.100:0/1366210219 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f97400cdd90 0x7f97400c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:44.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.363+0000 7f9745798640 1 -- 172.21.15.100:0/1366210219 >> 172.21.15.100:0/1366210219 conn(0x7f974006c610 msgr2=0x7f974015d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:44.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.363+0000 7f9745798640 1 -- 172.21.15.100:0/1366210219 shutdown_connections 2024-09-15T21:46:44.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:44.363+0000 7f9745798640 1 -- 172.21.15.100:0/1366210219 wait complete. 2024-09-15T21:46:46.620 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T21:46:46.623 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T21:46:47.378 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T21:46:47.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.488+0000 7f2c85d66640 1 Processor -- start 2024-09-15T21:46:47.490 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.488+0000 7f2c85d66640 1 -- start start 2024-09-15T21:46:47.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.488+0000 7f2c85d66640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f2c80073960 0x7f2c800c7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:47.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.488+0000 7f2c85d66640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c800c7f70 con 0x7f2c80159ca0 2024-09-15T21:46:47.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.488+0000 7f2c85d66640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c800c79d0 con 0x7f2c80073580 2024-09-15T21:46:47.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.488+0000 7f2c85d66640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f2c8006ccf0 con 0x7f2c80073960 2024-09-15T21:46:47.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.489+0000 7f2c7ffff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f2c80159ca0 0x7f2c80152ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:53908/0 (socket says 172.21.15.100:53908) 2024-09-15T21:46:47.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.489+0000 7f2c7ffff640 1 -- 172.21.15.100:0/839019072 learned_addr learned my addr 172.21.15.100:0/839019072 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:47.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.489+0000 7f2c7effd640 1 --2- 172.21.15.100:0/839019072 >> v2:172.21.15.122:3300/0 conn(0x7f2c80073960 0x7f2c800c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:47.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.489+0000 7f2c7e7fc640 1 -- 172.21.15.100:0/839019072 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2776029198 0 0) 0x7f2c800c7f70 con 0x7f2c80159ca0 2024-09-15T21:46:47.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.489+0000 7f2c7e7fc640 1 -- 172.21.15.100:0/839019072 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c68003520 con 0x7f2c80159ca0 2024-09-15T21:46:47.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.489+0000 7f2c7effd640 1 -- 172.21.15.100:0/839019072 >> v1:172.21.15.122:6789/0 conn(0x7f2c80073580 legacy=0x7f2c8006c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:47.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.489+0000 7f2c7effd640 1 -- 172.21.15.100:0/839019072 >> v1:172.21.15.100:6789/0 conn(0x7f2c80159ca0 legacy=0x7f2c80152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:47.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.489+0000 7f2c7effd640 1 -- 172.21.15.100:0/839019072 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c800c7c90 con 0x7f2c80073960 2024-09-15T21:46:47.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.490+0000 7f2c7effd640 1 --2- 172.21.15.100:0/839019072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c80073960 0x7f2c800c7290 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c700097a0 tx=0x7f2c7002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d46c109133baa8f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:47.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.490+0000 7f2c7e7fc640 1 -- 172.21.15.100:0/839019072 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70037070 con 0x7f2c80073960 2024-09-15T21:46:47.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.490+0000 7f2c7e7fc640 1 -- 172.21.15.100:0/839019072 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c70032440 con 0x7f2c80073960 2024-09-15T21:46:47.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.490+0000 7f2c7e7fc640 1 -- 172.21.15.100:0/839019072 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70032810 con 0x7f2c80073960 2024-09-15T21:46:47.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.490+0000 7f2c85d66640 1 -- 172.21.15.100:0/839019072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c80073960 msgr2=0x7f2c800c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:47.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.490+0000 7f2c85d66640 1 --2- 172.21.15.100:0/839019072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c80073960 0x7f2c800c7290 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c700097a0 tx=0x7f2c7002a230 comp rx=0 tx=0).stop 2024-09-15T21:46:47.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.491+0000 7f2c85d66640 1 -- 172.21.15.100:0/839019072 shutdown_connections 2024-09-15T21:46:47.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.491+0000 7f2c85d66640 1 --2- 172.21.15.100:0/839019072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c80073960 0x7f2c800c7290 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:47.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.491+0000 7f2c85d66640 1 -- 172.21.15.100:0/839019072 >> 172.21.15.100:0/839019072 conn(0x7f2c800cc890 msgr2=0x7f2c800ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:47.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.491+0000 7f2c85d66640 1 -- 172.21.15.100:0/839019072 shutdown_connections 2024-09-15T21:46:47.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.491+0000 7f2c85d66640 1 -- 172.21.15.100:0/839019072 wait complete. 2024-09-15T21:46:47.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.492+0000 7f2c85d66640 1 Processor -- start 2024-09-15T21:46:47.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.492+0000 7f2c85d66640 1 -- start start 2024-09-15T21:46:47.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c85d66640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c80073580 0x7f2c8014f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:47.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c85d66640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c80073960 0x7f2c80157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:47.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c85d66640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2c80159ca0 0x7f2c8014d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:47.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c85d66640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2c80159780 con 0x7f2c80073960 2024-09-15T21:46:47.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c85d66640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2c801594a0 con 0x7f2c80073580 2024-09-15T21:46:47.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c85d66640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2c801531d0 con 0x7f2c80159ca0 2024-09-15T21:46:47.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c7effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c80073960 0x7f2c80157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:47.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c7effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c80073960 0x7f2c80157c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52590/0 (socket says 172.21.15.100:52590) 2024-09-15T21:46:47.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c7effd640 1 -- 172.21.15.100:0/1535723843 learned_addr learned my addr 172.21.15.100:0/1535723843 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:47.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c7f7fe640 1 --2- 172.21.15.100:0/1535723843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c80073580 0x7f2c8014f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:47.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c7ffff640 1 --2- 172.21.15.100:0/1535723843 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2c80159ca0 0x7f2c8014d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:47.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c7effd640 1 -- 172.21.15.100:0/1535723843 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2c80159ca0 msgr2=0x7f2c8014d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:47.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c7effd640 1 --2- 172.21.15.100:0/1535723843 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2c80159ca0 0x7f2c8014d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:47.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c7effd640 1 -- 172.21.15.100:0/1535723843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c80073580 msgr2=0x7f2c8014f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:47.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c7effd640 1 --2- 172.21.15.100:0/1535723843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c80073580 0x7f2c8014f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:47.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c7effd640 1 -- 172.21.15.100:0/1535723843 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c801f4840 con 0x7f2c80073960 2024-09-15T21:46:47.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c7effd640 1 --2- 172.21.15.100:0/1535723843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c80073960 0x7f2c80157c60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2c700098c0 tx=0x7f2c700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:47.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c67fff640 1 -- 172.21.15.100:0/1535723843 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70037040 con 0x7f2c80073960 2024-09-15T21:46:47.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c67fff640 1 -- 172.21.15.100:0/1535723843 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c70030500 con 0x7f2c80073960 2024-09-15T21:46:47.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c85d66640 1 -- 172.21.15.100:0/1535723843 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c801f13a0 con 0x7f2c80073960 2024-09-15T21:46:47.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.493+0000 7f2c67fff640 1 -- 172.21.15.100:0/1535723843 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c700308d0 con 0x7f2c80073960 2024-09-15T21:46:47.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.494+0000 7f2c85d66640 1 -- 172.21.15.100:0/1535723843 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c801f1960 con 0x7f2c80073960 2024-09-15T21:46:47.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.494+0000 7f2c85d66640 1 -- 172.21.15.100:0/1535723843 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c800c7290 con 0x7f2c80073960 2024-09-15T21:46:47.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.495+0000 7f2c67fff640 1 -- 172.21.15.100:0/1535723843 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2c70044020 con 0x7f2c80073960 2024-09-15T21:46:47.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.495+0000 7f2c67fff640 1 --2- 172.21.15.100:0/1535723843 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2c6c06c2e0 0x7f2c6c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:47.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.496+0000 7f2c7f7fe640 1 --2- 172.21.15.100:0/1535723843 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2c6c06c2e0 0x7f2c6c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:47.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.496+0000 7f2c67fff640 1 -- 172.21.15.100:0/1535723843 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2c700ab570 con 0x7f2c80073960 2024-09-15T21:46:47.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.496+0000 7f2c7f7fe640 1 --2- 172.21.15.100:0/1535723843 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2c6c06c2e0 0x7f2c6c06e7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2c74001780 tx=0x7f2c74009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:47.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.500+0000 7f2c67fff640 1 -- 172.21.15.100:0/1535723843 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2c70079600 con 0x7f2c80073960 2024-09-15T21:46:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.704+0000 7f2c85d66640 1 -- 172.21.15.100:0/1535723843 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2c8014cb40 con 0x7f2c80073960 2024-09-15T21:46:47.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.705+0000 7f2c67fff640 1 -- 172.21.15.100:0/1535723843 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f2c7007d900 con 0x7f2c80073960 2024-09-15T21:46:47.706 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:47.707 INFO:teuthology.orchestra.run.smithi100.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T21:46:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.707+0000 7f2c85d66640 1 -- 172.21.15.100:0/1535723843 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2c6c06c2e0 msgr2=0x7f2c6c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.707+0000 7f2c85d66640 1 --2- 172.21.15.100:0/1535723843 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2c6c06c2e0 0x7f2c6c06e7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2c74001780 tx=0x7f2c74009040 comp rx=0 tx=0).stop 2024-09-15T21:46:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.707+0000 7f2c85d66640 1 -- 172.21.15.100:0/1535723843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c80073960 msgr2=0x7f2c80157c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.707+0000 7f2c85d66640 1 --2- 172.21.15.100:0/1535723843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c80073960 0x7f2c80157c60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2c700098c0 tx=0x7f2c700046c0 comp rx=0 tx=0).stop 2024-09-15T21:46:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.707+0000 7f2c85d66640 1 -- 172.21.15.100:0/1535723843 shutdown_connections 2024-09-15T21:46:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.707+0000 7f2c85d66640 1 --2- 172.21.15.100:0/1535723843 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2c6c06c2e0 0x7f2c6c06e7a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.707+0000 7f2c85d66640 1 --2- 172.21.15.100:0/1535723843 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2c80159ca0 0x7f2c8014d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:47.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.707+0000 7f2c85d66640 1 --2- 172.21.15.100:0/1535723843 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c80073960 0x7f2c80157c60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:47.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.707+0000 7f2c85d66640 1 --2- 172.21.15.100:0/1535723843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c80073580 0x7f2c8014f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:47.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.707+0000 7f2c85d66640 1 -- 172.21.15.100:0/1535723843 >> 172.21.15.100:0/1535723843 conn(0x7f2c800cc890 msgr2=0x7f2c8014b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:47.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.707+0000 7f2c85d66640 1 -- 172.21.15.100:0/1535723843 shutdown_connections 2024-09-15T21:46:47.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.707+0000 7f2c85d66640 1 -- 172.21.15.100:0/1535723843 wait complete. 2024-09-15T21:46:47.720 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-15T21:46:47.720 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T21:46:47.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.817+0000 7f7892313640 1 Processor -- start 2024-09-15T21:46:47.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.817+0000 7f7892313640 1 -- start start 2024-09-15T21:46:47.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.817+0000 7f7892313640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f788c1552b0 0x7f788c073cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:47.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.817+0000 7f7892313640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f788c0cae80 0x7f788c06c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:47.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.817+0000 7f7892313640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f788c06bd80 con 0x7f788c0720c0 2024-09-15T21:46:47.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.817+0000 7f7892313640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f788c15c090 con 0x7f788c0cae80 2024-09-15T21:46:47.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.817+0000 7f7892313640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f788c06c390 con 0x7f788c1552b0 2024-09-15T21:46:47.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.818+0000 7f788b7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f788c0cae80 0x7f788c06c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:47.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.818+0000 7f788b7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f788c0cae80 0x7f788c06c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52604/0 (socket says 172.21.15.100:52604) 2024-09-15T21:46:47.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.818+0000 7f788b7fe640 1 -- 172.21.15.100:0/2019095140 learned_addr learned my addr 172.21.15.100:0/2019095140 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:47.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.818+0000 7f788bfff640 1 --2- 172.21.15.100:0/2019095140 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f788c1552b0 0x7f788c073cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:47.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.818+0000 7f788b7fe640 1 -- 172.21.15.100:0/2019095140 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f788c1552b0 msgr2=0x7f788c073cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:47.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.818+0000 7f788b7fe640 1 --2- 172.21.15.100:0/2019095140 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f788c1552b0 0x7f788c073cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:47.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.818+0000 7f788b7fe640 1 -- 172.21.15.100:0/2019095140 >> v1:172.21.15.100:6789/0 conn(0x7f788c0720c0 legacy=0x7f788c073100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:47.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.818+0000 7f788b7fe640 1 -- 172.21.15.100:0/2019095140 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788c15baf0 con 0x7f788c0cae80 2024-09-15T21:46:47.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.819+0000 7f788b7fe640 1 --2- 172.21.15.100:0/2019095140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f788c0cae80 0x7f788c06c820 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7874009870 tx=0x7f787402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2889a498272fa11 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:47.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.819+0000 7f788affd640 1 -- 172.21.15.100:0/2019095140 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874037070 con 0x7f788c0cae80 2024-09-15T21:46:47.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.819+0000 7f788affd640 1 -- 172.21.15.100:0/2019095140 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f787402ae50 con 0x7f788c0cae80 2024-09-15T21:46:47.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.819+0000 7f788affd640 1 -- 172.21.15.100:0/2019095140 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874032860 con 0x7f788c0cae80 2024-09-15T21:46:47.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.819+0000 7f7892313640 1 -- 172.21.15.100:0/2019095140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f788c0cae80 msgr2=0x7f788c06c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:47.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.819+0000 7f7892313640 1 --2- 172.21.15.100:0/2019095140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f788c0cae80 0x7f788c06c820 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7874009870 tx=0x7f787402a300 comp rx=0 tx=0).stop 2024-09-15T21:46:47.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.819+0000 7f7892313640 1 -- 172.21.15.100:0/2019095140 shutdown_connections 2024-09-15T21:46:47.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.819+0000 7f7892313640 1 --2- 172.21.15.100:0/2019095140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f788c0cae80 0x7f788c06c820 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:47.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.819+0000 7f7892313640 1 --2- 172.21.15.100:0/2019095140 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f788c1552b0 0x7f788c073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:47.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.819+0000 7f7892313640 1 -- 172.21.15.100:0/2019095140 >> 172.21.15.100:0/2019095140 conn(0x7f788c0ca390 msgr2=0x7f788c0ca770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:47.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.819+0000 7f7892313640 1 -- 172.21.15.100:0/2019095140 shutdown_connections 2024-09-15T21:46:47.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.819+0000 7f7892313640 1 -- 172.21.15.100:0/2019095140 wait complete. 2024-09-15T21:46:47.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.820+0000 7f7892313640 1 Processor -- start 2024-09-15T21:46:47.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.821+0000 7f7892313640 1 -- start start 2024-09-15T21:46:47.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.821+0000 7f7892313640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f788c0720c0 0x7f788c14eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:47.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.821+0000 7f7892313640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f788c0cae80 0x7f788c150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:47.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.821+0000 7f7892313640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f788c1552b0 0x7f788c157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:47.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.821+0000 7f7892313640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f788c155690 con 0x7f788c0cae80 2024-09-15T21:46:47.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.821+0000 7f7892313640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f788c06c020 con 0x7f788c0720c0 2024-09-15T21:46:47.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.821+0000 7f7892313640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f788c073810 con 0x7f788c1552b0 2024-09-15T21:46:47.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.821+0000 7f788b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f788c0cae80 0x7f788c150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:47.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.821+0000 7f788b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f788c0cae80 0x7f788c150840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52612/0 (socket says 172.21.15.100:52612) 2024-09-15T21:46:47.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.821+0000 7f788b7fe640 1 -- 172.21.15.100:0/528436678 learned_addr learned my addr 172.21.15.100:0/528436678 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:47.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.822+0000 7f78908b0640 1 --2- 172.21.15.100:0/528436678 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f788c1552b0 0x7f788c157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:47.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.822+0000 7f788bfff640 1 --2- 172.21.15.100:0/528436678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f788c0720c0 0x7f788c14eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:47.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.822+0000 7f788b7fe640 1 -- 172.21.15.100:0/528436678 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f788c1552b0 msgr2=0x7f788c157690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:47.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.822+0000 7f788b7fe640 1 --2- 172.21.15.100:0/528436678 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f788c1552b0 0x7f788c157690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:47.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.822+0000 7f788b7fe640 1 -- 172.21.15.100:0/528436678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f788c0720c0 msgr2=0x7f788c14eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:47.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.822+0000 7f788b7fe640 1 --2- 172.21.15.100:0/528436678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f788c0720c0 0x7f788c14eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:47.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.822+0000 7f788b7fe640 1 -- 172.21.15.100:0/528436678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788c14ca20 con 0x7f788c0cae80 2024-09-15T21:46:47.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.822+0000 7f788b7fe640 1 --2- 172.21.15.100:0/528436678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f788c0cae80 0x7f788c150840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7874000c00 tx=0x7f78740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:47.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.822+0000 7f7888ff9640 1 -- 172.21.15.100:0/528436678 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874041020 con 0x7f788c0cae80 2024-09-15T21:46:47.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.822+0000 7f7888ff9640 1 -- 172.21.15.100:0/528436678 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f787403c670 con 0x7f788c0cae80 2024-09-15T21:46:47.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.822+0000 7f7892313640 1 -- 172.21.15.100:0/528436678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f788c151350 con 0x7f788c0cae80 2024-09-15T21:46:47.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.822+0000 7f7888ff9640 1 -- 172.21.15.100:0/528436678 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874037040 con 0x7f788c0cae80 2024-09-15T21:46:47.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.822+0000 7f7892313640 1 -- 172.21.15.100:0/528436678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f788c15afa0 con 0x7f788c0cae80 2024-09-15T21:46:47.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.823+0000 7f7892313640 1 -- 172.21.15.100:0/528436678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f788c14f050 con 0x7f788c0cae80 2024-09-15T21:46:47.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.823+0000 7f7888ff9640 1 -- 172.21.15.100:0/528436678 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7874044020 con 0x7f788c0cae80 2024-09-15T21:46:47.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.824+0000 7f7888ff9640 1 --2- 172.21.15.100:0/528436678 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f786c06c2b0 0x7f786c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:47.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.824+0000 7f788bfff640 1 --2- 172.21.15.100:0/528436678 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f786c06c2b0 0x7f786c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:47.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.824+0000 7f7888ff9640 1 -- 172.21.15.100:0/528436678 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f78740ab9d0 con 0x7f788c0cae80 2024-09-15T21:46:47.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.824+0000 7f788bfff640 1 --2- 172.21.15.100:0/528436678 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f786c06c2b0 0x7f786c06e770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f787c006fd0 tx=0x7f787c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:47.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:47.828+0000 7f7888ff9640 1 -- 172.21.15.100:0/528436678 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f788c14f050 con 0x7f788c0cae80 2024-09-15T21:46:48.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.028+0000 7f7892313640 1 -- 172.21.15.100:0/528436678 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f788c156f60 con 0x7f788c0cae80 2024-09-15T21:46:48.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.029+0000 7f7888ff9640 1 -- 172.21.15.100:0/528436678 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6954 (secure 0 0 0) 0x7f787404b030 con 0x7f788c0cae80 2024-09-15T21:46:48.031 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:48.031 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":16,"btime":"2024-09-15T21:46:46:560141+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2992633397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2992633397},{"type":"v1","addr":"172.21.15.122:6833","nonce":2992633397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T21:46:22.753977+0000","modified":"2024-09-15T21:46:46.560129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4291,"mds_2":4290,"mds_3":4288,"mds_4":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":4,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.100:6835/1809406468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1809406468},{"type":"v1","addr":"172.21.15.100:6835","nonce":1809406468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4288":{"gid":4288,"name":"f","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.122:6837/371886422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":371886422},{"type":"v1","addr":"172.21.15.122:6837","nonce":371886422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.100:6838/276075809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":276075809},{"type":"v1","addr":"172.21.15.100:6838","nonce":276075809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4291":{"gid":4291,"name":"b","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6836/2947743626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6835","nonce":2947743626},{"type":"v1","addr":"172.21.15.122:6836","nonce":2947743626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/602064326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6837","nonce":602064326},{"type":"v1","addr":"172.21.15.100:6839","nonce":602064326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4291,4290,4288]},"id":1}]} 2024-09-15T21:46:48.031 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 16 2024-09-15T21:46:48.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.031+0000 7f7892313640 1 -- 172.21.15.100:0/528436678 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f786c06c2b0 msgr2=0x7f786c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:48.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.031+0000 7f7892313640 1 --2- 172.21.15.100:0/528436678 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f786c06c2b0 0x7f786c06e770 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f787c006fd0 tx=0x7f787c008040 comp rx=0 tx=0).stop 2024-09-15T21:46:48.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.031+0000 7f7892313640 1 -- 172.21.15.100:0/528436678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f788c0cae80 msgr2=0x7f788c150840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:48.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.031+0000 7f7892313640 1 --2- 172.21.15.100:0/528436678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f788c0cae80 0x7f788c150840 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7874000c00 tx=0x7f78740046c0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.031+0000 7f7892313640 1 -- 172.21.15.100:0/528436678 shutdown_connections 2024-09-15T21:46:48.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.031+0000 7f7892313640 1 --2- 172.21.15.100:0/528436678 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f786c06c2b0 0x7f786c06e770 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.031+0000 7f7892313640 1 --2- 172.21.15.100:0/528436678 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f788c1552b0 0x7f788c157690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.031+0000 7f7892313640 1 --2- 172.21.15.100:0/528436678 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f788c0cae80 0x7f788c150840 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.031+0000 7f7892313640 1 --2- 172.21.15.100:0/528436678 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f788c0720c0 0x7f788c14eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.031+0000 7f7892313640 1 -- 172.21.15.100:0/528436678 >> 172.21.15.100:0/528436678 conn(0x7f788c0ca390 msgr2=0x7f788c0c4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:48.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.031+0000 7f7892313640 1 -- 172.21.15.100:0/528436678 shutdown_connections 2024-09-15T21:46:48.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.031+0000 7f7892313640 1 -- 172.21.15.100:0/528436678 wait complete. 2024-09-15T21:46:48.044 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T21:46:48.143 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.141+0000 7f952b791640 1 Processor -- start 2024-09-15T21:46:48.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.141+0000 7f952b791640 1 -- start start 2024-09-15T21:46:48.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.142+0000 7f952b791640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9524155440 0x7f95240cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:48.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.142+0000 7f952b791640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f952415e7a0 con 0x7f95240cb7d0 2024-09-15T21:46:48.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.142+0000 7f952b791640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f952415ea40 con 0x7f95240cc8f0 2024-09-15T21:46:48.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.142+0000 7f952b791640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f952415b470 con 0x7f9524155440 2024-09-15T21:46:48.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.142+0000 7f952952d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9524155440 0x7f95240cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:48.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.142+0000 7f952952d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9524155440 0x7f95240cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52620/0 (socket says 172.21.15.100:52620) 2024-09-15T21:46:48.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.142+0000 7f952952d640 1 -- 172.21.15.100:0/5617715 learned_addr learned my addr 172.21.15.100:0/5617715 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:48.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.143+0000 7f952952d640 1 -- 172.21.15.100:0/5617715 >> v1:172.21.15.122:6789/0 conn(0x7f95240cc8f0 legacy=0x7f9524165990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:48.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.143+0000 7f952952d640 1 -- 172.21.15.100:0/5617715 >> v1:172.21.15.100:6789/0 conn(0x7f95240cb7d0 legacy=0x7f95240cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:48.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.143+0000 7f952952d640 1 -- 172.21.15.100:0/5617715 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952415ed20 con 0x7f9524155440 2024-09-15T21:46:48.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.143+0000 7f952952d640 1 --2- 172.21.15.100:0/5617715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9524155440 0x7f95240cd420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9514009e50 tx=0x7f951402a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758ab7a3f613c238 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:48.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.143+0000 7f9523fff640 1 -- 172.21.15.100:0/5617715 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514036070 con 0x7f9524155440 2024-09-15T21:46:48.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.143+0000 7f9523fff640 1 -- 172.21.15.100:0/5617715 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9514031600 con 0x7f9524155440 2024-09-15T21:46:48.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.143+0000 7f9523fff640 1 -- 172.21.15.100:0/5617715 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95140319d0 con 0x7f9524155440 2024-09-15T21:46:48.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.144+0000 7f952b791640 1 -- 172.21.15.100:0/5617715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9524155440 msgr2=0x7f95240cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:48.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.144+0000 7f952b791640 1 --2- 172.21.15.100:0/5617715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9524155440 0x7f95240cd420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9514009e50 tx=0x7f951402a760 comp rx=0 tx=0).stop 2024-09-15T21:46:48.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.144+0000 7f952b791640 1 -- 172.21.15.100:0/5617715 shutdown_connections 2024-09-15T21:46:48.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.144+0000 7f952b791640 1 --2- 172.21.15.100:0/5617715 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9524155440 0x7f95240cd420 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.144+0000 7f952b791640 1 -- 172.21.15.100:0/5617715 >> 172.21.15.100:0/5617715 conn(0x7f95240680d0 msgr2=0x7f952406d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:48.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.144+0000 7f952b791640 1 -- 172.21.15.100:0/5617715 shutdown_connections 2024-09-15T21:46:48.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.144+0000 7f952b791640 1 -- 172.21.15.100:0/5617715 wait complete. 2024-09-15T21:46:48.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.145+0000 7f952b791640 1 Processor -- start 2024-09-15T21:46:48.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.145+0000 7f952b791640 1 -- start start 2024-09-15T21:46:48.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.146+0000 7f952b791640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95240cb7d0 0x7f9524151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:48.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.146+0000 7f952b791640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95240cc8f0 0x7f95240cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:48.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.146+0000 7f952b791640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9524155440 0x7f95241f1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:48.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.146+0000 7f952b791640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f952415f2e0 con 0x7f95240cb7d0 2024-09-15T21:46:48.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.146+0000 7f952b791640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9524155820 con 0x7f9524155440 2024-09-15T21:46:48.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.146+0000 7f952b791640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f952415b070 con 0x7f95240cc8f0 2024-09-15T21:46:48.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.146+0000 7f952952d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95240cb7d0 0x7f9524151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:48.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.146+0000 7f952952d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95240cb7d0 0x7f9524151460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52628/0 (socket says 172.21.15.100:52628) 2024-09-15T21:46:48.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.146+0000 7f952952d640 1 -- 172.21.15.100:0/935410453 learned_addr learned my addr 172.21.15.100:0/935410453 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:48.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.146+0000 7f952952d640 1 -- 172.21.15.100:0/935410453 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95240cc8f0 msgr2=0x7f95240cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:48.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.146+0000 7f952952d640 1 --2- 172.21.15.100:0/935410453 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95240cc8f0 0x7f95240cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.146+0000 7f952952d640 1 -- 172.21.15.100:0/935410453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9524155440 msgr2=0x7f95241f1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:48.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.146+0000 7f9529d2e640 1 --2- 172.21.15.100:0/935410453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9524155440 0x7f95241f1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:48.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.146+0000 7f952952d640 1 --2- 172.21.15.100:0/935410453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9524155440 0x7f95241f1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.146+0000 7f952952d640 1 -- 172.21.15.100:0/935410453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95241519a0 con 0x7f95240cb7d0 2024-09-15T21:46:48.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.146+0000 7f9529d2e640 1 --2- 172.21.15.100:0/935410453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9524155440 0x7f95241f1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:48.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.146+0000 7f952952d640 1 --2- 172.21.15.100:0/935410453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95240cb7d0 0x7f9524151460 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9514009e50 tx=0x7f9514004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:48.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.147+0000 7f9521ffb640 1 -- 172.21.15.100:0/935410453 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514036050 con 0x7f95240cb7d0 2024-09-15T21:46:48.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.147+0000 7f9521ffb640 1 -- 172.21.15.100:0/935410453 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9514042070 con 0x7f95240cb7d0 2024-09-15T21:46:48.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.147+0000 7f952b791640 1 -- 172.21.15.100:0/935410453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9524150d60 con 0x7f95240cb7d0 2024-09-15T21:46:48.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.147+0000 7f9521ffb640 1 -- 172.21.15.100:0/935410453 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514009510 con 0x7f95240cb7d0 2024-09-15T21:46:48.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.147+0000 7f952b791640 1 -- 172.21.15.100:0/935410453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95240b5b30 con 0x7f95240cb7d0 2024-09-15T21:46:48.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.148+0000 7f952b791640 1 -- 172.21.15.100:0/935410453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95240cc490 con 0x7f95240cb7d0 2024-09-15T21:46:48.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.148+0000 7f9521ffb640 1 -- 172.21.15.100:0/935410453 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9514007c50 con 0x7f95240cb7d0 2024-09-15T21:46:48.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.149+0000 7f9521ffb640 1 --2- 172.21.15.100:0/935410453 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f951006c2b0 0x7f951006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:48.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.149+0000 7f9521ffb640 1 -- 172.21.15.100:0/935410453 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f95140ac340 con 0x7f95240cb7d0 2024-09-15T21:46:48.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.149+0000 7f9528d2c640 1 --2- 172.21.15.100:0/935410453 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f951006c2b0 0x7f951006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:48.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.149+0000 7f9528d2c640 1 --2- 172.21.15.100:0/935410453 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f951006c2b0 0x7f951006e770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9518000f60 tx=0x7f95180079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:48.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.153+0000 7f9521ffb640 1 -- 172.21.15.100:0/935410453 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f95240cc490 con 0x7f95240cb7d0 2024-09-15T21:46:48.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.353+0000 7f952b791640 1 -- 172.21.15.100:0/935410453 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f952406d3e0 con 0x7f95240cb7d0 2024-09-15T21:46:48.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.354+0000 7f9521ffb640 1 -- 172.21.15.100:0/935410453 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6954 (secure 0 0 0) 0x7f951407aab0 con 0x7f95240cb7d0 2024-09-15T21:46:48.356 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:48.356 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":16,"btime":"2024-09-15T21:46:46:560141+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2992633397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2992633397},{"type":"v1","addr":"172.21.15.122:6833","nonce":2992633397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T21:46:22.753977+0000","modified":"2024-09-15T21:46:46.560129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4291,"mds_2":4290,"mds_3":4288,"mds_4":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":4,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.100:6835/1809406468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1809406468},{"type":"v1","addr":"172.21.15.100:6835","nonce":1809406468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4288":{"gid":4288,"name":"f","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.122:6837/371886422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":371886422},{"type":"v1","addr":"172.21.15.122:6837","nonce":371886422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.100:6838/276075809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":276075809},{"type":"v1","addr":"172.21.15.100:6838","nonce":276075809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4291":{"gid":4291,"name":"b","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6836/2947743626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6835","nonce":2947743626},{"type":"v1","addr":"172.21.15.122:6836","nonce":2947743626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/602064326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6837","nonce":602064326},{"type":"v1","addr":"172.21.15.100:6839","nonce":602064326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4291,4290,4288]},"id":1}]} 2024-09-15T21:46:48.356 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 16 2024-09-15T21:46:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.355+0000 7f952b791640 1 -- 172.21.15.100:0/935410453 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f951006c2b0 msgr2=0x7f951006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.355+0000 7f952b791640 1 --2- 172.21.15.100:0/935410453 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f951006c2b0 0x7f951006e770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9518000f60 tx=0x7f95180079d0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.355+0000 7f952b791640 1 -- 172.21.15.100:0/935410453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95240cb7d0 msgr2=0x7f9524151460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.355+0000 7f952b791640 1 --2- 172.21.15.100:0/935410453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95240cb7d0 0x7f9524151460 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9514009e50 tx=0x7f9514004a40 comp rx=0 tx=0).stop 2024-09-15T21:46:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.355+0000 7f952b791640 1 -- 172.21.15.100:0/935410453 shutdown_connections 2024-09-15T21:46:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.356+0000 7f952b791640 1 --2- 172.21.15.100:0/935410453 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f951006c2b0 0x7f951006e770 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.356+0000 7f952b791640 1 --2- 172.21.15.100:0/935410453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9524155440 0x7f95241f1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.356+0000 7f952b791640 1 --2- 172.21.15.100:0/935410453 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95240cc8f0 0x7f95240cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.356+0000 7f952b791640 1 --2- 172.21.15.100:0/935410453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95240cb7d0 0x7f9524151460 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.356+0000 7f952b791640 1 -- 172.21.15.100:0/935410453 >> 172.21.15.100:0/935410453 conn(0x7f95240680d0 msgr2=0x7f9524158cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:48.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.356+0000 7f952b791640 1 -- 172.21.15.100:0/935410453 shutdown_connections 2024-09-15T21:46:48.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.356+0000 7f952b791640 1 -- 172.21.15.100:0/935410453 wait complete. 2024-09-15T21:46:48.369 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:46:48.395 INFO:tasks.ceph.mds.e.smithi100.stderr:2024-09-15T21:46:48.393+0000 7fa09cc49640 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-09-15T21:46:48.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.465+0000 7fb817e2c640 1 Processor -- start 2024-09-15T21:46:48.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.465+0000 7fb817e2c640 1 -- start start 2024-09-15T21:46:48.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.466+0000 7fb817e2c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb8101531b0 0x7fb8100c6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:48.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.466+0000 7fb817e2c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb810072320 0x7fb8100711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.466+0000 7fb817e2c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb810071710 0x7fb81015c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.466+0000 7fb817e2c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fb810153590 con 0x7fb810072320 2024-09-15T21:46:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.466+0000 7fb817e2c640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fb8100c6c90 con 0x7fb8101531b0 2024-09-15T21:46:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.466+0000 7fb817e2c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb8100c6f50 con 0x7fb810071710 2024-09-15T21:46:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.466+0000 7fb8153c7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb810072320 0x7fb8100711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.466+0000 7fb8153c7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb810072320 0x7fb8100711d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52634/0 (socket says 172.21.15.100:52634) 2024-09-15T21:46:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.466+0000 7fb8153c7640 1 -- 172.21.15.100:0/147066992 learned_addr learned my addr 172.21.15.100:0/147066992 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.466+0000 7fb815bc8640 1 --2- 172.21.15.100:0/147066992 >> v2:172.21.15.122:3300/0 conn(0x7fb8101531b0 0x7fb8100c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.466+0000 7fb8163c9640 1 --2- 172.21.15.100:0/147066992 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb810071710 0x7fb81015c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.466+0000 7fb8153c7640 1 -- 172.21.15.100:0/147066992 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb810071710 msgr2=0x7fb81015c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.466+0000 7fb8153c7640 1 --2- 172.21.15.100:0/147066992 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb810071710 0x7fb81015c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.466+0000 7fb8153c7640 1 -- 172.21.15.100:0/147066992 >> v2:172.21.15.122:3300/0 conn(0x7fb8101531b0 msgr2=0x7fb8100c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.466+0000 7fb8153c7640 1 --2- 172.21.15.100:0/147066992 >> v2:172.21.15.122:3300/0 conn(0x7fb8101531b0 0x7fb8100c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.466+0000 7fb8153c7640 1 -- 172.21.15.100:0/147066992 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb81013e430 con 0x7fb810072320 2024-09-15T21:46:48.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.466+0000 7fb815bc8640 1 --2- 172.21.15.100:0/147066992 >> v2:172.21.15.122:3300/0 conn(0x7fb8101531b0 0x7fb8100c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:48.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.467+0000 7fb8153c7640 1 --2- 172.21.15.100:0/147066992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810072320 0x7fb8100711d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb81006fd80 tx=0x7fb80002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69289dfeae989b2d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:48.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.467+0000 7fb814bc6640 1 -- 172.21.15.100:0/147066992 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800037070 con 0x7fb810072320 2024-09-15T21:46:48.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.467+0000 7fb814bc6640 1 -- 172.21.15.100:0/147066992 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb800032440 con 0x7fb810072320 2024-09-15T21:46:48.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.467+0000 7fb814bc6640 1 -- 172.21.15.100:0/147066992 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800032810 con 0x7fb810072320 2024-09-15T21:46:48.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.468+0000 7fb817e2c640 1 -- 172.21.15.100:0/147066992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810072320 msgr2=0x7fb8100711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:48.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.468+0000 7fb817e2c640 1 --2- 172.21.15.100:0/147066992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810072320 0x7fb8100711d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb81006fd80 tx=0x7fb80002a270 comp rx=0 tx=0).stop 2024-09-15T21:46:48.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.468+0000 7fb817e2c640 1 -- 172.21.15.100:0/147066992 shutdown_connections 2024-09-15T21:46:48.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.468+0000 7fb817e2c640 1 --2- 172.21.15.100:0/147066992 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb810071710 0x7fb81015c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.468+0000 7fb817e2c640 1 --2- 172.21.15.100:0/147066992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810072320 0x7fb8100711d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.475 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.468+0000 7fb817e2c640 1 --2- 172.21.15.100:0/147066992 >> v2:172.21.15.122:3300/0 conn(0x7fb8101531b0 0x7fb8100c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.468+0000 7fb817e2c640 1 -- 172.21.15.100:0/147066992 >> 172.21.15.100:0/147066992 conn(0x7fb8100ccbd0 msgr2=0x7fb8100ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:48.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.468+0000 7fb817e2c640 1 -- 172.21.15.100:0/147066992 shutdown_connections 2024-09-15T21:46:48.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.468+0000 7fb817e2c640 1 -- 172.21.15.100:0/147066992 wait complete. 2024-09-15T21:46:48.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.469+0000 7fb817e2c640 1 Processor -- start 2024-09-15T21:46:48.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.469+0000 7fb817e2c640 1 -- start start 2024-09-15T21:46:48.476 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb817e2c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb810071710 0x7fb8101e4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:48.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb817e2c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810072320 0x7fb8101521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:48.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb817e2c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb8101531b0 0x7fb810152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:48.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb817e2c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb81013ecd0 con 0x7fb810072320 2024-09-15T21:46:48.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb817e2c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb81013e9f0 con 0x7fb810071710 2024-09-15T21:46:48.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb817e2c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb810072770 con 0x7fb8101531b0 2024-09-15T21:46:48.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb8153c7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810072320 0x7fb8101521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:48.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb8153c7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810072320 0x7fb8101521d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52646/0 (socket says 172.21.15.100:52646) 2024-09-15T21:46:48.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb8153c7640 1 -- 172.21.15.100:0/2354474806 learned_addr learned my addr 172.21.15.100:0/2354474806 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:48.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb815bc8640 1 --2- 172.21.15.100:0/2354474806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb810071710 0x7fb8101e4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:48.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb8163c9640 1 --2- 172.21.15.100:0/2354474806 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb8101531b0 0x7fb810152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:48.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb8153c7640 1 -- 172.21.15.100:0/2354474806 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb8101531b0 msgr2=0x7fb810152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:48.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb8153c7640 1 --2- 172.21.15.100:0/2354474806 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb8101531b0 0x7fb810152710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb8153c7640 1 -- 172.21.15.100:0/2354474806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb810071710 msgr2=0x7fb8101e4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:48.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb8153c7640 1 --2- 172.21.15.100:0/2354474806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb810071710 0x7fb8101e4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb8153c7640 1 -- 172.21.15.100:0/2354474806 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8101e4050 con 0x7fb810072320 2024-09-15T21:46:48.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb815bc8640 1 --2- 172.21.15.100:0/2354474806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb810071710 0x7fb8101e4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:48.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb8153c7640 1 --2- 172.21.15.100:0/2354474806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810072320 0x7fb8101521d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb800000c00 tx=0x7fb8000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:48.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb8067fc640 1 -- 172.21.15.100:0/2354474806 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800037040 con 0x7fb810072320 2024-09-15T21:46:48.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb8067fc640 1 -- 172.21.15.100:0/2354474806 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb800042070 con 0x7fb810072320 2024-09-15T21:46:48.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb817e2c640 1 -- 172.21.15.100:0/2354474806 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb810140620 con 0x7fb810072320 2024-09-15T21:46:48.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb8067fc640 1 -- 172.21.15.100:0/2354474806 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800030bb0 con 0x7fb810072320 2024-09-15T21:46:48.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.470+0000 7fb817e2c640 1 -- 172.21.15.100:0/2354474806 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb81013f8c0 con 0x7fb810072320 2024-09-15T21:46:48.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.471+0000 7fb817e2c640 1 -- 172.21.15.100:0/2354474806 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb81013e060 con 0x7fb810072320 2024-09-15T21:46:48.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.472+0000 7fb8067fc640 1 -- 172.21.15.100:0/2354474806 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb800047460 con 0x7fb810072320 2024-09-15T21:46:48.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.472+0000 7fb8067fc640 1 --2- 172.21.15.100:0/2354474806 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb7f406c2b0 0x7fb7f406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:48.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.473+0000 7fb815bc8640 1 --2- 172.21.15.100:0/2354474806 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb7f406c2b0 0x7fb7f406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:48.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.473+0000 7fb8067fc640 1 -- 172.21.15.100:0/2354474806 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb8000ac0b0 con 0x7fb810072320 2024-09-15T21:46:48.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.473+0000 7fb815bc8640 1 --2- 172.21.15.100:0/2354474806 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb7f406c2b0 0x7fb7f406e770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00d430 tx=0x7fb80c007530 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:48.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.476+0000 7fb8067fc640 1 -- 172.21.15.100:0/2354474806 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb800045360 con 0x7fb810072320 2024-09-15T21:46:48.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.645+0000 7fb817e2c640 1 -- 172.21.15.100:0/2354474806 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8100cab80 con 0x7fb810072320 2024-09-15T21:46:48.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.646+0000 7fb8067fc640 1 -- 172.21.15.100:0/2354474806 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16737 (secure 0 0 0) 0x7fb80007a790 con 0x7fb810072320 2024-09-15T21:46:48.649 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:48.649 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":22,"fsid":"30e2fff4-2171-4531-8806-0e03b63f494f","created":"2024-09-15T21:45:59.746897+0000","modified":"2024-09-15T21:46:24.409383+0000","last_up_change":"2024-09-15T21:46:13.224709+0000","last_in_change":"2024-09-15T21:46:01.812129+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:46:11.837603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T21:46:20.577982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T21:46:21.729231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5887900b-522d-476f-9704-cc2c5ee87a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6819","nonce":3920435546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6821","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6822","nonce":3920435546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6829","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6831","nonce":3920435546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6825","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6827","nonce":3920435546}]},"public_addr":"172.21.15.100:6819/3920435546","cluster_addr":"172.21.15.100:6822/3920435546","heartbeat_back_addr":"172.21.15.100:6831/3920435546","heartbeat_front_addr":"172.21.15.100:6827/3920435546","state":["exists","up"]},{"osd":1,"uuid":"adc17993-5c50-4251-85e1-4c7ac62e25e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6801","nonce":3802679972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6803","nonce":3802679972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6807","nonce":3802679972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6805","nonce":3802679972}]},"public_addr":"172.21.15.100:6801/3802679972","cluster_addr":"172.21.15.100:6803/3802679972","heartbeat_back_addr":"172.21.15.100:6807/3802679972","heartbeat_front_addr":"172.21.15.100:6805/3802679972","state":["exists","up"]},{"osd":2,"uuid":"8de3b7d5-1980-4a3b-a934-264f867c7c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6823","nonce":2316787974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6826","nonce":2316787974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6833","nonce":2316787974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6830","nonce":2316787974}]},"public_addr":"172.21.15.100:6823/2316787974","cluster_addr":"172.21.15.100:6826/2316787974","heartbeat_back_addr":"172.21.15.100:6833/2316787974","heartbeat_front_addr":"172.21.15.100:6830/2316787974","state":["exists","up"]},{"osd":3,"uuid":"93dbcd1c-6a6f-49cf-bc94-e64b883de5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6811","nonce":1287562741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6813","nonce":1287562741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6817","nonce":1287562741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6815","nonce":1287562741}]},"public_addr":"172.21.15.100:6811/1287562741","cluster_addr":"172.21.15.100:6813/1287562741","heartbeat_back_addr":"172.21.15.100:6817/1287562741","heartbeat_front_addr":"172.21.15.100:6815/1287562741","state":["exists","up"]},{"osd":4,"uuid":"7caee2a0-bab4-45a9-8109-724948c863c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6801","nonce":4045445512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6803","nonce":4045445512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6807","nonce":4045445512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6805","nonce":4045445512}]},"public_addr":"172.21.15.122:6801/4045445512","cluster_addr":"172.21.15.122:6803/4045445512","heartbeat_back_addr":"172.21.15.122:6807/4045445512","heartbeat_front_addr":"172.21.15.122:6805/4045445512","state":["exists","up"]},{"osd":5,"uuid":"6e64070a-671a-444b-a061-b3051a29b121","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6825","nonce":2400668532}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6827","nonce":2400668532}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6831","nonce":2400668532}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2400668532},{"type":"v1","addr":"172.21.15.122:6829","nonce":2400668532}]},"public_addr":"172.21.15.122:6825/2400668532","cluster_addr":"172.21.15.122:6827/2400668532","heartbeat_back_addr":"172.21.15.122:6831/2400668532","heartbeat_front_addr":"172.21.15.122:6829/2400668532","state":["exists","up"]},{"osd":6,"uuid":"47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6809","nonce":1437167311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6811","nonce":1437167311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6815","nonce":1437167311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6813","nonce":1437167311}]},"public_addr":"172.21.15.122:6809/1437167311","cluster_addr":"172.21.15.122:6811/1437167311","heartbeat_back_addr":"172.21.15.122:6815/1437167311","heartbeat_front_addr":"172.21.15.122:6813/1437167311","state":["exists","up"]},{"osd":7,"uuid":"9a916b6d-af82-486f-af40-4a7291c900fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6817","nonce":2431377162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6819","nonce":2431377162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6823","nonce":2431377162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6821","nonce":2431377162}]},"public_addr":"172.21.15.122:6817/2431377162","cluster_addr":"172.21.15.122:6819/2431377162","heartbeat_back_addr":"172.21.15.122:6823/2431377162","heartbeat_front_addr":"172.21.15.122:6821/2431377162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.168771+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.886614+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.154129+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.570531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.926773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.146320+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.409137+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.373359+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:46:48.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.648+0000 7fb817e2c640 1 -- 172.21.15.100:0/2354474806 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb7f406c2b0 msgr2=0x7fb7f406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:48.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.648+0000 7fb817e2c640 1 --2- 172.21.15.100:0/2354474806 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb7f406c2b0 0x7fb7f406e770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb80c00d430 tx=0x7fb80c007530 comp rx=0 tx=0).stop 2024-09-15T21:46:48.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.648+0000 7fb817e2c640 1 -- 172.21.15.100:0/2354474806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810072320 msgr2=0x7fb8101521d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:48.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.648+0000 7fb817e2c640 1 --2- 172.21.15.100:0/2354474806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810072320 0x7fb8101521d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb800000c00 tx=0x7fb8000046c0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.650 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.648+0000 7fb817e2c640 1 -- 172.21.15.100:0/2354474806 shutdown_connections 2024-09-15T21:46:48.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.648+0000 7fb817e2c640 1 --2- 172.21.15.100:0/2354474806 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb7f406c2b0 0x7fb7f406e770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.648+0000 7fb817e2c640 1 --2- 172.21.15.100:0/2354474806 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb8101531b0 0x7fb810152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.648+0000 7fb817e2c640 1 --2- 172.21.15.100:0/2354474806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb810072320 0x7fb8101521d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.648+0000 7fb817e2c640 1 --2- 172.21.15.100:0/2354474806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb810071710 0x7fb8101e4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.648+0000 7fb817e2c640 1 -- 172.21.15.100:0/2354474806 >> 172.21.15.100:0/2354474806 conn(0x7fb8100ccbd0 msgr2=0x7fb8100c9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:48.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.648+0000 7fb817e2c640 1 -- 172.21.15.100:0/2354474806 shutdown_connections 2024-09-15T21:46:48.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.648+0000 7fb817e2c640 1 -- 172.21.15.100:0/2354474806 wait complete. 2024-09-15T21:46:48.662 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T21:46:48.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.798+0000 7f9c954c3640 1 Processor -- start 2024-09-15T21:46:48.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.798+0000 7f9c954c3640 1 -- start start 2024-09-15T21:46:48.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.799+0000 7f9c954c3640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9c900cb7d0 0x7f9c900cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:48.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.799+0000 7f9c954c3640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9c9006d0e0 0x7f9c900cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:48.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.799+0000 7f9c954c3640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c9015e7a0 con 0x7f9c90155440 2024-09-15T21:46:48.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.799+0000 7f9c954c3640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f9c9015b470 con 0x7f9c9006d0e0 2024-09-15T21:46:48.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.799+0000 7f9c954c3640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9c90155820 con 0x7f9c900cb7d0 2024-09-15T21:46:48.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.799+0000 7f9c8f7fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9c9006d0e0 0x7f9c900cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:48.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.799+0000 7f9c867fc640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9c900cb7d0 0x7f9c900cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:48.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.799+0000 7f9c8f7fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9c9006d0e0 0x7f9c900cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:36774/0 (socket says 172.21.15.100:36774) 2024-09-15T21:46:48.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.799+0000 7f9c8f7fe640 1 -- 172.21.15.100:0/1594249463 learned_addr learned my addr 172.21.15.100:0/1594249463 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:48.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.799+0000 7f9c8effd640 1 --1- 172.21.15.100:0/1594249463 >> v1:172.21.15.122:6789/0 conn(0x7f9c90155440 0x7f9c900cd420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.100:43360/0 (socket says 172.21.15.100:43360) 2024-09-15T21:46:48.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.799+0000 7f9c8f7fe640 1 -- 172.21.15.100:0/1594249463 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9c900cb7d0 msgr2=0x7f9c900cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:48.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.799+0000 7f9c8f7fe640 1 --2- 172.21.15.100:0/1594249463 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9c900cb7d0 0x7f9c900cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.799+0000 7f9c8f7fe640 1 -- 172.21.15.100:0/1594249463 >> v1:172.21.15.122:6789/0 conn(0x7f9c90155440 legacy=0x7f9c900cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:48.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.799+0000 7f9c8f7fe640 1 -- 172.21.15.100:0/1594249463 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9015ea40 con 0x7f9c9006d0e0 2024-09-15T21:46:48.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.799+0000 7f9c8e7fc640 1 -- 172.21.15.100:0/1594249463 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2232038106 0 0) 0x7f9c9015e7a0 con 0x7f9c90155440 2024-09-15T21:46:48.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.800+0000 7f9c8f7fe640 1 --2- 172.21.15.100:0/1594249463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c9006d0e0 0x7f9c900cc720 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009e50 tx=0x7f9c8802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58a31d3017176164 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:48.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.800+0000 7f9c8e7fc640 1 -- 172.21.15.100:0/1594249463 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88036070 con 0x7f9c9006d0e0 2024-09-15T21:46:48.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.800+0000 7f9c8e7fc640 1 -- 172.21.15.100:0/1594249463 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c88031600 con 0x7f9c9006d0e0 2024-09-15T21:46:48.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.800+0000 7f9c8e7fc640 1 -- 172.21.15.100:0/1594249463 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c880319d0 con 0x7f9c9006d0e0 2024-09-15T21:46:48.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.801+0000 7f9c954c3640 1 -- 172.21.15.100:0/1594249463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c9006d0e0 msgr2=0x7f9c900cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:48.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.801+0000 7f9c954c3640 1 --2- 172.21.15.100:0/1594249463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c9006d0e0 0x7f9c900cc720 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009e50 tx=0x7f9c8802a680 comp rx=0 tx=0).stop 2024-09-15T21:46:48.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.801+0000 7f9c954c3640 1 -- 172.21.15.100:0/1594249463 shutdown_connections 2024-09-15T21:46:48.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.801+0000 7f9c954c3640 1 --2- 172.21.15.100:0/1594249463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c9006d0e0 0x7f9c900cc720 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.801+0000 7f9c954c3640 1 --2- 172.21.15.100:0/1594249463 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9c900cb7d0 0x7f9c900cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.801+0000 7f9c954c3640 1 -- 172.21.15.100:0/1594249463 >> 172.21.15.100:0/1594249463 conn(0x7f9c900680d0 msgr2=0x7f9c9006d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:48.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.801+0000 7f9c954c3640 1 -- 172.21.15.100:0/1594249463 shutdown_connections 2024-09-15T21:46:48.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.801+0000 7f9c954c3640 1 -- 172.21.15.100:0/1594249463 wait complete. 2024-09-15T21:46:48.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.802+0000 7f9c954c3640 1 Processor -- start 2024-09-15T21:46:48.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.803+0000 7f9c954c3640 1 -- start start 2024-09-15T21:46:48.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.803+0000 7f9c954c3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c9006d0e0 0x7f9c901e4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:48.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.803+0000 7f9c954c3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c900cb7d0 0x7f9c900b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:48.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.803+0000 7f9c954c3640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9c90155440 0x7f9c90154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:48.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.803+0000 7f9c954c3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9015f2e0 con 0x7f9c900cb7d0 2024-09-15T21:46:48.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.803+0000 7f9c954c3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9c9015f000 con 0x7f9c9006d0e0 2024-09-15T21:46:48.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.803+0000 7f9c954c3640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9c9015b070 con 0x7f9c90155440 2024-09-15T21:46:48.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.803+0000 7f9c867fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c900cb7d0 0x7f9c900b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:48.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.803+0000 7f9c867fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c900cb7d0 0x7f9c900b52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52658/0 (socket says 172.21.15.100:52658) 2024-09-15T21:46:48.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.803+0000 7f9c867fc640 1 -- 172.21.15.100:0/3469779045 learned_addr learned my addr 172.21.15.100:0/3469779045 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:48.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.803+0000 7f9c8effd640 1 --2- 172.21.15.100:0/3469779045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c9006d0e0 0x7f9c901e4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:48.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.803+0000 7f9c8f7fe640 1 --2- 172.21.15.100:0/3469779045 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9c90155440 0x7f9c90154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:48.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.804+0000 7f9c867fc640 1 -- 172.21.15.100:0/3469779045 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9c90155440 msgr2=0x7f9c90154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:48.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.804+0000 7f9c867fc640 1 --2- 172.21.15.100:0/3469779045 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9c90155440 0x7f9c90154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.804+0000 7f9c867fc640 1 -- 172.21.15.100:0/3469779045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c9006d0e0 msgr2=0x7f9c901e4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:48.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.804+0000 7f9c867fc640 1 --2- 172.21.15.100:0/3469779045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c9006d0e0 0x7f9c901e4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:48.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.804+0000 7f9c867fc640 1 -- 172.21.15.100:0/3469779045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9014a750 con 0x7f9c900cb7d0 2024-09-15T21:46:48.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.804+0000 7f9c867fc640 1 --2- 172.21.15.100:0/3469779045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c900cb7d0 0x7f9c900b52d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800e8a0 tx=0x7f9c7800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:48.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.804+0000 7f9c87fff640 1 -- 172.21.15.100:0/3469779045 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c78017070 con 0x7f9c900cb7d0 2024-09-15T21:46:48.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.804+0000 7f9c87fff640 1 -- 172.21.15.100:0/3469779045 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c7800c8e0 con 0x7f9c900cb7d0 2024-09-15T21:46:48.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.804+0000 7f9c87fff640 1 -- 172.21.15.100:0/3469779045 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7800ccd0 con 0x7f9c900cb7d0 2024-09-15T21:46:48.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.804+0000 7f9c954c3640 1 -- 172.21.15.100:0/3469779045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c901f19d0 con 0x7f9c900cb7d0 2024-09-15T21:46:48.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.804+0000 7f9c954c3640 1 -- 172.21.15.100:0/3469779045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c901489d0 con 0x7f9c900cb7d0 2024-09-15T21:46:48.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.805+0000 7f9c954c3640 1 -- 172.21.15.100:0/3469779045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c900c71c0 con 0x7f9c900cb7d0 2024-09-15T21:46:48.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.805+0000 7f9c87fff640 1 -- 172.21.15.100:0/3469779045 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9c7801a070 con 0x7f9c900cb7d0 2024-09-15T21:46:48.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.806+0000 7f9c87fff640 1 --2- 172.21.15.100:0/3469779045 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9c7006c2b0 0x7f9c7006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:48.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.806+0000 7f9c87fff640 1 -- 172.21.15.100:0/3469779045 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9c7808b640 con 0x7f9c900cb7d0 2024-09-15T21:46:48.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.806+0000 7f9c8effd640 1 --2- 172.21.15.100:0/3469779045 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9c7006c2b0 0x7f9c7006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:48.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.806+0000 7f9c8effd640 1 --2- 172.21.15.100:0/3469779045 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9c7006c2b0 0x7f9c7006e770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9c80000f60 tx=0x7f9c80007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:48.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:48.810+0000 7f9c87fff640 1 -- 172.21.15.100:0/3469779045 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9c900c71c0 con 0x7f9c900cb7d0 2024-09-15T21:46:49.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:49.009+0000 7f9c954c3640 1 -- 172.21.15.100:0/3469779045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c900b3940 con 0x7f9c900cb7d0 2024-09-15T21:46:49.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:49.009+0000 7f9c87fff640 1 -- 172.21.15.100:0/3469779045 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6954 (secure 0 0 0) 0x7f9c780596d0 con 0x7f9c900cb7d0 2024-09-15T21:46:49.012 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:49.012 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":16,"btime":"2024-09-15T21:46:46:560141+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2992633397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2992633397},{"type":"v1","addr":"172.21.15.122:6833","nonce":2992633397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T21:46:22.753977+0000","modified":"2024-09-15T21:46:46.560129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4291,"mds_2":4290,"mds_3":4288,"mds_4":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":4,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.100:6835/1809406468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1809406468},{"type":"v1","addr":"172.21.15.100:6835","nonce":1809406468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4288":{"gid":4288,"name":"f","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.122:6837/371886422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":371886422},{"type":"v1","addr":"172.21.15.122:6837","nonce":371886422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.100:6838/276075809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":276075809},{"type":"v1","addr":"172.21.15.100:6838","nonce":276075809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4291":{"gid":4291,"name":"b","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6836/2947743626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6835","nonce":2947743626},{"type":"v1","addr":"172.21.15.122:6836","nonce":2947743626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/602064326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6837","nonce":602064326},{"type":"v1","addr":"172.21.15.100:6839","nonce":602064326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4291,4290,4288]},"id":1}]} 2024-09-15T21:46:49.012 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 16 2024-09-15T21:46:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:49.012+0000 7f9c954c3640 1 -- 172.21.15.100:0/3469779045 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9c7006c2b0 msgr2=0x7f9c7006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:49.012+0000 7f9c954c3640 1 --2- 172.21.15.100:0/3469779045 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9c7006c2b0 0x7f9c7006e770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9c80000f60 tx=0x7f9c80007d50 comp rx=0 tx=0).stop 2024-09-15T21:46:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:49.012+0000 7f9c954c3640 1 -- 172.21.15.100:0/3469779045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c900cb7d0 msgr2=0x7f9c900b52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:49.012+0000 7f9c954c3640 1 --2- 172.21.15.100:0/3469779045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c900cb7d0 0x7f9c900b52d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800e8a0 tx=0x7f9c7800ed60 comp rx=0 tx=0).stop 2024-09-15T21:46:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:49.012+0000 7f9c954c3640 1 -- 172.21.15.100:0/3469779045 shutdown_connections 2024-09-15T21:46:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:49.012+0000 7f9c954c3640 1 --2- 172.21.15.100:0/3469779045 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9c7006c2b0 0x7f9c7006e770 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:49.012+0000 7f9c954c3640 1 --2- 172.21.15.100:0/3469779045 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9c90155440 0x7f9c90154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:49.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:49.012+0000 7f9c954c3640 1 --2- 172.21.15.100:0/3469779045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c900cb7d0 0x7f9c900b52d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:49.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:49.012+0000 7f9c954c3640 1 --2- 172.21.15.100:0/3469779045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c9006d0e0 0x7f9c901e4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:49.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:49.012+0000 7f9c954c3640 1 -- 172.21.15.100:0/3469779045 >> 172.21.15.100:0/3469779045 conn(0x7f9c900680d0 msgr2=0x7f9c9006d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:49.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:49.012+0000 7f9c954c3640 1 -- 172.21.15.100:0/3469779045 shutdown_connections 2024-09-15T21:46:49.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:49.012+0000 7f9c954c3640 1 -- 172.21.15.100:0/3469779045 wait complete. 2024-09-15T21:46:49.628 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-15T21:46:22.753977+0000', 'modified': '2024-09-15T21:46:46.560129+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4293, 'mds_1': 4291, 'mds_2': 4290, 'mds_3': 4288, 'mds_4': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'c', 'rank': 4, 'incarnation': 16, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.100:6835/1809406468', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 1809406468}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 1809406468}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4288': {'gid': 4288, 'name': 'f', 'rank': 3, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.122:6837/371886422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6834', 'nonce': 371886422}, {'type': 'v1', 'addr': '172.21.15.122:6837', 'nonce': 371886422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4290': {'gid': 4290, 'name': 'a', 'rank': 2, 'incarnation': 12, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.100:6838/276075809', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6836', 'nonce': 276075809}, {'type': 'v1', 'addr': '172.21.15.100:6838', 'nonce': 276075809}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4291': {'gid': 4291, 'name': 'b', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.122:6836/2947743626', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6835', 'nonce': 2947743626}, {'type': 'v1', 'addr': '172.21.15.122:6836', 'nonce': 2947743626}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4293': {'gid': 4293, 'name': 'e', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.100:6839/602064326', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6837', 'nonce': 602064326}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 602064326}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4293, 'qdb_cluster': [4293, 4291, 4290, 4288]} 2024-09-15T21:46:49.629 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4287:up:creating 2024-09-15T21:46:50.630 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T21:46:50.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.737+0000 7fbc38cf8640 1 Processor -- start 2024-09-15T21:46:50.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.738+0000 7fbc38cf8640 1 -- start start 2024-09-15T21:46:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.738+0000 7fbc38cf8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc340c92d0 0x7fbc340c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.739+0000 7fbc38cf8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbc341522e0 0x7fbc340723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.739+0000 7fbc38cf8640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc34157ef0 con 0x7fbc340c96b0 2024-09-15T21:46:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.739+0000 7fbc38cf8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fbc34072900 con 0x7fbc341522e0 2024-09-15T21:46:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.739+0000 7fbc38cf8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbc340c4980 con 0x7fbc340c92d0 2024-09-15T21:46:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.739+0000 7fbc32d9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbc341522e0 0x7fbc340723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.739+0000 7fbc32d9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbc341522e0 0x7fbc340723c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52668/0 (socket says 172.21.15.100:52668) 2024-09-15T21:46:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.739+0000 7fbc32d9d640 1 -- 172.21.15.100:0/471463005 learned_addr learned my addr 172.21.15.100:0/471463005 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.739+0000 7fbc3259c640 1 --2- 172.21.15.100:0/471463005 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc340c92d0 0x7fbc340c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.739+0000 7fbc32d9d640 1 -- 172.21.15.100:0/471463005 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc340c92d0 msgr2=0x7fbc340c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.739+0000 7fbc32d9d640 1 --2- 172.21.15.100:0/471463005 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc340c92d0 0x7fbc340c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:50.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.739+0000 7fbc32d9d640 1 -- 172.21.15.100:0/471463005 >> v1:172.21.15.122:6789/0 conn(0x7fbc340c96b0 legacy=0x7fbc34151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:50.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.739+0000 7fbc32d9d640 1 -- 172.21.15.100:0/471463005 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc341581b0 con 0x7fbc341522e0 2024-09-15T21:46:50.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.740+0000 7fbc32d9d640 1 --2- 172.21.15.100:0/471463005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc341522e0 0x7fbc340723c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009e50 tx=0x7fbc2c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90497ad6cf1be6b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:50.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.740+0000 7fbc3159a640 1 -- 172.21.15.100:0/471463005 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c036070 con 0x7fbc341522e0 2024-09-15T21:46:50.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.740+0000 7fbc3159a640 1 -- 172.21.15.100:0/471463005 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc2c031600 con 0x7fbc341522e0 2024-09-15T21:46:50.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.740+0000 7fbc3159a640 1 -- 172.21.15.100:0/471463005 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c0319d0 con 0x7fbc341522e0 2024-09-15T21:46:50.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.740+0000 7fbc38cf8640 1 -- 172.21.15.100:0/471463005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc341522e0 msgr2=0x7fbc340723c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:50.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.740+0000 7fbc38cf8640 1 --2- 172.21.15.100:0/471463005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc341522e0 0x7fbc340723c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009e50 tx=0x7fbc2c02a680 comp rx=0 tx=0).stop 2024-09-15T21:46:50.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.740+0000 7fbc38cf8640 1 -- 172.21.15.100:0/471463005 shutdown_connections 2024-09-15T21:46:50.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.740+0000 7fbc38cf8640 1 --2- 172.21.15.100:0/471463005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc341522e0 0x7fbc340723c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:50.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.741+0000 7fbc38cf8640 1 --2- 172.21.15.100:0/471463005 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc340c92d0 0x7fbc340c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:50.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.741+0000 7fbc38cf8640 1 -- 172.21.15.100:0/471463005 >> 172.21.15.100:0/471463005 conn(0x7fbc34067dd0 msgr2=0x7fbc3406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:50.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.741+0000 7fbc38cf8640 1 -- 172.21.15.100:0/471463005 shutdown_connections 2024-09-15T21:46:50.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.741+0000 7fbc38cf8640 1 -- 172.21.15.100:0/471463005 wait complete. 2024-09-15T21:46:50.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.742+0000 7fbc38cf8640 1 Processor -- start 2024-09-15T21:46:50.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.742+0000 7fbc38cf8640 1 -- start start 2024-09-15T21:46:50.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc38cf8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc340c92d0 0x7fbc341ef760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:50.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc38cf8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc340c96b0 0x7fbc34072d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:50.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc38cf8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc341522e0 0x7fbc3414d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:50.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc38cf8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbc34158a50 con 0x7fbc340c92d0 2024-09-15T21:46:50.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc38cf8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbc34158770 con 0x7fbc340c96b0 2024-09-15T21:46:50.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc38cf8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbc340c8d30 con 0x7fbc341522e0 2024-09-15T21:46:50.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc3259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc340c92d0 0x7fbc341ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:50.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc3259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc340c92d0 0x7fbc341ef760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52676/0 (socket says 172.21.15.100:52676) 2024-09-15T21:46:50.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc3259c640 1 -- 172.21.15.100:0/2614752577 learned_addr learned my addr 172.21.15.100:0/2614752577 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:50.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc32d9d640 1 --2- 172.21.15.100:0/2614752577 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc341522e0 0x7fbc3414d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:50.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc31d9b640 1 --2- 172.21.15.100:0/2614752577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc340c96b0 0x7fbc34072d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:50.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc3259c640 1 -- 172.21.15.100:0/2614752577 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc341522e0 msgr2=0x7fbc3414d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:50.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc3259c640 1 --2- 172.21.15.100:0/2614752577 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc341522e0 0x7fbc3414d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:50.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc3259c640 1 -- 172.21.15.100:0/2614752577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc340c96b0 msgr2=0x7fbc34072d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:50.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc3259c640 1 --2- 172.21.15.100:0/2614752577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc340c96b0 0x7fbc34072d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:50.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc3259c640 1 -- 172.21.15.100:0/2614752577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3406f410 con 0x7fbc340c92d0 2024-09-15T21:46:50.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc3259c640 1 --2- 172.21.15.100:0/2614752577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc340c92d0 0x7fbc341ef760 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00d620 tx=0x7fbc1c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:50.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc2affd640 1 -- 172.21.15.100:0/2614752577 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c00b620 con 0x7fbc340c92d0 2024-09-15T21:46:50.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc2affd640 1 -- 172.21.15.100:0/2614752577 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc1c019070 con 0x7fbc340c92d0 2024-09-15T21:46:50.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc38cf8640 1 -- 172.21.15.100:0/2614752577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc341f2270 con 0x7fbc340c92d0 2024-09-15T21:46:50.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc2affd640 1 -- 172.21.15.100:0/2614752577 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c00ba00 con 0x7fbc340c92d0 2024-09-15T21:46:50.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.743+0000 7fbc38cf8640 1 -- 172.21.15.100:0/2614752577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc341f2830 con 0x7fbc340c92d0 2024-09-15T21:46:50.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.744+0000 7fbc38cf8640 1 -- 172.21.15.100:0/2614752577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc340c8740 con 0x7fbc340c92d0 2024-09-15T21:46:50.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.745+0000 7fbc2affd640 1 -- 172.21.15.100:0/2614752577 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbc1c00bba0 con 0x7fbc340c92d0 2024-09-15T21:46:50.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.745+0000 7fbc2affd640 1 --2- 172.21.15.100:0/2614752577 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbc1406c2b0 0x7fbc1406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:50.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.745+0000 7fbc31d9b640 1 --2- 172.21.15.100:0/2614752577 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbc1406c2b0 0x7fbc1406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:50.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.745+0000 7fbc2affd640 1 -- 172.21.15.100:0/2614752577 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fbc1c08b710 con 0x7fbc340c92d0 2024-09-15T21:46:50.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.746+0000 7fbc31d9b640 1 --2- 172.21.15.100:0/2614752577 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbc1406c2b0 0x7fbc1406e770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc341e9620 tx=0x7fbc24009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:50.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.750+0000 7fbc2affd640 1 -- 172.21.15.100:0/2614752577 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbc1c059830 con 0x7fbc340c92d0 2024-09-15T21:46:50.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.950+0000 7fbc38cf8640 1 -- 172.21.15.100:0/2614752577 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc34071580 con 0x7fbc340c92d0 2024-09-15T21:46:50.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.950+0000 7fbc2affd640 1 -- 172.21.15.100:0/2614752577 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+6954 (secure 0 0 0) 0x7fbc1c05db30 con 0x7fbc340c92d0 2024-09-15T21:46:50.952 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:50.952 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":16,"btime":"2024-09-15T21:46:46:560141+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2992633397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2992633397},{"type":"v1","addr":"172.21.15.122:6833","nonce":2992633397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T21:46:22.753977+0000","modified":"2024-09-15T21:46:46.560129+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4291,"mds_2":4290,"mds_3":4288,"mds_4":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":4,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.100:6835/1809406468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1809406468},{"type":"v1","addr":"172.21.15.100:6835","nonce":1809406468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4288":{"gid":4288,"name":"f","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.122:6837/371886422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":371886422},{"type":"v1","addr":"172.21.15.122:6837","nonce":371886422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.100:6838/276075809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":276075809},{"type":"v1","addr":"172.21.15.100:6838","nonce":276075809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4291":{"gid":4291,"name":"b","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6836/2947743626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6835","nonce":2947743626},{"type":"v1","addr":"172.21.15.122:6836","nonce":2947743626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/602064326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6837","nonce":602064326},{"type":"v1","addr":"172.21.15.100:6839","nonce":602064326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4293,4291,4290,4288]},"id":1}]} 2024-09-15T21:46:50.953 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 16 2024-09-15T21:46:50.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.952+0000 7fbc38cf8640 1 -- 172.21.15.100:0/2614752577 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbc1406c2b0 msgr2=0x7fbc1406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:50.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.952+0000 7fbc38cf8640 1 --2- 172.21.15.100:0/2614752577 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbc1406c2b0 0x7fbc1406e770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbc341e9620 tx=0x7fbc24009040 comp rx=0 tx=0).stop 2024-09-15T21:46:50.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.952+0000 7fbc38cf8640 1 -- 172.21.15.100:0/2614752577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc340c92d0 msgr2=0x7fbc341ef760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:50.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.952+0000 7fbc38cf8640 1 --2- 172.21.15.100:0/2614752577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc340c92d0 0x7fbc341ef760 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00d620 tx=0x7fbc1c00dae0 comp rx=0 tx=0).stop 2024-09-15T21:46:50.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.952+0000 7fbc38cf8640 1 -- 172.21.15.100:0/2614752577 shutdown_connections 2024-09-15T21:46:50.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.952+0000 7fbc38cf8640 1 --2- 172.21.15.100:0/2614752577 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbc1406c2b0 0x7fbc1406e770 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:50.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.952+0000 7fbc38cf8640 1 --2- 172.21.15.100:0/2614752577 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc340c96b0 0x7fbc34072d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:50.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.952+0000 7fbc38cf8640 1 --2- 172.21.15.100:0/2614752577 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc341522e0 0x7fbc3414d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:50.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.952+0000 7fbc38cf8640 1 --2- 172.21.15.100:0/2614752577 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc340c92d0 0x7fbc341ef760 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:50.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.952+0000 7fbc38cf8640 1 -- 172.21.15.100:0/2614752577 >> 172.21.15.100:0/2614752577 conn(0x7fbc34067dd0 msgr2=0x7fbc341519a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:50.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.952+0000 7fbc38cf8640 1 -- 172.21.15.100:0/2614752577 shutdown_connections 2024-09-15T21:46:50.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:50.952+0000 7fbc38cf8640 1 -- 172.21.15.100:0/2614752577 wait complete. 2024-09-15T21:46:51.569 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-15T21:46:22.753977+0000', 'modified': '2024-09-15T21:46:46.560129+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4293, 'mds_1': 4291, 'mds_2': 4290, 'mds_3': 4288, 'mds_4': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'c', 'rank': 4, 'incarnation': 16, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.100:6835/1809406468', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 1809406468}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 1809406468}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4288': {'gid': 4288, 'name': 'f', 'rank': 3, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.122:6837/371886422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6834', 'nonce': 371886422}, {'type': 'v1', 'addr': '172.21.15.122:6837', 'nonce': 371886422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4290': {'gid': 4290, 'name': 'a', 'rank': 2, 'incarnation': 12, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.100:6838/276075809', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6836', 'nonce': 276075809}, {'type': 'v1', 'addr': '172.21.15.100:6838', 'nonce': 276075809}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4291': {'gid': 4291, 'name': 'b', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.122:6836/2947743626', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6835', 'nonce': 2947743626}, {'type': 'v1', 'addr': '172.21.15.122:6836', 'nonce': 2947743626}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4293': {'gid': 4293, 'name': 'e', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.100:6839/602064326', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6837', 'nonce': 602064326}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 602064326}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4293, 'qdb_cluster': [4293, 4291, 4290, 4288]} 2024-09-15T21:46:51.569 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4287:up:creating 2024-09-15T21:46:52.570 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T21:46:52.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.678+0000 7f9c02d8d640 1 Processor -- start 2024-09-15T21:46:52.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.678+0000 7f9c02d8d640 1 -- start start 2024-09-15T21:46:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.679+0000 7f9c02d8d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bfc071340 0x7f9bfc151450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.679+0000 7f9c02d8d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bfc0c91f0 con 0x7f9bfc070ee0 2024-09-15T21:46:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.679+0000 7f9c02d8d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bfc0c94d0 con 0x7f9bfc071990 2024-09-15T21:46:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.679+0000 7f9c02d8d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9bfc06ca30 con 0x7f9bfc071340 2024-09-15T21:46:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.679+0000 7f9c00b29640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f9bfc070ee0 0x7f9bfc0cca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:53938/0 (socket says 172.21.15.100:53938) 2024-09-15T21:46:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.679+0000 7f9c00b29640 1 -- 172.21.15.100:0/729303929 learned_addr learned my addr 172.21.15.100:0/729303929 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:52.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9bfb7fe640 1 -- 172.21.15.100:0/729303929 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1474687696 0 0) 0x7f9bfc0c91f0 con 0x7f9bfc070ee0 2024-09-15T21:46:52.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9bfb7fe640 1 -- 172.21.15.100:0/729303929 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9be4003520 con 0x7f9bfc070ee0 2024-09-15T21:46:52.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9c0132a640 1 --2- 172.21.15.100:0/729303929 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bfc071340 0x7f9bfc151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:52.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9bfb7fe640 1 -- 172.21.15.100:0/729303929 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2635338306 0 0) 0x7f9be4003520 con 0x7f9bfc070ee0 2024-09-15T21:46:52.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9bfb7fe640 1 -- 172.21.15.100:0/729303929 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bfc071340 msgr2=0x7f9bfc151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:52.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9bfb7fe640 1 --2- 172.21.15.100:0/729303929 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bfc071340 0x7f9bfc151450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:52.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9bfb7fe640 1 -- 172.21.15.100:0/729303929 >> v1:172.21.15.122:6789/0 conn(0x7f9bfc071990 legacy=0x7f9bfc06c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:52.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9bfb7fe640 1 -- 172.21.15.100:0/729303929 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bfc0c97b0 con 0x7f9bfc070ee0 2024-09-15T21:46:52.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9bfb7fe640 1 -- 172.21.15.100:0/729303929 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7f9bec001670 con 0x7f9bfc070ee0 2024-09-15T21:46:52.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9bfb7fe640 1 -- 172.21.15.100:0/729303929 >> v1:172.21.15.100:6789/0 conn(0x7f9bfc070ee0 legacy=0x7f9bfc0cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:52.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9bfb7fe640 1 --2- 172.21.15.100:0/729303929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be40039e0 0x7f9be4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:52.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9bfb7fe640 1 --2- 172.21.15.100:0/729303929 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bfc071340 0x7f9be4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:52.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9bfb7fe640 1 --2- 172.21.15.100:0/729303929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9be4004d40 0x7f9be40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:52.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9bfb7fe640 1 -- 172.21.15.100:0/729303929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc06ca30 con 0x7f9be4004d40 2024-09-15T21:46:52.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9bfb7fe640 1 -- 172.21.15.100:0/729303929 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9be400bb00 con 0x7f9be40039e0 2024-09-15T21:46:52.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9bfb7fe640 1 -- 172.21.15.100:0/729303929 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9be400bd00 con 0x7f9bfc071340 2024-09-15T21:46:52.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9c0132a640 1 --2- 172.21.15.100:0/729303929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9be4004d40 0x7f9be40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:52.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9c00b29640 1 --2- 172.21.15.100:0/729303929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be40039e0 0x7f9be4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:52.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9bfbfff640 1 --2- 172.21.15.100:0/729303929 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bfc071340 0x7f9be4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:52.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9c0132a640 1 -- 172.21.15.100:0/729303929 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bfc071340 msgr2=0x7f9be4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:52.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9c0132a640 1 --2- 172.21.15.100:0/729303929 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bfc071340 0x7f9be4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:52.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9c0132a640 1 -- 172.21.15.100:0/729303929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be40039e0 msgr2=0x7f9be4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:52.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9c0132a640 1 --2- 172.21.15.100:0/729303929 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be40039e0 0x7f9be4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:52.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.680+0000 7f9c0132a640 1 -- 172.21.15.100:0/729303929 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be4003520 con 0x7f9be4004d40 2024-09-15T21:46:52.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.681+0000 7f9c0132a640 1 --2- 172.21.15.100:0/729303929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9be4004d40 0x7f9be40051c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009a90 tx=0x7f9be4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51604d79cdb7a53 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:52.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.681+0000 7f9bfb7fe640 1 -- 172.21.15.100:0/729303929 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4036070 con 0x7f9be4004d40 2024-09-15T21:46:52.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.681+0000 7f9bfb7fe640 1 -- 172.21.15.100:0/729303929 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bf4031400 con 0x7f9be4004d40 2024-09-15T21:46:52.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.681+0000 7f9bfb7fe640 1 -- 172.21.15.100:0/729303929 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bf4009690 con 0x7f9be4004d40 2024-09-15T21:46:52.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.682+0000 7f9c02d8d640 1 -- 172.21.15.100:0/729303929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9be4004d40 msgr2=0x7f9be40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:52.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.682+0000 7f9c02d8d640 1 --2- 172.21.15.100:0/729303929 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9be4004d40 0x7f9be40051c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9bf4009a90 tx=0x7f9be4005830 comp rx=0 tx=0).stop 2024-09-15T21:46:52.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.682+0000 7f9c0132a640 1 -- 172.21.15.100:0/729303929 reap_dead start 2024-09-15T21:46:52.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.682+0000 7f9c02d8d640 1 -- 172.21.15.100:0/729303929 shutdown_connections 2024-09-15T21:46:52.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.682+0000 7f9c02d8d640 1 -- 172.21.15.100:0/729303929 >> 172.21.15.100:0/729303929 conn(0x7f9bfc067dd0 msgr2=0x7f9bfc06bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:52.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.682+0000 7f9c02d8d640 1 -- 172.21.15.100:0/729303929 shutdown_connections 2024-09-15T21:46:52.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.682+0000 7f9c02d8d640 1 -- 172.21.15.100:0/729303929 wait complete. 2024-09-15T21:46:52.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.683+0000 7f9c02d8d640 1 Processor -- start 2024-09-15T21:46:52.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.683+0000 7f9c02d8d640 1 -- start start 2024-09-15T21:46:52.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.684+0000 7f9c02d8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9be4004d40 0x7f9bfc0cc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:52.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.684+0000 7f9c02d8d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bfc14fa50 0x7f9bfc139d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:52.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.684+0000 7f9c02d8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bfc0ca470 0x7f9bfc142190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:52.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.684+0000 7f9c02d8d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc0c9d70 con 0x7f9be4004d40 2024-09-15T21:46:52.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.684+0000 7f9c02d8d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9bfc0717b0 con 0x7f9bfc0ca470 2024-09-15T21:46:52.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.684+0000 7f9c02d8d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9bfc151450 con 0x7f9bfc14fa50 2024-09-15T21:46:52.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.684+0000 7f9c00b29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9be4004d40 0x7f9bfc0cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:52.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.684+0000 7f9c00b29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9be4004d40 0x7f9bfc0cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52692/0 (socket says 172.21.15.100:52692) 2024-09-15T21:46:52.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.684+0000 7f9c00b29640 1 -- 172.21.15.100:0/1490558563 learned_addr learned my addr 172.21.15.100:0/1490558563 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:52.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.684+0000 7f9c0132a640 1 --2- 172.21.15.100:0/1490558563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bfc0ca470 0x7f9bfc142190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:52.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.684+0000 7f9bfbfff640 1 --2- 172.21.15.100:0/1490558563 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bfc14fa50 0x7f9bfc139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:52.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.684+0000 7f9c00b29640 1 -- 172.21.15.100:0/1490558563 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bfc14fa50 msgr2=0x7f9bfc139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:52.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.684+0000 7f9c00b29640 1 --2- 172.21.15.100:0/1490558563 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bfc14fa50 0x7f9bfc139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:52.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.684+0000 7f9c00b29640 1 -- 172.21.15.100:0/1490558563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bfc0ca470 msgr2=0x7f9bfc142190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:52.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.684+0000 7f9c00b29640 1 --2- 172.21.15.100:0/1490558563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bfc0ca470 0x7f9bfc142190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:52.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.684+0000 7f9c00b29640 1 -- 172.21.15.100:0/1490558563 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bfc159960 con 0x7f9be4004d40 2024-09-15T21:46:52.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.684+0000 7f9c00b29640 1 --2- 172.21.15.100:0/1490558563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9be4004d40 0x7f9bfc0cc150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9bec007340 tx=0x7f9bec007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:52.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.685+0000 7f9bf97fa640 1 -- 172.21.15.100:0/1490558563 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00f8c0 con 0x7f9be4004d40 2024-09-15T21:46:52.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.685+0000 7f9bf97fa640 1 -- 172.21.15.100:0/1490558563 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bec018070 con 0x7f9be4004d40 2024-09-15T21:46:52.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.685+0000 7f9c02d8d640 1 -- 172.21.15.100:0/1490558563 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bfc15a0a0 con 0x7f9be4004d40 2024-09-15T21:46:52.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.685+0000 7f9bf97fa640 1 -- 172.21.15.100:0/1490558563 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bec00fce0 con 0x7f9be4004d40 2024-09-15T21:46:52.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.685+0000 7f9c02d8d640 1 -- 172.21.15.100:0/1490558563 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bfc159c00 con 0x7f9be4004d40 2024-09-15T21:46:52.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.686+0000 7f9c02d8d640 1 -- 172.21.15.100:0/1490558563 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bfc15a270 con 0x7f9be4004d40 2024-09-15T21:46:52.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.686+0000 7f9bf97fa640 1 -- 172.21.15.100:0/1490558563 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9bec01d460 con 0x7f9be4004d40 2024-09-15T21:46:52.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.686+0000 7f9bf97fa640 1 --2- 172.21.15.100:0/1490558563 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9be806c2e0 0x7f9be806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:52.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.687+0000 7f9bfbfff640 1 --2- 172.21.15.100:0/1490558563 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9be806c2e0 0x7f9be806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:52.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.687+0000 7f9bf97fa640 1 -- 172.21.15.100:0/1490558563 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9bec08b300 con 0x7f9be4004d40 2024-09-15T21:46:52.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.687+0000 7f9bfbfff640 1 --2- 172.21.15.100:0/1490558563 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9be806c2e0 0x7f9be806e7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc13ab20 tx=0x7f9bf0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:52.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.691+0000 7f9bf97fa640 1 -- 172.21.15.100:0/1490558563 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9bec0594b0 con 0x7f9be4004d40 2024-09-15T21:46:52.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.893+0000 7f9c02d8d640 1 -- 172.21.15.100:0/1490558563 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bfc06c320 con 0x7f9be4004d40 2024-09-15T21:46:52.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.893+0000 7f9bf97fa640 1 -- 172.21.15.100:0/1490558563 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6958 (secure 0 0 0) 0x7f9bec05d7b0 con 0x7f9be4004d40 2024-09-15T21:46:52.895 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:46:52.895 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":17,"btime":"2024-09-15T21:46:52:569246+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2992633397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2992633397},{"type":"v1","addr":"172.21.15.122:6833","nonce":2992633397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T21:46:22.753977+0000","modified":"2024-09-15T21:46:52.569239+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4291,"mds_2":4290,"mds_3":4288,"mds_4":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.100:6835/1809406468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1809406468},{"type":"v1","addr":"172.21.15.100:6835","nonce":1809406468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4288":{"gid":4288,"name":"f","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.122:6837/371886422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":371886422},{"type":"v1","addr":"172.21.15.122:6837","nonce":371886422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.100:6838/276075809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":276075809},{"type":"v1","addr":"172.21.15.100:6838","nonce":276075809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4291":{"gid":4291,"name":"b","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6836/2947743626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6835","nonce":2947743626},{"type":"v1","addr":"172.21.15.122:6836","nonce":2947743626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/602064326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6837","nonce":602064326},{"type":"v1","addr":"172.21.15.100:6839","nonce":602064326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4287,4288,4290,4291,4293]},"id":1}]} 2024-09-15T21:46:52.896 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 17 2024-09-15T21:46:52.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.895+0000 7f9c02d8d640 1 -- 172.21.15.100:0/1490558563 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9be806c2e0 msgr2=0x7f9be806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:52.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.895+0000 7f9c02d8d640 1 --2- 172.21.15.100:0/1490558563 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9be806c2e0 0x7f9be806e7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc13ab20 tx=0x7f9bf0009040 comp rx=0 tx=0).stop 2024-09-15T21:46:52.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.895+0000 7f9c02d8d640 1 -- 172.21.15.100:0/1490558563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9be4004d40 msgr2=0x7f9bfc0cc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:52.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.895+0000 7f9c02d8d640 1 --2- 172.21.15.100:0/1490558563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9be4004d40 0x7f9bfc0cc150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9bec007340 tx=0x7f9bec007800 comp rx=0 tx=0).stop 2024-09-15T21:46:52.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.895+0000 7f9c02d8d640 1 -- 172.21.15.100:0/1490558563 shutdown_connections 2024-09-15T21:46:52.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.895+0000 7f9c02d8d640 1 --2- 172.21.15.100:0/1490558563 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9be806c2e0 0x7f9be806e7a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:52.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.895+0000 7f9c02d8d640 1 --2- 172.21.15.100:0/1490558563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9bfc0ca470 0x7f9bfc142190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:52.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.895+0000 7f9c02d8d640 1 --2- 172.21.15.100:0/1490558563 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9bfc14fa50 0x7f9bfc139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:52.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.895+0000 7f9c02d8d640 1 --2- 172.21.15.100:0/1490558563 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9be4004d40 0x7f9bfc0cc150 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:52.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.895+0000 7f9c02d8d640 1 -- 172.21.15.100:0/1490558563 >> 172.21.15.100:0/1490558563 conn(0x7f9bfc067dd0 msgr2=0x7f9bfc0c70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:52.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.895+0000 7f9c02d8d640 1 -- 172.21.15.100:0/1490558563 shutdown_connections 2024-09-15T21:46:52.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:52.896+0000 7f9c02d8d640 1 -- 172.21.15.100:0/1490558563 wait complete. 2024-09-15T21:46:53.395 INFO:tasks.ceph.mds.e.smithi100.stderr:2024-09-15T21:46:53.393+0000 7fa09cc49640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-15T21:46:53.511 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-15T21:46:22.753977+0000', 'modified': '2024-09-15T21:46:52.569239+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 4293, 'mds_1': 4291, 'mds_2': 4290, 'mds_3': 4288, 'mds_4': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'c', 'rank': 4, 'incarnation': 16, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.100:6835/1809406468', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 1809406468}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 1809406468}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4288': {'gid': 4288, 'name': 'f', 'rank': 3, 'incarnation': 14, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.122:6837/371886422', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6834', 'nonce': 371886422}, {'type': 'v1', 'addr': '172.21.15.122:6837', 'nonce': 371886422}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4290': {'gid': 4290, 'name': 'a', 'rank': 2, 'incarnation': 12, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.100:6838/276075809', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6836', 'nonce': 276075809}, {'type': 'v1', 'addr': '172.21.15.100:6838', 'nonce': 276075809}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4291': {'gid': 4291, 'name': 'b', 'rank': 1, 'incarnation': 10, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.122:6836/2947743626', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6835', 'nonce': 2947743626}, {'type': 'v1', 'addr': '172.21.15.122:6836', 'nonce': 2947743626}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4293': {'gid': 4293, 'name': 'e', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.100:6839/602064326', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6837', 'nonce': 602064326}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 602064326}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4293, 'qdb_cluster': [4287, 4288, 4290, 4291, 4293]} 2024-09-15T21:46:53.512 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-15T21:46:53.512 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-15T21:46:53.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.611+0000 7f8576e77640 1 Processor -- start 2024-09-15T21:46:53.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.611+0000 7f8576e77640 1 -- start start 2024-09-15T21:46:53.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.612+0000 7f8576e77640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8570159b80 0x7f8570073840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:53.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.612+0000 7f8576e77640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85700c9ba0 0x7f8570073d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:53.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.612+0000 7f8576e77640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85700c3f50 con 0x7f85700c97c0 2024-09-15T21:46:53.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.612+0000 7f8576e77640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f85700c41c0 con 0x7f8570159b80 2024-09-15T21:46:53.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.612+0000 7f8576e77640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f85700c44a0 con 0x7f85700c9ba0 2024-09-15T21:46:53.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.612+0000 7f856ffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8570159b80 0x7f8570073840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:53.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.612+0000 7f856ffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8570159b80 0x7f8570073840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52702/0 (socket says 172.21.15.100:52702) 2024-09-15T21:46:53.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.612+0000 7f856ffff640 1 -- 172.21.15.100:0/1672466083 learned_addr learned my addr 172.21.15.100:0/1672466083 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:53.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.613+0000 7f856ffff640 1 -- 172.21.15.100:0/1672466083 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85700c9ba0 msgr2=0x7f8570073d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:53.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.613+0000 7f8575414640 1 --2- 172.21.15.100:0/1672466083 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85700c9ba0 0x7f8570073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:53.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.613+0000 7f856ffff640 1 --2- 172.21.15.100:0/1672466083 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85700c9ba0 0x7f8570073d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:53.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.613+0000 7f856ffff640 1 -- 172.21.15.100:0/1672466083 >> v1:172.21.15.100:6789/0 conn(0x7f85700c97c0 legacy=0x7f8570159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:53.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.613+0000 7f856ffff640 1 -- 172.21.15.100:0/1672466083 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85700c4780 con 0x7f8570159b80 2024-09-15T21:46:53.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.613+0000 7f856f7fe640 1 -- 172.21.15.100:0/1672466083 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 864869895 0 0) 0x7f85700c3f50 con 0x7f85700c97c0 2024-09-15T21:46:53.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.613+0000 7f856ffff640 1 --2- 172.21.15.100:0/1672466083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8570159b80 0x7f8570073840 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8564009e50 tx=0x7f856402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4b31a0a6243ae4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:53.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.614+0000 7f856f7fe640 1 -- 172.21.15.100:0/1672466083 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8564036070 con 0x7f8570159b80 2024-09-15T21:46:53.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.614+0000 7f856f7fe640 1 -- 172.21.15.100:0/1672466083 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8564031600 con 0x7f8570159b80 2024-09-15T21:46:53.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.614+0000 7f856f7fe640 1 -- 172.21.15.100:0/1672466083 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85640319d0 con 0x7f8570159b80 2024-09-15T21:46:53.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.614+0000 7f8576e77640 1 -- 172.21.15.100:0/1672466083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8570159b80 msgr2=0x7f8570073840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:53.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.614+0000 7f8576e77640 1 --2- 172.21.15.100:0/1672466083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8570159b80 0x7f8570073840 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8564009e50 tx=0x7f856402a7a0 comp rx=0 tx=0).stop 2024-09-15T21:46:53.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.614+0000 7f8576e77640 1 -- 172.21.15.100:0/1672466083 shutdown_connections 2024-09-15T21:46:53.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.614+0000 7f8576e77640 1 --2- 172.21.15.100:0/1672466083 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85700c9ba0 0x7f8570073d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:53.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.614+0000 7f8576e77640 1 --2- 172.21.15.100:0/1672466083 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8570159b80 0x7f8570073840 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:53.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.614+0000 7f8576e77640 1 -- 172.21.15.100:0/1672466083 >> 172.21.15.100:0/1672466083 conn(0x7f857006c210 msgr2=0x7f8570067980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:53.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.614+0000 7f8576e77640 1 -- 172.21.15.100:0/1672466083 shutdown_connections 2024-09-15T21:46:53.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.614+0000 7f8576e77640 1 -- 172.21.15.100:0/1672466083 wait complete. 2024-09-15T21:46:53.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.615+0000 7f8576e77640 1 Processor -- start 2024-09-15T21:46:53.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.615+0000 7f8576e77640 1 -- start start 2024-09-15T21:46:53.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.616+0000 7f8576e77640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85700c97c0 0x7f85700c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:53.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.616+0000 7f8576e77640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85700c9ba0 0x7f857013e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:53.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.616+0000 7f8576e77640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8570159b80 0x7f857013e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:53.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.616+0000 7f8576e77640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f85700b4da0 con 0x7f8570159b80 2024-09-15T21:46:53.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.616+0000 7f8576e77640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8570158ee0 con 0x7f85700c9ba0 2024-09-15T21:46:53.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.616+0000 7f8576e77640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f85700b4f20 con 0x7f85700c97c0 2024-09-15T21:46:53.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.616+0000 7f8575414640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8570159b80 0x7f857013e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:53.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.616+0000 7f8575414640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8570159b80 0x7f857013e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52704/0 (socket says 172.21.15.100:52704) 2024-09-15T21:46:53.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.616+0000 7f8575414640 1 -- 172.21.15.100:0/416228807 learned_addr learned my addr 172.21.15.100:0/416228807 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:53.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.616+0000 7f8574c13640 1 --2- 172.21.15.100:0/416228807 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85700c97c0 0x7f85700c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:53.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.616+0000 7f856ffff640 1 --2- 172.21.15.100:0/416228807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85700c9ba0 0x7f857013e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:53.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.616+0000 7f8575414640 1 -- 172.21.15.100:0/416228807 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85700c97c0 msgr2=0x7f85700c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:53.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.616+0000 7f8575414640 1 --2- 172.21.15.100:0/416228807 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85700c97c0 0x7f85700c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:53.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.616+0000 7f8575414640 1 -- 172.21.15.100:0/416228807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85700c9ba0 msgr2=0x7f857013e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:53.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.616+0000 7f8575414640 1 --2- 172.21.15.100:0/416228807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85700c9ba0 0x7f857013e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:53.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.617+0000 7f8575414640 1 -- 172.21.15.100:0/416228807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85700c6310 con 0x7f8570159b80 2024-09-15T21:46:53.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.617+0000 7f8574c13640 1 --2- 172.21.15.100:0/416228807 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85700c97c0 0x7f85700c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:53.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.617+0000 7f8575414640 1 --2- 172.21.15.100:0/416228807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8570159b80 0x7f857013e9f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f856800c590 tx=0x7f856800ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:53.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.617+0000 7f856d7fa640 1 -- 172.21.15.100:0/416228807 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568015070 con 0x7f8570159b80 2024-09-15T21:46:53.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.617+0000 7f856d7fa640 1 -- 172.21.15.100:0/416228807 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8568004220 con 0x7f8570159b80 2024-09-15T21:46:53.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.617+0000 7f8576e77640 1 -- 172.21.15.100:0/416228807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85701f2dc0 con 0x7f8570159b80 2024-09-15T21:46:53.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.617+0000 7f856d7fa640 1 -- 172.21.15.100:0/416228807 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568003610 con 0x7f8570159b80 2024-09-15T21:46:53.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.617+0000 7f8576e77640 1 -- 172.21.15.100:0/416228807 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85700c6890 con 0x7f8570159b80 2024-09-15T21:46:53.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.618+0000 7f856d7fa640 1 -- 172.21.15.100:0/416228807 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f85680037b0 con 0x7f8570159b80 2024-09-15T21:46:53.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.619+0000 7f856d7fa640 1 --2- 172.21.15.100:0/416228807 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f855c06c2e0 0x7f855c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:53.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.619+0000 7f856d7fa640 1 -- 172.21.15.100:0/416228807 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f856808b1d0 con 0x7f8570159b80 2024-09-15T21:46:53.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.619+0000 7f8574c13640 1 --2- 172.21.15.100:0/416228807 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f855c06c2e0 0x7f855c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.619+0000 7f8574c13640 1 --2- 172.21.15.100:0/416228807 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f855c06c2e0 0x7f855c06e7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8560000f60 tx=0x7f8560009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:53.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.620+0000 7f8579c13740 1 Processor -- start 2024-09-15T21:46:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.620+0000 7f8579c13740 1 -- start start 2024-09-15T21:46:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.620+0000 7f8579c13740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490750 0x55b2fe4c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.620+0000 7f8579c13740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b2fe48df80 0x55b2fe4c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.620+0000 7f8579c13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe490b30 0x55b2fe48ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.620+0000 7f8579c13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b2fe3f1520 con 0x55b2fe490b30 2024-09-15T21:46:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.620+0000 7f8579c13740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b2fe3993e0 con 0x55b2fe490750 2024-09-15T21:46:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.620+0000 7f8579c13740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x55b2fe3ed4e0 con 0x55b2fe48df80 2024-09-15T21:46:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.620+0000 7f8575414640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe490b30 0x55b2fe48ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.620+0000 7f8575414640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe490b30 0x55b2fe48ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52708/0 (socket says 172.21.15.100:52708) 2024-09-15T21:46:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.620+0000 7f8575414640 1 -- 172.21.15.100:0/1932642352 learned_addr learned my addr 172.21.15.100:0/1932642352 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.621+0000 7f8575414640 1 -- 172.21.15.100:0/1932642352 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b2fe48df80 msgr2=0x55b2fe4c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.621+0000 7f8575414640 1 --2- 172.21.15.100:0/1932642352 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b2fe48df80 0x55b2fe4c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:53.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.621+0000 7f8575414640 1 -- 172.21.15.100:0/1932642352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490750 msgr2=0x55b2fe4c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.621+0000 7f8575414640 1 --2- 172.21.15.100:0/1932642352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490750 0x55b2fe4c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.621+0000 7f8575414640 1 -- 172.21.15.100:0/1932642352 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2fe3b42f0 con 0x55b2fe490b30 2024-09-15T21:46:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.621+0000 7f8575414640 1 --2- 172.21.15.100:0/1932642352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe490b30 0x55b2fe48ca60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8568089670 tx=0x7f8568059380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c2484b752a4ff4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.621+0000 7f85567fc640 1 -- 172.21.15.100:0/1932642352 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856801e040 con 0x55b2fe490b30 2024-09-15T21:46:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.621+0000 7f85567fc640 1 -- 172.21.15.100:0/1932642352 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8568057540 con 0x55b2fe490b30 2024-09-15T21:46:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.621+0000 7f85567fc640 1 -- 172.21.15.100:0/1932642352 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85680558b0 con 0x55b2fe490b30 2024-09-15T21:46:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.621+0000 7f8579c13740 1 -- 172.21.15.100:0/1932642352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe490b30 msgr2=0x55b2fe48ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.621+0000 7f8579c13740 1 --2- 172.21.15.100:0/1932642352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe490b30 0x55b2fe48ca60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8568089670 tx=0x7f8568059380 comp rx=0 tx=0).stop 2024-09-15T21:46:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.621+0000 7f8579c13740 1 -- 172.21.15.100:0/1932642352 shutdown_connections 2024-09-15T21:46:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.621+0000 7f8579c13740 1 --2- 172.21.15.100:0/1932642352 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe490b30 0x55b2fe48ca60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:53.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.621+0000 7f8579c13740 1 --2- 172.21.15.100:0/1932642352 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b2fe48df80 0x55b2fe4c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.621+0000 7f8579c13740 1 --2- 172.21.15.100:0/1932642352 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490750 0x55b2fe4c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.621+0000 7f8579c13740 1 -- 172.21.15.100:0/1932642352 >> 172.21.15.100:0/1932642352 conn(0x55b2fe48ee70 msgr2=0x55b2fe48f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.622+0000 7f8579c13740 1 -- 172.21.15.100:0/1932642352 shutdown_connections 2024-09-15T21:46:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.622+0000 7f8579c13740 1 -- 172.21.15.100:0/1932642352 wait complete. 2024-09-15T21:46:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.622+0000 7f8579c13740 1 -- start start 2024-09-15T21:46:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.622+0000 7f8579c13740 1 Processor -- start 2024-09-15T21:46:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.623+0000 7f8579c13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48df80 0x55b2fe4c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.623+0000 7f8579c13740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b2fe490750 0x55b2fe4c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.623+0000 7f8579c13740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490b30 0x55b2fe4b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.623+0000 7f8579c13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b2fe48ffe0 con 0x55b2fe48df80 2024-09-15T21:46:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.623+0000 7f8579c13740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b2fe37a450 con 0x55b2fe490b30 2024-09-15T21:46:53.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.623+0000 7f8579c13740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x55b2fe4c8550 con 0x55b2fe490750 2024-09-15T21:46:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.623+0000 7f8579c13740 10 client.0 _ll_register_callbacks cb 0x55b2fe103ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T21:46:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.623+0000 7f8574c13640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48df80 0x55b2fe4c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.623+0000 7f8574c13640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48df80 0x55b2fe4c2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52716/0 (socket says 172.21.15.100:52716) 2024-09-15T21:46:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.623+0000 7f8574c13640 1 -- 172.21.15.100:0/1057103784 learned_addr learned my addr 172.21.15.100:0/1057103784 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.623+0000 7f8575414640 1 --2- 172.21.15.100:0/1057103784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490b30 0x55b2fe4b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.623+0000 7f856ffff640 1 --2- 172.21.15.100:0/1057103784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b2fe490750 0x55b2fe4c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.623+0000 7f8574c13640 1 -- 172.21.15.100:0/1057103784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b2fe490750 msgr2=0x55b2fe4c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:53.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.623+0000 7f8574c13640 1 --2- 172.21.15.100:0/1057103784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b2fe490750 0x55b2fe4c90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.623+0000 7f8574c13640 1 -- 172.21.15.100:0/1057103784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490b30 msgr2=0x55b2fe4b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.623+0000 7f8574c13640 1 --2- 172.21.15.100:0/1057103784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490b30 0x55b2fe4b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.623+0000 7f8574c13640 1 -- 172.21.15.100:0/1057103784 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2fe1042d0 con 0x55b2fe48df80 2024-09-15T21:46:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.623+0000 7f8574c13640 1 --2- 172.21.15.100:0/1057103784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48df80 0x55b2fe4c2220 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f856000e7e0 tx=0x7f856000ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db33a7c74bcd0781 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.623+0000 7f854f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-15T21:46:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.624+0000 7f854f7fe640 1 -- 172.21.15.100:0/1057103784 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560003af0 con 0x55b2fe48df80 2024-09-15T21:46:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.624+0000 7f854f7fe640 1 -- 172.21.15.100:0/1057103784 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8560003c90 con 0x55b2fe48df80 2024-09-15T21:46:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.624+0000 7f8579c13740 1 -- 172.21.15.100:0/1057103784 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8568057540 con 0x55b2fe48df80 2024-09-15T21:46:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.624+0000 7f854f7fe640 1 -- 172.21.15.100:0/1057103784 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f8560013420 con 0x55b2fe48df80 2024-09-15T21:46:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.624+0000 7f8579c13740 10 client.4689 fetch_fsmap learned FSMap version 17 2024-09-15T21:46:53.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.624+0000 7f8579c13740 1 -- 172.21.15.100:0/1057103784 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55b2fe1ce6b0 con 0x55b2fe48df80 2024-09-15T21:46:53.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.624+0000 7f854f7fe640 1 -- 172.21.15.100:0/1057103784 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8568057540 con 0x55b2fe48df80 2024-09-15T21:46:53.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.624+0000 7f854f7fe640 1 -- 172.21.15.100:0/1057103784 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 17) ==== 4792+0+0 (secure 0 0 0) 0x7f856002c740 con 0x55b2fe48df80 2024-09-15T21:46:53.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.624+0000 7f8579c13740 10 client.4689 fetch_fsmap finished waiting for FSMap version 17 2024-09-15T21:46:53.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.624+0000 7f8579c13740 10 client.4689 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-15T21:46:53.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.624+0000 7f8579c13740 20 client.4689 populate_metadata read hostname 'smithi100' 2024-09-15T21:46:53.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.624+0000 7f8579c13740 1 --2- 172.21.15.100:0/1057103784 >> [v2:172.21.15.100:6834/1809406468,v1:172.21.15.100:6835/1809406468] conn(0x55b2fe4ba9f0 0x55b2fe4ee700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:53.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.624+0000 7f8579c13740 4 client.4689 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:46:53.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.624+0000 7f8579c13740 1 -- 172.21.15.100:0/1057103784 --> [v2:172.21.15.100:6834/1809406468,v1:172.21.15.100:6835/1809406468] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2fe3fec50 con 0x55b2fe4ba9f0 2024-09-15T21:46:53.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.624+0000 7f856ffff640 1 --2- 172.21.15.100:0/1057103784 >> [v2:172.21.15.100:6834/1809406468,v1:172.21.15.100:6835/1809406468] conn(0x55b2fe4ba9f0 0x55b2fe4ee700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:53.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.625+0000 7f854f7fe640 10 client.4689 ms_handle_connect on v2:172.21.15.100:6834/1809406468 2024-09-15T21:46:53.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:53.625+0000 7f856ffff640 1 --2- 172.21.15.100:0/1057103784 >> [v2:172.21.15.100:6834/1809406468,v1:172.21.15.100:6835/1809406468] conn(0x55b2fe4ba9f0 0x55b2fe4ee700 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cd2309ddba30956f server_cookie=a37078a9cc445255 in_seq=0 out_seq=0 2024-09-15T21:46:57.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.579+0000 7f854f7fe640 1 -- 172.21.15.100:0/1057103784 <== mds.4 v2:172.21.15.100:6834/1809406468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b2fe3fec50 con 0x55b2fe4ba9f0 2024-09-15T21:46:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.579+0000 7f854f7fe640 10 client.4689 handle_command_reply: tid=0 2024-09-15T21:46:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.579+0000 7f8579c13740 1 client.4689 shutdown 2024-09-15T21:46:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.579+0000 7f8579c13740 1 -- 172.21.15.100:0/1057103784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48df80 msgr2=0x55b2fe4c2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.579+0000 7f8579c13740 1 --2- 172.21.15.100:0/1057103784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48df80 0x55b2fe4c2220 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f856000e7e0 tx=0x7f856000ebe0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.579+0000 7f8579c13740 1 -- 172.21.15.100:0/1057103784 shutdown_connections 2024-09-15T21:46:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.579+0000 7f8579c13740 1 --2- 172.21.15.100:0/1057103784 >> [v2:172.21.15.100:6834/1809406468,v1:172.21.15.100:6835/1809406468] conn(0x55b2fe4ba9f0 0x55b2fe4ee700 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.579+0000 7f8579c13740 1 --2- 172.21.15.100:0/1057103784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490b30 0x55b2fe4b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.579+0000 7f8579c13740 1 --2- 172.21.15.100:0/1057103784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b2fe490750 0x55b2fe4c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.579+0000 7f8579c13740 1 --2- 172.21.15.100:0/1057103784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48df80 0x55b2fe4c2220 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.579+0000 7f854f7fe640 0 client.4689 ms_handle_reset on v2:172.21.15.100:6834/1809406468 2024-09-15T21:46:57.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.579+0000 7f8579c13740 1 -- 172.21.15.100:0/1057103784 >> 172.21.15.100:0/1057103784 conn(0x55b2fe48ee70 msgr2=0x55b2fe4d2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.579+0000 7f8579c13740 1 -- 172.21.15.100:0/1057103784 shutdown_connections 2024-09-15T21:46:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.579+0000 7f8579c13740 1 -- 172.21.15.100:0/1057103784 wait complete. 2024-09-15T21:46:57.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.579+0000 7f8579c13740 20 client.4689 trim_cache size 0 max 16384 2024-09-15T21:46:57.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.599+0000 7f8579c13740 1 Processor -- start 2024-09-15T21:46:57.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.599+0000 7f8579c13740 1 -- start start 2024-09-15T21:46:57.602 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.600+0000 7f8579c13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48ee70 0x55b2fe4c0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.600+0000 7f8579c13740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b2fe48df80 0x55b2fe489ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.600+0000 7f8579c13740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490750 0x55b2fe48a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.600+0000 7f8579c13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f85701fb290 con 0x55b2fe48ee70 2024-09-15T21:46:57.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.600+0000 7f8579c13740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f85700cea00 con 0x55b2fe490750 2024-09-15T21:46:57.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.600+0000 7f8579c13740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x55b2fe4b6320 con 0x55b2fe48df80 2024-09-15T21:46:57.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.600+0000 7f8574c13640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48ee70 0x55b2fe4c0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.600+0000 7f8574c13640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48ee70 0x55b2fe4c0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48594/0 (socket says 172.21.15.100:48594) 2024-09-15T21:46:57.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.600+0000 7f8574c13640 1 -- 172.21.15.100:0/3847642515 learned_addr learned my addr 172.21.15.100:0/3847642515 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:57.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.600+0000 7f856ffff640 1 --2- 172.21.15.100:0/3847642515 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b2fe48df80 0x55b2fe489ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.600+0000 7f8575414640 1 --2- 172.21.15.100:0/3847642515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490750 0x55b2fe48a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.600+0000 7f8574c13640 1 -- 172.21.15.100:0/3847642515 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b2fe48df80 msgr2=0x55b2fe489ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.600+0000 7f8574c13640 1 --2- 172.21.15.100:0/3847642515 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b2fe48df80 0x55b2fe489ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.600+0000 7f8574c13640 1 -- 172.21.15.100:0/3847642515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490750 msgr2=0x55b2fe48a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.600+0000 7f8574c13640 1 --2- 172.21.15.100:0/3847642515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490750 0x55b2fe48a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.600+0000 7f8574c13640 1 -- 172.21.15.100:0/3847642515 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2fe4c1b30 con 0x55b2fe48ee70 2024-09-15T21:46:57.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.601+0000 7f8574c13640 1 --2- 172.21.15.100:0/3847642515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48ee70 0x55b2fe4c0f30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8560001e70 tx=0x7f856001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8001d64bf15f820 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:57.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.601+0000 7f8555ffb640 1 -- 172.21.15.100:0/3847642515 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856002cdd0 con 0x55b2fe48ee70 2024-09-15T21:46:57.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.601+0000 7f8555ffb640 1 -- 172.21.15.100:0/3847642515 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85600052e0 con 0x55b2fe48ee70 2024-09-15T21:46:57.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.601+0000 7f8555ffb640 1 -- 172.21.15.100:0/3847642515 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560005490 con 0x55b2fe48ee70 2024-09-15T21:46:57.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.602+0000 7f8579c13740 1 -- 172.21.15.100:0/3847642515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48ee70 msgr2=0x55b2fe4c0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.602+0000 7f8579c13740 1 --2- 172.21.15.100:0/3847642515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48ee70 0x55b2fe4c0f30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8560001e70 tx=0x7f856001d000 comp rx=0 tx=0).stop 2024-09-15T21:46:57.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.602+0000 7f8579c13740 1 -- 172.21.15.100:0/3847642515 shutdown_connections 2024-09-15T21:46:57.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.602+0000 7f8579c13740 1 --2- 172.21.15.100:0/3847642515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490750 0x55b2fe48a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.602+0000 7f8579c13740 1 --2- 172.21.15.100:0/3847642515 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b2fe48df80 0x55b2fe489ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.602+0000 7f8579c13740 1 --2- 172.21.15.100:0/3847642515 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48ee70 0x55b2fe4c0f30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.602+0000 7f8579c13740 1 -- 172.21.15.100:0/3847642515 >> 172.21.15.100:0/3847642515 conn(0x55b2fe4c2570 msgr2=0x55b2fe4c8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:57.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.602+0000 7f8579c13740 1 -- 172.21.15.100:0/3847642515 shutdown_connections 2024-09-15T21:46:57.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.602+0000 7f8579c13740 1 -- 172.21.15.100:0/3847642515 wait complete. 2024-09-15T21:46:57.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.602+0000 7f8579c13740 1 -- start start 2024-09-15T21:46:57.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.603+0000 7f8579c13740 1 Processor -- start 2024-09-15T21:46:57.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.603+0000 7f8579c13740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48df80 0x55b2fe4d3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.603+0000 7f8579c13740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b2fe48ee70 0x55b2fe54c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.603+0000 7f8579c13740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490750 0x55b2fe54ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.603+0000 7f8579c13740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b2fe4c9610 con 0x55b2fe48df80 2024-09-15T21:46:57.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.603+0000 7f8579c13740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b2fe4b7310 con 0x55b2fe490750 2024-09-15T21:46:57.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.603+0000 7f8579c13740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x55b2fe4e31e0 con 0x55b2fe48ee70 2024-09-15T21:46:57.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.604+0000 7f8579c13740 10 client.0 _ll_register_callbacks cb 0x7f85600013c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T21:46:57.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.604+0000 7f8574c13640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48df80 0x55b2fe4d3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.604+0000 7f8574c13640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48df80 0x55b2fe4d3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48596/0 (socket says 172.21.15.100:48596) 2024-09-15T21:46:57.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.604+0000 7f8574c13640 1 -- 172.21.15.100:0/4232834165 learned_addr learned my addr 172.21.15.100:0/4232834165 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:57.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.604+0000 7f8575414640 1 --2- 172.21.15.100:0/4232834165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490750 0x55b2fe54ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.604+0000 7f856ffff640 1 --2- 172.21.15.100:0/4232834165 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b2fe48ee70 0x55b2fe54c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.604+0000 7f8574c13640 1 -- 172.21.15.100:0/4232834165 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b2fe48ee70 msgr2=0x55b2fe54c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.604+0000 7f8574c13640 1 --2- 172.21.15.100:0/4232834165 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b2fe48ee70 0x55b2fe54c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.604+0000 7f8574c13640 1 -- 172.21.15.100:0/4232834165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490750 msgr2=0x55b2fe54ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.604+0000 7f8574c13640 1 --2- 172.21.15.100:0/4232834165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490750 0x55b2fe54ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.604+0000 7f8574c13640 1 -- 172.21.15.100:0/4232834165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2fe4c1470 con 0x55b2fe48df80 2024-09-15T21:46:57.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.604+0000 7f8575414640 1 --2- 172.21.15.100:0/4232834165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490750 0x55b2fe54ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:46:57.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.604+0000 7f8574c13640 1 --2- 172.21.15.100:0/4232834165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48df80 0x55b2fe4d3020 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f856000e530 tx=0x7f8560012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fc317e2a3c7ccb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:57.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.604+0000 7f854f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-15T21:46:57.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.604+0000 7f854f7fe640 1 -- 172.21.15.100:0/4232834165 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85600137d0 con 0x55b2fe48df80 2024-09-15T21:46:57.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.604+0000 7f854f7fe640 1 -- 172.21.15.100:0/4232834165 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8560013970 con 0x55b2fe48df80 2024-09-15T21:46:57.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.604+0000 7f8579c13740 1 -- 172.21.15.100:0/4232834165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85600052e0 con 0x55b2fe48df80 2024-09-15T21:46:57.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.605+0000 7f854f7fe640 1 -- 172.21.15.100:0/4232834165 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f8560047050 con 0x55b2fe48df80 2024-09-15T21:46:57.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.605+0000 7f8579c13740 10 client.4695 fetch_fsmap learned FSMap version 17 2024-09-15T21:46:57.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.605+0000 7f8579c13740 1 -- 172.21.15.100:0/4232834165 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55b2fe490140 con 0x55b2fe48df80 2024-09-15T21:46:57.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.605+0000 7f854f7fe640 1 -- 172.21.15.100:0/4232834165 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85600052e0 con 0x55b2fe48df80 2024-09-15T21:46:57.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.605+0000 7f854f7fe640 1 -- 172.21.15.100:0/4232834165 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 17) ==== 4792+0+0 (secure 0 0 0) 0x7f856004a020 con 0x55b2fe48df80 2024-09-15T21:46:57.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.605+0000 7f8579c13740 10 client.4695 fetch_fsmap finished waiting for FSMap version 17 2024-09-15T21:46:57.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.605+0000 7f8579c13740 10 client.4695 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-15T21:46:57.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.605+0000 7f8579c13740 20 client.4695 populate_metadata read hostname 'smithi100' 2024-09-15T21:46:57.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.605+0000 7f8579c13740 1 --2- 172.21.15.100:0/4232834165 >> [v2:172.21.15.100:6834/1809406468,v1:172.21.15.100:6835/1809406468] conn(0x55b2fe490b30 0x55b2fe555dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.605+0000 7f8579c13740 4 client.4695 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T21:46:57.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.605+0000 7f8579c13740 1 -- 172.21.15.100:0/4232834165 --> [v2:172.21.15.100:6834/1809406468,v1:172.21.15.100:6835/1809406468] -- command(tid 0: {"prefix": "status"}) -- 0x55b2fe3f8fa0 con 0x55b2fe490b30 2024-09-15T21:46:57.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.605+0000 7f856ffff640 1 --2- 172.21.15.100:0/4232834165 >> [v2:172.21.15.100:6834/1809406468,v1:172.21.15.100:6835/1809406468] conn(0x55b2fe490b30 0x55b2fe555dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.606+0000 7f854f7fe640 10 client.4695 ms_handle_connect on v2:172.21.15.100:6834/1809406468 2024-09-15T21:46:57.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.606+0000 7f856ffff640 1 --2- 172.21.15.100:0/4232834165 >> [v2:172.21.15.100:6834/1809406468,v1:172.21.15.100:6835/1809406468] conn(0x55b2fe490b30 0x55b2fe555dd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9d4efad36331288a server_cookie=850037200e47a338 in_seq=0 out_seq=0 2024-09-15T21:46:57.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.606+0000 7f854f7fe640 1 -- 172.21.15.100:0/4232834165 <== mds.4 v2:172.21.15.100:6834/1809406468 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55b2fe3f8fa0 con 0x55b2fe490b30 2024-09-15T21:46:57.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.606+0000 7f854f7fe640 10 client.4695 handle_command_reply: tid=0 2024-09-15T21:46:57.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.606+0000 7f8579c13740 1 client.4695 shutdown 2024-09-15T21:46:57.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.606+0000 7f8579c13740 1 -- 172.21.15.100:0/4232834165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48df80 msgr2=0x55b2fe4d3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.606+0000 7f8579c13740 1 --2- 172.21.15.100:0/4232834165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48df80 0x55b2fe4d3020 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f856000e530 tx=0x7f8560012000 comp rx=0 tx=0).stop 2024-09-15T21:46:57.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.606+0000 7f8579c13740 1 -- 172.21.15.100:0/4232834165 shutdown_connections 2024-09-15T21:46:57.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.606+0000 7f8579c13740 1 --2- 172.21.15.100:0/4232834165 >> [v2:172.21.15.100:6834/1809406468,v1:172.21.15.100:6835/1809406468] conn(0x55b2fe490b30 0x55b2fe555dd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.606+0000 7f8579c13740 1 --2- 172.21.15.100:0/4232834165 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b2fe490750 0x55b2fe54ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.606+0000 7f8579c13740 1 --2- 172.21.15.100:0/4232834165 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b2fe48ee70 0x55b2fe54c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.606+0000 7f8579c13740 1 --2- 172.21.15.100:0/4232834165 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b2fe48df80 0x55b2fe4d3020 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.606+0000 7f854f7fe640 0 client.4695 ms_handle_reset on v2:172.21.15.100:6834/1809406468 2024-09-15T21:46:57.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.606+0000 7f8579c13740 1 -- 172.21.15.100:0/4232834165 >> 172.21.15.100:0/4232834165 conn(0x55b2fe4c2570 msgr2=0x55b2fe4c8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:57.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.606+0000 7f8579c13740 1 -- 172.21.15.100:0/4232834165 shutdown_connections 2024-09-15T21:46:57.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.606+0000 7f8579c13740 1 -- 172.21.15.100:0/4232834165 wait complete. 2024-09-15T21:46:57.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.606+0000 7f8579c13740 20 client.4695 trim_cache size 0 max 16384 2024-09-15T21:46:57.614 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:46:57.614 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "30e2fff4-2171-4531-8806-0e03b63f494f", 2024-09-15T21:46:57.614 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 4, 2024-09-15T21:46:57.615 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4287, 2024-09-15T21:46:57.615 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-09-15T21:46:57.615 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-09-15T21:46:57.615 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-09-15T21:46:57.615 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 11.045604002999999, 2024-09-15T21:46:57.615 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 17, 2024-09-15T21:46:57.615 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 22, 2024-09-15T21:46:57.615 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 22, 2024-09-15T21:46:57.615 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 40.224841048999998 2024-09-15T21:46:57.615 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:46:57.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.607+0000 7f8576e77640 1 -- 172.21.15.100:0/416228807 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f855c06c2e0 msgr2=0x7f855c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.607+0000 7f8576e77640 1 --2- 172.21.15.100:0/416228807 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f855c06c2e0 0x7f855c06e7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8560000f60 tx=0x7f8560009040 comp rx=0 tx=0).stop 2024-09-15T21:46:57.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.607+0000 7f8576e77640 1 -- 172.21.15.100:0/416228807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8570159b80 msgr2=0x7f857013e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.607+0000 7f8576e77640 1 --2- 172.21.15.100:0/416228807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8570159b80 0x7f857013e9f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f856800c590 tx=0x7f856800ca50 comp rx=0 tx=0).stop 2024-09-15T21:46:57.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.607+0000 7f8576e77640 1 -- 172.21.15.100:0/416228807 shutdown_connections 2024-09-15T21:46:57.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.607+0000 7f8576e77640 1 --2- 172.21.15.100:0/416228807 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f855c06c2e0 0x7f855c06e7a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.607+0000 7f8576e77640 1 --2- 172.21.15.100:0/416228807 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85700c9ba0 0x7f857013e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.607+0000 7f8576e77640 1 --2- 172.21.15.100:0/416228807 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8570159b80 0x7f857013e9f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.607+0000 7f8576e77640 1 --2- 172.21.15.100:0/416228807 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f85700c97c0 0x7f85700c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.607+0000 7f8576e77640 1 -- 172.21.15.100:0/416228807 >> 172.21.15.100:0/416228807 conn(0x7f857006c210 msgr2=0x7f8570156b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:57.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.607+0000 7f8576e77640 1 -- 172.21.15.100:0/416228807 shutdown_connections 2024-09-15T21:46:57.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.607+0000 7f8576e77640 1 -- 172.21.15.100:0/416228807 wait complete. 2024-09-15T21:46:57.623 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-09-15T21:46:57.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.727+0000 7f796eb3b640 1 Processor -- start 2024-09-15T21:46:57.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.727+0000 7f796eb3b640 1 -- start start 2024-09-15T21:46:57.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.728+0000 7f796eb3b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f79680cca30 0x7f7968071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.728+0000 7f796eb3b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7968070ee0 0x7f7968151450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.728+0000 7f796eb3b640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79680c91f0 con 0x7f79680cce10 2024-09-15T21:46:57.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.728+0000 7f796eb3b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f79680c94d0 con 0x7f7968070ee0 2024-09-15T21:46:57.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.728+0000 7f796eb3b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f79680c97b0 con 0x7f79680cca30 2024-09-15T21:46:57.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.728+0000 7f7967fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7968070ee0 0x7f7968151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.728+0000 7f7967fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7968070ee0 0x7f7968151450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48602/0 (socket says 172.21.15.100:48602) 2024-09-15T21:46:57.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.728+0000 7f7967fff640 1 -- 172.21.15.100:0/647643437 learned_addr learned my addr 172.21.15.100:0/647643437 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:57.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.728+0000 7f796c8d7640 1 --2- 172.21.15.100:0/647643437 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f79680cca30 0x7f7968071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.728+0000 7f7967fff640 1 -- 172.21.15.100:0/647643437 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f79680cca30 msgr2=0x7f7968071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.728+0000 7f7967fff640 1 --2- 172.21.15.100:0/647643437 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f79680cca30 0x7f7968071340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.728+0000 7f7967fff640 1 -- 172.21.15.100:0/647643437 >> v1:172.21.15.122:6789/0 conn(0x7f79680cce10 legacy=0x7f796815af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.728+0000 7f7967fff640 1 -- 172.21.15.100:0/647643437 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79680c9a90 con 0x7f7968070ee0 2024-09-15T21:46:57.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.729+0000 7f7967fff640 1 --2- 172.21.15.100:0/647643437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7968070ee0 0x7f7968151450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f795c00aa10 tx=0x7f795c02b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f22a88e9c46fc76 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:57.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.729+0000 7f79677fe640 1 -- 172.21.15.100:0/647643437 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c00b040 con 0x7f7968070ee0 2024-09-15T21:46:57.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.729+0000 7f79677fe640 1 -- 172.21.15.100:0/647643437 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f795c030710 con 0x7f7968070ee0 2024-09-15T21:46:57.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.729+0000 7f79677fe640 1 -- 172.21.15.100:0/647643437 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c030ae0 con 0x7f7968070ee0 2024-09-15T21:46:57.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.730+0000 7f796eb3b640 1 -- 172.21.15.100:0/647643437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7968070ee0 msgr2=0x7f7968151450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.730+0000 7f796eb3b640 1 --2- 172.21.15.100:0/647643437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7968070ee0 0x7f7968151450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f795c00aa10 tx=0x7f795c02b650 comp rx=0 tx=0).stop 2024-09-15T21:46:57.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.730+0000 7f796eb3b640 1 -- 172.21.15.100:0/647643437 shutdown_connections 2024-09-15T21:46:57.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.730+0000 7f796eb3b640 1 --2- 172.21.15.100:0/647643437 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7968070ee0 0x7f7968151450 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.730+0000 7f796eb3b640 1 --2- 172.21.15.100:0/647643437 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f79680cca30 0x7f7968071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.730+0000 7f796eb3b640 1 -- 172.21.15.100:0/647643437 >> 172.21.15.100:0/647643437 conn(0x7f7968067dd0 msgr2=0x7f796806bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:57.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.730+0000 7f796eb3b640 1 -- 172.21.15.100:0/647643437 shutdown_connections 2024-09-15T21:46:57.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.730+0000 7f796eb3b640 1 -- 172.21.15.100:0/647643437 wait complete. 2024-09-15T21:46:57.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.731+0000 7f796eb3b640 1 Processor -- start 2024-09-15T21:46:57.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.731+0000 7f796eb3b640 1 -- start start 2024-09-15T21:46:57.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.732+0000 7f796eb3b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7968070ee0 0x7f79681ef590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.732+0000 7f796eb3b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79680cca30 0x7f79681501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.732+0000 7f796eb3b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79680cce10 0x7f796814af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.732+0000 7f796eb3b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f796815b7e0 con 0x7f79680cca30 2024-09-15T21:46:57.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.732+0000 7f796eb3b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f796815b660 con 0x7f79680cce10 2024-09-15T21:46:57.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.732+0000 7f796eb3b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f79681f3180 con 0x7f7968070ee0 2024-09-15T21:46:57.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.732+0000 7f7967fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79680cca30 0x7f79681501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.732+0000 7f7967fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79680cca30 0x7f79681501a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48614/0 (socket says 172.21.15.100:48614) 2024-09-15T21:46:57.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.732+0000 7f7967fff640 1 -- 172.21.15.100:0/651566295 learned_addr learned my addr 172.21.15.100:0/651566295 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:57.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.732+0000 7f796c8d7640 1 --2- 172.21.15.100:0/651566295 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7968070ee0 0x7f79681ef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.732+0000 7f7967fff640 1 -- 172.21.15.100:0/651566295 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7968070ee0 msgr2=0x7f79681ef590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.732+0000 7f7967fff640 1 --2- 172.21.15.100:0/651566295 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7968070ee0 0x7f79681ef590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.732+0000 7f796d0d8640 1 --2- 172.21.15.100:0/651566295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79680cce10 0x7f796814af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.732+0000 7f7967fff640 1 -- 172.21.15.100:0/651566295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79680cce10 msgr2=0x7f796814af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.732+0000 7f7967fff640 1 --2- 172.21.15.100:0/651566295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79680cce10 0x7f796814af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.732+0000 7f7967fff640 1 -- 172.21.15.100:0/651566295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79681eedd0 con 0x7f79680cca30 2024-09-15T21:46:57.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.732+0000 7f7967fff640 1 --2- 172.21.15.100:0/651566295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79680cca30 0x7f79681501a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f795c000c00 tx=0x7f795c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:57.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.733+0000 7f79657fa640 1 -- 172.21.15.100:0/651566295 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c03f070 con 0x7f79680cca30 2024-09-15T21:46:57.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.733+0000 7f79657fa640 1 -- 172.21.15.100:0/651566295 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f795c030e30 con 0x7f79680cca30 2024-09-15T21:46:57.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.733+0000 7f796eb3b640 1 -- 172.21.15.100:0/651566295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7968150e20 con 0x7f79680cca30 2024-09-15T21:46:57.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.733+0000 7f79657fa640 1 -- 172.21.15.100:0/651566295 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c03b430 con 0x7f79680cca30 2024-09-15T21:46:57.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.733+0000 7f796eb3b640 1 -- 172.21.15.100:0/651566295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7968156bf0 con 0x7f79680cca30 2024-09-15T21:46:57.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.734+0000 7f79657fa640 1 -- 172.21.15.100:0/651566295 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f795c042070 con 0x7f79680cca30 2024-09-15T21:46:57.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.735+0000 7f79657fa640 1 --2- 172.21.15.100:0/651566295 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f795406c2b0 0x7f795406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.735+0000 7f796c8d7640 1 --2- 172.21.15.100:0/651566295 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f795406c2b0 0x7f795406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.735+0000 7f79657fa640 1 -- 172.21.15.100:0/651566295 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f795c0ac010 con 0x7f79680cca30 2024-09-15T21:46:57.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.735+0000 7f796c8d7640 1 --2- 172.21.15.100:0/651566295 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f795406c2b0 0x7f795406e770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7958006fd0 tx=0x7f7958008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:57.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.736+0000 7f79718d5740 1 Processor -- start 2024-09-15T21:46:57.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.736+0000 7f79718d5740 1 -- start start 2024-09-15T21:46:57.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.736+0000 7f79718d5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56550133c750 0x5655013749e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.736+0000 7f79718d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565501339f80 0x565501374f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.736+0000 7f79718d5740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56550133cb30 0x565501338a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.736+0000 7f79718d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56550129d520 con 0x565501339f80 2024-09-15T21:46:57.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.736+0000 7f79718d5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5655012453e0 con 0x56550133c750 2024-09-15T21:46:57.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.736+0000 7f79718d5740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x5655012994e0 con 0x56550133cb30 2024-09-15T21:46:57.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.737+0000 7f7967fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565501339f80 0x565501374f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.737+0000 7f7967fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565501339f80 0x565501374f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48626/0 (socket says 172.21.15.100:48626) 2024-09-15T21:46:57.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.737+0000 7f7967fff640 1 -- 172.21.15.100:0/3504216576 learned_addr learned my addr 172.21.15.100:0/3504216576 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:57.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.737+0000 7f796c8d7640 1 --2- 172.21.15.100:0/3504216576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56550133c750 0x5655013749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.737+0000 7f796d0d8640 1 --2- 172.21.15.100:0/3504216576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56550133cb30 0x565501338a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.737+0000 7f7967fff640 1 -- 172.21.15.100:0/3504216576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56550133cb30 msgr2=0x565501338a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.737+0000 7f7967fff640 1 --2- 172.21.15.100:0/3504216576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56550133cb30 0x565501338a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.737+0000 7f7967fff640 1 -- 172.21.15.100:0/3504216576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56550133c750 msgr2=0x5655013749e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.737+0000 7f7967fff640 1 --2- 172.21.15.100:0/3504216576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56550133c750 0x5655013749e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.737+0000 7f7967fff640 1 -- 172.21.15.100:0/3504216576 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5655012602f0 con 0x565501339f80 2024-09-15T21:46:57.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.737+0000 7f7967fff640 1 --2- 172.21.15.100:0/3504216576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565501339f80 0x565501374f20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f795c00a1c0 tx=0x7f795c079a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14efc79640720c22 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:57.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.737+0000 7f794e7fc640 1 -- 172.21.15.100:0/3504216576 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c049040 con 0x565501339f80 2024-09-15T21:46:57.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.737+0000 7f794e7fc640 1 -- 172.21.15.100:0/3504216576 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f795c07e4c0 con 0x565501339f80 2024-09-15T21:46:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.737+0000 7f794e7fc640 1 -- 172.21.15.100:0/3504216576 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795c082cf0 con 0x565501339f80 2024-09-15T21:46:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.738+0000 7f79718d5740 1 -- 172.21.15.100:0/3504216576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565501339f80 msgr2=0x565501374f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.738+0000 7f79718d5740 1 --2- 172.21.15.100:0/3504216576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565501339f80 0x565501374f20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f795c00a1c0 tx=0x7f795c079a70 comp rx=0 tx=0).stop 2024-09-15T21:46:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.738+0000 7f79718d5740 1 -- 172.21.15.100:0/3504216576 shutdown_connections 2024-09-15T21:46:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.738+0000 7f79718d5740 1 --2- 172.21.15.100:0/3504216576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56550133cb30 0x565501338a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.738+0000 7f79718d5740 1 --2- 172.21.15.100:0/3504216576 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x565501339f80 0x565501374f20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.738+0000 7f79718d5740 1 --2- 172.21.15.100:0/3504216576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56550133c750 0x5655013749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.738+0000 7f79718d5740 1 -- 172.21.15.100:0/3504216576 >> 172.21.15.100:0/3504216576 conn(0x56550133ae70 msgr2=0x56550133b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.738+0000 7f79718d5740 1 -- 172.21.15.100:0/3504216576 shutdown_connections 2024-09-15T21:46:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.738+0000 7f79718d5740 1 -- 172.21.15.100:0/3504216576 wait complete. 2024-09-15T21:46:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.738+0000 7f79718d5740 1 -- start start 2024-09-15T21:46:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.739+0000 7f79718d5740 1 Processor -- start 2024-09-15T21:46:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.739+0000 7f79718d5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565501339f80 0x565501364840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.739+0000 7f79718d5740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56550133c750 0x56550136c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.739+0000 7f79718d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 0x56550136ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.739+0000 7f79718d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5655012453e0 con 0x56550133cb30 2024-09-15T21:46:57.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.739+0000 7f79718d5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x565501226450 con 0x565501339f80 2024-09-15T21:46:57.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.739+0000 7f79718d5740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x56550129d520 con 0x56550133c750 2024-09-15T21:46:57.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.739+0000 7f79718d5740 10 client.0 _ll_register_callbacks cb 0x565500fafac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T21:46:57.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.739+0000 7f796d0d8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 0x56550136ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.739+0000 7f796d0d8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 0x56550136ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48636/0 (socket says 172.21.15.100:48636) 2024-09-15T21:46:57.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.739+0000 7f796d0d8640 1 -- 172.21.15.100:0/3565722413 learned_addr learned my addr 172.21.15.100:0/3565722413 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:57.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.739+0000 7f796c8d7640 1 --2- 172.21.15.100:0/3565722413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565501339f80 0x565501364840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.739+0000 7f7967fff640 1 --2- 172.21.15.100:0/3565722413 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56550133c750 0x56550136c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f796d0d8640 1 -- 172.21.15.100:0/3565722413 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56550133c750 msgr2=0x56550136c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f796d0d8640 1 --2- 172.21.15.100:0/3565722413 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56550133c750 0x56550136c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f796d0d8640 1 -- 172.21.15.100:0/3565722413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565501339f80 msgr2=0x565501364840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f796d0d8640 1 --2- 172.21.15.100:0/3565722413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565501339f80 0x565501364840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f796d0d8640 1 -- 172.21.15.100:0/3565722413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565500fb02d0 con 0x56550133cb30 2024-09-15T21:46:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f796d0d8640 1 --2- 172.21.15.100:0/3565722413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 0x56550136ed50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f796000e890 tx=0x7f796000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0ad9d9469f0fee server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f79477fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-15T21:46:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f79477fe640 1 -- 172.21.15.100:0/3565722413 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7960018020 con 0x56550133cb30 2024-09-15T21:46:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f79477fe640 1 -- 172.21.15.100:0/3565722413 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f796000cb60 con 0x56550133cb30 2024-09-15T21:46:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f79718d5740 1 -- 172.21.15.100:0/3565722413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f795c07e4c0 con 0x56550133cb30 2024-09-15T21:46:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f79477fe640 1 -- 172.21.15.100:0/3565722413 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f7960022050 con 0x56550133cb30 2024-09-15T21:46:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f79718d5740 10 client.4707 fetch_fsmap learned FSMap version 17 2024-09-15T21:46:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f79718d5740 1 -- 172.21.15.100:0/3565722413 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x56550107a6b0 con 0x56550133cb30 2024-09-15T21:46:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f79477fe640 1 -- 172.21.15.100:0/3565722413 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f795c07e4c0 con 0x56550133cb30 2024-09-15T21:46:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f79477fe640 1 -- 172.21.15.100:0/3565722413 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 17) ==== 4792+0+0 (secure 0 0 0) 0x7f7960003970 con 0x56550133cb30 2024-09-15T21:46:57.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f79718d5740 10 client.4707 fetch_fsmap finished waiting for FSMap version 17 2024-09-15T21:46:57.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f79718d5740 10 client.4707 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-15T21:46:57.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f79718d5740 20 client.4707 populate_metadata read hostname 'smithi100' 2024-09-15T21:46:57.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f79718d5740 1 --2- 172.21.15.100:0/3565722413 >> [v2:172.21.15.122:6834/371886422,v1:172.21.15.122:6837/371886422] conn(0x56550136f4e0 0x56550139a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f79718d5740 4 client.4707 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:46:57.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.740+0000 7f79718d5740 1 -- 172.21.15.100:0/3565722413 --> [v2:172.21.15.122:6834/371886422,v1:172.21.15.122:6837/371886422] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5655012aac50 con 0x56550136f4e0 2024-09-15T21:46:57.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.741+0000 7f796c8d7640 1 --2- 172.21.15.100:0/3565722413 >> [v2:172.21.15.122:6834/371886422,v1:172.21.15.122:6837/371886422] conn(0x56550136f4e0 0x56550139a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.741+0000 7f79477fe640 10 client.4707 ms_handle_connect on v2:172.21.15.122:6834/371886422 2024-09-15T21:46:57.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.741+0000 7f796c8d7640 1 --2- 172.21.15.100:0/3565722413 >> [v2:172.21.15.122:6834/371886422,v1:172.21.15.122:6837/371886422] conn(0x56550136f4e0 0x56550139a340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=52432480231767fa server_cookie=13b94b8985378821 in_seq=0 out_seq=0 2024-09-15T21:46:57.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.742+0000 7f79477fe640 1 -- 172.21.15.100:0/3565722413 <== mds.3 v2:172.21.15.122:6834/371886422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5655012aac50 con 0x56550136f4e0 2024-09-15T21:46:57.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.742+0000 7f79477fe640 10 client.4707 handle_command_reply: tid=0 2024-09-15T21:46:57.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.742+0000 7f79718d5740 1 client.4707 shutdown 2024-09-15T21:46:57.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.742+0000 7f79718d5740 1 -- 172.21.15.100:0/3565722413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 msgr2=0x56550136ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.742+0000 7f79718d5740 1 --2- 172.21.15.100:0/3565722413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 0x56550136ed50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f796000e890 tx=0x7f796000ed50 comp rx=0 tx=0).stop 2024-09-15T21:46:57.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.742+0000 7f79718d5740 1 -- 172.21.15.100:0/3565722413 shutdown_connections 2024-09-15T21:46:57.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.742+0000 7f79718d5740 1 --2- 172.21.15.100:0/3565722413 >> [v2:172.21.15.122:6834/371886422,v1:172.21.15.122:6837/371886422] conn(0x56550136f4e0 0x56550139a340 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.742+0000 7f79718d5740 1 --2- 172.21.15.100:0/3565722413 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 0x56550136ed50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.742+0000 7f79718d5740 1 --2- 172.21.15.100:0/3565722413 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56550133c750 0x56550136c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.742+0000 7f79477fe640 0 client.4707 ms_handle_reset on v2:172.21.15.122:6834/371886422 2024-09-15T21:46:57.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.742+0000 7f79718d5740 1 --2- 172.21.15.100:0/3565722413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565501339f80 0x565501364840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.742+0000 7f79718d5740 1 -- 172.21.15.100:0/3565722413 >> 172.21.15.100:0/3565722413 conn(0x56550133ae70 msgr2=0x5655013357c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:57.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.742+0000 7f79718d5740 1 -- 172.21.15.100:0/3565722413 shutdown_connections 2024-09-15T21:46:57.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.742+0000 7f79718d5740 1 -- 172.21.15.100:0/3565722413 wait complete. 2024-09-15T21:46:57.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.742+0000 7f79718d5740 20 client.4707 trim_cache size 0 max 16384 2024-09-15T21:46:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.759+0000 7f79718d5740 1 Processor -- start 2024-09-15T21:46:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.760+0000 7f79718d5740 1 -- start start 2024-09-15T21:46:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.760+0000 7f79718d5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565501339f80 0x56550133b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.760+0000 7f79718d5740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56550133c750 0x56550133b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.760+0000 7f79718d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 0x565501372ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.760+0000 7f79718d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f79681fb4c0 con 0x56550133cb30 2024-09-15T21:46:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.760+0000 7f79718d5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f796814b880 con 0x565501339f80 2024-09-15T21:46:57.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.760+0000 7f79718d5740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x565501360ee0 con 0x56550133c750 2024-09-15T21:46:57.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.760+0000 7f796d0d8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 0x565501372ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.760+0000 7f796d0d8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 0x565501372ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48644/0 (socket says 172.21.15.100:48644) 2024-09-15T21:46:57.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.760+0000 7f796d0d8640 1 -- 172.21.15.100:0/4207902005 learned_addr learned my addr 172.21.15.100:0/4207902005 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:57.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.761+0000 7f796d0d8640 1 -- 172.21.15.100:0/4207902005 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56550133c750 msgr2=0x56550133b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.761+0000 7f796d0d8640 1 --2- 172.21.15.100:0/4207902005 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56550133c750 0x56550133b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.761+0000 7f796d0d8640 1 -- 172.21.15.100:0/4207902005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565501339f80 msgr2=0x56550133b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.761+0000 7f796d0d8640 1 --2- 172.21.15.100:0/4207902005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565501339f80 0x56550133b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.761+0000 7f796d0d8640 1 -- 172.21.15.100:0/4207902005 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56550133d140 con 0x56550133cb30 2024-09-15T21:46:57.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.761+0000 7f796d0d8640 1 --2- 172.21.15.100:0/4207902005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 0x565501372ad0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7960000f80 tx=0x7f7960014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d59d7e21571bf25 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:57.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.761+0000 7f794dffb640 1 -- 172.21.15.100:0/4207902005 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7960018030 con 0x56550133cb30 2024-09-15T21:46:57.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.761+0000 7f794dffb640 1 -- 172.21.15.100:0/4207902005 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f796000ee80 con 0x56550133cb30 2024-09-15T21:46:57.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.761+0000 7f794dffb640 1 -- 172.21.15.100:0/4207902005 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796000d040 con 0x56550133cb30 2024-09-15T21:46:57.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.761+0000 7f79718d5740 1 -- 172.21.15.100:0/4207902005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 msgr2=0x565501372ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.761+0000 7f79718d5740 1 --2- 172.21.15.100:0/4207902005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 0x565501372ad0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7960000f80 tx=0x7f7960014000 comp rx=0 tx=0).stop 2024-09-15T21:46:57.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.761+0000 7f79718d5740 1 -- 172.21.15.100:0/4207902005 shutdown_connections 2024-09-15T21:46:57.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.761+0000 7f79718d5740 1 --2- 172.21.15.100:0/4207902005 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 0x565501372ad0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.761+0000 7f79718d5740 1 --2- 172.21.15.100:0/4207902005 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x56550133c750 0x56550133b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.761+0000 7f79718d5740 1 --2- 172.21.15.100:0/4207902005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x565501339f80 0x56550133b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.761+0000 7f79718d5740 1 -- 172.21.15.100:0/4207902005 >> 172.21.15.100:0/4207902005 conn(0x56550133ae70 msgr2=0x56550136f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:57.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.761+0000 7f79718d5740 1 -- 172.21.15.100:0/4207902005 shutdown_connections 2024-09-15T21:46:57.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.761+0000 7f79718d5740 1 -- 172.21.15.100:0/4207902005 wait complete. 2024-09-15T21:46:57.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.762+0000 7f79718d5740 1 -- start start 2024-09-15T21:46:57.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.762+0000 7f79718d5740 1 Processor -- start 2024-09-15T21:46:57.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f79718d5740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x565501339f80 0x5655013f7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f79718d5740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56550133c750 0x56550137ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f79718d5740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 0x56550137efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f79718d5740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x56550136f360 con 0x56550133cb30 2024-09-15T21:46:57.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f79718d5740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x565501362c40 con 0x56550133c750 2024-09-15T21:46:57.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f79718d5740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x56550138b780 con 0x565501339f80 2024-09-15T21:46:57.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f79718d5740 10 client.0 _ll_register_callbacks cb 0x7f7960007540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T21:46:57.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f796d0d8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 0x56550137efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f796d0d8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 0x56550137efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48658/0 (socket says 172.21.15.100:48658) 2024-09-15T21:46:57.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f796d0d8640 1 -- 172.21.15.100:0/240026459 learned_addr learned my addr 172.21.15.100:0/240026459 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:57.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f796c8d7640 1 --2- 172.21.15.100:0/240026459 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x565501339f80 0x5655013f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f7967fff640 1 --2- 172.21.15.100:0/240026459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56550133c750 0x56550137ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f796d0d8640 1 -- 172.21.15.100:0/240026459 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x565501339f80 msgr2=0x5655013f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f796d0d8640 1 --2- 172.21.15.100:0/240026459 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x565501339f80 0x5655013f7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f796d0d8640 1 -- 172.21.15.100:0/240026459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56550133c750 msgr2=0x56550137ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f796d0d8640 1 --2- 172.21.15.100:0/240026459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56550133c750 0x56550137ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f796d0d8640 1 -- 172.21.15.100:0/240026459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565501335a70 con 0x56550133cb30 2024-09-15T21:46:57.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f7967fff640 1 --2- 172.21.15.100:0/240026459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56550133c750 0x56550137ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:57.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f796d0d8640 1 --2- 172.21.15.100:0/240026459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 0x56550137efd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7960000f80 tx=0x7f7960004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a693b1883a8ca15e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:57.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.763+0000 7f79477fe640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-15T21:46:57.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.764+0000 7f79477fe640 1 -- 172.21.15.100:0/240026459 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796000e680 con 0x56550133cb30 2024-09-15T21:46:57.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.764+0000 7f79477fe640 1 -- 172.21.15.100:0/240026459 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f796000c960 con 0x56550133cb30 2024-09-15T21:46:57.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.764+0000 7f79718d5740 1 -- 172.21.15.100:0/240026459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5655012a4fa0 con 0x56550133cb30 2024-09-15T21:46:57.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.764+0000 7f79477fe640 1 -- 172.21.15.100:0/240026459 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f7960041050 con 0x56550133cb30 2024-09-15T21:46:57.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.764+0000 7f79718d5740 10 client.4713 fetch_fsmap learned FSMap version 17 2024-09-15T21:46:57.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.764+0000 7f79718d5740 1 -- 172.21.15.100:0/240026459 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x56550136e1e0 con 0x56550133cb30 2024-09-15T21:46:57.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.764+0000 7f79477fe640 1 -- 172.21.15.100:0/240026459 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5655012a4fa0 con 0x56550133cb30 2024-09-15T21:46:57.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.764+0000 7f79477fe640 1 -- 172.21.15.100:0/240026459 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 17) ==== 4792+0+0 (secure 0 0 0) 0x7f7960044020 con 0x56550133cb30 2024-09-15T21:46:57.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.764+0000 7f79718d5740 10 client.4713 fetch_fsmap finished waiting for FSMap version 17 2024-09-15T21:46:57.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.764+0000 7f79718d5740 10 client.4713 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-15T21:46:57.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.764+0000 7f79718d5740 20 client.4713 populate_metadata read hostname 'smithi100' 2024-09-15T21:46:57.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.764+0000 7f79718d5740 1 --2- 172.21.15.100:0/240026459 >> [v2:172.21.15.122:6834/371886422,v1:172.21.15.122:6837/371886422] conn(0x56550137de30 0x56550137e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.764+0000 7f79718d5740 4 client.4713 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T21:46:57.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.764+0000 7f79718d5740 1 -- 172.21.15.100:0/240026459 --> [v2:172.21.15.122:6834/371886422,v1:172.21.15.122:6837/371886422] -- command(tid 0: {"prefix": "status"}) -- 0x565501221b80 con 0x56550137de30 2024-09-15T21:46:57.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.764+0000 7f796c8d7640 1 --2- 172.21.15.100:0/240026459 >> [v2:172.21.15.122:6834/371886422,v1:172.21.15.122:6837/371886422] conn(0x56550137de30 0x56550137e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.765+0000 7f79477fe640 10 client.4713 ms_handle_connect on v2:172.21.15.122:6834/371886422 2024-09-15T21:46:57.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.765+0000 7f796c8d7640 1 --2- 172.21.15.100:0/240026459 >> [v2:172.21.15.122:6834/371886422,v1:172.21.15.122:6837/371886422] conn(0x56550137de30 0x56550137e210 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c7091a08b0f73c9b server_cookie=455f062bada158d4 in_seq=0 out_seq=0 2024-09-15T21:46:57.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.765+0000 7f79477fe640 1 -- 172.21.15.100:0/240026459 <== mds.3 v2:172.21.15.122:6834/371886422 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x565501221b80 con 0x56550137de30 2024-09-15T21:46:57.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.765+0000 7f79477fe640 10 client.4713 handle_command_reply: tid=0 2024-09-15T21:46:57.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.765+0000 7f79718d5740 1 client.4713 shutdown 2024-09-15T21:46:57.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.765+0000 7f79718d5740 1 -- 172.21.15.100:0/240026459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 msgr2=0x56550137efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.765+0000 7f79718d5740 1 --2- 172.21.15.100:0/240026459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 0x56550137efd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7960000f80 tx=0x7f7960004a70 comp rx=0 tx=0).stop 2024-09-15T21:46:57.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.765+0000 7f79718d5740 1 -- 172.21.15.100:0/240026459 shutdown_connections 2024-09-15T21:46:57.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.765+0000 7f79718d5740 1 --2- 172.21.15.100:0/240026459 >> [v2:172.21.15.122:6834/371886422,v1:172.21.15.122:6837/371886422] conn(0x56550137de30 0x56550137e210 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.765+0000 7f79718d5740 1 --2- 172.21.15.100:0/240026459 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56550133c750 0x56550137ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.765+0000 7f79718d5740 1 --2- 172.21.15.100:0/240026459 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56550133cb30 0x56550137efd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.765+0000 7f79477fe640 0 client.4713 ms_handle_reset on v2:172.21.15.122:6834/371886422 2024-09-15T21:46:57.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.765+0000 7f79718d5740 1 --2- 172.21.15.100:0/240026459 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x565501339f80 0x5655013f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.765+0000 7f79718d5740 1 -- 172.21.15.100:0/240026459 >> 172.21.15.100:0/240026459 conn(0x56550133ae70 msgr2=0x56550133b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:57.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.765+0000 7f79718d5740 1 -- 172.21.15.100:0/240026459 shutdown_connections 2024-09-15T21:46:57.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.765+0000 7f79718d5740 1 -- 172.21.15.100:0/240026459 wait complete. 2024-09-15T21:46:57.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.765+0000 7f79718d5740 20 client.4713 trim_cache size 0 max 16384 2024-09-15T21:46:57.774 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:46:57.774 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "30e2fff4-2171-4531-8806-0e03b63f494f", 2024-09-15T21:46:57.775 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 3, 2024-09-15T21:46:57.775 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4288, 2024-09-15T21:46:57.775 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-09-15T21:46:57.775 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-09-15T21:46:57.775 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-09-15T21:46:57.775 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 17.225038260000002, 2024-09-15T21:46:57.775 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 17, 2024-09-15T21:46:57.775 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 22, 2024-09-15T21:46:57.775 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 22, 2024-09-15T21:46:57.775 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 40.367802388000001 2024-09-15T21:46:57.775 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:46:57.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.766+0000 7f796eb3b640 1 -- 172.21.15.100:0/651566295 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f795406c2b0 msgr2=0x7f795406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.766+0000 7f796eb3b640 1 --2- 172.21.15.100:0/651566295 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f795406c2b0 0x7f795406e770 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7958006fd0 tx=0x7f7958008040 comp rx=0 tx=0).stop 2024-09-15T21:46:57.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.766+0000 7f796eb3b640 1 -- 172.21.15.100:0/651566295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79680cca30 msgr2=0x7f79681501a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.766+0000 7f796eb3b640 1 --2- 172.21.15.100:0/651566295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79680cca30 0x7f79681501a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f795c000c00 tx=0x7f795c0046c0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.766+0000 7f796eb3b640 1 -- 172.21.15.100:0/651566295 shutdown_connections 2024-09-15T21:46:57.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.766+0000 7f796eb3b640 1 --2- 172.21.15.100:0/651566295 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f795406c2b0 0x7f795406e770 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.766+0000 7f796eb3b640 1 --2- 172.21.15.100:0/651566295 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79680cce10 0x7f796814af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.766+0000 7f796eb3b640 1 --2- 172.21.15.100:0/651566295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79680cca30 0x7f79681501a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.766+0000 7f796eb3b640 1 --2- 172.21.15.100:0/651566295 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7968070ee0 0x7f79681ef590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.766+0000 7f796eb3b640 1 -- 172.21.15.100:0/651566295 >> 172.21.15.100:0/651566295 conn(0x7f7968067dd0 msgr2=0x7f79680c44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:57.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.766+0000 7f796eb3b640 1 -- 172.21.15.100:0/651566295 shutdown_connections 2024-09-15T21:46:57.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.766+0000 7f796eb3b640 1 -- 172.21.15.100:0/651566295 wait complete. 2024-09-15T21:46:57.778 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-15T21:46:57.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.876+0000 7f9399d3b640 1 Processor -- start 2024-09-15T21:46:57.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.876+0000 7f9399d3b640 1 -- start start 2024-09-15T21:46:57.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f9399d3b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93941522e0 0x7f93940c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f9399d3b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9394157ef0 con 0x7f93940c92d0 2024-09-15T21:46:57.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f9399d3b640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93941581b0 con 0x7f93940c96b0 2024-09-15T21:46:57.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f9399d3b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f93940c4980 con 0x7f93941522e0 2024-09-15T21:46:57.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f93937fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f93940c92d0 0x7f9394151bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:40716/0 (socket says 172.21.15.100:40716) 2024-09-15T21:46:57.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f93937fe640 1 -- 172.21.15.100:0/1936704035 learned_addr learned my addr 172.21.15.100:0/1936704035 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:46:57.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f93927fc640 1 -- 172.21.15.100:0/1936704035 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3826895991 0 0) 0x7f9394157ef0 con 0x7f93940c92d0 2024-09-15T21:46:57.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f9393fff640 1 --2- 172.21.15.100:0/1936704035 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93941522e0 0x7f93940c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f93927fc640 1 -- 172.21.15.100:0/1936704035 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f937c003520 con 0x7f93940c92d0 2024-09-15T21:46:57.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f93927fc640 1 -- 172.21.15.100:0/1936704035 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2618921744 0 0) 0x7f937c003520 con 0x7f93940c92d0 2024-09-15T21:46:57.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f93927fc640 1 -- 172.21.15.100:0/1936704035 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93941522e0 msgr2=0x7f93940c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f93927fc640 1 --2- 172.21.15.100:0/1936704035 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93941522e0 0x7f93940c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f93927fc640 1 -- 172.21.15.100:0/1936704035 >> v1:172.21.15.122:6789/0 conn(0x7f93940c96b0 legacy=0x7f93940c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f93927fc640 1 -- 172.21.15.100:0/1936704035 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9394158490 con 0x7f93940c92d0 2024-09-15T21:46:57.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f93927fc640 1 -- 172.21.15.100:0/1936704035 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7f9384001670 con 0x7f93940c92d0 2024-09-15T21:46:57.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f93927fc640 1 -- 172.21.15.100:0/1936704035 >> v1:172.21.15.100:6789/0 conn(0x7f93940c92d0 legacy=0x7f9394151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f93927fc640 1 --2- 172.21.15.100:0/1936704035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f937c0039e0 0x7f937c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f93927fc640 1 --2- 172.21.15.100:0/1936704035 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93941522e0 0x7f937c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f93927fc640 1 --2- 172.21.15.100:0/1936704035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f937c004d40 0x7f937c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f93927fc640 1 -- 172.21.15.100:0/1936704035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f937c00bb50 con 0x7f937c004d40 2024-09-15T21:46:57.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f93927fc640 1 -- 172.21.15.100:0/1936704035 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f937c00bcd0 con 0x7f937c0039e0 2024-09-15T21:46:57.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.877+0000 7f93927fc640 1 -- 172.21.15.100:0/1936704035 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f937c00bed0 con 0x7f93941522e0 2024-09-15T21:46:57.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.878+0000 7f9393fff640 1 --2- 172.21.15.100:0/1936704035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f937c004d40 0x7f937c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.878+0000 7f93937fe640 1 --2- 172.21.15.100:0/1936704035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f937c0039e0 0x7f937c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.878+0000 7f9392ffd640 1 --2- 172.21.15.100:0/1936704035 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93941522e0 0x7f937c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.878+0000 7f9393fff640 1 -- 172.21.15.100:0/1936704035 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93941522e0 msgr2=0x7f937c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.878+0000 7f9393fff640 1 --2- 172.21.15.100:0/1936704035 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93941522e0 0x7f937c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.878+0000 7f9393fff640 1 -- 172.21.15.100:0/1936704035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f937c0039e0 msgr2=0x7f937c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.878+0000 7f9393fff640 1 --2- 172.21.15.100:0/1936704035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f937c0039e0 0x7f937c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.878+0000 7f9393fff640 1 -- 172.21.15.100:0/1936704035 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937c003520 con 0x7f937c004d40 2024-09-15T21:46:57.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.878+0000 7f9393fff640 1 --2- 172.21.15.100:0/1936704035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f937c004d40 0x7f937c0051c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f938c009fc0 tx=0x7f937c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81453a624ae55678 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:57.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.879+0000 7f93927fc640 1 -- 172.21.15.100:0/1936704035 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c009c10 con 0x7f937c004d40 2024-09-15T21:46:57.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.879+0000 7f93927fc640 1 -- 172.21.15.100:0/1936704035 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f938c036070 con 0x7f937c004d40 2024-09-15T21:46:57.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.879+0000 7f93927fc640 1 -- 172.21.15.100:0/1936704035 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c031880 con 0x7f937c004d40 2024-09-15T21:46:57.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.879+0000 7f9399d3b640 1 -- 172.21.15.100:0/1936704035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f937c004d40 msgr2=0x7f937c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.879+0000 7f9399d3b640 1 --2- 172.21.15.100:0/1936704035 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f937c004d40 0x7f937c0051c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f938c009fc0 tx=0x7f937c005830 comp rx=0 tx=0).stop 2024-09-15T21:46:57.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.879+0000 7f9393fff640 1 -- 172.21.15.100:0/1936704035 reap_dead start 2024-09-15T21:46:57.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.879+0000 7f9399d3b640 1 -- 172.21.15.100:0/1936704035 shutdown_connections 2024-09-15T21:46:57.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.879+0000 7f9399d3b640 1 -- 172.21.15.100:0/1936704035 >> 172.21.15.100:0/1936704035 conn(0x7f9394067dd0 msgr2=0x7f939406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:57.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.879+0000 7f9399d3b640 1 -- 172.21.15.100:0/1936704035 shutdown_connections 2024-09-15T21:46:57.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.879+0000 7f9399d3b640 1 -- 172.21.15.100:0/1936704035 wait complete. 2024-09-15T21:46:57.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.880+0000 7f9399d3b640 1 Processor -- start 2024-09-15T21:46:57.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.881+0000 7f9399d3b640 1 -- start start 2024-09-15T21:46:57.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.881+0000 7f9399d3b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f937c004d40 0x7f939414e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.881+0000 7f9399d3b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93941e1c10 0x7f93941de0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.881+0000 7f9399d3b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93941de610 0x7f939414afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.881+0000 7f9399d3b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f93940c8d30 con 0x7f93941de610 2024-09-15T21:46:57.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.881+0000 7f9399d3b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9394158a50 con 0x7f937c004d40 2024-09-15T21:46:57.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.881+0000 7f9399d3b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f93940c8eb0 con 0x7f93941e1c10 2024-09-15T21:46:57.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.881+0000 7f9393fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93941de610 0x7f939414afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.881+0000 7f9393fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93941de610 0x7f939414afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48670/0 (socket says 172.21.15.100:48670) 2024-09-15T21:46:57.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.881+0000 7f9393fff640 1 -- 172.21.15.100:0/2256870813 learned_addr learned my addr 172.21.15.100:0/2256870813 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:57.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.881+0000 7f93937fe640 1 --2- 172.21.15.100:0/2256870813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f937c004d40 0x7f939414e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.881+0000 7f9392ffd640 1 --2- 172.21.15.100:0/2256870813 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93941e1c10 0x7f93941de0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.882+0000 7f9393fff640 1 -- 172.21.15.100:0/2256870813 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93941e1c10 msgr2=0x7f93941de0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.882+0000 7f9393fff640 1 --2- 172.21.15.100:0/2256870813 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93941e1c10 0x7f93941de0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.882+0000 7f9393fff640 1 -- 172.21.15.100:0/2256870813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f937c004d40 msgr2=0x7f939414e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.882+0000 7f9393fff640 1 --2- 172.21.15.100:0/2256870813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f937c004d40 0x7f939414e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.882+0000 7f9393fff640 1 -- 172.21.15.100:0/2256870813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939414d2d0 con 0x7f93941de610 2024-09-15T21:46:57.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.882+0000 7f9393fff640 1 --2- 172.21.15.100:0/2256870813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93941de610 0x7f939414afa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f938c00e230 tx=0x7f938c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:57.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.882+0000 7f937bfff640 1 -- 172.21.15.100:0/2256870813 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c009c10 con 0x7f93941de610 2024-09-15T21:46:57.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.882+0000 7f937bfff640 1 -- 172.21.15.100:0/2256870813 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f938c031db0 con 0x7f93941de610 2024-09-15T21:46:57.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.882+0000 7f937bfff640 1 -- 172.21.15.100:0/2256870813 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c003bd0 con 0x7f93941de610 2024-09-15T21:46:57.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.882+0000 7f9399d3b640 1 -- 172.21.15.100:0/2256870813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939414d5b0 con 0x7f93941de610 2024-09-15T21:46:57.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.882+0000 7f9399d3b640 1 -- 172.21.15.100:0/2256870813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f939414db70 con 0x7f93941de610 2024-09-15T21:46:57.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.883+0000 7f937bfff640 1 -- 172.21.15.100:0/2256870813 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f938c005d10 con 0x7f93941de610 2024-09-15T21:46:57.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.884+0000 7f937bfff640 1 --2- 172.21.15.100:0/2256870813 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f938006c2e0 0x7f938006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.884+0000 7f93937fe640 1 --2- 172.21.15.100:0/2256870813 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f938006c2e0 0x7f938006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.884+0000 7f937bfff640 1 -- 172.21.15.100:0/2256870813 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f938c0acb70 con 0x7f93941de610 2024-09-15T21:46:57.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.884+0000 7f93937fe640 1 --2- 172.21.15.100:0/2256870813 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f938006c2e0 0x7f938006e7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9384007c80 tx=0x7f9384001db0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:57.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.885+0000 7f939cad6740 1 Processor -- start 2024-09-15T21:46:57.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.885+0000 7f939cad6740 1 -- start start 2024-09-15T21:46:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.886+0000 7f939cad6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a4750 0x5572161dc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.886+0000 7f939cad6740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a1f80 0x5572161dcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.886+0000 7f939cad6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a4b30 0x5572161a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.886+0000 7f939cad6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x557216105520 con 0x5572161a4750 2024-09-15T21:46:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.886+0000 7f939cad6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5572160ad3e0 con 0x5572161a4b30 2024-09-15T21:46:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.886+0000 7f939cad6740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x5572161014e0 con 0x5572161a1f80 2024-09-15T21:46:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.886+0000 7f93937fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a4750 0x5572161dc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.886+0000 7f93937fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a4750 0x5572161dc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48680/0 (socket says 172.21.15.100:48680) 2024-09-15T21:46:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.886+0000 7f93937fe640 1 -- 172.21.15.100:0/4166816852 learned_addr learned my addr 172.21.15.100:0/4166816852 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.886+0000 7f9393fff640 1 --2- 172.21.15.100:0/4166816852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a4b30 0x5572161a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.886+0000 7f9392ffd640 1 --2- 172.21.15.100:0/4166816852 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a1f80 0x5572161dcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.886+0000 7f93937fe640 1 -- 172.21.15.100:0/4166816852 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a1f80 msgr2=0x5572161dcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.886+0000 7f93937fe640 1 --2- 172.21.15.100:0/4166816852 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a1f80 0x5572161dcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.886+0000 7f93937fe640 1 -- 172.21.15.100:0/4166816852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a4b30 msgr2=0x5572161a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.886+0000 7f93937fe640 1 --2- 172.21.15.100:0/4166816852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a4b30 0x5572161a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.886+0000 7f93937fe640 1 -- 172.21.15.100:0/4166816852 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572160c82f0 con 0x5572161a4750 2024-09-15T21:46:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.886+0000 7f9392ffd640 1 --2- 172.21.15.100:0/4166816852 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a1f80 0x5572161dcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.886+0000 7f93937fe640 1 --2- 172.21.15.100:0/4166816852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a4750 0x5572161dc9e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9384004ca0 tx=0x7f938400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec677a9c49ab09b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.886+0000 7f93797fa640 1 -- 172.21.15.100:0/4166816852 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384028070 con 0x5572161a4750 2024-09-15T21:46:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.887+0000 7f93797fa640 1 -- 172.21.15.100:0/4166816852 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9384004e50 con 0x5572161a4750 2024-09-15T21:46:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.887+0000 7f93797fa640 1 -- 172.21.15.100:0/4166816852 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384015dd0 con 0x5572161a4750 2024-09-15T21:46:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.887+0000 7f939cad6740 1 -- 172.21.15.100:0/4166816852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a4750 msgr2=0x5572161dc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.887+0000 7f939cad6740 1 --2- 172.21.15.100:0/4166816852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a4750 0x5572161dc9e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9384004ca0 tx=0x7f938400c000 comp rx=0 tx=0).stop 2024-09-15T21:46:57.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.887+0000 7f939cad6740 1 -- 172.21.15.100:0/4166816852 shutdown_connections 2024-09-15T21:46:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.887+0000 7f939cad6740 1 --2- 172.21.15.100:0/4166816852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a4b30 0x5572161a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.887+0000 7f939cad6740 1 --2- 172.21.15.100:0/4166816852 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a1f80 0x5572161dcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.887+0000 7f939cad6740 1 --2- 172.21.15.100:0/4166816852 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a4750 0x5572161dc9e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.887+0000 7f939cad6740 1 -- 172.21.15.100:0/4166816852 >> 172.21.15.100:0/4166816852 conn(0x5572161a2e70 msgr2=0x5572161a3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.887+0000 7f939cad6740 1 -- 172.21.15.100:0/4166816852 shutdown_connections 2024-09-15T21:46:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.887+0000 7f939cad6740 1 -- 172.21.15.100:0/4166816852 wait complete. 2024-09-15T21:46:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.888+0000 7f939cad6740 1 -- start start 2024-09-15T21:46:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.888+0000 7f939cad6740 1 Processor -- start 2024-09-15T21:46:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f939cad6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a1f80 0x5572161f4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f939cad6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a4750 0x5572161a1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f939cad6740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a4b30 0x5572161f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f939cad6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5572161a3fe0 con 0x5572161a4750 2024-09-15T21:46:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f939cad6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55721608e450 con 0x5572161a1f80 2024-09-15T21:46:57.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f939cad6740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x5572161dc550 con 0x5572161a4b30 2024-09-15T21:46:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f939cad6740 10 client.0 _ll_register_callbacks cb 0x557215e17ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T21:46:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f9392ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a4750 0x5572161a1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f9392ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a4750 0x5572161a1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48684/0 (socket says 172.21.15.100:48684) 2024-09-15T21:46:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f9392ffd640 1 -- 172.21.15.100:0/4066193567 learned_addr learned my addr 172.21.15.100:0/4066193567 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f93937fe640 1 --2- 172.21.15.100:0/4066193567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a1f80 0x5572161f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f9393fff640 1 --2- 172.21.15.100:0/4066193567 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a4b30 0x5572161f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f9392ffd640 1 -- 172.21.15.100:0/4066193567 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a4b30 msgr2=0x5572161f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f9392ffd640 1 --2- 172.21.15.100:0/4066193567 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a4b30 0x5572161f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f9392ffd640 1 -- 172.21.15.100:0/4066193567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a1f80 msgr2=0x5572161f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f9392ffd640 1 --2- 172.21.15.100:0/4066193567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a1f80 0x5572161f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f9392ffd640 1 -- 172.21.15.100:0/4066193567 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557215e182d0 con 0x5572161a4750 2024-09-15T21:46:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f9392ffd640 1 --2- 172.21.15.100:0/4066193567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a4750 0x5572161a1420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f93940c9ac0 tx=0x7f9388001790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5030316b5739a46d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:57.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f93727fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-15T21:46:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f93727fc640 1 -- 172.21.15.100:0/4066193567 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93880166f0 con 0x5572161a4750 2024-09-15T21:46:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f93727fc640 1 -- 172.21.15.100:0/4066193567 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9388010040 con 0x5572161a4750 2024-09-15T21:46:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f939cad6740 1 -- 172.21.15.100:0/4066193567 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9384004e50 con 0x5572161a4750 2024-09-15T21:46:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f93727fc640 1 -- 172.21.15.100:0/4066193567 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f9388016890 con 0x5572161a4750 2024-09-15T21:46:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f939cad6740 10 client.4725 fetch_fsmap learned FSMap version 17 2024-09-15T21:46:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.889+0000 7f939cad6740 1 -- 172.21.15.100:0/4066193567 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x557215ee26b0 con 0x5572161a4750 2024-09-15T21:46:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.890+0000 7f93727fc640 1 -- 172.21.15.100:0/4066193567 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9384004e50 con 0x5572161a4750 2024-09-15T21:46:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.890+0000 7f93727fc640 1 -- 172.21.15.100:0/4066193567 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 17) ==== 4792+0+0 (secure 0 0 0) 0x7f9388016a30 con 0x5572161a4750 2024-09-15T21:46:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.890+0000 7f939cad6740 10 client.4725 fetch_fsmap finished waiting for FSMap version 17 2024-09-15T21:46:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.890+0000 7f939cad6740 10 client.4725 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-15T21:46:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.890+0000 7f939cad6740 20 client.4725 populate_metadata read hostname 'smithi100' 2024-09-15T21:46:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.890+0000 7f939cad6740 1 --2- 172.21.15.100:0/4066193567 >> [v2:172.21.15.100:6836/276075809,v1:172.21.15.100:6838/276075809] conn(0x5572161d58a0 0x5572161c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.890+0000 7f939cad6740 4 client.4725 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:46:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.890+0000 7f939cad6740 1 -- 172.21.15.100:0/4066193567 --> [v2:172.21.15.100:6836/276075809,v1:172.21.15.100:6838/276075809] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557216112c50 con 0x5572161d58a0 2024-09-15T21:46:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.890+0000 7f93937fe640 1 --2- 172.21.15.100:0/4066193567 >> [v2:172.21.15.100:6836/276075809,v1:172.21.15.100:6838/276075809] conn(0x5572161d58a0 0x5572161c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.890+0000 7f93727fc640 10 client.4725 ms_handle_connect on v2:172.21.15.100:6836/276075809 2024-09-15T21:46:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.890+0000 7f93937fe640 1 --2- 172.21.15.100:0/4066193567 >> [v2:172.21.15.100:6836/276075809,v1:172.21.15.100:6838/276075809] conn(0x5572161d58a0 0x5572161c4de0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45ef63b6373a42f1 server_cookie=f0d18b5dd6e72c54 in_seq=0 out_seq=0 2024-09-15T21:46:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.891+0000 7f93727fc640 1 -- 172.21.15.100:0/4066193567 <== mds.2 v2:172.21.15.100:6836/276075809 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557216112c50 con 0x5572161d58a0 2024-09-15T21:46:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.891+0000 7f93727fc640 10 client.4725 handle_command_reply: tid=0 2024-09-15T21:46:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.891+0000 7f939cad6740 1 client.4725 shutdown 2024-09-15T21:46:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.891+0000 7f939cad6740 1 -- 172.21.15.100:0/4066193567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a4750 msgr2=0x5572161a1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.891+0000 7f939cad6740 1 --2- 172.21.15.100:0/4066193567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a4750 0x5572161a1420 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f93940c9ac0 tx=0x7f9388001790 comp rx=0 tx=0).stop 2024-09-15T21:46:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.891+0000 7f939cad6740 1 -- 172.21.15.100:0/4066193567 shutdown_connections 2024-09-15T21:46:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.891+0000 7f939cad6740 1 --2- 172.21.15.100:0/4066193567 >> [v2:172.21.15.100:6836/276075809,v1:172.21.15.100:6838/276075809] conn(0x5572161d58a0 0x5572161c4de0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.891+0000 7f939cad6740 1 --2- 172.21.15.100:0/4066193567 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a4b30 0x5572161f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.891+0000 7f939cad6740 1 --2- 172.21.15.100:0/4066193567 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a4750 0x5572161a1420 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.891+0000 7f939cad6740 1 --2- 172.21.15.100:0/4066193567 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a1f80 0x5572161f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.891+0000 7f93727fc640 0 client.4725 ms_handle_reset on v2:172.21.15.100:6836/276075809 2024-09-15T21:46:57.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.891+0000 7f939cad6740 1 -- 172.21.15.100:0/4066193567 >> 172.21.15.100:0/4066193567 conn(0x5572161a2e70 msgr2=0x5572161d7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:57.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.891+0000 7f939cad6740 1 -- 172.21.15.100:0/4066193567 shutdown_connections 2024-09-15T21:46:57.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.891+0000 7f939cad6740 1 -- 172.21.15.100:0/4066193567 wait complete. 2024-09-15T21:46:57.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.891+0000 7f939cad6740 20 client.4725 trim_cache size 0 max 16384 2024-09-15T21:46:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.909+0000 7f939cad6740 1 Processor -- start 2024-09-15T21:46:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.909+0000 7f939cad6740 1 -- start start 2024-09-15T21:46:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.909+0000 7f939cad6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a1f80 0x5572161e07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.909+0000 7f939cad6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a4750 0x5572161cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.909+0000 7f939cad6740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a4b30 0x5572161cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.909+0000 7f939cad6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f93941fb880 con 0x5572161a1f80 2024-09-15T21:46:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.909+0000 7f939cad6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f93941ef6f0 con 0x5572161a4750 2024-09-15T21:46:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.909+0000 7f939cad6740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x5572161f4dc0 con 0x5572161a4b30 2024-09-15T21:46:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.910+0000 7f93937fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a1f80 0x5572161e07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.910+0000 7f93937fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a1f80 0x5572161e07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48700/0 (socket says 172.21.15.100:48700) 2024-09-15T21:46:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.910+0000 7f93937fe640 1 -- 172.21.15.100:0/2123319925 learned_addr learned my addr 172.21.15.100:0/2123319925 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.910+0000 7f93937fe640 1 -- 172.21.15.100:0/2123319925 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a4b30 msgr2=0x5572161cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.910+0000 7f93937fe640 1 --2- 172.21.15.100:0/2123319925 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a4b30 0x5572161cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.910+0000 7f93937fe640 1 -- 172.21.15.100:0/2123319925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a4750 msgr2=0x5572161cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.910+0000 7f93937fe640 1 --2- 172.21.15.100:0/2123319925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a4750 0x5572161cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.910+0000 7f93937fe640 1 -- 172.21.15.100:0/2123319925 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572161a0580 con 0x5572161a1f80 2024-09-15T21:46:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.910+0000 7f93937fe640 1 --2- 172.21.15.100:0/2123319925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a1f80 0x5572161e07e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9384022000 tx=0x7f9384017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4118634bcbdd9fb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.910+0000 7f9378ff9640 1 -- 172.21.15.100:0/2123319925 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384028040 con 0x5572161a1f80 2024-09-15T21:46:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.910+0000 7f9378ff9640 1 -- 172.21.15.100:0/2123319925 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9384004e50 con 0x5572161a1f80 2024-09-15T21:46:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.910+0000 7f9378ff9640 1 -- 172.21.15.100:0/2123319925 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384039630 con 0x5572161a1f80 2024-09-15T21:46:57.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.911+0000 7f939cad6740 1 -- 172.21.15.100:0/2123319925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a1f80 msgr2=0x5572161e07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.911+0000 7f939cad6740 1 --2- 172.21.15.100:0/2123319925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a1f80 0x5572161e07e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9384022000 tx=0x7f9384017000 comp rx=0 tx=0).stop 2024-09-15T21:46:57.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.911+0000 7f939cad6740 1 -- 172.21.15.100:0/2123319925 shutdown_connections 2024-09-15T21:46:57.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.911+0000 7f939cad6740 1 --2- 172.21.15.100:0/2123319925 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a4750 0x5572161cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.911+0000 7f939cad6740 1 --2- 172.21.15.100:0/2123319925 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a4b30 0x5572161cdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.911+0000 7f939cad6740 1 --2- 172.21.15.100:0/2123319925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a1f80 0x5572161e07e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.911+0000 7f939cad6740 1 -- 172.21.15.100:0/2123319925 >> 172.21.15.100:0/2123319925 conn(0x5572161a2e70 msgr2=0x55721619ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:57.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.911+0000 7f939cad6740 1 -- 172.21.15.100:0/2123319925 shutdown_connections 2024-09-15T21:46:57.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.911+0000 7f939cad6740 1 -- 172.21.15.100:0/2123319925 wait complete. 2024-09-15T21:46:57.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.911+0000 7f939cad6740 1 -- start start 2024-09-15T21:46:57.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.911+0000 7f939cad6740 1 Processor -- start 2024-09-15T21:46:57.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.912+0000 7f939cad6740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a1f80 0x5572161d6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.912+0000 7f939cad6740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a4750 0x5572161d7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.912+0000 7f939cad6740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a4b30 0x557216261380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.912+0000 7f939cad6740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x5572161a1aa0 con 0x5572161a1f80 2024-09-15T21:46:57.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.912+0000 7f939cad6740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5572161dc0c0 con 0x5572161a4b30 2024-09-15T21:46:57.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.912+0000 7f939cad6740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x5572161f8100 con 0x5572161a4750 2024-09-15T21:46:57.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.912+0000 7f939cad6740 10 client.0 _ll_register_callbacks cb 0x7f9388005050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T21:46:57.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.912+0000 7f93937fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a1f80 0x5572161d6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.912+0000 7f93937fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a1f80 0x5572161d6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48702/0 (socket says 172.21.15.100:48702) 2024-09-15T21:46:57.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.912+0000 7f93937fe640 1 -- 172.21.15.100:0/269988139 learned_addr learned my addr 172.21.15.100:0/269988139 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:57.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.912+0000 7f9392ffd640 1 --2- 172.21.15.100:0/269988139 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a4750 0x5572161d7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.912+0000 7f9393fff640 1 --2- 172.21.15.100:0/269988139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a4b30 0x557216261380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.912+0000 7f93937fe640 1 -- 172.21.15.100:0/269988139 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a4750 msgr2=0x5572161d7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.912+0000 7f93937fe640 1 --2- 172.21.15.100:0/269988139 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a4750 0x5572161d7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.912+0000 7f93937fe640 1 -- 172.21.15.100:0/269988139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a4b30 msgr2=0x557216261380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:57.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.912+0000 7f93937fe640 1 --2- 172.21.15.100:0/269988139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a4b30 0x557216261380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f93937fe640 1 -- 172.21.15.100:0/269988139 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572161d0420 con 0x5572161a1f80 2024-09-15T21:46:57.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f9392ffd640 1 --2- 172.21.15.100:0/269988139 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a4750 0x5572161d7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:57.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f9393fff640 1 --2- 172.21.15.100:0/269988139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a4b30 0x557216261380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:57.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f93937fe640 1 --2- 172.21.15.100:0/269988139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a1f80 0x5572161d6870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9384004ca0 tx=0x7f93940c4c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab8e43c363e0096 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:57.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f93727fc640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-15T21:46:57.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f93727fc640 1 -- 172.21.15.100:0/269988139 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9384028070 con 0x5572161a1f80 2024-09-15T21:46:57.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f93727fc640 1 -- 172.21.15.100:0/269988139 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f9384015d50 con 0x5572161a1f80 2024-09-15T21:46:57.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f939cad6740 1 -- 172.21.15.100:0/269988139 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9384004e50 con 0x5572161a1f80 2024-09-15T21:46:57.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f93727fc640 1 -- 172.21.15.100:0/269988139 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f9384045050 con 0x5572161a1f80 2024-09-15T21:46:57.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f939cad6740 10 client.4731 fetch_fsmap learned FSMap version 17 2024-09-15T21:46:57.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f939cad6740 1 -- 172.21.15.100:0/269988139 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x5572162656e0 con 0x5572161a1f80 2024-09-15T21:46:57.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f93727fc640 1 -- 172.21.15.100:0/269988139 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9384004e50 con 0x5572161a1f80 2024-09-15T21:46:57.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f93727fc640 1 -- 172.21.15.100:0/269988139 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 17) ==== 4792+0+0 (secure 0 0 0) 0x7f9384048020 con 0x5572161a1f80 2024-09-15T21:46:57.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f939cad6740 10 client.4731 fetch_fsmap finished waiting for FSMap version 17 2024-09-15T21:46:57.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f939cad6740 10 client.4731 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-15T21:46:57.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f939cad6740 20 client.4731 populate_metadata read hostname 'smithi100' 2024-09-15T21:46:57.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f939cad6740 1 --2- 172.21.15.100:0/269988139 >> [v2:172.21.15.100:6836/276075809,v1:172.21.15.100:6838/276075809] conn(0x5572161d41a0 0x5572161d4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:57.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f939cad6740 4 client.4731 mds_command: new command op to 4290 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T21:46:57.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f939cad6740 1 -- 172.21.15.100:0/269988139 --> [v2:172.21.15.100:6836/276075809,v1:172.21.15.100:6838/276075809] -- command(tid 0: {"prefix": "status"}) -- 0x5572161cd400 con 0x5572161d41a0 2024-09-15T21:46:57.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.913+0000 7f9392ffd640 1 --2- 172.21.15.100:0/269988139 >> [v2:172.21.15.100:6836/276075809,v1:172.21.15.100:6838/276075809] conn(0x5572161d41a0 0x5572161d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:57.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.914+0000 7f93727fc640 10 client.4731 ms_handle_connect on v2:172.21.15.100:6836/276075809 2024-09-15T21:46:57.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.914+0000 7f9392ffd640 1 --2- 172.21.15.100:0/269988139 >> [v2:172.21.15.100:6836/276075809,v1:172.21.15.100:6838/276075809] conn(0x5572161d41a0 0x5572161d4580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bbed2e0db1234801 server_cookie=155d3aa9e1f84ab5 in_seq=0 out_seq=0 2024-09-15T21:46:57.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.914+0000 7f93727fc640 1 -- 172.21.15.100:0/269988139 <== mds.2 v2:172.21.15.100:6836/276075809 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5572161cd400 con 0x5572161d41a0 2024-09-15T21:46:57.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.914+0000 7f93727fc640 10 client.4731 handle_command_reply: tid=0 2024-09-15T21:46:57.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.914+0000 7f939cad6740 1 client.4731 shutdown 2024-09-15T21:46:57.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.914+0000 7f939cad6740 1 -- 172.21.15.100:0/269988139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a1f80 msgr2=0x5572161d6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.914+0000 7f939cad6740 1 --2- 172.21.15.100:0/269988139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a1f80 0x5572161d6870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9384004ca0 tx=0x7f93940c4c50 comp rx=0 tx=0).stop 2024-09-15T21:46:57.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.914+0000 7f939cad6740 1 -- 172.21.15.100:0/269988139 shutdown_connections 2024-09-15T21:46:57.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.914+0000 7f939cad6740 1 --2- 172.21.15.100:0/269988139 >> [v2:172.21.15.100:6836/276075809,v1:172.21.15.100:6838/276075809] conn(0x5572161d41a0 0x5572161d4580 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.914+0000 7f939cad6740 1 --2- 172.21.15.100:0/269988139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5572161a4b30 0x557216261380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.914+0000 7f939cad6740 1 --2- 172.21.15.100:0/269988139 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x5572161a4750 0x5572161d7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.914+0000 7f93727fc640 0 client.4731 ms_handle_reset on v2:172.21.15.100:6836/276075809 2024-09-15T21:46:57.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.914+0000 7f939cad6740 1 --2- 172.21.15.100:0/269988139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5572161a1f80 0x5572161d6870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.914+0000 7f939cad6740 1 -- 172.21.15.100:0/269988139 >> 172.21.15.100:0/269988139 conn(0x5572161a2e70 msgr2=0x55721619ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:57.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.914+0000 7f939cad6740 1 -- 172.21.15.100:0/269988139 shutdown_connections 2024-09-15T21:46:57.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.914+0000 7f939cad6740 1 -- 172.21.15.100:0/269988139 wait complete. 2024-09-15T21:46:57.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.914+0000 7f939cad6740 20 client.4731 trim_cache size 0 max 16384 2024-09-15T21:46:57.924 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:46:57.924 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "30e2fff4-2171-4531-8806-0e03b63f494f", 2024-09-15T21:46:57.924 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 2, 2024-09-15T21:46:57.924 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4290, 2024-09-15T21:46:57.924 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-09-15T21:46:57.924 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-09-15T21:46:57.924 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-09-15T21:46:57.925 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 22.443461059000001, 2024-09-15T21:46:57.925 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 17, 2024-09-15T21:46:57.925 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 22, 2024-09-15T21:46:57.925 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 22, 2024-09-15T21:46:57.925 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 40.531778420999999 2024-09-15T21:46:57.925 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:46:57.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.915+0000 7f9399d3b640 1 -- 172.21.15.100:0/2256870813 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f938006c2e0 msgr2=0x7f938006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.915+0000 7f9399d3b640 1 --2- 172.21.15.100:0/2256870813 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f938006c2e0 0x7f938006e7a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9384007c80 tx=0x7f9384001db0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.915+0000 7f9399d3b640 1 -- 172.21.15.100:0/2256870813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93941de610 msgr2=0x7f939414afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:57.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.915+0000 7f9399d3b640 1 --2- 172.21.15.100:0/2256870813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93941de610 0x7f939414afa0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f938c00e230 tx=0x7f938c009750 comp rx=0 tx=0).stop 2024-09-15T21:46:57.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.915+0000 7f9399d3b640 1 -- 172.21.15.100:0/2256870813 shutdown_connections 2024-09-15T21:46:57.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.915+0000 7f9399d3b640 1 --2- 172.21.15.100:0/2256870813 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f938006c2e0 0x7f938006e7a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.915+0000 7f9399d3b640 1 --2- 172.21.15.100:0/2256870813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93941de610 0x7f939414afa0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.915+0000 7f9399d3b640 1 --2- 172.21.15.100:0/2256870813 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93941e1c10 0x7f93941de0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.915+0000 7f9399d3b640 1 --2- 172.21.15.100:0/2256870813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f937c004d40 0x7f939414e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:57.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.915+0000 7f9399d3b640 1 -- 172.21.15.100:0/2256870813 >> 172.21.15.100:0/2256870813 conn(0x7f9394067dd0 msgr2=0x7f939406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:57.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.915+0000 7f9399d3b640 1 -- 172.21.15.100:0/2256870813 shutdown_connections 2024-09-15T21:46:57.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:57.915+0000 7f9399d3b640 1 -- 172.21.15.100:0/2256870813 wait complete. 2024-09-15T21:46:57.928 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-15T21:46:58.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.024+0000 7f87973bd640 1 Processor -- start 2024-09-15T21:46:58.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.024+0000 7f87973bd640 1 -- start start 2024-09-15T21:46:58.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.025+0000 7f87973bd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87901552e0 0x7f879006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.025+0000 7f87973bd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f87900cee00 0x7f87900730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.025+0000 7f87973bd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f87900735f0 0x7f879015c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.025+0000 7f87973bd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f879006ec60 con 0x7f87900735f0 2024-09-15T21:46:58.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.025+0000 7f87973bd640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f879015af40 con 0x7f87900cee00 2024-09-15T21:46:58.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.025+0000 7f87973bd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f879015b200 con 0x7f87901552e0 2024-09-15T21:46:58.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.025+0000 7f879595a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f87900735f0 0x7f879015c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.025+0000 7f879595a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f87900735f0 0x7f879015c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48706/0 (socket says 172.21.15.100:48706) 2024-09-15T21:46:58.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.025+0000 7f879595a640 1 -- 172.21.15.100:0/3802276231 learned_addr learned my addr 172.21.15.100:0/3802276231 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:58.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.025+0000 7f8795159640 1 --2- 172.21.15.100:0/3802276231 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87901552e0 0x7f879006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.025+0000 7f8794958640 1 --2- 172.21.15.100:0/3802276231 >> v2:172.21.15.122:3300/0 conn(0x7f87900cee00 0x7f87900730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.025+0000 7f879595a640 1 -- 172.21.15.100:0/3802276231 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87901552e0 msgr2=0x7f879006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.025+0000 7f879595a640 1 --2- 172.21.15.100:0/3802276231 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87901552e0 0x7f879006d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.025+0000 7f879595a640 1 -- 172.21.15.100:0/3802276231 >> v2:172.21.15.122:3300/0 conn(0x7f87900cee00 msgr2=0x7f87900730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.025+0000 7f879595a640 1 --2- 172.21.15.100:0/3802276231 >> v2:172.21.15.122:3300/0 conn(0x7f87900cee00 0x7f87900730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.025+0000 7f879595a640 1 -- 172.21.15.100:0/3802276231 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879015b4e0 con 0x7f87900735f0 2024-09-15T21:46:58.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.026+0000 7f879595a640 1 --2- 172.21.15.100:0/3802276231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87900735f0 0x7f879015c5e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f87900cc380 tx=0x7f878802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fd37e8ced17e48 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:58.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.026+0000 7f8787fff640 1 -- 172.21.15.100:0/3802276231 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788036070 con 0x7f87900735f0 2024-09-15T21:46:58.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.026+0000 7f8787fff640 1 -- 172.21.15.100:0/3802276231 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8788031600 con 0x7f87900735f0 2024-09-15T21:46:58.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.026+0000 7f8787fff640 1 -- 172.21.15.100:0/3802276231 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87880319d0 con 0x7f87900735f0 2024-09-15T21:46:58.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.027+0000 7f87973bd640 1 -- 172.21.15.100:0/3802276231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87900735f0 msgr2=0x7f879015c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.027+0000 7f87973bd640 1 --2- 172.21.15.100:0/3802276231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87900735f0 0x7f879015c5e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f87900cc380 tx=0x7f878802a6a0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.027+0000 7f87973bd640 1 -- 172.21.15.100:0/3802276231 shutdown_connections 2024-09-15T21:46:58.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.027+0000 7f87973bd640 1 --2- 172.21.15.100:0/3802276231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87900735f0 0x7f879015c5e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.027+0000 7f87973bd640 1 --2- 172.21.15.100:0/3802276231 >> v2:172.21.15.122:3300/0 conn(0x7f87900cee00 0x7f87900730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.027+0000 7f87973bd640 1 --2- 172.21.15.100:0/3802276231 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87901552e0 0x7f879006d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.027+0000 7f87973bd640 1 -- 172.21.15.100:0/3802276231 >> 172.21.15.100:0/3802276231 conn(0x7f8790070ca0 msgr2=0x7f8790071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:58.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.027+0000 7f87973bd640 1 -- 172.21.15.100:0/3802276231 shutdown_connections 2024-09-15T21:46:58.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.027+0000 7f87973bd640 1 -- 172.21.15.100:0/3802276231 wait complete. 2024-09-15T21:46:58.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.028+0000 7f87973bd640 1 Processor -- start 2024-09-15T21:46:58.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.028+0000 7f87973bd640 1 -- start start 2024-09-15T21:46:58.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f87973bd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87900735f0 0x7f87900ce470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f87973bd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87900cee00 0x7f87901f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f87973bd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87901552e0 0x7f87901e5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f87973bd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f87900c91a0 con 0x7f87900735f0 2024-09-15T21:46:58.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f87973bd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f879015baa0 con 0x7f87901552e0 2024-09-15T21:46:58.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f87973bd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f87900cf220 con 0x7f87900cee00 2024-09-15T21:46:58.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f8795159640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87900735f0 0x7f87900ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f8795159640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87900735f0 0x7f87900ce470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48718/0 (socket says 172.21.15.100:48718) 2024-09-15T21:46:58.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f8795159640 1 -- 172.21.15.100:0/665793660 learned_addr learned my addr 172.21.15.100:0/665793660 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:58.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f879595a640 1 --2- 172.21.15.100:0/665793660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87901552e0 0x7f87901e5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f8794958640 1 --2- 172.21.15.100:0/665793660 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87900cee00 0x7f87901f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f8795159640 1 -- 172.21.15.100:0/665793660 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87900cee00 msgr2=0x7f87901f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f8795159640 1 --2- 172.21.15.100:0/665793660 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87900cee00 0x7f87901f57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f8795159640 1 -- 172.21.15.100:0/665793660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87901552e0 msgr2=0x7f87901e5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f8795159640 1 --2- 172.21.15.100:0/665793660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87901552e0 0x7f87901e5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f8795159640 1 -- 172.21.15.100:0/665793660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87901e1480 con 0x7f87900735f0 2024-09-15T21:46:58.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f879595a640 1 --2- 172.21.15.100:0/665793660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87901552e0 0x7f87901e5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:58.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f8794958640 1 --2- 172.21.15.100:0/665793660 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87900cee00 0x7f87901f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:58.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f8795159640 1 --2- 172.21.15.100:0/665793660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87900735f0 0x7f87900ce470 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f878c00d5f0 tx=0x7f878c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:58.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f8785ffb640 1 -- 172.21.15.100:0/665793660 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c0125f0 con 0x7f87900735f0 2024-09-15T21:46:58.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f8785ffb640 1 -- 172.21.15.100:0/665793660 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f878c019070 con 0x7f87900735f0 2024-09-15T21:46:58.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f8785ffb640 1 -- 172.21.15.100:0/665793660 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c0129d0 con 0x7f87900735f0 2024-09-15T21:46:58.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f87973bd640 1 -- 172.21.15.100:0/665793660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87900c63b0 con 0x7f87900735f0 2024-09-15T21:46:58.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.029+0000 7f87973bd640 1 -- 172.21.15.100:0/665793660 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87900c6970 con 0x7f87900735f0 2024-09-15T21:46:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.031+0000 7f8785ffb640 1 -- 172.21.15.100:0/665793660 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f878c012b70 con 0x7f87900735f0 2024-09-15T21:46:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.031+0000 7f8785ffb640 1 --2- 172.21.15.100:0/665793660 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f877406c2b0 0x7f877406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.032+0000 7f8794958640 1 --2- 172.21.15.100:0/665793660 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f877406c2b0 0x7f877406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.032+0000 7f8785ffb640 1 -- 172.21.15.100:0/665793660 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f878c08b830 con 0x7f87900735f0 2024-09-15T21:46:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.032+0000 7f8794958640 1 --2- 172.21.15.100:0/665793660 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f877406c2b0 0x7f877406e770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f87901e43b0 tx=0x7f8780008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.033+0000 7f879a233740 1 Processor -- start 2024-09-15T21:46:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.033+0000 7f879a233740 1 -- start start 2024-09-15T21:46:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.033+0000 7f879a233740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc140750 0x561dbc1789e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.033+0000 7f879a233740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc13df80 0x561dbc178f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.033+0000 7f879a233740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc140b30 0x561dbc13ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.033+0000 7f879a233740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561dbc0a1520 con 0x561dbc13df80 2024-09-15T21:46:58.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.033+0000 7f879a233740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561dbc0493e0 con 0x561dbc140b30 2024-09-15T21:46:58.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.033+0000 7f879a233740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x561dbc09d4e0 con 0x561dbc140750 2024-09-15T21:46:58.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.033+0000 7f8794958640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc13df80 0x561dbc178f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.033+0000 7f8794958640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc13df80 0x561dbc178f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48730/0 (socket says 172.21.15.100:48730) 2024-09-15T21:46:58.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.033+0000 7f8794958640 1 -- 172.21.15.100:0/1541683962 learned_addr learned my addr 172.21.15.100:0/1541683962 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:58.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.033+0000 7f879595a640 1 --2- 172.21.15.100:0/1541683962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc140b30 0x561dbc13ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.033+0000 7f8795159640 1 --2- 172.21.15.100:0/1541683962 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc140750 0x561dbc1789e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.034+0000 7f8794958640 1 -- 172.21.15.100:0/1541683962 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc140750 msgr2=0x561dbc1789e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.034+0000 7f8794958640 1 --2- 172.21.15.100:0/1541683962 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc140750 0x561dbc1789e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.034+0000 7f8794958640 1 -- 172.21.15.100:0/1541683962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc140b30 msgr2=0x561dbc13ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.034+0000 7f8794958640 1 --2- 172.21.15.100:0/1541683962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc140b30 0x561dbc13ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.034+0000 7f8794958640 1 -- 172.21.15.100:0/1541683962 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dbc0642f0 con 0x561dbc13df80 2024-09-15T21:46:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.034+0000 7f879595a640 1 --2- 172.21.15.100:0/1541683962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc140b30 0x561dbc13ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:46:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.034+0000 7f8794958640 1 --2- 172.21.15.100:0/1541683962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc13df80 0x561dbc178f20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f878000fad0 tx=0x7f8780012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ca66785cf5ce4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.034+0000 7f8772ffd640 1 -- 172.21.15.100:0/1541683962 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878001b050 con 0x561dbc13df80 2024-09-15T21:46:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.034+0000 7f8772ffd640 1 -- 172.21.15.100:0/1541683962 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8780025070 con 0x561dbc13df80 2024-09-15T21:46:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.034+0000 7f8772ffd640 1 -- 172.21.15.100:0/1541683962 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780020440 con 0x561dbc13df80 2024-09-15T21:46:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.034+0000 7f879a233740 1 -- 172.21.15.100:0/1541683962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc13df80 msgr2=0x561dbc178f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.034+0000 7f879a233740 1 --2- 172.21.15.100:0/1541683962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc13df80 0x561dbc178f20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f878000fad0 tx=0x7f8780012000 comp rx=0 tx=0).stop 2024-09-15T21:46:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.035+0000 7f879a233740 1 -- 172.21.15.100:0/1541683962 shutdown_connections 2024-09-15T21:46:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.035+0000 7f879a233740 1 --2- 172.21.15.100:0/1541683962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc140b30 0x561dbc13ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.035+0000 7f879a233740 1 --2- 172.21.15.100:0/1541683962 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc13df80 0x561dbc178f20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.035+0000 7f879a233740 1 --2- 172.21.15.100:0/1541683962 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc140750 0x561dbc1789e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.035+0000 7f879a233740 1 -- 172.21.15.100:0/1541683962 >> 172.21.15.100:0/1541683962 conn(0x561dbc13ee70 msgr2=0x561dbc13f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.035+0000 7f879a233740 1 -- 172.21.15.100:0/1541683962 shutdown_connections 2024-09-15T21:46:58.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.035+0000 7f879a233740 1 -- 172.21.15.100:0/1541683962 wait complete. 2024-09-15T21:46:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.035+0000 7f879a233740 1 -- start start 2024-09-15T21:46:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.035+0000 7f879a233740 1 Processor -- start 2024-09-15T21:46:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.036+0000 7f879a233740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc13df80 0x561dbc171ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.036+0000 7f879a233740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140750 0x561dbc178ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.036+0000 7f879a233740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc140b30 0x561dbc16cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.036+0000 7f879a233740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561dbc13ffe0 con 0x561dbc140750 2024-09-15T21:46:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.036+0000 7f879a233740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561dbc02a450 con 0x561dbc140b30 2024-09-15T21:46:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.036+0000 7f879a233740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x561dbc178550 con 0x561dbc13df80 2024-09-15T21:46:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.036+0000 7f879a233740 10 client.0 _ll_register_callbacks cb 0x561dbbdb3ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T21:46:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.036+0000 7f8794958640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140750 0x561dbc178ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.036+0000 7f8794958640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140750 0x561dbc178ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48740/0 (socket says 172.21.15.100:48740) 2024-09-15T21:46:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.036+0000 7f8794958640 1 -- 172.21.15.100:0/588135957 learned_addr learned my addr 172.21.15.100:0/588135957 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:58.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.036+0000 7f879595a640 1 --2- 172.21.15.100:0/588135957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc140b30 0x561dbc16cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.036+0000 7f8795159640 1 --2- 172.21.15.100:0/588135957 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc13df80 0x561dbc171ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.036+0000 7f8794958640 1 -- 172.21.15.100:0/588135957 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc13df80 msgr2=0x561dbc171ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.036+0000 7f8794958640 1 --2- 172.21.15.100:0/588135957 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc13df80 0x561dbc171ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.036+0000 7f8794958640 1 -- 172.21.15.100:0/588135957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc140b30 msgr2=0x561dbc16cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.036+0000 7f8794958640 1 --2- 172.21.15.100:0/588135957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc140b30 0x561dbc16cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.036+0000 7f8794958640 1 -- 172.21.15.100:0/588135957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dbbdb42d0 con 0x561dbc140750 2024-09-15T21:46:58.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.037+0000 7f8794958640 1 --2- 172.21.15.100:0/588135957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140750 0x561dbc178ee0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f878000faa0 tx=0x7f8780004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c55e64205aab092 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:58.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.037+0000 7f876bfff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-15T21:46:58.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.037+0000 7f876bfff640 1 -- 172.21.15.100:0/588135957 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780020a90 con 0x561dbc140750 2024-09-15T21:46:58.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.037+0000 7f876bfff640 1 -- 172.21.15.100:0/588135957 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8780020c30 con 0x561dbc140750 2024-09-15T21:46:58.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.037+0000 7f879a233740 1 -- 172.21.15.100:0/588135957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8780025070 con 0x561dbc140750 2024-09-15T21:46:58.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.037+0000 7f876bfff640 1 -- 172.21.15.100:0/588135957 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f878001b020 con 0x561dbc140750 2024-09-15T21:46:58.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.037+0000 7f879a233740 10 client.4743 fetch_fsmap learned FSMap version 17 2024-09-15T21:46:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.037+0000 7f879a233740 1 -- 172.21.15.100:0/588135957 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x561dbbe7e6b0 con 0x561dbc140750 2024-09-15T21:46:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.037+0000 7f876bfff640 1 -- 172.21.15.100:0/588135957 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8780025070 con 0x561dbc140750 2024-09-15T21:46:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.037+0000 7f876bfff640 1 -- 172.21.15.100:0/588135957 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 17) ==== 4792+0+0 (secure 0 0 0) 0x7f878003e080 con 0x561dbc140750 2024-09-15T21:46:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.037+0000 7f879a233740 10 client.4743 fetch_fsmap finished waiting for FSMap version 17 2024-09-15T21:46:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.037+0000 7f879a233740 10 client.4743 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T21:46:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.037+0000 7f879a233740 20 client.4743 populate_metadata read hostname 'smithi100' 2024-09-15T21:46:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.037+0000 7f879a233740 1 --2- 172.21.15.100:0/588135957 >> [v2:172.21.15.122:6835/2947743626,v1:172.21.15.122:6836/2947743626] conn(0x561dbc190400 0x561dbc19e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.037+0000 7f879a233740 4 client.4743 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:46:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.037+0000 7f879a233740 1 -- 172.21.15.100:0/588135957 --> [v2:172.21.15.122:6835/2947743626,v1:172.21.15.122:6836/2947743626] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dbc0aec50 con 0x561dbc190400 2024-09-15T21:46:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.038+0000 7f8795159640 1 --2- 172.21.15.100:0/588135957 >> [v2:172.21.15.122:6835/2947743626,v1:172.21.15.122:6836/2947743626] conn(0x561dbc190400 0x561dbc19e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.038+0000 7f8795159640 1 --2- 172.21.15.100:0/588135957 >> [v2:172.21.15.122:6835/2947743626,v1:172.21.15.122:6836/2947743626] conn(0x561dbc190400 0x561dbc19e770 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5467ce9504b9533f server_cookie=5e4ce11e996d574a in_seq=0 out_seq=0 2024-09-15T21:46:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.038+0000 7f876bfff640 10 client.4743 ms_handle_connect on v2:172.21.15.122:6835/2947743626 2024-09-15T21:46:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.039+0000 7f876bfff640 1 -- 172.21.15.100:0/588135957 <== mds.1 v2:172.21.15.122:6835/2947743626 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561dbc0aec50 con 0x561dbc190400 2024-09-15T21:46:58.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.039+0000 7f876bfff640 10 client.4743 handle_command_reply: tid=0 2024-09-15T21:46:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.039+0000 7f879a233740 1 client.4743 shutdown 2024-09-15T21:46:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.039+0000 7f879a233740 1 -- 172.21.15.100:0/588135957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140750 msgr2=0x561dbc178ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.039+0000 7f879a233740 1 --2- 172.21.15.100:0/588135957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140750 0x561dbc178ee0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f878000faa0 tx=0x7f8780004730 comp rx=0 tx=0).stop 2024-09-15T21:46:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.039+0000 7f879a233740 1 -- 172.21.15.100:0/588135957 shutdown_connections 2024-09-15T21:46:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.039+0000 7f879a233740 1 --2- 172.21.15.100:0/588135957 >> [v2:172.21.15.122:6835/2947743626,v1:172.21.15.122:6836/2947743626] conn(0x561dbc190400 0x561dbc19e770 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.039+0000 7f879a233740 1 --2- 172.21.15.100:0/588135957 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc140b30 0x561dbc16cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.039+0000 7f879a233740 1 --2- 172.21.15.100:0/588135957 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140750 0x561dbc178ee0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.039+0000 7f879a233740 1 --2- 172.21.15.100:0/588135957 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc13df80 0x561dbc171ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.039+0000 7f876bfff640 0 client.4743 ms_handle_reset on v2:172.21.15.122:6835/2947743626 2024-09-15T21:46:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.039+0000 7f879a233740 1 -- 172.21.15.100:0/588135957 >> 172.21.15.100:0/588135957 conn(0x561dbc13ee70 msgr2=0x561dbc13e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.039+0000 7f879a233740 1 -- 172.21.15.100:0/588135957 shutdown_connections 2024-09-15T21:46:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.039+0000 7f879a233740 1 -- 172.21.15.100:0/588135957 wait complete. 2024-09-15T21:46:58.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.039+0000 7f879a233740 20 client.4743 trim_cache size 0 max 16384 2024-09-15T21:46:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.057+0000 7f879a233740 1 Processor -- start 2024-09-15T21:46:58.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.057+0000 7f879a233740 1 -- start start 2024-09-15T21:46:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.057+0000 7f879a233740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc13df80 0x561dbc139d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.057+0000 7f879a233740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc140750 0x561dbc13e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.057+0000 7f879a233740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140b30 0x561dbc13e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.057+0000 7f879a233740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f87900cd210 con 0x561dbc140b30 2024-09-15T21:46:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.057+0000 7f879a233740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561dbc13ffe0 con 0x561dbc13df80 2024-09-15T21:46:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.057+0000 7f879a233740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x561dbc166260 con 0x561dbc140750 2024-09-15T21:46:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.057+0000 7f879595a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140b30 0x561dbc13e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.058+0000 7f879595a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140b30 0x561dbc13e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48742/0 (socket says 172.21.15.100:48742) 2024-09-15T21:46:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.058+0000 7f879595a640 1 -- 172.21.15.100:0/1513964589 learned_addr learned my addr 172.21.15.100:0/1513964589 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.058+0000 7f8795159640 1 --2- 172.21.15.100:0/1513964589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc13df80 0x561dbc139d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.058+0000 7f8794958640 1 --2- 172.21.15.100:0/1513964589 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc140750 0x561dbc13e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.058+0000 7f879595a640 1 -- 172.21.15.100:0/1513964589 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc140750 msgr2=0x561dbc13e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.058+0000 7f879595a640 1 --2- 172.21.15.100:0/1513964589 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc140750 0x561dbc13e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.058+0000 7f879595a640 1 -- 172.21.15.100:0/1513964589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc13df80 msgr2=0x561dbc139d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.058+0000 7f879595a640 1 --2- 172.21.15.100:0/1513964589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc13df80 0x561dbc139d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.058+0000 7f879595a640 1 -- 172.21.15.100:0/1513964589 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dbc13c580 con 0x561dbc140b30 2024-09-15T21:46:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.058+0000 7f8795159640 1 --2- 172.21.15.100:0/1513964589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc13df80 0x561dbc139d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.058+0000 7f8794958640 1 --2- 172.21.15.100:0/1513964589 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc140750 0x561dbc13e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.058+0000 7f879595a640 1 --2- 172.21.15.100:0/1513964589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140b30 0x561dbc13e8a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f878802ac90 tx=0x7f878802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad1423715635df5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.058+0000 7f87727fc640 1 -- 172.21.15.100:0/1513964589 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788036040 con 0x561dbc140b30 2024-09-15T21:46:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.058+0000 7f87727fc640 1 -- 172.21.15.100:0/1513964589 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8788031780 con 0x561dbc140b30 2024-09-15T21:46:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.058+0000 7f87727fc640 1 -- 172.21.15.100:0/1513964589 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788031b30 con 0x561dbc140b30 2024-09-15T21:46:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.059+0000 7f879a233740 1 -- 172.21.15.100:0/1513964589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140b30 msgr2=0x561dbc13e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.059+0000 7f879a233740 1 --2- 172.21.15.100:0/1513964589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140b30 0x561dbc13e8a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f878802ac90 tx=0x7f878802e000 comp rx=0 tx=0).stop 2024-09-15T21:46:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.059+0000 7f879a233740 1 -- 172.21.15.100:0/1513964589 shutdown_connections 2024-09-15T21:46:58.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.059+0000 7f879a233740 1 --2- 172.21.15.100:0/1513964589 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140b30 0x561dbc13e8a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.059+0000 7f879a233740 1 --2- 172.21.15.100:0/1513964589 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc140750 0x561dbc13e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.059+0000 7f879a233740 1 --2- 172.21.15.100:0/1513964589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc13df80 0x561dbc139d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.059+0000 7f879a233740 1 -- 172.21.15.100:0/1513964589 >> 172.21.15.100:0/1513964589 conn(0x561dbc13ee70 msgr2=0x561dbc134ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:58.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.059+0000 7f879a233740 1 -- 172.21.15.100:0/1513964589 shutdown_connections 2024-09-15T21:46:58.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.059+0000 7f879a233740 1 -- 172.21.15.100:0/1513964589 wait complete. 2024-09-15T21:46:58.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.059+0000 7f879a233740 1 -- start start 2024-09-15T21:46:58.064 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.059+0000 7f879a233740 1 Processor -- start 2024-09-15T21:46:58.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.060+0000 7f879a233740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc13df80 0x561dbc1f7820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.060+0000 7f879a233740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140750 0x561dbc18cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.060+0000 7f879a233740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc140b30 0x561dbc18d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.060+0000 7f879a233740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x561dbc166f80 con 0x561dbc140750 2024-09-15T21:46:58.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.060+0000 7f879a233740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561dbc179540 con 0x561dbc13df80 2024-09-15T21:46:58.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.060+0000 7f879a233740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x561dbc16d870 con 0x561dbc140b30 2024-09-15T21:46:58.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.060+0000 7f879a233740 10 client.0 _ll_register_callbacks cb 0x7f878c055b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T21:46:58.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.060+0000 7f8794958640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140750 0x561dbc18cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.060+0000 7f8794958640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140750 0x561dbc18cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48744/0 (socket says 172.21.15.100:48744) 2024-09-15T21:46:58.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.060+0000 7f8794958640 1 -- 172.21.15.100:0/1137273257 learned_addr learned my addr 172.21.15.100:0/1137273257 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:58.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.060+0000 7f8795159640 1 --2- 172.21.15.100:0/1137273257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc13df80 0x561dbc1f7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.060+0000 7f879595a640 1 --2- 172.21.15.100:0/1137273257 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc140b30 0x561dbc18d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.060+0000 7f8794958640 1 -- 172.21.15.100:0/1137273257 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc140b30 msgr2=0x561dbc18d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.060+0000 7f8794958640 1 --2- 172.21.15.100:0/1137273257 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc140b30 0x561dbc18d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.060+0000 7f8794958640 1 -- 172.21.15.100:0/1137273257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc13df80 msgr2=0x561dbc1f7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.060+0000 7f8794958640 1 --2- 172.21.15.100:0/1137273257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc13df80 0x561dbc1f7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.060+0000 7f8794958640 1 -- 172.21.15.100:0/1137273257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dbc169500 con 0x561dbc140750 2024-09-15T21:46:58.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.060+0000 7f8795159640 1 --2- 172.21.15.100:0/1137273257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc13df80 0x561dbc1f7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:58.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.060+0000 7f879595a640 1 --2- 172.21.15.100:0/1137273257 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc140b30 0x561dbc18d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:58.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.061+0000 7f8794958640 1 --2- 172.21.15.100:0/1137273257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140750 0x561dbc18cca0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f878000f6a0 tx=0x7f878001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a107c233852bec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:58.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.061+0000 7f876bfff640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-15T21:46:58.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.061+0000 7f876bfff640 1 -- 172.21.15.100:0/1137273257 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780014830 con 0x561dbc140750 2024-09-15T21:46:58.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.061+0000 7f876bfff640 1 -- 172.21.15.100:0/1137273257 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8780033650 con 0x561dbc140750 2024-09-15T21:46:58.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.061+0000 7f879a233740 1 -- 172.21.15.100:0/1137273257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8788031780 con 0x561dbc140750 2024-09-15T21:46:58.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.061+0000 7f876bfff640 1 -- 172.21.15.100:0/1137273257 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f878004b050 con 0x561dbc140750 2024-09-15T21:46:58.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.061+0000 7f879a233740 10 client.4749 fetch_fsmap learned FSMap version 17 2024-09-15T21:46:58.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.061+0000 7f879a233740 1 -- 172.21.15.100:0/1137273257 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x561dbc18c6b0 con 0x561dbc140750 2024-09-15T21:46:58.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.061+0000 7f876bfff640 1 -- 172.21.15.100:0/1137273257 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8788031780 con 0x561dbc140750 2024-09-15T21:46:58.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.061+0000 7f876bfff640 1 -- 172.21.15.100:0/1137273257 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 17) ==== 4792+0+0 (secure 0 0 0) 0x7f878004e020 con 0x561dbc140750 2024-09-15T21:46:58.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.061+0000 7f879a233740 10 client.4749 fetch_fsmap finished waiting for FSMap version 17 2024-09-15T21:46:58.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.061+0000 7f879a233740 10 client.4749 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-15T21:46:58.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.061+0000 7f879a233740 20 client.4749 populate_metadata read hostname 'smithi100' 2024-09-15T21:46:58.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.061+0000 7f879a233740 1 --2- 172.21.15.100:0/1137273257 >> [v2:172.21.15.122:6835/2947743626,v1:172.21.15.122:6836/2947743626] conn(0x561dbc1708f0 0x561dbc20a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.061+0000 7f879a233740 4 client.4749 mds_command: new command op to 4291 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T21:46:58.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.061+0000 7f879a233740 1 -- 172.21.15.100:0/1137273257 --> [v2:172.21.15.122:6835/2947743626,v1:172.21.15.122:6836/2947743626] -- command(tid 0: {"prefix": "status"}) -- 0x561dbc0a8fa0 con 0x561dbc1708f0 2024-09-15T21:46:58.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.061+0000 7f8795159640 1 --2- 172.21.15.100:0/1137273257 >> [v2:172.21.15.122:6835/2947743626,v1:172.21.15.122:6836/2947743626] conn(0x561dbc1708f0 0x561dbc20a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.062+0000 7f876bfff640 10 client.4749 ms_handle_connect on v2:172.21.15.122:6835/2947743626 2024-09-15T21:46:58.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.062+0000 7f8795159640 1 --2- 172.21.15.100:0/1137273257 >> [v2:172.21.15.122:6835/2947743626,v1:172.21.15.122:6836/2947743626] conn(0x561dbc1708f0 0x561dbc20a0f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d8838b4de6529d20 server_cookie=a58761f6a0462bbc in_seq=0 out_seq=0 2024-09-15T21:46:58.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.062+0000 7f876bfff640 1 -- 172.21.15.100:0/1137273257 <== mds.1 v2:172.21.15.122:6835/2947743626 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x561dbc0a8fa0 con 0x561dbc1708f0 2024-09-15T21:46:58.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.062+0000 7f876bfff640 10 client.4749 handle_command_reply: tid=0 2024-09-15T21:46:58.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.062+0000 7f879a233740 1 client.4749 shutdown 2024-09-15T21:46:58.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.062+0000 7f879a233740 1 -- 172.21.15.100:0/1137273257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140750 msgr2=0x561dbc18cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.062+0000 7f879a233740 1 --2- 172.21.15.100:0/1137273257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140750 0x561dbc18cca0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f878000f6a0 tx=0x7f878001f000 comp rx=0 tx=0).stop 2024-09-15T21:46:58.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.062+0000 7f879a233740 1 -- 172.21.15.100:0/1137273257 shutdown_connections 2024-09-15T21:46:58.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.062+0000 7f879a233740 1 --2- 172.21.15.100:0/1137273257 >> [v2:172.21.15.122:6835/2947743626,v1:172.21.15.122:6836/2947743626] conn(0x561dbc1708f0 0x561dbc20a0f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.062+0000 7f879a233740 1 --2- 172.21.15.100:0/1137273257 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x561dbc140b30 0x561dbc18d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.062+0000 7f879a233740 1 --2- 172.21.15.100:0/1137273257 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x561dbc140750 0x561dbc18cca0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.062+0000 7f876bfff640 0 client.4749 ms_handle_reset on v2:172.21.15.122:6835/2947743626 2024-09-15T21:46:58.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.062+0000 7f879a233740 1 --2- 172.21.15.100:0/1137273257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561dbc13df80 0x561dbc1f7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.062+0000 7f879a233740 1 -- 172.21.15.100:0/1137273257 >> 172.21.15.100:0/1137273257 conn(0x561dbc13ee70 msgr2=0x561dbc134ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:58.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.062+0000 7f879a233740 1 -- 172.21.15.100:0/1137273257 shutdown_connections 2024-09-15T21:46:58.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.062+0000 7f879a233740 1 -- 172.21.15.100:0/1137273257 wait complete. 2024-09-15T21:46:58.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.062+0000 7f879a233740 20 client.4749 trim_cache size 0 max 16384 2024-09-15T21:46:58.072 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:46:58.072 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "30e2fff4-2171-4531-8806-0e03b63f494f", 2024-09-15T21:46:58.072 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 1, 2024-09-15T21:46:58.072 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4291, 2024-09-15T21:46:58.072 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-09-15T21:46:58.072 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-09-15T21:46:58.072 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-09-15T21:46:58.073 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 28.611472236000001, 2024-09-15T21:46:58.073 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 17, 2024-09-15T21:46:58.073 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 22, 2024-09-15T21:46:58.073 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 22, 2024-09-15T21:46:58.073 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 40.665078182999999 2024-09-15T21:46:58.073 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:46:58.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.063+0000 7f87973bd640 1 -- 172.21.15.100:0/665793660 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f877406c2b0 msgr2=0x7f877406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.063+0000 7f87973bd640 1 --2- 172.21.15.100:0/665793660 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f877406c2b0 0x7f877406e770 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f87901e43b0 tx=0x7f8780008040 comp rx=0 tx=0).stop 2024-09-15T21:46:58.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.063+0000 7f87973bd640 1 -- 172.21.15.100:0/665793660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87900735f0 msgr2=0x7f87900ce470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.063+0000 7f87973bd640 1 --2- 172.21.15.100:0/665793660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87900735f0 0x7f87900ce470 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f878c00d5f0 tx=0x7f878c00dab0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.063+0000 7f87973bd640 1 -- 172.21.15.100:0/665793660 shutdown_connections 2024-09-15T21:46:58.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.063+0000 7f87973bd640 1 --2- 172.21.15.100:0/665793660 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f877406c2b0 0x7f877406e770 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.063+0000 7f87973bd640 1 --2- 172.21.15.100:0/665793660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87901552e0 0x7f87901e5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.063+0000 7f87973bd640 1 --2- 172.21.15.100:0/665793660 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87900cee00 0x7f87901f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.063+0000 7f87973bd640 1 --2- 172.21.15.100:0/665793660 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87900735f0 0x7f87900ce470 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.063+0000 7f87973bd640 1 -- 172.21.15.100:0/665793660 >> 172.21.15.100:0/665793660 conn(0x7f8790070ca0 msgr2=0x7f87901e6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:58.074 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.063+0000 7f87973bd640 1 -- 172.21.15.100:0/665793660 shutdown_connections 2024-09-15T21:46:58.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.063+0000 7f87973bd640 1 -- 172.21.15.100:0/665793660 wait complete. 2024-09-15T21:46:58.076 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-09-15T21:46:58.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.173+0000 7fe9a05e8640 1 Processor -- start 2024-09-15T21:46:58.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.173+0000 7fe9a05e8640 1 -- start start 2024-09-15T21:46:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.174+0000 7fe9a05e8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe998155040 0x7fe998072280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.174+0000 7fe9a05e8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe998155420 0x7fe99806c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.174+0000 7fe9a05e8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe998071af0 con 0x7fe9980727c0 2024-09-15T21:46:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.174+0000 7fe9a05e8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fe9980712b0 con 0x7fe998155420 2024-09-15T21:46:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.174+0000 7fe9a05e8640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fe998071df0 con 0x7fe998155040 2024-09-15T21:46:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.174+0000 7fe99eb85640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe998155420 0x7fe99806c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.174+0000 7fe99eb85640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe998155420 0x7fe99806c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48756/0 (socket says 172.21.15.100:48756) 2024-09-15T21:46:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.174+0000 7fe99eb85640 1 -- 172.21.15.100:0/604530228 learned_addr learned my addr 172.21.15.100:0/604530228 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.174+0000 7fe99e384640 1 --2- 172.21.15.100:0/604530228 >> v2:172.21.15.122:3300/0 conn(0x7fe998155040 0x7fe998072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.174+0000 7fe99eb85640 1 -- 172.21.15.100:0/604530228 >> v2:172.21.15.122:3300/0 conn(0x7fe998155040 msgr2=0x7fe998072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.174+0000 7fe99eb85640 1 --2- 172.21.15.100:0/604530228 >> v2:172.21.15.122:3300/0 conn(0x7fe998155040 0x7fe998072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.174+0000 7fe99eb85640 1 -- 172.21.15.100:0/604530228 >> v1:172.21.15.100:6789/0 conn(0x7fe9980727c0 legacy=0x7fe99806bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.174+0000 7fe99eb85640 1 -- 172.21.15.100:0/604530228 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99815ba30 con 0x7fe998155420 2024-09-15T21:46:58.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.174+0000 7fe99e384640 1 --2- 172.21.15.100:0/604530228 >> v2:172.21.15.122:3300/0 conn(0x7fe998155040 0x7fe998072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.175+0000 7fe99eb85640 1 --2- 172.21.15.100:0/604530228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe998155420 0x7fe99806c2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe99806fd80 tx=0x7fe99002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51706da6129cfe1e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.175+0000 7fe99d382640 1 -- 172.21.15.100:0/604530228 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990037070 con 0x7fe998155420 2024-09-15T21:46:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.175+0000 7fe99d382640 1 -- 172.21.15.100:0/604530228 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe990032440 con 0x7fe998155420 2024-09-15T21:46:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.175+0000 7fe99d382640 1 -- 172.21.15.100:0/604530228 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990032810 con 0x7fe998155420 2024-09-15T21:46:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.176+0000 7fe9a05e8640 1 -- 172.21.15.100:0/604530228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe998155420 msgr2=0x7fe99806c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.176+0000 7fe9a05e8640 1 --2- 172.21.15.100:0/604530228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe998155420 0x7fe99806c2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe99806fd80 tx=0x7fe99002a270 comp rx=0 tx=0).stop 2024-09-15T21:46:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.176+0000 7fe9a05e8640 1 -- 172.21.15.100:0/604530228 shutdown_connections 2024-09-15T21:46:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.176+0000 7fe9a05e8640 1 --2- 172.21.15.100:0/604530228 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe998155420 0x7fe99806c2f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.176+0000 7fe9a05e8640 1 --2- 172.21.15.100:0/604530228 >> v2:172.21.15.122:3300/0 conn(0x7fe998155040 0x7fe998072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.176+0000 7fe9a05e8640 1 -- 172.21.15.100:0/604530228 >> 172.21.15.100:0/604530228 conn(0x7fe9980ccbd0 msgr2=0x7fe9980ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.176+0000 7fe9a05e8640 1 -- 172.21.15.100:0/604530228 shutdown_connections 2024-09-15T21:46:58.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.176+0000 7fe9a05e8640 1 -- 172.21.15.100:0/604530228 wait complete. 2024-09-15T21:46:58.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.177+0000 7fe9a05e8640 1 Processor -- start 2024-09-15T21:46:58.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.177+0000 7fe9a05e8640 1 -- start start 2024-09-15T21:46:58.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.177+0000 7fe9a05e8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9980727c0 0x7fe9980c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.177+0000 7fe9a05e8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe998155040 0x7fe998159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.177+0000 7fe9a05e8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe998155420 0x7fe99815a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.177+0000 7fe9a05e8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe99806d580 con 0x7fe998155040 2024-09-15T21:46:58.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.177+0000 7fe9a05e8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe99815bcd0 con 0x7fe9980727c0 2024-09-15T21:46:58.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.177+0000 7fe9a05e8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe99806d860 con 0x7fe998155420 2024-09-15T21:46:58.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.178+0000 7fe99db83640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe998155040 0x7fe998159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.178+0000 7fe99db83640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe998155040 0x7fe998159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48764/0 (socket says 172.21.15.100:48764) 2024-09-15T21:46:58.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.178+0000 7fe99db83640 1 -- 172.21.15.100:0/4089085888 learned_addr learned my addr 172.21.15.100:0/4089085888 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:58.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.178+0000 7fe99e384640 1 --2- 172.21.15.100:0/4089085888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9980727c0 0x7fe9980c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.178+0000 7fe99eb85640 1 --2- 172.21.15.100:0/4089085888 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe998155420 0x7fe99815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.178+0000 7fe99db83640 1 -- 172.21.15.100:0/4089085888 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe998155420 msgr2=0x7fe99815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.178+0000 7fe99db83640 1 --2- 172.21.15.100:0/4089085888 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe998155420 0x7fe99815a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.178+0000 7fe99db83640 1 -- 172.21.15.100:0/4089085888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9980727c0 msgr2=0x7fe9980c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.178+0000 7fe99db83640 1 --2- 172.21.15.100:0/4089085888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9980727c0 0x7fe9980c45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.178+0000 7fe99db83640 1 -- 172.21.15.100:0/4089085888 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe998155e40 con 0x7fe998155040 2024-09-15T21:46:58.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.178+0000 7fe99db83640 1 --2- 172.21.15.100:0/4089085888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe998155040 0x7fe998159fc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe98800c470 tx=0x7fe98800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:58.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.178+0000 7fe98effd640 1 -- 172.21.15.100:0/4089085888 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9880052d0 con 0x7fe998155040 2024-09-15T21:46:58.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.178+0000 7fe98effd640 1 -- 172.21.15.100:0/4089085888 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe988005490 con 0x7fe998155040 2024-09-15T21:46:58.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.178+0000 7fe98effd640 1 -- 172.21.15.100:0/4089085888 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988007260 con 0x7fe998155040 2024-09-15T21:46:58.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.178+0000 7fe9a05e8640 1 -- 172.21.15.100:0/4089085888 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe998156120 con 0x7fe998155040 2024-09-15T21:46:58.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.178+0000 7fe9a05e8640 1 -- 172.21.15.100:0/4089085888 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9981566e0 con 0x7fe998155040 2024-09-15T21:46:58.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.180+0000 7fe98effd640 1 -- 172.21.15.100:0/4089085888 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe988007400 con 0x7fe998155040 2024-09-15T21:46:58.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.180+0000 7fe98effd640 1 --2- 172.21.15.100:0/4089085888 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe97c06c2e0 0x7fe97c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.181+0000 7fe98effd640 1 -- 172.21.15.100:0/4089085888 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe98808a860 con 0x7fe998155040 2024-09-15T21:46:58.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.181+0000 7fe99e384640 1 --2- 172.21.15.100:0/4089085888 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe97c06c2e0 0x7fe97c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.181+0000 7fe99e384640 1 --2- 172.21.15.100:0/4089085888 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe97c06c2e0 0x7fe97c06e7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe99400f500 tx=0x7fe994007770 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:58.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.181+0000 7fe9a3461740 1 Processor -- start 2024-09-15T21:46:58.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.182+0000 7fe9a3461740 1 -- start start 2024-09-15T21:46:58.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.182+0000 7fe9a3461740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317c1750 0x55b3317f99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.182+0000 7fe9a3461740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 0x55b3317f9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.182+0000 7fe9a3461740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3317c1b30 0x55b3317bda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.182+0000 7fe9a3461740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b331722520 con 0x55b3317bef80 2024-09-15T21:46:58.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.182+0000 7fe9a3461740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b3316ca3e0 con 0x55b3317c1b30 2024-09-15T21:46:58.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.182+0000 7fe9a3461740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x55b33171e4e0 con 0x55b3317c1750 2024-09-15T21:46:58.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.182+0000 7fe99db83640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 0x55b3317f9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.182+0000 7fe99db83640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 0x55b3317f9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48776/0 (socket says 172.21.15.100:48776) 2024-09-15T21:46:58.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.182+0000 7fe99db83640 1 -- 172.21.15.100:0/2821036011 learned_addr learned my addr 172.21.15.100:0/2821036011 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:58.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.183+0000 7fe99e384640 1 --2- 172.21.15.100:0/2821036011 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317c1750 0x55b3317f99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.183+0000 7fe99eb85640 1 --2- 172.21.15.100:0/2821036011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3317c1b30 0x55b3317bda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.183+0000 7fe99db83640 1 -- 172.21.15.100:0/2821036011 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317c1750 msgr2=0x55b3317f99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.183+0000 7fe99db83640 1 --2- 172.21.15.100:0/2821036011 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317c1750 0x55b3317f99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.183+0000 7fe99db83640 1 -- 172.21.15.100:0/2821036011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3317c1b30 msgr2=0x55b3317bda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.183+0000 7fe99db83640 1 --2- 172.21.15.100:0/2821036011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3317c1b30 0x55b3317bda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.183+0000 7fe99db83640 1 -- 172.21.15.100:0/2821036011 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3316e52f0 con 0x55b3317bef80 2024-09-15T21:46:58.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.183+0000 7fe99db83640 1 --2- 172.21.15.100:0/2821036011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 0x55b3317f9f20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe988054d40 tx=0x7fe98800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4695bde307a8221 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:58.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.183+0000 7fe97bfff640 1 -- 172.21.15.100:0/2821036011 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98805f070 con 0x55b3317bef80 2024-09-15T21:46:58.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.183+0000 7fe97bfff640 1 -- 172.21.15.100:0/2821036011 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe988056be0 con 0x55b3317bef80 2024-09-15T21:46:58.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.183+0000 7fe97bfff640 1 -- 172.21.15.100:0/2821036011 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe988059b60 con 0x55b3317bef80 2024-09-15T21:46:58.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.184+0000 7fe9a3461740 1 -- 172.21.15.100:0/2821036011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 msgr2=0x55b3317f9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.184+0000 7fe9a3461740 1 --2- 172.21.15.100:0/2821036011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 0x55b3317f9f20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe988054d40 tx=0x7fe98800a000 comp rx=0 tx=0).stop 2024-09-15T21:46:58.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.184+0000 7fe9a3461740 1 -- 172.21.15.100:0/2821036011 shutdown_connections 2024-09-15T21:46:58.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.184+0000 7fe9a3461740 1 --2- 172.21.15.100:0/2821036011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3317c1b30 0x55b3317bda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.184+0000 7fe9a3461740 1 --2- 172.21.15.100:0/2821036011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 0x55b3317f9f20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.184+0000 7fe9a3461740 1 --2- 172.21.15.100:0/2821036011 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317c1750 0x55b3317f99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.184+0000 7fe9a3461740 1 -- 172.21.15.100:0/2821036011 >> 172.21.15.100:0/2821036011 conn(0x55b3317bfe70 msgr2=0x55b3317c0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:58.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.184+0000 7fe9a3461740 1 -- 172.21.15.100:0/2821036011 shutdown_connections 2024-09-15T21:46:58.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.184+0000 7fe9a3461740 1 -- 172.21.15.100:0/2821036011 wait complete. 2024-09-15T21:46:58.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.184+0000 7fe9a3461740 1 -- start start 2024-09-15T21:46:58.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.184+0000 7fe9a3461740 1 Processor -- start 2024-09-15T21:46:58.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.185+0000 7fe9a3461740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 0x55b3317f40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.185+0000 7fe9a3461740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3317c1750 0x55b3317e4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.185+0000 7fe9a3461740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317c1b30 0x55b3318145d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.185+0000 7fe9a3461740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b331722520 con 0x55b3317bef80 2024-09-15T21:46:58.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.185+0000 7fe9a3461740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b3316ab450 con 0x55b3317c1750 2024-09-15T21:46:58.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.185+0000 7fe9a3461740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x55b3317c0fe0 con 0x55b3317c1b30 2024-09-15T21:46:58.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.185+0000 7fe9a3461740 10 client.0 _ll_register_callbacks cb 0x55b331434ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T21:46:58.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.185+0000 7fe99e384640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 0x55b3317f40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.185+0000 7fe99e384640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 0x55b3317f40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48786/0 (socket says 172.21.15.100:48786) 2024-09-15T21:46:58.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.185+0000 7fe99e384640 1 -- 172.21.15.100:0/3707005704 learned_addr learned my addr 172.21.15.100:0/3707005704 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:58.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.185+0000 7fe99eb85640 1 --2- 172.21.15.100:0/3707005704 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317c1b30 0x55b3318145d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.185+0000 7fe99db83640 1 --2- 172.21.15.100:0/3707005704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3317c1750 0x55b3317e4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe99e384640 1 -- 172.21.15.100:0/3707005704 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317c1b30 msgr2=0x55b3318145d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe99e384640 1 --2- 172.21.15.100:0/3707005704 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317c1b30 0x55b3318145d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe99e384640 1 -- 172.21.15.100:0/3707005704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3317c1750 msgr2=0x55b3317e4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe99e384640 1 --2- 172.21.15.100:0/3707005704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3317c1750 0x55b3317e4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe99e384640 1 -- 172.21.15.100:0/3707005704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3314352d0 con 0x55b3317bef80 2024-09-15T21:46:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe99e384640 1 --2- 172.21.15.100:0/3707005704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 0x55b3317f40b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe99400deb0 tx=0x7fe994004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b79905646017158 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe9797fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-15T21:46:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe9797fa640 1 -- 172.21.15.100:0/3707005704 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994029060 con 0x55b3317bef80 2024-09-15T21:46:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe9797fa640 1 -- 172.21.15.100:0/3707005704 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe99401bbe0 con 0x55b3317bef80 2024-09-15T21:46:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe9a3461740 1 -- 172.21.15.100:0/3707005704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe988056be0 con 0x55b3317bef80 2024-09-15T21:46:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe9797fa640 1 -- 172.21.15.100:0/3707005704 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fe99402f050 con 0x55b3317bef80 2024-09-15T21:46:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe9a3461740 10 client.4761 fetch_fsmap learned FSMap version 17 2024-09-15T21:46:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe9a3461740 1 -- 172.21.15.100:0/3707005704 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55b3314ff6b0 con 0x55b3317bef80 2024-09-15T21:46:58.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe9797fa640 1 -- 172.21.15.100:0/3707005704 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe988056be0 con 0x55b3317bef80 2024-09-15T21:46:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe9797fa640 1 -- 172.21.15.100:0/3707005704 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 17) ==== 4792+0+0 (secure 0 0 0) 0x7fe994032020 con 0x55b3317bef80 2024-09-15T21:46:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe9a3461740 10 client.4761 fetch_fsmap finished waiting for FSMap version 17 2024-09-15T21:46:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe9a3461740 10 client.4761 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-15T21:46:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe9a3461740 20 client.4761 populate_metadata read hostname 'smithi100' 2024-09-15T21:46:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe9a3461740 1 --2- 172.21.15.100:0/3707005704 >> [v2:172.21.15.100:6837/602064326,v1:172.21.15.100:6839/602064326] conn(0x55b3317e2d20 0x55b33181f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe9a3461740 4 client.4761 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-15T21:46:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.186+0000 7fe9a3461740 1 -- 172.21.15.100:0/3707005704 --> [v2:172.21.15.100:6837/602064326,v1:172.21.15.100:6839/602064326] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b33172fc50 con 0x55b3317e2d20 2024-09-15T21:46:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.187+0000 7fe99db83640 1 --2- 172.21.15.100:0/3707005704 >> [v2:172.21.15.100:6837/602064326,v1:172.21.15.100:6839/602064326] conn(0x55b3317e2d20 0x55b33181f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.187+0000 7fe9797fa640 10 client.4761 ms_handle_connect on v2:172.21.15.100:6837/602064326 2024-09-15T21:46:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.187+0000 7fe99db83640 1 --2- 172.21.15.100:0/3707005704 >> [v2:172.21.15.100:6837/602064326,v1:172.21.15.100:6839/602064326] conn(0x55b3317e2d20 0x55b33181f710 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50e2a815dd8931ed server_cookie=cfc33bd1760ee4a9 in_seq=0 out_seq=0 2024-09-15T21:46:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.188+0000 7fe9797fa640 1 -- 172.21.15.100:0/3707005704 <== mds.0 v2:172.21.15.100:6837/602064326 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b33172fc50 con 0x55b3317e2d20 2024-09-15T21:46:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.188+0000 7fe9797fa640 10 client.4761 handle_command_reply: tid=0 2024-09-15T21:46:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.188+0000 7fe9a3461740 1 client.4761 shutdown 2024-09-15T21:46:58.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.188+0000 7fe9a3461740 1 -- 172.21.15.100:0/3707005704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 msgr2=0x55b3317f40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.188+0000 7fe9a3461740 1 --2- 172.21.15.100:0/3707005704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 0x55b3317f40b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe99400deb0 tx=0x7fe994004ab0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.188+0000 7fe9a3461740 1 -- 172.21.15.100:0/3707005704 shutdown_connections 2024-09-15T21:46:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.188+0000 7fe9a3461740 1 --2- 172.21.15.100:0/3707005704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3317c1750 0x55b3317e4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.188+0000 7fe9a3461740 1 --2- 172.21.15.100:0/3707005704 >> [v2:172.21.15.100:6837/602064326,v1:172.21.15.100:6839/602064326] conn(0x55b3317e2d20 0x55b33181f710 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.188+0000 7fe9a3461740 1 --2- 172.21.15.100:0/3707005704 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317c1b30 0x55b3318145d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.188+0000 7fe9a3461740 1 --2- 172.21.15.100:0/3707005704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 0x55b3317f40b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.188+0000 7fe9797fa640 0 client.4761 ms_handle_reset on v2:172.21.15.100:6837/602064326 2024-09-15T21:46:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.188+0000 7fe9a3461740 1 -- 172.21.15.100:0/3707005704 >> 172.21.15.100:0/3707005704 conn(0x55b3317bfe70 msgr2=0x55b3317fa400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.188+0000 7fe9a3461740 1 -- 172.21.15.100:0/3707005704 shutdown_connections 2024-09-15T21:46:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.188+0000 7fe9a3461740 1 -- 172.21.15.100:0/3707005704 wait complete. 2024-09-15T21:46:58.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.188+0000 7fe9a3461740 20 client.4761 trim_cache size 0 max 16384 2024-09-15T21:46:58.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.205+0000 7fe9a3461740 1 Processor -- start 2024-09-15T21:46:58.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.206+0000 7fe9a3461740 1 -- start start 2024-09-15T21:46:58.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.206+0000 7fe9a3461740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 0x55b3317c0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.206+0000 7fe9a3461740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3317c1750 0x55b3317e5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.206+0000 7fe9a3461740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317c1b30 0x55b3317e6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.206+0000 7fe9a3461740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe998150740 con 0x55b3317bef80 2024-09-15T21:46:58.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.206+0000 7fe9a3461740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b3317e76f0 con 0x55b3317c1750 2024-09-15T21:46:58.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.206+0000 7fe9a3461740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x55b3317e31f0 con 0x55b3317c1b30 2024-09-15T21:46:58.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.206+0000 7fe99e384640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 0x55b3317c0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.206+0000 7fe99e384640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 0x55b3317c0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48794/0 (socket says 172.21.15.100:48794) 2024-09-15T21:46:58.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.206+0000 7fe99e384640 1 -- 172.21.15.100:0/81792141 learned_addr learned my addr 172.21.15.100:0/81792141 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:58.210 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.206+0000 7fe99e384640 1 -- 172.21.15.100:0/81792141 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317c1b30 msgr2=0x55b3317e6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.206+0000 7fe99e384640 1 --2- 172.21.15.100:0/81792141 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317c1b30 0x55b3317e6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.206+0000 7fe99e384640 1 -- 172.21.15.100:0/81792141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3317c1750 msgr2=0x55b3317e5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.206+0000 7fe99e384640 1 --2- 172.21.15.100:0/81792141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3317c1750 0x55b3317e5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.206+0000 7fe99e384640 1 -- 172.21.15.100:0/81792141 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3317c21d0 con 0x55b3317bef80 2024-09-15T21:46:58.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.207+0000 7fe99e384640 1 --2- 172.21.15.100:0/81792141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 0x55b3317c0c70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe99401b8e0 tx=0x7fe994020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0d12ac565838e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:58.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.207+0000 7fe97b7fe640 1 -- 172.21.15.100:0/81792141 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe994037020 con 0x55b3317bef80 2024-09-15T21:46:58.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.207+0000 7fe97b7fe640 1 -- 172.21.15.100:0/81792141 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe99400db40 con 0x55b3317bef80 2024-09-15T21:46:58.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.207+0000 7fe97b7fe640 1 -- 172.21.15.100:0/81792141 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9940248e0 con 0x55b3317bef80 2024-09-15T21:46:58.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.207+0000 7fe9a3461740 1 -- 172.21.15.100:0/81792141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 msgr2=0x55b3317c0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.207+0000 7fe9a3461740 1 --2- 172.21.15.100:0/81792141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 0x55b3317c0c70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe99401b8e0 tx=0x7fe994020000 comp rx=0 tx=0).stop 2024-09-15T21:46:58.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.207+0000 7fe9a3461740 1 -- 172.21.15.100:0/81792141 shutdown_connections 2024-09-15T21:46:58.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.207+0000 7fe9a3461740 1 --2- 172.21.15.100:0/81792141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3317c1750 0x55b3317e5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.207+0000 7fe9a3461740 1 --2- 172.21.15.100:0/81792141 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317c1b30 0x55b3317e6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.207+0000 7fe9a3461740 1 --2- 172.21.15.100:0/81792141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317bef80 0x55b3317c0c70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.207+0000 7fe9a3461740 1 -- 172.21.15.100:0/81792141 >> 172.21.15.100:0/81792141 conn(0x55b3317bfe70 msgr2=0x55b331813ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:58.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.207+0000 7fe9a3461740 1 -- 172.21.15.100:0/81792141 shutdown_connections 2024-09-15T21:46:58.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.207+0000 7fe9a3461740 1 -- 172.21.15.100:0/81792141 wait complete. 2024-09-15T21:46:58.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.208+0000 7fe9a3461740 1 -- start start 2024-09-15T21:46:58.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.208+0000 7fe9a3461740 1 Processor -- start 2024-09-15T21:46:58.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe9a3461740 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317bef80 0x55b3317be140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe9a3461740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3317c1750 0x55b33180d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe9a3461740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317c1b30 0x55b33180dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe9a3461740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x55b3317e7e20 con 0x55b3317c1b30 2024-09-15T21:46:58.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe9a3461740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55b3317e6dc0 con 0x55b3317c1750 2024-09-15T21:46:58.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe9a3461740 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x55b3317f1380 con 0x55b3317bef80 2024-09-15T21:46:58.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe9a3461740 10 client.0 _ll_register_callbacks cb 0x7fe9940072a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-15T21:46:58.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe99eb85640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317c1b30 0x55b33180dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe99eb85640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317c1b30 0x55b33180dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48810/0 (socket says 172.21.15.100:48810) 2024-09-15T21:46:58.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe99eb85640 1 -- 172.21.15.100:0/446014070 learned_addr learned my addr 172.21.15.100:0/446014070 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:58.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe99e384640 1 --2- 172.21.15.100:0/446014070 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317bef80 0x55b3317be140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe99db83640 1 --2- 172.21.15.100:0/446014070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3317c1750 0x55b33180d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe99eb85640 1 -- 172.21.15.100:0/446014070 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317bef80 msgr2=0x55b3317be140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe99eb85640 1 --2- 172.21.15.100:0/446014070 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317bef80 0x55b3317be140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe99eb85640 1 -- 172.21.15.100:0/446014070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3317c1750 msgr2=0x55b33180d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe99eb85640 1 --2- 172.21.15.100:0/446014070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3317c1750 0x55b33180d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe99eb85640 1 -- 172.21.15.100:0/446014070 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b33180a180 con 0x55b3317c1b30 2024-09-15T21:46:58.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe99e384640 1 --2- 172.21.15.100:0/446014070 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317bef80 0x55b3317be140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:46:58.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe99eb85640 1 --2- 172.21.15.100:0/446014070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317c1b30 0x55b33180dae0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe990009930 tx=0x7fe9900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86caafbeaec31cd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:58.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe9797fa640 10 client.0 ms_handle_connect on v2:172.21.15.100:3300/0 2024-09-15T21:46:58.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.209+0000 7fe9797fa640 1 -- 172.21.15.100:0/446014070 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990037040 con 0x55b3317c1b30 2024-09-15T21:46:58.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.210+0000 7fe9797fa640 1 -- 172.21.15.100:0/446014070 <== mon.0 v2:172.21.15.100:3300/0 2 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe990046440 con 0x55b3317c1b30 2024-09-15T21:46:58.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.210+0000 7fe9a3461740 1 -- 172.21.15.100:0/446014070 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe99400db40 con 0x55b3317c1b30 2024-09-15T21:46:58.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.210+0000 7fe9797fa640 1 -- 172.21.15.100:0/446014070 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fe99004e050 con 0x55b3317c1b30 2024-09-15T21:46:58.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.210+0000 7fe9a3461740 10 client.4767 fetch_fsmap learned FSMap version 17 2024-09-15T21:46:58.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.210+0000 7fe9a3461740 1 -- 172.21.15.100:0/446014070 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55b33180af20 con 0x55b3317c1b30 2024-09-15T21:46:58.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.210+0000 7fe9797fa640 1 -- 172.21.15.100:0/446014070 <== mon.0 v2:172.21.15.100:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe99400db40 con 0x55b3317c1b30 2024-09-15T21:46:58.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.210+0000 7fe9797fa640 1 -- 172.21.15.100:0/446014070 <== mon.0 v2:172.21.15.100:3300/0 5 ==== fsmap(e 17) ==== 4792+0+0 (secure 0 0 0) 0x7fe990051020 con 0x55b3317c1b30 2024-09-15T21:46:58.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.210+0000 7fe9a3461740 10 client.4767 fetch_fsmap finished waiting for FSMap version 17 2024-09-15T21:46:58.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.210+0000 7fe9a3461740 10 client.4767 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-15T21:46:58.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.210+0000 7fe9a3461740 20 client.4767 populate_metadata read hostname 'smithi100' 2024-09-15T21:46:58.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.210+0000 7fe9a3461740 1 --2- 172.21.15.100:0/446014070 >> [v2:172.21.15.100:6837/602064326,v1:172.21.15.100:6839/602064326] conn(0x55b331878670 0x55b33188b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.210+0000 7fe9a3461740 4 client.4767 mds_command: new command op to 4293 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-15T21:46:58.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.210+0000 7fe9a3461740 1 -- 172.21.15.100:0/446014070 --> [v2:172.21.15.100:6837/602064326,v1:172.21.15.100:6839/602064326] -- command(tid 0: {"prefix": "status"}) -- 0x55b331729fa0 con 0x55b331878670 2024-09-15T21:46:58.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.210+0000 7fe99e384640 1 --2- 172.21.15.100:0/446014070 >> [v2:172.21.15.100:6837/602064326,v1:172.21.15.100:6839/602064326] conn(0x55b331878670 0x55b33188b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.210+0000 7fe99e384640 1 --2- 172.21.15.100:0/446014070 >> [v2:172.21.15.100:6837/602064326,v1:172.21.15.100:6839/602064326] conn(0x55b331878670 0x55b33188b1c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81167ad1515ac35e server_cookie=9c03e55d9781d2c0 in_seq=0 out_seq=0 2024-09-15T21:46:58.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.210+0000 7fe9797fa640 10 client.4767 ms_handle_connect on v2:172.21.15.100:6837/602064326 2024-09-15T21:46:58.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9797fa640 1 -- 172.21.15.100:0/446014070 <== mds.0 v2:172.21.15.100:6837/602064326 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55b331729fa0 con 0x55b331878670 2024-09-15T21:46:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9797fa640 10 client.4767 handle_command_reply: tid=0 2024-09-15T21:46:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a3461740 1 client.4767 shutdown 2024-09-15T21:46:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a3461740 1 -- 172.21.15.100:0/446014070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317c1b30 msgr2=0x55b33180dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a3461740 1 --2- 172.21.15.100:0/446014070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317c1b30 0x55b33180dae0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe990009930 tx=0x7fe9900046c0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a3461740 1 -- 172.21.15.100:0/446014070 shutdown_connections 2024-09-15T21:46:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a3461740 1 --2- 172.21.15.100:0/446014070 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55b3317c1750 0x55b33180d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a3461740 1 --2- 172.21.15.100:0/446014070 >> [v2:172.21.15.100:6837/602064326,v1:172.21.15.100:6839/602064326] conn(0x55b331878670 0x55b33188b1c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a3461740 1 --2- 172.21.15.100:0/446014070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55b3317c1b30 0x55b33180dae0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a3461740 1 --2- 172.21.15.100:0/446014070 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x55b3317bef80 0x55b3317be140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9797fa640 0 client.4767 ms_handle_reset on v2:172.21.15.100:6837/602064326 2024-09-15T21:46:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a3461740 1 -- 172.21.15.100:0/446014070 >> 172.21.15.100:0/446014070 conn(0x55b3317bfe70 msgr2=0x55b3317e5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a3461740 1 -- 172.21.15.100:0/446014070 shutdown_connections 2024-09-15T21:46:58.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a3461740 1 -- 172.21.15.100:0/446014070 wait complete. 2024-09-15T21:46:58.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a3461740 20 client.4767 trim_cache size 0 max 16384 2024-09-15T21:46:58.220 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:46:58.220 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_fsid": "30e2fff4-2171-4531-8806-0e03b63f494f", 2024-09-15T21:46:58.220 INFO:teuthology.orchestra.run.smithi100.stdout: "whoami": 0, 2024-09-15T21:46:58.220 INFO:teuthology.orchestra.run.smithi100.stdout: "id": 4293, 2024-09-15T21:46:58.220 INFO:teuthology.orchestra.run.smithi100.stdout: "want_state": "up:active", 2024-09-15T21:46:58.220 INFO:teuthology.orchestra.run.smithi100.stdout: "state": "up:active", 2024-09-15T21:46:58.220 INFO:teuthology.orchestra.run.smithi100.stdout: "fs_name": "cephfs", 2024-09-15T21:46:58.220 INFO:teuthology.orchestra.run.smithi100.stdout: "rank_uptime": 34.801974602000001, 2024-09-15T21:46:58.220 INFO:teuthology.orchestra.run.smithi100.stdout: "mdsmap_epoch": 17, 2024-09-15T21:46:58.221 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch": 22, 2024-09-15T21:46:58.221 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_epoch_barrier": 22, 2024-09-15T21:46:58.221 INFO:teuthology.orchestra.run.smithi100.stdout: "uptime": 40.827216456999999 2024-09-15T21:46:58.221 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:46:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a05e8640 1 -- 172.21.15.100:0/4089085888 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe97c06c2e0 msgr2=0x7fe97c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a05e8640 1 --2- 172.21.15.100:0/4089085888 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe97c06c2e0 0x7fe97c06e7a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe99400f500 tx=0x7fe994007770 comp rx=0 tx=0).stop 2024-09-15T21:46:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a05e8640 1 -- 172.21.15.100:0/4089085888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe998155040 msgr2=0x7fe998159fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a05e8640 1 --2- 172.21.15.100:0/4089085888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe998155040 0x7fe998159fc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe98800c470 tx=0x7fe98800c930 comp rx=0 tx=0).stop 2024-09-15T21:46:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a05e8640 1 -- 172.21.15.100:0/4089085888 shutdown_connections 2024-09-15T21:46:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a05e8640 1 --2- 172.21.15.100:0/4089085888 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe97c06c2e0 0x7fe97c06e7a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a05e8640 1 --2- 172.21.15.100:0/4089085888 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe998155420 0x7fe99815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a05e8640 1 --2- 172.21.15.100:0/4089085888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe998155040 0x7fe998159fc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a05e8640 1 --2- 172.21.15.100:0/4089085888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe9980727c0 0x7fe9980c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.211+0000 7fe9a05e8640 1 -- 172.21.15.100:0/4089085888 >> 172.21.15.100:0/4089085888 conn(0x7fe9980ccbd0 msgr2=0x7fe99806bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:58.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.212+0000 7fe9a05e8640 1 -- 172.21.15.100:0/4089085888 shutdown_connections 2024-09-15T21:46:58.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.212+0000 7fe9a05e8640 1 -- 172.21.15.100:0/4089085888 wait complete. 2024-09-15T21:46:58.224 INFO:teuthology.run_tasks:Running task sequential... 2024-09-15T21:46:58.234 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-15T21:46:58.234 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-15T21:46:58.235 INFO:teuthology.task.print:Enabling mgr modules 2024-09-15T21:46:58.235 INFO:teuthology.run_tasks:Running task kclient... 2024-09-15T21:46:58.373 INFO:tasks.kclient:Mounting kernel clients... 2024-09-15T21:46:58.373 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-15T21:46:58.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T21:46:58.373 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-15T21:46:58.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T21:46:58.390 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-09-15T21:46:58.456 INFO:teuthology.orchestra.run.smithi100.stderr:Cannot find device "ceph-brx" 2024-09-15T21:46:58.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:46:58.459 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-09-15T21:46:58.459 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-15T21:46:58.459 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-15T21:46:58.459 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-15T21:46:58.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.596+0000 7f5c94ba6640 1 Processor -- start 2024-09-15T21:46:58.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.596+0000 7f5c94ba6640 1 -- start start 2024-09-15T21:46:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.597+0000 7f5c94ba6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5c90155440 0x7f5c900cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.597+0000 7f5c94ba6640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f5c900cb7d0 0x7f5c900cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.597+0000 7f5c94ba6640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c9015e7a0 con 0x7f5c9006d0e0 2024-09-15T21:46:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.597+0000 7f5c94ba6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f5c9015b470 con 0x7f5c90155440 2024-09-15T21:46:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.597+0000 7f5c94ba6640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f5c90155820 con 0x7f5c900cb7d0 2024-09-15T21:46:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.597+0000 7f5c8e59c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5c90155440 0x7f5c900cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.597+0000 7f5c8e59c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5c90155440 0x7f5c900cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48826/0 (socket says 172.21.15.100:48826) 2024-09-15T21:46:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.597+0000 7f5c8e59c640 1 -- 172.21.15.100:0/2863007208 learned_addr learned my addr 172.21.15.100:0/2863007208 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.597+0000 7f5c8e59c640 1 -- 172.21.15.100:0/2863007208 >> v2:172.21.15.122:3300/0 conn(0x7f5c900cb7d0 msgr2=0x7f5c900cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.597+0000 7f5c8e59c640 1 --2- 172.21.15.100:0/2863007208 >> v2:172.21.15.122:3300/0 conn(0x7f5c900cb7d0 0x7f5c900cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.597+0000 7f5c8e59c640 1 -- 172.21.15.100:0/2863007208 >> v1:172.21.15.122:6789/0 conn(0x7f5c9006d0e0 legacy=0x7f5c900cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:46:58.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.597+0000 7f5c8e59c640 1 -- 172.21.15.100:0/2863007208 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9015ea40 con 0x7f5c90155440 2024-09-15T21:46:58.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.598+0000 7f5c8e59c640 1 --2- 172.21.15.100:0/2863007208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c90155440 0x7f5c900cd420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5c90158ca0 tx=0x7f5c8002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1150d3baddb63878 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:58.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.598+0000 7f5c8dd9b640 1 -- 172.21.15.100:0/2863007208 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80037070 con 0x7f5c90155440 2024-09-15T21:46:58.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.598+0000 7f5c8dd9b640 1 -- 172.21.15.100:0/2863007208 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c80032440 con 0x7f5c90155440 2024-09-15T21:46:58.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.598+0000 7f5c8dd9b640 1 -- 172.21.15.100:0/2863007208 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80032810 con 0x7f5c90155440 2024-09-15T21:46:58.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.598+0000 7f5c94ba6640 1 -- 172.21.15.100:0/2863007208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c90155440 msgr2=0x7f5c900cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.598+0000 7f5c94ba6640 1 --2- 172.21.15.100:0/2863007208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c90155440 0x7f5c900cd420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5c90158ca0 tx=0x7f5c8002a1a0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.598+0000 7f5c94ba6640 1 -- 172.21.15.100:0/2863007208 shutdown_connections 2024-09-15T21:46:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.598+0000 7f5c94ba6640 1 --2- 172.21.15.100:0/2863007208 >> v2:172.21.15.122:3300/0 conn(0x7f5c900cb7d0 0x7f5c900cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.598+0000 7f5c94ba6640 1 --2- 172.21.15.100:0/2863007208 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c90155440 0x7f5c900cd420 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.598+0000 7f5c94ba6640 1 -- 172.21.15.100:0/2863007208 >> 172.21.15.100:0/2863007208 conn(0x7f5c900680d0 msgr2=0x7f5c9006d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.599+0000 7f5c94ba6640 1 -- 172.21.15.100:0/2863007208 shutdown_connections 2024-09-15T21:46:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.599+0000 7f5c94ba6640 1 -- 172.21.15.100:0/2863007208 wait complete. 2024-09-15T21:46:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.600+0000 7f5c94ba6640 1 Processor -- start 2024-09-15T21:46:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.600+0000 7f5c94ba6640 1 -- start start 2024-09-15T21:46:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c94ba6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c9006d0e0 0x7f5c9015de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c94ba6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c900cb7d0 0x7f5c901e6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c94ba6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5c90155440 0x7f5c901e6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c94ba6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9015f2e0 con 0x7f5c9006d0e0 2024-09-15T21:46:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c94ba6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5c9015f000 con 0x7f5c900cb7d0 2024-09-15T21:46:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c94ba6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5c9015b070 con 0x7f5c90155440 2024-09-15T21:46:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c8e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c9006d0e0 0x7f5c9015de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c8e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c9006d0e0 0x7f5c9015de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48832/0 (socket says 172.21.15.100:48832) 2024-09-15T21:46:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c8e59c640 1 -- 172.21.15.100:0/2725302784 learned_addr learned my addr 172.21.15.100:0/2725302784 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:46:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c8ed9d640 1 --2- 172.21.15.100:0/2725302784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5c90155440 0x7f5c901e6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c85d9b640 1 --2- 172.21.15.100:0/2725302784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c900cb7d0 0x7f5c901e6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c8e59c640 1 -- 172.21.15.100:0/2725302784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5c90155440 msgr2=0x7f5c901e6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c8e59c640 1 --2- 172.21.15.100:0/2725302784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5c90155440 0x7f5c901e6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c8e59c640 1 -- 172.21.15.100:0/2725302784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c900cb7d0 msgr2=0x7f5c901e6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c8e59c640 1 --2- 172.21.15.100:0/2725302784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c900cb7d0 0x7f5c901e6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c8e59c640 1 -- 172.21.15.100:0/2725302784 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c90147690 con 0x7f5c9006d0e0 2024-09-15T21:46:58.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c8e59c640 1 --2- 172.21.15.100:0/2725302784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c9006d0e0 0x7f5c9015de10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5c80000c00 tx=0x7f5c800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:58.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c877fe640 1 -- 172.21.15.100:0/2725302784 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80041070 con 0x7f5c9006d0e0 2024-09-15T21:46:58.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c877fe640 1 -- 172.21.15.100:0/2725302784 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c8003c870 con 0x7f5c9006d0e0 2024-09-15T21:46:58.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c877fe640 1 -- 172.21.15.100:0/2725302784 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c80037040 con 0x7f5c9006d0e0 2024-09-15T21:46:58.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c94ba6640 1 -- 172.21.15.100:0/2725302784 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c90147910 con 0x7f5c9006d0e0 2024-09-15T21:46:58.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.601+0000 7f5c94ba6640 1 -- 172.21.15.100:0/2725302784 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c90152700 con 0x7f5c9006d0e0 2024-09-15T21:46:58.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.602+0000 7f5c94ba6640 1 -- 172.21.15.100:0/2725302784 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c9015c450 con 0x7f5c9006d0e0 2024-09-15T21:46:58.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.603+0000 7f5c877fe640 1 -- 172.21.15.100:0/2725302784 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5c80044070 con 0x7f5c9006d0e0 2024-09-15T21:46:58.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.603+0000 7f5c877fe640 1 --2- 172.21.15.100:0/2725302784 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5c7006c2b0 0x7f5c7006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:46:58.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.603+0000 7f5c85d9b640 1 --2- 172.21.15.100:0/2725302784 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5c7006c2b0 0x7f5c7006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:46:58.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.603+0000 7f5c877fe640 1 -- 172.21.15.100:0/2725302784 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f5c800abea0 con 0x7f5c9006d0e0 2024-09-15T21:46:58.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.604+0000 7f5c85d9b640 1 --2- 172.21.15.100:0/2725302784 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5c7006c2b0 0x7f5c7006e770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5c900b3330 tx=0x7f5c78009110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:46:58.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.608+0000 7f5c877fe640 1 -- 172.21.15.100:0/2725302784 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5c9015c450 con 0x7f5c9006d0e0 2024-09-15T21:46:58.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.806+0000 7f5c94ba6640 1 -- 172.21.15.100:0/2725302784 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5c900b3940 con 0x7f5c9006d0e0 2024-09-15T21:46:58.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.807+0000 7f5c877fe640 1 -- 172.21.15.100:0/2725302784 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f5c90147690 con 0x7f5c9006d0e0 2024-09-15T21:46:58.809 INFO:teuthology.orchestra.run.smithi100.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-15T21:46:58.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.808+0000 7f5c94ba6640 1 -- 172.21.15.100:0/2725302784 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5c7006c2b0 msgr2=0x7f5c7006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.808+0000 7f5c94ba6640 1 --2- 172.21.15.100:0/2725302784 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5c7006c2b0 0x7f5c7006e770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5c900b3330 tx=0x7f5c78009110 comp rx=0 tx=0).stop 2024-09-15T21:46:58.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.808+0000 7f5c94ba6640 1 -- 172.21.15.100:0/2725302784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c9006d0e0 msgr2=0x7f5c9015de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:46:58.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.808+0000 7f5c94ba6640 1 --2- 172.21.15.100:0/2725302784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c9006d0e0 0x7f5c9015de10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5c80000c00 tx=0x7f5c800046c0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.808+0000 7f5c94ba6640 1 -- 172.21.15.100:0/2725302784 shutdown_connections 2024-09-15T21:46:58.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.808+0000 7f5c94ba6640 1 --2- 172.21.15.100:0/2725302784 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5c7006c2b0 0x7f5c7006e770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.808+0000 7f5c94ba6640 1 --2- 172.21.15.100:0/2725302784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5c900cb7d0 0x7f5c901e6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.808+0000 7f5c94ba6640 1 --2- 172.21.15.100:0/2725302784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5c90155440 0x7f5c901e6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.808+0000 7f5c94ba6640 1 --2- 172.21.15.100:0/2725302784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c9006d0e0 0x7f5c9015de10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:46:58.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.808+0000 7f5c94ba6640 1 -- 172.21.15.100:0/2725302784 >> 172.21.15.100:0/2725302784 conn(0x7f5c900680d0 msgr2=0x7f5c900cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:46:58.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.809+0000 7f5c94ba6640 1 -- 172.21.15.100:0/2725302784 shutdown_connections 2024-09-15T21:46:58.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58.809+0000 7f5c94ba6640 1 -- 172.21.15.100:0/2725302784 wait complete. 2024-09-15T21:46:58.821 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-15T21:46:58.822 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-15T21:46:58.822 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi100.front.sepia.ceph.com 2024-09-15T21:46:58.822 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-15T21:46:58.822 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-15T21:46:58.822 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-15T21:46:58.822 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-15T21:46:58.822 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-15T21:46:58.822 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-15T21:46:58.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T21:46:58.822 DEBUG:teuthology.orchestra.run.smithi100:> ip addr 2024-09-15T21:46:58.838 INFO:teuthology.orchestra.run.smithi100.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-15T21:46:58.838 INFO:teuthology.orchestra.run.smithi100.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-15T21:46:58.839 INFO:teuthology.orchestra.run.smithi100.stdout: inet 127.0.0.1/8 scope host lo 2024-09-15T21:46:58.839 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-15T21:46:58.839 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 ::1/128 scope host 2024-09-15T21:46:58.839 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-15T21:46:58.839 INFO:teuthology.orchestra.run.smithi100.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T21:46:58.839 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:28 brd ff:ff:ff:ff:ff:ff 2024-09-15T21:46:58.839 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f0 2024-09-15T21:46:58.839 INFO:teuthology.orchestra.run.smithi100.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T21:46:58.839 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:d8:a1:29 brd ff:ff:ff:ff:ff:ff 2024-09-15T21:46:58.839 INFO:teuthology.orchestra.run.smithi100.stdout: altname enp7s0f1 2024-09-15T21:46:58.839 INFO:teuthology.orchestra.run.smithi100.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-15T21:46:58.839 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:32 brd ff:ff:ff:ff:ff:ff 2024-09-15T21:46:58.840 INFO:teuthology.orchestra.run.smithi100.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-15T21:46:58.840 INFO:teuthology.orchestra.run.smithi100.stdout: link/ether 0c:c4:7a:8f:d7:33 brd ff:ff:ff:ff:ff:ff 2024-09-15T21:46:58.840 INFO:teuthology.orchestra.run.smithi100.stdout: inet 172.21.15.100/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-15T21:46:58.840 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft 42734sec preferred_lft 42734sec 2024-09-15T21:46:58.840 INFO:teuthology.orchestra.run.smithi100.stdout: inet6 fe80::ec4:7aff:fe8f:d733/64 scope link noprefixroute 2024-09-15T21:46:58.840 INFO:teuthology.orchestra.run.smithi100.stdout: valid_lft forever preferred_lft forever 2024-09-15T21:46:58.840 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-15T21:46:58.840 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T21:46:58.841 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-15T21:46:58.841 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add name ceph-brx type bridge 2024-09-15T21:46:58.841 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr flush dev ceph-brx 2024-09-15T21:46:58.841 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set ceph-brx up 2024-09-15T21:46:58.841 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-15T21:46:58.841 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-15T21:46:58.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T21:46:59.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T21:46:59.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T21:46:59.046 DEBUG:teuthology.orchestra.run.smithi100:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-15T21:46:59.071 INFO:teuthology.orchestra.run.smithi100.stdout:1 2024-09-15T21:46:59.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T21:46:59.073 DEBUG:teuthology.orchestra.run.smithi100:> route 2024-09-15T21:46:59.150 INFO:teuthology.orchestra.run.smithi100.stdout:Kernel IP routing table 2024-09-15T21:46:59.150 INFO:teuthology.orchestra.run.smithi100.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-15T21:46:59.150 INFO:teuthology.orchestra.run.smithi100.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-15T21:46:59.150 INFO:teuthology.orchestra.run.smithi100.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-15T21:46:59.151 INFO:teuthology.orchestra.run.smithi100.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-15T21:46:59.151 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T21:46:59.151 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-15T21:46:59.151 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-15T21:46:59.151 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-15T21:46:59.151 DEBUG:teuthology.orchestra.run.smithi100:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-15T21:46:59.151 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-15T21:46:59.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T21:46:59.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T21:46:59.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T21:46:59.461 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-15T21:46:59.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T21:46:59.477 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list-id 2024-09-15T21:46:59.532 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T21:46:59.532 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-15T21:46:59.532 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-15T21:46:59.533 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-15T21:46:59.533 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-15T21:46:59.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T21:46:59.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T21:46:59.651 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-15T21:46:59.651 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T21:46:59.652 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-15T21:46:59.652 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-15T21:46:59.652 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-15T21:46:59.652 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-15T21:46:59.652 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-15T21:46:59.652 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-15T21:46:59.652 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-15T21:46:59.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T21:46:59.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T21:46:59.800 DEBUG:teuthology.orchestra.run.smithi100:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-15T21:46:59.800 DEBUG:teuthology.orchestra.run.smithi100:> set -e 2024-09-15T21:46:59.801 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set brx.0 up 2024-09-15T21:46:59.801 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link set dev brx.0 master ceph-brx 2024-09-15T21:46:59.801 DEBUG:teuthology.orchestra.run.smithi100:> ') 2024-09-15T21:46:59.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:59 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T21:46:59.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:46:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T21:46:59.878 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T21:46:59.878 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-15T21:46:59.893 INFO:teuthology.orchestra.run.smithi100.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-15T21:46:59.893 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-15T21:46:59.893 DEBUG:teuthology.orchestra.run.smithi100:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-15T21:46:59.949 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-15T21:46:59.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T21:46:59.949 DEBUG:teuthology.orchestra.run.smithi100:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-15T21:47:00.041 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-15T21:47:00.041 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-15T21:47:00.175 INFO:teuthology.orchestra.run.smithi100.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-15T21:47:00.175 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-15T21:47:00.175 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: resolved to: "172.21.15.100:3300,172.21.15.122:3300,172.21.15.122:3301" 2024-09-15T21:47:00.176 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: trying mount with new device syntax: 0@30e2fff4-2171-4531-8806-0e03b63f494f.cephfs=/ 2024-09-15T21:47:00.176 INFO:teuthology.orchestra.run.smithi100.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.100:3300/172.21.15.122:3300/172.21.15.122:3301" will pass to kernel 2024-09-15T21:47:00.176 INFO:teuthology.orchestra.run.smithi100.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-15T21:47:00.177 INFO:teuthology.orchestra.run.smithi100.stdout: You just mounted a file system that supports labels which does not 2024-09-15T21:47:00.177 INFO:teuthology.orchestra.run.smithi100.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-15T21:47:00.177 INFO:teuthology.orchestra.run.smithi100.stdout: applications will generate AVC messages and not be allowed access to 2024-09-15T21:47:00.177 INFO:teuthology.orchestra.run.smithi100.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-15T21:47:00.179 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-15T21:47:00.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T21:47:00.179 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-15T21:47:00.209 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T21:47:00.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T21:47:00.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T21:47:00.309 INFO:teuthology.orchestra.run.smithi100.stdout:client4776 2024-09-15T21:47:00.310 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:47:00.310 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T21:47:00.326 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T21:47:00.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T21:47:00.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T21:47:00.681 INFO:teuthology.orchestra.run.smithi100.stdout:client4776 2024-09-15T21:47:00.681 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-15T21:47:00.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-15T21:47:00.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T21:47:00.742 INFO:teuthology.orchestra.run.smithi100.stdout:client4776 2024-09-15T21:47:00.742 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/30e2fff4-2171-4531-8806-0e03b63f494f.client4776/status') 2024-09-15T21:47:00.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-15T21:47:00.803 INFO:teuthology.orchestra.run.smithi100.stdout:instance: client.4776 (3)192.168.0.1:0/806046571 2024-09-15T21:47:00.803 INFO:teuthology.orchestra.run.smithi100.stdout:blocklisted: false 2024-09-15T21:47:00.803 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records in 2024-09-15T21:47:00.803 INFO:teuthology.orchestra.run.smithi100.stderr:0+1 records out 2024-09-15T21:47:00.803 INFO:teuthology.orchestra.run.smithi100.stderr:68 bytes copied, 2.4144e-05 s, 2.8 MB/s 2024-09-15T21:47:00.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-15T21:47:00.811 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-15T21:47:00.833 INFO:tasks.thrashosds:config is {'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-15T21:47:00.833 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'max_mds': 5, 'session_timeout': 300}, 'conf': {'client': {'client mount timeout': 600, 'client_shutdown_timeout': 120, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds oft prefetch dirfrags': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'mds_sleep_rank_change': 5000000.0, 'mon client directed command retry': 5, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds osd', 'ms inject socket failures': 2500, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon client directed command retry': 5, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'ms inject delay max': 1, 'ms inject delay probability': 0.005, 'ms inject delay type': 'client mds', 'ms inject socket failures': 2500, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'but it is still running', 'objects unfound and apparently lost', 'MDS_SLOW_METADATA_IO', 'MDS_TRIM'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'counters': {'mds': ['mds.exported', 'mds.imported']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'ad35db06e8924912f01e32b35576297c0f1352b4'}} 2024-09-15T21:47:00.833 INFO:tasks.thrashosds:config is {'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-15T21:47:00.834 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-15T21:47:00.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.934+0000 7fd98fc9b640 1 Processor -- start 2024-09-15T21:47:00.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.934+0000 7fd98fc9b640 1 -- start start 2024-09-15T21:47:00.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.935+0000 7fd98fc9b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd988152060 0x7fd98806c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:00.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.935+0000 7fd98fc9b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd9880c9290 0x7fd9880c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:00.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.935+0000 7fd98fc9b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9880c50b0 con 0x7fd988158900 2024-09-15T21:47:00.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.935+0000 7fd98fc9b640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fd98806bff0 con 0x7fd9880c9290 2024-09-15T21:47:00.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.935+0000 7fd98fc9b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd988071800 con 0x7fd988152060 2024-09-15T21:47:00.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.935+0000 7fd98e238640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd988158900 0x7fd98815b470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:40730/0 (socket says 172.21.15.100:40730) 2024-09-15T21:47:00.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.935+0000 7fd98e238640 1 -- 172.21.15.100:0/1624307467 learned_addr learned my addr 172.21.15.100:0/1624307467 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:00.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98ca35640 1 -- 172.21.15.100:0/1624307467 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 885199675 0 0) 0x7fd9880c50b0 con 0x7fd988158900 2024-09-15T21:47:00.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98da37640 1 --2- 172.21.15.100:0/1624307467 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd988152060 0x7fd98806c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:00.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98d236640 1 --2- 172.21.15.100:0/1624307467 >> v2:172.21.15.122:3300/0 conn(0x7fd9880c9290 0x7fd9880c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:00.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98ca35640 1 -- 172.21.15.100:0/1624307467 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd970005730 con 0x7fd988158900 2024-09-15T21:47:00.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98ca35640 1 -- 172.21.15.100:0/1624307467 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1335323607 0 0) 0x7fd970005730 con 0x7fd988158900 2024-09-15T21:47:00.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98ca35640 1 -- 172.21.15.100:0/1624307467 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd988152060 msgr2=0x7fd98806c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:00.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98ca35640 1 --2- 172.21.15.100:0/1624307467 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd988152060 0x7fd98806c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:00.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98ca35640 1 -- 172.21.15.100:0/1624307467 >> v2:172.21.15.122:3300/0 conn(0x7fd9880c9290 msgr2=0x7fd9880c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:00.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98ca35640 1 --2- 172.21.15.100:0/1624307467 >> v2:172.21.15.122:3300/0 conn(0x7fd9880c9290 0x7fd9880c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:00.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98ca35640 1 -- 172.21.15.100:0/1624307467 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd988071ae0 con 0x7fd988158900 2024-09-15T21:47:00.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98ca35640 1 -- 172.21.15.100:0/1624307467 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7fd980001670 con 0x7fd988158900 2024-09-15T21:47:00.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98ca35640 1 -- 172.21.15.100:0/1624307467 >> v1:172.21.15.100:6789/0 conn(0x7fd988158900 legacy=0x7fd98815b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:00.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98ca35640 1 --2- 172.21.15.100:0/1624307467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd970005bc0 0x7fd970005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:00.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98ca35640 1 --2- 172.21.15.100:0/1624307467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd970006a10 0x7fd970006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:00.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98ca35640 1 --2- 172.21.15.100:0/1624307467 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd988152060 0x7fd9700073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:00.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98ca35640 1 -- 172.21.15.100:0/1624307467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd98806bff0 con 0x7fd970005bc0 2024-09-15T21:47:00.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98ca35640 1 -- 172.21.15.100:0/1624307467 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd988071800 con 0x7fd970006a10 2024-09-15T21:47:00.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98ca35640 1 -- 172.21.15.100:0/1624307467 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd97000dbd0 con 0x7fd988152060 2024-09-15T21:47:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98da37640 1 --2- 172.21.15.100:0/1624307467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd970005bc0 0x7fd970005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98d236640 1 --2- 172.21.15.100:0/1624307467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd970006a10 0x7fd970006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98e238640 1 --2- 172.21.15.100:0/1624307467 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd988152060 0x7fd9700073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98da37640 1 -- 172.21.15.100:0/1624307467 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd988152060 msgr2=0x7fd9700073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98da37640 1 --2- 172.21.15.100:0/1624307467 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd988152060 0x7fd9700073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98da37640 1 -- 172.21.15.100:0/1624307467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd970006a10 msgr2=0x7fd970006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98da37640 1 --2- 172.21.15.100:0/1624307467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd970006a10 0x7fd970006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.936+0000 7fd98da37640 1 -- 172.21.15.100:0/1624307467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd970005730 con 0x7fd970005bc0 2024-09-15T21:47:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.937+0000 7fd98da37640 1 --2- 172.21.15.100:0/1624307467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd970005bc0 0x7fd970005fe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd978009fc0 tx=0x7fd970007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dca0dd1217aa3ae server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.937+0000 7fd98ca35640 1 -- 172.21.15.100:0/1624307467 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978009c10 con 0x7fd970005bc0 2024-09-15T21:47:00.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.937+0000 7fd98ca35640 1 -- 172.21.15.100:0/1624307467 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd978036070 con 0x7fd970005bc0 2024-09-15T21:47:00.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.937+0000 7fd98ca35640 1 -- 172.21.15.100:0/1624307467 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978031880 con 0x7fd970005bc0 2024-09-15T21:47:00.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.938+0000 7fd98fc9b640 1 -- 172.21.15.100:0/1624307467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd970005bc0 msgr2=0x7fd970005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:00.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.938+0000 7fd98fc9b640 1 --2- 172.21.15.100:0/1624307467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd970005bc0 0x7fd970005fe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd978009fc0 tx=0x7fd970007a40 comp rx=0 tx=0).stop 2024-09-15T21:47:00.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.938+0000 7fd98e238640 1 -- 172.21.15.100:0/1624307467 reap_dead start 2024-09-15T21:47:00.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.938+0000 7fd98fc9b640 1 -- 172.21.15.100:0/1624307467 shutdown_connections 2024-09-15T21:47:00.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.938+0000 7fd98fc9b640 1 -- 172.21.15.100:0/1624307467 >> 172.21.15.100:0/1624307467 conn(0x7fd9880ccbd0 msgr2=0x7fd9880ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:00.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.938+0000 7fd98fc9b640 1 -- 172.21.15.100:0/1624307467 shutdown_connections 2024-09-15T21:47:00.944 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.938+0000 7fd98fc9b640 1 -- 172.21.15.100:0/1624307467 wait complete. 2024-09-15T21:47:00.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.939+0000 7fd98fc9b640 1 Processor -- start 2024-09-15T21:47:00.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.939+0000 7fd98fc9b640 1 -- start start 2024-09-15T21:47:00.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98fc9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd970005bc0 0x7fd9880cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:00.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98fc9b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd9881e0060 0x7fd9881ed530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:00.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98fc9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9881eda70 0x7fd98814ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:00.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98fc9b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd988072b00 con 0x7fd970005bc0 2024-09-15T21:47:00.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98fc9b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd988071dc0 con 0x7fd9881eda70 2024-09-15T21:47:00.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98fc9b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd988072c80 con 0x7fd9881e0060 2024-09-15T21:47:00.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98da37640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd970005bc0 0x7fd9880cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:00.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98da37640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd970005bc0 0x7fd9880cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48860/0 (socket says 172.21.15.100:48860) 2024-09-15T21:47:00.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98da37640 1 -- 172.21.15.100:0/1362953668 learned_addr learned my addr 172.21.15.100:0/1362953668 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:00.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98d236640 1 --2- 172.21.15.100:0/1362953668 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd9881e0060 0x7fd9881ed530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:00.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98e238640 1 --2- 172.21.15.100:0/1362953668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9881eda70 0x7fd98814ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:00.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98da37640 1 -- 172.21.15.100:0/1362953668 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd9881e0060 msgr2=0x7fd9881ed530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:00.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98da37640 1 --2- 172.21.15.100:0/1362953668 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd9881e0060 0x7fd9881ed530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98da37640 1 -- 172.21.15.100:0/1362953668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9881eda70 msgr2=0x7fd98814ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98da37640 1 --2- 172.21.15.100:0/1362953668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9881eda70 0x7fd98814ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98da37640 1 -- 172.21.15.100:0/1362953668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd988150b20 con 0x7fd970005bc0 2024-09-15T21:47:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98d236640 1 --2- 172.21.15.100:0/1362953668 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd9881e0060 0x7fd9881ed530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:47:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98e238640 1 --2- 172.21.15.100:0/1362953668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9881eda70 0x7fd98814ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:47:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98da37640 1 --2- 172.21.15.100:0/1362953668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd970005bc0 0x7fd9880cbec0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd97800e200 tx=0x7fd978009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd9867fc640 1 -- 172.21.15.100:0/1362953668 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978009c10 con 0x7fd970005bc0 2024-09-15T21:47:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd9867fc640 1 -- 172.21.15.100:0/1362953668 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd978003990 con 0x7fd970005bc0 2024-09-15T21:47:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd9867fc640 1 -- 172.21.15.100:0/1362953668 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978003d60 con 0x7fd970005bc0 2024-09-15T21:47:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98fc9b640 1 -- 172.21.15.100:0/1362953668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9881ecd90 con 0x7fd970005bc0 2024-09-15T21:47:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.940+0000 7fd98fc9b640 1 -- 172.21.15.100:0/1362953668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9881f0db0 con 0x7fd970005bc0 2024-09-15T21:47:00.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.942+0000 7fd98fc9b640 1 -- 172.21.15.100:0/1362953668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9880726d0 con 0x7fd970005bc0 2024-09-15T21:47:00.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.942+0000 7fd9867fc640 1 -- 172.21.15.100:0/1362953668 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd978005d10 con 0x7fd970005bc0 2024-09-15T21:47:00.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.942+0000 7fd9867fc640 1 --2- 172.21.15.100:0/1362953668 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd97406c2e0 0x7fd97406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:00.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.942+0000 7fd9867fc640 1 -- 172.21.15.100:0/1362953668 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd9780ac730 con 0x7fd970005bc0 2024-09-15T21:47:00.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.942+0000 7fd98d236640 1 --2- 172.21.15.100:0/1362953668 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd97406c2e0 0x7fd97406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:00.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.943+0000 7fd98d236640 1 --2- 172.21.15.100:0/1362953668 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd97406c2e0 0x7fd97406e7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd988157e70 tx=0x7fd97c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:00.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:00.947+0000 7fd9867fc640 1 -- 172.21.15.100:0/1362953668 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd9881ecd90 con 0x7fd970005bc0 2024-09-15T21:47:01.114 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.112+0000 7fd98fc9b640 1 -- 172.21.15.100:0/1362953668 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mgr_debug_aggressive_pg_num_changes}] v 0) -- 0x7fd988071440 con 0x7fd970005bc0 2024-09-15T21:47:01.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.116+0000 7fd9867fc640 1 -- 172.21.15.100:0/1362953668 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr_debug_aggressive_pg_num_changes}]=0 v10)=0 v10) ==== 136+0+0 (secure 0 0 0) 0x7fd97807af30 con 0x7fd970005bc0 2024-09-15T21:47:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.118+0000 7fd98fc9b640 1 -- 172.21.15.100:0/1362953668 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd97406c2e0 msgr2=0x7fd97406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.118+0000 7fd98fc9b640 1 --2- 172.21.15.100:0/1362953668 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd97406c2e0 0x7fd97406e7a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd988157e70 tx=0x7fd97c008040 comp rx=0 tx=0).stop 2024-09-15T21:47:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.118+0000 7fd98fc9b640 1 -- 172.21.15.100:0/1362953668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd970005bc0 msgr2=0x7fd9880cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.118+0000 7fd98fc9b640 1 --2- 172.21.15.100:0/1362953668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd970005bc0 0x7fd9880cbec0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd97800e200 tx=0x7fd978009750 comp rx=0 tx=0).stop 2024-09-15T21:47:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.118+0000 7fd98fc9b640 1 -- 172.21.15.100:0/1362953668 shutdown_connections 2024-09-15T21:47:01.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.118+0000 7fd98fc9b640 1 --2- 172.21.15.100:0/1362953668 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd97406c2e0 0x7fd97406e7a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.118+0000 7fd98fc9b640 1 --2- 172.21.15.100:0/1362953668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd9881eda70 0x7fd98814ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.118+0000 7fd98fc9b640 1 --2- 172.21.15.100:0/1362953668 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd9881e0060 0x7fd9881ed530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.118+0000 7fd98fc9b640 1 --2- 172.21.15.100:0/1362953668 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd970005bc0 0x7fd9880cbec0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.118+0000 7fd98fc9b640 1 -- 172.21.15.100:0/1362953668 >> 172.21.15.100:0/1362953668 conn(0x7fd9880ccbd0 msgr2=0x7fd9880c7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:01.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.118+0000 7fd98fc9b640 1 -- 172.21.15.100:0/1362953668 shutdown_connections 2024-09-15T21:47:01.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.118+0000 7fd98fc9b640 1 -- 172.21.15.100:0/1362953668 wait complete. 2024-09-15T21:47:01.130 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-15T21:47:01.131 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:47:01.131 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:47:01.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.227+0000 7f215ed98640 1 Processor -- start 2024-09-15T21:47:01.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.227+0000 7f215ed98640 1 -- start start 2024-09-15T21:47:01.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.228+0000 7f215ed98640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f21580c9a50 0x7f215815c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:01.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.228+0000 7f215ed98640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f2158153490 0x7f215815cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:01.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.228+0000 7f215ed98640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21580c8510 con 0x7f2158073b80 2024-09-15T21:47:01.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.228+0000 7f215ed98640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f21580c82c0 con 0x7f21580c9a50 2024-09-15T21:47:01.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.228+0000 7f215ed98640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f215806c890 con 0x7f2158153490 2024-09-15T21:47:01.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.228+0000 7f2157fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f21580c9a50 0x7f215815c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:01.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.228+0000 7f215cb34640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f2158073b80 0x7f2158152d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:40742/0 (socket says 172.21.15.100:40742) 2024-09-15T21:47:01.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.228+0000 7f215cb34640 1 -- 172.21.15.100:0/1621998926 learned_addr learned my addr 172.21.15.100:0/1621998926 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:01.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.228+0000 7f2157fff640 1 -- 172.21.15.100:0/1621998926 >> v2:172.21.15.122:3300/0 conn(0x7f2158153490 msgr2=0x7f215815cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:01.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.228+0000 7f2157fff640 1 --2- 172.21.15.100:0/1621998926 >> v2:172.21.15.122:3300/0 conn(0x7f2158153490 0x7f215815cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.228+0000 7f2157fff640 1 -- 172.21.15.100:0/1621998926 >> v1:172.21.15.100:6789/0 conn(0x7f2158073b80 legacy=0x7f2158152d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:01.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.228+0000 7f2157fff640 1 -- 172.21.15.100:0/1621998926 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2158159cb0 con 0x7f21580c9a50 2024-09-15T21:47:01.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.228+0000 7f21577fe640 1 -- 172.21.15.100:0/1621998926 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 212895478 0 0) 0x7f21580c8510 con 0x7f2158073b80 2024-09-15T21:47:01.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.229+0000 7f2157fff640 1 --2- 172.21.15.100:0/1621998926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21580c9a50 0x7f215815c570 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f215806ed70 tx=0x7f214c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff24ed43a62eff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:01.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.229+0000 7f21577fe640 1 -- 172.21.15.100:0/1621998926 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c037070 con 0x7f21580c9a50 2024-09-15T21:47:01.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.229+0000 7f21577fe640 1 -- 172.21.15.100:0/1621998926 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f214c032440 con 0x7f21580c9a50 2024-09-15T21:47:01.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.229+0000 7f21577fe640 1 -- 172.21.15.100:0/1621998926 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c032810 con 0x7f21580c9a50 2024-09-15T21:47:01.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.230+0000 7f215ed98640 1 -- 172.21.15.100:0/1621998926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21580c9a50 msgr2=0x7f215815c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:01.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.230+0000 7f215ed98640 1 --2- 172.21.15.100:0/1621998926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21580c9a50 0x7f215815c570 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f215806ed70 tx=0x7f214c02a230 comp rx=0 tx=0).stop 2024-09-15T21:47:01.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.230+0000 7f215ed98640 1 -- 172.21.15.100:0/1621998926 shutdown_connections 2024-09-15T21:47:01.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.230+0000 7f215ed98640 1 --2- 172.21.15.100:0/1621998926 >> v2:172.21.15.122:3300/0 conn(0x7f2158153490 0x7f215815cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.230+0000 7f215ed98640 1 --2- 172.21.15.100:0/1621998926 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21580c9a50 0x7f215815c570 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.230+0000 7f215ed98640 1 -- 172.21.15.100:0/1621998926 >> 172.21.15.100:0/1621998926 conn(0x7f215806d890 msgr2=0x7f215806bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:01.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.230+0000 7f215ed98640 1 -- 172.21.15.100:0/1621998926 shutdown_connections 2024-09-15T21:47:01.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.230+0000 7f215ed98640 1 -- 172.21.15.100:0/1621998926 wait complete. 2024-09-15T21:47:01.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.231+0000 7f215ed98640 1 Processor -- start 2024-09-15T21:47:01.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.231+0000 7f215ed98640 1 -- start start 2024-09-15T21:47:01.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.232+0000 7f215ed98640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2158073b80 0x7f21580cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:01.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.232+0000 7f215ed98640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21580c9a50 0x7f215814eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:01.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.232+0000 7f215ed98640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2158153490 0x7f21581541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:01.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.232+0000 7f215ed98640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f21580c78c0 con 0x7f2158073b80 2024-09-15T21:47:01.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.232+0000 7f215ed98640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2158159980 con 0x7f2158153490 2024-09-15T21:47:01.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.232+0000 7f215ed98640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f21580c7a40 con 0x7f21580c9a50 2024-09-15T21:47:01.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.232+0000 7f215cb34640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2158073b80 0x7f21580cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:01.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.232+0000 7f215cb34640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2158073b80 0x7f21580cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48872/0 (socket says 172.21.15.100:48872) 2024-09-15T21:47:01.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.232+0000 7f215cb34640 1 -- 172.21.15.100:0/3894272392 learned_addr learned my addr 172.21.15.100:0/3894272392 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:01.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.232+0000 7f215d335640 1 --2- 172.21.15.100:0/3894272392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2158153490 0x7f21581541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:01.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.232+0000 7f2157fff640 1 --2- 172.21.15.100:0/3894272392 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21580c9a50 0x7f215814eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:01.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.232+0000 7f215cb34640 1 -- 172.21.15.100:0/3894272392 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21580c9a50 msgr2=0x7f215814eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:01.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.232+0000 7f215cb34640 1 --2- 172.21.15.100:0/3894272392 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21580c9a50 0x7f215814eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.237 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.232+0000 7f215cb34640 1 -- 172.21.15.100:0/3894272392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2158153490 msgr2=0x7f21581541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:01.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.232+0000 7f215cb34640 1 --2- 172.21.15.100:0/3894272392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2158153490 0x7f21581541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.232+0000 7f215cb34640 1 -- 172.21.15.100:0/3894272392 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21581ef920 con 0x7f2158073b80 2024-09-15T21:47:01.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.232+0000 7f2157fff640 1 --2- 172.21.15.100:0/3894272392 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21580c9a50 0x7f215814eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:47:01.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.232+0000 7f215cb34640 1 --2- 172.21.15.100:0/3894272392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2158073b80 0x7f21580cd780 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f214800c4d0 tx=0x7f214800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:01.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.233+0000 7f21557fa640 1 -- 172.21.15.100:0/3894272392 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148018070 con 0x7f2158073b80 2024-09-15T21:47:01.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.233+0000 7f21557fa640 1 -- 172.21.15.100:0/3894272392 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2148007500 con 0x7f2158073b80 2024-09-15T21:47:01.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.233+0000 7f215ed98640 1 -- 172.21.15.100:0/3894272392 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21580caea0 con 0x7f2158073b80 2024-09-15T21:47:01.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.233+0000 7f21557fa640 1 -- 172.21.15.100:0/3894272392 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2148004a20 con 0x7f2158073b80 2024-09-15T21:47:01.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.233+0000 7f215ed98640 1 -- 172.21.15.100:0/3894272392 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21581539f0 con 0x7f2158073b80 2024-09-15T21:47:01.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.234+0000 7f215ed98640 1 -- 172.21.15.100:0/3894272392 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2158073f60 con 0x7f2158073b80 2024-09-15T21:47:01.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.234+0000 7f21557fa640 1 -- 172.21.15.100:0/3894272392 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f214801b070 con 0x7f2158073b80 2024-09-15T21:47:01.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.235+0000 7f21557fa640 1 --2- 172.21.15.100:0/3894272392 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f214406c2b0 0x7f214406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:01.238 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.235+0000 7f21557fa640 1 -- 172.21.15.100:0/3894272392 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f214808a7b0 con 0x7f2158073b80 2024-09-15T21:47:01.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.235+0000 7f2157fff640 1 --2- 172.21.15.100:0/3894272392 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f214406c2b0 0x7f214406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:01.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.235+0000 7f2157fff640 1 --2- 172.21.15.100:0/3894272392 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f214406c2b0 0x7f214406e770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f214c034000 tx=0x7f214c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:01.241 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.239+0000 7f21557fa640 1 -- 172.21.15.100:0/3894272392 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2148058f20 con 0x7f2158073b80 2024-09-15T21:47:01.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.402+0000 7f215ed98640 1 -- 172.21.15.100:0/3894272392 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2158156d50 con 0x7f214406c2b0 2024-09-15T21:47:01.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.408+0000 7f21557fa640 1 -- 172.21.15.100:0/3894272392 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251952 (secure 0 0 0) 0x7f2158156d50 con 0x7f214406c2b0 2024-09-15T21:47:01.413 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:47:01.414 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-15T21:47:01.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.411+0000 7f215ed98640 1 -- 172.21.15.100:0/3894272392 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f214406c2b0 msgr2=0x7f214406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:01.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.411+0000 7f215ed98640 1 --2- 172.21.15.100:0/3894272392 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f214406c2b0 0x7f214406e770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f214c034000 tx=0x7f214c02e000 comp rx=0 tx=0).stop 2024-09-15T21:47:01.415 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.411+0000 7f215ed98640 1 -- 172.21.15.100:0/3894272392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2158073b80 msgr2=0x7f21580cd780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:01.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.411+0000 7f215ed98640 1 --2- 172.21.15.100:0/3894272392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2158073b80 0x7f21580cd780 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f214800c4d0 tx=0x7f214800c990 comp rx=0 tx=0).stop 2024-09-15T21:47:01.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.412+0000 7f215ed98640 1 -- 172.21.15.100:0/3894272392 shutdown_connections 2024-09-15T21:47:01.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.412+0000 7f215ed98640 1 --2- 172.21.15.100:0/3894272392 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f214406c2b0 0x7f214406e770 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.412+0000 7f215ed98640 1 --2- 172.21.15.100:0/3894272392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2158153490 0x7f21581541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.412+0000 7f215ed98640 1 --2- 172.21.15.100:0/3894272392 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21580c9a50 0x7f215814eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.412+0000 7f215ed98640 1 --2- 172.21.15.100:0/3894272392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2158073b80 0x7f21580cd780 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.412+0000 7f215ed98640 1 -- 172.21.15.100:0/3894272392 >> 172.21.15.100:0/3894272392 conn(0x7f215806d890 msgr2=0x7f21580c59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:01.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.412+0000 7f215ed98640 1 -- 172.21.15.100:0/3894272392 shutdown_connections 2024-09-15T21:47:01.416 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.412+0000 7f215ed98640 1 -- 172.21.15.100:0/3894272392 wait complete. 2024-09-15T21:47:01.424 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-09-15T21:46:59.812084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598402,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":364,"num_write_kb":2478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223496,"kb_used_data":3144,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751224,"statfs":{"total":773094113280,"available":772865253376,"internally_reserved":0,"allocated":3219456,"data_stored":1812468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2812,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003542"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835668+0000","last_change":"2024-09-15T21:46:23.428662+0000","last_active":"2024-09-15T21:46:24.835668+0000","last_peered":"2024-09-15T21:46:24.835668+0000","last_clean":"2024-09-15T21:46:24.835668+0000","last_became_active":"2024-09-15T21:46:23.427735+0000","last_became_peered":"2024-09-15T21:46:23.427735+0000","last_unstale":"2024-09-15T21:46:24.835668+0000","last_undegraded":"2024-09-15T21:46:24.835668+0000","last_fullsized":"2024-09-15T21:46:24.835668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:29.702757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808513+0000","last_change":"2024-09-15T21:46:23.418716+0000","last_active":"2024-09-15T21:46:24.808513+0000","last_peered":"2024-09-15T21:46:24.808513+0000","last_clean":"2024-09-15T21:46:24.808513+0000","last_became_active":"2024-09-15T21:46:23.418471+0000","last_became_peered":"2024-09-15T21:46:23.418471+0000","last_unstale":"2024-09-15T21:46:24.808513+0000","last_undegraded":"2024-09-15T21:46:24.808513+0000","last_fullsized":"2024-09-15T21:46:24.808513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:49.599801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727016+0000","last_change":"2024-09-15T21:46:23.426404+0000","last_active":"2024-09-15T21:46:24.727016+0000","last_peered":"2024-09-15T21:46:24.727016+0000","last_clean":"2024-09-15T21:46:24.727016+0000","last_became_active":"2024-09-15T21:46:23.426146+0000","last_became_peered":"2024-09-15T21:46:23.426146+0000","last_unstale":"2024-09-15T21:46:24.727016+0000","last_undegraded":"2024-09-15T21:46:24.727016+0000","last_fullsized":"2024-09-15T21:46:24.727016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:23.626109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414109+0000","last_change":"2024-09-15T21:46:23.432515+0000","last_active":"2024-09-15T21:46:24.414109+0000","last_peered":"2024-09-15T21:46:24.414109+0000","last_clean":"2024-09-15T21:46:24.414109+0000","last_became_active":"2024-09-15T21:46:23.432283+0000","last_became_peered":"2024-09-15T21:46:23.432283+0000","last_unstale":"2024-09-15T21:46:24.414109+0000","last_undegraded":"2024-09-15T21:46:24.414109+0000","last_fullsized":"2024-09-15T21:46:24.414109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:53.288080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835464+0000","last_change":"2024-09-15T21:46:23.427120+0000","last_active":"2024-09-15T21:46:24.835464+0000","last_peered":"2024-09-15T21:46:24.835464+0000","last_clean":"2024-09-15T21:46:24.835464+0000","last_became_active":"2024-09-15T21:46:23.426706+0000","last_became_peered":"2024-09-15T21:46:23.426706+0000","last_unstale":"2024-09-15T21:46:24.835464+0000","last_undegraded":"2024-09-15T21:46:24.835464+0000","last_fullsized":"2024-09-15T21:46:24.835464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:29.513446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414665+0000","last_change":"2024-09-15T21:46:23.434690+0000","last_active":"2024-09-15T21:46:24.414665+0000","last_peered":"2024-09-15T21:46:24.414665+0000","last_clean":"2024-09-15T21:46:24.414665+0000","last_became_active":"2024-09-15T21:46:23.434146+0000","last_became_peered":"2024-09-15T21:46:23.434146+0000","last_unstale":"2024-09-15T21:46:24.414665+0000","last_undegraded":"2024-09-15T21:46:24.414665+0000","last_fullsized":"2024-09-15T21:46:24.414665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:26:59.905178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835273+0000","last_change":"2024-09-15T21:46:23.429541+0000","last_active":"2024-09-15T21:46:24.835273+0000","last_peered":"2024-09-15T21:46:24.835273+0000","last_clean":"2024-09-15T21:46:24.835273+0000","last_became_active":"2024-09-15T21:46:23.429263+0000","last_became_peered":"2024-09-15T21:46:23.429263+0000","last_unstale":"2024-09-15T21:46:24.835273+0000","last_undegraded":"2024-09-15T21:46:24.835273+0000","last_fullsized":"2024-09-15T21:46:24.835273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:34:51.530981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834382+0000","last_change":"2024-09-15T21:46:23.437143+0000","last_active":"2024-09-15T21:46:24.834382+0000","last_peered":"2024-09-15T21:46:24.834382+0000","last_clean":"2024-09-15T21:46:24.834382+0000","last_became_active":"2024-09-15T21:46:23.436864+0000","last_became_peered":"2024-09-15T21:46:23.436864+0000","last_unstale":"2024-09-15T21:46:24.834382+0000","last_undegraded":"2024-09-15T21:46:24.834382+0000","last_fullsized":"2024-09-15T21:46:24.834382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:45.870403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835817+0000","last_change":"2024-09-15T21:46:23.438642+0000","last_active":"2024-09-15T21:46:24.835817+0000","last_peered":"2024-09-15T21:46:24.835817+0000","last_clean":"2024-09-15T21:46:24.835817+0000","last_became_active":"2024-09-15T21:46:23.437521+0000","last_became_peered":"2024-09-15T21:46:23.437521+0000","last_unstale":"2024-09-15T21:46:24.835817+0000","last_undegraded":"2024-09-15T21:46:24.835817+0000","last_fullsized":"2024-09-15T21:46:24.835817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:04.648853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835520+0000","last_change":"2024-09-15T21:46:23.436106+0000","last_active":"2024-09-15T21:46:24.835520+0000","last_peered":"2024-09-15T21:46:24.835520+0000","last_clean":"2024-09-15T21:46:24.835520+0000","last_became_active":"2024-09-15T21:46:23.435821+0000","last_became_peered":"2024-09-15T21:46:23.435821+0000","last_unstale":"2024-09-15T21:46:24.835520+0000","last_undegraded":"2024-09-15T21:46:24.835520+0000","last_fullsized":"2024-09-15T21:46:24.835520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835070+0000","last_change":"2024-09-15T21:46:23.435442+0000","last_active":"2024-09-15T21:46:24.835070+0000","last_peered":"2024-09-15T21:46:24.835070+0000","last_clean":"2024-09-15T21:46:24.835070+0000","last_became_active":"2024-09-15T21:46:23.435165+0000","last_became_peered":"2024-09-15T21:46:23.435165+0000","last_unstale":"2024-09-15T21:46:24.835070+0000","last_undegraded":"2024-09-15T21:46:24.835070+0000","last_fullsized":"2024-09-15T21:46:24.835070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:02.361682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414099+0000","last_change":"2024-09-15T21:46:23.432534+0000","last_active":"2024-09-15T21:46:24.414099+0000","last_peered":"2024-09-15T21:46:24.414099+0000","last_clean":"2024-09-15T21:46:24.414099+0000","last_became_active":"2024-09-15T21:46:23.430170+0000","last_became_peered":"2024-09-15T21:46:23.430170+0000","last_unstale":"2024-09-15T21:46:24.414099+0000","last_undegraded":"2024-09-15T21:46:24.414099+0000","last_fullsized":"2024-09-15T21:46:24.414099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:11.463937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834691+0000","last_change":"2024-09-15T21:46:23.427841+0000","last_active":"2024-09-15T21:46:24.834691+0000","last_peered":"2024-09-15T21:46:24.834691+0000","last_clean":"2024-09-15T21:46:24.834691+0000","last_became_active":"2024-09-15T21:46:23.427621+0000","last_became_peered":"2024-09-15T21:46:23.427621+0000","last_unstale":"2024-09-15T21:46:24.834691+0000","last_undegraded":"2024-09-15T21:46:24.834691+0000","last_fullsized":"2024-09-15T21:46:24.834691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:18.331943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.726852+0000","last_change":"2024-09-15T21:46:23.424707+0000","last_active":"2024-09-15T21:46:24.726852+0000","last_peered":"2024-09-15T21:46:24.726852+0000","last_clean":"2024-09-15T21:46:24.726852+0000","last_became_active":"2024-09-15T21:46:23.424447+0000","last_became_peered":"2024-09-15T21:46:23.424447+0000","last_unstale":"2024-09-15T21:46:24.726852+0000","last_undegraded":"2024-09-15T21:46:24.726852+0000","last_fullsized":"2024-09-15T21:46:24.726852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:23.617214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808321+0000","last_change":"2024-09-15T21:46:23.430297+0000","last_active":"2024-09-15T21:46:24.808321+0000","last_peered":"2024-09-15T21:46:24.808321+0000","last_clean":"2024-09-15T21:46:24.808321+0000","last_became_active":"2024-09-15T21:46:23.430070+0000","last_became_peered":"2024-09-15T21:46:23.430070+0000","last_unstale":"2024-09-15T21:46:24.808321+0000","last_undegraded":"2024-09-15T21:46:24.808321+0000","last_fullsized":"2024-09-15T21:46:24.808321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:03:19.615022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414542+0000","last_change":"2024-09-15T21:46:23.430540+0000","last_active":"2024-09-15T21:46:24.414542+0000","last_peered":"2024-09-15T21:46:24.414542+0000","last_clean":"2024-09-15T21:46:24.414542+0000","last_became_active":"2024-09-15T21:46:23.430256+0000","last_became_peered":"2024-09-15T21:46:23.430256+0000","last_unstale":"2024-09-15T21:46:24.414542+0000","last_undegraded":"2024-09-15T21:46:24.414542+0000","last_fullsized":"2024-09-15T21:46:24.414542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:36.539762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.415216+0000","last_change":"2024-09-15T21:46:23.427802+0000","last_active":"2024-09-15T21:46:24.415216+0000","last_peered":"2024-09-15T21:46:24.415216+0000","last_clean":"2024-09-15T21:46:24.415216+0000","last_became_active":"2024-09-15T21:46:23.427502+0000","last_became_peered":"2024-09-15T21:46:23.427502+0000","last_unstale":"2024-09-15T21:46:24.415216+0000","last_undegraded":"2024-09-15T21:46:24.415216+0000","last_fullsized":"2024-09-15T21:46:24.415216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:31.062624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412600+0000","last_change":"2024-09-15T21:46:23.427570+0000","last_active":"2024-09-15T21:46:24.412600+0000","last_peered":"2024-09-15T21:46:24.412600+0000","last_clean":"2024-09-15T21:46:24.412600+0000","last_became_active":"2024-09-15T21:46:23.427304+0000","last_became_peered":"2024-09-15T21:46:23.427304+0000","last_unstale":"2024-09-15T21:46:24.412600+0000","last_undegraded":"2024-09-15T21:46:24.412600+0000","last_fullsized":"2024-09-15T21:46:24.412600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:37.345106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835899+0000","last_change":"2024-09-15T21:46:23.431434+0000","last_active":"2024-09-15T21:46:24.835899+0000","last_peered":"2024-09-15T21:46:24.835899+0000","last_clean":"2024-09-15T21:46:24.835899+0000","last_became_active":"2024-09-15T21:46:23.430987+0000","last_became_peered":"2024-09-15T21:46:23.430987+0000","last_unstale":"2024-09-15T21:46:24.835899+0000","last_undegraded":"2024-09-15T21:46:24.835899+0000","last_fullsized":"2024-09-15T21:46:24.835899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:48:30.532378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835198+0000","last_change":"2024-09-15T21:46:23.434558+0000","last_active":"2024-09-15T21:46:24.835198+0000","last_peered":"2024-09-15T21:46:24.835198+0000","last_clean":"2024-09-15T21:46:24.835198+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:46:24.835198+0000","last_undegraded":"2024-09-15T21:46:24.835198+0000","last_fullsized":"2024-09-15T21:46:24.835198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:03:39.562977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727140+0000","last_change":"2024-09-15T21:46:23.428323+0000","last_active":"2024-09-15T21:46:24.727140+0000","last_peered":"2024-09-15T21:46:24.727140+0000","last_clean":"2024-09-15T21:46:24.727140+0000","last_became_active":"2024-09-15T21:46:23.428016+0000","last_became_peered":"2024-09-15T21:46:23.428016+0000","last_unstale":"2024-09-15T21:46:24.727140+0000","last_undegraded":"2024-09-15T21:46:24.727140+0000","last_fullsized":"2024-09-15T21:46:24.727140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:28:48.678550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414612+0000","last_change":"2024-09-15T21:46:23.434465+0000","last_active":"2024-09-15T21:46:24.414612+0000","last_peered":"2024-09-15T21:46:24.414612+0000","last_clean":"2024-09-15T21:46:24.414612+0000","last_became_active":"2024-09-15T21:46:23.433849+0000","last_became_peered":"2024-09-15T21:46:23.433849+0000","last_unstale":"2024-09-15T21:46:24.414612+0000","last_undegraded":"2024-09-15T21:46:24.414612+0000","last_fullsized":"2024-09-15T21:46:24.414612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:46.093198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834522+0000","last_change":"2024-09-15T21:46:23.431527+0000","last_active":"2024-09-15T21:46:24.834522+0000","last_peered":"2024-09-15T21:46:24.834522+0000","last_clean":"2024-09-15T21:46:24.834522+0000","last_became_active":"2024-09-15T21:46:23.430719+0000","last_became_peered":"2024-09-15T21:46:23.430719+0000","last_unstale":"2024-09-15T21:46:24.834522+0000","last_undegraded":"2024-09-15T21:46:24.834522+0000","last_fullsized":"2024-09-15T21:46:24.834522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:16.326529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414532+0000","last_change":"2024-09-15T21:46:23.432282+0000","last_active":"2024-09-15T21:46:24.414532+0000","last_peered":"2024-09-15T21:46:24.414532+0000","last_clean":"2024-09-15T21:46:24.414532+0000","last_became_active":"2024-09-15T21:46:23.432035+0000","last_became_peered":"2024-09-15T21:46:23.432035+0000","last_unstale":"2024-09-15T21:46:24.414532+0000","last_undegraded":"2024-09-15T21:46:24.414532+0000","last_fullsized":"2024-09-15T21:46:24.414532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:22.163940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834776+0000","last_change":"2024-09-15T21:46:23.432646+0000","last_active":"2024-09-15T21:46:24.834776+0000","last_peered":"2024-09-15T21:46:24.834776+0000","last_clean":"2024-09-15T21:46:24.834776+0000","last_became_active":"2024-09-15T21:46:23.432317+0000","last_became_peered":"2024-09-15T21:46:23.432317+0000","last_unstale":"2024-09-15T21:46:24.834776+0000","last_undegraded":"2024-09-15T21:46:24.834776+0000","last_fullsized":"2024-09-15T21:46:24.834776+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:49:07.706450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835541+0000","last_change":"2024-09-15T21:46:23.428183+0000","last_active":"2024-09-15T21:46:24.835541+0000","last_peered":"2024-09-15T21:46:24.835541+0000","last_clean":"2024-09-15T21:46:24.835541+0000","last_became_active":"2024-09-15T21:46:23.427886+0000","last_became_peered":"2024-09-15T21:46:23.427886+0000","last_unstale":"2024-09-15T21:46:24.835541+0000","last_undegraded":"2024-09-15T21:46:24.835541+0000","last_fullsized":"2024-09-15T21:46:24.835541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:15.584377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808369+0000","last_change":"2024-09-15T21:46:23.421749+0000","last_active":"2024-09-15T21:46:24.808369+0000","last_peered":"2024-09-15T21:46:24.808369+0000","last_clean":"2024-09-15T21:46:24.808369+0000","last_became_active":"2024-09-15T21:46:23.421501+0000","last_became_peered":"2024-09-15T21:46:23.421501+0000","last_unstale":"2024-09-15T21:46:24.808369+0000","last_undegraded":"2024-09-15T21:46:24.808369+0000","last_fullsized":"2024-09-15T21:46:24.808369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:55.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834843+0000","last_change":"2024-09-15T21:46:23.432591+0000","last_active":"2024-09-15T21:46:24.834843+0000","last_peered":"2024-09-15T21:46:24.834843+0000","last_clean":"2024-09-15T21:46:24.834843+0000","last_became_active":"2024-09-15T21:46:23.432326+0000","last_became_peered":"2024-09-15T21:46:23.432326+0000","last_unstale":"2024-09-15T21:46:24.834843+0000","last_undegraded":"2024-09-15T21:46:24.834843+0000","last_fullsized":"2024-09-15T21:46:24.834843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:48.565579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727016+0000","last_change":"2024-09-15T21:46:23.423428+0000","last_active":"2024-09-15T21:46:24.727016+0000","last_peered":"2024-09-15T21:46:24.727016+0000","last_clean":"2024-09-15T21:46:24.727016+0000","last_became_active":"2024-09-15T21:46:23.422734+0000","last_became_peered":"2024-09-15T21:46:23.422734+0000","last_unstale":"2024-09-15T21:46:24.727016+0000","last_undegraded":"2024-09-15T21:46:24.727016+0000","last_fullsized":"2024-09-15T21:46:24.727016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:06.592891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414770+0000","last_change":"2024-09-15T21:46:23.435244+0000","last_active":"2024-09-15T21:46:24.414770+0000","last_peered":"2024-09-15T21:46:24.414770+0000","last_clean":"2024-09-15T21:46:24.414770+0000","last_became_active":"2024-09-15T21:46:23.434932+0000","last_became_peered":"2024-09-15T21:46:23.434932+0000","last_unstale":"2024-09-15T21:46:24.414770+0000","last_undegraded":"2024-09-15T21:46:24.414770+0000","last_fullsized":"2024-09-15T21:46:24.414770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:55.810146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412500+0000","last_change":"2024-09-15T21:46:23.437276+0000","last_active":"2024-09-15T21:46:24.412500+0000","last_peered":"2024-09-15T21:46:24.412500+0000","last_clean":"2024-09-15T21:46:24.412500+0000","last_became_active":"2024-09-15T21:46:23.436962+0000","last_became_peered":"2024-09-15T21:46:23.436962+0000","last_unstale":"2024-09-15T21:46:24.412500+0000","last_undegraded":"2024-09-15T21:46:24.412500+0000","last_fullsized":"2024-09-15T21:46:24.412500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:13.043999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834825+0000","last_change":"2024-09-15T21:46:23.429648+0000","last_active":"2024-09-15T21:46:24.834825+0000","last_peered":"2024-09-15T21:46:24.834825+0000","last_clean":"2024-09-15T21:46:24.834825+0000","last_became_active":"2024-09-15T21:46:23.429376+0000","last_became_peered":"2024-09-15T21:46:23.429376+0000","last_unstale":"2024-09-15T21:46:24.834825+0000","last_undegraded":"2024-09-15T21:46:24.834825+0000","last_fullsized":"2024-09-15T21:46:24.834825+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:31.278283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.413663+0000","last_change":"2024-09-15T21:46:23.429847+0000","last_active":"2024-09-15T21:46:24.413663+0000","last_peered":"2024-09-15T21:46:24.413663+0000","last_clean":"2024-09-15T21:46:24.413663+0000","last_became_active":"2024-09-15T21:46:23.429442+0000","last_became_peered":"2024-09-15T21:46:23.429442+0000","last_unstale":"2024-09-15T21:46:24.413663+0000","last_undegraded":"2024-09-15T21:46:24.413663+0000","last_fullsized":"2024-09-15T21:46:24.413663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:31.232512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:34.464539+0000","last_change":"2024-09-15T21:46:22.429407+0000","last_active":"2024-09-15T21:46:34.464539+0000","last_peered":"2024-09-15T21:46:34.464539+0000","last_clean":"2024-09-15T21:46:34.464539+0000","last_became_active":"2024-09-15T21:46:22.429107+0000","last_became_peered":"2024-09-15T21:46:22.429107+0000","last_unstale":"2024-09-15T21:46:34.464539+0000","last_undegraded":"2024-09-15T21:46:34.464539+0000","last_fullsized":"2024-09-15T21:46:34.464539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:14:54.680349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727169+0000","last_change":"2024-09-15T21:46:23.434198+0000","last_active":"2024-09-15T21:46:24.727169+0000","last_peered":"2024-09-15T21:46:24.727169+0000","last_clean":"2024-09-15T21:46:24.727169+0000","last_became_active":"2024-09-15T21:46:23.433944+0000","last_became_peered":"2024-09-15T21:46:23.433944+0000","last_unstale":"2024-09-15T21:46:24.727169+0000","last_undegraded":"2024-09-15T21:46:24.727169+0000","last_fullsized":"2024-09-15T21:46:24.727169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:25.412167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'7","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.550215+0000","last_change":"2024-09-15T21:46:22.425571+0000","last_active":"2024-09-15T21:46:45.550215+0000","last_peered":"2024-09-15T21:46:45.550215+0000","last_clean":"2024-09-15T21:46:45.550215+0000","last_became_active":"2024-09-15T21:46:22.425315+0000","last_became_peered":"2024-09-15T21:46:22.425315+0000","last_unstale":"2024-09-15T21:46:45.550215+0000","last_undegraded":"2024-09-15T21:46:45.550215+0000","last_fullsized":"2024-09-15T21:46:45.550215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:31.983301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:40.480412+0000","last_change":"2024-09-15T21:46:22.425492+0000","last_active":"2024-09-15T21:46:40.480412+0000","last_peered":"2024-09-15T21:46:40.480412+0000","last_clean":"2024-09-15T21:46:40.480412+0000","last_became_active":"2024-09-15T21:46:22.425226+0000","last_became_peered":"2024-09-15T21:46:22.425226+0000","last_unstale":"2024-09-15T21:46:40.480412+0000","last_undegraded":"2024-09-15T21:46:40.480412+0000","last_fullsized":"2024-09-15T21:46:40.480412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:24.322384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834460+0000","last_change":"2024-09-15T21:46:23.429140+0000","last_active":"2024-09-15T21:46:24.834460+0000","last_peered":"2024-09-15T21:46:24.834460+0000","last_clean":"2024-09-15T21:46:24.834460+0000","last_became_active":"2024-09-15T21:46:23.428902+0000","last_became_peered":"2024-09-15T21:46:23.428902+0000","last_unstale":"2024-09-15T21:46:24.834460+0000","last_undegraded":"2024-09-15T21:46:24.834460+0000","last_fullsized":"2024-09-15T21:46:24.834460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:11.256924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.551162+0000","last_change":"2024-09-15T21:46:22.428319+0000","last_active":"2024-09-15T21:46:45.551162+0000","last_peered":"2024-09-15T21:46:45.551162+0000","last_clean":"2024-09-15T21:46:45.551162+0000","last_became_active":"2024-09-15T21:46:22.428061+0000","last_became_peered":"2024-09-15T21:46:22.428061+0000","last_unstale":"2024-09-15T21:46:45.551162+0000","last_undegraded":"2024-09-15T21:46:45.551162+0000","last_fullsized":"2024-09-15T21:46:45.551162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:14.238988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808315+0000","last_change":"2024-09-15T21:46:23.427442+0000","last_active":"2024-09-15T21:46:24.808315+0000","last_peered":"2024-09-15T21:46:24.808315+0000","last_clean":"2024-09-15T21:46:24.808315+0000","last_became_active":"2024-09-15T21:46:23.427200+0000","last_became_peered":"2024-09-15T21:46:23.427200+0000","last_unstale":"2024-09-15T21:46:24.808315+0000","last_undegraded":"2024-09-15T21:46:24.808315+0000","last_fullsized":"2024-09-15T21:46:24.808315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:27.248545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:34.464224+0000","last_change":"2024-09-15T21:46:22.430599+0000","last_active":"2024-09-15T21:46:34.464224+0000","last_peered":"2024-09-15T21:46:34.464224+0000","last_clean":"2024-09-15T21:46:34.464224+0000","last_became_active":"2024-09-15T21:46:22.430327+0000","last_became_peered":"2024-09-15T21:46:22.430327+0000","last_unstale":"2024-09-15T21:46:34.464224+0000","last_undegraded":"2024-09-15T21:46:34.464224+0000","last_fullsized":"2024-09-15T21:46:34.464224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:51.904693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414860+0000","last_change":"2024-09-15T21:46:23.434006+0000","last_active":"2024-09-15T21:46:24.414860+0000","last_peered":"2024-09-15T21:46:24.414860+0000","last_clean":"2024-09-15T21:46:24.414860+0000","last_became_active":"2024-09-15T21:46:23.433634+0000","last_became_peered":"2024-09-15T21:46:23.433634+0000","last_unstale":"2024-09-15T21:46:24.414860+0000","last_undegraded":"2024-09-15T21:46:24.414860+0000","last_fullsized":"2024-09-15T21:46:24.414860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:22.288806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.571048+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:51.571048+0000","last_peered":"2024-09-15T21:46:51.571048+0000","last_clean":"2024-09-15T21:46:51.571048+0000","last_became_active":"2024-09-15T21:46:22.425236+0000","last_became_peered":"2024-09-15T21:46:22.425236+0000","last_unstale":"2024-09-15T21:46:51.571048+0000","last_undegraded":"2024-09-15T21:46:51.571048+0000","last_fullsized":"2024-09-15T21:46:51.571048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:45:21.332492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727237+0000","last_change":"2024-09-15T21:46:23.429587+0000","last_active":"2024-09-15T21:46:24.727237+0000","last_peered":"2024-09-15T21:46:24.727237+0000","last_clean":"2024-09-15T21:46:24.727237+0000","last_became_active":"2024-09-15T21:46:23.429334+0000","last_became_peered":"2024-09-15T21:46:23.429334+0000","last_unstale":"2024-09-15T21:46:24.727237+0000","last_undegraded":"2024-09-15T21:46:24.727237+0000","last_fullsized":"2024-09-15T21:46:24.727237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:48:56.361550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:34.466053+0000","last_change":"2024-09-15T21:46:22.421964+0000","last_active":"2024-09-15T21:46:34.466053+0000","last_peered":"2024-09-15T21:46:34.466053+0000","last_clean":"2024-09-15T21:46:34.466053+0000","last_became_active":"2024-09-15T21:46:22.421661+0000","last_became_peered":"2024-09-15T21:46:22.421661+0000","last_unstale":"2024-09-15T21:46:34.466053+0000","last_undegraded":"2024-09-15T21:46:34.466053+0000","last_fullsized":"2024-09-15T21:46:34.466053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:07.395070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414253+0000","last_change":"2024-09-15T21:46:23.427416+0000","last_active":"2024-09-15T21:46:24.414253+0000","last_peered":"2024-09-15T21:46:24.414253+0000","last_clean":"2024-09-15T21:46:24.414253+0000","last_became_active":"2024-09-15T21:46:23.427057+0000","last_became_peered":"2024-09-15T21:46:23.427057+0000","last_unstale":"2024-09-15T21:46:24.414253+0000","last_undegraded":"2024-09-15T21:46:24.414253+0000","last_fullsized":"2024-09-15T21:46:24.414253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:42.671501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'10","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.572146+0000","last_change":"2024-09-15T21:46:22.426151+0000","last_active":"2024-09-15T21:46:51.572146+0000","last_peered":"2024-09-15T21:46:51.572146+0000","last_clean":"2024-09-15T21:46:51.572146+0000","last_became_active":"2024-09-15T21:46:22.425866+0000","last_became_peered":"2024-09-15T21:46:22.425866+0000","last_unstale":"2024-09-15T21:46:51.572146+0000","last_undegraded":"2024-09-15T21:46:51.572146+0000","last_fullsized":"2024-09-15T21:46:51.572146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:10.684265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835569+0000","last_change":"2024-09-15T21:46:23.432292+0000","last_active":"2024-09-15T21:46:24.835569+0000","last_peered":"2024-09-15T21:46:24.835569+0000","last_clean":"2024-09-15T21:46:24.835569+0000","last_became_active":"2024-09-15T21:46:23.431877+0000","last_became_peered":"2024-09-15T21:46:23.431877+0000","last_unstale":"2024-09-15T21:46:24.835569+0000","last_undegraded":"2024-09-15T21:46:24.835569+0000","last_fullsized":"2024-09-15T21:46:24.835569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:16.857279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'9","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.567901+0000","last_change":"2024-09-15T21:46:22.427415+0000","last_active":"2024-09-15T21:46:51.567901+0000","last_peered":"2024-09-15T21:46:51.567901+0000","last_clean":"2024-09-15T21:46:51.567901+0000","last_became_active":"2024-09-15T21:46:22.427087+0000","last_became_peered":"2024-09-15T21:46:22.427087+0000","last_unstale":"2024-09-15T21:46:51.567901+0000","last_undegraded":"2024-09-15T21:46:51.567901+0000","last_fullsized":"2024-09-15T21:46:51.567901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:09.365202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.726943+0000","last_change":"2024-09-15T21:46:23.427238+0000","last_active":"2024-09-15T21:46:24.726943+0000","last_peered":"2024-09-15T21:46:24.726943+0000","last_clean":"2024-09-15T21:46:24.726943+0000","last_became_active":"2024-09-15T21:46:23.426993+0000","last_became_peered":"2024-09-15T21:46:23.426993+0000","last_unstale":"2024-09-15T21:46:24.726943+0000","last_undegraded":"2024-09-15T21:46:24.726943+0000","last_fullsized":"2024-09-15T21:46:24.726943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:52:24.175554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.550646+0000","last_change":"2024-09-15T21:46:22.427164+0000","last_active":"2024-09-15T21:46:45.550646+0000","last_peered":"2024-09-15T21:46:45.550646+0000","last_clean":"2024-09-15T21:46:45.550646+0000","last_became_active":"2024-09-15T21:46:22.426917+0000","last_became_peered":"2024-09-15T21:46:22.426917+0000","last_unstale":"2024-09-15T21:46:45.550646+0000","last_undegraded":"2024-09-15T21:46:45.550646+0000","last_fullsized":"2024-09-15T21:46:45.550646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:46.471935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835504+0000","last_change":"2024-09-15T21:46:23.433924+0000","last_active":"2024-09-15T21:46:24.835504+0000","last_peered":"2024-09-15T21:46:24.835504+0000","last_clean":"2024-09-15T21:46:24.835504+0000","last_became_active":"2024-09-15T21:46:23.433606+0000","last_became_peered":"2024-09-15T21:46:23.433606+0000","last_unstale":"2024-09-15T21:46:24.835504+0000","last_undegraded":"2024-09-15T21:46:24.835504+0000","last_fullsized":"2024-09-15T21:46:24.835504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:09:35.744149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.549348+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:45.549348+0000","last_peered":"2024-09-15T21:46:45.549348+0000","last_clean":"2024-09-15T21:46:45.549348+0000","last_became_active":"2024-09-15T21:46:22.425237+0000","last_became_peered":"2024-09-15T21:46:22.425237+0000","last_unstale":"2024-09-15T21:46:45.549348+0000","last_undegraded":"2024-09-15T21:46:45.549348+0000","last_fullsized":"2024-09-15T21:46:45.549348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:59.460420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834853+0000","last_change":"2024-09-15T21:46:23.427515+0000","last_active":"2024-09-15T21:46:24.834853+0000","last_peered":"2024-09-15T21:46:24.834853+0000","last_clean":"2024-09-15T21:46:24.834853+0000","last_became_active":"2024-09-15T21:46:23.427116+0000","last_became_peered":"2024-09-15T21:46:23.427116+0000","last_unstale":"2024-09-15T21:46:24.834853+0000","last_undegraded":"2024-09-15T21:46:24.834853+0000","last_fullsized":"2024-09-15T21:46:24.834853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:35.499970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'17","reported_seq":31,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.550667+0000","last_change":"2024-09-15T21:46:22.419905+0000","last_active":"2024-09-15T21:46:45.550667+0000","last_peered":"2024-09-15T21:46:45.550667+0000","last_clean":"2024-09-15T21:46:45.550667+0000","last_became_active":"2024-09-15T21:46:22.419620+0000","last_became_peered":"2024-09-15T21:46:22.419620+0000","last_unstale":"2024-09-15T21:46:45.550667+0000","last_undegraded":"2024-09-15T21:46:45.550667+0000","last_fullsized":"2024-09-15T21:46:45.550667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:26:51.215156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835459+0000","last_change":"2024-09-15T21:46:23.432975+0000","last_active":"2024-09-15T21:46:24.835459+0000","last_peered":"2024-09-15T21:46:24.835459+0000","last_clean":"2024-09-15T21:46:24.835459+0000","last_became_active":"2024-09-15T21:46:23.432718+0000","last_became_peered":"2024-09-15T21:46:23.432718+0000","last_unstale":"2024-09-15T21:46:24.835459+0000","last_undegraded":"2024-09-15T21:46:24.835459+0000","last_fullsized":"2024-09-15T21:46:24.835459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:59.789691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.419202+0000","last_change":"2024-09-15T21:46:22.429661+0000","last_active":"2024-09-15T21:46:28.419202+0000","last_peered":"2024-09-15T21:46:28.419202+0000","last_clean":"2024-09-15T21:46:28.419202+0000","last_became_active":"2024-09-15T21:46:22.429250+0000","last_became_peered":"2024-09-15T21:46:22.429250+0000","last_unstale":"2024-09-15T21:46:28.419202+0000","last_undegraded":"2024-09-15T21:46:28.419202+0000","last_fullsized":"2024-09-15T21:46:28.419202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:29.341739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835590+0000","last_change":"2024-09-15T21:46:23.435003+0000","last_active":"2024-09-15T21:46:24.835590+0000","last_peered":"2024-09-15T21:46:24.835590+0000","last_clean":"2024-09-15T21:46:24.835590+0000","last_became_active":"2024-09-15T21:46:23.434738+0000","last_became_peered":"2024-09-15T21:46:23.434738+0000","last_unstale":"2024-09-15T21:46:24.835590+0000","last_undegraded":"2024-09-15T21:46:24.835590+0000","last_fullsized":"2024-09-15T21:46:24.835590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:53:15.814759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.571870+0000","last_change":"2024-09-15T21:46:22.425870+0000","last_active":"2024-09-15T21:46:51.571870+0000","last_peered":"2024-09-15T21:46:51.571870+0000","last_clean":"2024-09-15T21:46:51.571870+0000","last_became_active":"2024-09-15T21:46:22.425588+0000","last_became_peered":"2024-09-15T21:46:22.425588+0000","last_unstale":"2024-09-15T21:46:51.571870+0000","last_undegraded":"2024-09-15T21:46:51.571870+0000","last_fullsized":"2024-09-15T21:46:51.571870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:40:07.586608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835837+0000","last_change":"2024-09-15T21:46:23.434657+0000","last_active":"2024-09-15T21:46:24.835837+0000","last_peered":"2024-09-15T21:46:24.835837+0000","last_clean":"2024-09-15T21:46:24.835837+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:46:24.835837+0000","last_undegraded":"2024-09-15T21:46:24.835837+0000","last_fullsized":"2024-09-15T21:46:24.835837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:02.545810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.549133+0000","last_change":"2024-09-15T21:46:22.425072+0000","last_active":"2024-09-15T21:46:45.549133+0000","last_peered":"2024-09-15T21:46:45.549133+0000","last_clean":"2024-09-15T21:46:45.549133+0000","last_became_active":"2024-09-15T21:46:22.424771+0000","last_became_peered":"2024-09-15T21:46:22.424771+0000","last_unstale":"2024-09-15T21:46:45.549133+0000","last_undegraded":"2024-09-15T21:46:45.549133+0000","last_fullsized":"2024-09-15T21:46:45.549133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:26.256785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414457+0000","last_change":"2024-09-15T21:46:23.432497+0000","last_active":"2024-09-15T21:46:24.414457+0000","last_peered":"2024-09-15T21:46:24.414457+0000","last_clean":"2024-09-15T21:46:24.414457+0000","last_became_active":"2024-09-15T21:46:23.431969+0000","last_became_peered":"2024-09-15T21:46:23.431969+0000","last_unstale":"2024-09-15T21:46:24.414457+0000","last_undegraded":"2024-09-15T21:46:24.414457+0000","last_fullsized":"2024-09-15T21:46:24.414457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:32:56.359477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.549727+0000","last_change":"2024-09-15T21:46:22.428668+0000","last_active":"2024-09-15T21:46:45.549727+0000","last_peered":"2024-09-15T21:46:45.549727+0000","last_clean":"2024-09-15T21:46:45.549727+0000","last_became_active":"2024-09-15T21:46:22.428341+0000","last_became_peered":"2024-09-15T21:46:22.428341+0000","last_unstale":"2024-09-15T21:46:45.549727+0000","last_undegraded":"2024-09-15T21:46:45.549727+0000","last_fullsized":"2024-09-15T21:46:45.549727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:29:45.521090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835717+0000","last_change":"2024-09-15T21:46:23.438665+0000","last_active":"2024-09-15T21:46:24.835717+0000","last_peered":"2024-09-15T21:46:24.835717+0000","last_clean":"2024-09-15T21:46:24.835717+0000","last_became_active":"2024-09-15T21:46:23.437606+0000","last_became_peered":"2024-09-15T21:46:23.437606+0000","last_unstale":"2024-09-15T21:46:24.835717+0000","last_undegraded":"2024-09-15T21:46:24.835717+0000","last_fullsized":"2024-09-15T21:46:24.835717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:14:53.751884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":171,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835126+0000","last_change":"2024-09-15T21:46:14.328094+0000","last_active":"2024-09-15T21:46:24.835126+0000","last_peered":"2024-09-15T21:46:24.835126+0000","last_clean":"2024-09-15T21:46:24.835126+0000","last_became_active":"2024-09-15T21:46:14.326991+0000","last_became_peered":"2024-09-15T21:46:14.326991+0000","last_unstale":"2024-09-15T21:46:24.835126+0000","last_undegraded":"2024-09-15T21:46:24.835126+0000","last_fullsized":"2024-09-15T21:46:24.835126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_clean_scrub_stamp":"2024-09-15T21:46:12.223400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:54:27.704130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.550469+0000","last_change":"2024-09-15T21:46:22.428887+0000","last_active":"2024-09-15T21:46:45.550469+0000","last_peered":"2024-09-15T21:46:45.550469+0000","last_clean":"2024-09-15T21:46:45.550469+0000","last_became_active":"2024-09-15T21:46:22.428619+0000","last_became_peered":"2024-09-15T21:46:22.428619+0000","last_unstale":"2024-09-15T21:46:45.550469+0000","last_undegraded":"2024-09-15T21:46:45.550469+0000","last_fullsized":"2024-09-15T21:46:45.550469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:48.434763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834644+0000","last_change":"2024-09-15T21:46:23.435459+0000","last_active":"2024-09-15T21:46:24.834644+0000","last_peered":"2024-09-15T21:46:24.834644+0000","last_clean":"2024-09-15T21:46:24.834644+0000","last_became_active":"2024-09-15T21:46:23.435141+0000","last_became_peered":"2024-09-15T21:46:23.435141+0000","last_unstale":"2024-09-15T21:46:24.834644+0000","last_undegraded":"2024-09-15T21:46:24.834644+0000","last_fullsized":"2024-09-15T21:46:24.834644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:40:50.707124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:40.480640+0000","last_change":"2024-09-15T21:46:22.431113+0000","last_active":"2024-09-15T21:46:40.480640+0000","last_peered":"2024-09-15T21:46:40.480640+0000","last_clean":"2024-09-15T21:46:40.480640+0000","last_became_active":"2024-09-15T21:46:22.430832+0000","last_became_peered":"2024-09-15T21:46:22.430832+0000","last_unstale":"2024-09-15T21:46:40.480640+0000","last_undegraded":"2024-09-15T21:46:40.480640+0000","last_fullsized":"2024-09-15T21:46:40.480640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:28.849259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.726839+0000","last_change":"2024-09-15T21:46:23.430396+0000","last_active":"2024-09-15T21:46:24.726839+0000","last_peered":"2024-09-15T21:46:24.726839+0000","last_clean":"2024-09-15T21:46:24.726839+0000","last_became_active":"2024-09-15T21:46:23.430138+0000","last_became_peered":"2024-09-15T21:46:23.430138+0000","last_unstale":"2024-09-15T21:46:24.726839+0000","last_undegraded":"2024-09-15T21:46:24.726839+0000","last_fullsized":"2024-09-15T21:46:24.726839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:50.622949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:40.483066+0000","last_change":"2024-09-15T21:46:22.425808+0000","last_active":"2024-09-15T21:46:40.483066+0000","last_peered":"2024-09-15T21:46:40.483066+0000","last_clean":"2024-09-15T21:46:40.483066+0000","last_became_active":"2024-09-15T21:46:22.425510+0000","last_became_peered":"2024-09-15T21:46:22.425510+0000","last_unstale":"2024-09-15T21:46:40.483066+0000","last_undegraded":"2024-09-15T21:46:40.483066+0000","last_fullsized":"2024-09-15T21:46:40.483066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:11:00.338076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808338+0000","last_change":"2024-09-15T21:46:23.427051+0000","last_active":"2024-09-15T21:46:24.808338+0000","last_peered":"2024-09-15T21:46:24.808338+0000","last_clean":"2024-09-15T21:46:24.808338+0000","last_became_active":"2024-09-15T21:46:23.426185+0000","last_became_peered":"2024-09-15T21:46:23.426185+0000","last_unstale":"2024-09-15T21:46:24.808338+0000","last_undegraded":"2024-09-15T21:46:24.808338+0000","last_fullsized":"2024-09-15T21:46:24.808338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:48.318405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.572233+0000","last_change":"2024-09-15T21:46:22.424645+0000","last_active":"2024-09-15T21:46:51.572233+0000","last_peered":"2024-09-15T21:46:51.572233+0000","last_clean":"2024-09-15T21:46:51.572233+0000","last_became_active":"2024-09-15T21:46:22.421229+0000","last_became_peered":"2024-09-15T21:46:22.421229+0000","last_unstale":"2024-09-15T21:46:51.572233+0000","last_undegraded":"2024-09-15T21:46:51.572233+0000","last_fullsized":"2024-09-15T21:46:51.572233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:49.165670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414000+0000","last_change":"2024-09-15T21:46:23.425699+0000","last_active":"2024-09-15T21:46:24.414000+0000","last_peered":"2024-09-15T21:46:24.414000+0000","last_clean":"2024-09-15T21:46:24.414000+0000","last_became_active":"2024-09-15T21:46:23.425416+0000","last_became_peered":"2024-09-15T21:46:23.425416+0000","last_unstale":"2024-09-15T21:46:24.414000+0000","last_undegraded":"2024-09-15T21:46:24.414000+0000","last_fullsized":"2024-09-15T21:46:24.414000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:00.325179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808477+0000","last_change":"2024-09-15T21:46:22.426492+0000","last_active":"2024-09-15T21:46:24.808477+0000","last_peered":"2024-09-15T21:46:24.808477+0000","last_clean":"2024-09-15T21:46:24.808477+0000","last_became_active":"2024-09-15T21:46:22.426210+0000","last_became_peered":"2024-09-15T21:46:22.426210+0000","last_unstale":"2024-09-15T21:46:24.808477+0000","last_undegraded":"2024-09-15T21:46:24.808477+0000","last_fullsized":"2024-09-15T21:46:24.808477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:05.248091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414861+0000","last_change":"2024-09-15T21:46:23.429837+0000","last_active":"2024-09-15T21:46:24.414861+0000","last_peered":"2024-09-15T21:46:24.414861+0000","last_clean":"2024-09-15T21:46:24.414861+0000","last_became_active":"2024-09-15T21:46:23.429254+0000","last_became_peered":"2024-09-15T21:46:23.429254+0000","last_unstale":"2024-09-15T21:46:24.414861+0000","last_undegraded":"2024-09-15T21:46:24.414861+0000","last_fullsized":"2024-09-15T21:46:24.414861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:58.465583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.551360+0000","last_change":"2024-09-15T21:46:22.430710+0000","last_active":"2024-09-15T21:46:45.551360+0000","last_peered":"2024-09-15T21:46:45.551360+0000","last_clean":"2024-09-15T21:46:45.551360+0000","last_became_active":"2024-09-15T21:46:22.430375+0000","last_became_peered":"2024-09-15T21:46:22.430375+0000","last_unstale":"2024-09-15T21:46:45.551360+0000","last_undegraded":"2024-09-15T21:46:45.551360+0000","last_fullsized":"2024-09-15T21:46:45.551360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:04.081179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808447+0000","last_change":"2024-09-15T21:46:23.426729+0000","last_active":"2024-09-15T21:46:24.808447+0000","last_peered":"2024-09-15T21:46:24.808447+0000","last_clean":"2024-09-15T21:46:24.808447+0000","last_became_active":"2024-09-15T21:46:23.426495+0000","last_became_peered":"2024-09-15T21:46:23.426495+0000","last_unstale":"2024-09-15T21:46:24.808447+0000","last_undegraded":"2024-09-15T21:46:24.808447+0000","last_fullsized":"2024-09-15T21:46:24.808447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:03.277997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'7","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.549857+0000","last_change":"2024-09-15T21:46:22.427137+0000","last_active":"2024-09-15T21:46:45.549857+0000","last_peered":"2024-09-15T21:46:45.549857+0000","last_clean":"2024-09-15T21:46:45.549857+0000","last_became_active":"2024-09-15T21:46:22.426855+0000","last_became_peered":"2024-09-15T21:46:22.426855+0000","last_unstale":"2024-09-15T21:46:45.549857+0000","last_undegraded":"2024-09-15T21:46:45.549857+0000","last_fullsized":"2024-09-15T21:46:45.549857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:40:14.277940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834714+0000","last_change":"2024-09-15T21:46:23.427327+0000","last_active":"2024-09-15T21:46:24.834714+0000","last_peered":"2024-09-15T21:46:24.834714+0000","last_clean":"2024-09-15T21:46:24.834714+0000","last_became_active":"2024-09-15T21:46:23.427096+0000","last_became_peered":"2024-09-15T21:46:23.427096+0000","last_unstale":"2024-09-15T21:46:24.834714+0000","last_undegraded":"2024-09-15T21:46:24.834714+0000","last_fullsized":"2024-09-15T21:46:24.834714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:32.370390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'7","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.570649+0000","last_change":"2024-09-15T21:46:22.427798+0000","last_active":"2024-09-15T21:46:51.570649+0000","last_peered":"2024-09-15T21:46:51.570649+0000","last_clean":"2024-09-15T21:46:51.570649+0000","last_became_active":"2024-09-15T21:46:22.427537+0000","last_became_peered":"2024-09-15T21:46:22.427537+0000","last_unstale":"2024-09-15T21:46:51.570649+0000","last_undegraded":"2024-09-15T21:46:51.570649+0000","last_fullsized":"2024-09-15T21:46:51.570649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:18:54.963834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727157+0000","last_change":"2024-09-15T21:46:23.424266+0000","last_active":"2024-09-15T21:46:24.727157+0000","last_peered":"2024-09-15T21:46:24.727157+0000","last_clean":"2024-09-15T21:46:24.727157+0000","last_became_active":"2024-09-15T21:46:23.424007+0000","last_became_peered":"2024-09-15T21:46:23.424007+0000","last_unstale":"2024-09-15T21:46:24.727157+0000","last_undegraded":"2024-09-15T21:46:24.727157+0000","last_fullsized":"2024-09-15T21:46:24.727157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:23.492253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:34.461771+0000","last_change":"2024-09-15T21:46:22.430901+0000","last_active":"2024-09-15T21:46:34.461771+0000","last_peered":"2024-09-15T21:46:34.461771+0000","last_clean":"2024-09-15T21:46:34.461771+0000","last_became_active":"2024-09-15T21:46:22.430452+0000","last_became_peered":"2024-09-15T21:46:22.430452+0000","last_unstale":"2024-09-15T21:46:34.461771+0000","last_undegraded":"2024-09-15T21:46:34.461771+0000","last_fullsized":"2024-09-15T21:46:34.461771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:59:45.663976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.413661+0000","last_change":"2024-09-15T21:46:23.430836+0000","last_active":"2024-09-15T21:46:24.413661+0000","last_peered":"2024-09-15T21:46:24.413661+0000","last_clean":"2024-09-15T21:46:24.413661+0000","last_became_active":"2024-09-15T21:46:23.429283+0000","last_became_peered":"2024-09-15T21:46:23.429283+0000","last_unstale":"2024-09-15T21:46:24.413661+0000","last_undegraded":"2024-09-15T21:46:24.413661+0000","last_fullsized":"2024-09-15T21:46:24.413661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:30.571376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'7","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.569263+0000","last_change":"2024-09-15T21:46:22.423795+0000","last_active":"2024-09-15T21:46:51.569263+0000","last_peered":"2024-09-15T21:46:51.569263+0000","last_clean":"2024-09-15T21:46:51.569263+0000","last_became_active":"2024-09-15T21:46:22.422327+0000","last_became_peered":"2024-09-15T21:46:22.422327+0000","last_unstale":"2024-09-15T21:46:51.569263+0000","last_undegraded":"2024-09-15T21:46:51.569263+0000","last_fullsized":"2024-09-15T21:46:51.569263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:16.670599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834522+0000","last_change":"2024-09-15T21:46:23.428634+0000","last_active":"2024-09-15T21:46:24.834522+0000","last_peered":"2024-09-15T21:46:24.834522+0000","last_clean":"2024-09-15T21:46:24.834522+0000","last_became_active":"2024-09-15T21:46:23.428301+0000","last_became_peered":"2024-09-15T21:46:23.428301+0000","last_unstale":"2024-09-15T21:46:24.834522+0000","last_undegraded":"2024-09-15T21:46:24.834522+0000","last_fullsized":"2024-09-15T21:46:24.834522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:59.492942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.547973+0000","last_change":"2024-09-15T21:46:22.425810+0000","last_active":"2024-09-15T21:46:45.547973+0000","last_peered":"2024-09-15T21:46:45.547973+0000","last_clean":"2024-09-15T21:46:45.547973+0000","last_became_active":"2024-09-15T21:46:22.425478+0000","last_became_peered":"2024-09-15T21:46:22.425478+0000","last_unstale":"2024-09-15T21:46:45.547973+0000","last_undegraded":"2024-09-15T21:46:45.547973+0000","last_fullsized":"2024-09-15T21:46:45.547973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:53.296599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834743+0000","last_change":"2024-09-15T21:46:23.426054+0000","last_active":"2024-09-15T21:46:24.834743+0000","last_peered":"2024-09-15T21:46:24.834743+0000","last_clean":"2024-09-15T21:46:24.834743+0000","last_became_active":"2024-09-15T21:46:23.425547+0000","last_became_peered":"2024-09-15T21:46:23.425547+0000","last_unstale":"2024-09-15T21:46:24.834743+0000","last_undegraded":"2024-09-15T21:46:24.834743+0000","last_fullsized":"2024-09-15T21:46:24.834743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:56.838503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:34.458317+0000","last_change":"2024-09-15T21:46:22.428524+0000","last_active":"2024-09-15T21:46:34.458317+0000","last_peered":"2024-09-15T21:46:34.458317+0000","last_clean":"2024-09-15T21:46:34.458317+0000","last_became_active":"2024-09-15T21:46:22.428043+0000","last_became_peered":"2024-09-15T21:46:22.428043+0000","last_unstale":"2024-09-15T21:46:34.458317+0000","last_undegraded":"2024-09-15T21:46:34.458317+0000","last_fullsized":"2024-09-15T21:46:34.458317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:08.393381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834850+0000","last_change":"2024-09-15T21:46:23.434719+0000","last_active":"2024-09-15T21:46:24.834850+0000","last_peered":"2024-09-15T21:46:24.834850+0000","last_clean":"2024-09-15T21:46:24.834850+0000","last_became_active":"2024-09-15T21:46:23.434448+0000","last_became_peered":"2024-09-15T21:46:23.434448+0000","last_unstale":"2024-09-15T21:46:24.834850+0000","last_undegraded":"2024-09-15T21:46:24.834850+0000","last_fullsized":"2024-09-15T21:46:24.834850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:14.901671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414691+0000","last_change":"2024-09-15T21:46:23.430058+0000","last_active":"2024-09-15T21:46:24.414691+0000","last_peered":"2024-09-15T21:46:24.414691+0000","last_clean":"2024-09-15T21:46:24.414691+0000","last_became_active":"2024-09-15T21:46:23.429598+0000","last_became_peered":"2024-09-15T21:46:23.429598+0000","last_unstale":"2024-09-15T21:46:24.414691+0000","last_undegraded":"2024-09-15T21:46:24.414691+0000","last_fullsized":"2024-09-15T21:46:24.414691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:44.142207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.570689+0000","last_change":"2024-09-15T21:46:22.431172+0000","last_active":"2024-09-15T21:46:51.570689+0000","last_peered":"2024-09-15T21:46:51.570689+0000","last_clean":"2024-09-15T21:46:51.570689+0000","last_became_active":"2024-09-15T21:46:22.430800+0000","last_became_peered":"2024-09-15T21:46:22.430800+0000","last_unstale":"2024-09-15T21:46:51.570689+0000","last_undegraded":"2024-09-15T21:46:51.570689+0000","last_fullsized":"2024-09-15T21:46:51.570689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:55.072466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414264+0000","last_change":"2024-09-15T21:46:23.434353+0000","last_active":"2024-09-15T21:46:24.414264+0000","last_peered":"2024-09-15T21:46:24.414264+0000","last_clean":"2024-09-15T21:46:24.414264+0000","last_became_active":"2024-09-15T21:46:23.434097+0000","last_became_peered":"2024-09-15T21:46:23.434097+0000","last_unstale":"2024-09-15T21:46:24.414264+0000","last_undegraded":"2024-09-15T21:46:24.414264+0000","last_fullsized":"2024-09-15T21:46:24.414264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:46.007227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'12","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.571743+0000","last_change":"2024-09-15T21:46:22.429870+0000","last_active":"2024-09-15T21:46:51.571743+0000","last_peered":"2024-09-15T21:46:51.571743+0000","last_clean":"2024-09-15T21:46:51.571743+0000","last_became_active":"2024-09-15T21:46:22.429578+0000","last_became_peered":"2024-09-15T21:46:22.429578+0000","last_unstale":"2024-09-15T21:46:51.571743+0000","last_undegraded":"2024-09-15T21:46:51.571743+0000","last_fullsized":"2024-09-15T21:46:51.571743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:41.678078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727142+0000","last_change":"2024-09-15T21:46:23.429604+0000","last_active":"2024-09-15T21:46:24.727142+0000","last_peered":"2024-09-15T21:46:24.727142+0000","last_clean":"2024-09-15T21:46:24.727142+0000","last_became_active":"2024-09-15T21:46:23.429355+0000","last_became_peered":"2024-09-15T21:46:23.429355+0000","last_unstale":"2024-09-15T21:46:24.727142+0000","last_undegraded":"2024-09-15T21:46:24.727142+0000","last_fullsized":"2024-09-15T21:46:24.727142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:45.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.416728+0000","last_change":"2024-09-15T21:46:22.428529+0000","last_active":"2024-09-15T21:46:28.416728+0000","last_peered":"2024-09-15T21:46:28.416728+0000","last_clean":"2024-09-15T21:46:28.416728+0000","last_became_active":"2024-09-15T21:46:22.428263+0000","last_became_peered":"2024-09-15T21:46:22.428263+0000","last_unstale":"2024-09-15T21:46:28.416728+0000","last_undegraded":"2024-09-15T21:46:28.416728+0000","last_fullsized":"2024-09-15T21:46:28.416728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:22:40.160062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834701+0000","last_change":"2024-09-15T21:46:23.427968+0000","last_active":"2024-09-15T21:46:24.834701+0000","last_peered":"2024-09-15T21:46:24.834701+0000","last_clean":"2024-09-15T21:46:24.834701+0000","last_became_active":"2024-09-15T21:46:23.427546+0000","last_became_peered":"2024-09-15T21:46:23.427546+0000","last_unstale":"2024-09-15T21:46:24.834701+0000","last_undegraded":"2024-09-15T21:46:24.834701+0000","last_fullsized":"2024-09-15T21:46:24.834701+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:47.226097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'13","reported_seq":27,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.569304+0000","last_change":"2024-09-15T21:46:22.428258+0000","last_active":"2024-09-15T21:46:51.569304+0000","last_peered":"2024-09-15T21:46:51.569304+0000","last_clean":"2024-09-15T21:46:51.569304+0000","last_became_active":"2024-09-15T21:46:22.427954+0000","last_became_peered":"2024-09-15T21:46:22.427954+0000","last_unstale":"2024-09-15T21:46:51.569304+0000","last_undegraded":"2024-09-15T21:46:51.569304+0000","last_fullsized":"2024-09-15T21:46:51.569304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:22.440456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8034,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":205,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":16068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":179,"ondisk_log_size":179,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476748,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":843776,"data_stored":668200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509452,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":669426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":79360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":80890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509453,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":78288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509452,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":78420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542156,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":78490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":79394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:47:01.429 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:47:01.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.562+0000 7f43a1711640 1 Processor -- start 2024-09-15T21:47:01.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.562+0000 7f43a1711640 1 -- start start 2024-09-15T21:47:01.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.563+0000 7f43a1711640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f439c155440 0x7f439c0cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:01.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.563+0000 7f43a1711640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f439c0cb7d0 0x7f439c0cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:01.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.563+0000 7f43a1711640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f439c15e7a0 con 0x7f439c06d0e0 2024-09-15T21:47:01.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.563+0000 7f43a1711640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f439c15b470 con 0x7f439c0cb7d0 2024-09-15T21:47:01.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.563+0000 7f43a1711640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f439c155820 con 0x7f439c155440 2024-09-15T21:47:01.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.563+0000 7f439affd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f439c155440 0x7f439c0cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:01.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.563+0000 7f4393fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f439c0cb7d0 0x7f439c0cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:01.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.563+0000 7f439affd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f439c155440 0x7f439c0cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:54204/0 (socket says 172.21.15.100:54204) 2024-09-15T21:47:01.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.563+0000 7f439affd640 1 -- 172.21.15.100:0/3811276803 learned_addr learned my addr 172.21.15.100:0/3811276803 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:01.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.563+0000 7f4393fff640 1 -- 172.21.15.100:0/3811276803 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f439c155440 msgr2=0x7f439c0cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:01.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.563+0000 7f4393fff640 1 --2- 172.21.15.100:0/3811276803 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f439c155440 0x7f439c0cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.563+0000 7f4393fff640 1 -- 172.21.15.100:0/3811276803 >> v1:172.21.15.122:6789/0 conn(0x7f439c06d0e0 legacy=0x7f439c0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:01.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.563+0000 7f4393fff640 1 -- 172.21.15.100:0/3811276803 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439c15ea40 con 0x7f439c0cb7d0 2024-09-15T21:47:01.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.563+0000 7f439affd640 1 --2- 172.21.15.100:0/3811276803 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f439c155440 0x7f439c0cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:01.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.564+0000 7f4393fff640 1 --2- 172.21.15.100:0/3811276803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f439c0cb7d0 0x7f439c0cd960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f43840097a0 tx=0x7f438402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab8d07fcc1dd41a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:01.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.564+0000 7f439a7fc640 1 -- 172.21.15.100:0/3811276803 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4384037070 con 0x7f439c0cb7d0 2024-09-15T21:47:01.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.564+0000 7f439a7fc640 1 -- 172.21.15.100:0/3811276803 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4384032440 con 0x7f439c0cb7d0 2024-09-15T21:47:01.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.564+0000 7f439a7fc640 1 -- 172.21.15.100:0/3811276803 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4384032810 con 0x7f439c0cb7d0 2024-09-15T21:47:01.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.565+0000 7f43a1711640 1 -- 172.21.15.100:0/3811276803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f439c0cb7d0 msgr2=0x7f439c0cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:01.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.565+0000 7f43a1711640 1 --2- 172.21.15.100:0/3811276803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f439c0cb7d0 0x7f439c0cd960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f43840097a0 tx=0x7f438402a180 comp rx=0 tx=0).stop 2024-09-15T21:47:01.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.565+0000 7f43a1711640 1 -- 172.21.15.100:0/3811276803 shutdown_connections 2024-09-15T21:47:01.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.565+0000 7f43a1711640 1 --2- 172.21.15.100:0/3811276803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f439c0cb7d0 0x7f439c0cd960 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.565+0000 7f43a1711640 1 --2- 172.21.15.100:0/3811276803 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f439c155440 0x7f439c0cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.565+0000 7f43a1711640 1 -- 172.21.15.100:0/3811276803 >> 172.21.15.100:0/3811276803 conn(0x7f439c0680d0 msgr2=0x7f439c06d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:01.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.565+0000 7f43a1711640 1 -- 172.21.15.100:0/3811276803 shutdown_connections 2024-09-15T21:47:01.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.565+0000 7f43a1711640 1 -- 172.21.15.100:0/3811276803 wait complete. 2024-09-15T21:47:01.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.566+0000 7f43a1711640 1 Processor -- start 2024-09-15T21:47:01.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.566+0000 7f43a1711640 1 -- start start 2024-09-15T21:47:01.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f43a1711640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f439c06d0e0 0x7f439c0cfd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:01.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f43a1711640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f439c0cb7d0 0x7f439c1f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:01.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f43a1711640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f439c155440 0x7f439c1f51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:01.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f43a1711640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f439c15f2e0 con 0x7f439c06d0e0 2024-09-15T21:47:01.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f43a1711640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f439c15f000 con 0x7f439c155440 2024-09-15T21:47:01.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f43a1711640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f439c15b070 con 0x7f439c0cb7d0 2024-09-15T21:47:01.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f439affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f439c06d0e0 0x7f439c0cfd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:01.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f439affd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f439c06d0e0 0x7f439c0cfd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48882/0 (socket says 172.21.15.100:48882) 2024-09-15T21:47:01.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f439affd640 1 -- 172.21.15.100:0/797127354 learned_addr learned my addr 172.21.15.100:0/797127354 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:01.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f4393fff640 1 --2- 172.21.15.100:0/797127354 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f439c0cb7d0 0x7f439c1f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:01.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f439b7fe640 1 --2- 172.21.15.100:0/797127354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f439c155440 0x7f439c1f51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:01.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f439affd640 1 -- 172.21.15.100:0/797127354 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f439c0cb7d0 msgr2=0x7f439c1f4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:01.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f439affd640 1 --2- 172.21.15.100:0/797127354 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f439c0cb7d0 0x7f439c1f4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f439affd640 1 -- 172.21.15.100:0/797127354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f439c155440 msgr2=0x7f439c1f51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:01.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f439affd640 1 --2- 172.21.15.100:0/797127354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f439c155440 0x7f439c1f51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f439affd640 1 -- 172.21.15.100:0/797127354 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439c153ce0 con 0x7f439c06d0e0 2024-09-15T21:47:01.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f439affd640 1 --2- 172.21.15.100:0/797127354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f439c06d0e0 0x7f439c0cfd50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f438c00e9d0 tx=0x7f438c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:01.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f43937fe640 1 -- 172.21.15.100:0/797127354 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c017070 con 0x7f439c06d0e0 2024-09-15T21:47:01.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f43937fe640 1 -- 172.21.15.100:0/797127354 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f438c00ca60 con 0x7f439c06d0e0 2024-09-15T21:47:01.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f43a1711640 1 -- 172.21.15.100:0/797127354 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439c153f50 con 0x7f439c06d0e0 2024-09-15T21:47:01.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.567+0000 7f43937fe640 1 -- 172.21.15.100:0/797127354 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c00ce10 con 0x7f439c06d0e0 2024-09-15T21:47:01.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.568+0000 7f43a1711640 1 -- 172.21.15.100:0/797127354 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f439c154510 con 0x7f439c06d0e0 2024-09-15T21:47:01.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.569+0000 7f43a1711640 1 -- 172.21.15.100:0/797127354 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f439c0769a0 con 0x7f439c06d0e0 2024-09-15T21:47:01.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.569+0000 7f43937fe640 1 -- 172.21.15.100:0/797127354 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f438c01a070 con 0x7f439c06d0e0 2024-09-15T21:47:01.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.569+0000 7f43937fe640 1 --2- 172.21.15.100:0/797127354 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f437c06c2b0 0x7f437c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:01.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.569+0000 7f43937fe640 1 -- 172.21.15.100:0/797127354 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f438c08b3a0 con 0x7f439c06d0e0 2024-09-15T21:47:01.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.569+0000 7f4393fff640 1 --2- 172.21.15.100:0/797127354 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f437c06c2b0 0x7f437c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:01.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.570+0000 7f4393fff640 1 --2- 172.21.15.100:0/797127354 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f437c06c2b0 0x7f437c06e770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4384034000 tx=0x7f438402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:01.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.574+0000 7f43937fe640 1 -- 172.21.15.100:0/797127354 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f439c0769a0 con 0x7f439c06d0e0 2024-09-15T21:47:01.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.737+0000 7f43a1711640 1 -- 172.21.15.100:0/797127354 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f439c0b3940 con 0x7f437c06c2b0 2024-09-15T21:47:01.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.743+0000 7f43937fe640 1 -- 172.21.15.100:0/797127354 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251952 (secure 0 0 0) 0x7f439c0b3940 con 0x7f437c06c2b0 2024-09-15T21:47:01.747 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:47:01.749 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-15T21:47:01.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.746+0000 7f43a1711640 1 -- 172.21.15.100:0/797127354 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f437c06c2b0 msgr2=0x7f437c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:01.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.746+0000 7f43a1711640 1 --2- 172.21.15.100:0/797127354 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f437c06c2b0 0x7f437c06e770 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4384034000 tx=0x7f438402e000 comp rx=0 tx=0).stop 2024-09-15T21:47:01.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.746+0000 7f43a1711640 1 -- 172.21.15.100:0/797127354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f439c06d0e0 msgr2=0x7f439c0cfd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:01.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.746+0000 7f43a1711640 1 --2- 172.21.15.100:0/797127354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f439c06d0e0 0x7f439c0cfd50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f438c00e9d0 tx=0x7f438c00ee90 comp rx=0 tx=0).stop 2024-09-15T21:47:01.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.746+0000 7f43a1711640 1 -- 172.21.15.100:0/797127354 shutdown_connections 2024-09-15T21:47:01.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.746+0000 7f43a1711640 1 --2- 172.21.15.100:0/797127354 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f437c06c2b0 0x7f437c06e770 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.746+0000 7f43a1711640 1 --2- 172.21.15.100:0/797127354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f439c155440 0x7f439c1f51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.746+0000 7f43a1711640 1 --2- 172.21.15.100:0/797127354 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f439c0cb7d0 0x7f439c1f4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.747+0000 7f43a1711640 1 --2- 172.21.15.100:0/797127354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f439c06d0e0 0x7f439c0cfd50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.747+0000 7f43a1711640 1 -- 172.21.15.100:0/797127354 >> 172.21.15.100:0/797127354 conn(0x7f439c0680d0 msgr2=0x7f439c14b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:01.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.747+0000 7f43a1711640 1 -- 172.21.15.100:0/797127354 shutdown_connections 2024-09-15T21:47:01.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.747+0000 7f43a1711640 1 -- 172.21.15.100:0/797127354 wait complete. 2024-09-15T21:47:01.759 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-09-15T21:46:59.812084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598402,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":364,"num_write_kb":2478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223496,"kb_used_data":3144,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751224,"statfs":{"total":773094113280,"available":772865253376,"internally_reserved":0,"allocated":3219456,"data_stored":1812468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2812,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003542"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835668+0000","last_change":"2024-09-15T21:46:23.428662+0000","last_active":"2024-09-15T21:46:24.835668+0000","last_peered":"2024-09-15T21:46:24.835668+0000","last_clean":"2024-09-15T21:46:24.835668+0000","last_became_active":"2024-09-15T21:46:23.427735+0000","last_became_peered":"2024-09-15T21:46:23.427735+0000","last_unstale":"2024-09-15T21:46:24.835668+0000","last_undegraded":"2024-09-15T21:46:24.835668+0000","last_fullsized":"2024-09-15T21:46:24.835668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:29.702757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808513+0000","last_change":"2024-09-15T21:46:23.418716+0000","last_active":"2024-09-15T21:46:24.808513+0000","last_peered":"2024-09-15T21:46:24.808513+0000","last_clean":"2024-09-15T21:46:24.808513+0000","last_became_active":"2024-09-15T21:46:23.418471+0000","last_became_peered":"2024-09-15T21:46:23.418471+0000","last_unstale":"2024-09-15T21:46:24.808513+0000","last_undegraded":"2024-09-15T21:46:24.808513+0000","last_fullsized":"2024-09-15T21:46:24.808513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:49.599801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727016+0000","last_change":"2024-09-15T21:46:23.426404+0000","last_active":"2024-09-15T21:46:24.727016+0000","last_peered":"2024-09-15T21:46:24.727016+0000","last_clean":"2024-09-15T21:46:24.727016+0000","last_became_active":"2024-09-15T21:46:23.426146+0000","last_became_peered":"2024-09-15T21:46:23.426146+0000","last_unstale":"2024-09-15T21:46:24.727016+0000","last_undegraded":"2024-09-15T21:46:24.727016+0000","last_fullsized":"2024-09-15T21:46:24.727016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:23.626109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414109+0000","last_change":"2024-09-15T21:46:23.432515+0000","last_active":"2024-09-15T21:46:24.414109+0000","last_peered":"2024-09-15T21:46:24.414109+0000","last_clean":"2024-09-15T21:46:24.414109+0000","last_became_active":"2024-09-15T21:46:23.432283+0000","last_became_peered":"2024-09-15T21:46:23.432283+0000","last_unstale":"2024-09-15T21:46:24.414109+0000","last_undegraded":"2024-09-15T21:46:24.414109+0000","last_fullsized":"2024-09-15T21:46:24.414109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:53.288080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835464+0000","last_change":"2024-09-15T21:46:23.427120+0000","last_active":"2024-09-15T21:46:24.835464+0000","last_peered":"2024-09-15T21:46:24.835464+0000","last_clean":"2024-09-15T21:46:24.835464+0000","last_became_active":"2024-09-15T21:46:23.426706+0000","last_became_peered":"2024-09-15T21:46:23.426706+0000","last_unstale":"2024-09-15T21:46:24.835464+0000","last_undegraded":"2024-09-15T21:46:24.835464+0000","last_fullsized":"2024-09-15T21:46:24.835464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:29.513446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414665+0000","last_change":"2024-09-15T21:46:23.434690+0000","last_active":"2024-09-15T21:46:24.414665+0000","last_peered":"2024-09-15T21:46:24.414665+0000","last_clean":"2024-09-15T21:46:24.414665+0000","last_became_active":"2024-09-15T21:46:23.434146+0000","last_became_peered":"2024-09-15T21:46:23.434146+0000","last_unstale":"2024-09-15T21:46:24.414665+0000","last_undegraded":"2024-09-15T21:46:24.414665+0000","last_fullsized":"2024-09-15T21:46:24.414665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:26:59.905178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835273+0000","last_change":"2024-09-15T21:46:23.429541+0000","last_active":"2024-09-15T21:46:24.835273+0000","last_peered":"2024-09-15T21:46:24.835273+0000","last_clean":"2024-09-15T21:46:24.835273+0000","last_became_active":"2024-09-15T21:46:23.429263+0000","last_became_peered":"2024-09-15T21:46:23.429263+0000","last_unstale":"2024-09-15T21:46:24.835273+0000","last_undegraded":"2024-09-15T21:46:24.835273+0000","last_fullsized":"2024-09-15T21:46:24.835273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:34:51.530981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834382+0000","last_change":"2024-09-15T21:46:23.437143+0000","last_active":"2024-09-15T21:46:24.834382+0000","last_peered":"2024-09-15T21:46:24.834382+0000","last_clean":"2024-09-15T21:46:24.834382+0000","last_became_active":"2024-09-15T21:46:23.436864+0000","last_became_peered":"2024-09-15T21:46:23.436864+0000","last_unstale":"2024-09-15T21:46:24.834382+0000","last_undegraded":"2024-09-15T21:46:24.834382+0000","last_fullsized":"2024-09-15T21:46:24.834382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:45.870403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835817+0000","last_change":"2024-09-15T21:46:23.438642+0000","last_active":"2024-09-15T21:46:24.835817+0000","last_peered":"2024-09-15T21:46:24.835817+0000","last_clean":"2024-09-15T21:46:24.835817+0000","last_became_active":"2024-09-15T21:46:23.437521+0000","last_became_peered":"2024-09-15T21:46:23.437521+0000","last_unstale":"2024-09-15T21:46:24.835817+0000","last_undegraded":"2024-09-15T21:46:24.835817+0000","last_fullsized":"2024-09-15T21:46:24.835817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:04.648853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835520+0000","last_change":"2024-09-15T21:46:23.436106+0000","last_active":"2024-09-15T21:46:24.835520+0000","last_peered":"2024-09-15T21:46:24.835520+0000","last_clean":"2024-09-15T21:46:24.835520+0000","last_became_active":"2024-09-15T21:46:23.435821+0000","last_became_peered":"2024-09-15T21:46:23.435821+0000","last_unstale":"2024-09-15T21:46:24.835520+0000","last_undegraded":"2024-09-15T21:46:24.835520+0000","last_fullsized":"2024-09-15T21:46:24.835520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835070+0000","last_change":"2024-09-15T21:46:23.435442+0000","last_active":"2024-09-15T21:46:24.835070+0000","last_peered":"2024-09-15T21:46:24.835070+0000","last_clean":"2024-09-15T21:46:24.835070+0000","last_became_active":"2024-09-15T21:46:23.435165+0000","last_became_peered":"2024-09-15T21:46:23.435165+0000","last_unstale":"2024-09-15T21:46:24.835070+0000","last_undegraded":"2024-09-15T21:46:24.835070+0000","last_fullsized":"2024-09-15T21:46:24.835070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:02.361682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414099+0000","last_change":"2024-09-15T21:46:23.432534+0000","last_active":"2024-09-15T21:46:24.414099+0000","last_peered":"2024-09-15T21:46:24.414099+0000","last_clean":"2024-09-15T21:46:24.414099+0000","last_became_active":"2024-09-15T21:46:23.430170+0000","last_became_peered":"2024-09-15T21:46:23.430170+0000","last_unstale":"2024-09-15T21:46:24.414099+0000","last_undegraded":"2024-09-15T21:46:24.414099+0000","last_fullsized":"2024-09-15T21:46:24.414099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:11.463937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834691+0000","last_change":"2024-09-15T21:46:23.427841+0000","last_active":"2024-09-15T21:46:24.834691+0000","last_peered":"2024-09-15T21:46:24.834691+0000","last_clean":"2024-09-15T21:46:24.834691+0000","last_became_active":"2024-09-15T21:46:23.427621+0000","last_became_peered":"2024-09-15T21:46:23.427621+0000","last_unstale":"2024-09-15T21:46:24.834691+0000","last_undegraded":"2024-09-15T21:46:24.834691+0000","last_fullsized":"2024-09-15T21:46:24.834691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:18.331943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.726852+0000","last_change":"2024-09-15T21:46:23.424707+0000","last_active":"2024-09-15T21:46:24.726852+0000","last_peered":"2024-09-15T21:46:24.726852+0000","last_clean":"2024-09-15T21:46:24.726852+0000","last_became_active":"2024-09-15T21:46:23.424447+0000","last_became_peered":"2024-09-15T21:46:23.424447+0000","last_unstale":"2024-09-15T21:46:24.726852+0000","last_undegraded":"2024-09-15T21:46:24.726852+0000","last_fullsized":"2024-09-15T21:46:24.726852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:23.617214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808321+0000","last_change":"2024-09-15T21:46:23.430297+0000","last_active":"2024-09-15T21:46:24.808321+0000","last_peered":"2024-09-15T21:46:24.808321+0000","last_clean":"2024-09-15T21:46:24.808321+0000","last_became_active":"2024-09-15T21:46:23.430070+0000","last_became_peered":"2024-09-15T21:46:23.430070+0000","last_unstale":"2024-09-15T21:46:24.808321+0000","last_undegraded":"2024-09-15T21:46:24.808321+0000","last_fullsized":"2024-09-15T21:46:24.808321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:03:19.615022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414542+0000","last_change":"2024-09-15T21:46:23.430540+0000","last_active":"2024-09-15T21:46:24.414542+0000","last_peered":"2024-09-15T21:46:24.414542+0000","last_clean":"2024-09-15T21:46:24.414542+0000","last_became_active":"2024-09-15T21:46:23.430256+0000","last_became_peered":"2024-09-15T21:46:23.430256+0000","last_unstale":"2024-09-15T21:46:24.414542+0000","last_undegraded":"2024-09-15T21:46:24.414542+0000","last_fullsized":"2024-09-15T21:46:24.414542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:36.539762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.415216+0000","last_change":"2024-09-15T21:46:23.427802+0000","last_active":"2024-09-15T21:46:24.415216+0000","last_peered":"2024-09-15T21:46:24.415216+0000","last_clean":"2024-09-15T21:46:24.415216+0000","last_became_active":"2024-09-15T21:46:23.427502+0000","last_became_peered":"2024-09-15T21:46:23.427502+0000","last_unstale":"2024-09-15T21:46:24.415216+0000","last_undegraded":"2024-09-15T21:46:24.415216+0000","last_fullsized":"2024-09-15T21:46:24.415216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:31.062624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412600+0000","last_change":"2024-09-15T21:46:23.427570+0000","last_active":"2024-09-15T21:46:24.412600+0000","last_peered":"2024-09-15T21:46:24.412600+0000","last_clean":"2024-09-15T21:46:24.412600+0000","last_became_active":"2024-09-15T21:46:23.427304+0000","last_became_peered":"2024-09-15T21:46:23.427304+0000","last_unstale":"2024-09-15T21:46:24.412600+0000","last_undegraded":"2024-09-15T21:46:24.412600+0000","last_fullsized":"2024-09-15T21:46:24.412600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:37.345106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835899+0000","last_change":"2024-09-15T21:46:23.431434+0000","last_active":"2024-09-15T21:46:24.835899+0000","last_peered":"2024-09-15T21:46:24.835899+0000","last_clean":"2024-09-15T21:46:24.835899+0000","last_became_active":"2024-09-15T21:46:23.430987+0000","last_became_peered":"2024-09-15T21:46:23.430987+0000","last_unstale":"2024-09-15T21:46:24.835899+0000","last_undegraded":"2024-09-15T21:46:24.835899+0000","last_fullsized":"2024-09-15T21:46:24.835899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:48:30.532378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835198+0000","last_change":"2024-09-15T21:46:23.434558+0000","last_active":"2024-09-15T21:46:24.835198+0000","last_peered":"2024-09-15T21:46:24.835198+0000","last_clean":"2024-09-15T21:46:24.835198+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:46:24.835198+0000","last_undegraded":"2024-09-15T21:46:24.835198+0000","last_fullsized":"2024-09-15T21:46:24.835198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:03:39.562977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727140+0000","last_change":"2024-09-15T21:46:23.428323+0000","last_active":"2024-09-15T21:46:24.727140+0000","last_peered":"2024-09-15T21:46:24.727140+0000","last_clean":"2024-09-15T21:46:24.727140+0000","last_became_active":"2024-09-15T21:46:23.428016+0000","last_became_peered":"2024-09-15T21:46:23.428016+0000","last_unstale":"2024-09-15T21:46:24.727140+0000","last_undegraded":"2024-09-15T21:46:24.727140+0000","last_fullsized":"2024-09-15T21:46:24.727140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:28:48.678550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414612+0000","last_change":"2024-09-15T21:46:23.434465+0000","last_active":"2024-09-15T21:46:24.414612+0000","last_peered":"2024-09-15T21:46:24.414612+0000","last_clean":"2024-09-15T21:46:24.414612+0000","last_became_active":"2024-09-15T21:46:23.433849+0000","last_became_peered":"2024-09-15T21:46:23.433849+0000","last_unstale":"2024-09-15T21:46:24.414612+0000","last_undegraded":"2024-09-15T21:46:24.414612+0000","last_fullsized":"2024-09-15T21:46:24.414612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:46.093198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834522+0000","last_change":"2024-09-15T21:46:23.431527+0000","last_active":"2024-09-15T21:46:24.834522+0000","last_peered":"2024-09-15T21:46:24.834522+0000","last_clean":"2024-09-15T21:46:24.834522+0000","last_became_active":"2024-09-15T21:46:23.430719+0000","last_became_peered":"2024-09-15T21:46:23.430719+0000","last_unstale":"2024-09-15T21:46:24.834522+0000","last_undegraded":"2024-09-15T21:46:24.834522+0000","last_fullsized":"2024-09-15T21:46:24.834522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:16.326529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414532+0000","last_change":"2024-09-15T21:46:23.432282+0000","last_active":"2024-09-15T21:46:24.414532+0000","last_peered":"2024-09-15T21:46:24.414532+0000","last_clean":"2024-09-15T21:46:24.414532+0000","last_became_active":"2024-09-15T21:46:23.432035+0000","last_became_peered":"2024-09-15T21:46:23.432035+0000","last_unstale":"2024-09-15T21:46:24.414532+0000","last_undegraded":"2024-09-15T21:46:24.414532+0000","last_fullsized":"2024-09-15T21:46:24.414532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:22.163940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834776+0000","last_change":"2024-09-15T21:46:23.432646+0000","last_active":"2024-09-15T21:46:24.834776+0000","last_peered":"2024-09-15T21:46:24.834776+0000","last_clean":"2024-09-15T21:46:24.834776+0000","last_became_active":"2024-09-15T21:46:23.432317+0000","last_became_peered":"2024-09-15T21:46:23.432317+0000","last_unstale":"2024-09-15T21:46:24.834776+0000","last_undegraded":"2024-09-15T21:46:24.834776+0000","last_fullsized":"2024-09-15T21:46:24.834776+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:49:07.706450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835541+0000","last_change":"2024-09-15T21:46:23.428183+0000","last_active":"2024-09-15T21:46:24.835541+0000","last_peered":"2024-09-15T21:46:24.835541+0000","last_clean":"2024-09-15T21:46:24.835541+0000","last_became_active":"2024-09-15T21:46:23.427886+0000","last_became_peered":"2024-09-15T21:46:23.427886+0000","last_unstale":"2024-09-15T21:46:24.835541+0000","last_undegraded":"2024-09-15T21:46:24.835541+0000","last_fullsized":"2024-09-15T21:46:24.835541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:15.584377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808369+0000","last_change":"2024-09-15T21:46:23.421749+0000","last_active":"2024-09-15T21:46:24.808369+0000","last_peered":"2024-09-15T21:46:24.808369+0000","last_clean":"2024-09-15T21:46:24.808369+0000","last_became_active":"2024-09-15T21:46:23.421501+0000","last_became_peered":"2024-09-15T21:46:23.421501+0000","last_unstale":"2024-09-15T21:46:24.808369+0000","last_undegraded":"2024-09-15T21:46:24.808369+0000","last_fullsized":"2024-09-15T21:46:24.808369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:55.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834843+0000","last_change":"2024-09-15T21:46:23.432591+0000","last_active":"2024-09-15T21:46:24.834843+0000","last_peered":"2024-09-15T21:46:24.834843+0000","last_clean":"2024-09-15T21:46:24.834843+0000","last_became_active":"2024-09-15T21:46:23.432326+0000","last_became_peered":"2024-09-15T21:46:23.432326+0000","last_unstale":"2024-09-15T21:46:24.834843+0000","last_undegraded":"2024-09-15T21:46:24.834843+0000","last_fullsized":"2024-09-15T21:46:24.834843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:48.565579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727016+0000","last_change":"2024-09-15T21:46:23.423428+0000","last_active":"2024-09-15T21:46:24.727016+0000","last_peered":"2024-09-15T21:46:24.727016+0000","last_clean":"2024-09-15T21:46:24.727016+0000","last_became_active":"2024-09-15T21:46:23.422734+0000","last_became_peered":"2024-09-15T21:46:23.422734+0000","last_unstale":"2024-09-15T21:46:24.727016+0000","last_undegraded":"2024-09-15T21:46:24.727016+0000","last_fullsized":"2024-09-15T21:46:24.727016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:06.592891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414770+0000","last_change":"2024-09-15T21:46:23.435244+0000","last_active":"2024-09-15T21:46:24.414770+0000","last_peered":"2024-09-15T21:46:24.414770+0000","last_clean":"2024-09-15T21:46:24.414770+0000","last_became_active":"2024-09-15T21:46:23.434932+0000","last_became_peered":"2024-09-15T21:46:23.434932+0000","last_unstale":"2024-09-15T21:46:24.414770+0000","last_undegraded":"2024-09-15T21:46:24.414770+0000","last_fullsized":"2024-09-15T21:46:24.414770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:55.810146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412500+0000","last_change":"2024-09-15T21:46:23.437276+0000","last_active":"2024-09-15T21:46:24.412500+0000","last_peered":"2024-09-15T21:46:24.412500+0000","last_clean":"2024-09-15T21:46:24.412500+0000","last_became_active":"2024-09-15T21:46:23.436962+0000","last_became_peered":"2024-09-15T21:46:23.436962+0000","last_unstale":"2024-09-15T21:46:24.412500+0000","last_undegraded":"2024-09-15T21:46:24.412500+0000","last_fullsized":"2024-09-15T21:46:24.412500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:13.043999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834825+0000","last_change":"2024-09-15T21:46:23.429648+0000","last_active":"2024-09-15T21:46:24.834825+0000","last_peered":"2024-09-15T21:46:24.834825+0000","last_clean":"2024-09-15T21:46:24.834825+0000","last_became_active":"2024-09-15T21:46:23.429376+0000","last_became_peered":"2024-09-15T21:46:23.429376+0000","last_unstale":"2024-09-15T21:46:24.834825+0000","last_undegraded":"2024-09-15T21:46:24.834825+0000","last_fullsized":"2024-09-15T21:46:24.834825+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:31.278283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.413663+0000","last_change":"2024-09-15T21:46:23.429847+0000","last_active":"2024-09-15T21:46:24.413663+0000","last_peered":"2024-09-15T21:46:24.413663+0000","last_clean":"2024-09-15T21:46:24.413663+0000","last_became_active":"2024-09-15T21:46:23.429442+0000","last_became_peered":"2024-09-15T21:46:23.429442+0000","last_unstale":"2024-09-15T21:46:24.413663+0000","last_undegraded":"2024-09-15T21:46:24.413663+0000","last_fullsized":"2024-09-15T21:46:24.413663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:31.232512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:34.464539+0000","last_change":"2024-09-15T21:46:22.429407+0000","last_active":"2024-09-15T21:46:34.464539+0000","last_peered":"2024-09-15T21:46:34.464539+0000","last_clean":"2024-09-15T21:46:34.464539+0000","last_became_active":"2024-09-15T21:46:22.429107+0000","last_became_peered":"2024-09-15T21:46:22.429107+0000","last_unstale":"2024-09-15T21:46:34.464539+0000","last_undegraded":"2024-09-15T21:46:34.464539+0000","last_fullsized":"2024-09-15T21:46:34.464539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:14:54.680349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727169+0000","last_change":"2024-09-15T21:46:23.434198+0000","last_active":"2024-09-15T21:46:24.727169+0000","last_peered":"2024-09-15T21:46:24.727169+0000","last_clean":"2024-09-15T21:46:24.727169+0000","last_became_active":"2024-09-15T21:46:23.433944+0000","last_became_peered":"2024-09-15T21:46:23.433944+0000","last_unstale":"2024-09-15T21:46:24.727169+0000","last_undegraded":"2024-09-15T21:46:24.727169+0000","last_fullsized":"2024-09-15T21:46:24.727169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:25.412167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'7","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.550215+0000","last_change":"2024-09-15T21:46:22.425571+0000","last_active":"2024-09-15T21:46:45.550215+0000","last_peered":"2024-09-15T21:46:45.550215+0000","last_clean":"2024-09-15T21:46:45.550215+0000","last_became_active":"2024-09-15T21:46:22.425315+0000","last_became_peered":"2024-09-15T21:46:22.425315+0000","last_unstale":"2024-09-15T21:46:45.550215+0000","last_undegraded":"2024-09-15T21:46:45.550215+0000","last_fullsized":"2024-09-15T21:46:45.550215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:31.983301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:40.480412+0000","last_change":"2024-09-15T21:46:22.425492+0000","last_active":"2024-09-15T21:46:40.480412+0000","last_peered":"2024-09-15T21:46:40.480412+0000","last_clean":"2024-09-15T21:46:40.480412+0000","last_became_active":"2024-09-15T21:46:22.425226+0000","last_became_peered":"2024-09-15T21:46:22.425226+0000","last_unstale":"2024-09-15T21:46:40.480412+0000","last_undegraded":"2024-09-15T21:46:40.480412+0000","last_fullsized":"2024-09-15T21:46:40.480412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:24.322384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834460+0000","last_change":"2024-09-15T21:46:23.429140+0000","last_active":"2024-09-15T21:46:24.834460+0000","last_peered":"2024-09-15T21:46:24.834460+0000","last_clean":"2024-09-15T21:46:24.834460+0000","last_became_active":"2024-09-15T21:46:23.428902+0000","last_became_peered":"2024-09-15T21:46:23.428902+0000","last_unstale":"2024-09-15T21:46:24.834460+0000","last_undegraded":"2024-09-15T21:46:24.834460+0000","last_fullsized":"2024-09-15T21:46:24.834460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:11.256924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.551162+0000","last_change":"2024-09-15T21:46:22.428319+0000","last_active":"2024-09-15T21:46:45.551162+0000","last_peered":"2024-09-15T21:46:45.551162+0000","last_clean":"2024-09-15T21:46:45.551162+0000","last_became_active":"2024-09-15T21:46:22.428061+0000","last_became_peered":"2024-09-15T21:46:22.428061+0000","last_unstale":"2024-09-15T21:46:45.551162+0000","last_undegraded":"2024-09-15T21:46:45.551162+0000","last_fullsized":"2024-09-15T21:46:45.551162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:14.238988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808315+0000","last_change":"2024-09-15T21:46:23.427442+0000","last_active":"2024-09-15T21:46:24.808315+0000","last_peered":"2024-09-15T21:46:24.808315+0000","last_clean":"2024-09-15T21:46:24.808315+0000","last_became_active":"2024-09-15T21:46:23.427200+0000","last_became_peered":"2024-09-15T21:46:23.427200+0000","last_unstale":"2024-09-15T21:46:24.808315+0000","last_undegraded":"2024-09-15T21:46:24.808315+0000","last_fullsized":"2024-09-15T21:46:24.808315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:27.248545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:34.464224+0000","last_change":"2024-09-15T21:46:22.430599+0000","last_active":"2024-09-15T21:46:34.464224+0000","last_peered":"2024-09-15T21:46:34.464224+0000","last_clean":"2024-09-15T21:46:34.464224+0000","last_became_active":"2024-09-15T21:46:22.430327+0000","last_became_peered":"2024-09-15T21:46:22.430327+0000","last_unstale":"2024-09-15T21:46:34.464224+0000","last_undegraded":"2024-09-15T21:46:34.464224+0000","last_fullsized":"2024-09-15T21:46:34.464224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:51.904693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414860+0000","last_change":"2024-09-15T21:46:23.434006+0000","last_active":"2024-09-15T21:46:24.414860+0000","last_peered":"2024-09-15T21:46:24.414860+0000","last_clean":"2024-09-15T21:46:24.414860+0000","last_became_active":"2024-09-15T21:46:23.433634+0000","last_became_peered":"2024-09-15T21:46:23.433634+0000","last_unstale":"2024-09-15T21:46:24.414860+0000","last_undegraded":"2024-09-15T21:46:24.414860+0000","last_fullsized":"2024-09-15T21:46:24.414860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:22.288806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.571048+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:51.571048+0000","last_peered":"2024-09-15T21:46:51.571048+0000","last_clean":"2024-09-15T21:46:51.571048+0000","last_became_active":"2024-09-15T21:46:22.425236+0000","last_became_peered":"2024-09-15T21:46:22.425236+0000","last_unstale":"2024-09-15T21:46:51.571048+0000","last_undegraded":"2024-09-15T21:46:51.571048+0000","last_fullsized":"2024-09-15T21:46:51.571048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:45:21.332492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727237+0000","last_change":"2024-09-15T21:46:23.429587+0000","last_active":"2024-09-15T21:46:24.727237+0000","last_peered":"2024-09-15T21:46:24.727237+0000","last_clean":"2024-09-15T21:46:24.727237+0000","last_became_active":"2024-09-15T21:46:23.429334+0000","last_became_peered":"2024-09-15T21:46:23.429334+0000","last_unstale":"2024-09-15T21:46:24.727237+0000","last_undegraded":"2024-09-15T21:46:24.727237+0000","last_fullsized":"2024-09-15T21:46:24.727237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:48:56.361550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:34.466053+0000","last_change":"2024-09-15T21:46:22.421964+0000","last_active":"2024-09-15T21:46:34.466053+0000","last_peered":"2024-09-15T21:46:34.466053+0000","last_clean":"2024-09-15T21:46:34.466053+0000","last_became_active":"2024-09-15T21:46:22.421661+0000","last_became_peered":"2024-09-15T21:46:22.421661+0000","last_unstale":"2024-09-15T21:46:34.466053+0000","last_undegraded":"2024-09-15T21:46:34.466053+0000","last_fullsized":"2024-09-15T21:46:34.466053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:07.395070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414253+0000","last_change":"2024-09-15T21:46:23.427416+0000","last_active":"2024-09-15T21:46:24.414253+0000","last_peered":"2024-09-15T21:46:24.414253+0000","last_clean":"2024-09-15T21:46:24.414253+0000","last_became_active":"2024-09-15T21:46:23.427057+0000","last_became_peered":"2024-09-15T21:46:23.427057+0000","last_unstale":"2024-09-15T21:46:24.414253+0000","last_undegraded":"2024-09-15T21:46:24.414253+0000","last_fullsized":"2024-09-15T21:46:24.414253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:42.671501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'10","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.572146+0000","last_change":"2024-09-15T21:46:22.426151+0000","last_active":"2024-09-15T21:46:51.572146+0000","last_peered":"2024-09-15T21:46:51.572146+0000","last_clean":"2024-09-15T21:46:51.572146+0000","last_became_active":"2024-09-15T21:46:22.425866+0000","last_became_peered":"2024-09-15T21:46:22.425866+0000","last_unstale":"2024-09-15T21:46:51.572146+0000","last_undegraded":"2024-09-15T21:46:51.572146+0000","last_fullsized":"2024-09-15T21:46:51.572146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:10.684265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835569+0000","last_change":"2024-09-15T21:46:23.432292+0000","last_active":"2024-09-15T21:46:24.835569+0000","last_peered":"2024-09-15T21:46:24.835569+0000","last_clean":"2024-09-15T21:46:24.835569+0000","last_became_active":"2024-09-15T21:46:23.431877+0000","last_became_peered":"2024-09-15T21:46:23.431877+0000","last_unstale":"2024-09-15T21:46:24.835569+0000","last_undegraded":"2024-09-15T21:46:24.835569+0000","last_fullsized":"2024-09-15T21:46:24.835569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:16.857279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'9","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.567901+0000","last_change":"2024-09-15T21:46:22.427415+0000","last_active":"2024-09-15T21:46:51.567901+0000","last_peered":"2024-09-15T21:46:51.567901+0000","last_clean":"2024-09-15T21:46:51.567901+0000","last_became_active":"2024-09-15T21:46:22.427087+0000","last_became_peered":"2024-09-15T21:46:22.427087+0000","last_unstale":"2024-09-15T21:46:51.567901+0000","last_undegraded":"2024-09-15T21:46:51.567901+0000","last_fullsized":"2024-09-15T21:46:51.567901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:09.365202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.726943+0000","last_change":"2024-09-15T21:46:23.427238+0000","last_active":"2024-09-15T21:46:24.726943+0000","last_peered":"2024-09-15T21:46:24.726943+0000","last_clean":"2024-09-15T21:46:24.726943+0000","last_became_active":"2024-09-15T21:46:23.426993+0000","last_became_peered":"2024-09-15T21:46:23.426993+0000","last_unstale":"2024-09-15T21:46:24.726943+0000","last_undegraded":"2024-09-15T21:46:24.726943+0000","last_fullsized":"2024-09-15T21:46:24.726943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:52:24.175554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.550646+0000","last_change":"2024-09-15T21:46:22.427164+0000","last_active":"2024-09-15T21:46:45.550646+0000","last_peered":"2024-09-15T21:46:45.550646+0000","last_clean":"2024-09-15T21:46:45.550646+0000","last_became_active":"2024-09-15T21:46:22.426917+0000","last_became_peered":"2024-09-15T21:46:22.426917+0000","last_unstale":"2024-09-15T21:46:45.550646+0000","last_undegraded":"2024-09-15T21:46:45.550646+0000","last_fullsized":"2024-09-15T21:46:45.550646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:46.471935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835504+0000","last_change":"2024-09-15T21:46:23.433924+0000","last_active":"2024-09-15T21:46:24.835504+0000","last_peered":"2024-09-15T21:46:24.835504+0000","last_clean":"2024-09-15T21:46:24.835504+0000","last_became_active":"2024-09-15T21:46:23.433606+0000","last_became_peered":"2024-09-15T21:46:23.433606+0000","last_unstale":"2024-09-15T21:46:24.835504+0000","last_undegraded":"2024-09-15T21:46:24.835504+0000","last_fullsized":"2024-09-15T21:46:24.835504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:09:35.744149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.549348+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:45.549348+0000","last_peered":"2024-09-15T21:46:45.549348+0000","last_clean":"2024-09-15T21:46:45.549348+0000","last_became_active":"2024-09-15T21:46:22.425237+0000","last_became_peered":"2024-09-15T21:46:22.425237+0000","last_unstale":"2024-09-15T21:46:45.549348+0000","last_undegraded":"2024-09-15T21:46:45.549348+0000","last_fullsized":"2024-09-15T21:46:45.549348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:59.460420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834853+0000","last_change":"2024-09-15T21:46:23.427515+0000","last_active":"2024-09-15T21:46:24.834853+0000","last_peered":"2024-09-15T21:46:24.834853+0000","last_clean":"2024-09-15T21:46:24.834853+0000","last_became_active":"2024-09-15T21:46:23.427116+0000","last_became_peered":"2024-09-15T21:46:23.427116+0000","last_unstale":"2024-09-15T21:46:24.834853+0000","last_undegraded":"2024-09-15T21:46:24.834853+0000","last_fullsized":"2024-09-15T21:46:24.834853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:35.499970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'17","reported_seq":31,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.550667+0000","last_change":"2024-09-15T21:46:22.419905+0000","last_active":"2024-09-15T21:46:45.550667+0000","last_peered":"2024-09-15T21:46:45.550667+0000","last_clean":"2024-09-15T21:46:45.550667+0000","last_became_active":"2024-09-15T21:46:22.419620+0000","last_became_peered":"2024-09-15T21:46:22.419620+0000","last_unstale":"2024-09-15T21:46:45.550667+0000","last_undegraded":"2024-09-15T21:46:45.550667+0000","last_fullsized":"2024-09-15T21:46:45.550667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:26:51.215156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835459+0000","last_change":"2024-09-15T21:46:23.432975+0000","last_active":"2024-09-15T21:46:24.835459+0000","last_peered":"2024-09-15T21:46:24.835459+0000","last_clean":"2024-09-15T21:46:24.835459+0000","last_became_active":"2024-09-15T21:46:23.432718+0000","last_became_peered":"2024-09-15T21:46:23.432718+0000","last_unstale":"2024-09-15T21:46:24.835459+0000","last_undegraded":"2024-09-15T21:46:24.835459+0000","last_fullsized":"2024-09-15T21:46:24.835459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:59.789691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.419202+0000","last_change":"2024-09-15T21:46:22.429661+0000","last_active":"2024-09-15T21:46:28.419202+0000","last_peered":"2024-09-15T21:46:28.419202+0000","last_clean":"2024-09-15T21:46:28.419202+0000","last_became_active":"2024-09-15T21:46:22.429250+0000","last_became_peered":"2024-09-15T21:46:22.429250+0000","last_unstale":"2024-09-15T21:46:28.419202+0000","last_undegraded":"2024-09-15T21:46:28.419202+0000","last_fullsized":"2024-09-15T21:46:28.419202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:29.341739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835590+0000","last_change":"2024-09-15T21:46:23.435003+0000","last_active":"2024-09-15T21:46:24.835590+0000","last_peered":"2024-09-15T21:46:24.835590+0000","last_clean":"2024-09-15T21:46:24.835590+0000","last_became_active":"2024-09-15T21:46:23.434738+0000","last_became_peered":"2024-09-15T21:46:23.434738+0000","last_unstale":"2024-09-15T21:46:24.835590+0000","last_undegraded":"2024-09-15T21:46:24.835590+0000","last_fullsized":"2024-09-15T21:46:24.835590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:53:15.814759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.571870+0000","last_change":"2024-09-15T21:46:22.425870+0000","last_active":"2024-09-15T21:46:51.571870+0000","last_peered":"2024-09-15T21:46:51.571870+0000","last_clean":"2024-09-15T21:46:51.571870+0000","last_became_active":"2024-09-15T21:46:22.425588+0000","last_became_peered":"2024-09-15T21:46:22.425588+0000","last_unstale":"2024-09-15T21:46:51.571870+0000","last_undegraded":"2024-09-15T21:46:51.571870+0000","last_fullsized":"2024-09-15T21:46:51.571870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:40:07.586608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835837+0000","last_change":"2024-09-15T21:46:23.434657+0000","last_active":"2024-09-15T21:46:24.835837+0000","last_peered":"2024-09-15T21:46:24.835837+0000","last_clean":"2024-09-15T21:46:24.835837+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:46:24.835837+0000","last_undegraded":"2024-09-15T21:46:24.835837+0000","last_fullsized":"2024-09-15T21:46:24.835837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:02.545810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.549133+0000","last_change":"2024-09-15T21:46:22.425072+0000","last_active":"2024-09-15T21:46:45.549133+0000","last_peered":"2024-09-15T21:46:45.549133+0000","last_clean":"2024-09-15T21:46:45.549133+0000","last_became_active":"2024-09-15T21:46:22.424771+0000","last_became_peered":"2024-09-15T21:46:22.424771+0000","last_unstale":"2024-09-15T21:46:45.549133+0000","last_undegraded":"2024-09-15T21:46:45.549133+0000","last_fullsized":"2024-09-15T21:46:45.549133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:26.256785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414457+0000","last_change":"2024-09-15T21:46:23.432497+0000","last_active":"2024-09-15T21:46:24.414457+0000","last_peered":"2024-09-15T21:46:24.414457+0000","last_clean":"2024-09-15T21:46:24.414457+0000","last_became_active":"2024-09-15T21:46:23.431969+0000","last_became_peered":"2024-09-15T21:46:23.431969+0000","last_unstale":"2024-09-15T21:46:24.414457+0000","last_undegraded":"2024-09-15T21:46:24.414457+0000","last_fullsized":"2024-09-15T21:46:24.414457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:32:56.359477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.549727+0000","last_change":"2024-09-15T21:46:22.428668+0000","last_active":"2024-09-15T21:46:45.549727+0000","last_peered":"2024-09-15T21:46:45.549727+0000","last_clean":"2024-09-15T21:46:45.549727+0000","last_became_active":"2024-09-15T21:46:22.428341+0000","last_became_peered":"2024-09-15T21:46:22.428341+0000","last_unstale":"2024-09-15T21:46:45.549727+0000","last_undegraded":"2024-09-15T21:46:45.549727+0000","last_fullsized":"2024-09-15T21:46:45.549727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:29:45.521090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835717+0000","last_change":"2024-09-15T21:46:23.438665+0000","last_active":"2024-09-15T21:46:24.835717+0000","last_peered":"2024-09-15T21:46:24.835717+0000","last_clean":"2024-09-15T21:46:24.835717+0000","last_became_active":"2024-09-15T21:46:23.437606+0000","last_became_peered":"2024-09-15T21:46:23.437606+0000","last_unstale":"2024-09-15T21:46:24.835717+0000","last_undegraded":"2024-09-15T21:46:24.835717+0000","last_fullsized":"2024-09-15T21:46:24.835717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:14:53.751884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":171,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835126+0000","last_change":"2024-09-15T21:46:14.328094+0000","last_active":"2024-09-15T21:46:24.835126+0000","last_peered":"2024-09-15T21:46:24.835126+0000","last_clean":"2024-09-15T21:46:24.835126+0000","last_became_active":"2024-09-15T21:46:14.326991+0000","last_became_peered":"2024-09-15T21:46:14.326991+0000","last_unstale":"2024-09-15T21:46:24.835126+0000","last_undegraded":"2024-09-15T21:46:24.835126+0000","last_fullsized":"2024-09-15T21:46:24.835126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_clean_scrub_stamp":"2024-09-15T21:46:12.223400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:54:27.704130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.550469+0000","last_change":"2024-09-15T21:46:22.428887+0000","last_active":"2024-09-15T21:46:45.550469+0000","last_peered":"2024-09-15T21:46:45.550469+0000","last_clean":"2024-09-15T21:46:45.550469+0000","last_became_active":"2024-09-15T21:46:22.428619+0000","last_became_peered":"2024-09-15T21:46:22.428619+0000","last_unstale":"2024-09-15T21:46:45.550469+0000","last_undegraded":"2024-09-15T21:46:45.550469+0000","last_fullsized":"2024-09-15T21:46:45.550469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:48.434763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834644+0000","last_change":"2024-09-15T21:46:23.435459+0000","last_active":"2024-09-15T21:46:24.834644+0000","last_peered":"2024-09-15T21:46:24.834644+0000","last_clean":"2024-09-15T21:46:24.834644+0000","last_became_active":"2024-09-15T21:46:23.435141+0000","last_became_peered":"2024-09-15T21:46:23.435141+0000","last_unstale":"2024-09-15T21:46:24.834644+0000","last_undegraded":"2024-09-15T21:46:24.834644+0000","last_fullsized":"2024-09-15T21:46:24.834644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:40:50.707124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:40.480640+0000","last_change":"2024-09-15T21:46:22.431113+0000","last_active":"2024-09-15T21:46:40.480640+0000","last_peered":"2024-09-15T21:46:40.480640+0000","last_clean":"2024-09-15T21:46:40.480640+0000","last_became_active":"2024-09-15T21:46:22.430832+0000","last_became_peered":"2024-09-15T21:46:22.430832+0000","last_unstale":"2024-09-15T21:46:40.480640+0000","last_undegraded":"2024-09-15T21:46:40.480640+0000","last_fullsized":"2024-09-15T21:46:40.480640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:28.849259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.726839+0000","last_change":"2024-09-15T21:46:23.430396+0000","last_active":"2024-09-15T21:46:24.726839+0000","last_peered":"2024-09-15T21:46:24.726839+0000","last_clean":"2024-09-15T21:46:24.726839+0000","last_became_active":"2024-09-15T21:46:23.430138+0000","last_became_peered":"2024-09-15T21:46:23.430138+0000","last_unstale":"2024-09-15T21:46:24.726839+0000","last_undegraded":"2024-09-15T21:46:24.726839+0000","last_fullsized":"2024-09-15T21:46:24.726839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:50.622949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:40.483066+0000","last_change":"2024-09-15T21:46:22.425808+0000","last_active":"2024-09-15T21:46:40.483066+0000","last_peered":"2024-09-15T21:46:40.483066+0000","last_clean":"2024-09-15T21:46:40.483066+0000","last_became_active":"2024-09-15T21:46:22.425510+0000","last_became_peered":"2024-09-15T21:46:22.425510+0000","last_unstale":"2024-09-15T21:46:40.483066+0000","last_undegraded":"2024-09-15T21:46:40.483066+0000","last_fullsized":"2024-09-15T21:46:40.483066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:11:00.338076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808338+0000","last_change":"2024-09-15T21:46:23.427051+0000","last_active":"2024-09-15T21:46:24.808338+0000","last_peered":"2024-09-15T21:46:24.808338+0000","last_clean":"2024-09-15T21:46:24.808338+0000","last_became_active":"2024-09-15T21:46:23.426185+0000","last_became_peered":"2024-09-15T21:46:23.426185+0000","last_unstale":"2024-09-15T21:46:24.808338+0000","last_undegraded":"2024-09-15T21:46:24.808338+0000","last_fullsized":"2024-09-15T21:46:24.808338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:48.318405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.572233+0000","last_change":"2024-09-15T21:46:22.424645+0000","last_active":"2024-09-15T21:46:51.572233+0000","last_peered":"2024-09-15T21:46:51.572233+0000","last_clean":"2024-09-15T21:46:51.572233+0000","last_became_active":"2024-09-15T21:46:22.421229+0000","last_became_peered":"2024-09-15T21:46:22.421229+0000","last_unstale":"2024-09-15T21:46:51.572233+0000","last_undegraded":"2024-09-15T21:46:51.572233+0000","last_fullsized":"2024-09-15T21:46:51.572233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:49.165670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414000+0000","last_change":"2024-09-15T21:46:23.425699+0000","last_active":"2024-09-15T21:46:24.414000+0000","last_peered":"2024-09-15T21:46:24.414000+0000","last_clean":"2024-09-15T21:46:24.414000+0000","last_became_active":"2024-09-15T21:46:23.425416+0000","last_became_peered":"2024-09-15T21:46:23.425416+0000","last_unstale":"2024-09-15T21:46:24.414000+0000","last_undegraded":"2024-09-15T21:46:24.414000+0000","last_fullsized":"2024-09-15T21:46:24.414000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:00.325179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808477+0000","last_change":"2024-09-15T21:46:22.426492+0000","last_active":"2024-09-15T21:46:24.808477+0000","last_peered":"2024-09-15T21:46:24.808477+0000","last_clean":"2024-09-15T21:46:24.808477+0000","last_became_active":"2024-09-15T21:46:22.426210+0000","last_became_peered":"2024-09-15T21:46:22.426210+0000","last_unstale":"2024-09-15T21:46:24.808477+0000","last_undegraded":"2024-09-15T21:46:24.808477+0000","last_fullsized":"2024-09-15T21:46:24.808477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:05.248091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414861+0000","last_change":"2024-09-15T21:46:23.429837+0000","last_active":"2024-09-15T21:46:24.414861+0000","last_peered":"2024-09-15T21:46:24.414861+0000","last_clean":"2024-09-15T21:46:24.414861+0000","last_became_active":"2024-09-15T21:46:23.429254+0000","last_became_peered":"2024-09-15T21:46:23.429254+0000","last_unstale":"2024-09-15T21:46:24.414861+0000","last_undegraded":"2024-09-15T21:46:24.414861+0000","last_fullsized":"2024-09-15T21:46:24.414861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:58.465583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.551360+0000","last_change":"2024-09-15T21:46:22.430710+0000","last_active":"2024-09-15T21:46:45.551360+0000","last_peered":"2024-09-15T21:46:45.551360+0000","last_clean":"2024-09-15T21:46:45.551360+0000","last_became_active":"2024-09-15T21:46:22.430375+0000","last_became_peered":"2024-09-15T21:46:22.430375+0000","last_unstale":"2024-09-15T21:46:45.551360+0000","last_undegraded":"2024-09-15T21:46:45.551360+0000","last_fullsized":"2024-09-15T21:46:45.551360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:04.081179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808447+0000","last_change":"2024-09-15T21:46:23.426729+0000","last_active":"2024-09-15T21:46:24.808447+0000","last_peered":"2024-09-15T21:46:24.808447+0000","last_clean":"2024-09-15T21:46:24.808447+0000","last_became_active":"2024-09-15T21:46:23.426495+0000","last_became_peered":"2024-09-15T21:46:23.426495+0000","last_unstale":"2024-09-15T21:46:24.808447+0000","last_undegraded":"2024-09-15T21:46:24.808447+0000","last_fullsized":"2024-09-15T21:46:24.808447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:03.277997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'7","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.549857+0000","last_change":"2024-09-15T21:46:22.427137+0000","last_active":"2024-09-15T21:46:45.549857+0000","last_peered":"2024-09-15T21:46:45.549857+0000","last_clean":"2024-09-15T21:46:45.549857+0000","last_became_active":"2024-09-15T21:46:22.426855+0000","last_became_peered":"2024-09-15T21:46:22.426855+0000","last_unstale":"2024-09-15T21:46:45.549857+0000","last_undegraded":"2024-09-15T21:46:45.549857+0000","last_fullsized":"2024-09-15T21:46:45.549857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:40:14.277940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834714+0000","last_change":"2024-09-15T21:46:23.427327+0000","last_active":"2024-09-15T21:46:24.834714+0000","last_peered":"2024-09-15T21:46:24.834714+0000","last_clean":"2024-09-15T21:46:24.834714+0000","last_became_active":"2024-09-15T21:46:23.427096+0000","last_became_peered":"2024-09-15T21:46:23.427096+0000","last_unstale":"2024-09-15T21:46:24.834714+0000","last_undegraded":"2024-09-15T21:46:24.834714+0000","last_fullsized":"2024-09-15T21:46:24.834714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:32.370390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'7","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.570649+0000","last_change":"2024-09-15T21:46:22.427798+0000","last_active":"2024-09-15T21:46:51.570649+0000","last_peered":"2024-09-15T21:46:51.570649+0000","last_clean":"2024-09-15T21:46:51.570649+0000","last_became_active":"2024-09-15T21:46:22.427537+0000","last_became_peered":"2024-09-15T21:46:22.427537+0000","last_unstale":"2024-09-15T21:46:51.570649+0000","last_undegraded":"2024-09-15T21:46:51.570649+0000","last_fullsized":"2024-09-15T21:46:51.570649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:18:54.963834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727157+0000","last_change":"2024-09-15T21:46:23.424266+0000","last_active":"2024-09-15T21:46:24.727157+0000","last_peered":"2024-09-15T21:46:24.727157+0000","last_clean":"2024-09-15T21:46:24.727157+0000","last_became_active":"2024-09-15T21:46:23.424007+0000","last_became_peered":"2024-09-15T21:46:23.424007+0000","last_unstale":"2024-09-15T21:46:24.727157+0000","last_undegraded":"2024-09-15T21:46:24.727157+0000","last_fullsized":"2024-09-15T21:46:24.727157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:23.492253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:34.461771+0000","last_change":"2024-09-15T21:46:22.430901+0000","last_active":"2024-09-15T21:46:34.461771+0000","last_peered":"2024-09-15T21:46:34.461771+0000","last_clean":"2024-09-15T21:46:34.461771+0000","last_became_active":"2024-09-15T21:46:22.430452+0000","last_became_peered":"2024-09-15T21:46:22.430452+0000","last_unstale":"2024-09-15T21:46:34.461771+0000","last_undegraded":"2024-09-15T21:46:34.461771+0000","last_fullsized":"2024-09-15T21:46:34.461771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:59:45.663976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.413661+0000","last_change":"2024-09-15T21:46:23.430836+0000","last_active":"2024-09-15T21:46:24.413661+0000","last_peered":"2024-09-15T21:46:24.413661+0000","last_clean":"2024-09-15T21:46:24.413661+0000","last_became_active":"2024-09-15T21:46:23.429283+0000","last_became_peered":"2024-09-15T21:46:23.429283+0000","last_unstale":"2024-09-15T21:46:24.413661+0000","last_undegraded":"2024-09-15T21:46:24.413661+0000","last_fullsized":"2024-09-15T21:46:24.413661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:30.571376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'7","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.569263+0000","last_change":"2024-09-15T21:46:22.423795+0000","last_active":"2024-09-15T21:46:51.569263+0000","last_peered":"2024-09-15T21:46:51.569263+0000","last_clean":"2024-09-15T21:46:51.569263+0000","last_became_active":"2024-09-15T21:46:22.422327+0000","last_became_peered":"2024-09-15T21:46:22.422327+0000","last_unstale":"2024-09-15T21:46:51.569263+0000","last_undegraded":"2024-09-15T21:46:51.569263+0000","last_fullsized":"2024-09-15T21:46:51.569263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:16.670599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834522+0000","last_change":"2024-09-15T21:46:23.428634+0000","last_active":"2024-09-15T21:46:24.834522+0000","last_peered":"2024-09-15T21:46:24.834522+0000","last_clean":"2024-09-15T21:46:24.834522+0000","last_became_active":"2024-09-15T21:46:23.428301+0000","last_became_peered":"2024-09-15T21:46:23.428301+0000","last_unstale":"2024-09-15T21:46:24.834522+0000","last_undegraded":"2024-09-15T21:46:24.834522+0000","last_fullsized":"2024-09-15T21:46:24.834522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:59.492942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.547973+0000","last_change":"2024-09-15T21:46:22.425810+0000","last_active":"2024-09-15T21:46:45.547973+0000","last_peered":"2024-09-15T21:46:45.547973+0000","last_clean":"2024-09-15T21:46:45.547973+0000","last_became_active":"2024-09-15T21:46:22.425478+0000","last_became_peered":"2024-09-15T21:46:22.425478+0000","last_unstale":"2024-09-15T21:46:45.547973+0000","last_undegraded":"2024-09-15T21:46:45.547973+0000","last_fullsized":"2024-09-15T21:46:45.547973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:53.296599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834743+0000","last_change":"2024-09-15T21:46:23.426054+0000","last_active":"2024-09-15T21:46:24.834743+0000","last_peered":"2024-09-15T21:46:24.834743+0000","last_clean":"2024-09-15T21:46:24.834743+0000","last_became_active":"2024-09-15T21:46:23.425547+0000","last_became_peered":"2024-09-15T21:46:23.425547+0000","last_unstale":"2024-09-15T21:46:24.834743+0000","last_undegraded":"2024-09-15T21:46:24.834743+0000","last_fullsized":"2024-09-15T21:46:24.834743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:56.838503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:34.458317+0000","last_change":"2024-09-15T21:46:22.428524+0000","last_active":"2024-09-15T21:46:34.458317+0000","last_peered":"2024-09-15T21:46:34.458317+0000","last_clean":"2024-09-15T21:46:34.458317+0000","last_became_active":"2024-09-15T21:46:22.428043+0000","last_became_peered":"2024-09-15T21:46:22.428043+0000","last_unstale":"2024-09-15T21:46:34.458317+0000","last_undegraded":"2024-09-15T21:46:34.458317+0000","last_fullsized":"2024-09-15T21:46:34.458317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:08.393381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834850+0000","last_change":"2024-09-15T21:46:23.434719+0000","last_active":"2024-09-15T21:46:24.834850+0000","last_peered":"2024-09-15T21:46:24.834850+0000","last_clean":"2024-09-15T21:46:24.834850+0000","last_became_active":"2024-09-15T21:46:23.434448+0000","last_became_peered":"2024-09-15T21:46:23.434448+0000","last_unstale":"2024-09-15T21:46:24.834850+0000","last_undegraded":"2024-09-15T21:46:24.834850+0000","last_fullsized":"2024-09-15T21:46:24.834850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:14.901671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414691+0000","last_change":"2024-09-15T21:46:23.430058+0000","last_active":"2024-09-15T21:46:24.414691+0000","last_peered":"2024-09-15T21:46:24.414691+0000","last_clean":"2024-09-15T21:46:24.414691+0000","last_became_active":"2024-09-15T21:46:23.429598+0000","last_became_peered":"2024-09-15T21:46:23.429598+0000","last_unstale":"2024-09-15T21:46:24.414691+0000","last_undegraded":"2024-09-15T21:46:24.414691+0000","last_fullsized":"2024-09-15T21:46:24.414691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:44.142207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.570689+0000","last_change":"2024-09-15T21:46:22.431172+0000","last_active":"2024-09-15T21:46:51.570689+0000","last_peered":"2024-09-15T21:46:51.570689+0000","last_clean":"2024-09-15T21:46:51.570689+0000","last_became_active":"2024-09-15T21:46:22.430800+0000","last_became_peered":"2024-09-15T21:46:22.430800+0000","last_unstale":"2024-09-15T21:46:51.570689+0000","last_undegraded":"2024-09-15T21:46:51.570689+0000","last_fullsized":"2024-09-15T21:46:51.570689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:55.072466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414264+0000","last_change":"2024-09-15T21:46:23.434353+0000","last_active":"2024-09-15T21:46:24.414264+0000","last_peered":"2024-09-15T21:46:24.414264+0000","last_clean":"2024-09-15T21:46:24.414264+0000","last_became_active":"2024-09-15T21:46:23.434097+0000","last_became_peered":"2024-09-15T21:46:23.434097+0000","last_unstale":"2024-09-15T21:46:24.414264+0000","last_undegraded":"2024-09-15T21:46:24.414264+0000","last_fullsized":"2024-09-15T21:46:24.414264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:46.007227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'12","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.571743+0000","last_change":"2024-09-15T21:46:22.429870+0000","last_active":"2024-09-15T21:46:51.571743+0000","last_peered":"2024-09-15T21:46:51.571743+0000","last_clean":"2024-09-15T21:46:51.571743+0000","last_became_active":"2024-09-15T21:46:22.429578+0000","last_became_peered":"2024-09-15T21:46:22.429578+0000","last_unstale":"2024-09-15T21:46:51.571743+0000","last_undegraded":"2024-09-15T21:46:51.571743+0000","last_fullsized":"2024-09-15T21:46:51.571743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:41.678078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727142+0000","last_change":"2024-09-15T21:46:23.429604+0000","last_active":"2024-09-15T21:46:24.727142+0000","last_peered":"2024-09-15T21:46:24.727142+0000","last_clean":"2024-09-15T21:46:24.727142+0000","last_became_active":"2024-09-15T21:46:23.429355+0000","last_became_peered":"2024-09-15T21:46:23.429355+0000","last_unstale":"2024-09-15T21:46:24.727142+0000","last_undegraded":"2024-09-15T21:46:24.727142+0000","last_fullsized":"2024-09-15T21:46:24.727142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:45.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.416728+0000","last_change":"2024-09-15T21:46:22.428529+0000","last_active":"2024-09-15T21:46:28.416728+0000","last_peered":"2024-09-15T21:46:28.416728+0000","last_clean":"2024-09-15T21:46:28.416728+0000","last_became_active":"2024-09-15T21:46:22.428263+0000","last_became_peered":"2024-09-15T21:46:22.428263+0000","last_unstale":"2024-09-15T21:46:28.416728+0000","last_undegraded":"2024-09-15T21:46:28.416728+0000","last_fullsized":"2024-09-15T21:46:28.416728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:22:40.160062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834701+0000","last_change":"2024-09-15T21:46:23.427968+0000","last_active":"2024-09-15T21:46:24.834701+0000","last_peered":"2024-09-15T21:46:24.834701+0000","last_clean":"2024-09-15T21:46:24.834701+0000","last_became_active":"2024-09-15T21:46:23.427546+0000","last_became_peered":"2024-09-15T21:46:23.427546+0000","last_unstale":"2024-09-15T21:46:24.834701+0000","last_undegraded":"2024-09-15T21:46:24.834701+0000","last_fullsized":"2024-09-15T21:46:24.834701+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:47.226097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'13","reported_seq":27,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.569304+0000","last_change":"2024-09-15T21:46:22.428258+0000","last_active":"2024-09-15T21:46:51.569304+0000","last_peered":"2024-09-15T21:46:51.569304+0000","last_clean":"2024-09-15T21:46:51.569304+0000","last_became_active":"2024-09-15T21:46:22.427954+0000","last_became_peered":"2024-09-15T21:46:22.427954+0000","last_unstale":"2024-09-15T21:46:51.569304+0000","last_undegraded":"2024-09-15T21:46:51.569304+0000","last_fullsized":"2024-09-15T21:46:51.569304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:22.440456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8034,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":205,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":16068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":179,"ondisk_log_size":179,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476748,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":843776,"data_stored":668200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509452,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":669426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":79360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":80890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509453,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":78288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509452,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":78420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542156,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":78490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":79394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:47:01.763 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:47:01.763 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-15T21:47:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.896+0000 7f2dd8511640 1 Processor -- start 2024-09-15T21:47:01.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.896+0000 7f2dd8511640 1 -- start start 2024-09-15T21:47:01.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.897+0000 7f2dd8511640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2dd00cc8f0 0x7f2dd01669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:01.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.897+0000 7f2dd8511640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dd015e7a0 con 0x7f2dd00cb7d0 2024-09-15T21:47:01.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.897+0000 7f2dd8511640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dd015ea40 con 0x7f2dd0155440 2024-09-15T21:47:01.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.897+0000 7f2dd8511640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f2dd015b470 con 0x7f2dd00cc8f0 2024-09-15T21:47:01.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.897+0000 7f2dd6aae640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2dd00cc8f0 0x7f2dd01669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:01.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.897+0000 7f2dd5aac640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f2dd00cb7d0 0x7f2dd00cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:40748/0 (socket says 172.21.15.100:40748) 2024-09-15T21:47:01.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.897+0000 7f2dd5aac640 1 -- 172.21.15.100:0/3167541816 learned_addr learned my addr 172.21.15.100:0/3167541816 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:01.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.897+0000 7f2dd6aae640 1 -- 172.21.15.100:0/3167541816 >> v1:172.21.15.122:6789/0 conn(0x7f2dd0155440 legacy=0x7f2dd00cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:01.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.897+0000 7f2dd6aae640 1 -- 172.21.15.100:0/3167541816 >> v1:172.21.15.100:6789/0 conn(0x7f2dd00cb7d0 legacy=0x7f2dd00cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:01.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.897+0000 7f2dd52ab640 1 -- 172.21.15.100:0/3167541816 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1242752568 0 0) 0x7f2dd015e7a0 con 0x7f2dd00cb7d0 2024-09-15T21:47:01.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.897+0000 7f2dd6aae640 1 -- 172.21.15.100:0/3167541816 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd015ed20 con 0x7f2dd00cc8f0 2024-09-15T21:47:01.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.898+0000 7f2dd6aae640 1 --2- 172.21.15.100:0/3167541816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd00cc8f0 0x7f2dd01669a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009d90 tx=0x7f2dc802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=892d1dce3d4d5fcc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:01.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.898+0000 7f2dd52ab640 1 -- 172.21.15.100:0/3167541816 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8007740 con 0x7f2dd00cc8f0 2024-09-15T21:47:01.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.898+0000 7f2dd52ab640 1 -- 172.21.15.100:0/3167541816 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dc80078e0 con 0x7f2dd00cc8f0 2024-09-15T21:47:01.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.898+0000 7f2dd52ab640 1 -- 172.21.15.100:0/3167541816 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8007cb0 con 0x7f2dd00cc8f0 2024-09-15T21:47:01.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.899+0000 7f2dd8511640 1 -- 172.21.15.100:0/3167541816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd00cc8f0 msgr2=0x7f2dd01669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:01.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.899+0000 7f2dd8511640 1 --2- 172.21.15.100:0/3167541816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd00cc8f0 0x7f2dd01669a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009d90 tx=0x7f2dc802a6a0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.899+0000 7f2dd8511640 1 -- 172.21.15.100:0/3167541816 shutdown_connections 2024-09-15T21:47:01.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.899+0000 7f2dd8511640 1 --2- 172.21.15.100:0/3167541816 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd00cc8f0 0x7f2dd01669a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.899+0000 7f2dd8511640 1 -- 172.21.15.100:0/3167541816 >> 172.21.15.100:0/3167541816 conn(0x7f2dd00680d0 msgr2=0x7f2dd006d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:01.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.899+0000 7f2dd8511640 1 -- 172.21.15.100:0/3167541816 shutdown_connections 2024-09-15T21:47:01.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.899+0000 7f2dd8511640 1 -- 172.21.15.100:0/3167541816 wait complete. 2024-09-15T21:47:01.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.900+0000 7f2dd8511640 1 Processor -- start 2024-09-15T21:47:01.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.900+0000 7f2dd8511640 1 -- start start 2024-09-15T21:47:01.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dd8511640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2dd00cb7d0 0x7f2dd01f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:01.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dd8511640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd00cc8f0 0x7f2dd01f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:01.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dd8511640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd0155440 0x7f2dd0140330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:01.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dd8511640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd015f2e0 con 0x7f2dd00cc8f0 2024-09-15T21:47:01.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dd8511640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0155820 con 0x7f2dd0155440 2024-09-15T21:47:01.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dd8511640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2dd015b070 con 0x7f2dd00cb7d0 2024-09-15T21:47:01.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dd5aac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd00cc8f0 0x7f2dd01f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:01.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dd5aac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd00cc8f0 0x7f2dd01f3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48904/0 (socket says 172.21.15.100:48904) 2024-09-15T21:47:01.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dd5aac640 1 -- 172.21.15.100:0/2481811212 learned_addr learned my addr 172.21.15.100:0/2481811212 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:01.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dd5aac640 1 -- 172.21.15.100:0/2481811212 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2dd00cb7d0 msgr2=0x7f2dd01f3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:01.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dd5aac640 1 --2- 172.21.15.100:0/2481811212 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2dd00cb7d0 0x7f2dd01f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dd5aac640 1 -- 172.21.15.100:0/2481811212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd0155440 msgr2=0x7f2dd0140330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:01.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dd5aac640 1 --2- 172.21.15.100:0/2481811212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd0155440 0x7f2dd0140330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:01.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dd5aac640 1 -- 172.21.15.100:0/2481811212 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd01f1900 con 0x7f2dd00cc8f0 2024-09-15T21:47:01.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dd5aac640 1 --2- 172.21.15.100:0/2481811212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd00cc8f0 0x7f2dd01f3790 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000c460 tx=0x7f2dc000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:01.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dc6ffd640 1 -- 172.21.15.100:0/2481811212 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0018070 con 0x7f2dd00cc8f0 2024-09-15T21:47:01.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dc6ffd640 1 -- 172.21.15.100:0/2481811212 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dc0007500 con 0x7f2dd00cc8f0 2024-09-15T21:47:01.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dd8511640 1 -- 172.21.15.100:0/2481811212 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd01f1bc0 con 0x7f2dd00cc8f0 2024-09-15T21:47:01.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dc6ffd640 1 -- 172.21.15.100:0/2481811212 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0004a20 con 0x7f2dd00cc8f0 2024-09-15T21:47:01.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.901+0000 7f2dd8511640 1 -- 172.21.15.100:0/2481811212 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd01f2770 con 0x7f2dd00cc8f0 2024-09-15T21:47:01.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.902+0000 7f2dd8511640 1 -- 172.21.15.100:0/2481811212 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dd00b3940 con 0x7f2dd00cc8f0 2024-09-15T21:47:01.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.903+0000 7f2dc6ffd640 1 -- 172.21.15.100:0/2481811212 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2dc001b070 con 0x7f2dd00cc8f0 2024-09-15T21:47:01.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.903+0000 7f2dc6ffd640 1 --2- 172.21.15.100:0/2481811212 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2db406c2b0 0x7f2db406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:01.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.903+0000 7f2dc6ffd640 1 -- 172.21.15.100:0/2481811212 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2dc008a720 con 0x7f2dd00cc8f0 2024-09-15T21:47:01.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.903+0000 7f2dd62ad640 1 --2- 172.21.15.100:0/2481811212 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2db406c2b0 0x7f2db406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:01.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.904+0000 7f2dd62ad640 1 --2- 172.21.15.100:0/2481811212 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2db406c2b0 0x7f2db406e770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc003cc0 tx=0x7f2dcc0079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:01.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:01.907+0000 7f2dc6ffd640 1 -- 172.21.15.100:0/2481811212 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2dc0058e90 con 0x7f2dd00cc8f0 2024-09-15T21:47:02.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.075+0000 7f2dd8511640 1 -- 172.21.15.100:0/2481811212 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f2dd00caed0 con 0x7f2dd00cc8f0 2024-09-15T21:47:02.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.075+0000 7f2dc6ffd640 1 -- 172.21.15.100:0/2481811212 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v22) ==== 56+0+3267 (secure 0 0 0) 0x7f2dc0058790 con 0x7f2dd00cc8f0 2024-09-15T21:47:02.077 INFO:teuthology.orchestra.run.smithi100.stdout:epoch 22 2024-09-15T21:47:02.077 INFO:teuthology.orchestra.run.smithi100.stdout:fsid 30e2fff4-2171-4531-8806-0e03b63f494f 2024-09-15T21:47:02.077 INFO:teuthology.orchestra.run.smithi100.stdout:created 2024-09-15T21:45:59.746897+0000 2024-09-15T21:47:02.078 INFO:teuthology.orchestra.run.smithi100.stdout:modified 2024-09-15T21:46:24.409383+0000 2024-09-15T21:47:02.078 INFO:teuthology.orchestra.run.smithi100.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-15T21:47:02.078 INFO:teuthology.orchestra.run.smithi100.stdout:crush_version 5 2024-09-15T21:47:02.078 INFO:teuthology.orchestra.run.smithi100.stdout:full_ratio 0.95 2024-09-15T21:47:02.078 INFO:teuthology.orchestra.run.smithi100.stdout:backfillfull_ratio 0.9 2024-09-15T21:47:02.078 INFO:teuthology.orchestra.run.smithi100.stdout:nearfull_ratio 0.85 2024-09-15T21:47:02.078 INFO:teuthology.orchestra.run.smithi100.stdout:require_min_compat_client luminous 2024-09-15T21:47:02.078 INFO:teuthology.orchestra.run.smithi100.stdout:min_compat_client jewel 2024-09-15T21:47:02.078 INFO:teuthology.orchestra.run.smithi100.stdout:require_osd_release squid 2024-09-15T21:47:02.078 INFO:teuthology.orchestra.run.smithi100.stdout:stretch_mode_enabled false 2024-09-15T21:47:02.078 INFO:teuthology.orchestra.run.smithi100.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-15T21:47:02.078 INFO:teuthology.orchestra.run.smithi100.stdout:pool 2 'cephfs_metadata' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.50 2024-09-15T21:47:02.078 INFO:teuthology.orchestra.run.smithi100.stdout:pool 3 'cephfs_data' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-15T21:47:02.079 INFO:teuthology.orchestra.run.smithi100.stdout:max_osd 8 2024-09-15T21:47:02.079 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0 up in weight 1 up_from 16 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] [v2:172.21.15.100:6821/3920435546,v1:172.21.15.100:6822/3920435546] exists,up 5887900b-522d-476f-9704-cc2c5ee87a60 2024-09-15T21:47:02.079 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] [v2:172.21.15.100:6802/3802679972,v1:172.21.15.100:6803/3802679972] exists,up adc17993-5c50-4251-85e1-4c7ac62e25e6 2024-09-15T21:47:02.079 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] [v2:172.21.15.100:6824/2316787974,v1:172.21.15.100:6826/2316787974] exists,up 8de3b7d5-1980-4a3b-a934-264f867c7c8d 2024-09-15T21:47:02.079 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3 up in weight 1 up_from 13 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] [v2:172.21.15.100:6812/1287562741,v1:172.21.15.100:6813/1287562741] exists,up 93dbcd1c-6a6f-49cf-bc94-e64b883de5f6 2024-09-15T21:47:02.079 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] [v2:172.21.15.122:6802/4045445512,v1:172.21.15.122:6803/4045445512] exists,up 7caee2a0-bab4-45a9-8109-724948c863c6 2024-09-15T21:47:02.079 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] [v2:172.21.15.122:6826/2400668532,v1:172.21.15.122:6827/2400668532] exists,up 6e64070a-671a-444b-a061-b3051a29b121 2024-09-15T21:47:02.079 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] [v2:172.21.15.122:6810/1437167311,v1:172.21.15.122:6811/1437167311] exists,up 47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3 2024-09-15T21:47:02.079 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] [v2:172.21.15.122:6818/2431377162,v1:172.21.15.122:6819/2431377162] exists,up 9a916b6d-af82-486f-af40-4a7291c900fe 2024-09-15T21:47:02.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.077+0000 7f2dd8511640 1 -- 172.21.15.100:0/2481811212 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2db406c2b0 msgr2=0x7f2db406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.077+0000 7f2dd8511640 1 --2- 172.21.15.100:0/2481811212 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2db406c2b0 0x7f2db406e770 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc003cc0 tx=0x7f2dcc0079b0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.077+0000 7f2dd8511640 1 -- 172.21.15.100:0/2481811212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd00cc8f0 msgr2=0x7f2dd01f3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.077+0000 7f2dd8511640 1 --2- 172.21.15.100:0/2481811212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd00cc8f0 0x7f2dd01f3790 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000c460 tx=0x7f2dc000c920 comp rx=0 tx=0).stop 2024-09-15T21:47:02.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.077+0000 7f2dd8511640 1 -- 172.21.15.100:0/2481811212 shutdown_connections 2024-09-15T21:47:02.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.077+0000 7f2dd8511640 1 --2- 172.21.15.100:0/2481811212 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2db406c2b0 0x7f2db406e770 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.077+0000 7f2dd8511640 1 --2- 172.21.15.100:0/2481811212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd0155440 0x7f2dd0140330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.077+0000 7f2dd8511640 1 --2- 172.21.15.100:0/2481811212 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2dd00cc8f0 0x7f2dd01f3790 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.077+0000 7f2dd8511640 1 --2- 172.21.15.100:0/2481811212 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2dd00cb7d0 0x7f2dd01f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.077+0000 7f2dd8511640 1 -- 172.21.15.100:0/2481811212 >> 172.21.15.100:0/2481811212 conn(0x7f2dd00680d0 msgr2=0x7f2dd006bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:02.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.077+0000 7f2dd8511640 1 -- 172.21.15.100:0/2481811212 shutdown_connections 2024-09-15T21:47:02.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.077+0000 7f2dd8511640 1 -- 172.21.15.100:0/2481811212 wait complete. 2024-09-15T21:47:02.090 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 16 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] [v2:172.21.15.100:6821/3920435546,v1:172.21.15.100:6822/3920435546] exists,up 5887900b-522d-476f-9704-cc2c5ee87a60', 'osd.1 up in weight 1 up_from 14 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] [v2:172.21.15.100:6802/3802679972,v1:172.21.15.100:6803/3802679972] exists,up adc17993-5c50-4251-85e1-4c7ac62e25e6', 'osd.2 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] [v2:172.21.15.100:6824/2316787974,v1:172.21.15.100:6826/2316787974] exists,up 8de3b7d5-1980-4a3b-a934-264f867c7c8d', 'osd.3 up in weight 1 up_from 13 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] [v2:172.21.15.100:6812/1287562741,v1:172.21.15.100:6813/1287562741] exists,up 93dbcd1c-6a6f-49cf-bc94-e64b883de5f6', 'osd.4 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] [v2:172.21.15.122:6802/4045445512,v1:172.21.15.122:6803/4045445512] exists,up 7caee2a0-bab4-45a9-8109-724948c863c6', 'osd.5 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] [v2:172.21.15.122:6826/2400668532,v1:172.21.15.122:6827/2400668532] exists,up 6e64070a-671a-444b-a061-b3051a29b121', 'osd.6 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] [v2:172.21.15.122:6810/1437167311,v1:172.21.15.122:6811/1437167311] exists,up 47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3', 'osd.7 up in weight 1 up_from 15 up_thru 20 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] [v2:172.21.15.122:6818/2431377162,v1:172.21.15.122:6819/2431377162] exists,up 9a916b6d-af82-486f-af40-4a7291c900fe'] 2024-09-15T21:47:02.091 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-15T21:47:02.189 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:02.189 INFO:teuthology.orchestra.run.smithi100.stdout: "name": "mon.a", 2024-09-15T21:47:02.189 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster": "ceph", 2024-09-15T21:47:02.189 INFO:teuthology.orchestra.run.smithi100.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-15T21:47:02.190 INFO:teuthology.orchestra.run.smithi100.stdout: "admin_socket_mode": "", 2024-09-15T21:47:02.190 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-15T21:47:02.190 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_client_required": "cephx, none", 2024-09-15T21:47:02.190 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_cluster_required": "cephx", 2024-09-15T21:47:02.190 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_debug": "true", 2024-09-15T21:47:02.190 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-15T21:47:02.190 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-15T21:47:02.190 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_service_required": "cephx", 2024-09-15T21:47:02.190 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-15T21:47:02.190 INFO:teuthology.orchestra.run.smithi100.stdout: "auth_supported": "cephx", 2024-09-15T21:47:02.190 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_aio": "true", 2024-09-15T21:47:02.190 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-15T21:47:02.190 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_aio_poll_ms": "250", 2024-09-15T21:47:02.191 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_aio_reap_max": "16", 2024-09-15T21:47:02.191 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_async_discard": "false", 2024-09-15T21:47:02.191 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_block_size": "4096", 2024-09-15T21:47:02.191 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_debug_aio": "false", 2024-09-15T21:47:02.191 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-15T21:47:02.191 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-15T21:47:02.191 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_debug_inflight_ios": "false", 2024-09-15T21:47:02.191 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_enable_discard": "false", 2024-09-15T21:47:02.191 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_flock_retry": "3", 2024-09-15T21:47:02.191 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-15T21:47:02.191 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_inject_crash": "0", 2024-09-15T21:47:02.191 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-15T21:47:02.191 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_ioring": "false", 2024-09-15T21:47:02.191 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_ioring_hipri": "false", 2024-09-15T21:47:02.192 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-15T21:47:02.192 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-15T21:47:02.192 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-15T21:47:02.192 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-15T21:47:02.192 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-15T21:47:02.192 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-15T21:47:02.192 INFO:teuthology.orchestra.run.smithi100.stdout: "bdev_type": "", 2024-09-15T21:47:02.192 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_alloc_size": "1048576", 2024-09-15T21:47:02.192 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_allocator": "hybrid", 2024-09-15T21:47:02.192 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_buffered_io": "true", 2024-09-15T21:47:02.192 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_check_for_zeros": "false", 2024-09-15T21:47:02.192 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-15T21:47:02.193 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-15T21:47:02.193 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_compact_log_sync": "false", 2024-09-15T21:47:02.193 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-15T21:47:02.193 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-15T21:47:02.193 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-15T21:47:02.193 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-15T21:47:02.193 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_max_log_runway": "4194304", 2024-09-15T21:47:02.193 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_max_prefetch": "1048576", 2024-09-15T21:47:02.193 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_min_flush_size": "524288", 2024-09-15T21:47:02.193 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_min_log_runway": "1048576", 2024-09-15T21:47:02.193 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_replay_recovery": "false", 2024-09-15T21:47:02.193 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-15T21:47:02.193 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-15T21:47:02.193 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_sync_write": "false", 2024-09-15T21:47:02.194 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-15T21:47:02.194 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-15T21:47:02.194 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-15T21:47:02.194 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_allocation_from_file": "true", 2024-09-15T21:47:02.194 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_allocator": "hybrid", 2024-09-15T21:47:02.194 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-15T21:47:02.194 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-15T21:47:02.194 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-15T21:47:02.194 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-15T21:47:02.194 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_bdev_label_multi": "true", 2024-09-15T21:47:02.194 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-15T21:47:02.194 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-15T21:47:02.194 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-15T21:47:02.194 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-15T21:47:02.194 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-15T21:47:02.195 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_create": "true", 2024-09-15T21:47:02.195 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_db_create": "false", 2024-09-15T21:47:02.195 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_db_path": "", 2024-09-15T21:47:02.195 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_db_size": "0", 2024-09-15T21:47:02.195 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_path": "", 2024-09-15T21:47:02.195 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_preallocate_file": "false", 2024-09-15T21:47:02.195 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_size": "107374182400", 2024-09-15T21:47:02.195 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_wal_create": "false", 2024-09-15T21:47:02.195 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_wal_path": "", 2024-09-15T21:47:02.195 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_block_wal_size": "100663296", 2024-09-15T21:47:02.195 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_bluefs": "true", 2024-09-15T21:47:02.195 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-15T21:47:02.195 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-15T21:47:02.195 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-15T21:47:02.196 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-15T21:47:02.196 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T21:47:02.196 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T21:47:02.196 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-15T21:47:02.196 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T21:47:02.196 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_autotune": "true", 2024-09-15T21:47:02.196 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-15T21:47:02.196 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-15T21:47:02.196 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-15T21:47:02.196 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-15T21:47:02.196 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_size": "0", 2024-09-15T21:47:02.196 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-15T21:47:02.196 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-15T21:47:02.196 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-15T21:47:02.197 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-15T21:47:02.197 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_type": "2q", 2024-09-15T21:47:02.197 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-15T21:47:02.197 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_clone_cow": "true", 2024-09-15T21:47:02.197 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-15T21:47:02.197 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-15T21:47:02.197 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-15T21:47:02.197 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-15T21:47:02.197 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-15T21:47:02.197 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-15T21:47:02.197 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-15T21:47:02.197 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_compression_mode": "none", 2024-09-15T21:47:02.197 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-15T21:47:02.198 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_csum_type": "crc32c", 2024-09-15T21:47:02.198 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-15T21:47:02.198 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_freelist": "false", 2024-09-15T21:47:02.198 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-15T21:47:02.198 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-15T21:47:02.198 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-15T21:47:02.198 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-15T21:47:02.198 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-15T21:47:02.198 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-15T21:47:02.198 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-15T21:47:02.198 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-15T21:47:02.198 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-15T21:47:02.198 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-15T21:47:02.198 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-15T21:47:02.198 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-15T21:47:02.198 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_small_allocations": "0", 2024-09-15T21:47:02.199 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-15T21:47:02.199 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_default_buffered_read": "true", 2024-09-15T21:47:02.199 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_default_buffered_write": "false", 2024-09-15T21:47:02.199 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-15T21:47:02.199 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-15T21:47:02.199 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-15T21:47:02.199 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-15T21:47:02.199 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-15T21:47:02.199 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-15T21:47:02.199 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-15T21:47:02.199 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-15T21:47:02.199 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-15T21:47:02.199 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fail_eio": "false", 2024-09-15T21:47:02.200 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-15T21:47:02.200 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-15T21:47:02.200 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-15T21:47:02.200 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-15T21:47:02.200 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-15T21:47:02.200 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-15T21:47:02.200 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_on_mount": "false", 2024-09-15T21:47:02.200 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-15T21:47:02.200 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_on_umount": "false", 2024-09-15T21:47:02.200 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-15T21:47:02.200 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-15T21:47:02.200 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-15T21:47:02.200 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-15T21:47:02.200 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-15T21:47:02.201 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-15T21:47:02.201 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-15T21:47:02.201 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-15T21:47:02.201 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_ignore_data_csum": "false", 2024-09-15T21:47:02.201 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-15T21:47:02.201 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-15T21:47:02.201 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-15T21:47:02.201 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-15T21:47:02.201 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_log_op_age": "5.000000", 2024-09-15T21:47:02.201 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_max_alloc_size": "0", 2024-09-15T21:47:02.201 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_max_blob_size": "0", 2024-09-15T21:47:02.201 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-15T21:47:02.201 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-15T21:47:02.201 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-15T21:47:02.202 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_max_deferred_txc": "32", 2024-09-15T21:47:02.202 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_min_alloc_size": "0", 2024-09-15T21:47:02.202 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-15T21:47:02.202 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-15T21:47:02.202 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_nid_prealloc": "1024", 2024-09-15T21:47:02.202 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-15T21:47:02.202 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-15T21:47:02.202 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-15T21:47:02.202 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-15T21:47:02.202 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_retry_disk_reads": "3", 2024-09-15T21:47:02.202 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_rocksdb_cf": "true", 2024-09-15T21:47:02.202 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-15T21:47:02.203 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-15T21:47:02.203 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-15T21:47:02.203 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-15T21:47:02.203 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-15T21:47:02.203 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-15T21:47:02.203 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-15T21:47:02.203 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-15T21:47:02.203 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_spdk_mem": "512", 2024-09-15T21:47:02.203 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-15T21:47:02.203 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-15T21:47:02.203 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-15T21:47:02.203 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-15T21:47:02.203 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-15T21:47:02.204 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-15T21:47:02.204 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-15T21:47:02.204 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_tracing": "false", 2024-09-15T21:47:02.204 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-15T21:47:02.204 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-15T21:47:02.204 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-15T21:47:02.204 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-15T21:47:02.204 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-15T21:47:02.204 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-15T21:47:02.204 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-15T21:47:02.204 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-15T21:47:02.204 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-15T21:47:02.204 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_zero_block_detection": "false", 2024-09-15T21:47:02.204 INFO:teuthology.orchestra.run.smithi100.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-15T21:47:02.205 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-15T21:47:02.205 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-15T21:47:02.205 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-15T21:47:02.205 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-15T21:47:02.205 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-15T21:47:02.205 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-15T21:47:02.205 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-15T21:47:02.205 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-15T21:47:02.205 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-15T21:47:02.205 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-15T21:47:02.205 INFO:teuthology.orchestra.run.smithi100.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-15T21:47:02.205 INFO:teuthology.orchestra.run.smithi100.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-15T21:47:02.206 INFO:teuthology.orchestra.run.smithi100.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-15T21:47:02.206 INFO:teuthology.orchestra.run.smithi100.stdout: "cephx_cluster_require_signatures": "false", 2024-09-15T21:47:02.206 INFO:teuthology.orchestra.run.smithi100.stdout: "cephx_cluster_require_version": "2", 2024-09-15T21:47:02.206 INFO:teuthology.orchestra.run.smithi100.stdout: "cephx_require_signatures": "false", 2024-09-15T21:47:02.206 INFO:teuthology.orchestra.run.smithi100.stdout: "cephx_require_version": "2", 2024-09-15T21:47:02.206 INFO:teuthology.orchestra.run.smithi100.stdout: "cephx_service_require_signatures": "false", 2024-09-15T21:47:02.206 INFO:teuthology.orchestra.run.smithi100.stdout: "cephx_service_require_version": "2", 2024-09-15T21:47:02.206 INFO:teuthology.orchestra.run.smithi100.stdout: "cephx_sign_messages": "true", 2024-09-15T21:47:02.206 INFO:teuthology.orchestra.run.smithi100.stdout: "chdir": "", 2024-09-15T21:47:02.206 INFO:teuthology.orchestra.run.smithi100.stdout: "client_acl_type": "", 2024-09-15T21:47:02.206 INFO:teuthology.orchestra.run.smithi100.stdout: "client_asio_thread_count": "2", 2024-09-15T21:47:02.206 INFO:teuthology.orchestra.run.smithi100.stdout: "client_cache_mid": "0.750000", 2024-09-15T21:47:02.206 INFO:teuthology.orchestra.run.smithi100.stdout: "client_cache_size": "16384", 2024-09-15T21:47:02.207 INFO:teuthology.orchestra.run.smithi100.stdout: "client_caps_release_delay": "5", 2024-09-15T21:47:02.207 INFO:teuthology.orchestra.run.smithi100.stdout: "client_check_pool_perm": "true", 2024-09-15T21:47:02.207 INFO:teuthology.orchestra.run.smithi100.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-15T21:47:02.207 INFO:teuthology.orchestra.run.smithi100.stdout: "client_debug_force_sync_read": "false", 2024-09-15T21:47:02.207 INFO:teuthology.orchestra.run.smithi100.stdout: "client_debug_getattr_caps": "false", 2024-09-15T21:47:02.207 INFO:teuthology.orchestra.run.smithi100.stdout: "client_debug_inject_features": "", 2024-09-15T21:47:02.207 INFO:teuthology.orchestra.run.smithi100.stdout: "client_debug_inject_tick_delay": "0", 2024-09-15T21:47:02.207 INFO:teuthology.orchestra.run.smithi100.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-15T21:47:02.207 INFO:teuthology.orchestra.run.smithi100.stdout: "client_die_on_failed_remount": "false", 2024-09-15T21:47:02.207 INFO:teuthology.orchestra.run.smithi100.stdout: "client_dirsize_rbytes": "true", 2024-09-15T21:47:02.207 INFO:teuthology.orchestra.run.smithi100.stdout: "client_force_lazyio": "false", 2024-09-15T21:47:02.207 INFO:teuthology.orchestra.run.smithi100.stdout: "client_fs": "", 2024-09-15T21:47:02.207 INFO:teuthology.orchestra.run.smithi100.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-15T21:47:02.208 INFO:teuthology.orchestra.run.smithi100.stdout: "client_inject_release_failure": "false", 2024-09-15T21:47:02.208 INFO:teuthology.orchestra.run.smithi100.stdout: "client_max_inline_size": "4096", 2024-09-15T21:47:02.208 INFO:teuthology.orchestra.run.smithi100.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-15T21:47:02.208 INFO:teuthology.orchestra.run.smithi100.stdout: "client_mds_namespace": "", 2024-09-15T21:47:02.208 INFO:teuthology.orchestra.run.smithi100.stdout: "client_metadata": "", 2024-09-15T21:47:02.208 INFO:teuthology.orchestra.run.smithi100.stdout: "client_mount_gid": "-1", 2024-09-15T21:47:02.208 INFO:teuthology.orchestra.run.smithi100.stdout: "client_mount_timeout": "300", 2024-09-15T21:47:02.208 INFO:teuthology.orchestra.run.smithi100.stdout: "client_mount_uid": "-1", 2024-09-15T21:47:02.208 INFO:teuthology.orchestra.run.smithi100.stdout: "client_mountpoint": "/", 2024-09-15T21:47:02.208 INFO:teuthology.orchestra.run.smithi100.stdout: "client_notify_timeout": "10", 2024-09-15T21:47:02.208 INFO:teuthology.orchestra.run.smithi100.stdout: "client_oc": "true", 2024-09-15T21:47:02.208 INFO:teuthology.orchestra.run.smithi100.stdout: "client_oc_max_dirty": "104857600", 2024-09-15T21:47:02.208 INFO:teuthology.orchestra.run.smithi100.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-15T21:47:02.208 INFO:teuthology.orchestra.run.smithi100.stdout: "client_oc_max_objects": "1000", 2024-09-15T21:47:02.209 INFO:teuthology.orchestra.run.smithi100.stdout: "client_oc_size": "209715200", 2024-09-15T21:47:02.209 INFO:teuthology.orchestra.run.smithi100.stdout: "client_oc_target_dirty": "8388608", 2024-09-15T21:47:02.209 INFO:teuthology.orchestra.run.smithi100.stdout: "client_permissions": "true", 2024-09-15T21:47:02.209 INFO:teuthology.orchestra.run.smithi100.stdout: "client_quota": "true", 2024-09-15T21:47:02.209 INFO:teuthology.orchestra.run.smithi100.stdout: "client_quota_df": "true", 2024-09-15T21:47:02.209 INFO:teuthology.orchestra.run.smithi100.stdout: "client_readahead_max_bytes": "0", 2024-09-15T21:47:02.209 INFO:teuthology.orchestra.run.smithi100.stdout: "client_readahead_max_periods": "4", 2024-09-15T21:47:02.209 INFO:teuthology.orchestra.run.smithi100.stdout: "client_readahead_min": "131072", 2024-09-15T21:47:02.209 INFO:teuthology.orchestra.run.smithi100.stdout: "client_reconnect_stale": "false", 2024-09-15T21:47:02.209 INFO:teuthology.orchestra.run.smithi100.stdout: "client_shutdown_timeout": "30", 2024-09-15T21:47:02.209 INFO:teuthology.orchestra.run.smithi100.stdout: "client_snapdir": ".snap", 2024-09-15T21:47:02.209 INFO:teuthology.orchestra.run.smithi100.stdout: "client_tick_interval": "1", 2024-09-15T21:47:02.210 INFO:teuthology.orchestra.run.smithi100.stdout: "client_trace": "", 2024-09-15T21:47:02.210 INFO:teuthology.orchestra.run.smithi100.stdout: "client_try_dentry_invalidate": "false", 2024-09-15T21:47:02.210 INFO:teuthology.orchestra.run.smithi100.stdout: "client_use_faked_inos": "false", 2024-09-15T21:47:02.210 INFO:teuthology.orchestra.run.smithi100.stdout: "client_use_random_mds": "false", 2024-09-15T21:47:02.210 INFO:teuthology.orchestra.run.smithi100.stdout: "clog_to_graylog": "false", 2024-09-15T21:47:02.210 INFO:teuthology.orchestra.run.smithi100.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-15T21:47:02.210 INFO:teuthology.orchestra.run.smithi100.stdout: "clog_to_graylog_port": "12201", 2024-09-15T21:47:02.210 INFO:teuthology.orchestra.run.smithi100.stdout: "clog_to_monitors": "default=true", 2024-09-15T21:47:02.210 INFO:teuthology.orchestra.run.smithi100.stdout: "clog_to_syslog": "false", 2024-09-15T21:47:02.210 INFO:teuthology.orchestra.run.smithi100.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-15T21:47:02.210 INFO:teuthology.orchestra.run.smithi100.stdout: "clog_to_syslog_level": "info", 2024-09-15T21:47:02.210 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "-", 2024-09-15T21:47:02.211 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_network": "", 2024-09-15T21:47:02.211 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_network_interface": "", 2024-09-15T21:47:02.211 INFO:teuthology.orchestra.run.smithi100.stdout: "compressor_zlib_isal": "false", 2024-09-15T21:47:02.211 INFO:teuthology.orchestra.run.smithi100.stdout: "compressor_zlib_level": "5", 2024-09-15T21:47:02.211 INFO:teuthology.orchestra.run.smithi100.stdout: "compressor_zlib_winsize": "-15", 2024-09-15T21:47:02.211 INFO:teuthology.orchestra.run.smithi100.stdout: "compressor_zstd_level": "1", 2024-09-15T21:47:02.211 INFO:teuthology.orchestra.run.smithi100.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-15T21:47:02.211 INFO:teuthology.orchestra.run.smithi100.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-15T21:47:02.211 INFO:teuthology.orchestra.run.smithi100.stdout: "crimson_alien_op_num_threads": "6", 2024-09-15T21:47:02.211 INFO:teuthology.orchestra.run.smithi100.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-15T21:47:02.211 INFO:teuthology.orchestra.run.smithi100.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-15T21:47:02.211 INFO:teuthology.orchestra.run.smithi100.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-15T21:47:02.212 INFO:teuthology.orchestra.run.smithi100.stdout: "crimson_osd_stat_interval": "0", 2024-09-15T21:47:02.212 INFO:teuthology.orchestra.run.smithi100.stdout: "crimson_seastar_cpu_cores": "", 2024-09-15T21:47:02.212 INFO:teuthology.orchestra.run.smithi100.stdout: "crimson_seastar_num_threads": "0", 2024-09-15T21:47:02.212 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_location": "", 2024-09-15T21:47:02.212 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_location_hook": "", 2024-09-15T21:47:02.212 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_location_hook_timeout": "10", 2024-09-15T21:47:02.212 INFO:teuthology.orchestra.run.smithi100.stdout: "daemonize": "false", 2024-09-15T21:47:02.212 INFO:teuthology.orchestra.run.smithi100.stdout: "daos_pool": "tank", 2024-09-15T21:47:02.212 INFO:teuthology.orchestra.run.smithi100.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-15T21:47:02.212 INFO:teuthology.orchestra.run.smithi100.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-15T21:47:02.212 INFO:teuthology.orchestra.run.smithi100.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-15T21:47:02.212 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_alienstore": "0/5", 2024-09-15T21:47:02.213 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_allow_any_pool_priority": "false", 2024-09-15T21:47:02.213 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_asok": "1/5", 2024-09-15T21:47:02.213 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_asok_assert_abort": "false", 2024-09-15T21:47:02.213 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_asserts_on_shutdown": "true", 2024-09-15T21:47:02.213 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_auth": "20/20", 2024-09-15T21:47:02.213 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_bdev": "1/3", 2024-09-15T21:47:02.213 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_bluefs": "1/5", 2024-09-15T21:47:02.213 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_bluestore": "1/5", 2024-09-15T21:47:02.213 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_buffer": "0/1", 2024-09-15T21:47:02.213 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_ceph_exporter": "1/5", 2024-09-15T21:47:02.213 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_cephfs_mirror": "0/5", 2024-09-15T21:47:02.213 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_cephsqlite": "0/5", 2024-09-15T21:47:02.213 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_client": "0/5", 2024-09-15T21:47:02.214 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_compressor": "1/5", 2024-09-15T21:47:02.214 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_context": "0/1", 2024-09-15T21:47:02.214 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_crimson_interrupt": "0/5", 2024-09-15T21:47:02.214 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_crush": "1/1", 2024-09-15T21:47:02.214 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_crypto": "1/5", 2024-09-15T21:47:02.214 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_cyanstore": "0/5", 2024-09-15T21:47:02.214 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_deliberately_leak_memory": "false", 2024-09-15T21:47:02.214 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_disable_randomized_ping": "false", 2024-09-15T21:47:02.214 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_dpdk": "1/5", 2024-09-15T21:47:02.214 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_eventtrace": "1/5", 2024-09-15T21:47:02.214 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_filer": "0/1", 2024-09-15T21:47:02.214 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_filestore": "1/3", 2024-09-15T21:47:02.214 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_finisher": "1/1", 2024-09-15T21:47:02.215 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_fuse": "1/5", 2024-09-15T21:47:02.215 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_heartbeat_testing_span": "0", 2024-09-15T21:47:02.215 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_heartbeatmap": "1/5", 2024-09-15T21:47:02.215 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-15T21:47:02.215 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_javaclient": "1/5", 2024-09-15T21:47:02.215 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_journal": "1/3", 2024-09-15T21:47:02.215 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_journaler": "0/5", 2024-09-15T21:47:02.215 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_kstore": "1/5", 2024-09-15T21:47:02.215 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_lockdep": "0/1", 2024-09-15T21:47:02.215 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mclock": "1/5", 2024-09-15T21:47:02.215 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mds": "1/5", 2024-09-15T21:47:02.215 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mds_balancer": "1/5", 2024-09-15T21:47:02.215 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mds_locker": "1/5", 2024-09-15T21:47:02.216 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mds_log": "1/5", 2024-09-15T21:47:02.216 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mds_log_expire": "1/5", 2024-09-15T21:47:02.216 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mds_migrator": "1/5", 2024-09-15T21:47:02.216 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mds_quiesce": "3/5", 2024-09-15T21:47:02.216 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_memstore": "1/5", 2024-09-15T21:47:02.216 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mgr": "2/5", 2024-09-15T21:47:02.216 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mgrc": "1/5", 2024-09-15T21:47:02.216 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_mon": "20/20", 2024-09-15T21:47:02.216 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_monc": "0/10", 2024-09-15T21:47:02.216 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_ms": "1/1", 2024-09-15T21:47:02.216 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_none": "0/5", 2024-09-15T21:47:02.216 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_objclass": "0/5", 2024-09-15T21:47:02.216 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_objectcacher": "0/5", 2024-09-15T21:47:02.217 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_objecter": "0/1", 2024-09-15T21:47:02.217 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_optracker": "0/5", 2024-09-15T21:47:02.217 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_osd": "1/5", 2024-09-15T21:47:02.217 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_paxos": "20/20", 2024-09-15T21:47:02.217 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_perfcounter": "1/5", 2024-09-15T21:47:02.217 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_prioritycache": "1/5", 2024-09-15T21:47:02.217 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rados": "0/5", 2024-09-15T21:47:02.217 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rbd": "0/5", 2024-09-15T21:47:02.217 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rbd_mirror": "0/5", 2024-09-15T21:47:02.217 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rbd_pwl": "0/5", 2024-09-15T21:47:02.217 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rbd_replay": "0/5", 2024-09-15T21:47:02.217 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_refs": "0/0", 2024-09-15T21:47:02.217 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_reserver": "1/1", 2024-09-15T21:47:02.218 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rgw": "1/5", 2024-09-15T21:47:02.218 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rgw_access": "1/5", 2024-09-15T21:47:02.218 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rgw_datacache": "1/5", 2024-09-15T21:47:02.218 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rgw_dbstore": "1/5", 2024-09-15T21:47:02.218 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rgw_flight": "1/5", 2024-09-15T21:47:02.218 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-15T21:47:02.218 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rgw_sync": "1/5", 2024-09-15T21:47:02.218 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_rocksdb": "4/5", 2024-09-15T21:47:02.218 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore": "0/5", 2024-09-15T21:47:02.218 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_backref": "0/5", 2024-09-15T21:47:02.218 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_cache": "0/5", 2024-09-15T21:47:02.218 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_cleaner": "0/5", 2024-09-15T21:47:02.218 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_device": "0/5", 2024-09-15T21:47:02.218 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_epm": "0/5", 2024-09-15T21:47:02.219 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-15T21:47:02.219 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_journal": "0/5", 2024-09-15T21:47:02.219 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_lba": "0/5", 2024-09-15T21:47:02.219 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_odata": "0/5", 2024-09-15T21:47:02.219 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_omap": "0/5", 2024-09-15T21:47:02.219 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_onode": "0/5", 2024-09-15T21:47:02.219 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_t": "0/5", 2024-09-15T21:47:02.219 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_seastore_tm": "0/5", 2024-09-15T21:47:02.219 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_striper": "0/1", 2024-09-15T21:47:02.219 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_test": "0/5", 2024-09-15T21:47:02.219 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_throttle": "1/1", 2024-09-15T21:47:02.219 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_timer": "0/1", 2024-09-15T21:47:02.219 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_tp": "0/5", 2024-09-15T21:47:02.220 INFO:teuthology.orchestra.run.smithi100.stdout: "debug_trace": "1/5", 2024-09-15T21:47:02.220 INFO:teuthology.orchestra.run.smithi100.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-15T21:47:02.220 INFO:teuthology.orchestra.run.smithi100.stdout: "device_failure_prediction_mode": "none", 2024-09-15T21:47:02.220 INFO:teuthology.orchestra.run.smithi100.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-15T21:47:02.220 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-15T21:47:02.220 INFO:teuthology.orchestra.run.smithi100.stdout: "err_to_graylog": "false", 2024-09-15T21:47:02.220 INFO:teuthology.orchestra.run.smithi100.stdout: "err_to_journald": "false", 2024-09-15T21:47:02.220 INFO:teuthology.orchestra.run.smithi100.stdout: "err_to_stderr": "true", 2024-09-15T21:47:02.220 INFO:teuthology.orchestra.run.smithi100.stdout: "err_to_syslog": "false", 2024-09-15T21:47:02.220 INFO:teuthology.orchestra.run.smithi100.stdout: "event_tracing": "false", 2024-09-15T21:47:02.220 INFO:teuthology.orchestra.run.smithi100.stdout: "exporter_addr": "0.0.0.0", 2024-09-15T21:47:02.220 INFO:teuthology.orchestra.run.smithi100.stdout: "exporter_http_port": "9926", 2024-09-15T21:47:02.221 INFO:teuthology.orchestra.run.smithi100.stdout: "exporter_prio_limit": "5", 2024-09-15T21:47:02.221 INFO:teuthology.orchestra.run.smithi100.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-15T21:47:02.221 INFO:teuthology.orchestra.run.smithi100.stdout: "exporter_sort_metrics": "true", 2024-09-15T21:47:02.221 INFO:teuthology.orchestra.run.smithi100.stdout: "exporter_stats_period": "5", 2024-09-15T21:47:02.221 INFO:teuthology.orchestra.run.smithi100.stdout: "fake_statfs_for_testing": "0", 2024-09-15T21:47:02.221 INFO:teuthology.orchestra.run.smithi100.stdout: "fatal_signal_handlers": "true", 2024-09-15T21:47:02.221 INFO:teuthology.orchestra.run.smithi100.stdout: "filer_max_purge_ops": "10", 2024-09-15T21:47:02.221 INFO:teuthology.orchestra.run.smithi100.stdout: "filer_max_truncate_ops": "128", 2024-09-15T21:47:02.221 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_apply_finisher_threads": "1", 2024-09-15T21:47:02.221 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_blackhole": "false", 2024-09-15T21:47:02.221 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_btrfs_clone_range": "true", 2024-09-15T21:47:02.221 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_btrfs_snap": "true", 2024-09-15T21:47:02.221 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_caller_concurrency": "10", 2024-09-15T21:47:02.222 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_collect_device_partition_information": "true", 2024-09-15T21:47:02.222 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_commit_timeout": "600.000000", 2024-09-15T21:47:02.222 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_debug_inject_read_err": "false", 2024-09-15T21:47:02.222 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_debug_omap_check": "false", 2024-09-15T21:47:02.222 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-15T21:47:02.222 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_debug_verify_split": "false", 2024-09-15T21:47:02.222 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_dump_file": "", 2024-09-15T21:47:02.222 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-15T21:47:02.222 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-15T21:47:02.222 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_fadvise": "true", 2024-09-15T21:47:02.222 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_fail_eio": "true", 2024-09-15T21:47:02.222 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_fd_cache_shards": "16", 2024-09-15T21:47:02.222 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_fd_cache_size": "128", 2024-09-15T21:47:02.223 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_fiemap": "false", 2024-09-15T21:47:02.223 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_fiemap_threshold": "4096", 2024-09-15T21:47:02.223 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-15T21:47:02.223 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-15T21:47:02.223 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_inject_stall": "0", 2024-09-15T21:47:02.223 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_journal_parallel": "false", 2024-09-15T21:47:02.223 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_journal_trailing": "false", 2024-09-15T21:47:02.223 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_journal_writeahead": "false", 2024-09-15T21:47:02.223 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_kill_at": "0", 2024-09-15T21:47:02.223 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-15T21:47:02.223 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-15T21:47:02.223 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-15T21:47:02.223 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-15T21:47:02.224 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-15T21:47:02.224 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_inline_xattrs": "0", 2024-09-15T21:47:02.224 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-15T21:47:02.224 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-15T21:47:02.224 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-15T21:47:02.224 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-15T21:47:02.224 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_xattr_value_size": "0", 2024-09-15T21:47:02.224 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-15T21:47:02.224 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-15T21:47:02.224 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-15T21:47:02.224 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_merge_threshold": "-10", 2024-09-15T21:47:02.224 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-15T21:47:02.224 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_odsync_write": "false", 2024-09-15T21:47:02.224 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_omap_backend": "rocksdb", 2024-09-15T21:47:02.225 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_omap_backend_path": "", 2024-09-15T21:47:02.225 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-15T21:47:02.225 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-15T21:47:02.225 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-15T21:47:02.225 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_op_thread_timeout": "60", 2024-09-15T21:47:02.225 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_op_threads": "2", 2024-09-15T21:47:02.225 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_punch_hole": "false", 2024-09-15T21:47:02.225 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-15T21:47:02.225 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-15T21:47:02.225 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-15T21:47:02.225 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-15T21:47:02.225 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-15T21:47:02.225 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-15T21:47:02.225 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-15T21:47:02.226 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-15T21:47:02.226 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-15T21:47:02.226 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_queue_max_ops": "50", 2024-09-15T21:47:02.226 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-15T21:47:02.226 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_seek_data_hole": "false", 2024-09-15T21:47:02.226 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_sloppy_crc": "false", 2024-09-15T21:47:02.226 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-15T21:47:02.226 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_splice": "false", 2024-09-15T21:47:02.226 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_split_multiple": "2", 2024-09-15T21:47:02.226 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_split_rand_factor": "20", 2024-09-15T21:47:02.226 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_update_to": "1000", 2024-09-15T21:47:02.226 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-15T21:47:02.226 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-15T21:47:02.226 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-15T21:47:02.227 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-15T21:47:02.227 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-15T21:47:02.227 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-15T21:47:02.227 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_enable": "true", 2024-09-15T21:47:02.227 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-15T21:47:02.227 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-15T21:47:02.227 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-15T21:47:02.227 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-15T21:47:02.227 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-15T21:47:02.227 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-15T21:47:02.227 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_xfs_extsize": "false", 2024-09-15T21:47:02.227 INFO:teuthology.orchestra.run.smithi100.stdout: "filestore_zfs_snap": "false", 2024-09-15T21:47:02.227 INFO:teuthology.orchestra.run.smithi100.stdout: "fio_dir": "/tmp/fio", 2024-09-15T21:47:02.227 INFO:teuthology.orchestra.run.smithi100.stdout: "fsid": "30e2fff4-2171-4531-8806-0e03b63f494f", 2024-09-15T21:47:02.228 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_allow_other": "true", 2024-09-15T21:47:02.228 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_atomic_o_trunc": "true", 2024-09-15T21:47:02.228 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_big_writes": "true", 2024-09-15T21:47:02.228 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_debug": "false", 2024-09-15T21:47:02.228 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_default_permissions": "false", 2024-09-15T21:47:02.228 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_disable_pagecache": "false", 2024-09-15T21:47:02.228 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_max_write": "0", 2024-09-15T21:47:02.228 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_multithreaded": "true", 2024-09-15T21:47:02.228 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_require_active_mds": "true", 2024-09-15T21:47:02.228 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_set_user_groups": "true", 2024-09-15T21:47:02.228 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_splice_move": "true", 2024-09-15T21:47:02.228 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_splice_read": "true", 2024-09-15T21:47:02.228 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_splice_write": "true", 2024-09-15T21:47:02.229 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-15T21:47:02.229 INFO:teuthology.orchestra.run.smithi100.stdout: "fuse_use_invalidate_cb": "true", 2024-09-15T21:47:02.229 INFO:teuthology.orchestra.run.smithi100.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-15T21:47:02.229 INFO:teuthology.orchestra.run.smithi100.stdout: "gss_target_name": "ceph", 2024-09-15T21:47:02.229 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_file": "", 2024-09-15T21:47:02.229 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_inject_failure": "0", 2024-09-15T21:47:02.229 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_interval": "5", 2024-09-15T21:47:02.229 INFO:teuthology.orchestra.run.smithi100.stdout: "host": "smithi100", 2024-09-15T21:47:02.229 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-15T21:47:02.229 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-15T21:47:02.229 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-15T21:47:02.229 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-15T21:47:02.229 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-15T21:47:02.229 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-15T21:47:02.230 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-15T21:47:02.230 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-15T21:47:02.230 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-15T21:47:02.230 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-15T21:47:02.230 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-15T21:47:02.230 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-15T21:47:02.230 INFO:teuthology.orchestra.run.smithi100.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-15T21:47:02.230 INFO:teuthology.orchestra.run.smithi100.stdout: "inject_early_sigterm": "false", 2024-09-15T21:47:02.230 INFO:teuthology.orchestra.run.smithi100.stdout: "jaeger_agent_port": "6799", 2024-09-15T21:47:02.230 INFO:teuthology.orchestra.run.smithi100.stdout: "jaeger_tracing_enable": "false", 2024-09-15T21:47:02.230 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_aio": "true", 2024-09-15T21:47:02.230 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_align_min_size": "65536", 2024-09-15T21:47:02.230 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_block_align": "true", 2024-09-15T21:47:02.231 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_block_size": "4096", 2024-09-15T21:47:02.231 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_dio": "true", 2024-09-15T21:47:02.231 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_discard": "false", 2024-09-15T21:47:02.231 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_force_aio": "false", 2024-09-15T21:47:02.231 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_ignore_corruption": "false", 2024-09-15T21:47:02.231 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_max_write_bytes": "10485760", 2024-09-15T21:47:02.231 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_max_write_entries": "100", 2024-09-15T21:47:02.231 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_replay_from": "0", 2024-09-15T21:47:02.231 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-15T21:47:02.231 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-15T21:47:02.231 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-15T21:47:02.231 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-15T21:47:02.231 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_write_header_frequency": "0", 2024-09-15T21:47:02.232 INFO:teuthology.orchestra.run.smithi100.stdout: "journal_zero_on_create": "false", 2024-09-15T21:47:02.232 INFO:teuthology.orchestra.run.smithi100.stdout: "journaler_prefetch_periods": "10", 2024-09-15T21:47:02.232 INFO:teuthology.orchestra.run.smithi100.stdout: "journaler_prezero_periods": "5", 2024-09-15T21:47:02.232 INFO:teuthology.orchestra.run.smithi100.stdout: "journaler_write_head_interval": "15", 2024-09-15T21:47:02.232 INFO:teuthology.orchestra.run.smithi100.stdout: "key": "", 2024-09-15T21:47:02.232 INFO:teuthology.orchestra.run.smithi100.stdout: "keyfile": "", 2024-09-15T21:47:02.232 INFO:teuthology.orchestra.run.smithi100.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-15T21:47:02.232 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_backend": "rocksdb", 2024-09-15T21:47:02.232 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_default_stripe_size": "65536", 2024-09-15T21:47:02.232 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_fsck_on_mount": "false", 2024-09-15T21:47:02.232 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-15T21:47:02.232 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_max_bytes": "67108864", 2024-09-15T21:47:02.232 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_max_ops": "512", 2024-09-15T21:47:02.233 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_nid_prealloc": "1024", 2024-09-15T21:47:02.233 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_onode_map_size": "1024", 2024-09-15T21:47:02.233 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-15T21:47:02.233 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_sync_submit_transaction": "false", 2024-09-15T21:47:02.233 INFO:teuthology.orchestra.run.smithi100.stdout: "kstore_sync_transaction": "false", 2024-09-15T21:47:02.233 INFO:teuthology.orchestra.run.smithi100.stdout: "librados_thread_count": "2", 2024-09-15T21:47:02.233 INFO:teuthology.orchestra.run.smithi100.stdout: "lockdep": "false", 2024-09-15T21:47:02.233 INFO:teuthology.orchestra.run.smithi100.stdout: "lockdep_force_backtrace": "false", 2024-09-15T21:47:02.233 INFO:teuthology.orchestra.run.smithi100.stdout: "log_coarse_timestamps": "true", 2024-09-15T21:47:02.233 INFO:teuthology.orchestra.run.smithi100.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-15T21:47:02.233 INFO:teuthology.orchestra.run.smithi100.stdout: "log_flush_on_exit": "false", 2024-09-15T21:47:02.233 INFO:teuthology.orchestra.run.smithi100.stdout: "log_graylog_host": "127.0.0.1", 2024-09-15T21:47:02.233 INFO:teuthology.orchestra.run.smithi100.stdout: "log_graylog_port": "12201", 2024-09-15T21:47:02.233 INFO:teuthology.orchestra.run.smithi100.stdout: "log_max_new": "1000", 2024-09-15T21:47:02.234 INFO:teuthology.orchestra.run.smithi100.stdout: "log_max_recent": "10000", 2024-09-15T21:47:02.234 INFO:teuthology.orchestra.run.smithi100.stdout: "log_stderr_prefix": "", 2024-09-15T21:47:02.234 INFO:teuthology.orchestra.run.smithi100.stdout: "log_stop_at_utilization": "0.970000", 2024-09-15T21:47:02.234 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_file": "true", 2024-09-15T21:47:02.234 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_graylog": "false", 2024-09-15T21:47:02.234 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_journald": "false", 2024-09-15T21:47:02.234 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_stderr": "false", 2024-09-15T21:47:02.234 INFO:teuthology.orchestra.run.smithi100.stdout: "log_to_syslog": "false", 2024-09-15T21:47:02.234 INFO:teuthology.orchestra.run.smithi100.stdout: "max_rotating_auth_attempts": "10", 2024-09-15T21:47:02.234 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-15T21:47:02.234 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_action_on_write_error": "1", 2024-09-15T21:47:02.234 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_alternate_name_max": "8192", 2024-09-15T21:47:02.234 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_asio_thread_count": "2", 2024-09-15T21:47:02.234 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_export_pin": "true", 2024-09-15T21:47:02.234 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_fragment_dirs": "true", 2024-09-15T21:47:02.235 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-15T21:47:02.235 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_fragment_interval": "5", 2024-09-15T21:47:02.235 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-15T21:47:02.235 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-15T21:47:02.235 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_interval": "10", 2024-09-15T21:47:02.235 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_max": "-1", 2024-09-15T21:47:02.235 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_max_until": "-1", 2024-09-15T21:47:02.235 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_merge_size": "50", 2024-09-15T21:47:02.235 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_midchunk": "0.300000", 2024-09-15T21:47:02.235 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-15T21:47:02.235 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_min_start": "0.200000", 2024-09-15T21:47:02.235 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_minchunk": "0.001000", 2024-09-15T21:47:02.235 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_mode": "0", 2024-09-15T21:47:02.235 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_need_max": "1.200000", 2024-09-15T21:47:02.236 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_need_min": "0.800000", 2024-09-15T21:47:02.236 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_overload_epochs": "2", 2024-09-15T21:47:02.236 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-15T21:47:02.236 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-15T21:47:02.236 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_split_bits": "3", 2024-09-15T21:47:02.236 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-15T21:47:02.236 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_split_size": "10000", 2024-09-15T21:47:02.236 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-15T21:47:02.236 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_target_decay": "10.000000", 2024-09-15T21:47:02.236 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-15T21:47:02.236 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_beacon_grace": "15.000000", 2024-09-15T21:47:02.236 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_beacon_interval": "4.000000", 2024-09-15T21:47:02.236 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-15T21:47:02.237 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-15T21:47:02.237 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache_mid": "0.700000", 2024-09-15T21:47:02.237 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-15T21:47:02.237 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache_quiesce_delay": "0", 2024-09-15T21:47:02.237 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-15T21:47:02.237 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-15T21:47:02.237 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-15T21:47:02.237 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache_release_free_interval": "10", 2024-09-15T21:47:02.237 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache_reservation": "0.050000", 2024-09-15T21:47:02.237 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-15T21:47:02.237 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache_trim_interval": "1", 2024-09-15T21:47:02.237 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache_trim_threshold": "262144", 2024-09-15T21:47:02.238 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-15T21:47:02.238 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-15T21:47:02.238 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-15T21:47:02.238 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_client_prealloc_inos": "1000", 2024-09-15T21:47:02.238 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-15T21:47:02.238 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_connect_bootstrapping": "false", 2024-09-15T21:47:02.238 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_damage_table_max_entries": "10000", 2024-09-15T21:47:02.238 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-15T21:47:02.238 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_debug_auth_pins": "false", 2024-09-15T21:47:02.238 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_debug_frag": "false", 2024-09-15T21:47:02.238 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_debug_scatterstat": "false", 2024-09-15T21:47:02.238 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_debug_subtrees": "false", 2024-09-15T21:47:02.238 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_decay_halflife": "5.000000", 2024-09-15T21:47:02.238 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_default_dir_hash": "2", 2024-09-15T21:47:02.239 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_defer_session_stale": "true", 2024-09-15T21:47:02.239 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_deny_all_reconnect": "false", 2024-09-15T21:47:02.239 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_dir_keys_per_op": "16384", 2024-09-15T21:47:02.239 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_dir_max_commit_size": "10", 2024-09-15T21:47:02.239 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_dir_max_entries": "0", 2024-09-15T21:47:02.239 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_dir_prefetch": "true", 2024-09-15T21:47:02.239 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-15T21:47:02.239 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-15T21:47:02.239 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_dump_cache_on_map": "false", 2024-09-15T21:47:02.239 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-15T21:47:02.239 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-15T21:47:02.239 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_early_reply": "true", 2024-09-15T21:47:02.239 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_enable_op_tracker": "true", 2024-09-15T21:47:02.240 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_enforce_unique_name": "true", 2024-09-15T21:47:02.240 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-15T21:47:02.240 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-15T21:47:02.240 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_export_ephemeral_random": "true", 2024-09-15T21:47:02.240 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-15T21:47:02.240 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-15T21:47:02.240 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-15T21:47:02.240 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-15T21:47:02.240 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-15T21:47:02.240 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-15T21:47:02.240 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-15T21:47:02.240 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-15T21:47:02.240 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_health_summarize_threshold": "10", 2024-09-15T21:47:02.240 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-15T21:47:02.241 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-15T21:47:02.241 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_inject_health_dummy": "false", 2024-09-15T21:47:02.241 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-15T21:47:02.241 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_inject_migrator_session_race": "false", 2024-09-15T21:47:02.241 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-15T21:47:02.241 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-15T21:47:02.241 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-15T21:47:02.241 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_join_fs": "", 2024-09-15T21:47:02.241 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_journal_format": "1", 2024-09-15T21:47:02.241 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-15T21:47:02.241 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_create_at": "0", 2024-09-15T21:47:02.241 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_export_at": "0", 2024-09-15T21:47:02.241 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_import_at": "0", 2024-09-15T21:47:02.242 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_journal_at": "0", 2024-09-15T21:47:02.242 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_journal_expire_at": "0", 2024-09-15T21:47:02.242 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_journal_replay_at": "0", 2024-09-15T21:47:02.242 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_link_at": "0", 2024-09-15T21:47:02.242 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_mdstable_at": "0", 2024-09-15T21:47:02.242 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_openc_at": "0", 2024-09-15T21:47:02.242 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_rename_at": "0", 2024-09-15T21:47:02.242 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_kill_shutdown_at": "0", 2024-09-15T21:47:02.242 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_event_large_threshold": "524288", 2024-09-15T21:47:02.242 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_events_per_segment": "1024", 2024-09-15T21:47:02.242 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-15T21:47:02.242 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_max_events": "-1", 2024-09-15T21:47:02.242 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_max_segments": "128", 2024-09-15T21:47:02.243 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_pause": "false", 2024-09-15T21:47:02.243 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_segment_size": "0", 2024-09-15T21:47:02.243 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-15T21:47:02.243 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-15T21:47:02.243 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-15T21:47:02.243 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_trim_threshold": "128", 2024-09-15T21:47:02.243 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-15T21:47:02.243 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log_warn_factor": "2.000000", 2024-09-15T21:47:02.243 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_caps_per_client": "1048576", 2024-09-15T21:47:02.243 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_completed_flushes": "100000", 2024-09-15T21:47:02.243 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_completed_requests": "100000", 2024-09-15T21:47:02.243 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_export_size": "20971520", 2024-09-15T21:47:02.243 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_file_recover": "32", 2024-09-15T21:47:02.244 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_purge_files": "64", 2024-09-15T21:47:02.244 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_purge_ops": "8192", 2024-09-15T21:47:02.244 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-15T21:47:02.244 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-15T21:47:02.244 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_max_snaps_per_dir": "100", 2024-09-15T21:47:02.244 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_metrics_update_interval": "2", 2024-09-15T21:47:02.244 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_min_caps_per_client": "100", 2024-09-15T21:47:02.244 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_min_caps_working_set": "10000", 2024-09-15T21:47:02.244 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-15T21:47:02.244 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_numa_node": "-1", 2024-09-15T21:47:02.244 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-15T21:47:02.244 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_op_complaint_time": "30.000000", 2024-09-15T21:47:02.244 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_op_history_duration": "600", 2024-09-15T21:47:02.244 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_op_history_size": "20", 2024-09-15T21:47:02.245 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_op_history_slow_op_size": "20", 2024-09-15T21:47:02.245 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-15T21:47:02.245 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_op_log_threshold": "5", 2024-09-15T21:47:02.245 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_ping_grace": "15", 2024-09-15T21:47:02.245 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_ping_interval": "5", 2024-09-15T21:47:02.245 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-15T21:47:02.245 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-15T21:47:02.245 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_recall_max_caps": "30000", 2024-09-15T21:47:02.245 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-15T21:47:02.245 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-15T21:47:02.245 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-15T21:47:02.245 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_recall_warning_threshold": "262144", 2024-09-15T21:47:02.245 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-15T21:47:02.246 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_replay_interval": "1.000000", 2024-09-15T21:47:02.246 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-15T21:47:02.246 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-15T21:47:02.246 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_root_ino_gid": "0", 2024-09-15T21:47:02.246 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_root_ino_uid": "0", 2024-09-15T21:47:02.246 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-15T21:47:02.246 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-15T21:47:02.246 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-15T21:47:02.246 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-15T21:47:02.246 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-15T21:47:02.246 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-15T21:47:02.246 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-15T21:47:02.246 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-15T21:47:02.246 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-15T21:47:02.246 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-15T21:47:02.247 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_shutdown_check": "0", 2024-09-15T21:47:02.247 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_skip_ino": "0", 2024-09-15T21:47:02.247 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-15T21:47:02.247 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_snap_max_uid": "4294967294", 2024-09-15T21:47:02.247 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_snap_min_uid": "0", 2024-09-15T21:47:02.247 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_snap_rstat": "false", 2024-09-15T21:47:02.247 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_standby_replay_damaged": "false", 2024-09-15T21:47:02.247 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_symlink_recovery": "true", 2024-09-15T21:47:02.247 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-15T21:47:02.247 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_thrash_exports": "0", 2024-09-15T21:47:02.247 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_thrash_fragments": "0", 2024-09-15T21:47:02.247 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_tick_interval": "5.000000", 2024-09-15T21:47:02.247 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_valgrind_exit": "false", 2024-09-15T21:47:02.248 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_verify_backtrace": "1", 2024-09-15T21:47:02.248 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_verify_scatter": "false", 2024-09-15T21:47:02.248 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-15T21:47:02.248 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_wipe_sessions": "false", 2024-09-15T21:47:02.248 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool_debug": "false", 2024-09-15T21:47:02.248 INFO:teuthology.orchestra.run.smithi100.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-15T21:47:02.248 INFO:teuthology.orchestra.run.smithi100.stdout: "memstore_device_bytes": "1073741824", 2024-09-15T21:47:02.248 INFO:teuthology.orchestra.run.smithi100.stdout: "memstore_page_set": "false", 2024-09-15T21:47:02.248 INFO:teuthology.orchestra.run.smithi100.stdout: "memstore_page_size": "65536", 2024-09-15T21:47:02.248 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_client_bytes": "134217728", 2024-09-15T21:47:02.248 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_client_messages": "512", 2024-09-15T21:47:02.248 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-15T21:47:02.249 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-15T21:47:02.249 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-15T21:47:02.249 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-15T21:47:02.249 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_disabled_modules": "", 2024-09-15T21:47:02.249 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_enable_op_tracker": "true", 2024-09-15T21:47:02.249 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-15T21:47:02.249 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_max_pg_num_change": "128", 2024-09-15T21:47:02.249 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_mds_bytes": "134217728", 2024-09-15T21:47:02.249 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_mds_messages": "128", 2024-09-15T21:47:02.249 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-15T21:47:02.249 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_mon_bytes": "134217728", 2024-09-15T21:47:02.249 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_mon_messages": "128", 2024-09-15T21:47:02.249 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-15T21:47:02.249 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-15T21:47:02.249 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_op_history_duration": "600", 2024-09-15T21:47:02.250 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_op_history_size": "20", 2024-09-15T21:47:02.250 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-15T21:47:02.250 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-15T21:47:02.250 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_op_log_threshold": "5", 2024-09-15T21:47:02.250 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_osd_bytes": "536870912", 2024-09-15T21:47:02.250 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_osd_messages": "8192", 2024-09-15T21:47:02.250 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_pool": "true", 2024-09-15T21:47:02.250 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-15T21:47:02.250 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_standby_modules": "true", 2024-09-15T21:47:02.250 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_stats_period": "5", 2024-09-15T21:47:02.250 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_stats_threshold": "5", 2024-09-15T21:47:02.250 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_tick_period": "2", 2024-09-15T21:47:02.250 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-15T21:47:02.250 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-15T21:47:02.251 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_allow_pool_delete": "true", 2024-09-15T21:47:02.251 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_allow_pool_size_one": "true", 2024-09-15T21:47:02.251 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_auth_validate_all_caps": "true", 2024-09-15T21:47:02.251 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-15T21:47:02.251 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-15T21:47:02.251 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_bytes": "104857600", 2024-09-15T21:47:02.251 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_directed_command_retry": "2", 2024-09-15T21:47:02.251 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-15T21:47:02.251 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-15T21:47:02.251 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-15T21:47:02.251 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-15T21:47:02.251 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_hunt_parallel": "3", 2024-09-15T21:47:02.251 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_log_interval": "1.000000", 2024-09-15T21:47:02.251 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-15T21:47:02.252 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_ping_interval": "10.000000", 2024-09-15T21:47:02.252 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-15T21:47:02.252 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_client_target_rank": "-1", 2024-09-15T21:47:02.252 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-15T21:47:02.252 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-15T21:47:02.252 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-15T21:47:02.252 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_file_level": "debug", 2024-09-15T21:47:02.252 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_to_file": "true", 2024-09-15T21:47:02.252 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-15T21:47:02.252 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-15T21:47:02.252 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-15T21:47:02.252 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_to_journald": "false", 2024-09-15T21:47:02.252 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-15T21:47:02.253 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-15T21:47:02.253 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-15T21:47:02.253 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-15T21:47:02.253 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_compact_on_bootstrap": "false", 2024-09-15T21:47:02.253 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_compact_on_start": "false", 2024-09-15T21:47:02.253 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_compact_on_trim": "true", 2024-09-15T21:47:02.253 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-15T21:47:02.253 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-15T21:47:02.253 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-15T21:47:02.253 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_cpu_threads": "4", 2024-09-15T21:47:02.253 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_crush_min_required_version": "hammer", 2024-09-15T21:47:02.253 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_daemon_bytes": "419430400", 2024-09-15T21:47:02.253 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-15T21:47:02.253 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_data_avail_crit": "5", 2024-09-15T21:47:02.254 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_data_avail_warn": "5", 2024-09-15T21:47:02.254 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_data_size_warn": "16106127360", 2024-09-15T21:47:02.254 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-15T21:47:02.254 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-15T21:47:02.254 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_dump_json": "false", 2024-09-15T21:47:02.254 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-15T21:47:02.254 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_dump_transactions": "false", 2024-09-15T21:47:02.254 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_extra_checks": "false", 2024-09-15T21:47:02.254 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-15T21:47:02.254 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-15T21:47:02.254 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_no_require_reef": "false", 2024-09-15T21:47:02.254 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_no_require_squid": "false", 2024-09-15T21:47:02.254 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-15T21:47:02.254 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-15T21:47:02.255 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-15T21:47:02.255 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_down_mkfs_grace": "300", 2024-09-15T21:47:02.255 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_down_uptime_grace": "60", 2024-09-15T21:47:02.255 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_election_default_strategy": "1", 2024-09-15T21:47:02.255 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_election_timeout": "5.000000", 2024-09-15T21:47:02.255 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-15T21:47:02.255 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_elector_ping_divisor": "2", 2024-09-15T21:47:02.255 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-15T21:47:02.255 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_enable_op_tracker": "true", 2024-09-15T21:47:02.255 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_fake_pool_delete": "false", 2024-09-15T21:47:02.255 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_force_quorum_join": "false", 2024-09-15T21:47:02.255 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-15T21:47:02.255 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_globalid_prealloc": "10000", 2024-09-15T21:47:02.255 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_health_detail_to_clog": "false", 2024-09-15T21:47:02.255 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_health_log_update_period": "5", 2024-09-15T21:47:02.256 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_health_max_detail": "50", 2024-09-15T21:47:02.256 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_health_to_clog": "true", 2024-09-15T21:47:02.256 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_health_to_clog_interval": "600", 2024-09-15T21:47:02.256 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-15T21:47:02.256 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_host": "172.21.15.100,172.21.15.122,[v2:172.21.15.122:3301,v1:172.21.15.122:6790]", 2024-09-15T21:47:02.256 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_host_override": "", 2024-09-15T21:47:02.256 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_initial_members": "", 2024-09-15T21:47:02.256 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-15T21:47:02.256 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-15T21:47:02.256 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-15T21:47:02.256 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-15T21:47:02.256 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-15T21:47:02.257 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_lease": "5.000000", 2024-09-15T21:47:02.257 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-15T21:47:02.257 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-15T21:47:02.257 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_log_full_interval": "50", 2024-09-15T21:47:02.257 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_log_max": "10000", 2024-09-15T21:47:02.257 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_log_max_summary": "50", 2024-09-15T21:47:02.257 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-15T21:47:02.257 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_max_log_epochs": "500", 2024-09-15T21:47:02.257 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-15T21:47:02.257 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-15T21:47:02.257 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_max_osd": "10000", 2024-09-15T21:47:02.257 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_max_pg_per_osd": "10000", 2024-09-15T21:47:02.257 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_max_pool_pg_num": "65536", 2024-09-15T21:47:02.258 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-15T21:47:02.258 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-15T21:47:02.258 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_mds_force_trim_to": "0", 2024-09-15T21:47:02.258 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_mds_skip_sanity": "false", 2024-09-15T21:47:02.258 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_memory_autotune": "true", 2024-09-15T21:47:02.258 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_memory_target": "2147483648", 2024-09-15T21:47:02.258 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_mgr_beacon_grace": "30", 2024-09-15T21:47:02.258 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-15T21:47:02.258 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_mgr_digest_period": "5", 2024-09-15T21:47:02.258 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_mgr_inactive_grace": "60", 2024-09-15T21:47:02.258 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-15T21:47:02.258 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-15T21:47:02.258 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_min_osdmap_epochs": "500", 2024-09-15T21:47:02.258 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_op_complaint_time": "120", 2024-09-15T21:47:02.259 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_op_history_duration": "600", 2024-09-15T21:47:02.259 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_op_history_size": "20", 2024-09-15T21:47:02.259 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_op_history_slow_op_size": "20", 2024-09-15T21:47:02.259 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-15T21:47:02.259 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_op_log_threshold": "5", 2024-09-15T21:47:02.259 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-15T21:47:02.259 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-15T21:47:02.259 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-15T21:47:02.259 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_auto_mark_in": "false", 2024-09-15T21:47:02.259 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-15T21:47:02.259 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-15T21:47:02.259 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-15T21:47:02.260 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_cache_size": "500", 2024-09-15T21:47:02.260 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-15T21:47:02.260 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-15T21:47:02.260 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-15T21:47:02.260 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_down_out_interval": "600", 2024-09-15T21:47:02.260 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-15T21:47:02.260 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-15T21:47:02.260 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_force_trim_to": "0", 2024-09-15T21:47:02.260 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-15T21:47:02.260 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-15T21:47:02.260 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-15T21:47:02.260 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-15T21:47:02.260 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-15T21:47:02.261 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-15T21:47:02.261 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-15T21:47:02.261 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-15T21:47:02.261 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_min_down_reporters": "2", 2024-09-15T21:47:02.261 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-15T21:47:02.261 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-15T21:47:02.261 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-15T21:47:02.261 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-15T21:47:02.261 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-15T21:47:02.261 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-15T21:47:02.261 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_report_timeout": "900", 2024-09-15T21:47:02.261 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-15T21:47:02.262 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-15T21:47:02.262 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-15T21:47:02.262 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-15T21:47:02.262 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-15T21:47:02.262 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-15T21:47:02.262 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-15T21:47:02.262 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-15T21:47:02.262 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-15T21:47:02.262 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_pg_stuck_threshold": "60", 2024-09-15T21:47:02.262 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-15T21:47:02.262 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-15T21:47:02.262 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-15T21:47:02.262 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-15T21:47:02.262 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-15T21:47:02.263 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-15T21:47:02.263 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_probe_timeout": "2.000000", 2024-09-15T21:47:02.263 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_reweight_max_change": "0.050000", 2024-09-15T21:47:02.263 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_reweight_max_osds": "4", 2024-09-15T21:47:02.263 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-15T21:47:02.263 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-15T21:47:02.263 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-15T21:47:02.263 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-15T21:47:02.263 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-15T21:47:02.263 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_scrub_interval": "86400", 2024-09-15T21:47:02.263 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_scrub_max_keys": "100", 2024-09-15T21:47:02.263 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_scrub_timeout": "300", 2024-09-15T21:47:02.263 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_session_timeout": "300", 2024-09-15T21:47:02.264 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_smart_report_timeout": "5", 2024-09-15T21:47:02.264 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_stat_smooth_intervals": "6", 2024-09-15T21:47:02.264 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-15T21:47:02.264 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_stretch_pool_min_size": "2", 2024-09-15T21:47:02.264 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_stretch_pool_size": "4", 2024-09-15T21:47:02.264 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-15T21:47:02.264 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-15T21:47:02.264 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_sync_debug": "false", 2024-09-15T21:47:02.264 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-15T21:47:02.264 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-15T21:47:02.264 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_sync_provider_kill_at": "0", 2024-09-15T21:47:02.264 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_sync_requester_kill_at": "0", 2024-09-15T21:47:02.265 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_sync_timeout": "60.000000", 2024-09-15T21:47:02.265 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_target_pg_per_osd": "100", 2024-09-15T21:47:02.265 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_tick_interval": "5", 2024-09-15T21:47:02.265 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_timecheck_interval": "300.000000", 2024-09-15T21:47:02.265 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-15T21:47:02.265 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_use_min_delay_socket": "false", 2024-09-15T21:47:02.265 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_older_version_delay": "604800", 2024-09-15T21:47:02.265 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-15T21:47:02.265 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-15T21:47:02.265 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-15T21:47:02.265 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-15T21:47:02.265 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-15T21:47:02.265 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-15T21:47:02.265 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-15T21:47:02.266 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_misplaced": "false", 2024-09-15T21:47:02.266 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-15T21:47:02.266 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_older_version": "true", 2024-09-15T21:47:02.266 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-15T21:47:02.266 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-15T21:47:02.266 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-15T21:47:02.266 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-15T21:47:02.266 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-15T21:47:02.266 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-15T21:47:02.266 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-15T21:47:02.267 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-15T21:47:02.267 INFO:teuthology.orchestra.run.smithi100.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-15T21:47:02.267 INFO:teuthology.orchestra.run.smithi100.stdout: "monmap": "", 2024-09-15T21:47:02.267 INFO:teuthology.orchestra.run.smithi100.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-15T21:47:02.267 INFO:teuthology.orchestra.run.smithi100.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-15T21:47:02.267 INFO:teuthology.orchestra.run.smithi100.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-15T21:47:02.267 INFO:teuthology.orchestra.run.smithi100.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-15T21:47:02.267 INFO:teuthology.orchestra.run.smithi100.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-15T21:47:02.267 INFO:teuthology.orchestra.run.smithi100.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-15T21:47:02.267 INFO:teuthology.orchestra.run.smithi100.stdout: "motr_tracing_enabled": "false", 2024-09-15T21:47:02.267 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_op_threads": "3", 2024-09-15T21:47:02.267 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-15T21:47:02.268 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_cm": "false", 2024-09-15T21:47:02.268 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_device_name": "", 2024-09-15T21:47:02.268 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_dscp": "96", 2024-09-15T21:47:02.268 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-15T21:47:02.268 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-15T21:47:02.268 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_local_gid": "", 2024-09-15T21:47:02.268 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-15T21:47:02.268 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_port_num": "1", 2024-09-15T21:47:02.268 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-15T21:47:02.268 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-15T21:47:02.268 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-15T21:47:02.268 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-15T21:47:02.268 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_sl": "3", 2024-09-15T21:47:02.268 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_support_srq": "true", 2024-09-15T21:47:02.269 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_rdma_type": "ib", 2024-09-15T21:47:02.269 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_async_reap_threshold": "5", 2024-09-15T21:47:02.269 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_before_connect": "false", 2024-09-15T21:47:02.269 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_ipv4": "true", 2024-09-15T21:47:02.269 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_ipv6": "false", 2024-09-15T21:47:02.269 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_msgr1": "true", 2024-09-15T21:47:02.269 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_msgr2": "true", 2024-09-15T21:47:02.269 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_port_max": "7568", 2024-09-15T21:47:02.269 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_port_min": "6800", 2024-09-15T21:47:02.269 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-15T21:47:02.269 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_retry_count": "3", 2024-09-15T21:47:02.269 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_bind_retry_delay": "5", 2024-09-15T21:47:02.269 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_blackhole_client": "false", 2024-09-15T21:47:02.270 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_blackhole_mds": "false", 2024-09-15T21:47:02.270 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_blackhole_mgr": "false", 2024-09-15T21:47:02.270 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_blackhole_mon": "false", 2024-09-15T21:47:02.270 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_blackhole_osd": "false", 2024-09-15T21:47:02.270 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_client_mode": "crc secure", 2024-09-15T21:47:02.270 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_cluster_mode": "crc secure", 2024-09-15T21:47:02.270 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_cluster_type": "", 2024-09-15T21:47:02.270 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_compress_secure": "false", 2024-09-15T21:47:02.270 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_connection_idle_timeout": "900", 2024-09-15T21:47:02.270 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_connection_ready_timeout": "10", 2024-09-15T21:47:02.270 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_crc_data": "true", 2024-09-15T21:47:02.270 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_crc_header": "true", 2024-09-15T21:47:02.270 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_die_on_bad_msg": "false", 2024-09-15T21:47:02.270 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_die_on_bug": "true", 2024-09-15T21:47:02.270 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_die_on_old_message": "true", 2024-09-15T21:47:02.271 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_die_on_skipped_message": "false", 2024-09-15T21:47:02.271 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-15T21:47:02.271 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-15T21:47:02.271 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_coremask": "0xF", 2024-09-15T21:47:02.271 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-15T21:47:02.271 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-15T21:47:02.271 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_enable_tso": "true", 2024-09-15T21:47:02.271 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-15T21:47:02.271 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-15T21:47:02.271 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_hugepages": "", 2024-09-15T21:47:02.271 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-15T21:47:02.271 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-15T21:47:02.271 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_lro": "true", 2024-09-15T21:47:02.271 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_memory_channel": "4", 2024-09-15T21:47:02.272 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-15T21:47:02.272 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_pmd": "", 2024-09-15T21:47:02.272 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_port_id": "0", 2024-09-15T21:47:02.272 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-15T21:47:02.272 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-15T21:47:02.272 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_dump_on_send": "false", 2024-09-15T21:47:02.272 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_initial_backoff": "0.200000", 2024-09-15T21:47:02.272 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_inject_delay_max": "1.000000", 2024-09-15T21:47:02.272 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-15T21:47:02.272 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_inject_delay_type": "", 2024-09-15T21:47:02.272 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-15T21:47:02.272 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_inject_network_congestion": "0", 2024-09-15T21:47:02.272 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_inject_socket_failures": "0", 2024-09-15T21:47:02.273 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_learn_addr_from_peer": "true", 2024-09-15T21:47:02.273 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_max_accept_failures": "4", 2024-09-15T21:47:02.273 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_max_backoff": "15.000000", 2024-09-15T21:47:02.273 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_mon_client_mode": "secure crc", 2024-09-15T21:47:02.273 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-15T21:47:02.273 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_mon_service_mode": "secure crc", 2024-09-15T21:47:02.273 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_osd_compress_min_size": "1024", 2024-09-15T21:47:02.273 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_osd_compress_mode": "none", 2024-09-15T21:47:02.273 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-15T21:47:02.273 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-15T21:47:02.273 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_pq_min_cost": "65536", 2024-09-15T21:47:02.273 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_public_type": "", 2024-09-15T21:47:02.273 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_service_mode": "crc secure", 2024-09-15T21:47:02.273 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_tcp_listen_backlog": "512", 2024-09-15T21:47:02.274 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_tcp_nodelay": "true", 2024-09-15T21:47:02.274 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-15T21:47:02.274 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_tcp_rcvbuf": "0", 2024-09-15T21:47:02.274 INFO:teuthology.orchestra.run.smithi100.stdout: "ms_type": "async+posix", 2024-09-15T21:47:02.274 INFO:teuthology.orchestra.run.smithi100.stdout: "no_config_file": "false", 2024-09-15T21:47:02.274 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter_completion_locks_per_session": "32", 2024-09-15T21:47:02.274 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-15T21:47:02.274 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-15T21:47:02.274 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter_inflight_ops": "1024", 2024-09-15T21:47:02.274 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-15T21:47:02.274 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-15T21:47:02.274 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter_tick_interval": "5.000000", 2024-09-15T21:47:02.274 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter_timeout": "10.000000", 2024-09-15T21:47:02.274 INFO:teuthology.orchestra.run.smithi100.stdout: "objectstore_blackhole": "false", 2024-09-15T21:47:02.275 INFO:teuthology.orchestra.run.smithi100.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-15T21:47:02.275 INFO:teuthology.orchestra.run.smithi100.stdout: "openssl_engine_opts": "", 2024-09-15T21:47:02.275 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_agent_delay_time": "5.000000", 2024-09-15T21:47:02.275 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_agent_hist_halflife": "1000", 2024-09-15T21:47:02.275 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_agent_max_low_ops": "2", 2024-09-15T21:47:02.275 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_agent_max_ops": "4", 2024-09-15T21:47:02.275 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-15T21:47:02.275 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-15T21:47:02.275 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_agent_slop": "0.020000", 2024-09-15T21:47:02.275 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-15T21:47:02.275 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-15T21:47:02.275 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_asio_thread_count": "2", 2024-09-15T21:47:02.275 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_async_recovery_min_cost": "100", 2024-09-15T21:47:02.275 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-15T21:47:02.276 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-15T21:47:02.276 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_backfill_scan_max": "512", 2024-09-15T21:47:02.276 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_backfill_scan_min": "64", 2024-09-15T21:47:02.276 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_backoff_on_degraded": "false", 2024-09-15T21:47:02.276 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_backoff_on_peering": "false", 2024-09-15T21:47:02.276 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_backoff_on_unfound": "true", 2024-09-15T21:47:02.276 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_beacon_report_interval": "300", 2024-09-15T21:47:02.276 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bench_duration": "30", 2024-09-15T21:47:02.276 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-15T21:47:02.276 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bench_max_block_size": "67108864", 2024-09-15T21:47:02.276 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-15T21:47:02.276 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_blkin_trace_all": "false", 2024-09-15T21:47:02.276 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-15T21:47:02.276 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-15T21:47:02.277 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-15T21:47:02.277 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-15T21:47:02.277 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_check_for_log_corruption": "false", 2024-09-15T21:47:02.277 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-15T21:47:02.277 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-15T21:47:02.277 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-15T21:47:02.277 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-15T21:47:02.277 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_class_update_on_start": "true", 2024-09-15T21:47:02.277 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_client_message_cap": "256", 2024-09-15T21:47:02.277 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_client_message_size_cap": "524288000", 2024-09-15T21:47:02.277 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_client_op_priority": "63", 2024-09-15T21:47:02.277 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_client_watch_timeout": "30", 2024-09-15T21:47:02.277 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_command_max_records": "256", 2024-09-15T21:47:02.278 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_compact_on_start": "false", 2024-09-15T21:47:02.278 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-15T21:47:02.278 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-15T21:47:02.278 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-15T21:47:02.278 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_crush_update_on_start": "true", 2024-09-15T21:47:02.278 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_crush_update_weight_set": "true", 2024-09-15T21:47:02.278 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-15T21:47:02.278 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-15T21:47:02.278 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-15T21:47:02.278 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-15T21:47:02.278 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-15T21:47:02.278 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_feed_pullee": "-1", 2024-09-15T21:47:02.278 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-15T21:47:02.279 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-15T21:47:02.279 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-15T21:47:02.279 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_misdirected_ops": "false", 2024-09-15T21:47:02.279 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_no_acting_change": "false", 2024-09-15T21:47:02.279 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_no_purge_strays": "false", 2024-09-15T21:47:02.279 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_op_order": "false", 2024-09-15T21:47:02.279 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-15T21:47:02.279 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-15T21:47:02.279 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-15T21:47:02.279 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-15T21:47:02.279 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_shutdown": "false", 2024-09-15T21:47:02.279 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-15T21:47:02.279 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-15T21:47:02.280 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_trim_objects": "false", 2024-09-15T21:47:02.280 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-15T21:47:02.280 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-15T21:47:02.280 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_verify_snaps": "false", 2024-09-15T21:47:02.280 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-15T21:47:02.280 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-15T21:47:02.280 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_deep_scrub_keys": "1024", 2024-09-15T21:47:02.280 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-15T21:47:02.280 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-15T21:47:02.280 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-15T21:47:02.280 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_deep_scrub_stride": "524288", 2024-09-15T21:47:02.280 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-15T21:47:02.280 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-15T21:47:02.281 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_default_notify_timeout": "30", 2024-09-15T21:47:02.281 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_delete_sleep": "0.000000", 2024-09-15T21:47:02.281 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-15T21:47:02.281 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-15T21:47:02.281 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-15T21:47:02.281 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_discard_disconnected_ops": "true", 2024-09-15T21:47:02.281 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_enable_op_tracker": "true", 2024-09-15T21:47:02.281 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-15T21:47:02.281 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-15T21:47:02.281 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-15T21:47:02.281 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-15T21:47:02.281 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_fast_info": "true", 2024-09-15T21:47:02.281 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_fast_shutdown": "true", 2024-09-15T21:47:02.282 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-15T21:47:02.282 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-15T21:47:02.282 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-15T21:47:02.282 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-15T21:47:02.282 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-15T21:47:02.282 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_function_tracing": "false", 2024-09-15T21:47:02.282 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_heartbeat_grace": "20", 2024-09-15T21:47:02.282 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_heartbeat_interval": "6", 2024-09-15T21:47:02.282 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-15T21:47:02.282 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_heartbeat_min_peers": "10", 2024-09-15T21:47:02.282 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_heartbeat_min_size": "2000", 2024-09-15T21:47:02.282 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_heartbeat_stale": "600", 2024-09-15T21:47:02.282 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-15T21:47:02.282 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_hit_set_max_size": "100000", 2024-09-15T21:47:02.283 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_hit_set_min_size": "1000", 2024-09-15T21:47:02.283 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-15T21:47:02.283 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-15T21:47:02.283 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-15T21:47:02.283 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-15T21:47:02.283 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-15T21:47:02.283 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-15T21:47:02.283 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_journal_size": "5120", 2024-09-15T21:47:02.283 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_kill_backfill_at": "0", 2024-09-15T21:47:02.283 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-15T21:47:02.283 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_map_cache_size": "50", 2024-09-15T21:47:02.283 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_map_dedup": "true", 2024-09-15T21:47:02.284 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_map_message_max": "40", 2024-09-15T21:47:02.284 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-15T21:47:02.284 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_map_share_max_epochs": "40", 2024-09-15T21:47:02.284 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_attr_name_len": "100", 2024-09-15T21:47:02.284 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_attr_size": "0", 2024-09-15T21:47:02.284 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_backfills": "1", 2024-09-15T21:47:02.284 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_markdown_count": "5", 2024-09-15T21:47:02.284 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_markdown_period": "600", 2024-09-15T21:47:02.284 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_object_name_len": "2048", 2024-09-15T21:47:02.284 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_object_namespace_len": "256", 2024-09-15T21:47:02.284 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_object_size": "134217728", 2024-09-15T21:47:02.284 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-15T21:47:02.284 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-15T21:47:02.285 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_pg_blocked_by": "16", 2024-09-15T21:47:02.285 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_pg_log_entries": "10000", 2024-09-15T21:47:02.285 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-15T21:47:02.285 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_pgls": "1024", 2024-09-15T21:47:02.285 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_push_cost": "8388608", 2024-09-15T21:47:02.285 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_push_objects": "10", 2024-09-15T21:47:02.285 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_scrubs": "3", 2024-09-15T21:47:02.285 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-15T21:47:02.285 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_trimming_pgs": "2", 2024-09-15T21:47:02.285 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_write_op_reply_len": "64", 2024-09-15T21:47:02.285 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_max_write_size": "90", 2024-09-15T21:47:02.285 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-15T21:47:02.286 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-15T21:47:02.286 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-15T21:47:02.286 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-15T21:47:02.286 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-15T21:47:02.286 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-15T21:47:02.286 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-15T21:47:02.286 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-15T21:47:02.286 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_profile": "balanced", 2024-09-15T21:47:02.286 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-15T21:47:02.286 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-15T21:47:02.286 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-15T21:47:02.286 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-15T21:47:02.286 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-15T21:47:02.286 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-15T21:47:02.287 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-15T21:47:02.287 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-15T21:47:02.287 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-15T21:47:02.287 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-15T21:47:02.287 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-15T21:47:02.287 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_memory_base": "805306368", 2024-09-15T21:47:02.287 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_memory_cache_min": "134217728", 2024-09-15T21:47:02.287 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-15T21:47:02.287 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-15T21:47:02.287 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_memory_target": "4294967296", 2024-09-15T21:47:02.288 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_memory_target_autotune": "false", 2024-09-15T21:47:02.288 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-15T21:47:02.288 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_min_pg_log_entries": "250", 2024-09-15T21:47:02.288 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_min_recovery_priority": "0", 2024-09-15T21:47:02.288 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-15T21:47:02.288 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-15T21:47:02.288 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mon_report_interval": "5", 2024-09-15T21:47:02.288 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-15T21:47:02.288 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-15T21:47:02.288 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_num_cache_shards": "32", 2024-09-15T21:47:02.288 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_num_op_tracker_shard": "32", 2024-09-15T21:47:02.288 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_numa_auto_affinity": "true", 2024-09-15T21:47:02.288 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_numa_node": "-1", 2024-09-15T21:47:02.289 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_numa_prefer_iface": "true", 2024-09-15T21:47:02.289 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-15T21:47:02.289 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_objecter_finishers": "1", 2024-09-15T21:47:02.289 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_objectstore": "bluestore", 2024-09-15T21:47:02.289 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_objectstore_fuse": "false", 2024-09-15T21:47:02.289 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_objectstore_tracing": "false", 2024-09-15T21:47:02.289 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_complaint_time": "30.000000", 2024-09-15T21:47:02.289 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_history_duration": "600", 2024-09-15T21:47:02.289 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_history_size": "20", 2024-09-15T21:47:02.289 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_history_slow_op_size": "20", 2024-09-15T21:47:02.289 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-15T21:47:02.289 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_log_threshold": "5", 2024-09-15T21:47:02.289 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_num_shards": "0", 2024-09-15T21:47:02.290 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_num_shards_hdd": "5", 2024-09-15T21:47:02.290 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_num_shards_ssd": "8", 2024-09-15T21:47:02.290 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-15T21:47:02.290 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-15T21:47:02.290 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-15T21:47:02.290 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-15T21:47:02.290 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_pq_min_cost": "65536", 2024-09-15T21:47:02.290 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-15T21:47:02.290 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_queue_cut_off": "high", 2024-09-15T21:47:02.290 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-15T21:47:02.290 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_op_thread_timeout": "15", 2024-09-15T21:47:02.290 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_open_classes_on_start": "true", 2024-09-15T21:47:02.290 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_os_flags": "0", 2024-09-15T21:47:02.290 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_peering_op_priority": "255", 2024-09-15T21:47:02.291 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_delete_cost": "1048576", 2024-09-15T21:47:02.291 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_delete_priority": "5", 2024-09-15T21:47:02.291 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-15T21:47:02.291 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-15T21:47:02.291 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-15T21:47:02.291 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_log_trim_max": "10000", 2024-09-15T21:47:02.291 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_log_trim_min": "100", 2024-09-15T21:47:02.291 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-15T21:47:02.291 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-15T21:47:02.291 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-15T21:47:02.291 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-15T21:47:02.291 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-15T21:47:02.292 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-15T21:47:02.292 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-15T21:47:02.292 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-15T21:47:02.292 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-15T21:47:02.292 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-15T21:47:02.292 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_crimson": "false", 2024-09-15T21:47:02.292 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-15T21:47:02.292 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-15T21:47:02.292 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-15T21:47:02.292 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-15T21:47:02.292 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-15T21:47:02.292 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-15T21:47:02.292 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-15T21:47:02.293 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-15T21:47:02.293 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_flags": "0", 2024-09-15T21:47:02.293 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-15T21:47:02.293 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_min_size": "0", 2024-09-15T21:47:02.293 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-15T21:47:02.293 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_pg_num": "32", 2024-09-15T21:47:02.293 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_pgp_num": "0", 2024-09-15T21:47:02.293 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-15T21:47:02.293 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_read_ratio": "70", 2024-09-15T21:47:02.293 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_size": "2", 2024-09-15T21:47:02.293 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_default_type": "replicated", 2024-09-15T21:47:02.293 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-15T21:47:02.293 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-15T21:47:02.294 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_push_per_object_cost": "1000", 2024-09-15T21:47:02.294 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-15T21:47:02.294 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recover_clone_overlap": "true", 2024-09-15T21:47:02.294 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-15T21:47:02.294 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_cost": "20971520", 2024-09-15T21:47:02.294 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-15T21:47:02.294 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_max_active": "0", 2024-09-15T21:47:02.294 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-15T21:47:02.294 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-15T21:47:02.294 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-15T21:47:02.294 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-15T21:47:02.294 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_max_single_start": "1", 2024-09-15T21:47:02.294 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_op_priority": "3", 2024-09-15T21:47:02.294 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-15T21:47:02.295 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_priority": "5", 2024-09-15T21:47:02.295 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-15T21:47:02.295 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_sleep": "0.000000", 2024-09-15T21:47:02.295 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-15T21:47:02.295 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-15T21:47:02.295 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-15T21:47:02.295 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_repair_during_recovery": "false", 2024-09-15T21:47:02.295 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_requested_scrub_priority": "120", 2024-09-15T21:47:02.295 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-15T21:47:02.295 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-15T21:47:02.295 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_auto_repair": "false", 2024-09-15T21:47:02.295 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-15T21:47:02.295 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-15T21:47:02.295 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_begin_hour": "0", 2024-09-15T21:47:02.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_begin_week_day": "0", 2024-09-15T21:47:02.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_chunk_max": "25", 2024-09-15T21:47:02.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_chunk_min": "5", 2024-09-15T21:47:02.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_cost": "52428800", 2024-09-15T21:47:02.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-15T21:47:02.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_during_recovery": "false", 2024-09-15T21:47:02.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_end_hour": "0", 2024-09-15T21:47:02.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_end_week_day": "0", 2024-09-15T21:47:02.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_event_cost": "4096", 2024-09-15T21:47:02.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-15T21:47:02.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-15T21:47:02.296 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_invalid_stats": "true", 2024-09-15T21:47:02.297 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-15T21:47:02.297 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-15T21:47:02.297 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_max_preemptions": "5", 2024-09-15T21:47:02.297 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-15T21:47:02.297 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_priority": "5", 2024-09-15T21:47:02.297 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_scrub_sleep": "0.000000", 2024-09-15T21:47:02.297 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-15T21:47:02.297 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-15T21:47:02.297 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-15T21:47:02.297 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_skip_data_digest": "false", 2024-09-15T21:47:02.297 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_smart_report_timeout": "5", 2024-09-15T21:47:02.297 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_snap_trim_cost": "1048576", 2024-09-15T21:47:02.297 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_snap_trim_priority": "5", 2024-09-15T21:47:02.297 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-15T21:47:02.298 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-15T21:47:02.298 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-15T21:47:02.298 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-15T21:47:02.298 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-15T21:47:02.298 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-15T21:47:02.298 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-15T21:47:02.298 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_target_transaction_size": "30", 2024-09-15T21:47:02.298 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-15T21:47:02.298 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-15T21:47:02.298 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-15T21:47:02.298 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-15T21:47:02.298 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-15T21:47:02.298 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-15T21:47:02.298 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-15T21:47:02.299 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-15T21:47:02.299 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-15T21:47:02.299 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-15T21:47:02.299 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_tracing": "false", 2024-09-15T21:47:02.299 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_use_stale_snap": "false", 2024-09-15T21:47:02.299 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-15T21:47:02.299 INFO:teuthology.orchestra.run.smithi100.stdout: "osdc_blkin_trace_all": "false", 2024-09-15T21:47:02.299 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_kill_at": "0", 2024-09-15T21:47:02.299 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_max_join_drift": "10", 2024-09-15T21:47:02.299 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_min": "500", 2024-09-15T21:47:02.299 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_min_wait": "0.050000", 2024-09-15T21:47:02.299 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_propose_interval": "1.000000", 2024-09-15T21:47:02.299 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_service_trim_max": "500", 2024-09-15T21:47:02.300 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-15T21:47:02.300 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_service_trim_min": "250", 2024-09-15T21:47:02.300 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_stash_full_interval": "25", 2024-09-15T21:47:02.300 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_trim_max": "500", 2024-09-15T21:47:02.300 INFO:teuthology.orchestra.run.smithi100.stdout: "paxos_trim_min": "250", 2024-09-15T21:47:02.300 INFO:teuthology.orchestra.run.smithi100.stdout: "perf": "true", 2024-09-15T21:47:02.300 INFO:teuthology.orchestra.run.smithi100.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-15T21:47:02.300 INFO:teuthology.orchestra.run.smithi100.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-15T21:47:02.300 INFO:teuthology.orchestra.run.smithi100.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-15T21:47:02.300 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "-", 2024-09-15T21:47:02.300 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrv": "", 2024-09-15T21:47:02.300 INFO:teuthology.orchestra.run.smithi100.stdout: "public_bind_addr": "-", 2024-09-15T21:47:02.300 INFO:teuthology.orchestra.run.smithi100.stdout: "public_network": "", 2024-09-15T21:47:02.300 INFO:teuthology.orchestra.run.smithi100.stdout: "public_network_interface": "", 2024-09-15T21:47:02.301 INFO:teuthology.orchestra.run.smithi100.stdout: "qat_compressor_busy_polling": "false", 2024-09-15T21:47:02.301 INFO:teuthology.orchestra.run.smithi100.stdout: "qat_compressor_enabled": "false", 2024-09-15T21:47:02.301 INFO:teuthology.orchestra.run.smithi100.stdout: "qat_compressor_session_max_number": "256", 2024-09-15T21:47:02.301 INFO:teuthology.orchestra.run.smithi100.stdout: "rados_mon_op_timeout": "0", 2024-09-15T21:47:02.301 INFO:teuthology.orchestra.run.smithi100.stdout: "rados_osd_op_timeout": "0", 2024-09-15T21:47:02.301 INFO:teuthology.orchestra.run.smithi100.stdout: "rados_tracing": "false", 2024-09-15T21:47:02.301 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_atime_update_interval": "60", 2024-09-15T21:47:02.301 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-15T21:47:02.301 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_balance_parent_reads": "false", 2024-09-15T21:47:02.301 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_balance_snap_reads": "false", 2024-09-15T21:47:02.301 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_blkin_trace_all": "false", 2024-09-15T21:47:02.301 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-15T21:47:02.301 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-15T21:47:02.302 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_cache": "true", 2024-09-15T21:47:02.302 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-15T21:47:02.302 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-15T21:47:02.302 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-15T21:47:02.302 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-15T21:47:02.302 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_cache_policy": "writearound", 2024-09-15T21:47:02.302 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_cache_size": "33554432", 2024-09-15T21:47:02.302 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-15T21:47:02.302 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-15T21:47:02.302 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_clone_copy_on_read": "false", 2024-09-15T21:47:02.302 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_compression_hint": "none", 2024-09-15T21:47:02.302 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_concurrent_management_ops": "10", 2024-09-15T21:47:02.302 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-15T21:47:02.302 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_clone_format": "auto", 2024-09-15T21:47:02.303 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_data_pool": "", 2024-09-15T21:47:02.303 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_features": "61", 2024-09-15T21:47:02.303 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_format": "2", 2024-09-15T21:47:02.303 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_map_options": "", 2024-09-15T21:47:02.303 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_order": "22", 2024-09-15T21:47:02.303 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_pool": "rbd", 2024-09-15T21:47:02.303 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-15T21:47:02.303 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_stripe_count": "0", 2024-09-15T21:47:02.303 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_default_stripe_unit": "0", 2024-09-15T21:47:02.303 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-15T21:47:02.303 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-15T21:47:02.303 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-15T21:47:02.303 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_enable_alloc_hint": "true", 2024-09-15T21:47:02.303 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-15T21:47:02.304 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_io_scheduler": "simple", 2024-09-15T21:47:02.304 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-15T21:47:02.304 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-15T21:47:02.304 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-15T21:47:02.304 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-15T21:47:02.304 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-15T21:47:02.304 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-15T21:47:02.304 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-15T21:47:02.304 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-15T21:47:02.304 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-15T21:47:02.304 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_order": "24", 2024-09-15T21:47:02.304 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_pool": "", 2024-09-15T21:47:02.304 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_journal_splay_width": "4", 2024-09-15T21:47:02.305 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_localize_parent_reads": "false", 2024-09-15T21:47:02.305 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_localize_snap_reads": "false", 2024-09-15T21:47:02.305 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-15T21:47:02.305 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-15T21:47:02.305 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-15T21:47:02.305 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-15T21:47:02.305 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-15T21:47:02.305 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-15T21:47:02.305 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-15T21:47:02.305 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-15T21:47:02.305 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-15T21:47:02.305 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-15T21:47:02.305 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-15T21:47:02.306 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-15T21:47:02.306 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-15T21:47:02.306 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-15T21:47:02.306 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-15T21:47:02.306 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-15T21:47:02.306 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-15T21:47:02.306 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-15T21:47:02.306 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-15T21:47:02.306 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-15T21:47:02.306 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-15T21:47:02.306 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-15T21:47:02.306 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-15T21:47:02.306 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-15T21:47:02.306 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-15T21:47:02.307 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-15T21:47:02.307 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-15T21:47:02.307 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-15T21:47:02.307 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-15T21:47:02.307 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-15T21:47:02.307 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-15T21:47:02.307 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_mtime_update_interval": "60", 2024-09-15T21:47:02.307 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_non_blocking_aio": "true", 2024-09-15T21:47:02.307 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_op_thread_timeout": "60", 2024-09-15T21:47:02.307 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_op_threads": "1", 2024-09-15T21:47:02.307 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_parent_cache_enabled": "false", 2024-09-15T21:47:02.307 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-15T21:47:02.307 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-15T21:47:02.308 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-15T21:47:02.308 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_plugins": "", 2024-09-15T21:47:02.308 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_bps_burst": "0", 2024-09-15T21:47:02.308 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-15T21:47:02.308 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_bps_limit": "0", 2024-09-15T21:47:02.308 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_exclude_ops": "0", 2024-09-15T21:47:02.308 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_iops_burst": "0", 2024-09-15T21:47:02.308 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-15T21:47:02.308 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_iops_limit": "0", 2024-09-15T21:47:02.308 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-15T21:47:02.308 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-15T21:47:02.308 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-15T21:47:02.308 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-15T21:47:02.309 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-15T21:47:02.309 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-15T21:47:02.309 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-15T21:47:02.309 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-15T21:47:02.309 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-15T21:47:02.309 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-15T21:47:02.309 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-15T21:47:02.309 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-15T21:47:02.309 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-15T21:47:02.309 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-15T21:47:02.309 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_read_from_replica_policy": "default", 2024-09-15T21:47:02.309 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-15T21:47:02.309 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-15T21:47:02.310 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-15T21:47:02.310 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-15T21:47:02.310 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_skip_partial_discard": "true", 2024-09-15T21:47:02.310 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-15T21:47:02.310 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_tracing": "false", 2024-09-15T21:47:02.310 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_validate_names": "true", 2024-09-15T21:47:02.310 INFO:teuthology.orchestra.run.smithi100.stdout: "rbd_validate_pool": "true", 2024-09-15T21:47:02.310 INFO:teuthology.orchestra.run.smithi100.stdout: "restapi_base_url": "", 2024-09-15T21:47:02.310 INFO:teuthology.orchestra.run.smithi100.stdout: "restapi_log_level": "", 2024-09-15T21:47:02.310 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-15T21:47:02.311 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-15T21:47:02.311 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_acl_grants_max_num": "100", 2024-09-15T21:47:02.311 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_admin_entry": "admin", 2024-09-15T21:47:02.311 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-15T21:47:02.311 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_backend_store": "rados", 2024-09-15T21:47:02.311 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_barbican_url": "", 2024-09-15T21:47:02.311 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_beast_enable_async": "true", 2024-09-15T21:47:02.311 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_bucket_counters_cache": "false", 2024-09-15T21:47:02.311 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-15T21:47:02.311 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-15T21:47:02.311 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-15T21:47:02.311 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-15T21:47:02.311 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-15T21:47:02.312 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-15T21:47:02.312 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-15T21:47:02.312 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-15T21:47:02.312 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_cache_enabled": "true", 2024-09-15T21:47:02.312 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_cache_expiry_interval": "900", 2024-09-15T21:47:02.312 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_cache_lru_size": "25000", 2024-09-15T21:47:02.312 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_config_store": "rados", 2024-09-15T21:47:02.312 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_content_length_compat": "false", 2024-09-15T21:47:02.312 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_copy_obj_progress": "true", 2024-09-15T21:47:02.312 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-15T21:47:02.312 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_cors_rules_max_num": "100", 2024-09-15T21:47:02.312 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_cross_domain_policy": "", 2024-09-15T21:47:02.312 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-15T21:47:02.313 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_kmip_addr": "", 2024-09-15T21:47:02.313 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-15T21:47:02.313 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-15T21:47:02.313 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-15T21:47:02.313 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-15T21:47:02.313 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_kmip_password": "", 2024-09-15T21:47:02.313 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-15T21:47:02.313 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_kmip_username": "", 2024-09-15T21:47:02.313 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_require_ssl": "true", 2024-09-15T21:47:02.313 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-15T21:47:02.313 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-15T21:47:02.313 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-15T21:47:02.313 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-15T21:47:02.313 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-15T21:47:02.314 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-15T21:47:02.314 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-15T21:47:02.314 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-15T21:47:02.314 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-15T21:47:02.314 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-15T21:47:02.314 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-15T21:47:02.314 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-15T21:47:02.314 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-15T21:47:02.314 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-15T21:47:02.314 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-15T21:47:02.314 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_addr": "", 2024-09-15T21:47:02.314 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_auth": "token", 2024-09-15T21:47:02.314 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_namespace": "", 2024-09-15T21:47:02.314 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_prefix": "", 2024-09-15T21:47:02.315 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-15T21:47:02.315 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-15T21:47:02.315 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-15T21:47:02.315 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-15T21:47:02.315 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_token_file": "", 2024-09-15T21:47:02.315 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-15T21:47:02.315 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_curl_buffersize": "524288", 2024-09-15T21:47:02.315 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-15T21:47:02.315 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_curl_low_speed_time": "300", 2024-09-15T21:47:02.315 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-15T21:47:02.315 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-15T21:47:02.315 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-15T21:47:02.315 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-15T21:47:02.316 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-15T21:47:02.316 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-15T21:47:02.316 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-15T21:47:02.316 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-15T21:47:02.316 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-15T21:47:02.316 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-15T21:47:02.316 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-15T21:47:02.316 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_d4n_port": "6379", 2024-09-15T21:47:02.316 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-15T21:47:02.316 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_data_log_changes_size": "1000", 2024-09-15T21:47:02.316 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_data_log_num_shards": "128", 2024-09-15T21:47:02.316 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-15T21:47:02.316 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_data_log_window": "30", 2024-09-15T21:47:02.316 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-15T21:47:02.317 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-15T21:47:02.317 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-15T21:47:02.317 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-15T21:47:02.317 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-15T21:47:02.317 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-15T21:47:02.317 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-15T21:47:02.317 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-15T21:47:02.317 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-15T21:47:02.317 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-15T21:47:02.317 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-15T21:47:02.317 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-15T21:47:02.317 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-15T21:47:02.317 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-15T21:47:02.318 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-15T21:47:02.318 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-15T21:47:02.318 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-15T21:47:02.318 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-15T21:47:02.318 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-15T21:47:02.318 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-15T21:47:02.318 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-15T21:47:02.318 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-15T21:47:02.318 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-15T21:47:02.318 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-15T21:47:02.318 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-15T21:47:02.318 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dns_name": "", 2024-09-15T21:47:02.318 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dns_s3website_name": "", 2024-09-15T21:47:02.319 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_dynamic_resharding": "true", 2024-09-15T21:47:02.319 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-15T21:47:02.319 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_enable_gc_threads": "true", 2024-09-15T21:47:02.319 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_enable_lc_threads": "true", 2024-09-15T21:47:02.319 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_enable_ops_log": "false", 2024-09-15T21:47:02.319 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_enable_quota_threads": "true", 2024-09-15T21:47:02.319 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_enable_static_website": "false", 2024-09-15T21:47:02.319 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_enable_usage_log": "false", 2024-09-15T21:47:02.319 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_enforce_swift_acls": "true", 2024-09-15T21:47:02.319 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_exit_timeout_secs": "120", 2024-09-15T21:47:02.319 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_expose_bucket": "false", 2024-09-15T21:47:02.319 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_extended_http_attrs": "", 2024-09-15T21:47:02.319 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_filter": "none", 2024-09-15T21:47:02.319 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-15T21:47:02.320 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_frontends": "beast port=7480", 2024-09-15T21:47:02.320 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-15T21:47:02.320 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_gc_max_deferred": "50", 2024-09-15T21:47:02.320 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-15T21:47:02.320 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_gc_max_objs": "32", 2024-09-15T21:47:02.320 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-15T21:47:02.320 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-15T21:47:02.320 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-15T21:47:02.320 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-15T21:47:02.320 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_gc_processor_period": "3600", 2024-09-15T21:47:02.320 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-15T21:47:02.320 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-15T21:47:02.320 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-15T21:47:02.321 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-15T21:47:02.321 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_init_timeout": "300", 2024-09-15T21:47:02.321 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-15T21:47:02.321 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-15T21:47:02.321 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_kafka_connection_idle": "300", 2024-09-15T21:47:02.321 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-15T21:47:02.321 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-15T21:47:02.321 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-15T21:47:02.321 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-15T21:47:02.321 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-15T21:47:02.321 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_admin_domain": "", 2024-09-15T21:47:02.321 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_admin_password": "", 2024-09-15T21:47:02.321 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_admin_password_path": "", 2024-09-15T21:47:02.322 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_admin_project": "", 2024-09-15T21:47:02.322 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_admin_tenant": "", 2024-09-15T21:47:02.322 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_admin_token": "", 2024-09-15T21:47:02.322 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_admin_token_path": "", 2024-09-15T21:47:02.322 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_admin_user": "", 2024-09-15T21:47:02.322 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_api_version": "2", 2024-09-15T21:47:02.322 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_barbican_domain": "", 2024-09-15T21:47:02.322 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_barbican_password": "", 2024-09-15T21:47:02.322 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_barbican_project": "", 2024-09-15T21:47:02.322 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-15T21:47:02.322 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_barbican_user": "", 2024-09-15T21:47:02.322 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-15T21:47:02.322 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-15T21:47:02.322 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-15T21:47:02.323 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-15T21:47:02.323 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-15T21:47:02.323 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_url": "", 2024-09-15T21:47:02.323 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-15T21:47:02.323 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_lc_debug_interval": "-1", 2024-09-15T21:47:02.323 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_lc_lock_max_time": "90", 2024-09-15T21:47:02.323 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_lc_max_objs": "32", 2024-09-15T21:47:02.323 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_lc_max_rules": "1000", 2024-09-15T21:47:02.323 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_lc_max_worker": "3", 2024-09-15T21:47:02.323 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-15T21:47:02.323 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_lc_thread_delay": "0", 2024-09-15T21:47:02.323 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-15T21:47:02.324 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ldap_dnattr": "uid", 2024-09-15T21:47:02.324 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-15T21:47:02.324 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ldap_searchfilter": "", 2024-09-15T21:47:02.324 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-15T21:47:02.324 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ldap_uri": "", 2024-09-15T21:47:02.324 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-15T21:47:02.324 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-15T21:47:02.324 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-15T21:47:02.324 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_log_http_headers": "", 2024-09-15T21:47:02.324 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-15T21:47:02.324 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-15T21:47:02.324 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_log_object_name_utc": "false", 2024-09-15T21:47:02.324 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-15T21:47:02.325 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-15T21:47:02.325 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_attr_name_len": "0", 2024-09-15T21:47:02.325 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_attr_size": "0", 2024-09-15T21:47:02.325 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-15T21:47:02.325 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_chunk_size": "4194304", 2024-09-15T21:47:02.325 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-15T21:47:02.325 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-15T21:47:02.325 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-15T21:47:02.325 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_listing_results": "1000", 2024-09-15T21:47:02.325 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_notify_retries": "10", 2024-09-15T21:47:02.325 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-15T21:47:02.325 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_put_param_size": "1048576", 2024-09-15T21:47:02.325 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_put_size": "5368709120", 2024-09-15T21:47:02.326 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_max_slo_entries": "1000", 2024-09-15T21:47:02.326 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_md_log_max_shards": "64", 2024-09-15T21:47:02.326 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-15T21:47:02.326 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-15T21:47:02.326 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-15T21:47:02.326 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-15T21:47:02.326 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_mp_lock_max_time": "600", 2024-09-15T21:47:02.326 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-15T21:47:02.326 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-15T21:47:02.326 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-15T21:47:02.327 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-15T21:47:02.327 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-15T21:47:02.327 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-15T21:47:02.327 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-15T21:47:02.327 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-15T21:47:02.327 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_max_gc": "300", 2024-09-15T21:47:02.327 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-15T21:47:02.327 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-15T21:47:02.327 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-15T21:47:02.327 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-15T21:47:02.327 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-15T21:47:02.327 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-15T21:47:02.327 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-15T21:47:02.328 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_num_async_rados_threads": "32", 2024-09-15T21:47:02.328 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_num_control_oids": "8", 2024-09-15T21:47:02.328 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_numa_node": "-1", 2024-09-15T21:47:02.328 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-15T21:47:02.328 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-15T21:47:02.328 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_objexp_chunk_size": "100", 2024-09-15T21:47:02.328 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_objexp_gc_interval": "600", 2024-09-15T21:47:02.328 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-15T21:47:02.328 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-15T21:47:02.328 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-15T21:47:02.328 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_op_thread_timeout": "600", 2024-09-15T21:47:02.328 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_op_tracing": "false", 2024-09-15T21:47:02.328 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_opa_token": "", 2024-09-15T21:47:02.328 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_opa_url": "", 2024-09-15T21:47:02.329 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_opa_verify_ssl": "true", 2024-09-15T21:47:02.329 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-15T21:47:02.329 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-15T21:47:02.329 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ops_log_rados": "false", 2024-09-15T21:47:02.329 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_ops_log_socket_path": "", 2024-09-15T21:47:02.329 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-15T21:47:02.329 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-15T21:47:02.329 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-15T21:47:02.329 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_period_push_interval": "2.000000", 2024-09-15T21:47:02.329 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-15T21:47:02.329 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-15T21:47:02.329 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-15T21:47:02.329 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-15T21:47:02.330 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_posix_cache_lanes": "3", 2024-09-15T21:47:02.330 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-15T21:47:02.330 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-15T21:47:02.330 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_posix_cache_partitions": "3", 2024-09-15T21:47:02.330 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-15T21:47:02.330 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_print_continue": "true", 2024-09-15T21:47:02.330 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-15T21:47:02.330 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-15T21:47:02.330 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-15T21:47:02.330 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-15T21:47:02.330 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-15T21:47:02.330 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_rados_tracing": "false", 2024-09-15T21:47:02.330 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_realm": "", 2024-09-15T21:47:02.331 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_realm_id": "", 2024-09-15T21:47:02.331 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-15T21:47:02.331 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_region": "", 2024-09-15T21:47:02.331 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-15T21:47:02.331 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-15T21:47:02.331 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-15T21:47:02.331 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-15T21:47:02.331 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_request_uri": "", 2024-09-15T21:47:02.331 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_reshard_batch_size": "64", 2024-09-15T21:47:02.331 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-15T21:47:02.331 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_reshard_max_aio": "128", 2024-09-15T21:47:02.331 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_reshard_num_logs": "16", 2024-09-15T21:47:02.331 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_reshard_thread_interval": "600", 2024-09-15T21:47:02.332 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_resolve_cname": "false", 2024-09-15T21:47:02.332 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-15T21:47:02.332 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_run_sync_thread": "true", 2024-09-15T21:47:02.332 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-15T21:47:02.332 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-15T21:47:02.332 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-15T21:47:02.332 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-15T21:47:02.332 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-15T21:47:02.332 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-15T21:47:02.332 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-15T21:47:02.332 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-15T21:47:02.332 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-15T21:47:02.332 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_scheduler_type": "throttler", 2024-09-15T21:47:02.333 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_script_uri": "", 2024-09-15T21:47:02.333 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_service_provider_name": "", 2024-09-15T21:47:02.333 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-15T21:47:02.333 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sts_client_id": "", 2024-09-15T21:47:02.333 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sts_client_secret": "", 2024-09-15T21:47:02.333 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sts_entry": "sts", 2024-09-15T21:47:02.333 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sts_key": "sts", 2024-09-15T21:47:02.333 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-15T21:47:02.333 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sts_min_session_duration": "900", 2024-09-15T21:47:02.333 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sts_token_introspection_url": "", 2024-09-15T21:47:02.336 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_account_in_url": "false", 2024-09-15T21:47:02.336 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_auth_entry": "auth", 2024-09-15T21:47:02.336 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_auth_url": "", 2024-09-15T21:47:02.336 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_custom_header": "", 2024-09-15T21:47:02.337 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-15T21:47:02.337 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_need_stats": "true", 2024-09-15T21:47:02.337 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_tenant_name": "", 2024-09-15T21:47:02.337 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_token_expiration": "86400", 2024-09-15T21:47:02.337 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_url": "", 2024-09-15T21:47:02.337 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_url_prefix": "swift", 2024-09-15T21:47:02.337 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-15T21:47:02.337 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-15T21:47:02.337 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-15T21:47:02.337 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_lease_period": "120", 2024-09-15T21:47:02.337 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-15T21:47:02.337 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-15T21:47:02.337 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-15T21:47:02.338 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-15T21:47:02.338 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-15T21:47:02.338 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-15T21:47:02.338 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-15T21:47:02.338 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-15T21:47:02.338 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-15T21:47:02.338 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_thread_pool_size": "512", 2024-09-15T21:47:02.338 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-15T21:47:02.338 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-15T21:47:02.338 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-15T21:47:02.338 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-15T21:47:02.338 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_torrent_comment": "", 2024-09-15T21:47:02.339 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_torrent_createby": "", 2024-09-15T21:47:02.339 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_torrent_encoding": "", 2024-09-15T21:47:02.339 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_torrent_flag": "false", 2024-09-15T21:47:02.339 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-15T21:47:02.339 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_torrent_origin": "", 2024-09-15T21:47:02.339 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-15T21:47:02.339 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_torrent_tracker": "", 2024-09-15T21:47:02.339 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_trust_forwarded_https": "false", 2024-09-15T21:47:02.339 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-15T21:47:02.339 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-15T21:47:02.339 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_usage_max_shards": "32", 2024-09-15T21:47:02.339 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_usage_max_user_shards": "1", 2024-09-15T21:47:02.339 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_use_opa_authz": "false", 2024-09-15T21:47:02.339 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_counters_cache": "false", 2024-09-15T21:47:02.340 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-15T21:47:02.340 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-15T21:47:02.340 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-15T21:47:02.340 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_max_buckets": "1000", 2024-09-15T21:47:02.340 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_policies_max_num": "100", 2024-09-15T21:47:02.340 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-15T21:47:02.340 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-15T21:47:02.340 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-15T21:47:02.340 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-15T21:47:02.340 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_user_unique_email": "true", 2024-09-15T21:47:02.340 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_verify_ssl": "true", 2024-09-15T21:47:02.340 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-15T21:47:02.340 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_zone": "", 2024-09-15T21:47:02.341 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_zone_id": "", 2024-09-15T21:47:02.341 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-15T21:47:02.341 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_zonegroup": "", 2024-09-15T21:47:02.341 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_zonegroup_id": "", 2024-09-15T21:47:02.341 INFO:teuthology.orchestra.run.smithi100.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-15T21:47:02.341 INFO:teuthology.orchestra.run.smithi100.stdout: "rgwlc_auto_session_clear": "true", 2024-09-15T21:47:02.341 INFO:teuthology.orchestra.run.smithi100.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-15T21:47:02.341 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_block_size": "4096", 2024-09-15T21:47:02.341 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-15T21:47:02.341 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-15T21:47:02.341 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-15T21:47:02.341 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-15T21:47:02.341 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-15T21:47:02.341 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_cache_size": "536870912", 2024-09-15T21:47:02.342 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-15T21:47:02.342 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-15T21:47:02.342 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-15T21:47:02.342 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-15T21:47:02.342 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-15T21:47:02.342 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-15T21:47:02.342 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-15T21:47:02.342 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-15T21:47:02.342 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_index_type": "binary_search", 2024-09-15T21:47:02.342 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-15T21:47:02.342 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-15T21:47:02.342 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_partition_filters": "false", 2024-09-15T21:47:02.342 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_perf": "false", 2024-09-15T21:47:02.342 INFO:teuthology.orchestra.run.smithi100.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-15T21:47:02.343 INFO:teuthology.orchestra.run.smithi100.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-15T21:47:02.343 INFO:teuthology.orchestra.run.smithi100.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-15T21:47:02.343 INFO:teuthology.orchestra.run.smithi100.stdout: "run_dir": "/var/run/ceph", 2024-09-15T21:47:02.343 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_block_create": "true", 2024-09-15T21:47:02.343 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_cache_lru_size": "67108864", 2024-09-15T21:47:02.343 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-15T21:47:02.343 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-15T21:47:02.343 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_default_max_object_size": "16777216", 2024-09-15T21:47:02.343 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-15T21:47:02.343 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_device_size": "53687091200", 2024-09-15T21:47:02.343 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-15T21:47:02.343 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_full_integrity_check": "false", 2024-09-15T21:47:02.343 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_journal_batch_capacity": "16", 2024-09-15T21:47:02.344 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-15T21:47:02.344 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-15T21:47:02.344 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-15T21:47:02.344 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_main_device_type": "SSD", 2024-09-15T21:47:02.344 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-15T21:47:02.344 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-15T21:47:02.344 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-15T21:47:02.344 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-15T21:47:02.344 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-15T21:47:02.344 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-15T21:47:02.344 INFO:teuthology.orchestra.run.smithi100.stdout: "seastore_segment_size": "67108864", 2024-09-15T21:47:02.344 INFO:teuthology.orchestra.run.smithi100.stdout: "set_keepcaps": "false", 2024-09-15T21:47:02.344 INFO:teuthology.orchestra.run.smithi100.stdout: "setgroup": "", 2024-09-15T21:47:02.344 INFO:teuthology.orchestra.run.smithi100.stdout: "setuser": "", 2024-09-15T21:47:02.345 INFO:teuthology.orchestra.run.smithi100.stdout: "setuser_match_path": "", 2024-09-15T21:47:02.345 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-15T21:47:02.345 INFO:teuthology.orchestra.run.smithi100.stdout: "thp": "false", 2024-09-15T21:47:02.345 INFO:teuthology.orchestra.run.smithi100.stdout: "threadpool_default_timeout": "60", 2024-09-15T21:47:02.345 INFO:teuthology.orchestra.run.smithi100.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-15T21:47:02.345 INFO:teuthology.orchestra.run.smithi100.stdout: "throttler_perf_counter": "true" 2024-09-15T21:47:02.345 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:02.347 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-15T21:47:02.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.443+0000 7f6a9b79f640 1 Processor -- start 2024-09-15T21:47:02.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.443+0000 7f6a9b79f640 1 -- start start 2024-09-15T21:47:02.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.444+0000 7f6a9b79f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6a94155210 0x7f6a9415ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.444+0000 7f6a9b79f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6a94072080 0x7f6a9406f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.444+0000 7f6a9b79f640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a940ca090 con 0x7f6a9406fe50 2024-09-15T21:47:02.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.444+0000 7f6a9b79f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f6a9415f4d0 con 0x7f6a94072080 2024-09-15T21:47:02.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.444+0000 7f6a9b79f640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f6a9415bf80 con 0x7f6a94155210 2024-09-15T21:47:02.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.444+0000 7f6a98d3a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6a94072080 0x7f6a9406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.444+0000 7f6a98d3a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6a94072080 0x7f6a9406f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48920/0 (socket says 172.21.15.100:48920) 2024-09-15T21:47:02.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.444+0000 7f6a98d3a640 1 -- 172.21.15.100:0/4091632347 learned_addr learned my addr 172.21.15.100:0/4091632347 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:02.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.445+0000 7f6a9953b640 1 --2- 172.21.15.100:0/4091632347 >> v2:172.21.15.122:3300/0 conn(0x7f6a94155210 0x7f6a9415ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.445+0000 7f6a98d3a640 1 -- 172.21.15.100:0/4091632347 >> v2:172.21.15.122:3300/0 conn(0x7f6a94155210 msgr2=0x7f6a9415ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:02.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.445+0000 7f6a98d3a640 1 --2- 172.21.15.100:0/4091632347 >> v2:172.21.15.122:3300/0 conn(0x7f6a94155210 0x7f6a9415ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.445+0000 7f6a98d3a640 1 -- 172.21.15.100:0/4091632347 >> v1:172.21.15.122:6789/0 conn(0x7f6a9406fe50 legacy=0x7f6a9406be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:02.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.445+0000 7f6a98d3a640 1 -- 172.21.15.100:0/4091632347 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a94072550 con 0x7f6a94072080 2024-09-15T21:47:02.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.445+0000 7f6a98d3a640 1 --2- 172.21.15.100:0/4091632347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94072080 0x7f6a9406f910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6a940ce9f0 tx=0x7f6a8802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=368a4fd88adaeed server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.445+0000 7f6a93fff640 1 -- 172.21.15.100:0/4091632347 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88037070 con 0x7f6a94072080 2024-09-15T21:47:02.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.445+0000 7f6a93fff640 1 -- 172.21.15.100:0/4091632347 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a88032440 con 0x7f6a94072080 2024-09-15T21:47:02.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.446+0000 7f6a93fff640 1 -- 172.21.15.100:0/4091632347 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88032810 con 0x7f6a94072080 2024-09-15T21:47:02.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.446+0000 7f6a9b79f640 1 -- 172.21.15.100:0/4091632347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94072080 msgr2=0x7f6a9406f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.446+0000 7f6a9b79f640 1 --2- 172.21.15.100:0/4091632347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94072080 0x7f6a9406f910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6a940ce9f0 tx=0x7f6a8802a2c0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.446+0000 7f6a9b79f640 1 -- 172.21.15.100:0/4091632347 shutdown_connections 2024-09-15T21:47:02.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.446+0000 7f6a9b79f640 1 --2- 172.21.15.100:0/4091632347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94072080 0x7f6a9406f910 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.449 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.446+0000 7f6a9b79f640 1 --2- 172.21.15.100:0/4091632347 >> v2:172.21.15.122:3300/0 conn(0x7f6a94155210 0x7f6a9415ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.446+0000 7f6a9b79f640 1 -- 172.21.15.100:0/4091632347 >> 172.21.15.100:0/4091632347 conn(0x7f6a940cd9a0 msgr2=0x7f6a940cdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:02.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.446+0000 7f6a9b79f640 1 -- 172.21.15.100:0/4091632347 shutdown_connections 2024-09-15T21:47:02.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.446+0000 7f6a9b79f640 1 -- 172.21.15.100:0/4091632347 wait complete. 2024-09-15T21:47:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.447+0000 7f6a9b79f640 1 Processor -- start 2024-09-15T21:47:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.447+0000 7f6a9b79f640 1 -- start start 2024-09-15T21:47:02.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.448+0000 7f6a9b79f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6a9406fe50 0x7f6a94155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.448+0000 7f6a9b79f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a94072080 0x7f6a94156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.448+0000 7f6a9b79f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94155210 0x7f6a94072e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.448+0000 7f6a9b79f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9415eba0 con 0x7f6a94155210 2024-09-15T21:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.448+0000 7f6a9b79f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9415e8c0 con 0x7f6a94072080 2024-09-15T21:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.448+0000 7f6a9b79f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6a9415ee80 con 0x7f6a9406fe50 2024-09-15T21:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.448+0000 7f6a99d3c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94155210 0x7f6a94072e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.448+0000 7f6a99d3c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94155210 0x7f6a94072e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:48922/0 (socket says 172.21.15.100:48922) 2024-09-15T21:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.448+0000 7f6a99d3c640 1 -- 172.21.15.100:0/1795583471 learned_addr learned my addr 172.21.15.100:0/1795583471 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.448+0000 7f6a98d3a640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a94072080 0x7f6a94156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.448+0000 7f6a9953b640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6a9406fe50 0x7f6a94155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.448+0000 7f6a99d3c640 1 -- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6a9406fe50 msgr2=0x7f6a94155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.448+0000 7f6a99d3c640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6a9406fe50 0x7f6a94155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.448+0000 7f6a99d3c640 1 -- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a94072080 msgr2=0x7f6a94156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.448+0000 7f6a99d3c640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a94072080 0x7f6a94156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.448+0000 7f6a99d3c640 1 -- 172.21.15.100:0/1795583471 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a94156270 con 0x7f6a94155210 2024-09-15T21:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.448+0000 7f6a99d3c640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94155210 0x7f6a94072e90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00b3e0 tx=0x7f6a8c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.449+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c015020 con 0x7f6a94155210 2024-09-15T21:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.449+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a8c011830 con 0x7f6a94155210 2024-09-15T21:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.449+0000 7f6a9b79f640 1 -- 172.21.15.100:0/1795583471 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a94155f90 con 0x7f6a94155210 2024-09-15T21:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.449+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8c011c60 con 0x7f6a94155210 2024-09-15T21:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.449+0000 7f6a9b79f640 1 -- 172.21.15.100:0/1795583471 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a941f45d0 con 0x7f6a94155210 2024-09-15T21:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.450+0000 7f6a9b79f640 1 -- 172.21.15.100:0/1795583471 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6a9415f160 con 0x7f6a94155210 2024-09-15T21:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.450+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6a8c018020 con 0x7f6a94155210 2024-09-15T21:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.450+0000 7f6a91ffb640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6a8006c2b0 0x7f6a8006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.451+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6a8c08aee0 con 0x7f6a94155210 2024-09-15T21:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.451+0000 7f6a9953b640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6a8006c2b0 0x7f6a8006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.451+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1289 (secure 0 0 0) 0x7f6a8c01f030 con 0x7f6a94155210 2024-09-15T21:47:02.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.451+0000 7f6a9b79f640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a941e51e0 0x7f6a941f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.451+0000 7f6a9b79f640 1 -- 172.21.15.100:0/1795583471 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6a940cdf20 con 0x7f6a941e51e0 2024-09-15T21:47:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.451+0000 7f6a9953b640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6a8006c2b0 0x7f6a8006e770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6a84003b80 tx=0x7f6a840098d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.451+0000 7f6a98d3a640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a941e51e0 0x7f6a941f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.451+0000 7f6a98d3a640 1 -- 172.21.15.100:0/1795583471 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- command(tid 2: {"prefix": "get_command_descriptions"}) -- 0x7f6a940caeb0 con 0x7f6a941e51e0 2024-09-15T21:47:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.451+0000 7f6a98d3a640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a941e51e0 0x7f6a941f4970 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6a94068650 tx=0x7f6a880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.451+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88043070 con 0x7f6a941e51e0 2024-09-15T21:47:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.452+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 <== mon.0 v2:172.21.15.100:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6a88032e40 con 0x7f6a941e51e0 2024-09-15T21:47:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.452+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a941e51e0 msgr2=0x7f6a941f4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.452+0000 7f6a91ffb640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a941e51e0 0x7f6a941f4970 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6a94068650 tx=0x7f6a880046c0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.452+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a941e51e0 msgr2=0x7f6a941f4970 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:47:02.456 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.452+0000 7f6a91ffb640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a941e51e0 0x7f6a941f4970 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.458+0000 7f6a9b79f640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94159eb0 0x7f6a9415a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.458+0000 7f6a9b79f640 1 -- 172.21.15.100:0/1795583471 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6a94077360 con 0x7f6a94159eb0 2024-09-15T21:47:02.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.458+0000 7f6a98d3a640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94159eb0 0x7f6a9415a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.458+0000 7f6a98d3a640 1 -- 172.21.15.100:0/1795583471 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- command(tid 3: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) -- 0x7f6a941573d0 con 0x7f6a94159eb0 2024-09-15T21:47:02.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.458+0000 7f6a98d3a640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94159eb0 0x7f6a9415a290 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6a8802a300 tx=0x7f6a88008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.460 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.458+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88037040 con 0x7f6a94159eb0 2024-09-15T21:47:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.459+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 <== mon.0 v2:172.21.15.100:3300/0 2 ==== command_reply(tid 3: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f6a88003a00 con 0x7f6a94159eb0 2024-09-15T21:47:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.459+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94159eb0 msgr2=0x7f6a9415a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.459+0000 7f6a91ffb640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94159eb0 0x7f6a9415a290 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6a8802a300 tx=0x7f6a88008030 comp rx=0 tx=0).stop 2024-09-15T21:47:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.459+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94159eb0 msgr2=0x7f6a9415a290 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:47:02.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.459+0000 7f6a91ffb640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94159eb0 0x7f6a9415a290 secure :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6a8802a300 tx=0x7f6a88008030 comp rx=0 tx=0).stop 2024-09-15T21:47:02.462 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a: {} 2024-09-15T21:47:02.463 INFO:teuthology.orchestra.run.smithi100.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T21:47:02.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.460+0000 7f6a9b79f640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a94153c60 0x7f6a94154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.460+0000 7f6a9b79f640 1 -- 172.21.15.100:0/1795583471 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a940733d0 con 0x7f6a94153c60 2024-09-15T21:47:02.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.460+0000 7f6a98d3a640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a94153c60 0x7f6a94154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.460+0000 7f6a98d3a640 1 -- 172.21.15.100:0/1795583471 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- command(tid 4: {"prefix": "get_command_descriptions"}) -- 0x7f6a941545b0 con 0x7f6a94153c60 2024-09-15T21:47:02.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.460+0000 7f6a98d3a640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a94153c60 0x7f6a94154070 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6a940774e0 tx=0x7f6a880046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.461+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8805f070 con 0x7f6a94153c60 2024-09-15T21:47:02.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.461+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 <== mon.1 v2:172.21.15.122:3300/0 2 ==== command_reply(tid 4: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6a88059460 con 0x7f6a94153c60 2024-09-15T21:47:02.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.461+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a94153c60 msgr2=0x7f6a94154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.461+0000 7f6a91ffb640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a94153c60 0x7f6a94154070 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6a940774e0 tx=0x7f6a880046c0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.461+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a94153c60 msgr2=0x7f6a94154070 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:47:02.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.461+0000 7f6a91ffb640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a94153c60 0x7f6a94154070 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.461+0000 7f6a99d3c640 1 -- 172.21.15.100:0/1795583471 reap_dead start 2024-09-15T21:47:02.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.467+0000 7f6a9b79f640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a94156e90 0x7f6a94155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.467+0000 7f6a9b79f640 1 -- 172.21.15.100:0/1795583471 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6a942035c0 con 0x7f6a94156e90 2024-09-15T21:47:02.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.467+0000 7f6a98d3a640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a94156e90 0x7f6a94155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.467+0000 7f6a98d3a640 1 -- 172.21.15.100:0/1795583471 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- command(tid 5: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) -- 0x7f6a94203790 con 0x7f6a94156e90 2024-09-15T21:47:02.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.467+0000 7f6a98d3a640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a94156e90 0x7f6a94155940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6a88003630 tx=0x7f6a8804f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.468+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8806a070 con 0x7f6a94156e90 2024-09-15T21:47:02.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.469+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 <== mon.1 v2:172.21.15.122:3300/0 2 ==== command_reply(tid 5: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f6a88064450 con 0x7f6a94156e90 2024-09-15T21:47:02.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.469+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a94156e90 msgr2=0x7f6a94155940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.469+0000 7f6a91ffb640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a94156e90 0x7f6a94155940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6a88003630 tx=0x7f6a8804f000 comp rx=0 tx=0).stop 2024-09-15T21:47:02.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.469+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a94156e90 msgr2=0x7f6a94155940 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:47:02.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.469+0000 7f6a91ffb640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6a94156e90 0x7f6a94155940 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6a88003630 tx=0x7f6a8804f000 comp rx=0 tx=0).stop 2024-09-15T21:47:02.472 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b: {} 2024-09-15T21:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T21:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.469+0000 7f6a9b79f640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6a941578c0 0x7f6a94157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.469+0000 7f6a9b79f640 1 -- 172.21.15.100:0/1795583471 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6a941581e0 con 0x7f6a941578c0 2024-09-15T21:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.470+0000 7f6a98d3a640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6a941578c0 0x7f6a94157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.470+0000 7f6a98d3a640 1 -- 172.21.15.100:0/1795583471 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7f6a94158390 con 0x7f6a941578c0 2024-09-15T21:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.470+0000 7f6a98d3a640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6a941578c0 0x7f6a94157ca0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6a88004770 tx=0x7f6a880046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.470+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88037020 con 0x7f6a941578c0 2024-09-15T21:47:02.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.471+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 <== mon.2 v2:172.21.15.122:3301/0 2 ==== command_reply(tid 6: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6a88006450 con 0x7f6a941578c0 2024-09-15T21:47:02.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.471+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6a941578c0 msgr2=0x7f6a94157ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.471+0000 7f6a91ffb640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6a941578c0 0x7f6a94157ca0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6a88004770 tx=0x7f6a880046c0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.471+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6a941578c0 msgr2=0x7f6a94157ca0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:47:02.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.471+0000 7f6a91ffb640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6a941578c0 0x7f6a94157ca0 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6a88004770 tx=0x7f6a880046c0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.476+0000 7f6a9b79f640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6a941e76c0 0x7f6a941e7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.476+0000 7f6a9b79f640 1 -- 172.21.15.100:0/1795583471 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6a941585a0 con 0x7f6a941e76c0 2024-09-15T21:47:02.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.477+0000 7f6a98d3a640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6a941e76c0 0x7f6a941e7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.477+0000 7f6a98d3a640 1 -- 172.21.15.100:0/1795583471 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=0"]}) -- 0x7f6a941e7fe0 con 0x7f6a941e76c0 2024-09-15T21:47:02.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.477+0000 7f6a98d3a640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6a941e76c0 0x7f6a941e7aa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6a88003630 tx=0x7f6a88054000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.477+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a88037030 con 0x7f6a941e76c0 2024-09-15T21:47:02.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.479+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 <== mon.2 v2:172.21.15.122:3301/0 2 ==== command_reply(tid 7: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f6a88081070 con 0x7f6a941e76c0 2024-09-15T21:47:02.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.479+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6a941e76c0 msgr2=0x7f6a941e7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.479+0000 7f6a91ffb640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6a941e76c0 0x7f6a941e7aa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6a88003630 tx=0x7f6a88054000 comp rx=0 tx=0).stop 2024-09-15T21:47:02.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.479+0000 7f6a91ffb640 1 -- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6a941e76c0 msgr2=0x7f6a941e7aa0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:47:02.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.479+0000 7f6a91ffb640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6a941e76c0 0x7f6a941e7aa0 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6a88003630 tx=0x7f6a88054000 comp rx=0 tx=0).stop 2024-09-15T21:47:02.481 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c: {} 2024-09-15T21:47:02.482 INFO:teuthology.orchestra.run.smithi100.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T21:47:02.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.480+0000 7f6a9b79f640 1 -- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6a8006c2b0 msgr2=0x7f6a8006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.482 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.480+0000 7f6a9b79f640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6a8006c2b0 0x7f6a8006e770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6a84003b80 tx=0x7f6a840098d0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.480+0000 7f6a9b79f640 1 -- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94155210 msgr2=0x7f6a94072e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.480+0000 7f6a9b79f640 1 --2- 172.21.15.100:0/1795583471 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6a94155210 0x7f6a94072e90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6a8c00b3e0 tx=0x7f6a8c00b8a0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.480+0000 7f6a99d3c640 1 -- 172.21.15.100:0/1795583471 reap_dead start 2024-09-15T21:47:02.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.480+0000 7f6a9b79f640 1 -- 172.21.15.100:0/1795583471 shutdown_connections 2024-09-15T21:47:02.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.480+0000 7f6a9b79f640 1 -- 172.21.15.100:0/1795583471 >> 172.21.15.100:0/1795583471 conn(0x7f6a940cd9a0 msgr2=0x7f6a940c7150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:02.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.480+0000 7f6a9b79f640 1 -- 172.21.15.100:0/1795583471 shutdown_connections 2024-09-15T21:47:02.483 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.480+0000 7f6a9b79f640 1 -- 172.21.15.100:0/1795583471 wait complete. 2024-09-15T21:47:02.492 DEBUG:teuthology.orchestra.run.smithi122:> type ceph-objectstore-tool 2024-09-15T21:47:02.510 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T21:47:02.510 DEBUG:teuthology.orchestra.run.smithi100:> type ceph-objectstore-tool 2024-09-15T21:47:02.525 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T21:47:02.525 INFO:teuthology.run_tasks:Running task workunit... 2024-09-15T21:47:02.569 INFO:tasks.workunit:Pulling workunits from ref ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-15T21:47:02.569 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-15T21:47:02.569 INFO:tasks.workunit:timeout=3h 2024-09-15T21:47:02.569 INFO:tasks.workunit:cleanup=True 2024-09-15T21:47:02.570 DEBUG:teuthology.orchestra.run.smithi100:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-15T21:47:02.570 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-15T21:47:02.570 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:47:02.570 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:47:02.571 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-15T21:47:02.571 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T21:47:02.571 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-15T21:47:02.572 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-15T21:47:02.572 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:47:02.572 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-15T21:47:02.572 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:47:02.573 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-15T21:47:02.573 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:47:02.574 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:47:02.571+0000 7f74313af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37847) UID: 0 2024-09-15T21:47:02.591 INFO:teuthology.orchestra.run.smithi100.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-15T21:47:02.591 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-15T21:47:02.591 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 29h/41d Inode: 1 Links: 2 2024-09-15T21:47:02.592 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:47:02.592 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-15T21:47:02.592 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-15T21:47:02.592 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-09-15 21:46:28.410564920 +0000 2024-09-15T21:47:02.592 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-09-15 21:47:00.200538569 +0000 2024-09-15T21:47:02.592 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-09-15 21:46:28.410564920 +0000 2024-09-15T21:47:02.592 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-15T21:47:02.592 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-15T21:47:02.675 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:02.674+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:02.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.683+0000 7f965f467640 1 Processor -- start 2024-09-15T21:47:02.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.683+0000 7f965f467640 1 -- start start 2024-09-15T21:47:02.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.683+0000 7f366312a640 1 Processor -- start 2024-09-15T21:47:02.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.683+0000 7f366312a640 1 -- start start 2024-09-15T21:47:02.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.684+0000 7f6ca3542640 1 Processor -- start 2024-09-15T21:47:02.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.684+0000 7f6ca3542640 1 -- start start 2024-09-15T21:47:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.684+0000 7f965f467640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9658155290 0x7f965815e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.684+0000 7f965f467640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9658155670 0x7f965815bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.684+0000 7f965f467640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f96580c65b0 0x7f96580c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.684+0000 7f965f467640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f965815c0a0 con 0x7f9658155670 2024-09-15T21:47:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.684+0000 7f965f467640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f965815edb0 con 0x7f96580c65b0 2024-09-15T21:47:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.684+0000 7f965f467640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f965815f090 con 0x7f9658155290 2024-09-15T21:47:02.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.684+0000 7f366312a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f365c0c4600 0x7f365c06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.684+0000 7f366312a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f365c157e30 con 0x7f365c06d230 2024-09-15T21:47:02.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.684+0000 7f366312a640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f365c1580d0 con 0x7f365c14e9c0 2024-09-15T21:47:02.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.684+0000 7f366312a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f365c1583b0 con 0x7f365c0c4600 2024-09-15T21:47:02.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f965d203640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9658155290 0x7f965815e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.688 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.684+0000 7f36616c7640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f365c06d230 0x7f365c072f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:40760/0 (socket says 172.21.15.100:40760) 2024-09-15T21:47:02.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f965da04640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f96580c65b0 0x7f96580c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f965da04640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f96580c65b0 0x7f96580c69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:45976/0 (socket says 172.21.15.100:45976) 2024-09-15T21:47:02.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f965da04640 1 -- 172.21.15.100:0/2570044008 learned_addr learned my addr 172.21.15.100:0/2570044008 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:02.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f965d203640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9658155290 0x7f965815e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:54258/0 (socket says 172.21.15.100:54258) 2024-09-15T21:47:02.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.684+0000 7f36616c7640 1 -- 172.21.15.100:0/1304026343 learned_addr learned my addr 172.21.15.100:0/1304026343 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:02.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f365bfff640 1 --2- 172.21.15.100:0/1304026343 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f365c0c4600 0x7f365c06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f965ca02640 1 --2- 172.21.15.100:0/2570044008 >> v2:172.21.15.100:3300/0 conn(0x7f9658155670 0x7f965815bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f6ca3542640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6c9c06c230 0x7f6c9c0c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f365b7fe640 1 -- 172.21.15.100:0/1304026343 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 68569703 0 0) 0x7f365c157e30 con 0x7f365c06d230 2024-09-15T21:47:02.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f365b7fe640 1 -- 172.21.15.100:0/1304026343 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f364c005730 con 0x7f365c06d230 2024-09-15T21:47:02.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f6ca3542640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c9c15b020 con 0x7f6c9c067980 2024-09-15T21:47:02.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f6ca3542640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c9c0cdb50 con 0x7f6c9c06be50 2024-09-15T21:47:02.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f6ca3542640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f6c9c15b930 con 0x7f6c9c06c230 2024-09-15T21:47:02.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f6ca1adf640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f6c9c067980 0x7f6c9c1659a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:40762/0 (socket says 172.21.15.100:40762) 2024-09-15T21:47:02.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f6ca1adf640 1 -- 172.21.15.100:0/3756693063 learned_addr learned my addr 172.21.15.100:0/3756693063 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:02.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f6ca12de640 1 --1- 172.21.15.100:0/3756693063 >> v1:172.21.15.122:6789/0 conn(0x7f6c9c06be50 0x7f6c9c15e610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.100:39958/0 (socket says 172.21.15.100:39958) 2024-09-15T21:47:02.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f6ca0add640 1 --2- 172.21.15.100:0/3756693063 >> v2:172.21.15.100:3300/0 conn(0x7f6c9c06c230 0x7f6c9c0c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f965da04640 1 -- 172.21.15.100:0/2570044008 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9658155290 msgr2=0x7f965815e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:02.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f965da04640 1 --2- 172.21.15.100:0/2570044008 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9658155290 0x7f965815e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f965da04640 1 -- 172.21.15.100:0/2570044008 >> v2:172.21.15.100:3300/0 conn(0x7f9658155670 msgr2=0x7f965815bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:02.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f965da04640 1 --2- 172.21.15.100:0/2570044008 >> v2:172.21.15.100:3300/0 conn(0x7f9658155670 0x7f965815bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.692 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f965da04640 1 -- 172.21.15.100:0/2570044008 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f965815f370 con 0x7f96580c65b0 2024-09-15T21:47:02.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f365b7fe640 1 -- 172.21.15.100:0/1304026343 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 668536445 0 0) 0x7f365c1580d0 con 0x7f365c14e9c0 2024-09-15T21:47:02.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f365b7fe640 1 -- 172.21.15.100:0/1304026343 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f365c157e30 con 0x7f365c14e9c0 2024-09-15T21:47:02.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f365b7fe640 1 -- 172.21.15.100:0/1304026343 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1336349357 0 0) 0x7f364c005730 con 0x7f365c06d230 2024-09-15T21:47:02.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f365b7fe640 1 -- 172.21.15.100:0/1304026343 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f365c0c4600 msgr2=0x7f365c06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:02.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f365b7fe640 1 --2- 172.21.15.100:0/1304026343 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f365c0c4600 0x7f365c06ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f365b7fe640 1 -- 172.21.15.100:0/1304026343 >> v1:172.21.15.122:6789/0 conn(0x7f365c14e9c0 legacy=0x7f365c06df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f365b7fe640 1 -- 172.21.15.100:0/1304026343 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365c158690 con 0x7f365c06d230 2024-09-15T21:47:02.693 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f365b7fe640 1 -- 172.21.15.100:0/1304026343 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7f36540019f0 con 0x7f365c06d230 2024-09-15T21:47:02.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f365b7fe640 1 -- 172.21.15.100:0/1304026343 >> v1:172.21.15.100:6789/0 conn(0x7f365c06d230 legacy=0x7f365c072f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f365b7fe640 1 --2- 172.21.15.100:0/1304026343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f364c005d80 0x7f364c006160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f365b7fe640 1 --2- 172.21.15.100:0/1304026343 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f364c006a10 0x7f364c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f365b7fe640 1 --2- 172.21.15.100:0/1304026343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364c0073d0 0x7f364c0147c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f365b7fe640 1 -- 172.21.15.100:0/1304026343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f365c1583b0 con 0x7f364c0073d0 2024-09-15T21:47:02.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f365b7fe640 1 -- 172.21.15.100:0/1304026343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f364c006840 con 0x7f364c005d80 2024-09-15T21:47:02.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f365b7fe640 1 -- 172.21.15.100:0/1304026343 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f364c00bc40 con 0x7f364c006a10 2024-09-15T21:47:02.694 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f6ca0add640 1 -- 172.21.15.100:0/3756693063 >> v1:172.21.15.122:6789/0 conn(0x7f6c9c06be50 legacy=0x7f6c9c15e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:02.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f6ca0add640 1 -- 172.21.15.100:0/3756693063 >> v1:172.21.15.100:6789/0 conn(0x7f6c9c067980 legacy=0x7f6c9c1659a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.685+0000 7f6ca0add640 1 -- 172.21.15.100:0/3756693063 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c9c0cde30 con 0x7f6c9c06c230 2024-09-15T21:47:02.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.686+0000 7f36616c7640 1 --2- 172.21.15.100:0/1304026343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364c0073d0 0x7f364c0147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.686+0000 7f365bfff640 1 --2- 172.21.15.100:0/1304026343 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f364c006a10 0x7f364c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.686+0000 7f3660ec6640 1 --2- 172.21.15.100:0/1304026343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f364c005d80 0x7f364c006160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.686+0000 7f965da04640 1 --2- 172.21.15.100:0/2570044008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96580c65b0 0x7f96580c69e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f965806d5b0 tx=0x7f965002a690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23bec96cf63aaf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.686+0000 7f36616c7640 1 -- 172.21.15.100:0/1304026343 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f364c006a10 msgr2=0x7f364c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:02.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.686+0000 7f36616c7640 1 --2- 172.21.15.100:0/1304026343 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f364c006a10 0x7f364c006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.686+0000 7f36616c7640 1 -- 172.21.15.100:0/1304026343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f364c005d80 msgr2=0x7f364c006160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:02.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.686+0000 7f36616c7640 1 --2- 172.21.15.100:0/1304026343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f364c005d80 0x7f364c006160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.686+0000 7f36616c7640 1 -- 172.21.15.100:0/1304026343 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364c005730 con 0x7f364c0073d0 2024-09-15T21:47:02.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.686+0000 7f3660ec6640 1 --2- 172.21.15.100:0/1304026343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f364c005d80 0x7f364c006160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:47:02.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.686+0000 7f964ffff640 1 -- 172.21.15.100:0/2570044008 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650036070 con 0x7f96580c65b0 2024-09-15T21:47:02.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.686+0000 7f964ffff640 1 -- 172.21.15.100:0/2570044008 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9650031600 con 0x7f96580c65b0 2024-09-15T21:47:02.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.686+0000 7f964ffff640 1 -- 172.21.15.100:0/2570044008 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96500319d0 con 0x7f96580c65b0 2024-09-15T21:47:02.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.686+0000 7f6ca0add640 1 --2- 172.21.15.100:0/3756693063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9c06c230 0x7f6c9c0c6ba0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0097d0 tx=0x7f6c8c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407dd29f072787b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f6c93fff640 1 -- 172.21.15.100:0/3756693063 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c037070 con 0x7f6c9c06c230 2024-09-15T21:47:02.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f6c93fff640 1 -- 172.21.15.100:0/3756693063 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c8c032440 con 0x7f6c9c06c230 2024-09-15T21:47:02.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f6c93fff640 1 -- 172.21.15.100:0/3756693063 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c032810 con 0x7f6c9c06c230 2024-09-15T21:47:02.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f36616c7640 1 --2- 172.21.15.100:0/1304026343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364c0073d0 0x7f364c0147c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f365400a730 tx=0x7f364c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=408f90002030e8d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f965f467640 1 -- 172.21.15.100:0/2570044008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96580c65b0 msgr2=0x7f96580c69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f965f467640 1 --2- 172.21.15.100:0/2570044008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96580c65b0 0x7f96580c69e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f965806d5b0 tx=0x7f965002a690 comp rx=0 tx=0).stop 2024-09-15T21:47:02.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f965f467640 1 -- 172.21.15.100:0/2570044008 shutdown_connections 2024-09-15T21:47:02.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f965f467640 1 --2- 172.21.15.100:0/2570044008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f96580c65b0 0x7f96580c69e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f365b7fe640 1 -- 172.21.15.100:0/1304026343 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365400a350 con 0x7f364c0073d0 2024-09-15T21:47:02.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f365b7fe640 1 -- 172.21.15.100:0/1304026343 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f365402f070 con 0x7f364c0073d0 2024-09-15T21:47:02.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f365b7fe640 1 -- 172.21.15.100:0/1304026343 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365402bb90 con 0x7f364c0073d0 2024-09-15T21:47:02.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f965f467640 1 --2- 172.21.15.100:0/2570044008 >> v2:172.21.15.100:3300/0 conn(0x7f9658155670 0x7f965815bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f965f467640 1 --2- 172.21.15.100:0/2570044008 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9658155290 0x7f965815e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f965f467640 1 -- 172.21.15.100:0/2570044008 >> 172.21.15.100:0/2570044008 conn(0x7f9658073310 msgr2=0x7f9658073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:02.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f6ca3542640 1 -- 172.21.15.100:0/3756693063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9c06c230 msgr2=0x7f6c9c0c6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f6ca3542640 1 --2- 172.21.15.100:0/3756693063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9c06c230 0x7f6c9c0c6ba0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0097d0 tx=0x7f6c8c02a210 comp rx=0 tx=0).stop 2024-09-15T21:47:02.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f965f467640 1 -- 172.21.15.100:0/2570044008 shutdown_connections 2024-09-15T21:47:02.701 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f965f467640 1 -- 172.21.15.100:0/2570044008 wait complete. 2024-09-15T21:47:02.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f6ca3542640 1 -- 172.21.15.100:0/3756693063 shutdown_connections 2024-09-15T21:47:02.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f6ca3542640 1 --2- 172.21.15.100:0/3756693063 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9c06c230 0x7f6c9c0c6ba0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.687+0000 7f6ca3542640 1 -- 172.21.15.100:0/3756693063 >> 172.21.15.100:0/3756693063 conn(0x7f6c9c06fa40 msgr2=0x7f6c9c0cd0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:02.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.688+0000 7f6ca3542640 1 -- 172.21.15.100:0/3756693063 shutdown_connections 2024-09-15T21:47:02.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.688+0000 7f6ca3542640 1 -- 172.21.15.100:0/3756693063 wait complete. 2024-09-15T21:47:02.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.688+0000 7f366312a640 1 -- 172.21.15.100:0/1304026343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364c0073d0 msgr2=0x7f364c0147c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.688+0000 7f366312a640 1 --2- 172.21.15.100:0/1304026343 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364c0073d0 0x7f364c0147c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f365400a730 tx=0x7f364c007a40 comp rx=0 tx=0).stop 2024-09-15T21:47:02.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.688+0000 7f36616c7640 1 -- 172.21.15.100:0/1304026343 reap_dead start 2024-09-15T21:47:02.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.688+0000 7f366312a640 1 -- 172.21.15.100:0/1304026343 shutdown_connections 2024-09-15T21:47:02.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.688+0000 7f366312a640 1 -- 172.21.15.100:0/1304026343 >> 172.21.15.100:0/1304026343 conn(0x7f365c0ca430 msgr2=0x7f365c0ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:02.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.688+0000 7f366312a640 1 -- 172.21.15.100:0/1304026343 shutdown_connections 2024-09-15T21:47:02.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.688+0000 7f366312a640 1 -- 172.21.15.100:0/1304026343 wait complete. 2024-09-15T21:47:02.703 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.689+0000 7f965f467640 1 Processor -- start 2024-09-15T21:47:02.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.689+0000 7f6ca3542640 1 Processor -- start 2024-09-15T21:47:02.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.689+0000 7f965f467640 1 -- start start 2024-09-15T21:47:02.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.689+0000 7f366312a640 1 Processor -- start 2024-09-15T21:47:02.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.689+0000 7f6ca3542640 1 -- start start 2024-09-15T21:47:02.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.689+0000 7f366312a640 1 -- start start 2024-09-15T21:47:02.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.690+0000 7f366312a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364c0073d0 0x7f365c14d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.690+0000 7f3660ec6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364c0073d0 0x7f365c14d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.690+0000 7f3660ec6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364c0073d0 0x7f365c14d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49002/0 (socket says 172.21.15.100:49002) 2024-09-15T21:47:02.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.690+0000 7f366312a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f365c1597b0 0x7f365c0c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.690+0000 7f3660ec6640 1 -- 172.21.15.100:0/3740202194 learned_addr learned my addr 172.21.15.100:0/3740202194 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:02.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.690+0000 7f366312a640 1 --2- 172.21.15.100:0/3740202194 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f365c147b10 0x7f365c147f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.690+0000 7f366312a640 1 -- 172.21.15.100:0/3740202194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f365c14eda0 con 0x7f364c0073d0 2024-09-15T21:47:02.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f6ca3542640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9c067980 0x7f6c9c157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f6ca3542640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c9c06be50 0x7f6c9c0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.690+0000 7f366312a640 1 -- 172.21.15.100:0/3740202194 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f365c0caf20 con 0x7f365c1597b0 2024-09-15T21:47:02.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.690+0000 7f366312a640 1 -- 172.21.15.100:0/3740202194 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f365c073630 con 0x7f365c147b10 2024-09-15T21:47:02.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f6ca3542640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c9c06c230 0x7f6c9c155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f365bfff640 1 --2- 172.21.15.100:0/3740202194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f365c1597b0 0x7f365c0c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f6ca0add640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c9c06be50 0x7f6c9c0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f6ca0add640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c9c06be50 0x7f6c9c0cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:46016/0 (socket says 172.21.15.100:46016) 2024-09-15T21:47:02.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f6ca0add640 1 -- 172.21.15.100:0/2649006922 learned_addr learned my addr 172.21.15.100:0/2649006922 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:02.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f6ca12de640 1 --2- 172.21.15.100:0/2649006922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9c067980 0x7f6c9c157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f6ca3542640 1 -- 172.21.15.100:0/2649006922 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c0cd7f0 con 0x7f6c9c067980 2024-09-15T21:47:02.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f6ca1adf640 1 --2- 172.21.15.100:0/2649006922 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c9c06c230 0x7f6c9c155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f6ca3542640 1 -- 172.21.15.100:0/2649006922 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6c9c15b5c0 con 0x7f6c9c06be50 2024-09-15T21:47:02.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f6ca3542640 1 -- 172.21.15.100:0/2649006922 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6c9c0c8600 con 0x7f6c9c06c230 2024-09-15T21:47:02.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f36616c7640 1 --2- 172.21.15.100:0/3740202194 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f365c147b10 0x7f365c147f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f3660ec6640 1 -- 172.21.15.100:0/3740202194 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f365c147b10 msgr2=0x7f365c147f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.690+0000 7f965f467640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f96580c65b0 0x7f9658158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.690+0000 7f965f467640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9658155290 0x7f96580cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.690+0000 7f965f467640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9658155670 0x7f965814e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f3660ec6640 1 --2- 172.21.15.100:0/3740202194 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f365c147b10 0x7f365c147f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f6ca12de640 1 -- 172.21.15.100:0/2649006922 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c9c06c230 msgr2=0x7f6c9c155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f6ca12de640 1 --2- 172.21.15.100:0/2649006922 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c9c06c230 0x7f6c9c155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f6ca12de640 1 -- 172.21.15.100:0/2649006922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c9c06be50 msgr2=0x7f6c9c0cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f965f467640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f96580b39e0 con 0x7f9658155290 2024-09-15T21:47:02.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f965f467640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f96580c6090 con 0x7f9658155670 2024-09-15T21:47:02.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f965f467640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f965815c590 con 0x7f96580c65b0 2024-09-15T21:47:02.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f965ca02640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9658155290 0x7f96580cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f965da04640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9658155670 0x7f965814e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f965d203640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f96580c65b0 0x7f9658158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f965ca02640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9658155290 0x7f96580cc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49004/0 (socket says 172.21.15.100:49004) 2024-09-15T21:47:02.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f965da04640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9658155670 0x7f965814e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:46010/0 (socket says 172.21.15.100:46010) 2024-09-15T21:47:02.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f965ca02640 1 -- 172.21.15.100:0/2500103331 learned_addr learned my addr 172.21.15.100:0/2500103331 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:02.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f6ca12de640 1 --2- 172.21.15.100:0/2649006922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c9c06be50 0x7f6c9c0cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f6ca12de640 1 -- 172.21.15.100:0/2649006922 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c9c1f1130 con 0x7f6c9c067980 2024-09-15T21:47:02.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f965da04640 1 -- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f96580c65b0 msgr2=0x7f9658158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f965da04640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f96580c65b0 0x7f9658158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f965da04640 1 -- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9658155290 msgr2=0x7f96580cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f965da04640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9658155290 0x7f96580cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f965da04640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9658156ff0 con 0x7f9658155670 2024-09-15T21:47:02.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f965da04640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9658155670 0x7f965814e4f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9650030000 tx=0x7f9650004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f3660ec6640 1 -- 172.21.15.100:0/3740202194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f365c1597b0 msgr2=0x7f365c0c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f3660ec6640 1 --2- 172.21.15.100:0/3740202194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f365c1597b0 0x7f365c0c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f3660ec6640 1 -- 172.21.15.100:0/3740202194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f365c159d30 con 0x7f364c0073d0 2024-09-15T21:47:02.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f365bfff640 1 --2- 172.21.15.100:0/3740202194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f365c1597b0 0x7f365c0c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:47:02.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.691+0000 7f3660ec6640 1 --2- 172.21.15.100:0/3740202194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364c0073d0 0x7f365c14d4a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f365000e620 tx=0x7f365000eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.692+0000 7f6ca0add640 1 --2- 172.21.15.100:0/2649006922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c9c06be50 0x7f6c9c0cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:02.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.692+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650036050 con 0x7f9658155670 2024-09-15T21:47:02.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.692+0000 7f6ca12de640 1 --2- 172.21.15.100:0/2649006922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9c067980 0x7f6c9c157fb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800b3e0 tx=0x7f6c9800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.692+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9650042070 con 0x7f9658155670 2024-09-15T21:47:02.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.692+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96580b5730 con 0x7f9658155670 2024-09-15T21:47:02.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.692+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9650009510 con 0x7f9658155670 2024-09-15T21:47:02.715 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.692+0000 7f36597fa640 1 -- 172.21.15.100:0/3740202194 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36500052e0 con 0x7f364c0073d0 2024-09-15T21:47:02.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.692+0000 7f6c91ffb640 1 -- 172.21.15.100:0/2649006922 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98015020 con 0x7f6c9c067980 2024-09-15T21:47:02.716 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.692+0000 7f36597fa640 1 -- 172.21.15.100:0/3740202194 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3650007020 con 0x7f364c0073d0 2024-09-15T21:47:02.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.692+0000 7f6c91ffb640 1 -- 172.21.15.100:0/2649006922 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c98011830 con 0x7f6c9c067980 2024-09-15T21:47:02.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.692+0000 7f6ca3542640 1 -- 172.21.15.100:0/2649006922 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c9c14de10 con 0x7f6c9c067980 2024-09-15T21:47:02.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.692+0000 7f6c91ffb640 1 -- 172.21.15.100:0/2649006922 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c98011c20 con 0x7f6c9c067980 2024-09-15T21:47:02.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.692+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96581f1940 con 0x7f9658155670 2024-09-15T21:47:02.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.692+0000 7f36597fa640 1 -- 172.21.15.100:0/3740202194 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36500071d0 con 0x7f364c0073d0 2024-09-15T21:47:02.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.692+0000 7f366312a640 1 -- 172.21.15.100:0/3740202194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f365c0c90f0 con 0x7f364c0073d0 2024-09-15T21:47:02.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.692+0000 7f6ca3542640 1 -- 172.21.15.100:0/2649006922 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c9c14e6b0 con 0x7f6c9c067980 2024-09-15T21:47:02.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.693+0000 7f366312a640 1 -- 172.21.15.100:0/3740202194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f365c0c6180 con 0x7f364c0073d0 2024-09-15T21:47:02.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.693+0000 7f96437fe640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f9630005270 con 0x7f9658155670 2024-09-15T21:47:02.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.694+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9650047460 con 0x7f9658155670 2024-09-15T21:47:02.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.694+0000 7f6ca3542640 1 -- 172.21.15.100:0/2649006922 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c9c0c95e0 con 0x7f6c9c067980 2024-09-15T21:47:02.719 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:02.719 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:02.719 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:02.719 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:02.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.694+0000 7f6c91ffb640 1 -- 172.21.15.100:0/2649006922 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6c98018020 con 0x7f6c9c067980 2024-09-15T21:47:02.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.694+0000 7f366312a640 1 -- 172.21.15.100:0/3740202194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f365c069650 con 0x7f364c0073d0 2024-09-15T21:47:02.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.695+0000 7f964dffb640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f963c06c330 0x7f963c06e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.695+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f96500abff0 con 0x7f9658155670 2024-09-15T21:47:02.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.695+0000 7f965d203640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f963c06c330 0x7f963c06e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.695+0000 7f6c91ffb640 1 --2- 172.21.15.100:0/2649006922 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6c8006c2e0 0x7f6c8006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.695+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v22) ==== 54+0+16 (secure 0 0 0) 0x7f96500ae590 con 0x7f9658155670 2024-09-15T21:47:02.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.695+0000 7f36597fa640 1 -- 172.21.15.100:0/3740202194 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f365001a020 con 0x7f364c0073d0 2024-09-15T21:47:02.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.695+0000 7f6ca0add640 1 --2- 172.21.15.100:0/2649006922 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6c8006c2e0 0x7f6c8006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.695+0000 7f6c91ffb640 1 -- 172.21.15.100:0/2649006922 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f6c9808b1f0 con 0x7f6c9c067980 2024-09-15T21:47:02.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.695+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f9658156850 0x7f965815d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.696+0000 7f965d203640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f963c06c330 0x7f963c06e7f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9654009ac0 tx=0x7f9654009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.696+0000 7f36597fa640 1 --2- 172.21.15.100:0/3740202194 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f364006c300 0x7f364006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.696+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9658157570 con 0x7f9658156850 2024-09-15T21:47:02.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.696+0000 7f965ca02640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f9658156850 0x7f965815d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.696+0000 7f36597fa640 1 -- 172.21.15.100:0/3740202194 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f365008a990 con 0x7f364c0073d0 2024-09-15T21:47:02.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.696+0000 7f6ca0add640 1 --2- 172.21.15.100:0/2649006922 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6c8006c2e0 0x7f6c8006e7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0091d0 tx=0x7f6c8c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.696+0000 7f365bfff640 1 --2- 172.21.15.100:0/3740202194 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f364006c300 0x7f364006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.696+0000 7f965ca02640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f9658156850 0x7f965815d830 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.696+0000 7f365bfff640 1 --2- 172.21.15.100:0/3740202194 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f364006c300 0x7f364006e7c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f365c157560 tx=0x7f3648008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.697+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9658157570 con 0x7f9658156850 2024-09-15T21:47:02.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.700+0000 7f6c91ffb640 1 -- 172.21.15.100:0/2649006922 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6c9801f020 con 0x7f6c9c067980 2024-09-15T21:47:02.726 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:47:02.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.709+0000 7f36597fa640 1 -- 172.21.15.100:0/3740202194 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f365c069650 con 0x7f364c0073d0 2024-09-15T21:47:02.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.716+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9658158860 con 0x7f9658156850 2024-09-15T21:47:02.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.717+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9658158860 con 0x7f9658156850 2024-09-15T21:47:02.728 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:02.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.717+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f9658157ca0 0x7f96581f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.717+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f965815a2f0 con 0x7f9658157ca0 2024-09-15T21:47:02.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.718+0000 7f965d203640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f9658157ca0 0x7f96581f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.718+0000 7f965d203640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f9658157ca0 0x7f96581f3fd0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.719+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f965815a2f0 con 0x7f9658157ca0 2024-09-15T21:47:02.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.733+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f965815a010 con 0x7f9658157ca0 2024-09-15T21:47:02.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.735+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f965815a010 con 0x7f9658157ca0 2024-09-15T21:47:02.737 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:02.738 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:02.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.735+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f96581f4a20 0x7f9658062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.736+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f96581f4e00 con 0x7f96581f4a20 2024-09-15T21:47:02.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.736+0000 7f965ca02640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f96581f4a20 0x7f9658062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.736+0000 7f965ca02640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f96581f4a20 0x7f9658062630 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.737+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96581f4e00 con 0x7f96581f4a20 2024-09-15T21:47:02.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.752+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f96581f4e00 con 0x7f96581f4a20 2024-09-15T21:47:02.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.754+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f96581f4e00 con 0x7f96581f4a20 2024-09-15T21:47:02.756 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:02.756 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:02.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.754+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f9658063080 0x7f96580ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.754+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f96580634a0 con 0x7f9658063080 2024-09-15T21:47:02.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.754+0000 7f965da04640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f9658063080 0x7f96580ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.755+0000 7f965da04640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f9658063080 0x7f96580ce960 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.755+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96580634a0 con 0x7f9658063080 2024-09-15T21:47:02.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.769+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f96580cf170 con 0x7f9658063080 2024-09-15T21:47:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.771+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f96580cf170 con 0x7f9658063080 2024-09-15T21:47:02.773 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:02.773 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:02.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.771+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f965802c640 0x7f965802caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.771+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f96580634a0 con 0x7f965802c640 2024-09-15T21:47:02.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.771+0000 7f965d203640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f965802c640 0x7f965802caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.772+0000 7f965d203640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f965802c640 0x7f965802caa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.772+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f96580634a0 con 0x7f965802c640 2024-09-15T21:47:02.775 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:02.773+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:02.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.786+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f96580634a0 con 0x7f965802c640 2024-09-15T21:47:02.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.788+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f96580634a0 con 0x7f965802c640 2024-09-15T21:47:02.790 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:02.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.789+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f965802d4f0 0x7f9658207360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.789+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f965802d970 con 0x7f965802d4f0 2024-09-15T21:47:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.789+0000 7f965ca02640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f965802d4f0 0x7f9658207360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.789+0000 7f965ca02640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f965802d4f0 0x7f9658207360 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.790+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== osd.5 v2:172.21.15.122:6824/2400668532 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f965802d970 con 0x7f965802d4f0 2024-09-15T21:47:02.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.804+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f965802d970 con 0x7f965802d4f0 2024-09-15T21:47:02.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.806+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== osd.5 v2:172.21.15.122:6824/2400668532 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f965802d970 con 0x7f965802d4f0 2024-09-15T21:47:02.808 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-15T21:47:02.809 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:47:02.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.806+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f9658207db0 0x7f965820a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.806+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f965802d970 con 0x7f9658207db0 2024-09-15T21:47:02.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.807+0000 7f965da04640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f9658207db0 0x7f965820a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.807+0000 7f965da04640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f9658207db0 0x7f965820a1a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.808+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f965802d970 con 0x7f9658207db0 2024-09-15T21:47:02.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.821+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f965802d970 con 0x7f9658207db0 2024-09-15T21:47:02.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.824+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f965802d970 con 0x7f9658207db0 2024-09-15T21:47:02.825 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.824+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f965820abf0 0x7f965820cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.824+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f965820fdb0 con 0x7f965820abf0 2024-09-15T21:47:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.824+0000 7f965d203640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f965820abf0 0x7f965820cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:02.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.825+0000 7f965d203640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f965820abf0 0x7f965820cfe0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:02.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.825+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f965820fdb0 con 0x7f965820abf0 2024-09-15T21:47:02.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.855+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f965820d520 con 0x7f965820abf0 2024-09-15T21:47:02.859 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:02.859 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:02.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.857+0000 7f964dffb640 1 -- 172.21.15.100:0/2500103331 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f965820d520 con 0x7f965820abf0 2024-09-15T21:47:02.860 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:02.862 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:02.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f9658156850 msgr2=0x7f965815d830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f9658156850 0x7f965815d830 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f9658157ca0 msgr2=0x7f96581f3fd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f9658157ca0 0x7f96581f3fd0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f96581f4a20 msgr2=0x7f9658062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f96581f4a20 0x7f9658062630 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f9658063080 msgr2=0x7f96580ce960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f9658063080 0x7f96580ce960 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f965802c640 msgr2=0x7f965802caa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f965802c640 0x7f965802caa0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f965802d4f0 msgr2=0x7f9658207360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f965802d4f0 0x7f9658207360 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965da04640 1 -- 172.21.15.100:0/2500103331 reap_dead start 2024-09-15T21:47:02.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f9658207db0 msgr2=0x7f965820a1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f9658207db0 0x7f965820a1a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965da04640 1 -- 172.21.15.100:0/2500103331 reap_dead start 2024-09-15T21:47:02.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965da04640 1 -- 172.21.15.100:0/2500103331 reap_dead start 2024-09-15T21:47:02.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f965820abf0 msgr2=0x7f965820cfe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f965820abf0 0x7f965820cfe0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f963c06c330 msgr2=0x7f963c06e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f963c06c330 0x7f963c06e7f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9654009ac0 tx=0x7f9654009190 comp rx=0 tx=0).stop 2024-09-15T21:47:02.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9658155670 msgr2=0x7f965814e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:02.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.858+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9658155670 0x7f965814e4f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9650030000 tx=0x7f9650004a40 comp rx=0 tx=0).stop 2024-09-15T21:47:02.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.859+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 shutdown_connections 2024-09-15T21:47:02.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.859+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f9658207db0 0x7f965820a1a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.859+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f965820abf0 0x7f965820cfe0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.859+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f963c06c330 0x7f963c06e7f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.859+0000 7f965f467640 1 --2- 172.21.15.100:0/2500103331 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9658155670 0x7f965814e4f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:02.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.859+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 >> 172.21.15.100:0/2500103331 conn(0x7f9658073310 msgr2=0x7f96580d10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:02.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.859+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 shutdown_connections 2024-09-15T21:47:02.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.859+0000 7f965f467640 1 -- 172.21.15.100:0/2500103331 wait complete. 2024-09-15T21:47:02.869 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:47:02.877 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:02.875+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:02.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.961+0000 7f6ca3542640 1 -- 172.21.15.100:0/2649006922 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f6c9c0723e0 con 0x7f6c9c067980 2024-09-15T21:47:02.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.962+0000 7f6c91ffb640 1 -- 172.21.15.100:0/2649006922 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v22) ==== 81+0+34268 (secure 0 0 0) 0x7f6c98059270 con 0x7f6c9c067980 2024-09-15T21:47:02.965 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:47:02.965 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:02.965 INFO:teuthology.orchestra.run.smithi100.stdout: "epoch": 22, 2024-09-15T21:47:02.965 INFO:teuthology.orchestra.run.smithi100.stdout: "fsid": "30e2fff4-2171-4531-8806-0e03b63f494f", 2024-09-15T21:47:02.965 INFO:teuthology.orchestra.run.smithi100.stdout: "created": "2024-09-15T21:45:59.746897+0000", 2024-09-15T21:47:02.965 INFO:teuthology.orchestra.run.smithi100.stdout: "modified": "2024-09-15T21:46:24.409383+0000", 2024-09-15T21:47:02.965 INFO:teuthology.orchestra.run.smithi100.stdout: "last_up_change": "2024-09-15T21:46:13.224709+0000", 2024-09-15T21:47:02.965 INFO:teuthology.orchestra.run.smithi100.stdout: "last_in_change": "2024-09-15T21:46:01.812129+0000", 2024-09-15T21:47:02.965 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T21:47:02.965 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_num": 5799936, 2024-09-15T21:47:02.965 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_set": [ 2024-09-15T21:47:02.965 INFO:teuthology.orchestra.run.smithi100.stdout: "pglog_hardlimit", 2024-09-15T21:47:02.966 INFO:teuthology.orchestra.run.smithi100.stdout: "purged_snapdirs", 2024-09-15T21:47:02.966 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_deletes", 2024-09-15T21:47:02.966 INFO:teuthology.orchestra.run.smithi100.stdout: "sortbitwise" 2024-09-15T21:47:02.966 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-15T21:47:02.966 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_version": 5, 2024-09-15T21:47:02.966 INFO:teuthology.orchestra.run.smithi100.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T21:47:02.966 INFO:teuthology.orchestra.run.smithi100.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T21:47:02.966 INFO:teuthology.orchestra.run.smithi100.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T21:47:02.966 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_snapshot": "", 2024-09-15T21:47:02.966 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_max": 3, 2024-09-15T21:47:02.966 INFO:teuthology.orchestra.run.smithi100.stdout: "max_osd": 8, 2024-09-15T21:47:02.966 INFO:teuthology.orchestra.run.smithi100.stdout: "require_min_compat_client": "luminous", 2024-09-15T21:47:02.966 INFO:teuthology.orchestra.run.smithi100.stdout: "min_compat_client": "jewel", 2024-09-15T21:47:02.966 INFO:teuthology.orchestra.run.smithi100.stdout: "require_osd_release": "squid", 2024-09-15T21:47:02.967 INFO:teuthology.orchestra.run.smithi100.stdout: "allow_crimson": false, 2024-09-15T21:47:02.967 INFO:teuthology.orchestra.run.smithi100.stdout: "pools": [ 2024-09-15T21:47:02.967 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.967 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 1, 2024-09-15T21:47:02.967 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": ".mgr", 2024-09-15T21:47:02.967 INFO:teuthology.orchestra.run.smithi100.stdout: "create_time": "2024-09-15T21:46:11.837603+0000", 2024-09-15T21:47:02.967 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-15T21:47:02.967 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_names": "hashpspool", 2024-09-15T21:47:02.967 INFO:teuthology.orchestra.run.smithi100.stdout: "type": 1, 2024-09-15T21:47:02.967 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 2, 2024-09-15T21:47:02.967 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": 1, 2024-09-15T21:47:02.967 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_rule": 0, 2024-09-15T21:47:02.967 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:47:02.968 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:47:02.968 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:47:02.968 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:47:02.968 INFO:teuthology.orchestra.run.smithi100.stdout: "is_stretch_pool": false, 2024-09-15T21:47:02.968 INFO:teuthology.orchestra.run.smithi100.stdout: "object_hash": 2, 2024-09-15T21:47:02.968 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:47:02.968 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": 1, 2024-09-15T21:47:02.968 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num": 1, 2024-09-15T21:47:02.968 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num_target": 1, 2024-09-15T21:47:02.968 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_target": 1, 2024-09-15T21:47:02.968 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_pending": 1, 2024-09-15T21:47:02.968 INFO:teuthology.orchestra.run.smithi100.stdout: "last_pg_merge_meta": { 2024-09-15T21:47:02.968 INFO:teuthology.orchestra.run.smithi100.stdout: "source_pgid": "0.0", 2024-09-15T21:47:02.968 INFO:teuthology.orchestra.run.smithi100.stdout: "ready_epoch": 0, 2024-09-15T21:47:02.969 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_started": 0, 2024-09-15T21:47:02.969 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_clean": 0, 2024-09-15T21:47:02.969 INFO:teuthology.orchestra.run.smithi100.stdout: "source_version": "0'0", 2024-09-15T21:47:02.969 INFO:teuthology.orchestra.run.smithi100.stdout: "target_version": "0'0" 2024-09-15T21:47:02.969 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.969 INFO:teuthology.orchestra.run.smithi100.stdout: "last_change": "18", 2024-09-15T21:47:02.969 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend": "0", 2024-09-15T21:47:02.969 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:47:02.969 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:47:02.969 INFO:teuthology.orchestra.run.smithi100.stdout: "auid": 0, 2024-09-15T21:47:02.969 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:47:02.969 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_seq": 0, 2024-09-15T21:47:02.969 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_epoch": 0, 2024-09-15T21:47:02.970 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_snaps": [], 2024-09-15T21:47:02.970 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps": "[]", 2024-09-15T21:47:02.970 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_bytes": 0, 2024-09-15T21:47:02.970 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_objects": 0, 2024-09-15T21:47:02.970 INFO:teuthology.orchestra.run.smithi100.stdout: "tiers": [], 2024-09-15T21:47:02.970 INFO:teuthology.orchestra.run.smithi100.stdout: "tier_of": -1, 2024-09-15T21:47:02.970 INFO:teuthology.orchestra.run.smithi100.stdout: "read_tier": -1, 2024-09-15T21:47:02.970 INFO:teuthology.orchestra.run.smithi100.stdout: "write_tier": -1, 2024-09-15T21:47:02.970 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_mode": "none", 2024-09-15T21:47:02.970 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_bytes": 0, 2024-09-15T21:47:02.970 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_objects": 0, 2024-09-15T21:47:02.970 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:47:02.970 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:47:02.970 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:47:02.971 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_flush_age": 0, 2024-09-15T21:47:02.971 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_evict_age": 0, 2024-09-15T21:47:02.971 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profile": "", 2024-09-15T21:47:02.971 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_params": { 2024-09-15T21:47:02.971 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "none" 2024-09-15T21:47:02.971 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.971 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_period": 0, 2024-09-15T21:47:02.971 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_count": 0, 2024-09-15T21:47:02.971 INFO:teuthology.orchestra.run.smithi100.stdout: "use_gmt_hitset": true, 2024-09-15T21:47:02.971 INFO:teuthology.orchestra.run.smithi100.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:47:02.971 INFO:teuthology.orchestra.run.smithi100.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:47:02.971 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:47:02.971 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:47:02.971 INFO:teuthology.orchestra.run.smithi100.stdout: "grade_table": [], 2024-09-15T21:47:02.972 INFO:teuthology.orchestra.run.smithi100.stdout: "stripe_width": 0, 2024-09-15T21:47:02.972 INFO:teuthology.orchestra.run.smithi100.stdout: "expected_num_objects": 0, 2024-09-15T21:47:02.972 INFO:teuthology.orchestra.run.smithi100.stdout: "fast_read": false, 2024-09-15T21:47:02.972 INFO:teuthology.orchestra.run.smithi100.stdout: "options": { 2024-09-15T21:47:02.972 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_max": 32, 2024-09-15T21:47:02.972 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_min": 1 2024-09-15T21:47:02.972 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.972 INFO:teuthology.orchestra.run.smithi100.stdout: "application_metadata": { 2024-09-15T21:47:02.972 INFO:teuthology.orchestra.run.smithi100.stdout: "mgr": {} 2024-09-15T21:47:02.973 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.973 INFO:teuthology.orchestra.run.smithi100.stdout: "read_balance": { 2024-09-15T21:47:02.973 INFO:teuthology.orchestra.run.smithi100.stdout: "score_type": "Fair distribution", 2024-09-15T21:47:02.973 INFO:teuthology.orchestra.run.smithi100.stdout: "score_acting": 8, 2024-09-15T21:47:02.973 INFO:teuthology.orchestra.run.smithi100.stdout: "score_stable": 8, 2024-09-15T21:47:02.973 INFO:teuthology.orchestra.run.smithi100.stdout: "optimal_score": 0.25, 2024-09-15T21:47:02.973 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_acting": 2, 2024-09-15T21:47:02.973 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_stable": 2, 2024-09-15T21:47:02.973 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity_weighted": 1, 2024-09-15T21:47:02.973 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity": 1, 2024-09-15T21:47:02.973 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity_weighted": 1 2024-09-15T21:47:02.973 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:02.973 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.974 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.974 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 2, 2024-09-15T21:47:02.974 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": "cephfs_metadata", 2024-09-15T21:47:02.974 INFO:teuthology.orchestra.run.smithi100.stdout: "create_time": "2024-09-15T21:46:20.577982+0000", 2024-09-15T21:47:02.974 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-15T21:47:02.974 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_names": "hashpspool", 2024-09-15T21:47:02.974 INFO:teuthology.orchestra.run.smithi100.stdout: "type": 1, 2024-09-15T21:47:02.974 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 2, 2024-09-15T21:47:02.974 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": 1, 2024-09-15T21:47:02.974 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_rule": 0, 2024-09-15T21:47:02.974 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:47:02.974 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:47:02.974 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:47:02.974 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:47:02.975 INFO:teuthology.orchestra.run.smithi100.stdout: "is_stretch_pool": false, 2024-09-15T21:47:02.975 INFO:teuthology.orchestra.run.smithi100.stdout: "object_hash": 2, 2024-09-15T21:47:02.975 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:47:02.975 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": 32, 2024-09-15T21:47:02.975 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num": 32, 2024-09-15T21:47:02.975 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num_target": 32, 2024-09-15T21:47:02.975 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_target": 32, 2024-09-15T21:47:02.975 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_pending": 32, 2024-09-15T21:47:02.975 INFO:teuthology.orchestra.run.smithi100.stdout: "last_pg_merge_meta": { 2024-09-15T21:47:02.975 INFO:teuthology.orchestra.run.smithi100.stdout: "source_pgid": "0.0", 2024-09-15T21:47:02.975 INFO:teuthology.orchestra.run.smithi100.stdout: "ready_epoch": 0, 2024-09-15T21:47:02.975 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_started": 0, 2024-09-15T21:47:02.975 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_clean": 0, 2024-09-15T21:47:02.976 INFO:teuthology.orchestra.run.smithi100.stdout: "source_version": "0'0", 2024-09-15T21:47:02.976 INFO:teuthology.orchestra.run.smithi100.stdout: "target_version": "0'0" 2024-09-15T21:47:02.976 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.976 INFO:teuthology.orchestra.run.smithi100.stdout: "last_change": "21", 2024-09-15T21:47:02.976 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend": "0", 2024-09-15T21:47:02.976 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:47:02.976 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:47:02.976 INFO:teuthology.orchestra.run.smithi100.stdout: "auid": 0, 2024-09-15T21:47:02.976 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:47:02.976 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_seq": 0, 2024-09-15T21:47:02.976 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_epoch": 0, 2024-09-15T21:47:02.976 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_snaps": [], 2024-09-15T21:47:02.976 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps": "[]", 2024-09-15T21:47:02.976 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_bytes": 0, 2024-09-15T21:47:02.977 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_objects": 0, 2024-09-15T21:47:02.977 INFO:teuthology.orchestra.run.smithi100.stdout: "tiers": [], 2024-09-15T21:47:02.977 INFO:teuthology.orchestra.run.smithi100.stdout: "tier_of": -1, 2024-09-15T21:47:02.977 INFO:teuthology.orchestra.run.smithi100.stdout: "read_tier": -1, 2024-09-15T21:47:02.977 INFO:teuthology.orchestra.run.smithi100.stdout: "write_tier": -1, 2024-09-15T21:47:02.977 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_mode": "none", 2024-09-15T21:47:02.977 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_bytes": 0, 2024-09-15T21:47:02.977 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_objects": 0, 2024-09-15T21:47:02.977 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:47:02.977 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:47:02.977 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:47:02.977 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_flush_age": 0, 2024-09-15T21:47:02.977 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_evict_age": 0, 2024-09-15T21:47:02.978 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profile": "", 2024-09-15T21:47:02.978 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_params": { 2024-09-15T21:47:02.978 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "none" 2024-09-15T21:47:02.978 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.978 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_period": 0, 2024-09-15T21:47:02.978 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_count": 0, 2024-09-15T21:47:02.978 INFO:teuthology.orchestra.run.smithi100.stdout: "use_gmt_hitset": true, 2024-09-15T21:47:02.978 INFO:teuthology.orchestra.run.smithi100.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:47:02.978 INFO:teuthology.orchestra.run.smithi100.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:47:02.978 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:47:02.978 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:47:02.978 INFO:teuthology.orchestra.run.smithi100.stdout: "grade_table": [], 2024-09-15T21:47:02.978 INFO:teuthology.orchestra.run.smithi100.stdout: "stripe_width": 0, 2024-09-15T21:47:02.978 INFO:teuthology.orchestra.run.smithi100.stdout: "expected_num_objects": 0, 2024-09-15T21:47:02.979 INFO:teuthology.orchestra.run.smithi100.stdout: "fast_read": false, 2024-09-15T21:47:02.979 INFO:teuthology.orchestra.run.smithi100.stdout: "options": { 2024-09-15T21:47:02.979 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscale_bias": 4, 2024-09-15T21:47:02.979 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_min": 16, 2024-09-15T21:47:02.979 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_priority": 5 2024-09-15T21:47:02.979 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.979 INFO:teuthology.orchestra.run.smithi100.stdout: "application_metadata": { 2024-09-15T21:47:02.979 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs": { 2024-09-15T21:47:02.979 INFO:teuthology.orchestra.run.smithi100.stdout: "metadata": "cephfs" 2024-09-15T21:47:02.979 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:02.979 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.979 INFO:teuthology.orchestra.run.smithi100.stdout: "read_balance": { 2024-09-15T21:47:02.979 INFO:teuthology.orchestra.run.smithi100.stdout: "score_type": "Fair distribution", 2024-09-15T21:47:02.979 INFO:teuthology.orchestra.run.smithi100.stdout: "score_acting": 1.5, 2024-09-15T21:47:02.980 INFO:teuthology.orchestra.run.smithi100.stdout: "score_stable": 1.5, 2024-09-15T21:47:02.980 INFO:teuthology.orchestra.run.smithi100.stdout: "optimal_score": 1, 2024-09-15T21:47:02.980 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_acting": 1.5, 2024-09-15T21:47:02.980 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_stable": 1.5, 2024-09-15T21:47:02.980 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity_weighted": 1, 2024-09-15T21:47:02.980 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity": 1, 2024-09-15T21:47:02.980 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity_weighted": 1 2024-09-15T21:47:02.980 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:02.980 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.980 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.980 INFO:teuthology.orchestra.run.smithi100.stdout: "pool": 3, 2024-09-15T21:47:02.980 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_name": "cephfs_data", 2024-09-15T21:47:02.980 INFO:teuthology.orchestra.run.smithi100.stdout: "create_time": "2024-09-15T21:46:21.729231+0000", 2024-09-15T21:47:02.980 INFO:teuthology.orchestra.run.smithi100.stdout: "flags": 1, 2024-09-15T21:47:02.981 INFO:teuthology.orchestra.run.smithi100.stdout: "flags_names": "hashpspool", 2024-09-15T21:47:02.981 INFO:teuthology.orchestra.run.smithi100.stdout: "type": 1, 2024-09-15T21:47:02.981 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 2, 2024-09-15T21:47:02.981 INFO:teuthology.orchestra.run.smithi100.stdout: "min_size": 1, 2024-09-15T21:47:02.981 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_rule": 0, 2024-09-15T21:47:02.981 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:47:02.981 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:47:02.981 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:47:02.981 INFO:teuthology.orchestra.run.smithi100.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:47:02.981 INFO:teuthology.orchestra.run.smithi100.stdout: "is_stretch_pool": false, 2024-09-15T21:47:02.981 INFO:teuthology.orchestra.run.smithi100.stdout: "object_hash": 2, 2024-09-15T21:47:02.981 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:47:02.981 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num": 64, 2024-09-15T21:47:02.982 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num": 64, 2024-09-15T21:47:02.982 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_placement_num_target": 64, 2024-09-15T21:47:02.982 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_target": 64, 2024-09-15T21:47:02.982 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_pending": 64, 2024-09-15T21:47:02.982 INFO:teuthology.orchestra.run.smithi100.stdout: "last_pg_merge_meta": { 2024-09-15T21:47:02.982 INFO:teuthology.orchestra.run.smithi100.stdout: "source_pgid": "0.0", 2024-09-15T21:47:02.982 INFO:teuthology.orchestra.run.smithi100.stdout: "ready_epoch": 0, 2024-09-15T21:47:02.982 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_started": 0, 2024-09-15T21:47:02.982 INFO:teuthology.orchestra.run.smithi100.stdout: "last_epoch_clean": 0, 2024-09-15T21:47:02.982 INFO:teuthology.orchestra.run.smithi100.stdout: "source_version": "0'0", 2024-09-15T21:47:02.982 INFO:teuthology.orchestra.run.smithi100.stdout: "target_version": "0'0" 2024-09-15T21:47:02.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.982 INFO:teuthology.orchestra.run.smithi100.stdout: "last_change": "22", 2024-09-15T21:47:02.982 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend": "0", 2024-09-15T21:47:02.983 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:47:02.983 INFO:teuthology.orchestra.run.smithi100.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:47:02.983 INFO:teuthology.orchestra.run.smithi100.stdout: "auid": 0, 2024-09-15T21:47:02.983 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:47:02.983 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_seq": 0, 2024-09-15T21:47:02.983 INFO:teuthology.orchestra.run.smithi100.stdout: "snap_epoch": 0, 2024-09-15T21:47:02.983 INFO:teuthology.orchestra.run.smithi100.stdout: "pool_snaps": [], 2024-09-15T21:47:02.983 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps": "[]", 2024-09-15T21:47:02.983 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_bytes": 0, 2024-09-15T21:47:02.983 INFO:teuthology.orchestra.run.smithi100.stdout: "quota_max_objects": 0, 2024-09-15T21:47:02.983 INFO:teuthology.orchestra.run.smithi100.stdout: "tiers": [], 2024-09-15T21:47:02.983 INFO:teuthology.orchestra.run.smithi100.stdout: "tier_of": -1, 2024-09-15T21:47:02.983 INFO:teuthology.orchestra.run.smithi100.stdout: "read_tier": -1, 2024-09-15T21:47:02.984 INFO:teuthology.orchestra.run.smithi100.stdout: "write_tier": -1, 2024-09-15T21:47:02.984 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_mode": "none", 2024-09-15T21:47:02.984 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_bytes": 0, 2024-09-15T21:47:02.984 INFO:teuthology.orchestra.run.smithi100.stdout: "target_max_objects": 0, 2024-09-15T21:47:02.984 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:47:02.984 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:47:02.984 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:47:02.984 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_flush_age": 0, 2024-09-15T21:47:02.984 INFO:teuthology.orchestra.run.smithi100.stdout: "cache_min_evict_age": 0, 2024-09-15T21:47:02.984 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profile": "", 2024-09-15T21:47:02.984 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_params": { 2024-09-15T21:47:02.984 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "none" 2024-09-15T21:47:02.984 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.985 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_period": 0, 2024-09-15T21:47:02.985 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_count": 0, 2024-09-15T21:47:02.985 INFO:teuthology.orchestra.run.smithi100.stdout: "use_gmt_hitset": true, 2024-09-15T21:47:02.985 INFO:teuthology.orchestra.run.smithi100.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:47:02.985 INFO:teuthology.orchestra.run.smithi100.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:47:02.985 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:47:02.985 INFO:teuthology.orchestra.run.smithi100.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:47:02.985 INFO:teuthology.orchestra.run.smithi100.stdout: "grade_table": [], 2024-09-15T21:47:02.985 INFO:teuthology.orchestra.run.smithi100.stdout: "stripe_width": 0, 2024-09-15T21:47:02.985 INFO:teuthology.orchestra.run.smithi100.stdout: "expected_num_objects": 0, 2024-09-15T21:47:02.985 INFO:teuthology.orchestra.run.smithi100.stdout: "fast_read": false, 2024-09-15T21:47:02.985 INFO:teuthology.orchestra.run.smithi100.stdout: "options": { 2024-09-15T21:47:02.985 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_num_min": 64, 2024-09-15T21:47:02.985 INFO:teuthology.orchestra.run.smithi100.stdout: "target_size_ratio": 0.89999997615814209 2024-09-15T21:47:02.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.986 INFO:teuthology.orchestra.run.smithi100.stdout: "application_metadata": { 2024-09-15T21:47:02.986 INFO:teuthology.orchestra.run.smithi100.stdout: "cephfs": { 2024-09-15T21:47:02.986 INFO:teuthology.orchestra.run.smithi100.stdout: "data": "cephfs" 2024-09-15T21:47:02.986 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:02.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.986 INFO:teuthology.orchestra.run.smithi100.stdout: "read_balance": { 2024-09-15T21:47:02.986 INFO:teuthology.orchestra.run.smithi100.stdout: "score_type": "Fair distribution", 2024-09-15T21:47:02.986 INFO:teuthology.orchestra.run.smithi100.stdout: "score_acting": 1.5, 2024-09-15T21:47:02.986 INFO:teuthology.orchestra.run.smithi100.stdout: "score_stable": 1.5, 2024-09-15T21:47:02.986 INFO:teuthology.orchestra.run.smithi100.stdout: "optimal_score": 1, 2024-09-15T21:47:02.986 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_acting": 1.5, 2024-09-15T21:47:02.986 INFO:teuthology.orchestra.run.smithi100.stdout: "raw_score_stable": 1.5, 2024-09-15T21:47:02.986 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity_weighted": 1, 2024-09-15T21:47:02.986 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity": 1, 2024-09-15T21:47:02.987 INFO:teuthology.orchestra.run.smithi100.stdout: "average_primary_affinity_weighted": 1 2024-09-15T21:47:02.987 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:02.987 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:02.987 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-15T21:47:02.987 INFO:teuthology.orchestra.run.smithi100.stdout: "osds": [ 2024-09-15T21:47:02.987 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.987 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 0, 2024-09-15T21:47:02.987 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "5887900b-522d-476f-9704-cc2c5ee87a60", 2024-09-15T21:47:02.987 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-15T21:47:02.987 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-15T21:47:02.987 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-15T21:47:02.987 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-15T21:47:02.987 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-15T21:47:02.988 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-15T21:47:02.988 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 16, 2024-09-15T21:47:02.988 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 20, 2024-09-15T21:47:02.988 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-15T21:47:02.988 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-15T21:47:02.988 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-15T21:47:02.988 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:02.988 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.988 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:02.988 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6818", 2024-09-15T21:47:02.988 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3920435546 2024-09-15T21:47:02.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.988 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.988 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:02.989 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6819", 2024-09-15T21:47:02.989 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3920435546 2024-09-15T21:47:02.989 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:02.989 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:02.989 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.989 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-15T21:47:02.989 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:02.989 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.989 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:02.989 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6821", 2024-09-15T21:47:02.989 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3920435546 2024-09-15T21:47:02.989 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.989 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.990 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:02.990 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6822", 2024-09-15T21:47:02.990 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3920435546 2024-09-15T21:47:02.990 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:02.990 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:02.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.990 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-15T21:47:02.990 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:02.990 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.990 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:02.990 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6829", 2024-09-15T21:47:02.990 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3920435546 2024-09-15T21:47:02.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.990 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.991 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:02.991 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6831", 2024-09-15T21:47:02.991 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3920435546 2024-09-15T21:47:02.991 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:02.991 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:02.991 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.991 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-15T21:47:02.991 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:02.991 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.991 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:02.991 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6825", 2024-09-15T21:47:02.991 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3920435546 2024-09-15T21:47:02.991 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.991 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.992 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:02.992 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6827", 2024-09-15T21:47:02.992 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3920435546 2024-09-15T21:47:02.992 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:02.992 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:02.992 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.992 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6819/3920435546", 2024-09-15T21:47:02.992 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6822/3920435546", 2024-09-15T21:47:02.992 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6831/3920435546", 2024-09-15T21:47:02.992 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6827/3920435546", 2024-09-15T21:47:02.992 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-15T21:47:02.992 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-15T21:47:02.992 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-15T21:47:02.992 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:02.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.993 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.993 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 1, 2024-09-15T21:47:02.993 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "adc17993-5c50-4251-85e1-4c7ac62e25e6", 2024-09-15T21:47:02.993 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-15T21:47:02.993 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-15T21:47:02.993 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-15T21:47:02.993 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-15T21:47:02.993 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-15T21:47:02.993 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-15T21:47:02.993 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 14, 2024-09-15T21:47:02.993 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 20, 2024-09-15T21:47:02.993 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-15T21:47:02.993 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-15T21:47:02.994 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-15T21:47:02.994 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:02.994 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.994 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:02.994 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6800", 2024-09-15T21:47:02.994 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3802679972 2024-09-15T21:47:02.994 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.994 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.994 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:02.994 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6801", 2024-09-15T21:47:02.994 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3802679972 2024-09-15T21:47:02.994 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:02.994 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:02.994 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.995 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-15T21:47:02.995 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:02.995 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.995 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:02.995 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6802", 2024-09-15T21:47:02.995 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3802679972 2024-09-15T21:47:02.995 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.995 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.995 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:02.995 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6803", 2024-09-15T21:47:02.995 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3802679972 2024-09-15T21:47:02.995 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:02.995 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:02.995 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.996 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-15T21:47:02.996 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:02.996 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.996 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:02.996 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6806", 2024-09-15T21:47:02.996 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3802679972 2024-09-15T21:47:02.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.996 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.996 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:02.996 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6807", 2024-09-15T21:47:02.996 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3802679972 2024-09-15T21:47:02.996 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:02.996 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:02.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.997 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-15T21:47:02.997 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:02.997 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.997 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:02.997 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6804", 2024-09-15T21:47:02.997 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3802679972 2024-09-15T21:47:02.997 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.997 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.997 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:02.997 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6805", 2024-09-15T21:47:02.997 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 3802679972 2024-09-15T21:47:02.997 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:02.997 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:02.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.998 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6801/3802679972", 2024-09-15T21:47:02.998 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6803/3802679972", 2024-09-15T21:47:02.998 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6807/3802679972", 2024-09-15T21:47:02.998 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6805/3802679972", 2024-09-15T21:47:02.998 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-15T21:47:02.998 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-15T21:47:02.998 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-15T21:47:02.998 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:02.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:02.998 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.998 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 2, 2024-09-15T21:47:02.998 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "8de3b7d5-1980-4a3b-a934-264f867c7c8d", 2024-09-15T21:47:02.998 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-15T21:47:02.999 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-15T21:47:02.999 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-15T21:47:02.999 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-15T21:47:02.999 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-15T21:47:02.999 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-15T21:47:02.999 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 15, 2024-09-15T21:47:02.999 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 20, 2024-09-15T21:47:02.999 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-15T21:47:02.999 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-15T21:47:02.999 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-15T21:47:02.999 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:02.999 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:02.999 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.000 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6820", 2024-09-15T21:47:03.000 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2316787974 2024-09-15T21:47:03.000 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.000 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.000 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.000 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6823", 2024-09-15T21:47:03.000 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2316787974 2024-09-15T21:47:03.000 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.000 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.000 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.000 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-15T21:47:03.000 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.001 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.001 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.001 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6824", 2024-09-15T21:47:03.001 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2316787974 2024-09-15T21:47:03.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.001 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.001 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.001 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6826", 2024-09-15T21:47:03.001 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2316787974 2024-09-15T21:47:03.001 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.001 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.001 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-15T21:47:03.001 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.002 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.002 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.002 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6832", 2024-09-15T21:47:03.002 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2316787974 2024-09-15T21:47:03.002 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.002 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.002 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.002 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6833", 2024-09-15T21:47:03.002 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2316787974 2024-09-15T21:47:03.002 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.002 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.002 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.002 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-15T21:47:03.002 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.003 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.003 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.003 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6828", 2024-09-15T21:47:03.003 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2316787974 2024-09-15T21:47:03.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.003 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.003 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.003 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6830", 2024-09-15T21:47:03.003 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2316787974 2024-09-15T21:47:03.003 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.003 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.003 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6823/2316787974", 2024-09-15T21:47:03.003 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6826/2316787974", 2024-09-15T21:47:03.004 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6833/2316787974", 2024-09-15T21:47:03.004 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6830/2316787974", 2024-09-15T21:47:03.004 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-15T21:47:03.004 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-15T21:47:03.004 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-15T21:47:03.004 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.004 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.004 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 3, 2024-09-15T21:47:03.004 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "93dbcd1c-6a6f-49cf-bc94-e64b883de5f6", 2024-09-15T21:47:03.004 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-15T21:47:03.004 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-15T21:47:03.004 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-15T21:47:03.005 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-15T21:47:03.005 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-15T21:47:03.005 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-15T21:47:03.005 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 13, 2024-09-15T21:47:03.005 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 20, 2024-09-15T21:47:03.005 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-15T21:47:03.005 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-15T21:47:03.005 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-15T21:47:03.005 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.005 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.005 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.005 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6810", 2024-09-15T21:47:03.005 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1287562741 2024-09-15T21:47:03.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.006 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.006 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.006 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6811", 2024-09-15T21:47:03.006 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1287562741 2024-09-15T21:47:03.006 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.006 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.006 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.006 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-15T21:47:03.006 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.006 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.006 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.006 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6812", 2024-09-15T21:47:03.006 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1287562741 2024-09-15T21:47:03.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.007 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.007 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.007 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6813", 2024-09-15T21:47:03.007 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1287562741 2024-09-15T21:47:03.007 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.007 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.007 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-15T21:47:03.007 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.007 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.007 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.007 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6816", 2024-09-15T21:47:03.007 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1287562741 2024-09-15T21:47:03.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.008 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.008 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.008 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6817", 2024-09-15T21:47:03.008 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1287562741 2024-09-15T21:47:03.008 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.008 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.008 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.008 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-15T21:47:03.008 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.008 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.008 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.008 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6814", 2024-09-15T21:47:03.008 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1287562741 2024-09-15T21:47:03.008 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.008 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.009 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.009 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.100:6815", 2024-09-15T21:47:03.009 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1287562741 2024-09-15T21:47:03.009 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.009 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.009 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.009 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.100:6811/1287562741", 2024-09-15T21:47:03.009 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.100:6813/1287562741", 2024-09-15T21:47:03.009 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.100:6817/1287562741", 2024-09-15T21:47:03.009 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.100:6815/1287562741", 2024-09-15T21:47:03.009 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-15T21:47:03.009 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-15T21:47:03.009 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-15T21:47:03.010 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.010 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.010 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.010 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 4, 2024-09-15T21:47:03.010 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "7caee2a0-bab4-45a9-8109-724948c863c6", 2024-09-15T21:47:03.010 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-15T21:47:03.010 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-15T21:47:03.010 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-15T21:47:03.010 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-15T21:47:03.010 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-15T21:47:03.010 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-15T21:47:03.010 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 15, 2024-09-15T21:47:03.010 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 20, 2024-09-15T21:47:03.010 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-15T21:47:03.011 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-15T21:47:03.011 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-15T21:47:03.011 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.011 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.011 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.011 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6800", 2024-09-15T21:47:03.011 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4045445512 2024-09-15T21:47:03.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.011 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.011 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.011 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6801", 2024-09-15T21:47:03.011 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4045445512 2024-09-15T21:47:03.011 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.012 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.012 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.012 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-15T21:47:03.012 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.012 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.012 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.012 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6802", 2024-09-15T21:47:03.012 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4045445512 2024-09-15T21:47:03.012 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.012 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.012 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.012 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6803", 2024-09-15T21:47:03.012 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4045445512 2024-09-15T21:47:03.012 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.013 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.013 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-15T21:47:03.013 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.013 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.013 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.013 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6806", 2024-09-15T21:47:03.013 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4045445512 2024-09-15T21:47:03.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.013 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.013 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.013 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6807", 2024-09-15T21:47:03.013 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4045445512 2024-09-15T21:47:03.014 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.014 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.014 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-15T21:47:03.014 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.014 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.014 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.014 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6804", 2024-09-15T21:47:03.014 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4045445512 2024-09-15T21:47:03.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.014 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.014 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.014 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6805", 2024-09-15T21:47:03.015 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 4045445512 2024-09-15T21:47:03.015 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.015 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.015 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.015 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.122:6801/4045445512", 2024-09-15T21:47:03.015 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.122:6803/4045445512", 2024-09-15T21:47:03.015 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.122:6807/4045445512", 2024-09-15T21:47:03.015 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.122:6805/4045445512", 2024-09-15T21:47:03.015 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-15T21:47:03.015 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-15T21:47:03.015 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-15T21:47:03.015 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.015 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.015 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.016 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 5, 2024-09-15T21:47:03.016 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "6e64070a-671a-444b-a061-b3051a29b121", 2024-09-15T21:47:03.016 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-15T21:47:03.016 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-15T21:47:03.016 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-15T21:47:03.016 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-15T21:47:03.016 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-15T21:47:03.016 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-15T21:47:03.016 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 15, 2024-09-15T21:47:03.016 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 20, 2024-09-15T21:47:03.016 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-15T21:47:03.016 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-15T21:47:03.016 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-15T21:47:03.017 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.017 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.017 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.017 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6824", 2024-09-15T21:47:03.017 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2400668532 2024-09-15T21:47:03.017 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.017 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.017 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.017 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6825", 2024-09-15T21:47:03.017 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2400668532 2024-09-15T21:47:03.017 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.017 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.017 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.017 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-15T21:47:03.018 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.018 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.018 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.018 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6826", 2024-09-15T21:47:03.018 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2400668532 2024-09-15T21:47:03.018 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.018 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.018 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.018 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6827", 2024-09-15T21:47:03.018 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2400668532 2024-09-15T21:47:03.018 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.018 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.018 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.018 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-15T21:47:03.019 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.019 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.019 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.019 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6830", 2024-09-15T21:47:03.019 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2400668532 2024-09-15T21:47:03.019 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.019 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.019 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.019 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6831", 2024-09-15T21:47:03.019 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2400668532 2024-09-15T21:47:03.019 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.019 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.019 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.020 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-15T21:47:03.020 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.020 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.020 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.020 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6828", 2024-09-15T21:47:03.020 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2400668532 2024-09-15T21:47:03.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.020 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.020 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.020 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6829", 2024-09-15T21:47:03.020 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2400668532 2024-09-15T21:47:03.020 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.020 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.021 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.122:6825/2400668532", 2024-09-15T21:47:03.021 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.122:6827/2400668532", 2024-09-15T21:47:03.021 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.122:6831/2400668532", 2024-09-15T21:47:03.021 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.122:6829/2400668532", 2024-09-15T21:47:03.021 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-15T21:47:03.021 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-15T21:47:03.021 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-15T21:47:03.021 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.021 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.021 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 6, 2024-09-15T21:47:03.021 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3", 2024-09-15T21:47:03.021 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-15T21:47:03.021 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-15T21:47:03.022 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-15T21:47:03.022 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-15T21:47:03.022 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-15T21:47:03.022 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-15T21:47:03.022 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 15, 2024-09-15T21:47:03.022 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 20, 2024-09-15T21:47:03.022 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-15T21:47:03.022 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-15T21:47:03.022 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-15T21:47:03.022 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.023 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.023 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.023 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6808", 2024-09-15T21:47:03.023 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1437167311 2024-09-15T21:47:03.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.023 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.023 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.023 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6809", 2024-09-15T21:47:03.023 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1437167311 2024-09-15T21:47:03.023 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.023 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.023 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-15T21:47:03.024 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.024 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.024 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.024 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6810", 2024-09-15T21:47:03.024 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1437167311 2024-09-15T21:47:03.024 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.024 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.024 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.024 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6811", 2024-09-15T21:47:03.024 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1437167311 2024-09-15T21:47:03.024 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.024 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.024 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.025 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-15T21:47:03.025 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.025 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.025 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.025 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6814", 2024-09-15T21:47:03.025 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1437167311 2024-09-15T21:47:03.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.025 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.025 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.025 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6815", 2024-09-15T21:47:03.025 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1437167311 2024-09-15T21:47:03.025 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.025 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.026 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-15T21:47:03.026 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.026 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.026 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.026 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6812", 2024-09-15T21:47:03.026 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1437167311 2024-09-15T21:47:03.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.026 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.026 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.026 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6813", 2024-09-15T21:47:03.026 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 1437167311 2024-09-15T21:47:03.026 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.026 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.027 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.122:6809/1437167311", 2024-09-15T21:47:03.027 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.122:6811/1437167311", 2024-09-15T21:47:03.027 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.122:6815/1437167311", 2024-09-15T21:47:03.027 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.122:6813/1437167311", 2024-09-15T21:47:03.027 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-15T21:47:03.027 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-15T21:47:03.027 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-15T21:47:03.027 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.027 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.027 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 7, 2024-09-15T21:47:03.027 INFO:teuthology.orchestra.run.smithi100.stdout: "uuid": "9a916b6d-af82-486f-af40-4a7291c900fe", 2024-09-15T21:47:03.028 INFO:teuthology.orchestra.run.smithi100.stdout: "up": 1, 2024-09-15T21:47:03.028 INFO:teuthology.orchestra.run.smithi100.stdout: "in": 1, 2024-09-15T21:47:03.028 INFO:teuthology.orchestra.run.smithi100.stdout: "weight": 1, 2024-09-15T21:47:03.028 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_affinity": 1, 2024-09-15T21:47:03.028 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_begin": 0, 2024-09-15T21:47:03.028 INFO:teuthology.orchestra.run.smithi100.stdout: "last_clean_end": 0, 2024-09-15T21:47:03.028 INFO:teuthology.orchestra.run.smithi100.stdout: "up_from": 15, 2024-09-15T21:47:03.028 INFO:teuthology.orchestra.run.smithi100.stdout: "up_thru": 20, 2024-09-15T21:47:03.028 INFO:teuthology.orchestra.run.smithi100.stdout: "down_at": 0, 2024-09-15T21:47:03.028 INFO:teuthology.orchestra.run.smithi100.stdout: "lost_at": 0, 2024-09-15T21:47:03.028 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addrs": { 2024-09-15T21:47:03.028 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.028 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.029 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.029 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6816", 2024-09-15T21:47:03.029 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2431377162 2024-09-15T21:47:03.029 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.029 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.029 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.029 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6817", 2024-09-15T21:47:03.029 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2431377162 2024-09-15T21:47:03.029 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.029 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.029 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.029 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addrs": { 2024-09-15T21:47:03.029 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.029 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.030 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.030 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6818", 2024-09-15T21:47:03.030 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2431377162 2024-09-15T21:47:03.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.030 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.030 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.030 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6819", 2024-09-15T21:47:03.030 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2431377162 2024-09-15T21:47:03.030 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.030 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.031 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addrs": { 2024-09-15T21:47:03.031 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.031 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.031 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.031 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6822", 2024-09-15T21:47:03.031 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2431377162 2024-09-15T21:47:03.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.031 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.031 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.031 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6823", 2024-09-15T21:47:03.031 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2431377162 2024-09-15T21:47:03.031 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.031 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.032 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addrs": { 2024-09-15T21:47:03.032 INFO:teuthology.orchestra.run.smithi100.stdout: "addrvec": [ 2024-09-15T21:47:03.032 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.032 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v2", 2024-09-15T21:47:03.032 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6820", 2024-09-15T21:47:03.032 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2431377162 2024-09-15T21:47:03.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.032 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.032 INFO:teuthology.orchestra.run.smithi100.stdout: "type": "v1", 2024-09-15T21:47:03.032 INFO:teuthology.orchestra.run.smithi100.stdout: "addr": "172.21.15.122:6821", 2024-09-15T21:47:03.032 INFO:teuthology.orchestra.run.smithi100.stdout: "nonce": 2431377162 2024-09-15T21:47:03.032 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.032 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.033 INFO:teuthology.orchestra.run.smithi100.stdout: "public_addr": "172.21.15.122:6817/2431377162", 2024-09-15T21:47:03.033 INFO:teuthology.orchestra.run.smithi100.stdout: "cluster_addr": "172.21.15.122:6819/2431377162", 2024-09-15T21:47:03.033 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_back_addr": "172.21.15.122:6823/2431377162", 2024-09-15T21:47:03.033 INFO:teuthology.orchestra.run.smithi100.stdout: "heartbeat_front_addr": "172.21.15.122:6821/2431377162", 2024-09-15T21:47:03.033 INFO:teuthology.orchestra.run.smithi100.stdout: "state": [ 2024-09-15T21:47:03.033 INFO:teuthology.orchestra.run.smithi100.stdout: "exists", 2024-09-15T21:47:03.033 INFO:teuthology.orchestra.run.smithi100.stdout: "up" 2024-09-15T21:47:03.033 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:03.033 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.033 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-15T21:47:03.033 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_xinfo": [ 2024-09-15T21:47:03.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.034 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 0, 2024-09-15T21:47:03.034 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-15T21:47:03.034 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-15T21:47:03.034 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-15T21:47:03.034 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540701547738038271, 2024-09-15T21:47:03.034 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-15T21:47:03.034 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-15T21:46:09.168771+0000", 2024-09-15T21:47:03.034 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-15T21:47:03.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.034 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 1, 2024-09-15T21:47:03.034 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-15T21:47:03.035 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-15T21:47:03.035 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-15T21:47:03.035 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540701547738038271, 2024-09-15T21:47:03.035 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-15T21:47:03.035 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-15T21:46:08.886614+0000", 2024-09-15T21:47:03.035 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-15T21:47:03.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.035 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.035 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 2, 2024-09-15T21:47:03.035 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-15T21:47:03.035 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-15T21:47:03.035 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-15T21:47:03.035 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540701547738038271, 2024-09-15T21:47:03.035 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-15T21:47:03.036 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-15T21:46:09.154129+0000", 2024-09-15T21:47:03.036 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-15T21:47:03.036 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.036 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.036 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 3, 2024-09-15T21:47:03.036 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-15T21:47:03.036 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-15T21:47:03.036 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-15T21:47:03.036 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540701547738038271, 2024-09-15T21:47:03.036 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-15T21:47:03.036 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-15T21:46:08.570531+0000", 2024-09-15T21:47:03.036 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-15T21:47:03.036 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.036 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.037 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 4, 2024-09-15T21:47:03.037 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-15T21:47:03.037 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-15T21:47:03.037 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-15T21:47:03.037 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540701547738038271, 2024-09-15T21:47:03.037 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-15T21:47:03.037 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-15T21:46:08.926773+0000", 2024-09-15T21:47:03.037 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-15T21:47:03.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.037 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.037 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 5, 2024-09-15T21:47:03.037 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-15T21:47:03.037 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-15T21:47:03.038 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-15T21:47:03.038 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540701547738038271, 2024-09-15T21:47:03.038 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-15T21:47:03.038 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-15T21:46:09.146320+0000", 2024-09-15T21:47:03.038 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-15T21:47:03.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.038 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.038 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 6, 2024-09-15T21:47:03.038 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-15T21:47:03.038 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-15T21:47:03.038 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-15T21:47:03.038 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540701547738038271, 2024-09-15T21:47:03.038 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-15T21:47:03.038 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-15T21:46:09.409137+0000", 2024-09-15T21:47:03.039 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-15T21:47:03.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.039 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:03.039 INFO:teuthology.orchestra.run.smithi100.stdout: "osd": 7, 2024-09-15T21:47:03.039 INFO:teuthology.orchestra.run.smithi100.stdout: "down_stamp": "0.000000", 2024-09-15T21:47:03.039 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_probability": 0, 2024-09-15T21:47:03.039 INFO:teuthology.orchestra.run.smithi100.stdout: "laggy_interval": 0, 2024-09-15T21:47:03.039 INFO:teuthology.orchestra.run.smithi100.stdout: "features": 4540701547738038271, 2024-09-15T21:47:03.039 INFO:teuthology.orchestra.run.smithi100.stdout: "old_weight": 0, 2024-09-15T21:47:03.039 INFO:teuthology.orchestra.run.smithi100.stdout: "last_purged_snaps_scrub": "2024-09-15T21:46:09.373359+0000", 2024-09-15T21:47:03.039 INFO:teuthology.orchestra.run.smithi100.stdout: "dead_epoch": 0 2024-09-15T21:47:03.039 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.039 INFO:teuthology.orchestra.run.smithi100.stdout: ], 2024-09-15T21:47:03.039 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_upmap": [], 2024-09-15T21:47:03.040 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_upmap_items": [], 2024-09-15T21:47:03.040 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_upmap_primaries": [], 2024-09-15T21:47:03.040 INFO:teuthology.orchestra.run.smithi100.stdout: "pg_temp": [], 2024-09-15T21:47:03.040 INFO:teuthology.orchestra.run.smithi100.stdout: "primary_temp": [], 2024-09-15T21:47:03.040 INFO:teuthology.orchestra.run.smithi100.stdout: "blocklist": {}, 2024-09-15T21:47:03.040 INFO:teuthology.orchestra.run.smithi100.stdout: "range_blocklist": {}, 2024-09-15T21:47:03.040 INFO:teuthology.orchestra.run.smithi100.stdout: "erasure_code_profiles": { 2024-09-15T21:47:03.040 INFO:teuthology.orchestra.run.smithi100.stdout: "default": { 2024-09-15T21:47:03.040 INFO:teuthology.orchestra.run.smithi100.stdout: "crush-failure-domain": "osd", 2024-09-15T21:47:03.040 INFO:teuthology.orchestra.run.smithi100.stdout: "k": "2", 2024-09-15T21:47:03.040 INFO:teuthology.orchestra.run.smithi100.stdout: "m": "1", 2024-09-15T21:47:03.040 INFO:teuthology.orchestra.run.smithi100.stdout: "plugin": "jerasure", 2024-09-15T21:47:03.040 INFO:teuthology.orchestra.run.smithi100.stdout: "technique": "reed_sol_van" 2024-09-15T21:47:03.040 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:03.041 INFO:teuthology.orchestra.run.smithi100.stdout: "removed_snaps_queue": [], 2024-09-15T21:47:03.041 INFO:teuthology.orchestra.run.smithi100.stdout: "new_removed_snaps": [], 2024-09-15T21:47:03.041 INFO:teuthology.orchestra.run.smithi100.stdout: "new_purged_snaps": [], 2024-09-15T21:47:03.041 INFO:teuthology.orchestra.run.smithi100.stdout: "crush_node_flags": {}, 2024-09-15T21:47:03.041 INFO:teuthology.orchestra.run.smithi100.stdout: "device_class_flags": {}, 2024-09-15T21:47:03.041 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_mode": { 2024-09-15T21:47:03.041 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_mode_enabled": false, 2024-09-15T21:47:03.041 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_bucket_count": 0, 2024-09-15T21:47:03.041 INFO:teuthology.orchestra.run.smithi100.stdout: "degraded_stretch_mode": 0, 2024-09-15T21:47:03.041 INFO:teuthology.orchestra.run.smithi100.stdout: "recovering_stretch_mode": 0, 2024-09-15T21:47:03.041 INFO:teuthology.orchestra.run.smithi100.stdout: "stretch_mode_bucket": 0 2024-09-15T21:47:03.041 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:03.041 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:03.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.964+0000 7f6ca3542640 1 -- 172.21.15.100:0/2649006922 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6c8006c2e0 msgr2=0x7f6c8006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:03.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.964+0000 7f6ca3542640 1 --2- 172.21.15.100:0/2649006922 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6c8006c2e0 0x7f6c8006e7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0091d0 tx=0x7f6c8c02e000 comp rx=0 tx=0).stop 2024-09-15T21:47:03.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.964+0000 7f6ca3542640 1 -- 172.21.15.100:0/2649006922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9c067980 msgr2=0x7f6c9c157fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:03.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.964+0000 7f6ca3542640 1 --2- 172.21.15.100:0/2649006922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9c067980 0x7f6c9c157fb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6c9800b3e0 tx=0x7f6c9800b8a0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.964+0000 7f6ca3542640 1 -- 172.21.15.100:0/2649006922 shutdown_connections 2024-09-15T21:47:03.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.964+0000 7f6ca3542640 1 --2- 172.21.15.100:0/2649006922 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6c8006c2e0 0x7f6c8006e7a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.964+0000 7f6ca3542640 1 --2- 172.21.15.100:0/2649006922 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c9c06be50 0x7f6c9c0cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.964+0000 7f6ca3542640 1 --2- 172.21.15.100:0/2649006922 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6c9c06c230 0x7f6c9c155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.964+0000 7f6ca3542640 1 --2- 172.21.15.100:0/2649006922 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6c9c067980 0x7f6c9c157fb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.964+0000 7f6ca3542640 1 -- 172.21.15.100:0/2649006922 >> 172.21.15.100:0/2649006922 conn(0x7f6c9c06fa40 msgr2=0x7f6c9c072900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:03.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.965+0000 7f6ca3542640 1 -- 172.21.15.100:0/2649006922 shutdown_connections 2024-09-15T21:47:03.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.965+0000 7f6ca3542640 1 -- 172.21.15.100:0/2649006922 wait complete. 2024-09-15T21:47:03.045 INFO:tasks.thrashosds.thrasher:j is {'epoch': 22, 'fsid': '30e2fff4-2171-4531-8806-0e03b63f494f', 'created': '2024-09-15T21:45:59.746897+0000', 'modified': '2024-09-15T21:46:24.409383+0000', 'last_up_change': '2024-09-15T21:46:13.224709+0000', 'last_in_change': '2024-09-15T21:46:01.812129+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T21:46:11.837603+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-15T21:46:20.577982+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-15T21:46:21.729231+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '5887900b-522d-476f-9704-cc2c5ee87a60', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6818', 'nonce': 3920435546}, {'type': 'v1', 'addr': '172.21.15.100:6819', 'nonce': 3920435546}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6821', 'nonce': 3920435546}, {'type': 'v1', 'addr': '172.21.15.100:6822', 'nonce': 3920435546}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6829', 'nonce': 3920435546}, {'type': 'v1', 'addr': '172.21.15.100:6831', 'nonce': 3920435546}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6825', 'nonce': 3920435546}, {'type': 'v1', 'addr': '172.21.15.100:6827', 'nonce': 3920435546}]}, 'public_addr': '172.21.15.100:6819/3920435546', 'cluster_addr': '172.21.15.100:6822/3920435546', 'heartbeat_back_addr': '172.21.15.100:6831/3920435546', 'heartbeat_front_addr': '172.21.15.100:6827/3920435546', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'adc17993-5c50-4251-85e1-4c7ac62e25e6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6800', 'nonce': 3802679972}, {'type': 'v1', 'addr': '172.21.15.100:6801', 'nonce': 3802679972}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6802', 'nonce': 3802679972}, {'type': 'v1', 'addr': '172.21.15.100:6803', 'nonce': 3802679972}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6806', 'nonce': 3802679972}, {'type': 'v1', 'addr': '172.21.15.100:6807', 'nonce': 3802679972}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6804', 'nonce': 3802679972}, {'type': 'v1', 'addr': '172.21.15.100:6805', 'nonce': 3802679972}]}, 'public_addr': '172.21.15.100:6801/3802679972', 'cluster_addr': '172.21.15.100:6803/3802679972', 'heartbeat_back_addr': '172.21.15.100:6807/3802679972', 'heartbeat_front_addr': '172.21.15.100:6805/3802679972', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '8de3b7d5-1980-4a3b-a934-264f867c7c8d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6820', 'nonce': 2316787974}, {'type': 'v1', 'addr': '172.21.15.100:6823', 'nonce': 2316787974}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6824', 'nonce': 2316787974}, {'type': 'v1', 'addr': '172.21.15.100:6826', 'nonce': 2316787974}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6832', 'nonce': 2316787974}, {'type': 'v1', 'addr': '172.21.15.100:6833', 'nonce': 2316787974}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6828', 'nonce': 2316787974}, {'type': 'v1', 'addr': '172.21.15.100:6830', 'nonce': 2316787974}]}, 'public_addr': '172.21.15.100:6823/2316787974', 'cluster_addr': '172.21.15.100:6826/2316787974', 'heartbeat_back_addr': '172.21.15.100:6833/2316787974', 'heartbeat_front_addr': '172.21.15.100:6830/2316787974', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '93dbcd1c-6a6f-49cf-bc94-e64b883de5f6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6810', 'nonce': 1287562741}, {'type': 'v1', 'addr': '172.21.15.100:6811', 'nonce': 1287562741}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6812', 'nonce': 1287562741}, {'type': 'v1', 'addr': '172.21.15.100:6813', 'nonce': 1287562741}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6816', 'nonce': 1287562741}, {'type': 'v1', 'addr': '172.21.15.100:6817', 'nonce': 1287562741}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6814', 'nonce': 1287562741}, {'type': 'v1', 'addr': '172.21.15.100:6815', 'nonce': 1287562741}]}, 'public_addr': '172.21.15.100:6811/1287562741', 'cluster_addr': '172.21.15.100:6813/1287562741', 'heartbeat_back_addr': '172.21.15.100:6817/1287562741', 'heartbeat_front_addr': '172.21.15.100:6815/1287562741', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '7caee2a0-bab4-45a9-8109-724948c863c6', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6800', 'nonce': 4045445512}, {'type': 'v1', 'addr': '172.21.15.122:6801', 'nonce': 4045445512}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6802', 'nonce': 4045445512}, {'type': 'v1', 'addr': '172.21.15.122:6803', 'nonce': 4045445512}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6806', 'nonce': 4045445512}, {'type': 'v1', 'addr': '172.21.15.122:6807', 'nonce': 4045445512}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6804', 'nonce': 4045445512}, {'type': 'v1', 'addr': '172.21.15.122:6805', 'nonce': 4045445512}]}, 'public_addr': '172.21.15.122:6801/4045445512', 'cluster_addr': '172.21.15.122:6803/4045445512', 'heartbeat_back_addr': '172.21.15.122:6807/4045445512', 'heartbeat_front_addr': '172.21.15.122:6805/4045445512', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '6e64070a-671a-444b-a061-b3051a29b121', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6824', 'nonce': 2400668532}, {'type': 'v1', 'addr': '172.21.15.122:6825', 'nonce': 2400668532}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6826', 'nonce': 2400668532}, {'type': 'v1', 'addr': '172.21.15.122:6827', 'nonce': 2400668532}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6830', 'nonce': 2400668532}, {'type': 'v1', 'addr': '172.21.15.122:6831', 'nonce': 2400668532}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6828', 'nonce': 2400668532}, {'type': 'v1', 'addr': '172.21.15.122:6829', 'nonce': 2400668532}]}, 'public_addr': '172.21.15.122:6825/2400668532', 'cluster_addr': '172.21.15.122:6827/2400668532', 'heartbeat_back_addr': '172.21.15.122:6831/2400668532', 'heartbeat_front_addr': '172.21.15.122:6829/2400668532', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6808', 'nonce': 1437167311}, {'type': 'v1', 'addr': '172.21.15.122:6809', 'nonce': 1437167311}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6810', 'nonce': 1437167311}, {'type': 'v1', 'addr': '172.21.15.122:6811', 'nonce': 1437167311}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6814', 'nonce': 1437167311}, {'type': 'v1', 'addr': '172.21.15.122:6815', 'nonce': 1437167311}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6812', 'nonce': 1437167311}, {'type': 'v1', 'addr': '172.21.15.122:6813', 'nonce': 1437167311}]}, 'public_addr': '172.21.15.122:6809/1437167311', 'cluster_addr': '172.21.15.122:6811/1437167311', 'heartbeat_back_addr': '172.21.15.122:6815/1437167311', 'heartbeat_front_addr': '172.21.15.122:6813/1437167311', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '9a916b6d-af82-486f-af40-4a7291c900fe', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 15, 'up_thru': 20, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6816', 'nonce': 2431377162}, {'type': 'v1', 'addr': '172.21.15.122:6817', 'nonce': 2431377162}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6818', 'nonce': 2431377162}, {'type': 'v1', 'addr': '172.21.15.122:6819', 'nonce': 2431377162}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6822', 'nonce': 2431377162}, {'type': 'v1', 'addr': '172.21.15.122:6823', 'nonce': 2431377162}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6820', 'nonce': 2431377162}, {'type': 'v1', 'addr': '172.21.15.122:6821', 'nonce': 2431377162}]}, 'public_addr': '172.21.15.122:6817/2431377162', 'cluster_addr': '172.21.15.122:6819/2431377162', 'heartbeat_back_addr': '172.21.15.122:6823/2431377162', 'heartbeat_front_addr': '172.21.15.122:6821/2431377162', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:46:09.168771+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:46:08.886614+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:46:09.154129+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:46:08.570531+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:46:08.926773+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:46:09.146320+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:46:09.409137+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:46:09.373359+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T21:47:03.046 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:47:03.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:02.980+0000 7f366312a640 1 -- 172.21.15.100:0/3740202194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f365c072f20 con 0x7f364c0073d0 2024-09-15T21:47:03.047 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:03.047 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:03.047 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:47:03.048 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:03.042+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:03.144 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:03.142+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:03.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.185+0000 7fb58b9d6640 1 Processor -- start 2024-09-15T21:47:03.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.185+0000 7fb58b9d6640 1 -- start start 2024-09-15T21:47:03.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.186+0000 7fb58b9d6640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb58415aef0 0x7fb58406ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:03.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.186+0000 7fb58b9d6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb58406cfd0 0x7fb584073960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:03.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.186+0000 7fb58b9d6640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb58415b880 con 0x7fb584155330 2024-09-15T21:47:03.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.186+0000 7fb58b9d6640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fb58415bb60 con 0x7fb58415aef0 2024-09-15T21:47:03.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.186+0000 7fb58b9d6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb58406bfb0 con 0x7fb58406cfd0 2024-09-15T21:47:03.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.186+0000 7fb588f71640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb58415aef0 0x7fb58406ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:03.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.186+0000 7fb588f71640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb58415aef0 0x7fb58406ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:46020/0 (socket says 172.21.15.100:46020) 2024-09-15T21:47:03.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.186+0000 7fb588f71640 1 -- 172.21.15.100:0/1862988649 learned_addr learned my addr 172.21.15.100:0/1862988649 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:03.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.186+0000 7fb589f73640 1 --2- 172.21.15.100:0/1862988649 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb58406cfd0 0x7fb584073960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:03.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.186+0000 7fb588f71640 1 -- 172.21.15.100:0/1862988649 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb58406cfd0 msgr2=0x7fb584073960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:03.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.186+0000 7fb588f71640 1 --2- 172.21.15.100:0/1862988649 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb58406cfd0 0x7fb584073960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.186+0000 7fb588f71640 1 -- 172.21.15.100:0/1862988649 >> v1:172.21.15.122:6789/0 conn(0x7fb584155330 legacy=0x7fb584073030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:03.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.186+0000 7fb588f71640 1 -- 172.21.15.100:0/1862988649 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb58406c290 con 0x7fb58415aef0 2024-09-15T21:47:03.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.187+0000 7fb588f71640 1 --2- 172.21.15.100:0/1862988649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb58415aef0 0x7fb58406ca90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb574009870 tx=0x7fb57402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a966e29c9ded75 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:03.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.187+0000 7fb57bfff640 1 -- 172.21.15.100:0/1862988649 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574037070 con 0x7fb58415aef0 2024-09-15T21:47:03.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.187+0000 7fb57bfff640 1 -- 172.21.15.100:0/1862988649 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb574032440 con 0x7fb58415aef0 2024-09-15T21:47:03.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.187+0000 7fb57bfff640 1 -- 172.21.15.100:0/1862988649 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574032810 con 0x7fb58415aef0 2024-09-15T21:47:03.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.188+0000 7fb58b9d6640 1 -- 172.21.15.100:0/1862988649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb58415aef0 msgr2=0x7fb58406ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:03.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.188+0000 7fb58b9d6640 1 --2- 172.21.15.100:0/1862988649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb58415aef0 0x7fb58406ca90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb574009870 tx=0x7fb57402a1c0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.188+0000 7fb58b9d6640 1 -- 172.21.15.100:0/1862988649 shutdown_connections 2024-09-15T21:47:03.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.188+0000 7fb58b9d6640 1 --2- 172.21.15.100:0/1862988649 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb58406cfd0 0x7fb584073960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.188+0000 7fb58b9d6640 1 --2- 172.21.15.100:0/1862988649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb58415aef0 0x7fb58406ca90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.188+0000 7fb58b9d6640 1 -- 172.21.15.100:0/1862988649 >> 172.21.15.100:0/1862988649 conn(0x7fb5840ca430 msgr2=0x7fb5840ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:03.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.188+0000 7fb58b9d6640 1 -- 172.21.15.100:0/1862988649 shutdown_connections 2024-09-15T21:47:03.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.188+0000 7fb58b9d6640 1 -- 172.21.15.100:0/1862988649 wait complete. 2024-09-15T21:47:03.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.189+0000 7fb58b9d6640 1 Processor -- start 2024-09-15T21:47:03.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.189+0000 7fb58b9d6640 1 -- start start 2024-09-15T21:47:03.192 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:03.192 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:03.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.190+0000 7fb58b9d6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb58406cfd0 0x7fb584142970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:03.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.190+0000 7fb58b9d6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb584155330 0x7fb5840cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:03.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.190+0000 7fb58b9d6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb58415aef0 0x7fb584148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:03.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.190+0000 7fb58b9d6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb5840c84b0 con 0x7fb584155330 2024-09-15T21:47:03.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.190+0000 7fb58b9d6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb5840c8330 con 0x7fb58406cfd0 2024-09-15T21:47:03.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.190+0000 7fb58b9d6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb58415b300 con 0x7fb58415aef0 2024-09-15T21:47:03.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.190+0000 7fb588f71640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb584155330 0x7fb5840cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:03.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.190+0000 7fb588f71640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb584155330 0x7fb5840cb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49022/0 (socket says 172.21.15.100:49022) 2024-09-15T21:47:03.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.190+0000 7fb588f71640 1 -- 172.21.15.100:0/3004797924 learned_addr learned my addr 172.21.15.100:0/3004797924 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:03.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.190+0000 7fb589772640 1 --2- 172.21.15.100:0/3004797924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb58406cfd0 0x7fb584142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:03.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.190+0000 7fb589f73640 1 --2- 172.21.15.100:0/3004797924 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb58415aef0 0x7fb584148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:03.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.190+0000 7fb588f71640 1 -- 172.21.15.100:0/3004797924 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb58415aef0 msgr2=0x7fb584148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:03.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.190+0000 7fb588f71640 1 --2- 172.21.15.100:0/3004797924 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb58415aef0 0x7fb584148d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.190+0000 7fb588f71640 1 -- 172.21.15.100:0/3004797924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb58406cfd0 msgr2=0x7fb584142970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:03.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.190+0000 7fb588f71640 1 --2- 172.21.15.100:0/3004797924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb58406cfd0 0x7fb584142970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.190+0000 7fb588f71640 1 -- 172.21.15.100:0/3004797924 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5840cbde0 con 0x7fb584155330 2024-09-15T21:47:03.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.190+0000 7fb589772640 1 --2- 172.21.15.100:0/3004797924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb58406cfd0 0x7fb584142970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:03.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.191+0000 7fb588f71640 1 --2- 172.21.15.100:0/3004797924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb584155330 0x7fb5840cb730 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb574000c00 tx=0x7fb5740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:03.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.191+0000 7fb579ffb640 1 -- 172.21.15.100:0/3004797924 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574037040 con 0x7fb584155330 2024-09-15T21:47:03.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.191+0000 7fb579ffb640 1 -- 172.21.15.100:0/3004797924 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb574042070 con 0x7fb584155330 2024-09-15T21:47:03.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.191+0000 7fb58b9d6640 1 -- 172.21.15.100:0/3004797924 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5840cc170 con 0x7fb584155330 2024-09-15T21:47:03.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.191+0000 7fb579ffb640 1 -- 172.21.15.100:0/3004797924 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb574030860 con 0x7fb584155330 2024-09-15T21:47:03.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.191+0000 7fb58b9d6640 1 -- 172.21.15.100:0/3004797924 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5840cc730 con 0x7fb584155330 2024-09-15T21:47:03.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.192+0000 7fb58b9d6640 1 -- 172.21.15.100:0/3004797924 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5840769a0 con 0x7fb584155330 2024-09-15T21:47:03.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.193+0000 7fb579ffb640 1 -- 172.21.15.100:0/3004797924 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb57402ad10 con 0x7fb584155330 2024-09-15T21:47:03.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.193+0000 7fb579ffb640 1 --2- 172.21.15.100:0/3004797924 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb56806c2b0 0x7fb56806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:03.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.193+0000 7fb579ffb640 1 -- 172.21.15.100:0/3004797924 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fb5740ab8f0 con 0x7fb584155330 2024-09-15T21:47:03.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.193+0000 7fb589772640 1 --2- 172.21.15.100:0/3004797924 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb56806c2b0 0x7fb56806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:03.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.194+0000 7fb589772640 1 --2- 172.21.15.100:0/3004797924 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb56806c2b0 0x7fb56806e770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb580001b70 tx=0x7fb580009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:03.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.197+0000 7fb579ffb640 1 -- 172.21.15.100:0/3004797924 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb5840769a0 con 0x7fb584155330 2024-09-15T21:47:03.202 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:47:03.244 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:03.243+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:03.345 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:03.343+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:03.346 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:03.347 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:03.355 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:47:03.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.367+0000 7fb58b9d6640 1 -- 172.21.15.100:0/3004797924 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb58406ca90 con 0x7fb56806c2b0 2024-09-15T21:47:03.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.373+0000 7fb579ffb640 1 -- 172.21.15.100:0/3004797924 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251928 (secure 0 0 0) 0x7fb58406ca90 con 0x7fb56806c2b0 2024-09-15T21:47:03.377 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:47:03.379 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-15T21:47:03.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.376+0000 7fb58b9d6640 1 -- 172.21.15.100:0/3004797924 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb56806c2b0 msgr2=0x7fb56806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:03.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.376+0000 7fb58b9d6640 1 --2- 172.21.15.100:0/3004797924 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb56806c2b0 0x7fb56806e770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb580001b70 tx=0x7fb580009040 comp rx=0 tx=0).stop 2024-09-15T21:47:03.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.376+0000 7fb58b9d6640 1 -- 172.21.15.100:0/3004797924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb584155330 msgr2=0x7fb5840cb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:03.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.376+0000 7fb58b9d6640 1 --2- 172.21.15.100:0/3004797924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb584155330 0x7fb5840cb730 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb574000c00 tx=0x7fb5740046c0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.377+0000 7fb58b9d6640 1 -- 172.21.15.100:0/3004797924 shutdown_connections 2024-09-15T21:47:03.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.377+0000 7fb58b9d6640 1 --2- 172.21.15.100:0/3004797924 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb56806c2b0 0x7fb56806e770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.380 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.377+0000 7fb58b9d6640 1 --2- 172.21.15.100:0/3004797924 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb58415aef0 0x7fb584148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.377+0000 7fb58b9d6640 1 --2- 172.21.15.100:0/3004797924 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb584155330 0x7fb5840cb730 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.377+0000 7fb58b9d6640 1 --2- 172.21.15.100:0/3004797924 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb58406cfd0 0x7fb584142970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.377+0000 7fb58b9d6640 1 -- 172.21.15.100:0/3004797924 >> 172.21.15.100:0/3004797924 conn(0x7fb5840ca430 msgr2=0x7fb5840c9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:03.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.377+0000 7fb58b9d6640 1 -- 172.21.15.100:0/3004797924 shutdown_connections 2024-09-15T21:47:03.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.377+0000 7fb58b9d6640 1 -- 172.21.15.100:0/3004797924 wait complete. 2024-09-15T21:47:03.389 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-09-15T21:47:01.812571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":598402,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":364,"num_write_kb":2478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":296,"ondisk_log_size":296,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223496,"kb_used_data":3144,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751224,"statfs":{"total":773094113280,"available":772865253376,"internally_reserved":0,"allocated":3219456,"data_stored":1812468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1406,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003559"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835668+0000","last_change":"2024-09-15T21:46:23.428662+0000","last_active":"2024-09-15T21:46:24.835668+0000","last_peered":"2024-09-15T21:46:24.835668+0000","last_clean":"2024-09-15T21:46:24.835668+0000","last_became_active":"2024-09-15T21:46:23.427735+0000","last_became_peered":"2024-09-15T21:46:23.427735+0000","last_unstale":"2024-09-15T21:46:24.835668+0000","last_undegraded":"2024-09-15T21:46:24.835668+0000","last_fullsized":"2024-09-15T21:46:24.835668+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:29.702757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808513+0000","last_change":"2024-09-15T21:46:23.418716+0000","last_active":"2024-09-15T21:46:24.808513+0000","last_peered":"2024-09-15T21:46:24.808513+0000","last_clean":"2024-09-15T21:46:24.808513+0000","last_became_active":"2024-09-15T21:46:23.418471+0000","last_became_peered":"2024-09-15T21:46:23.418471+0000","last_unstale":"2024-09-15T21:46:24.808513+0000","last_undegraded":"2024-09-15T21:46:24.808513+0000","last_fullsized":"2024-09-15T21:46:24.808513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:49.599801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727016+0000","last_change":"2024-09-15T21:46:23.426404+0000","last_active":"2024-09-15T21:46:24.727016+0000","last_peered":"2024-09-15T21:46:24.727016+0000","last_clean":"2024-09-15T21:46:24.727016+0000","last_became_active":"2024-09-15T21:46:23.426146+0000","last_became_peered":"2024-09-15T21:46:23.426146+0000","last_unstale":"2024-09-15T21:46:24.727016+0000","last_undegraded":"2024-09-15T21:46:24.727016+0000","last_fullsized":"2024-09-15T21:46:24.727016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:23.626109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414109+0000","last_change":"2024-09-15T21:46:23.432515+0000","last_active":"2024-09-15T21:46:24.414109+0000","last_peered":"2024-09-15T21:46:24.414109+0000","last_clean":"2024-09-15T21:46:24.414109+0000","last_became_active":"2024-09-15T21:46:23.432283+0000","last_became_peered":"2024-09-15T21:46:23.432283+0000","last_unstale":"2024-09-15T21:46:24.414109+0000","last_undegraded":"2024-09-15T21:46:24.414109+0000","last_fullsized":"2024-09-15T21:46:24.414109+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:53.288080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835464+0000","last_change":"2024-09-15T21:46:23.427120+0000","last_active":"2024-09-15T21:46:24.835464+0000","last_peered":"2024-09-15T21:46:24.835464+0000","last_clean":"2024-09-15T21:46:24.835464+0000","last_became_active":"2024-09-15T21:46:23.426706+0000","last_became_peered":"2024-09-15T21:46:23.426706+0000","last_unstale":"2024-09-15T21:46:24.835464+0000","last_undegraded":"2024-09-15T21:46:24.835464+0000","last_fullsized":"2024-09-15T21:46:24.835464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:29.513446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414665+0000","last_change":"2024-09-15T21:46:23.434690+0000","last_active":"2024-09-15T21:46:24.414665+0000","last_peered":"2024-09-15T21:46:24.414665+0000","last_clean":"2024-09-15T21:46:24.414665+0000","last_became_active":"2024-09-15T21:46:23.434146+0000","last_became_peered":"2024-09-15T21:46:23.434146+0000","last_unstale":"2024-09-15T21:46:24.414665+0000","last_undegraded":"2024-09-15T21:46:24.414665+0000","last_fullsized":"2024-09-15T21:46:24.414665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:26:59.905178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835273+0000","last_change":"2024-09-15T21:46:23.429541+0000","last_active":"2024-09-15T21:46:24.835273+0000","last_peered":"2024-09-15T21:46:24.835273+0000","last_clean":"2024-09-15T21:46:24.835273+0000","last_became_active":"2024-09-15T21:46:23.429263+0000","last_became_peered":"2024-09-15T21:46:23.429263+0000","last_unstale":"2024-09-15T21:46:24.835273+0000","last_undegraded":"2024-09-15T21:46:24.835273+0000","last_fullsized":"2024-09-15T21:46:24.835273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:34:51.530981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834382+0000","last_change":"2024-09-15T21:46:23.437143+0000","last_active":"2024-09-15T21:46:24.834382+0000","last_peered":"2024-09-15T21:46:24.834382+0000","last_clean":"2024-09-15T21:46:24.834382+0000","last_became_active":"2024-09-15T21:46:23.436864+0000","last_became_peered":"2024-09-15T21:46:23.436864+0000","last_unstale":"2024-09-15T21:46:24.834382+0000","last_undegraded":"2024-09-15T21:46:24.834382+0000","last_fullsized":"2024-09-15T21:46:24.834382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:45.870403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835817+0000","last_change":"2024-09-15T21:46:23.438642+0000","last_active":"2024-09-15T21:46:24.835817+0000","last_peered":"2024-09-15T21:46:24.835817+0000","last_clean":"2024-09-15T21:46:24.835817+0000","last_became_active":"2024-09-15T21:46:23.437521+0000","last_became_peered":"2024-09-15T21:46:23.437521+0000","last_unstale":"2024-09-15T21:46:24.835817+0000","last_undegraded":"2024-09-15T21:46:24.835817+0000","last_fullsized":"2024-09-15T21:46:24.835817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:04.648853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835520+0000","last_change":"2024-09-15T21:46:23.436106+0000","last_active":"2024-09-15T21:46:24.835520+0000","last_peered":"2024-09-15T21:46:24.835520+0000","last_clean":"2024-09-15T21:46:24.835520+0000","last_became_active":"2024-09-15T21:46:23.435821+0000","last_became_peered":"2024-09-15T21:46:23.435821+0000","last_unstale":"2024-09-15T21:46:24.835520+0000","last_undegraded":"2024-09-15T21:46:24.835520+0000","last_fullsized":"2024-09-15T21:46:24.835520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:32.962320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835070+0000","last_change":"2024-09-15T21:46:23.435442+0000","last_active":"2024-09-15T21:46:24.835070+0000","last_peered":"2024-09-15T21:46:24.835070+0000","last_clean":"2024-09-15T21:46:24.835070+0000","last_became_active":"2024-09-15T21:46:23.435165+0000","last_became_peered":"2024-09-15T21:46:23.435165+0000","last_unstale":"2024-09-15T21:46:24.835070+0000","last_undegraded":"2024-09-15T21:46:24.835070+0000","last_fullsized":"2024-09-15T21:46:24.835070+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:02.361682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414099+0000","last_change":"2024-09-15T21:46:23.432534+0000","last_active":"2024-09-15T21:46:24.414099+0000","last_peered":"2024-09-15T21:46:24.414099+0000","last_clean":"2024-09-15T21:46:24.414099+0000","last_became_active":"2024-09-15T21:46:23.430170+0000","last_became_peered":"2024-09-15T21:46:23.430170+0000","last_unstale":"2024-09-15T21:46:24.414099+0000","last_undegraded":"2024-09-15T21:46:24.414099+0000","last_fullsized":"2024-09-15T21:46:24.414099+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:11.463937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834691+0000","last_change":"2024-09-15T21:46:23.427841+0000","last_active":"2024-09-15T21:46:24.834691+0000","last_peered":"2024-09-15T21:46:24.834691+0000","last_clean":"2024-09-15T21:46:24.834691+0000","last_became_active":"2024-09-15T21:46:23.427621+0000","last_became_peered":"2024-09-15T21:46:23.427621+0000","last_unstale":"2024-09-15T21:46:24.834691+0000","last_undegraded":"2024-09-15T21:46:24.834691+0000","last_fullsized":"2024-09-15T21:46:24.834691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:18.331943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.726852+0000","last_change":"2024-09-15T21:46:23.424707+0000","last_active":"2024-09-15T21:46:24.726852+0000","last_peered":"2024-09-15T21:46:24.726852+0000","last_clean":"2024-09-15T21:46:24.726852+0000","last_became_active":"2024-09-15T21:46:23.424447+0000","last_became_peered":"2024-09-15T21:46:23.424447+0000","last_unstale":"2024-09-15T21:46:24.726852+0000","last_undegraded":"2024-09-15T21:46:24.726852+0000","last_fullsized":"2024-09-15T21:46:24.726852+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:23.617214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808321+0000","last_change":"2024-09-15T21:46:23.430297+0000","last_active":"2024-09-15T21:46:24.808321+0000","last_peered":"2024-09-15T21:46:24.808321+0000","last_clean":"2024-09-15T21:46:24.808321+0000","last_became_active":"2024-09-15T21:46:23.430070+0000","last_became_peered":"2024-09-15T21:46:23.430070+0000","last_unstale":"2024-09-15T21:46:24.808321+0000","last_undegraded":"2024-09-15T21:46:24.808321+0000","last_fullsized":"2024-09-15T21:46:24.808321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:03:19.615022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414542+0000","last_change":"2024-09-15T21:46:23.430540+0000","last_active":"2024-09-15T21:46:24.414542+0000","last_peered":"2024-09-15T21:46:24.414542+0000","last_clean":"2024-09-15T21:46:24.414542+0000","last_became_active":"2024-09-15T21:46:23.430256+0000","last_became_peered":"2024-09-15T21:46:23.430256+0000","last_unstale":"2024-09-15T21:46:24.414542+0000","last_undegraded":"2024-09-15T21:46:24.414542+0000","last_fullsized":"2024-09-15T21:46:24.414542+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:39:36.539762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.415216+0000","last_change":"2024-09-15T21:46:23.427802+0000","last_active":"2024-09-15T21:46:24.415216+0000","last_peered":"2024-09-15T21:46:24.415216+0000","last_clean":"2024-09-15T21:46:24.415216+0000","last_became_active":"2024-09-15T21:46:23.427502+0000","last_became_peered":"2024-09-15T21:46:23.427502+0000","last_unstale":"2024-09-15T21:46:24.415216+0000","last_undegraded":"2024-09-15T21:46:24.415216+0000","last_fullsized":"2024-09-15T21:46:24.415216+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:31.062624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412600+0000","last_change":"2024-09-15T21:46:23.427570+0000","last_active":"2024-09-15T21:46:24.412600+0000","last_peered":"2024-09-15T21:46:24.412600+0000","last_clean":"2024-09-15T21:46:24.412600+0000","last_became_active":"2024-09-15T21:46:23.427304+0000","last_became_peered":"2024-09-15T21:46:23.427304+0000","last_unstale":"2024-09-15T21:46:24.412600+0000","last_undegraded":"2024-09-15T21:46:24.412600+0000","last_fullsized":"2024-09-15T21:46:24.412600+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:37.345106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835899+0000","last_change":"2024-09-15T21:46:23.431434+0000","last_active":"2024-09-15T21:46:24.835899+0000","last_peered":"2024-09-15T21:46:24.835899+0000","last_clean":"2024-09-15T21:46:24.835899+0000","last_became_active":"2024-09-15T21:46:23.430987+0000","last_became_peered":"2024-09-15T21:46:23.430987+0000","last_unstale":"2024-09-15T21:46:24.835899+0000","last_undegraded":"2024-09-15T21:46:24.835899+0000","last_fullsized":"2024-09-15T21:46:24.835899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:48:30.532378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835198+0000","last_change":"2024-09-15T21:46:23.434558+0000","last_active":"2024-09-15T21:46:24.835198+0000","last_peered":"2024-09-15T21:46:24.835198+0000","last_clean":"2024-09-15T21:46:24.835198+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:46:24.835198+0000","last_undegraded":"2024-09-15T21:46:24.835198+0000","last_fullsized":"2024-09-15T21:46:24.835198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:03:39.562977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727140+0000","last_change":"2024-09-15T21:46:23.428323+0000","last_active":"2024-09-15T21:46:24.727140+0000","last_peered":"2024-09-15T21:46:24.727140+0000","last_clean":"2024-09-15T21:46:24.727140+0000","last_became_active":"2024-09-15T21:46:23.428016+0000","last_became_peered":"2024-09-15T21:46:23.428016+0000","last_unstale":"2024-09-15T21:46:24.727140+0000","last_undegraded":"2024-09-15T21:46:24.727140+0000","last_fullsized":"2024-09-15T21:46:24.727140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:28:48.678550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414612+0000","last_change":"2024-09-15T21:46:23.434465+0000","last_active":"2024-09-15T21:46:24.414612+0000","last_peered":"2024-09-15T21:46:24.414612+0000","last_clean":"2024-09-15T21:46:24.414612+0000","last_became_active":"2024-09-15T21:46:23.433849+0000","last_became_peered":"2024-09-15T21:46:23.433849+0000","last_unstale":"2024-09-15T21:46:24.414612+0000","last_undegraded":"2024-09-15T21:46:24.414612+0000","last_fullsized":"2024-09-15T21:46:24.414612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:08:46.093198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834522+0000","last_change":"2024-09-15T21:46:23.431527+0000","last_active":"2024-09-15T21:46:24.834522+0000","last_peered":"2024-09-15T21:46:24.834522+0000","last_clean":"2024-09-15T21:46:24.834522+0000","last_became_active":"2024-09-15T21:46:23.430719+0000","last_became_peered":"2024-09-15T21:46:23.430719+0000","last_unstale":"2024-09-15T21:46:24.834522+0000","last_undegraded":"2024-09-15T21:46:24.834522+0000","last_fullsized":"2024-09-15T21:46:24.834522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:16.326529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414532+0000","last_change":"2024-09-15T21:46:23.432282+0000","last_active":"2024-09-15T21:46:24.414532+0000","last_peered":"2024-09-15T21:46:24.414532+0000","last_clean":"2024-09-15T21:46:24.414532+0000","last_became_active":"2024-09-15T21:46:23.432035+0000","last_became_peered":"2024-09-15T21:46:23.432035+0000","last_unstale":"2024-09-15T21:46:24.414532+0000","last_undegraded":"2024-09-15T21:46:24.414532+0000","last_fullsized":"2024-09-15T21:46:24.414532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:22.163940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834776+0000","last_change":"2024-09-15T21:46:23.432646+0000","last_active":"2024-09-15T21:46:24.834776+0000","last_peered":"2024-09-15T21:46:24.834776+0000","last_clean":"2024-09-15T21:46:24.834776+0000","last_became_active":"2024-09-15T21:46:23.432317+0000","last_became_peered":"2024-09-15T21:46:23.432317+0000","last_unstale":"2024-09-15T21:46:24.834776+0000","last_undegraded":"2024-09-15T21:46:24.834776+0000","last_fullsized":"2024-09-15T21:46:24.834776+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:49:07.706450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835541+0000","last_change":"2024-09-15T21:46:23.428183+0000","last_active":"2024-09-15T21:46:24.835541+0000","last_peered":"2024-09-15T21:46:24.835541+0000","last_clean":"2024-09-15T21:46:24.835541+0000","last_became_active":"2024-09-15T21:46:23.427886+0000","last_became_peered":"2024-09-15T21:46:23.427886+0000","last_unstale":"2024-09-15T21:46:24.835541+0000","last_undegraded":"2024-09-15T21:46:24.835541+0000","last_fullsized":"2024-09-15T21:46:24.835541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:15.584377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808369+0000","last_change":"2024-09-15T21:46:23.421749+0000","last_active":"2024-09-15T21:46:24.808369+0000","last_peered":"2024-09-15T21:46:24.808369+0000","last_clean":"2024-09-15T21:46:24.808369+0000","last_became_active":"2024-09-15T21:46:23.421501+0000","last_became_peered":"2024-09-15T21:46:23.421501+0000","last_unstale":"2024-09-15T21:46:24.808369+0000","last_undegraded":"2024-09-15T21:46:24.808369+0000","last_fullsized":"2024-09-15T21:46:24.808369+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:55.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834843+0000","last_change":"2024-09-15T21:46:23.432591+0000","last_active":"2024-09-15T21:46:24.834843+0000","last_peered":"2024-09-15T21:46:24.834843+0000","last_clean":"2024-09-15T21:46:24.834843+0000","last_became_active":"2024-09-15T21:46:23.432326+0000","last_became_peered":"2024-09-15T21:46:23.432326+0000","last_unstale":"2024-09-15T21:46:24.834843+0000","last_undegraded":"2024-09-15T21:46:24.834843+0000","last_fullsized":"2024-09-15T21:46:24.834843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:48.565579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727016+0000","last_change":"2024-09-15T21:46:23.423428+0000","last_active":"2024-09-15T21:46:24.727016+0000","last_peered":"2024-09-15T21:46:24.727016+0000","last_clean":"2024-09-15T21:46:24.727016+0000","last_became_active":"2024-09-15T21:46:23.422734+0000","last_became_peered":"2024-09-15T21:46:23.422734+0000","last_unstale":"2024-09-15T21:46:24.727016+0000","last_undegraded":"2024-09-15T21:46:24.727016+0000","last_fullsized":"2024-09-15T21:46:24.727016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:06.592891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414770+0000","last_change":"2024-09-15T21:46:23.435244+0000","last_active":"2024-09-15T21:46:24.414770+0000","last_peered":"2024-09-15T21:46:24.414770+0000","last_clean":"2024-09-15T21:46:24.414770+0000","last_became_active":"2024-09-15T21:46:23.434932+0000","last_became_peered":"2024-09-15T21:46:23.434932+0000","last_unstale":"2024-09-15T21:46:24.414770+0000","last_undegraded":"2024-09-15T21:46:24.414770+0000","last_fullsized":"2024-09-15T21:46:24.414770+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:55.810146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.412500+0000","last_change":"2024-09-15T21:46:23.437276+0000","last_active":"2024-09-15T21:46:24.412500+0000","last_peered":"2024-09-15T21:46:24.412500+0000","last_clean":"2024-09-15T21:46:24.412500+0000","last_became_active":"2024-09-15T21:46:23.436962+0000","last_became_peered":"2024-09-15T21:46:23.436962+0000","last_unstale":"2024-09-15T21:46:24.412500+0000","last_undegraded":"2024-09-15T21:46:24.412500+0000","last_fullsized":"2024-09-15T21:46:24.412500+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:13.043999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834825+0000","last_change":"2024-09-15T21:46:23.429648+0000","last_active":"2024-09-15T21:46:24.834825+0000","last_peered":"2024-09-15T21:46:24.834825+0000","last_clean":"2024-09-15T21:46:24.834825+0000","last_became_active":"2024-09-15T21:46:23.429376+0000","last_became_peered":"2024-09-15T21:46:23.429376+0000","last_unstale":"2024-09-15T21:46:24.834825+0000","last_undegraded":"2024-09-15T21:46:24.834825+0000","last_fullsized":"2024-09-15T21:46:24.834825+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:31.278283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.413663+0000","last_change":"2024-09-15T21:46:23.429847+0000","last_active":"2024-09-15T21:46:24.413663+0000","last_peered":"2024-09-15T21:46:24.413663+0000","last_clean":"2024-09-15T21:46:24.413663+0000","last_became_active":"2024-09-15T21:46:23.429442+0000","last_became_peered":"2024-09-15T21:46:23.429442+0000","last_unstale":"2024-09-15T21:46:24.413663+0000","last_undegraded":"2024-09-15T21:46:24.413663+0000","last_fullsized":"2024-09-15T21:46:24.413663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:31.232512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:34.464539+0000","last_change":"2024-09-15T21:46:22.429407+0000","last_active":"2024-09-15T21:46:34.464539+0000","last_peered":"2024-09-15T21:46:34.464539+0000","last_clean":"2024-09-15T21:46:34.464539+0000","last_became_active":"2024-09-15T21:46:22.429107+0000","last_became_peered":"2024-09-15T21:46:22.429107+0000","last_unstale":"2024-09-15T21:46:34.464539+0000","last_undegraded":"2024-09-15T21:46:34.464539+0000","last_fullsized":"2024-09-15T21:46:34.464539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:14:54.680349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727169+0000","last_change":"2024-09-15T21:46:23.434198+0000","last_active":"2024-09-15T21:46:24.727169+0000","last_peered":"2024-09-15T21:46:24.727169+0000","last_clean":"2024-09-15T21:46:24.727169+0000","last_became_active":"2024-09-15T21:46:23.433944+0000","last_became_peered":"2024-09-15T21:46:23.433944+0000","last_unstale":"2024-09-15T21:46:24.727169+0000","last_undegraded":"2024-09-15T21:46:24.727169+0000","last_fullsized":"2024-09-15T21:46:24.727169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:25.412167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'7","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.550215+0000","last_change":"2024-09-15T21:46:22.425571+0000","last_active":"2024-09-15T21:46:45.550215+0000","last_peered":"2024-09-15T21:46:45.550215+0000","last_clean":"2024-09-15T21:46:45.550215+0000","last_became_active":"2024-09-15T21:46:22.425315+0000","last_became_peered":"2024-09-15T21:46:22.425315+0000","last_unstale":"2024-09-15T21:46:45.550215+0000","last_undegraded":"2024-09-15T21:46:45.550215+0000","last_fullsized":"2024-09-15T21:46:45.550215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:31.983301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:40.480412+0000","last_change":"2024-09-15T21:46:22.425492+0000","last_active":"2024-09-15T21:46:40.480412+0000","last_peered":"2024-09-15T21:46:40.480412+0000","last_clean":"2024-09-15T21:46:40.480412+0000","last_became_active":"2024-09-15T21:46:22.425226+0000","last_became_peered":"2024-09-15T21:46:22.425226+0000","last_unstale":"2024-09-15T21:46:40.480412+0000","last_undegraded":"2024-09-15T21:46:40.480412+0000","last_fullsized":"2024-09-15T21:46:40.480412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:24.322384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834460+0000","last_change":"2024-09-15T21:46:23.429140+0000","last_active":"2024-09-15T21:46:24.834460+0000","last_peered":"2024-09-15T21:46:24.834460+0000","last_clean":"2024-09-15T21:46:24.834460+0000","last_became_active":"2024-09-15T21:46:23.428902+0000","last_became_peered":"2024-09-15T21:46:23.428902+0000","last_unstale":"2024-09-15T21:46:24.834460+0000","last_undegraded":"2024-09-15T21:46:24.834460+0000","last_fullsized":"2024-09-15T21:46:24.834460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:11.256924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.551162+0000","last_change":"2024-09-15T21:46:22.428319+0000","last_active":"2024-09-15T21:46:45.551162+0000","last_peered":"2024-09-15T21:46:45.551162+0000","last_clean":"2024-09-15T21:46:45.551162+0000","last_became_active":"2024-09-15T21:46:22.428061+0000","last_became_peered":"2024-09-15T21:46:22.428061+0000","last_unstale":"2024-09-15T21:46:45.551162+0000","last_undegraded":"2024-09-15T21:46:45.551162+0000","last_fullsized":"2024-09-15T21:46:45.551162+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:46:14.238988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808315+0000","last_change":"2024-09-15T21:46:23.427442+0000","last_active":"2024-09-15T21:46:24.808315+0000","last_peered":"2024-09-15T21:46:24.808315+0000","last_clean":"2024-09-15T21:46:24.808315+0000","last_became_active":"2024-09-15T21:46:23.427200+0000","last_became_peered":"2024-09-15T21:46:23.427200+0000","last_unstale":"2024-09-15T21:46:24.808315+0000","last_undegraded":"2024-09-15T21:46:24.808315+0000","last_fullsized":"2024-09-15T21:46:24.808315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:27.248545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:34.464224+0000","last_change":"2024-09-15T21:46:22.430599+0000","last_active":"2024-09-15T21:46:34.464224+0000","last_peered":"2024-09-15T21:46:34.464224+0000","last_clean":"2024-09-15T21:46:34.464224+0000","last_became_active":"2024-09-15T21:46:22.430327+0000","last_became_peered":"2024-09-15T21:46:22.430327+0000","last_unstale":"2024-09-15T21:46:34.464224+0000","last_undegraded":"2024-09-15T21:46:34.464224+0000","last_fullsized":"2024-09-15T21:46:34.464224+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:51.904693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414860+0000","last_change":"2024-09-15T21:46:23.434006+0000","last_active":"2024-09-15T21:46:24.414860+0000","last_peered":"2024-09-15T21:46:24.414860+0000","last_clean":"2024-09-15T21:46:24.414860+0000","last_became_active":"2024-09-15T21:46:23.433634+0000","last_became_peered":"2024-09-15T21:46:23.433634+0000","last_unstale":"2024-09-15T21:46:24.414860+0000","last_undegraded":"2024-09-15T21:46:24.414860+0000","last_fullsized":"2024-09-15T21:46:24.414860+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:22.288806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.571048+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:51.571048+0000","last_peered":"2024-09-15T21:46:51.571048+0000","last_clean":"2024-09-15T21:46:51.571048+0000","last_became_active":"2024-09-15T21:46:22.425236+0000","last_became_peered":"2024-09-15T21:46:22.425236+0000","last_unstale":"2024-09-15T21:46:51.571048+0000","last_undegraded":"2024-09-15T21:46:51.571048+0000","last_fullsized":"2024-09-15T21:46:51.571048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:45:21.332492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727237+0000","last_change":"2024-09-15T21:46:23.429587+0000","last_active":"2024-09-15T21:46:24.727237+0000","last_peered":"2024-09-15T21:46:24.727237+0000","last_clean":"2024-09-15T21:46:24.727237+0000","last_became_active":"2024-09-15T21:46:23.429334+0000","last_became_peered":"2024-09-15T21:46:23.429334+0000","last_unstale":"2024-09-15T21:46:24.727237+0000","last_undegraded":"2024-09-15T21:46:24.727237+0000","last_fullsized":"2024-09-15T21:46:24.727237+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:48:56.361550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:34.466053+0000","last_change":"2024-09-15T21:46:22.421964+0000","last_active":"2024-09-15T21:46:34.466053+0000","last_peered":"2024-09-15T21:46:34.466053+0000","last_clean":"2024-09-15T21:46:34.466053+0000","last_became_active":"2024-09-15T21:46:22.421661+0000","last_became_peered":"2024-09-15T21:46:22.421661+0000","last_unstale":"2024-09-15T21:46:34.466053+0000","last_undegraded":"2024-09-15T21:46:34.466053+0000","last_fullsized":"2024-09-15T21:46:34.466053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:20:07.395070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414253+0000","last_change":"2024-09-15T21:46:23.427416+0000","last_active":"2024-09-15T21:46:24.414253+0000","last_peered":"2024-09-15T21:46:24.414253+0000","last_clean":"2024-09-15T21:46:24.414253+0000","last_became_active":"2024-09-15T21:46:23.427057+0000","last_became_peered":"2024-09-15T21:46:23.427057+0000","last_unstale":"2024-09-15T21:46:24.414253+0000","last_undegraded":"2024-09-15T21:46:24.414253+0000","last_fullsized":"2024-09-15T21:46:24.414253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:42.671501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'10","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.572146+0000","last_change":"2024-09-15T21:46:22.426151+0000","last_active":"2024-09-15T21:46:51.572146+0000","last_peered":"2024-09-15T21:46:51.572146+0000","last_clean":"2024-09-15T21:46:51.572146+0000","last_became_active":"2024-09-15T21:46:22.425866+0000","last_became_peered":"2024-09-15T21:46:22.425866+0000","last_unstale":"2024-09-15T21:46:51.572146+0000","last_undegraded":"2024-09-15T21:46:51.572146+0000","last_fullsized":"2024-09-15T21:46:51.572146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:10.684265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835569+0000","last_change":"2024-09-15T21:46:23.432292+0000","last_active":"2024-09-15T21:46:24.835569+0000","last_peered":"2024-09-15T21:46:24.835569+0000","last_clean":"2024-09-15T21:46:24.835569+0000","last_became_active":"2024-09-15T21:46:23.431877+0000","last_became_peered":"2024-09-15T21:46:23.431877+0000","last_unstale":"2024-09-15T21:46:24.835569+0000","last_undegraded":"2024-09-15T21:46:24.835569+0000","last_fullsized":"2024-09-15T21:46:24.835569+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:16.857279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'9","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.567901+0000","last_change":"2024-09-15T21:46:22.427415+0000","last_active":"2024-09-15T21:46:51.567901+0000","last_peered":"2024-09-15T21:46:51.567901+0000","last_clean":"2024-09-15T21:46:51.567901+0000","last_became_active":"2024-09-15T21:46:22.427087+0000","last_became_peered":"2024-09-15T21:46:22.427087+0000","last_unstale":"2024-09-15T21:46:51.567901+0000","last_undegraded":"2024-09-15T21:46:51.567901+0000","last_fullsized":"2024-09-15T21:46:51.567901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:09.365202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.726943+0000","last_change":"2024-09-15T21:46:23.427238+0000","last_active":"2024-09-15T21:46:24.726943+0000","last_peered":"2024-09-15T21:46:24.726943+0000","last_clean":"2024-09-15T21:46:24.726943+0000","last_became_active":"2024-09-15T21:46:23.426993+0000","last_became_peered":"2024-09-15T21:46:23.426993+0000","last_unstale":"2024-09-15T21:46:24.726943+0000","last_undegraded":"2024-09-15T21:46:24.726943+0000","last_fullsized":"2024-09-15T21:46:24.726943+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:52:24.175554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.550646+0000","last_change":"2024-09-15T21:46:22.427164+0000","last_active":"2024-09-15T21:46:45.550646+0000","last_peered":"2024-09-15T21:46:45.550646+0000","last_clean":"2024-09-15T21:46:45.550646+0000","last_became_active":"2024-09-15T21:46:22.426917+0000","last_became_peered":"2024-09-15T21:46:22.426917+0000","last_unstale":"2024-09-15T21:46:45.550646+0000","last_undegraded":"2024-09-15T21:46:45.550646+0000","last_fullsized":"2024-09-15T21:46:45.550646+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:46.471935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835504+0000","last_change":"2024-09-15T21:46:23.433924+0000","last_active":"2024-09-15T21:46:24.835504+0000","last_peered":"2024-09-15T21:46:24.835504+0000","last_clean":"2024-09-15T21:46:24.835504+0000","last_became_active":"2024-09-15T21:46:23.433606+0000","last_became_peered":"2024-09-15T21:46:23.433606+0000","last_unstale":"2024-09-15T21:46:24.835504+0000","last_undegraded":"2024-09-15T21:46:24.835504+0000","last_fullsized":"2024-09-15T21:46:24.835504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:09:35.744149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.549348+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:46:45.549348+0000","last_peered":"2024-09-15T21:46:45.549348+0000","last_clean":"2024-09-15T21:46:45.549348+0000","last_became_active":"2024-09-15T21:46:22.425237+0000","last_became_peered":"2024-09-15T21:46:22.425237+0000","last_unstale":"2024-09-15T21:46:45.549348+0000","last_undegraded":"2024-09-15T21:46:45.549348+0000","last_fullsized":"2024-09-15T21:46:45.549348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:59.460420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834853+0000","last_change":"2024-09-15T21:46:23.427515+0000","last_active":"2024-09-15T21:46:24.834853+0000","last_peered":"2024-09-15T21:46:24.834853+0000","last_clean":"2024-09-15T21:46:24.834853+0000","last_became_active":"2024-09-15T21:46:23.427116+0000","last_became_peered":"2024-09-15T21:46:23.427116+0000","last_unstale":"2024-09-15T21:46:24.834853+0000","last_undegraded":"2024-09-15T21:46:24.834853+0000","last_fullsized":"2024-09-15T21:46:24.834853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:35.499970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'17","reported_seq":31,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.550667+0000","last_change":"2024-09-15T21:46:22.419905+0000","last_active":"2024-09-15T21:46:45.550667+0000","last_peered":"2024-09-15T21:46:45.550667+0000","last_clean":"2024-09-15T21:46:45.550667+0000","last_became_active":"2024-09-15T21:46:22.419620+0000","last_became_peered":"2024-09-15T21:46:22.419620+0000","last_unstale":"2024-09-15T21:46:45.550667+0000","last_undegraded":"2024-09-15T21:46:45.550667+0000","last_fullsized":"2024-09-15T21:46:45.550667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:26:51.215156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835459+0000","last_change":"2024-09-15T21:46:23.432975+0000","last_active":"2024-09-15T21:46:24.835459+0000","last_peered":"2024-09-15T21:46:24.835459+0000","last_clean":"2024-09-15T21:46:24.835459+0000","last_became_active":"2024-09-15T21:46:23.432718+0000","last_became_peered":"2024-09-15T21:46:23.432718+0000","last_unstale":"2024-09-15T21:46:24.835459+0000","last_undegraded":"2024-09-15T21:46:24.835459+0000","last_fullsized":"2024-09-15T21:46:24.835459+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:59.789691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.419202+0000","last_change":"2024-09-15T21:46:22.429661+0000","last_active":"2024-09-15T21:46:28.419202+0000","last_peered":"2024-09-15T21:46:28.419202+0000","last_clean":"2024-09-15T21:46:28.419202+0000","last_became_active":"2024-09-15T21:46:22.429250+0000","last_became_peered":"2024-09-15T21:46:22.429250+0000","last_unstale":"2024-09-15T21:46:28.419202+0000","last_undegraded":"2024-09-15T21:46:28.419202+0000","last_fullsized":"2024-09-15T21:46:28.419202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:29.341739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835590+0000","last_change":"2024-09-15T21:46:23.435003+0000","last_active":"2024-09-15T21:46:24.835590+0000","last_peered":"2024-09-15T21:46:24.835590+0000","last_clean":"2024-09-15T21:46:24.835590+0000","last_became_active":"2024-09-15T21:46:23.434738+0000","last_became_peered":"2024-09-15T21:46:23.434738+0000","last_unstale":"2024-09-15T21:46:24.835590+0000","last_undegraded":"2024-09-15T21:46:24.835590+0000","last_fullsized":"2024-09-15T21:46:24.835590+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:53:15.814759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.571870+0000","last_change":"2024-09-15T21:46:22.425870+0000","last_active":"2024-09-15T21:46:51.571870+0000","last_peered":"2024-09-15T21:46:51.571870+0000","last_clean":"2024-09-15T21:46:51.571870+0000","last_became_active":"2024-09-15T21:46:22.425588+0000","last_became_peered":"2024-09-15T21:46:22.425588+0000","last_unstale":"2024-09-15T21:46:51.571870+0000","last_undegraded":"2024-09-15T21:46:51.571870+0000","last_fullsized":"2024-09-15T21:46:51.571870+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:40:07.586608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835837+0000","last_change":"2024-09-15T21:46:23.434657+0000","last_active":"2024-09-15T21:46:24.835837+0000","last_peered":"2024-09-15T21:46:24.835837+0000","last_clean":"2024-09-15T21:46:24.835837+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:46:24.835837+0000","last_undegraded":"2024-09-15T21:46:24.835837+0000","last_fullsized":"2024-09-15T21:46:24.835837+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:02.545810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.549133+0000","last_change":"2024-09-15T21:46:22.425072+0000","last_active":"2024-09-15T21:46:45.549133+0000","last_peered":"2024-09-15T21:46:45.549133+0000","last_clean":"2024-09-15T21:46:45.549133+0000","last_became_active":"2024-09-15T21:46:22.424771+0000","last_became_peered":"2024-09-15T21:46:22.424771+0000","last_unstale":"2024-09-15T21:46:45.549133+0000","last_undegraded":"2024-09-15T21:46:45.549133+0000","last_fullsized":"2024-09-15T21:46:45.549133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:26.256785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414457+0000","last_change":"2024-09-15T21:46:23.432497+0000","last_active":"2024-09-15T21:46:24.414457+0000","last_peered":"2024-09-15T21:46:24.414457+0000","last_clean":"2024-09-15T21:46:24.414457+0000","last_became_active":"2024-09-15T21:46:23.431969+0000","last_became_peered":"2024-09-15T21:46:23.431969+0000","last_unstale":"2024-09-15T21:46:24.414457+0000","last_undegraded":"2024-09-15T21:46:24.414457+0000","last_fullsized":"2024-09-15T21:46:24.414457+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:32:56.359477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.549727+0000","last_change":"2024-09-15T21:46:22.428668+0000","last_active":"2024-09-15T21:46:45.549727+0000","last_peered":"2024-09-15T21:46:45.549727+0000","last_clean":"2024-09-15T21:46:45.549727+0000","last_became_active":"2024-09-15T21:46:22.428341+0000","last_became_peered":"2024-09-15T21:46:22.428341+0000","last_unstale":"2024-09-15T21:46:45.549727+0000","last_undegraded":"2024-09-15T21:46:45.549727+0000","last_fullsized":"2024-09-15T21:46:45.549727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:29:45.521090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835717+0000","last_change":"2024-09-15T21:46:23.438665+0000","last_active":"2024-09-15T21:46:24.835717+0000","last_peered":"2024-09-15T21:46:24.835717+0000","last_clean":"2024-09-15T21:46:24.835717+0000","last_became_active":"2024-09-15T21:46:23.437606+0000","last_became_peered":"2024-09-15T21:46:23.437606+0000","last_unstale":"2024-09-15T21:46:24.835717+0000","last_undegraded":"2024-09-15T21:46:24.835717+0000","last_fullsized":"2024-09-15T21:46:24.835717+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:14:53.751884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":171,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.835126+0000","last_change":"2024-09-15T21:46:14.328094+0000","last_active":"2024-09-15T21:46:24.835126+0000","last_peered":"2024-09-15T21:46:24.835126+0000","last_clean":"2024-09-15T21:46:24.835126+0000","last_became_active":"2024-09-15T21:46:14.326991+0000","last_became_peered":"2024-09-15T21:46:14.326991+0000","last_unstale":"2024-09-15T21:46:24.835126+0000","last_undegraded":"2024-09-15T21:46:24.835126+0000","last_fullsized":"2024-09-15T21:46:24.835126+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_clean_scrub_stamp":"2024-09-15T21:46:12.223400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:54:27.704130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'6","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.550469+0000","last_change":"2024-09-15T21:46:22.428887+0000","last_active":"2024-09-15T21:46:45.550469+0000","last_peered":"2024-09-15T21:46:45.550469+0000","last_clean":"2024-09-15T21:46:45.550469+0000","last_became_active":"2024-09-15T21:46:22.428619+0000","last_became_peered":"2024-09-15T21:46:22.428619+0000","last_unstale":"2024-09-15T21:46:45.550469+0000","last_undegraded":"2024-09-15T21:46:45.550469+0000","last_fullsized":"2024-09-15T21:46:45.550469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:48.434763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834644+0000","last_change":"2024-09-15T21:46:23.435459+0000","last_active":"2024-09-15T21:46:24.834644+0000","last_peered":"2024-09-15T21:46:24.834644+0000","last_clean":"2024-09-15T21:46:24.834644+0000","last_became_active":"2024-09-15T21:46:23.435141+0000","last_became_peered":"2024-09-15T21:46:23.435141+0000","last_unstale":"2024-09-15T21:46:24.834644+0000","last_undegraded":"2024-09-15T21:46:24.834644+0000","last_fullsized":"2024-09-15T21:46:24.834644+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:40:50.707124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:40.480640+0000","last_change":"2024-09-15T21:46:22.431113+0000","last_active":"2024-09-15T21:46:40.480640+0000","last_peered":"2024-09-15T21:46:40.480640+0000","last_clean":"2024-09-15T21:46:40.480640+0000","last_became_active":"2024-09-15T21:46:22.430832+0000","last_became_peered":"2024-09-15T21:46:22.430832+0000","last_unstale":"2024-09-15T21:46:40.480640+0000","last_undegraded":"2024-09-15T21:46:40.480640+0000","last_fullsized":"2024-09-15T21:46:40.480640+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:28.849259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.726839+0000","last_change":"2024-09-15T21:46:23.430396+0000","last_active":"2024-09-15T21:46:24.726839+0000","last_peered":"2024-09-15T21:46:24.726839+0000","last_clean":"2024-09-15T21:46:24.726839+0000","last_became_active":"2024-09-15T21:46:23.430138+0000","last_became_peered":"2024-09-15T21:46:23.430138+0000","last_unstale":"2024-09-15T21:46:24.726839+0000","last_undegraded":"2024-09-15T21:46:24.726839+0000","last_fullsized":"2024-09-15T21:46:24.726839+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:50.622949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:40.483066+0000","last_change":"2024-09-15T21:46:22.425808+0000","last_active":"2024-09-15T21:46:40.483066+0000","last_peered":"2024-09-15T21:46:40.483066+0000","last_clean":"2024-09-15T21:46:40.483066+0000","last_became_active":"2024-09-15T21:46:22.425510+0000","last_became_peered":"2024-09-15T21:46:22.425510+0000","last_unstale":"2024-09-15T21:46:40.483066+0000","last_undegraded":"2024-09-15T21:46:40.483066+0000","last_fullsized":"2024-09-15T21:46:40.483066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:11:00.338076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808338+0000","last_change":"2024-09-15T21:46:23.427051+0000","last_active":"2024-09-15T21:46:24.808338+0000","last_peered":"2024-09-15T21:46:24.808338+0000","last_clean":"2024-09-15T21:46:24.808338+0000","last_became_active":"2024-09-15T21:46:23.426185+0000","last_became_peered":"2024-09-15T21:46:23.426185+0000","last_unstale":"2024-09-15T21:46:24.808338+0000","last_undegraded":"2024-09-15T21:46:24.808338+0000","last_fullsized":"2024-09-15T21:46:24.808338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:48.318405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.572233+0000","last_change":"2024-09-15T21:46:22.424645+0000","last_active":"2024-09-15T21:46:51.572233+0000","last_peered":"2024-09-15T21:46:51.572233+0000","last_clean":"2024-09-15T21:46:51.572233+0000","last_became_active":"2024-09-15T21:46:22.421229+0000","last_became_peered":"2024-09-15T21:46:22.421229+0000","last_unstale":"2024-09-15T21:46:51.572233+0000","last_undegraded":"2024-09-15T21:46:51.572233+0000","last_fullsized":"2024-09-15T21:46:51.572233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:33:49.165670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414000+0000","last_change":"2024-09-15T21:46:23.425699+0000","last_active":"2024-09-15T21:46:24.414000+0000","last_peered":"2024-09-15T21:46:24.414000+0000","last_clean":"2024-09-15T21:46:24.414000+0000","last_became_active":"2024-09-15T21:46:23.425416+0000","last_became_peered":"2024-09-15T21:46:23.425416+0000","last_unstale":"2024-09-15T21:46:24.414000+0000","last_undegraded":"2024-09-15T21:46:24.414000+0000","last_fullsized":"2024-09-15T21:46:24.414000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:00.325179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"22'1","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808477+0000","last_change":"2024-09-15T21:46:22.426492+0000","last_active":"2024-09-15T21:46:24.808477+0000","last_peered":"2024-09-15T21:46:24.808477+0000","last_clean":"2024-09-15T21:46:24.808477+0000","last_became_active":"2024-09-15T21:46:22.426210+0000","last_became_peered":"2024-09-15T21:46:22.426210+0000","last_unstale":"2024-09-15T21:46:24.808477+0000","last_undegraded":"2024-09-15T21:46:24.808477+0000","last_fullsized":"2024-09-15T21:46:24.808477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:05.248091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414861+0000","last_change":"2024-09-15T21:46:23.429837+0000","last_active":"2024-09-15T21:46:24.414861+0000","last_peered":"2024-09-15T21:46:24.414861+0000","last_clean":"2024-09-15T21:46:24.414861+0000","last_became_active":"2024-09-15T21:46:23.429254+0000","last_became_peered":"2024-09-15T21:46:23.429254+0000","last_unstale":"2024-09-15T21:46:24.414861+0000","last_undegraded":"2024-09-15T21:46:24.414861+0000","last_fullsized":"2024-09-15T21:46:24.414861+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:39:58.465583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.551360+0000","last_change":"2024-09-15T21:46:22.430710+0000","last_active":"2024-09-15T21:46:45.551360+0000","last_peered":"2024-09-15T21:46:45.551360+0000","last_clean":"2024-09-15T21:46:45.551360+0000","last_became_active":"2024-09-15T21:46:22.430375+0000","last_became_peered":"2024-09-15T21:46:22.430375+0000","last_unstale":"2024-09-15T21:46:45.551360+0000","last_undegraded":"2024-09-15T21:46:45.551360+0000","last_fullsized":"2024-09-15T21:46:45.551360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:04.081179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.808447+0000","last_change":"2024-09-15T21:46:23.426729+0000","last_active":"2024-09-15T21:46:24.808447+0000","last_peered":"2024-09-15T21:46:24.808447+0000","last_clean":"2024-09-15T21:46:24.808447+0000","last_became_active":"2024-09-15T21:46:23.426495+0000","last_became_peered":"2024-09-15T21:46:23.426495+0000","last_unstale":"2024-09-15T21:46:24.808447+0000","last_undegraded":"2024-09-15T21:46:24.808447+0000","last_fullsized":"2024-09-15T21:46:24.808447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:03.277997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'7","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.549857+0000","last_change":"2024-09-15T21:46:22.427137+0000","last_active":"2024-09-15T21:46:45.549857+0000","last_peered":"2024-09-15T21:46:45.549857+0000","last_clean":"2024-09-15T21:46:45.549857+0000","last_became_active":"2024-09-15T21:46:22.426855+0000","last_became_peered":"2024-09-15T21:46:22.426855+0000","last_unstale":"2024-09-15T21:46:45.549857+0000","last_undegraded":"2024-09-15T21:46:45.549857+0000","last_fullsized":"2024-09-15T21:46:45.549857+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:40:14.277940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834714+0000","last_change":"2024-09-15T21:46:23.427327+0000","last_active":"2024-09-15T21:46:24.834714+0000","last_peered":"2024-09-15T21:46:24.834714+0000","last_clean":"2024-09-15T21:46:24.834714+0000","last_became_active":"2024-09-15T21:46:23.427096+0000","last_became_peered":"2024-09-15T21:46:23.427096+0000","last_unstale":"2024-09-15T21:46:24.834714+0000","last_undegraded":"2024-09-15T21:46:24.834714+0000","last_fullsized":"2024-09-15T21:46:24.834714+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:32.370390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'7","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.570649+0000","last_change":"2024-09-15T21:46:22.427798+0000","last_active":"2024-09-15T21:46:51.570649+0000","last_peered":"2024-09-15T21:46:51.570649+0000","last_clean":"2024-09-15T21:46:51.570649+0000","last_became_active":"2024-09-15T21:46:22.427537+0000","last_became_peered":"2024-09-15T21:46:22.427537+0000","last_unstale":"2024-09-15T21:46:51.570649+0000","last_undegraded":"2024-09-15T21:46:51.570649+0000","last_fullsized":"2024-09-15T21:46:51.570649+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:18:54.963834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727157+0000","last_change":"2024-09-15T21:46:23.424266+0000","last_active":"2024-09-15T21:46:24.727157+0000","last_peered":"2024-09-15T21:46:24.727157+0000","last_clean":"2024-09-15T21:46:24.727157+0000","last_became_active":"2024-09-15T21:46:23.424007+0000","last_became_peered":"2024-09-15T21:46:23.424007+0000","last_unstale":"2024-09-15T21:46:24.727157+0000","last_undegraded":"2024-09-15T21:46:24.727157+0000","last_fullsized":"2024-09-15T21:46:24.727157+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:23.492253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:34.461771+0000","last_change":"2024-09-15T21:46:22.430901+0000","last_active":"2024-09-15T21:46:34.461771+0000","last_peered":"2024-09-15T21:46:34.461771+0000","last_clean":"2024-09-15T21:46:34.461771+0000","last_became_active":"2024-09-15T21:46:22.430452+0000","last_became_peered":"2024-09-15T21:46:22.430452+0000","last_unstale":"2024-09-15T21:46:34.461771+0000","last_undegraded":"2024-09-15T21:46:34.461771+0000","last_fullsized":"2024-09-15T21:46:34.461771+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:59:45.663976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.413661+0000","last_change":"2024-09-15T21:46:23.430836+0000","last_active":"2024-09-15T21:46:24.413661+0000","last_peered":"2024-09-15T21:46:24.413661+0000","last_clean":"2024-09-15T21:46:24.413661+0000","last_became_active":"2024-09-15T21:46:23.429283+0000","last_became_peered":"2024-09-15T21:46:23.429283+0000","last_unstale":"2024-09-15T21:46:24.413661+0000","last_undegraded":"2024-09-15T21:46:24.413661+0000","last_fullsized":"2024-09-15T21:46:24.413661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:30.571376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'7","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.569263+0000","last_change":"2024-09-15T21:46:22.423795+0000","last_active":"2024-09-15T21:46:51.569263+0000","last_peered":"2024-09-15T21:46:51.569263+0000","last_clean":"2024-09-15T21:46:51.569263+0000","last_became_active":"2024-09-15T21:46:22.422327+0000","last_became_peered":"2024-09-15T21:46:22.422327+0000","last_unstale":"2024-09-15T21:46:51.569263+0000","last_undegraded":"2024-09-15T21:46:51.569263+0000","last_fullsized":"2024-09-15T21:46:51.569263+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:16.670599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834522+0000","last_change":"2024-09-15T21:46:23.428634+0000","last_active":"2024-09-15T21:46:24.834522+0000","last_peered":"2024-09-15T21:46:24.834522+0000","last_clean":"2024-09-15T21:46:24.834522+0000","last_became_active":"2024-09-15T21:46:23.428301+0000","last_became_peered":"2024-09-15T21:46:23.428301+0000","last_unstale":"2024-09-15T21:46:24.834522+0000","last_undegraded":"2024-09-15T21:46:24.834522+0000","last_fullsized":"2024-09-15T21:46:24.834522+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:59.492942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:45.547973+0000","last_change":"2024-09-15T21:46:22.425810+0000","last_active":"2024-09-15T21:46:45.547973+0000","last_peered":"2024-09-15T21:46:45.547973+0000","last_clean":"2024-09-15T21:46:45.547973+0000","last_became_active":"2024-09-15T21:46:22.425478+0000","last_became_peered":"2024-09-15T21:46:22.425478+0000","last_unstale":"2024-09-15T21:46:45.547973+0000","last_undegraded":"2024-09-15T21:46:45.547973+0000","last_fullsized":"2024-09-15T21:46:45.547973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:53.296599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834743+0000","last_change":"2024-09-15T21:46:23.426054+0000","last_active":"2024-09-15T21:46:24.834743+0000","last_peered":"2024-09-15T21:46:24.834743+0000","last_clean":"2024-09-15T21:46:24.834743+0000","last_became_active":"2024-09-15T21:46:23.425547+0000","last_became_peered":"2024-09-15T21:46:23.425547+0000","last_unstale":"2024-09-15T21:46:24.834743+0000","last_undegraded":"2024-09-15T21:46:24.834743+0000","last_fullsized":"2024-09-15T21:46:24.834743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:56.838503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'4","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:34.458317+0000","last_change":"2024-09-15T21:46:22.428524+0000","last_active":"2024-09-15T21:46:34.458317+0000","last_peered":"2024-09-15T21:46:34.458317+0000","last_clean":"2024-09-15T21:46:34.458317+0000","last_became_active":"2024-09-15T21:46:22.428043+0000","last_became_peered":"2024-09-15T21:46:22.428043+0000","last_unstale":"2024-09-15T21:46:34.458317+0000","last_undegraded":"2024-09-15T21:46:34.458317+0000","last_fullsized":"2024-09-15T21:46:34.458317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:08.393381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834850+0000","last_change":"2024-09-15T21:46:23.434719+0000","last_active":"2024-09-15T21:46:24.834850+0000","last_peered":"2024-09-15T21:46:24.834850+0000","last_clean":"2024-09-15T21:46:24.834850+0000","last_became_active":"2024-09-15T21:46:23.434448+0000","last_became_peered":"2024-09-15T21:46:23.434448+0000","last_unstale":"2024-09-15T21:46:24.834850+0000","last_undegraded":"2024-09-15T21:46:24.834850+0000","last_fullsized":"2024-09-15T21:46:24.834850+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:14.901671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414691+0000","last_change":"2024-09-15T21:46:23.430058+0000","last_active":"2024-09-15T21:46:24.414691+0000","last_peered":"2024-09-15T21:46:24.414691+0000","last_clean":"2024-09-15T21:46:24.414691+0000","last_became_active":"2024-09-15T21:46:23.429598+0000","last_became_peered":"2024-09-15T21:46:23.429598+0000","last_unstale":"2024-09-15T21:46:24.414691+0000","last_undegraded":"2024-09-15T21:46:24.414691+0000","last_fullsized":"2024-09-15T21:46:24.414691+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:01:44.142207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'5","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.570689+0000","last_change":"2024-09-15T21:46:22.431172+0000","last_active":"2024-09-15T21:46:51.570689+0000","last_peered":"2024-09-15T21:46:51.570689+0000","last_clean":"2024-09-15T21:46:51.570689+0000","last_became_active":"2024-09-15T21:46:22.430800+0000","last_became_peered":"2024-09-15T21:46:22.430800+0000","last_unstale":"2024-09-15T21:46:51.570689+0000","last_undegraded":"2024-09-15T21:46:51.570689+0000","last_fullsized":"2024-09-15T21:46:51.570689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:42:55.072466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.414264+0000","last_change":"2024-09-15T21:46:23.434353+0000","last_active":"2024-09-15T21:46:24.414264+0000","last_peered":"2024-09-15T21:46:24.414264+0000","last_clean":"2024-09-15T21:46:24.414264+0000","last_became_active":"2024-09-15T21:46:23.434097+0000","last_became_peered":"2024-09-15T21:46:23.434097+0000","last_unstale":"2024-09-15T21:46:24.414264+0000","last_undegraded":"2024-09-15T21:46:24.414264+0000","last_fullsized":"2024-09-15T21:46:24.414264+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:40:46.007227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'12","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.571743+0000","last_change":"2024-09-15T21:46:22.429870+0000","last_active":"2024-09-15T21:46:51.571743+0000","last_peered":"2024-09-15T21:46:51.571743+0000","last_clean":"2024-09-15T21:46:51.571743+0000","last_became_active":"2024-09-15T21:46:22.429578+0000","last_became_peered":"2024-09-15T21:46:22.429578+0000","last_unstale":"2024-09-15T21:46:51.571743+0000","last_undegraded":"2024-09-15T21:46:51.571743+0000","last_fullsized":"2024-09-15T21:46:51.571743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:41.678078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.727142+0000","last_change":"2024-09-15T21:46:23.429604+0000","last_active":"2024-09-15T21:46:24.727142+0000","last_peered":"2024-09-15T21:46:24.727142+0000","last_clean":"2024-09-15T21:46:24.727142+0000","last_became_active":"2024-09-15T21:46:23.429355+0000","last_became_peered":"2024-09-15T21:46:23.429355+0000","last_unstale":"2024-09-15T21:46:24.727142+0000","last_undegraded":"2024-09-15T21:46:24.727142+0000","last_fullsized":"2024-09-15T21:46:24.727142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:45.360901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'2","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:28.416728+0000","last_change":"2024-09-15T21:46:22.428529+0000","last_active":"2024-09-15T21:46:28.416728+0000","last_peered":"2024-09-15T21:46:28.416728+0000","last_clean":"2024-09-15T21:46:28.416728+0000","last_became_active":"2024-09-15T21:46:22.428263+0000","last_became_peered":"2024-09-15T21:46:22.428263+0000","last_unstale":"2024-09-15T21:46:28.416728+0000","last_undegraded":"2024-09-15T21:46:28.416728+0000","last_fullsized":"2024-09-15T21:46:28.416728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:22:40.160062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:24.834701+0000","last_change":"2024-09-15T21:46:23.427968+0000","last_active":"2024-09-15T21:46:24.834701+0000","last_peered":"2024-09-15T21:46:24.834701+0000","last_clean":"2024-09-15T21:46:24.834701+0000","last_became_active":"2024-09-15T21:46:23.427546+0000","last_became_peered":"2024-09-15T21:46:23.427546+0000","last_unstale":"2024-09-15T21:46:24.834701+0000","last_undegraded":"2024-09-15T21:46:24.834701+0000","last_fullsized":"2024-09-15T21:46:24.834701+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:47.226097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'13","reported_seq":27,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:46:51.569304+0000","last_change":"2024-09-15T21:46:22.428258+0000","last_active":"2024-09-15T21:46:51.569304+0000","last_peered":"2024-09-15T21:46:51.569304+0000","last_clean":"2024-09-15T21:46:51.569304+0000","last_became_active":"2024-09-15T21:46:22.427954+0000","last_became_peered":"2024-09-15T21:46:22.427954+0000","last_unstale":"2024-09-15T21:46:51.569304+0000","last_undegraded":"2024-09-15T21:46:51.569304+0000","last_fullsized":"2024-09-15T21:46:51.569304+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:22.440456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":8034,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":205,"num_write_kb":62,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":16068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":179,"ondisk_log_size":179,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476748,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32008,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339832,"statfs":{"total":96636764160,"available":96603987968,"internally_reserved":0,"allocated":843776,"data_stored":668200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509452,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":669426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":79360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":80890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509453,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":245760,"data_stored":78288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509452,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":78420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542156,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":249856,"data_stored":78490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":253952,"data_stored":79394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":2436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:47:03.394 INFO:tasks.thrashosds.thrasher:Setting 2.8 to [0, 5] 2024-09-15T21:47:03.394 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.8', '0', '5'] 2024-09-15T21:47:03.394 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.8 0 5 2024-09-15T21:47:03.447 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:03.445+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:03.447 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-15T21:47:03.478 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:03.479 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:03.488 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:47:03.495 INFO:tasks.workunit.client.0.smithi100.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-15T21:47:03.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.535+0000 7fa9149b7640 1 Processor -- start 2024-09-15T21:47:03.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.535+0000 7fa9149b7640 1 -- start start 2024-09-15T21:47:03.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.536+0000 7fa9149b7640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9100cba90 0x7fa910154f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:03.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.536+0000 7fa9149b7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa91006d700 0x7fa910155450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:03.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.536+0000 7fa9149b7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa91015b3c0 con 0x7fa9100cb6b0 2024-09-15T21:47:03.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.536+0000 7fa9149b7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa910070960 con 0x7fa91006d700 2024-09-15T21:47:03.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.536+0000 7fa9149b7640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa9100c9440 con 0x7fa9100cba90 2024-09-15T21:47:03.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.536+0000 7fa90ed9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa91006d700 0x7fa910155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.536+0000 7fa90e59c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fa9100cb6b0 0x7fa910073940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:40772/0 (socket says 172.21.15.100:40772) 2024-09-15T21:47:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.536+0000 7fa90e59c640 1 -- 172.21.15.100:0/2134682861 learned_addr learned my addr 172.21.15.100:0/2134682861 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.536+0000 7fa90dd9b640 1 --2- 172.21.15.100:0/2134682861 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9100cba90 0x7fa910154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.536+0000 7fa90ed9d640 1 -- 172.21.15.100:0/2134682861 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9100cba90 msgr2=0x7fa910154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.536+0000 7fa90ed9d640 1 --2- 172.21.15.100:0/2134682861 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9100cba90 0x7fa910154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.536+0000 7fa90ed9d640 1 -- 172.21.15.100:0/2134682861 >> v1:172.21.15.100:6789/0 conn(0x7fa9100cb6b0 legacy=0x7fa910073940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.536+0000 7fa90ed9d640 1 -- 172.21.15.100:0/2134682861 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91015b680 con 0x7fa91006d700 2024-09-15T21:47:03.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.536+0000 7fa90d59a640 1 -- 172.21.15.100:0/2134682861 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3605376847 0 0) 0x7fa91015b3c0 con 0x7fa9100cb6b0 2024-09-15T21:47:03.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.537+0000 7fa90ed9d640 1 --2- 172.21.15.100:0/2134682861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa91006d700 0x7fa910155450 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa908009d90 tx=0x7fa90802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d1b6b00d1c27f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:03.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.537+0000 7fa90d59a640 1 -- 172.21.15.100:0/2134682861 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908007740 con 0x7fa91006d700 2024-09-15T21:47:03.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.537+0000 7fa90d59a640 1 -- 172.21.15.100:0/2134682861 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9080078e0 con 0x7fa91006d700 2024-09-15T21:47:03.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.537+0000 7fa90d59a640 1 -- 172.21.15.100:0/2134682861 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908007cb0 con 0x7fa91006d700 2024-09-15T21:47:03.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.538+0000 7fa9149b7640 1 -- 172.21.15.100:0/2134682861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa91006d700 msgr2=0x7fa910155450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:03.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.538+0000 7fa9149b7640 1 --2- 172.21.15.100:0/2134682861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa91006d700 0x7fa910155450 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa908009d90 tx=0x7fa90802a660 comp rx=0 tx=0).stop 2024-09-15T21:47:03.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.538+0000 7fa9149b7640 1 -- 172.21.15.100:0/2134682861 shutdown_connections 2024-09-15T21:47:03.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.538+0000 7fa9149b7640 1 --2- 172.21.15.100:0/2134682861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa91006d700 0x7fa910155450 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.538+0000 7fa9149b7640 1 --2- 172.21.15.100:0/2134682861 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9100cba90 0x7fa910154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.538+0000 7fa9149b7640 1 -- 172.21.15.100:0/2134682861 >> 172.21.15.100:0/2134682861 conn(0x7fa910067980 msgr2=0x7fa910067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:03.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.538+0000 7fa9149b7640 1 -- 172.21.15.100:0/2134682861 shutdown_connections 2024-09-15T21:47:03.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.538+0000 7fa9149b7640 1 -- 172.21.15.100:0/2134682861 wait complete. 2024-09-15T21:47:03.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.539+0000 7fa9149b7640 1 Processor -- start 2024-09-15T21:47:03.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.539+0000 7fa9149b7640 1 -- start start 2024-09-15T21:47:03.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.540+0000 7fa9149b7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa91006d700 0x7fa9101f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:03.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.540+0000 7fa9149b7640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9100cb6b0 0x7fa9101f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:03.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.540+0000 7fa9149b7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9100cba90 0x7fa9100ce9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:03.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.540+0000 7fa9149b7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa91015bf20 con 0x7fa9100cba90 2024-09-15T21:47:03.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.540+0000 7fa9149b7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa91015bc40 con 0x7fa91006d700 2024-09-15T21:47:03.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.540+0000 7fa9149b7640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa91015b0b0 con 0x7fa9100cb6b0 2024-09-15T21:47:03.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.540+0000 7fa90ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9100cba90 0x7fa9100ce9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:03.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.540+0000 7fa90ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9100cba90 0x7fa9100ce9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49038/0 (socket says 172.21.15.100:49038) 2024-09-15T21:47:03.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.540+0000 7fa90ed9d640 1 -- 172.21.15.100:0/1414035931 learned_addr learned my addr 172.21.15.100:0/1414035931 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:03.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.540+0000 7fa90dd9b640 1 --2- 172.21.15.100:0/1414035931 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9100cb6b0 0x7fa9101f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:03.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.540+0000 7fa90e59c640 1 --2- 172.21.15.100:0/1414035931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa91006d700 0x7fa9101f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:03.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.540+0000 7fa90ed9d640 1 -- 172.21.15.100:0/1414035931 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9100cb6b0 msgr2=0x7fa9101f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:03.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.540+0000 7fa90ed9d640 1 --2- 172.21.15.100:0/1414035931 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9100cb6b0 0x7fa9101f4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.540+0000 7fa90ed9d640 1 -- 172.21.15.100:0/1414035931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa91006d700 msgr2=0x7fa9101f17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:03.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.540+0000 7fa90ed9d640 1 --2- 172.21.15.100:0/1414035931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa91006d700 0x7fa9101f17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.540+0000 7fa90ed9d640 1 -- 172.21.15.100:0/1414035931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9100cf100 con 0x7fa9100cba90 2024-09-15T21:47:03.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.541+0000 7fa90ed9d640 1 --2- 172.21.15.100:0/1414035931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9100cba90 0x7fa9100ce9a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa908009d90 tx=0x7fa908004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:03.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.541+0000 7fa8feffd640 1 -- 172.21.15.100:0/1414035931 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908040070 con 0x7fa9100cba90 2024-09-15T21:47:03.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.541+0000 7fa8feffd640 1 -- 172.21.15.100:0/1414035931 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa908031c40 con 0x7fa9100cba90 2024-09-15T21:47:03.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.541+0000 7fa9149b7640 1 -- 172.21.15.100:0/1414035931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9100cf3e0 con 0x7fa9100cba90 2024-09-15T21:47:03.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.541+0000 7fa8feffd640 1 -- 172.21.15.100:0/1414035931 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa908009450 con 0x7fa9100cba90 2024-09-15T21:47:03.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.541+0000 7fa9149b7640 1 -- 172.21.15.100:0/1414035931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9101f0cf0 con 0x7fa9100cba90 2024-09-15T21:47:03.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.542+0000 7fa9149b7640 1 -- 172.21.15.100:0/1414035931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa91015a5f0 con 0x7fa9100cba90 2024-09-15T21:47:03.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.543+0000 7fa8feffd640 1 -- 172.21.15.100:0/1414035931 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa908039040 con 0x7fa9100cba90 2024-09-15T21:47:03.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.543+0000 7fa8feffd640 1 --2- 172.21.15.100:0/1414035931 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa8f006c2b0 0x7fa8f006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.543+0000 7fa8feffd640 1 -- 172.21.15.100:0/1414035931 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fa9080ac240 con 0x7fa9100cba90 2024-09-15T21:47:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.543+0000 7fa90e59c640 1 --2- 172.21.15.100:0/1414035931 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa8f006c2b0 0x7fa8f006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:03.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.544+0000 7fa90e59c640 1 --2- 172.21.15.100:0/1414035931 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa8f006c2b0 0x7fa8f006e770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa900000f60 tx=0x7fa900007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:03.548 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:03.546+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:03.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.548+0000 7fa8feffd640 1 -- 172.21.15.100:0/1414035931 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa91015a5f0 con 0x7fa9100cba90 2024-09-15T21:47:03.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.596+0000 7f36597fa640 1 -- 172.21.15.100:0/3740202194 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v23) ==== 87+0+0 (secure 0 0 0) 0x7f3650059270 con 0x7f364c0073d0 2024-09-15T21:47:03.598 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is set 2024-09-15T21:47:03.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.598+0000 7f366312a640 1 -- 172.21.15.100:0/3740202194 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f364006c300 msgr2=0x7f364006e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:03.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.598+0000 7f366312a640 1 --2- 172.21.15.100:0/3740202194 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f364006c300 0x7f364006e7c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f365c157560 tx=0x7f3648008040 comp rx=0 tx=0).stop 2024-09-15T21:47:03.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.598+0000 7f366312a640 1 -- 172.21.15.100:0/3740202194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364c0073d0 msgr2=0x7f365c14d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:03.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.598+0000 7f366312a640 1 --2- 172.21.15.100:0/3740202194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364c0073d0 0x7f365c14d4a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f365000e620 tx=0x7f365000eae0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.599+0000 7f366312a640 1 -- 172.21.15.100:0/3740202194 shutdown_connections 2024-09-15T21:47:03.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.599+0000 7f366312a640 1 --2- 172.21.15.100:0/3740202194 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f364006c300 0x7f364006e7c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.599+0000 7f366312a640 1 --2- 172.21.15.100:0/3740202194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f365c1597b0 0x7f365c0c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.599+0000 7f366312a640 1 --2- 172.21.15.100:0/3740202194 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f365c147b10 0x7f365c147f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.599+0000 7f366312a640 1 --2- 172.21.15.100:0/3740202194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f364c0073d0 0x7f365c14d4a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:03.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.599+0000 7f366312a640 1 -- 172.21.15.100:0/3740202194 >> 172.21.15.100:0/3740202194 conn(0x7f365c0ca430 msgr2=0x7f365c06cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:03.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.599+0000 7f366312a640 1 -- 172.21.15.100:0/3740202194 shutdown_connections 2024-09-15T21:47:03.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.599+0000 7f366312a640 1 -- 172.21.15.100:0/3740202194 wait complete. 2024-09-15T21:47:03.606 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:03.606 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:03.617 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:47:03.648 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:03.647+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:03.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:03.746+0000 7fa9149b7640 1 -- 172.21.15.100:0/1414035931 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pg-upmap", "pgid": "2.8", "id": [0, 5]} v 0) -- 0x7fa910073940 con 0x7fa9100cba90 2024-09-15T21:47:03.749 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:03.748+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:03.761 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:03.761 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:03.771 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:47:03.851 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:03.849+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:03.878 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:03.878 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:03.887 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:47:03.952 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:03.950+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:03.992 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:03.992 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:04.002 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:47:04.053 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:04.051+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:04.106 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:04.107 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:04.115 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:47:04.154 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:04.152+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:04.254 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:04.253+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:04.261 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:04.261 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:04.270 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:47:04.355 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:04.354+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:04.386 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:04.386 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:04.395 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:47:04.478 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:04.477+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:04.539 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:04.539 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:04.547 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:47:04.557 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:04.556+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:04.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.610+0000 7fa8feffd640 1 -- 172.21.15.100:0/1414035931 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pg-upmap", "pgid": "2.8", "id": [0, 5]}]=0 set 2.8 pg_upmap mapping to [0,5] v24) ==== 122+0+0 (secure 0 0 0) 0x7fa90807a360 con 0x7fa9100cba90 2024-09-15T21:47:04.612 INFO:teuthology.orchestra.run.smithi100.stderr:set 2.8 pg_upmap mapping to [0,5] 2024-09-15T21:47:04.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.613+0000 7fa9149b7640 1 -- 172.21.15.100:0/1414035931 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa8f006c2b0 msgr2=0x7fa8f006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:04.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.613+0000 7fa9149b7640 1 --2- 172.21.15.100:0/1414035931 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa8f006c2b0 0x7fa8f006e770 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa900000f60 tx=0x7fa900007d50 comp rx=0 tx=0).stop 2024-09-15T21:47:04.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.613+0000 7fa9149b7640 1 -- 172.21.15.100:0/1414035931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9100cba90 msgr2=0x7fa9100ce9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:04.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.613+0000 7fa9149b7640 1 --2- 172.21.15.100:0/1414035931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9100cba90 0x7fa9100ce9a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa908009d90 tx=0x7fa908004a40 comp rx=0 tx=0).stop 2024-09-15T21:47:04.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.613+0000 7fa9149b7640 1 -- 172.21.15.100:0/1414035931 shutdown_connections 2024-09-15T21:47:04.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.613+0000 7fa9149b7640 1 --2- 172.21.15.100:0/1414035931 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa8f006c2b0 0x7fa8f006e770 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:04.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.613+0000 7fa9149b7640 1 --2- 172.21.15.100:0/1414035931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9100cba90 0x7fa9100ce9a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:04.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.613+0000 7fa9149b7640 1 --2- 172.21.15.100:0/1414035931 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9100cb6b0 0x7fa9101f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:04.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.613+0000 7fa9149b7640 1 --2- 172.21.15.100:0/1414035931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa91006d700 0x7fa9101f17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:04.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.613+0000 7fa9149b7640 1 -- 172.21.15.100:0/1414035931 >> 172.21.15.100:0/1414035931 conn(0x7fa910067980 msgr2=0x7fa9101595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:04.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.613+0000 7fa9149b7640 1 -- 172.21.15.100:0/1414035931 shutdown_connections 2024-09-15T21:47:04.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.613+0000 7fa9149b7640 1 -- 172.21.15.100:0/1414035931 wait complete. 2024-09-15T21:47:04.658 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:04.657+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:04.838 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:04.838 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:04.839 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:47:04.839 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:04.837+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:04.877 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:47:04.938 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:04.937+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:04.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.979+0000 7f5f6c3bd640 1 Processor -- start 2024-09-15T21:47:04.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.979+0000 7f5f6c3bd640 1 -- start start 2024-09-15T21:47:04.981 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:04.982 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:04.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.980+0000 7f5f6c3bd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f64152280 0x7f5f641588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:04.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.980+0000 7f5f6c3bd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5f64072fa0 0x7f5f64162810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:04.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.980+0000 7f5f6c3bd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f64071580 con 0x7f5f640c3880 2024-09-15T21:47:04.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.980+0000 7f5f6c3bd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f5f64152660 con 0x7f5f64072fa0 2024-09-15T21:47:04.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.980+0000 7f5f6c3bd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5f640c4890 con 0x7f5f64152280 2024-09-15T21:47:04.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.980+0000 7f5f6a95a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5f64072fa0 0x7f5f64162810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:04.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.980+0000 7f5f69958640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f5f640c3880 0x7f5f640c8390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:40788/0 (socket says 172.21.15.100:40788) 2024-09-15T21:47:04.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.980+0000 7f5f69958640 1 -- 172.21.15.100:0/4097710477 learned_addr learned my addr 172.21.15.100:0/4097710477 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:04.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.980+0000 7f5f6a159640 1 --2- 172.21.15.100:0/4097710477 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f64152280 0x7f5f641588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:04.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.980+0000 7f5f6a95a640 1 -- 172.21.15.100:0/4097710477 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f64152280 msgr2=0x7f5f641588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:04.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.980+0000 7f5f6a95a640 1 --2- 172.21.15.100:0/4097710477 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f64152280 0x7f5f641588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:04.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.980+0000 7f5f6a95a640 1 -- 172.21.15.100:0/4097710477 >> v1:172.21.15.100:6789/0 conn(0x7f5f640c3880 legacy=0x7f5f640c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:04.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.980+0000 7f5f6a95a640 1 -- 172.21.15.100:0/4097710477 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f64071860 con 0x7f5f64072fa0 2024-09-15T21:47:04.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.980+0000 7f5f69157640 1 -- 172.21.15.100:0/4097710477 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3913899649 0 0) 0x7f5f64071580 con 0x7f5f640c3880 2024-09-15T21:47:04.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.981+0000 7f5f6a95a640 1 --2- 172.21.15.100:0/4097710477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f64072fa0 0x7f5f64162810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009d90 tx=0x7f5f5c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b3467322e2f3fe server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:04.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.981+0000 7f5f69157640 1 -- 172.21.15.100:0/4097710477 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c007740 con 0x7f5f64072fa0 2024-09-15T21:47:04.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.981+0000 7f5f69157640 1 -- 172.21.15.100:0/4097710477 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f5c0078e0 con 0x7f5f64072fa0 2024-09-15T21:47:04.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.981+0000 7f5f69157640 1 -- 172.21.15.100:0/4097710477 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c007cb0 con 0x7f5f64072fa0 2024-09-15T21:47:04.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.982+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/4097710477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f64072fa0 msgr2=0x7f5f64162810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:04.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.982+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/4097710477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f64072fa0 0x7f5f64162810 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009d90 tx=0x7f5f5c02a6f0 comp rx=0 tx=0).stop 2024-09-15T21:47:04.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.982+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/4097710477 shutdown_connections 2024-09-15T21:47:04.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.982+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/4097710477 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f64072fa0 0x7f5f64162810 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:04.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.982+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/4097710477 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f64152280 0x7f5f641588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:04.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.982+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/4097710477 >> 172.21.15.100:0/4097710477 conn(0x7f5f640c9380 msgr2=0x7f5f640c9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:04.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.982+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/4097710477 shutdown_connections 2024-09-15T21:47:04.985 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.982+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/4097710477 wait complete. 2024-09-15T21:47:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.983+0000 7f5f6c3bd640 1 Processor -- start 2024-09-15T21:47:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.983+0000 7f5f6c3bd640 1 -- start start 2024-09-15T21:47:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.983+0000 7f5f6c3bd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f64072fa0 0x7f5f64155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.983+0000 7f5f6c3bd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f640c3880 0x7f5f641f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.983+0000 7f5f6c3bd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f64152280 0x7f5f64155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.983+0000 7f5f6c3bd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5f64158380 con 0x7f5f640c3880 2024-09-15T21:47:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.983+0000 7f5f6c3bd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5f641580a0 con 0x7f5f64072fa0 2024-09-15T21:47:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.983+0000 7f5f6c3bd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5f64158de0 con 0x7f5f64152280 2024-09-15T21:47:04.988 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.984+0000 7f5f69958640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f640c3880 0x7f5f641f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.984+0000 7f5f69958640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f640c3880 0x7f5f641f26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49042/0 (socket says 172.21.15.100:49042) 2024-09-15T21:47:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.984+0000 7f5f69958640 1 -- 172.21.15.100:0/2605170416 learned_addr learned my addr 172.21.15.100:0/2605170416 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.984+0000 7f5f6a95a640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f64152280 0x7f5f64155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.984+0000 7f5f6a159640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f64072fa0 0x7f5f64155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.984+0000 7f5f69958640 1 -- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f64152280 msgr2=0x7f5f64155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.984+0000 7f5f69958640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5f64152280 0x7f5f64155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.984+0000 7f5f69958640 1 -- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f64072fa0 msgr2=0x7f5f64155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.984+0000 7f5f69958640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f64072fa0 0x7f5f64155f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.984+0000 7f5f69958640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f641f1bf0 con 0x7f5f640c3880 2024-09-15T21:47:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.984+0000 7f5f6a159640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5f64072fa0 0x7f5f64155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:47:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.984+0000 7f5f69958640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f640c3880 0x7f5f641f26c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400c500 tx=0x7f5f5400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.984+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54018070 con 0x7f5f640c3880 2024-09-15T21:47:04.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.984+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f54007580 con 0x7f5f640c3880 2024-09-15T21:47:04.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.984+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54004a20 con 0x7f5f640c3880 2024-09-15T21:47:04.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.984+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f6415b3e0 con 0x7f5f640c3880 2024-09-15T21:47:04.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.984+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f6415b9a0 con 0x7f5f640c3880 2024-09-15T21:47:04.990 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.985+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f5f64069420 con 0x7f5f640c3880 2024-09-15T21:47:04.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.986+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5f5401b070 con 0x7f5f640c3880 2024-09-15T21:47:04.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.986+0000 7f5f5affd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5f4806c2b0 0x7f5f4806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:04.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.986+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5854+0+0 (secure 0 0 0) 0x7f5f5408a6f0 con 0x7f5f640c3880 2024-09-15T21:47:04.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.986+0000 7f5f6a159640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5f4806c2b0 0x7f5f4806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:04.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.986+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v24) ==== 54+0+16 (secure 0 0 0) 0x7f5f54090170 con 0x7f5f640c3880 2024-09-15T21:47:04.997 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:47:04.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.987+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f5f641f1e20 0x7f5f640ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:04.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.987+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5f640ca1b0 con 0x7f5f641f1e20 2024-09-15T21:47:04.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.987+0000 7f5f6a159640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5f4806c2b0 0x7f5f4806e770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009af0 tx=0x7f5f60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:04.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.987+0000 7f5f6a95a640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f5f641f1e20 0x7f5f640ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:04.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.987+0000 7f5f6a95a640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f5f641f1e20 0x7f5f640ca6f0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:04.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:04.988+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5f640ca1b0 con 0x7f5f641f1e20 2024-09-15T21:47:05.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.002+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5f64150c30 con 0x7f5f641f1e20 2024-09-15T21:47:05.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.003+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5f64150c30 con 0x7f5f641f1e20 2024-09-15T21:47:05.005 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:05.006 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:05.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.003+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f5f6414a090 0x7f5f6414a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:05.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.003+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f5f64150950 con 0x7f5f6414a090 2024-09-15T21:47:05.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.003+0000 7f5f6a159640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f5f6414a090 0x7f5f6414a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:05.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.004+0000 7f5f6a159640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f5f6414a090 0x7f5f6414a470 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:05.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.004+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5f64150950 con 0x7f5f6414a090 2024-09-15T21:47:05.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.018+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5f6414a9b0 con 0x7f5f6414a090 2024-09-15T21:47:05.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.020+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5f6414a9b0 con 0x7f5f6414a090 2024-09-15T21:47:05.022 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:05.023 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:05.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.020+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f5f640c7900 0x7f5f640c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:05.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.020+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f5f6414b110 con 0x7f5f640c7900 2024-09-15T21:47:05.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.020+0000 7f5f69958640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f5f640c7900 0x7f5f640c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:05.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.020+0000 7f5f69958640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f5f640c7900 0x7f5f640c7d00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:05.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.021+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5f6414b110 con 0x7f5f640c7900 2024-09-15T21:47:05.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.034+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5f6414b620 con 0x7f5f640c7900 2024-09-15T21:47:05.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.036+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5f6414b620 con 0x7f5f640c7900 2024-09-15T21:47:05.039 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:05.039 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:05.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.036+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f5f64062630 0x7f5f64062a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:05.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.036+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f5f64062f50 con 0x7f5f64062630 2024-09-15T21:47:05.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.036+0000 7f5f6a95a640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f5f64062630 0x7f5f64062a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:05.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.037+0000 7f5f6a95a640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f5f64062630 0x7f5f64062a10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:05.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.037+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5f64062f50 con 0x7f5f64062630 2024-09-15T21:47:05.041 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:05.039+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:05.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.050+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5f64063460 con 0x7f5f64062630 2024-09-15T21:47:05.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.052+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5f64063460 con 0x7f5f64062630 2024-09-15T21:47:05.055 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:05.056 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:05.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.052+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f5f6402cb50 0x7f5f6402cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:05.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.052+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f5f640633e0 con 0x7f5f6402cb50 2024-09-15T21:47:05.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.052+0000 7f5f6a159640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f5f6402cb50 0x7f5f6402cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:05.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.053+0000 7f5f6a159640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f5f6402cb50 0x7f5f6402cf70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:05.056 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.053+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5f640633e0 con 0x7f5f6402cb50 2024-09-15T21:47:05.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.066+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5f6402d4b0 con 0x7f5f6402cb50 2024-09-15T21:47:05.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.068+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5f6402d4b0 con 0x7f5f6402cb50 2024-09-15T21:47:05.070 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:05.071 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:05.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.068+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f5f64203000 0x7f5f642053f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:05.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.068+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f5f6402d9c0 con 0x7f5f64203000 2024-09-15T21:47:05.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.069+0000 7f5f69958640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f5f64203000 0x7f5f642053f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:05.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.069+0000 7f5f69958640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f5f64203000 0x7f5f642053f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:05.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.070+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== osd.5 v2:172.21.15.122:6824/2400668532 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5f6402d9c0 con 0x7f5f64203000 2024-09-15T21:47:05.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.083+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5f6402d9c0 con 0x7f5f64203000 2024-09-15T21:47:05.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.085+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== osd.5 v2:172.21.15.122:6824/2400668532 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5f6402d9c0 con 0x7f5f64203000 2024-09-15T21:47:05.087 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-15T21:47:05.088 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:47:05.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.085+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f5f64205e40 0x7f5f64208230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:05.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.085+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f5f6420b000 con 0x7f5f64205e40 2024-09-15T21:47:05.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.086+0000 7f5f6a95a640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f5f64205e40 0x7f5f64208230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:05.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.086+0000 7f5f6a95a640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f5f64205e40 0x7f5f64208230 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:05.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.087+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5f6420b000 con 0x7f5f64205e40 2024-09-15T21:47:05.100 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:05.100 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:05.101 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:05.101 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:05.101 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:05.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.100+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5f642087f0 con 0x7f5f64205e40 2024-09-15T21:47:05.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.103+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5f642087f0 con 0x7f5f64205e40 2024-09-15T21:47:05.105 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:05.106 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:05.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.103+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f5f64208cd0 0x7f5f6420b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:05.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.103+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f5f6420e1c0 con 0x7f5f64208cd0 2024-09-15T21:47:05.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.103+0000 7f5f6a159640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f5f64208cd0 0x7f5f6420b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:05.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.103+0000 7f5f6a159640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f5f64208cd0 0x7f5f6420b120 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:05.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.104+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5f6420e1c0 con 0x7f5f64208cd0 2024-09-15T21:47:05.110 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:47:05.119 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.117+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5f6420cee0 con 0x7f5f64208cd0 2024-09-15T21:47:05.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f5affd640 1 -- 172.21.15.100:0/2605170416 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5f6420cee0 con 0x7f5f64208cd0 2024-09-15T21:47:05.121 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:05.123 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:05.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f5f641f1e20 msgr2=0x7f5f640ca6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:05.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f5f641f1e20 0x7f5f640ca6f0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:05.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f5f6414a090 msgr2=0x7f5f6414a470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:05.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f5f6414a090 0x7f5f6414a470 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:05.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f5f640c7900 msgr2=0x7f5f640c7d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:05.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f5f640c7900 0x7f5f640c7d00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:05.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f5f64062630 msgr2=0x7f5f64062a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:05.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f5f64062630 0x7f5f64062a10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:05.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f5f6402cb50 msgr2=0x7f5f6402cf70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:05.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f5f6402cb50 0x7f5f6402cf70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:05.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f5f64203000 msgr2=0x7f5f642053f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:05.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f5f64203000 0x7f5f642053f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:05.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6a95a640 1 -- 172.21.15.100:0/2605170416 reap_dead start 2024-09-15T21:47:05.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6a95a640 1 -- 172.21.15.100:0/2605170416 reap_dead start 2024-09-15T21:47:05.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6a95a640 1 -- 172.21.15.100:0/2605170416 reap_dead start 2024-09-15T21:47:05.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f5f64205e40 msgr2=0x7f5f64208230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:05.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f5f64205e40 0x7f5f64208230 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:05.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f5f64208cd0 msgr2=0x7f5f6420b120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:05.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f5f64208cd0 0x7f5f6420b120 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:05.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5f4806c2b0 msgr2=0x7f5f4806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:05.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5f4806c2b0 0x7f5f4806e770 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5f60009af0 tx=0x7f5f60008040 comp rx=0 tx=0).stop 2024-09-15T21:47:05.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f640c3880 msgr2=0x7f5f641f26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:05.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.119+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f640c3880 0x7f5f641f26c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400c500 tx=0x7f5f5400c9c0 comp rx=0 tx=0).stop 2024-09-15T21:47:05.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.120+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 shutdown_connections 2024-09-15T21:47:05.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.120+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f5f64205e40 0x7f5f64208230 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:05.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.120+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f5f64208cd0 0x7f5f6420b120 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:05.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.120+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5f4806c2b0 0x7f5f4806e770 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:05.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.120+0000 7f5f6c3bd640 1 --2- 172.21.15.100:0/2605170416 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f640c3880 0x7f5f641f26c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:05.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.120+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 >> 172.21.15.100:0/2605170416 conn(0x7f5f640c9380 msgr2=0x7f5f64151150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:05.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.120+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 shutdown_connections 2024-09-15T21:47:05.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.120+0000 7f5f6c3bd640 1 -- 172.21.15.100:0/2605170416 wait complete. 2024-09-15T21:47:05.173 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:05.171+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:05.215 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:05.215 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:05.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:05.215 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:05.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.215 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:12 2.12 2:48cd7578:::60e.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.215 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455402+0000", 2024-09-15T21:47:05.215 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.756686985000002, 2024-09-15T21:47:05.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.008447151, 2024-09-15T21:47:05.216 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.216 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.216 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.216 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455402+0000", 2024-09-15T21:47:05.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455402+0000", 2024-09-15T21:47:05.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455403+0000", 2024-09-15T21:47:05.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2800000000000002e-07 2024-09-15T21:47:05.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455412+0000", 2024-09-15T21:47:05.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5899999999999997e-06 2024-09-15T21:47:05.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456441+0000", 2024-09-15T21:47:05.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0010282609999999999 2024-09-15T21:47:05.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456448+0000", 2024-09-15T21:47:05.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8849999999999999e-06 2024-09-15T21:47:05.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456529+0000", 2024-09-15T21:47:05.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0657999999999999e-05 2024-09-15T21:47:05.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456615+0000", 2024-09-15T21:47:05.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5377999999999995e-05 2024-09-15T21:47:05.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:05.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456737+0000", 2024-09-15T21:47:05.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012286799999999999 2024-09-15T21:47:05.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458057+0000", 2024-09-15T21:47:05.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013198039999999999 2024-09-15T21:47:05.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463754+0000", 2024-09-15T21:47:05.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005697237 2024-09-15T21:47:05.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463809+0000", 2024-09-15T21:47:05.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5040000000000002e-05 2024-09-15T21:47:05.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463849+0000", 2024-09-15T21:47:05.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0002000000000001e-05 2024-09-15T21:47:05.221 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.221 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.221 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.221 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:13 2.16 2:6f2c300e:::60f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.221 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455414+0000", 2024-09-15T21:47:05.221 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.756675334000001, 2024-09-15T21:47:05.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090734720000000008, 2024-09-15T21:47:05.221 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.222 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.222 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.222 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455414+0000", 2024-09-15T21:47:05.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455414+0000", 2024-09-15T21:47:05.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455414+0000", 2024-09-15T21:47:05.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9700000000000002e-07 2024-09-15T21:47:05.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455417+0000", 2024-09-15T21:47:05.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.813e-06 2024-09-15T21:47:05.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458176+0000", 2024-09-15T21:47:05.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027589649999999999 2024-09-15T21:47:05.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458185+0000", 2024-09-15T21:47:05.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5120000000000008e-06 2024-09-15T21:47:05.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458851+0000", 2024-09-15T21:47:05.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00066587500000000002 2024-09-15T21:47:05.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459142+0000", 2024-09-15T21:47:05.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000291966 2024-09-15T21:47:05.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:05.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459273+0000", 2024-09-15T21:47:05.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013040600000000001 2024-09-15T21:47:05.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463485+0000", 2024-09-15T21:47:05.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004212304 2024-09-15T21:47:05.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464418+0000", 2024-09-15T21:47:05.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00093258000000000002 2024-09-15T21:47:05.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464451+0000", 2024-09-15T21:47:05.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3246e-05 2024-09-15T21:47:05.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464487+0000", 2024-09-15T21:47:05.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6408000000000001e-05 2024-09-15T21:47:05.229 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.229 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.229 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.229 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:14 2.16 2:6f2c300e:::60f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.229 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455418+0000", 2024-09-15T21:47:05.229 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.756670976999999, 2024-09-15T21:47:05.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0092032969999999992, 2024-09-15T21:47:05.229 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.229 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.229 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.230 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455418+0000", 2024-09-15T21:47:05.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455418+0000", 2024-09-15T21:47:05.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455419+0000", 2024-09-15T21:47:05.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9900000000000002e-07 2024-09-15T21:47:05.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455421+0000", 2024-09-15T21:47:05.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4540000000000001e-06 2024-09-15T21:47:05.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458278+0000", 2024-09-15T21:47:05.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028568370000000001 2024-09-15T21:47:05.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458290+0000", 2024-09-15T21:47:05.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1715e-05 2024-09-15T21:47:05.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459558+0000", 2024-09-15T21:47:05.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001268714 2024-09-15T21:47:05.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459637+0000", 2024-09-15T21:47:05.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8400999999999997e-05 2024-09-15T21:47:05.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:05.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459792+0000", 2024-09-15T21:47:05.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000155072 2024-09-15T21:47:05.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463490+0000", 2024-09-15T21:47:05.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003698355 2024-09-15T21:47:05.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464534+0000", 2024-09-15T21:47:05.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0010436740000000001 2024-09-15T21:47:05.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464594+0000", 2024-09-15T21:47:05.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9759000000000002e-05 2024-09-15T21:47:05.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464622+0000", 2024-09-15T21:47:05.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8017e-05 2024-09-15T21:47:05.235 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.235 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.235 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.235 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:24 2.3 2:c112adae:::101.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.235 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455422+0000", 2024-09-15T21:47:05.235 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.756666975999998, 2024-09-15T21:47:05.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0079742749999999994, 2024-09-15T21:47:05.235 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.235 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.235 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.236 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455422+0000", 2024-09-15T21:47:05.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455422+0000", 2024-09-15T21:47:05.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455423+0000", 2024-09-15T21:47:05.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:47:05.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455433+0000", 2024-09-15T21:47:05.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9860000000000006e-06 2024-09-15T21:47:05.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458310+0000", 2024-09-15T21:47:05.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028777540000000002 2024-09-15T21:47:05.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458316+0000", 2024-09-15T21:47:05.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3720000000000004e-06 2024-09-15T21:47:05.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458653+0000", 2024-09-15T21:47:05.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033772899999999999 2024-09-15T21:47:05.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458788+0000", 2024-09-15T21:47:05.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134282 2024-09-15T21:47:05.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-15T21:47:05.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458969+0000", 2024-09-15T21:47:05.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000180865 2024-09-15T21:47:05.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462175+0000", 2024-09-15T21:47:05.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032066249999999998 2024-09-15T21:47:05.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463301+0000", 2024-09-15T21:47:05.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001125338 2024-09-15T21:47:05.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463352+0000", 2024-09-15T21:47:05.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0967999999999997e-05 2024-09-15T21:47:05.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463397+0000", 2024-09-15T21:47:05.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5049999999999997e-05 2024-09-15T21:47:05.240 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.241 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.241 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.241 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:26 2.3 2:c27f454c:::mds1_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.241 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455434+0000", 2024-09-15T21:47:05.241 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.756655297999998, 2024-09-15T21:47:05.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0080572170000000002, 2024-09-15T21:47:05.241 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.241 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.241 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.241 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455434+0000", 2024-09-15T21:47:05.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455434+0000", 2024-09-15T21:47:05.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455434+0000", 2024-09-15T21:47:05.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3000000000000002e-07 2024-09-15T21:47:05.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455439+0000", 2024-09-15T21:47:05.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.464e-06 2024-09-15T21:47:05.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458340+0000", 2024-09-15T21:47:05.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029012740000000001 2024-09-15T21:47:05.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458346+0000", 2024-09-15T21:47:05.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9370000000000001e-06 2024-09-15T21:47:05.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459336+0000", 2024-09-15T21:47:05.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0009897930000000001 2024-09-15T21:47:05.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459454+0000", 2024-09-15T21:47:05.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011808500000000001 2024-09-15T21:47:05.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-15T21:47:05.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459603+0000", 2024-09-15T21:47:05.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014933000000000001 2024-09-15T21:47:05.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462232+0000", 2024-09-15T21:47:05.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026290329999999998 2024-09-15T21:47:05.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463406+0000", 2024-09-15T21:47:05.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001173299 2024-09-15T21:47:05.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463450+0000", 2024-09-15T21:47:05.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4530999999999998e-05 2024-09-15T21:47:05.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463491+0000", 2024-09-15T21:47:05.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1140999999999998e-05 2024-09-15T21:47:05.246 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.246 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.246 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.246 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:30 2.12 2:4bb29eec:::201.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.246 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455440+0000", 2024-09-15T21:47:05.246 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.756649537000001, 2024-09-15T21:47:05.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086051700000000005, 2024-09-15T21:47:05.247 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.247 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.247 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:05.247 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455440+0000", 2024-09-15T21:47:05.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455440+0000", 2024-09-15T21:47:05.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455440+0000", 2024-09-15T21:47:05.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9099999999999999e-07 2024-09-15T21:47:05.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455442+0000", 2024-09-15T21:47:05.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0999999999999998e-06 2024-09-15T21:47:05.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458359+0000", 2024-09-15T21:47:05.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029165939999999998 2024-09-15T21:47:05.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458369+0000", 2024-09-15T21:47:05.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0261999999999999e-05 2024-09-15T21:47:05.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458466+0000", 2024-09-15T21:47:05.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6898999999999995e-05 2024-09-15T21:47:05.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458576+0000", 2024-09-15T21:47:05.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000109514 2024-09-15T21:47:05.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464045+0000", 2024-09-15T21:47:05.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054694100000000001 2024-09-15T21:47:05.250 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.250 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.250 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.250 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:32 2.18 2:1d00e16c:::201.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.250 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455443+0000", 2024-09-15T21:47:05.251 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.756645879000001, 2024-09-15T21:47:05.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0079644680000000006, 2024-09-15T21:47:05.251 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.251 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.251 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:05.251 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455443+0000", 2024-09-15T21:47:05.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455443+0000", 2024-09-15T21:47:05.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455444+0000", 2024-09-15T21:47:05.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-15T21:47:05.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455445+0000", 2024-09-15T21:47:05.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.39e-06 2024-09-15T21:47:05.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459133+0000", 2024-09-15T21:47:05.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003688322 2024-09-15T21:47:05.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459143+0000", 2024-09-15T21:47:05.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1300000000000007e-06 2024-09-15T21:47:05.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459217+0000", 2024-09-15T21:47:05.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4046000000000002e-05 2024-09-15T21:47:05.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459319+0000", 2024-09-15T21:47:05.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000102866 2024-09-15T21:47:05.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463408+0000", 2024-09-15T21:47:05.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040884420000000003 2024-09-15T21:47:05.254 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.254 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.254 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.254 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.1.10:9 2.17 e22/19 2:e8b9846e:::60d.00000000:head v 22'1, mlcod=0'0)", 2024-09-15T21:47:05.254 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.459214+0000", 2024-09-15T21:47:05.255 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.752875063000001, 2024-09-15T21:47:05.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0042427130000000004, 2024-09-15T21:47:05.255 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.255 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.255 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.255 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459214+0000", 2024-09-15T21:47:05.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459214+0000", 2024-09-15T21:47:05.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459215+0000", 2024-09-15T21:47:05.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4099999999999997e-07 2024-09-15T21:47:05.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459223+0000", 2024-09-15T21:47:05.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1540000000000002e-06 2024-09-15T21:47:05.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459233+0000", 2024-09-15T21:47:05.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0006e-05 2024-09-15T21:47:05.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459240+0000", 2024-09-15T21:47:05.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6950000000000003e-06 2024-09-15T21:47:05.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459305+0000", 2024-09-15T21:47:05.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4907999999999996e-05 2024-09-15T21:47:05.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459324+0000", 2024-09-15T21:47:05.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9066e-05 2024-09-15T21:47:05.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463355+0000", 2024-09-15T21:47:05.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040313969999999999 2024-09-15T21:47:05.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463457+0000", 2024-09-15T21:47:05.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010154600000000001 2024-09-15T21:47:05.259 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.259 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.259 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.259 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:24 2.3 2:c216c0d1:::mds2_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.259 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.476395+0000", 2024-09-15T21:47:05.259 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 24.735694093999999, 2024-09-15T21:47:05.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004716858, 2024-09-15T21:47:05.259 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.259 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.259 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.259 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476395+0000", 2024-09-15T21:47:05.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476395+0000", 2024-09-15T21:47:05.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476397+0000", 2024-09-15T21:47:05.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1619999999999998e-06 2024-09-15T21:47:05.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476417+0000", 2024-09-15T21:47:05.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9449999999999998e-05 2024-09-15T21:47:05.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476447+0000", 2024-09-15T21:47:05.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0139999999999999e-05 2024-09-15T21:47:05.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476457+0000", 2024-09-15T21:47:05.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7850000000000003e-06 2024-09-15T21:47:05.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476543+0000", 2024-09-15T21:47:05.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6544000000000004e-05 2024-09-15T21:47:05.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476692+0000", 2024-09-15T21:47:05.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000148722 2024-09-15T21:47:05.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-15T21:47:05.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476803+0000", 2024-09-15T21:47:05.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000111189 2024-09-15T21:47:05.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480059+0000", 2024-09-15T21:47:05.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032562170000000001 2024-09-15T21:47:05.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.481049+0000", 2024-09-15T21:47:05.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00098924400000000002 2024-09-15T21:47:05.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.481083+0000", 2024-09-15T21:47:05.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4005999999999999e-05 2024-09-15T21:47:05.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.481112+0000", 2024-09-15T21:47:05.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9399000000000001e-05 2024-09-15T21:47:05.264 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.265 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.265 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.265 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:10 2.3 2:c04be1b2:::621.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.265 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544911+0000", 2024-09-15T21:47:05.265 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.667177805000001, 2024-09-15T21:47:05.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056076110000000002, 2024-09-15T21:47:05.265 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.265 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.266 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.266 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544911+0000", 2024-09-15T21:47:05.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544911+0000", 2024-09-15T21:47:05.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544912+0000", 2024-09-15T21:47:05.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3799999999999997e-07 2024-09-15T21:47:05.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544915+0000", 2024-09-15T21:47:05.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1920000000000001e-06 2024-09-15T21:47:05.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545131+0000", 2024-09-15T21:47:05.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021561700000000001 2024-09-15T21:47:05.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545138+0000", 2024-09-15T21:47:05.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.379e-06 2024-09-15T21:47:05.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545659+0000", 2024-09-15T21:47:05.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00052105299999999995 2024-09-15T21:47:05.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545744+0000", 2024-09-15T21:47:05.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4333999999999996e-05 2024-09-15T21:47:05.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-15T21:47:05.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545899+0000", 2024-09-15T21:47:05.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000155103 2024-09-15T21:47:05.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547920+0000", 2024-09-15T21:47:05.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002021508 2024-09-15T21:47:05.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550414+0000", 2024-09-15T21:47:05.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002493557 2024-09-15T21:47:05.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550467+0000", 2024-09-15T21:47:05.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2966000000000001e-05 2024-09-15T21:47:05.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550519+0000", 2024-09-15T21:47:05.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2364000000000002e-05 2024-09-15T21:47:05.270 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.270 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.271 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.271 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:23 2.18 2:1c1db442:::103.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.271 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544917+0000", 2024-09-15T21:47:05.271 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.667172423, 2024-09-15T21:47:05.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0062879529999999998, 2024-09-15T21:47:05.271 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.271 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.271 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.271 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544917+0000", 2024-09-15T21:47:05.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.272 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.272 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544917+0000", 2024-09-15T21:47:05.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.272 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.272 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544917+0000", 2024-09-15T21:47:05.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5600000000000002e-07 2024-09-15T21:47:05.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544921+0000", 2024-09-15T21:47:05.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1620000000000001e-06 2024-09-15T21:47:05.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545157+0000", 2024-09-15T21:47:05.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00023608099999999999 2024-09-15T21:47:05.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545162+0000", 2024-09-15T21:47:05.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5619999999999997e-06 2024-09-15T21:47:05.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545221+0000", 2024-09-15T21:47:05.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9138000000000002e-05 2024-09-15T21:47:05.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545485+0000", 2024-09-15T21:47:05.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00026413499999999998 2024-09-15T21:47:05.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:05.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545621+0000", 2024-09-15T21:47:05.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013621099999999999 2024-09-15T21:47:05.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548523+0000", 2024-09-15T21:47:05.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029016739999999999 2024-09-15T21:47:05.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.551104+0000", 2024-09-15T21:47:05.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002580962 2024-09-15T21:47:05.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.551158+0000", 2024-09-15T21:47:05.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4052999999999999e-05 2024-09-15T21:47:05.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.551205+0000", 2024-09-15T21:47:05.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6718999999999998e-05 2024-09-15T21:47:05.276 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.276 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.276 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.276 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:27 2.12 2:4960cc90:::503.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.276 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544923+0000", 2024-09-15T21:47:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 19.667166572999999, 2024-09-15T21:47:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0057630820000000001, 2024-09-15T21:47:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544923+0000", 2024-09-15T21:47:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544923+0000", 2024-09-15T21:47:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544923+0000", 2024-09-15T21:47:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0899999999999997e-07 2024-09-15T21:47:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544926+0000", 2024-09-15T21:47:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5459999999999998e-06 2024-09-15T21:47:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545184+0000", 2024-09-15T21:47:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025825700000000002 2024-09-15T21:47:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545191+0000", 2024-09-15T21:47:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5070000000000003e-06 2024-09-15T21:47:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545296+0000", 2024-09-15T21:47:05.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000105092 2024-09-15T21:47:05.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545404+0000", 2024-09-15T21:47:05.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010791299999999999 2024-09-15T21:47:05.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:05.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545523+0000", 2024-09-15T21:47:05.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0001182 2024-09-15T21:47:05.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547713+0000", 2024-09-15T21:47:05.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021907989999999998 2024-09-15T21:47:05.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550575+0000", 2024-09-15T21:47:05.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028621480000000001 2024-09-15T21:47:05.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550644+0000", 2024-09-15T21:47:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8462999999999998e-05 2024-09-15T21:47:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550686+0000", 2024-09-15T21:47:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1848000000000003e-05 2024-09-15T21:47:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.284 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:7 2.16 2:6b7c230a:::62a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.284 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565187+0000", 2024-09-15T21:47:05.284 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.646902187, 2024-09-15T21:47:05.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059014230000000003, 2024-09-15T21:47:05.284 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.284 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.284 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.284 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565187+0000", 2024-09-15T21:47:05.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565187+0000", 2024-09-15T21:47:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565189+0000", 2024-09-15T21:47:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3099999999999999e-06 2024-09-15T21:47:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565197+0000", 2024-09-15T21:47:05.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0839999999999992e-06 2024-09-15T21:47:05.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565262+0000", 2024-09-15T21:47:05.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4169000000000002e-05 2024-09-15T21:47:05.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565272+0000", 2024-09-15T21:47:05.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.004e-05 2024-09-15T21:47:05.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565377+0000", 2024-09-15T21:47:05.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010527900000000001 2024-09-15T21:47:05.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565534+0000", 2024-09-15T21:47:05.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015667599999999999 2024-09-15T21:47:05.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:05.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565665+0000", 2024-09-15T21:47:05.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013145999999999999 2024-09-15T21:47:05.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568520+0000", 2024-09-15T21:47:05.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028552 2024-09-15T21:47:05.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570992+0000", 2024-09-15T21:47:05.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002471563 2024-09-15T21:47:05.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571046+0000", 2024-09-15T21:47:05.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4540000000000003e-05 2024-09-15T21:47:05.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571089+0000", 2024-09-15T21:47:05.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2101999999999998e-05 2024-09-15T21:47:05.289 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.289 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.289 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.289 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:8 2.16 2:6b7c230a:::62a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.290 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565201+0000", 2024-09-15T21:47:05.290 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.646888411000001, 2024-09-15T21:47:05.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061535160000000004, 2024-09-15T21:47:05.290 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.290 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.290 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.290 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565201+0000", 2024-09-15T21:47:05.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565201+0000", 2024-09-15T21:47:05.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565201+0000", 2024-09-15T21:47:05.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3500000000000002e-07 2024-09-15T21:47:05.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565204+0000", 2024-09-15T21:47:05.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7070000000000001e-06 2024-09-15T21:47:05.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565980+0000", 2024-09-15T21:47:05.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00077637599999999995 2024-09-15T21:47:05.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565990+0000", 2024-09-15T21:47:05.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1069999999999994e-06 2024-09-15T21:47:05.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566049+0000", 2024-09-15T21:47:05.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9246000000000002e-05 2024-09-15T21:47:05.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566133+0000", 2024-09-15T21:47:05.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4507999999999998e-05 2024-09-15T21:47:05.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:05.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566272+0000", 2024-09-15T21:47:05.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013888600000000001 2024-09-15T21:47:05.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569473+0000", 2024-09-15T21:47:05.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032005760000000001 2024-09-15T21:47:05.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571247+0000", 2024-09-15T21:47:05.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017738280000000001 2024-09-15T21:47:05.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571315+0000", 2024-09-15T21:47:05.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8077000000000006e-05 2024-09-15T21:47:05.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571354+0000", 2024-09-15T21:47:05.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.977e-05 2024-09-15T21:47:05.295 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.295 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.295 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.295 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:11 2.2 2:447438e6:::62c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.295 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565205+0000", 2024-09-15T21:47:05.295 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.646883880000001, 2024-09-15T21:47:05.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0067688100000000001, 2024-09-15T21:47:05.296 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.296 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.296 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.296 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565205+0000", 2024-09-15T21:47:05.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565205+0000", 2024-09-15T21:47:05.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565206+0000", 2024-09-15T21:47:05.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4400000000000001e-07 2024-09-15T21:47:05.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565208+0000", 2024-09-15T21:47:05.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.588e-06 2024-09-15T21:47:05.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566022+0000", 2024-09-15T21:47:05.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00081349700000000005 2024-09-15T21:47:05.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566028+0000", 2024-09-15T21:47:05.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1149999999999999e-06 2024-09-15T21:47:05.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566119+0000", 2024-09-15T21:47:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1116e-05 2024-09-15T21:47:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566260+0000", 2024-09-15T21:47:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014067699999999999 2024-09-15T21:47:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566384+0000", 2024-09-15T21:47:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012431399999999999 2024-09-15T21:47:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569521+0000", 2024-09-15T21:47:05.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0031368189999999999 2024-09-15T21:47:05.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571922+0000", 2024-09-15T21:47:05.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024014449999999999 2024-09-15T21:47:05.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571952+0000", 2024-09-15T21:47:05.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9719000000000001e-05 2024-09-15T21:47:05.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571974+0000", 2024-09-15T21:47:05.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2175999999999999e-05 2024-09-15T21:47:05.302 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.303 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.303 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.303 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:12 2.2 2:447438e6:::62c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.303 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565210+0000", 2024-09-15T21:47:05.303 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.646879466, 2024-09-15T21:47:05.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069441429999999998, 2024-09-15T21:47:05.303 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.303 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.303 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.303 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565210+0000", 2024-09-15T21:47:05.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565210+0000", 2024-09-15T21:47:05.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565210+0000", 2024-09-15T21:47:05.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8900000000000001e-07 2024-09-15T21:47:05.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565214+0000", 2024-09-15T21:47:05.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8399999999999997e-06 2024-09-15T21:47:05.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566054+0000", 2024-09-15T21:47:05.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00083961099999999998 2024-09-15T21:47:05.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566060+0000", 2024-09-15T21:47:05.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9769999999999997e-06 2024-09-15T21:47:05.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566839+0000", 2024-09-15T21:47:05.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00077965100000000002 2024-09-15T21:47:05.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566929+0000", 2024-09-15T21:47:05.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9378999999999999e-05 2024-09-15T21:47:05.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:05.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567087+0000", 2024-09-15T21:47:05.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015816699999999999 2024-09-15T21:47:05.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569526+0000", 2024-09-15T21:47:05.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024392770000000001 2024-09-15T21:47:05.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572090+0000", 2024-09-15T21:47:05.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025637770000000002 2024-09-15T21:47:05.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572128+0000", 2024-09-15T21:47:05.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8571000000000001e-05 2024-09-15T21:47:05.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572154+0000", 2024-09-15T21:47:05.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5604000000000001e-05 2024-09-15T21:47:05.308 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.308 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.308 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.308 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:24 2.3 2:c4549ada:::mds4_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.308 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565215+0000", 2024-09-15T21:47:05.309 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.646874218000001, 2024-09-15T21:47:05.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0064577590000000004, 2024-09-15T21:47:05.309 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.309 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.309 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.309 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565215+0000", 2024-09-15T21:47:05.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565215+0000", 2024-09-15T21:47:05.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.310 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565215+0000", 2024-09-15T21:47:05.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9499999999999998e-07 2024-09-15T21:47:05.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.310 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565217+0000", 2024-09-15T21:47:05.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0959999999999999e-06 2024-09-15T21:47:05.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566074+0000", 2024-09-15T21:47:05.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00085624900000000001 2024-09-15T21:47:05.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566079+0000", 2024-09-15T21:47:05.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9470000000000002e-06 2024-09-15T21:47:05.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566145+0000", 2024-09-15T21:47:05.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5843e-05 2024-09-15T21:47:05.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566299+0000", 2024-09-15T21:47:05.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000154673 2024-09-15T21:47:05.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 2", 2024-09-15T21:47:05.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566438+0000", 2024-09-15T21:47:05.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013868999999999999 2024-09-15T21:47:05.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569805+0000", 2024-09-15T21:47:05.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033668830000000002 2024-09-15T21:47:05.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571565+0000", 2024-09-15T21:47:05.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017602760000000001 2024-09-15T21:47:05.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571632+0000", 2024-09-15T21:47:05.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6740000000000001e-05 2024-09-15T21:47:05.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.314 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.314 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571673+0000", 2024-09-15T21:47:05.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1066999999999998e-05 2024-09-15T21:47:05.314 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.314 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.315 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.315 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:26 2.12 2:4dad84a4:::mds4_sessionmap:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.315 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565222+0000", 2024-09-15T21:47:05.315 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.646866922999999, 2024-09-15T21:47:05.315 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066780520000000003, 2024-09-15T21:47:05.315 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.315 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.315 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.315 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565222+0000", 2024-09-15T21:47:05.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.316 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565222+0000", 2024-09-15T21:47:05.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.316 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565223+0000", 2024-09-15T21:47:05.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2e-07 2024-09-15T21:47:05.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565226+0000", 2024-09-15T21:47:05.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.58e-06 2024-09-15T21:47:05.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566100+0000", 2024-09-15T21:47:05.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00087394100000000004 2024-09-15T21:47:05.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566105+0000", 2024-09-15T21:47:05.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7450000000000003e-06 2024-09-15T21:47:05.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566793+0000", 2024-09-15T21:47:05.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068750500000000004 2024-09-15T21:47:05.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568056+0000", 2024-09-15T21:47:05.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001263059 2024-09-15T21:47:05.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:05.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568169+0000", 2024-09-15T21:47:05.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011367899999999999 2024-09-15T21:47:05.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569814+0000", 2024-09-15T21:47:05.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001644232 2024-09-15T21:47:05.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571801+0000", 2024-09-15T21:47:05.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019878909999999999 2024-09-15T21:47:05.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571851+0000", 2024-09-15T21:47:05.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0006999999999997e-05 2024-09-15T21:47:05.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571900+0000", 2024-09-15T21:47:05.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8992999999999997e-05 2024-09-15T21:47:05.320 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.320 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.320 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.320 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:28 2.2 2:4182a243:::204.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.321 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565228+0000", 2024-09-15T21:47:05.321 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.646861783, 2024-09-15T21:47:05.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070650210000000003, 2024-09-15T21:47:05.321 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.321 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.321 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:05.321 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.321 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.321 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565228+0000", 2024-09-15T21:47:05.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565228+0000", 2024-09-15T21:47:05.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565228+0000", 2024-09-15T21:47:05.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4400000000000001e-07 2024-09-15T21:47:05.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565229+0000", 2024-09-15T21:47:05.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5889999999999999e-06 2024-09-15T21:47:05.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566119+0000", 2024-09-15T21:47:05.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00088996699999999997 2024-09-15T21:47:05.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566125+0000", 2024-09-15T21:47:05.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1460000000000004e-06 2024-09-15T21:47:05.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567330+0000", 2024-09-15T21:47:05.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001205081 2024-09-15T21:47:05.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567435+0000", 2024-09-15T21:47:05.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000105371 2024-09-15T21:47:05.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572293+0000", 2024-09-15T21:47:05.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048575229999999999 2024-09-15T21:47:05.325 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.325 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.325 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.325 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:30 2.18 2:1f0916d5:::204.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.325 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565236+0000", 2024-09-15T21:47:05.325 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 13.646853367, 2024-09-15T21:47:05.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043773299999999996, 2024-09-15T21:47:05.325 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.325 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.325 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:05.326 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565236+0000", 2024-09-15T21:47:05.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565236+0000", 2024-09-15T21:47:05.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565236+0000", 2024-09-15T21:47:05.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.15e-07 2024-09-15T21:47:05.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565238+0000", 2024-09-15T21:47:05.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7090000000000001e-06 2024-09-15T21:47:05.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.327 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.327 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566136+0000", 2024-09-15T21:47:05.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00089817100000000002 2024-09-15T21:47:05.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566143+0000", 2024-09-15T21:47:05.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1119999999999998e-06 2024-09-15T21:47:05.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566195+0000", 2024-09-15T21:47:05.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2089000000000002e-05 2024-09-15T21:47:05.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566756+0000", 2024-09-15T21:47:05.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00056101600000000001 2024-09-15T21:47:05.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569613+0000", 2024-09-15T21:47:05.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002856918 2024-09-15T21:47:05.329 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.329 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.329 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.329 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.329 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.329 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:05.331 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:47:05.331 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:05.329+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:05.432 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:47:05.430+0000 7f74313af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37847) UID: 0 2024-09-15T21:47:05.434 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:05.435 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:05.435 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:05.435 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:05.443 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:47:05.533 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:05.532+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:05.587 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:05.587 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:05.587 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:05.587 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:05.587 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:05.595 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:47:05.616 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:47:05.659 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:05.658+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:05.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.715+0000 7f9dfc21e640 1 Processor -- start 2024-09-15T21:47:05.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.715+0000 7f9dfc21e640 1 -- start start 2024-09-15T21:47:05.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.716+0000 7f9dfc21e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9df40cca30 0x7f9df4071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:05.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.716+0000 7f9dfc21e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9df40cce10 0x7f9df415af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:05.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.716+0000 7f9dfc21e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9df40c91f0 con 0x7f9df4070ee0 2024-09-15T21:47:05.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.716+0000 7f9dfc21e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f9df40c94d0 con 0x7f9df40cce10 2024-09-15T21:47:05.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.716+0000 7f9dfc21e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f9df40c97b0 con 0x7f9df40cca30 2024-09-15T21:47:05.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.716+0000 7f9dfa7bb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9df40cce10 0x7f9df415af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:05.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.716+0000 7f9dfa7bb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9df40cce10 0x7f9df415af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49046/0 (socket says 172.21.15.100:49046) 2024-09-15T21:47:05.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.716+0000 7f9dfa7bb640 1 -- 172.21.15.100:0/3994680568 learned_addr learned my addr 172.21.15.100:0/3994680568 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:05.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.716+0000 7f9df9fba640 1 --2- 172.21.15.100:0/3994680568 >> v2:172.21.15.122:3300/0 conn(0x7f9df40cca30 0x7f9df4071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:05.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.716+0000 7f9dfa7bb640 1 -- 172.21.15.100:0/3994680568 >> v2:172.21.15.122:3300/0 conn(0x7f9df40cca30 msgr2=0x7f9df4071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:05.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.716+0000 7f9dfa7bb640 1 --2- 172.21.15.100:0/3994680568 >> v2:172.21.15.122:3300/0 conn(0x7f9df40cca30 0x7f9df4071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:05.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.716+0000 7f9dfa7bb640 1 -- 172.21.15.100:0/3994680568 >> v1:172.21.15.122:6789/0 conn(0x7f9df4070ee0 legacy=0x7f9df4151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:05.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.716+0000 7f9dfa7bb640 1 -- 172.21.15.100:0/3994680568 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df40c9a90 con 0x7f9df40cce10 2024-09-15T21:47:05.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.717+0000 7f9dfa7bb640 1 --2- 172.21.15.100:0/3994680568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9df40cce10 0x7f9df415af50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9df40cc790 tx=0x7f9dec02c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c037a18787c05d53 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:05.720 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.717+0000 7f9df8fb8640 1 -- 172.21.15.100:0/3994680568 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec035020 con 0x7f9df40cce10 2024-09-15T21:47:05.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.717+0000 7f9df8fb8640 1 -- 172.21.15.100:0/3994680568 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dec009600 con 0x7f9df40cce10 2024-09-15T21:47:05.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.717+0000 7f9df8fb8640 1 -- 172.21.15.100:0/3994680568 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0099d0 con 0x7f9df40cce10 2024-09-15T21:47:05.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.718+0000 7f9dfc21e640 1 -- 172.21.15.100:0/3994680568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9df40cce10 msgr2=0x7f9df415af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:05.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.718+0000 7f9dfc21e640 1 --2- 172.21.15.100:0/3994680568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9df40cce10 0x7f9df415af50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9df40cc790 tx=0x7f9dec02c9e0 comp rx=0 tx=0).stop 2024-09-15T21:47:05.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.718+0000 7f9dfc21e640 1 -- 172.21.15.100:0/3994680568 shutdown_connections 2024-09-15T21:47:05.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.718+0000 7f9dfc21e640 1 --2- 172.21.15.100:0/3994680568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9df40cce10 0x7f9df415af50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:05.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.718+0000 7f9dfc21e640 1 --2- 172.21.15.100:0/3994680568 >> v2:172.21.15.122:3300/0 conn(0x7f9df40cca30 0x7f9df4071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:05.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.718+0000 7f9dfc21e640 1 -- 172.21.15.100:0/3994680568 >> 172.21.15.100:0/3994680568 conn(0x7f9df4067dd0 msgr2=0x7f9df406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:05.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.718+0000 7f9dfc21e640 1 -- 172.21.15.100:0/3994680568 shutdown_connections 2024-09-15T21:47:05.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.718+0000 7f9dfc21e640 1 -- 172.21.15.100:0/3994680568 wait complete. 2024-09-15T21:47:05.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.719+0000 7f9dfc21e640 1 Processor -- start 2024-09-15T21:47:05.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.719+0000 7f9dfc21e640 1 -- start start 2024-09-15T21:47:05.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9dfc21e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9df4070ee0 0x7f9df41edb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:05.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9dfc21e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9df40cca30 0x7f9df40cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9dfc21e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9df40cce10 0x7f9df4156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9dfc21e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9df4151ce0 con 0x7f9df40cca30 2024-09-15T21:47:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9dfc21e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9df4151b60 con 0x7f9df4070ee0 2024-09-15T21:47:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9dfc21e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9df4071990 con 0x7f9df40cce10 2024-09-15T21:47:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9df97b9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9df40cca30 0x7f9df40cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9df97b9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9df40cca30 0x7f9df40cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49052/0 (socket says 172.21.15.100:49052) 2024-09-15T21:47:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9df97b9640 1 -- 172.21.15.100:0/774470838 learned_addr learned my addr 172.21.15.100:0/774470838 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9df9fba640 1 --2- 172.21.15.100:0/774470838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9df4070ee0 0x7f9df41edb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9dfa7bb640 1 --2- 172.21.15.100:0/774470838 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9df40cce10 0x7f9df4156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9df97b9640 1 -- 172.21.15.100:0/774470838 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9df40cce10 msgr2=0x7f9df4156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9df97b9640 1 --2- 172.21.15.100:0/774470838 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9df40cce10 0x7f9df4156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9df97b9640 1 -- 172.21.15.100:0/774470838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9df4070ee0 msgr2=0x7f9df41edb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:05.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9df97b9640 1 --2- 172.21.15.100:0/774470838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9df4070ee0 0x7f9df41edb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9df97b9640 1 -- 172.21.15.100:0/774470838 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9df414b800 con 0x7f9df40cca30 2024-09-15T21:47:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9df97b9640 1 --2- 172.21.15.100:0/774470838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9df40cca30 0x7f9df40cb890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9de400c400 tx=0x7f9de400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9dea7fc640 1 -- 172.21.15.100:0/774470838 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de40052d0 con 0x7f9df40cca30 2024-09-15T21:47:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9dea7fc640 1 -- 172.21.15.100:0/774470838 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9de4005490 con 0x7f9df40cca30 2024-09-15T21:47:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9dfc21e640 1 -- 172.21.15.100:0/774470838 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9df4156d00 con 0x7f9df40cca30 2024-09-15T21:47:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9dea7fc640 1 -- 172.21.15.100:0/774470838 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4007260 con 0x7f9df40cca30 2024-09-15T21:47:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.720+0000 7f9dfc21e640 1 -- 172.21.15.100:0/774470838 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9df40cadb0 con 0x7f9df40cca30 2024-09-15T21:47:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.721+0000 7f9dfc21e640 1 -- 172.21.15.100:0/774470838 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9df40caba0 con 0x7f9df40cca30 2024-09-15T21:47:05.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.722+0000 7f9dea7fc640 1 -- 172.21.15.100:0/774470838 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9de4007400 con 0x7f9df40cca30 2024-09-15T21:47:05.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.722+0000 7f9dea7fc640 1 --2- 172.21.15.100:0/774470838 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9dd806c2b0 0x7f9dd806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:05.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.723+0000 7f9dea7fc640 1 -- 172.21.15.100:0/774470838 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5854+0+0 (secure 0 0 0) 0x7f9de408a910 con 0x7f9df40cca30 2024-09-15T21:47:05.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.723+0000 7f9df9fba640 1 --2- 172.21.15.100:0/774470838 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9dd806c2b0 0x7f9dd806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:05.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.723+0000 7f9df9fba640 1 --2- 172.21.15.100:0/774470838 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9dd806c2b0 0x7f9dd806e770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009b40 tx=0x7f9df0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:05.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.727+0000 7f9dea7fc640 1 -- 172.21.15.100:0/774470838 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9df414b800 con 0x7f9df40cca30 2024-09-15T21:47:05.735 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:05.733+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:05.742 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:05.742 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:05.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:05.742 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:05.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.742 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:05.742 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:05.742 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.325944585999999, 2024-09-15T21:47:05.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0050344960000000003, 2024-09-15T21:47:05.742 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0199999999999998e-07 2024-09-15T21:47:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413470+0000", 2024-09-15T21:47:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4329999999999998e-06 2024-09-15T21:47:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413579+0000", 2024-09-15T21:47:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108593 2024-09-15T21:47:05.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413587+0000", 2024-09-15T21:47:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3100000000000001e-06 2024-09-15T21:47:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413691+0000", 2024-09-15T21:47:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010357 2024-09-15T21:47:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413887+0000", 2024-09-15T21:47:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019666799999999999 2024-09-15T21:47:05.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:05.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414027+0000", 2024-09-15T21:47:05.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013996700000000001 2024-09-15T21:47:05.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416360+0000", 2024-09-15T21:47:05.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002332204 2024-09-15T21:47:05.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418437+0000", 2024-09-15T21:47:05.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002077279 2024-09-15T21:47:05.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418469+0000", 2024-09-15T21:47:05.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2359e-05 2024-09-15T21:47:05.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418502+0000", 2024-09-15T21:47:05.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2811000000000001e-05 2024-09-15T21:47:05.747 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.747 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.747 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.325940545999998, 2024-09-15T21:47:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0053538780000000003, 2024-09-15T21:47:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:05.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2000000000000001e-07 2024-09-15T21:47:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413474+0000", 2024-09-15T21:47:05.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-15T21:47:05.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413612+0000", 2024-09-15T21:47:05.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000137727 2024-09-15T21:47:05.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413618+0000", 2024-09-15T21:47:05.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.241e-06 2024-09-15T21:47:05.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414443+0000", 2024-09-15T21:47:05.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082447 2024-09-15T21:47:05.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414523+0000", 2024-09-15T21:47:05.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0059e-05 2024-09-15T21:47:05.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:05.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414653+0000", 2024-09-15T21:47:05.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013031699999999999 2024-09-15T21:47:05.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416366+0000", 2024-09-15T21:47:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001712917 2024-09-15T21:47:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418769+0000", 2024-09-15T21:47:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024031529999999999 2024-09-15T21:47:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418802+0000", 2024-09-15T21:47:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2925000000000001e-05 2024-09-15T21:47:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418825+0000", 2024-09-15T21:47:05.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3204e-05 2024-09-15T21:47:05.753 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.753 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.753 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.753 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:05.753 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:05.753 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.325932524999999, 2024-09-15T21:47:05.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059708640000000002, 2024-09-15T21:47:05.753 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.753 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.753 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:05.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7300000000000002e-07 2024-09-15T21:47:05.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413482+0000", 2024-09-15T21:47:05.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2359999999999999e-06 2024-09-15T21:47:05.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414323+0000", 2024-09-15T21:47:05.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00084095399999999999 2024-09-15T21:47:05.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414332+0000", 2024-09-15T21:47:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0880000000000001e-06 2024-09-15T21:47:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415439+0000", 2024-09-15T21:47:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011071869999999999 2024-09-15T21:47:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415554+0000", 2024-09-15T21:47:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114499 2024-09-15T21:47:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:05.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415682+0000", 2024-09-15T21:47:05.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012802500000000001 2024-09-15T21:47:05.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.417957+0000", 2024-09-15T21:47:05.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022746720000000002 2024-09-15T21:47:05.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419385+0000", 2024-09-15T21:47:05.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001428826 2024-09-15T21:47:05.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419420+0000", 2024-09-15T21:47:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4875000000000001e-05 2024-09-15T21:47:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419450+0000", 2024-09-15T21:47:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0128999999999999e-05 2024-09-15T21:47:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:05.758 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.325928707000003, 2024-09-15T21:47:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0087186220000000005, 2024-09-15T21:47:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:05.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413484+0000", 2024-09-15T21:47:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-15T21:47:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413487+0000", 2024-09-15T21:47:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.534e-06 2024-09-15T21:47:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414353+0000", 2024-09-15T21:47:05.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00086616999999999998 2024-09-15T21:47:05.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414359+0000", 2024-09-15T21:47:05.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3279999999999996e-06 2024-09-15T21:47:05.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416041+0000", 2024-09-15T21:47:05.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001682467 2024-09-15T21:47:05.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416158+0000", 2024-09-15T21:47:05.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117242 2024-09-15T21:47:05.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:05.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416284+0000", 2024-09-15T21:47:05.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012509000000000001 2024-09-15T21:47:05.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.417992+0000", 2024-09-15T21:47:05.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017087059999999999 2024-09-15T21:47:05.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422142+0000", 2024-09-15T21:47:05.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041501180000000004 2024-09-15T21:47:05.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422176+0000", 2024-09-15T21:47:05.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4109999999999997e-05 2024-09-15T21:47:05.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422202+0000", 2024-09-15T21:47:05.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5585e-05 2024-09-15T21:47:05.763 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.763 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.325923281999998, 2024-09-15T21:47:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0088000370000000001, 2024-09-15T21:47:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413491+0000", 2024-09-15T21:47:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.773e-06 2024-09-15T21:47:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414372+0000", 2024-09-15T21:47:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00088122000000000005 2024-09-15T21:47:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414378+0000", 2024-09-15T21:47:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4380000000000003e-06 2024-09-15T21:47:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416631+0000", 2024-09-15T21:47:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022530319999999999 2024-09-15T21:47:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416765+0000", 2024-09-15T21:47:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013423499999999999 2024-09-15T21:47:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422289+0000", 2024-09-15T21:47:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055240530000000001 2024-09-15T21:47:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 31.284354962999998, 2024-09-15T21:47:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056900609999999997, 2024-09-15T21:47:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455058+0000", 2024-09-15T21:47:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.51e-07 2024-09-15T21:47:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455061+0000", 2024-09-15T21:47:05.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.145e-06 2024-09-15T21:47:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455158+0000", 2024-09-15T21:47:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7101000000000004e-05 2024-09-15T21:47:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455165+0000", 2024-09-15T21:47:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1019999999999999e-06 2024-09-15T21:47:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456433+0000", 2024-09-15T21:47:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012678259999999999 2024-09-15T21:47:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456517+0000", 2024-09-15T21:47:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4373e-05 2024-09-15T21:47:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456677+0000", 2024-09-15T21:47:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015963300000000001 2024-09-15T21:47:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458636+0000", 2024-09-15T21:47:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019594030000000002 2024-09-15T21:47:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460665+0000", 2024-09-15T21:47:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020283549999999999 2024-09-15T21:47:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460707+0000", 2024-09-15T21:47:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2398000000000003e-05 2024-09-15T21:47:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460747+0000", 2024-09-15T21:47:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0274000000000001e-05 2024-09-15T21:47:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 31.284347769, 2024-09-15T21:47:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086420030000000005, 2024-09-15T21:47:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.773 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455065+0000", 2024-09-15T21:47:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3700000000000001e-07 2024-09-15T21:47:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455068+0000", 2024-09-15T21:47:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.915e-06 2024-09-15T21:47:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455366+0000", 2024-09-15T21:47:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00029825400000000002 2024-09-15T21:47:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455374+0000", 2024-09-15T21:47:05.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6240000000000008e-06 2024-09-15T21:47:05.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456199+0000", 2024-09-15T21:47:05.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082448500000000002 2024-09-15T21:47:05.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457034+0000", 2024-09-15T21:47:05.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00083553399999999997 2024-09-15T21:47:05.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:05.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457171+0000", 2024-09-15T21:47:05.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013653600000000001 2024-09-15T21:47:05.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462663+0000", 2024-09-15T21:47:05.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054924420000000002 2024-09-15T21:47:05.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463613+0000", 2024-09-15T21:47:05.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094971499999999996 2024-09-15T21:47:05.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463663+0000", 2024-09-15T21:47:05.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9636999999999999e-05 2024-09-15T21:47:05.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463706+0000", 2024-09-15T21:47:05.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3523999999999999e-05 2024-09-15T21:47:05.778 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.778 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.778 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.778 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.778 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:05.778 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 31.284343240999998, 2024-09-15T21:47:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090408420000000003, 2024-09-15T21:47:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:05.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.79e-07 2024-09-15T21:47:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455072+0000", 2024-09-15T21:47:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6189999999999998e-06 2024-09-15T21:47:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455403+0000", 2024-09-15T21:47:05.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033167299999999999 2024-09-15T21:47:05.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455420+0000", 2024-09-15T21:47:05.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6518999999999999e-05 2024-09-15T21:47:05.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457448+0000", 2024-09-15T21:47:05.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020275710000000001 2024-09-15T21:47:05.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3844999999999994e-05 2024-09-15T21:47:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457684+0000", 2024-09-15T21:47:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015239 2024-09-15T21:47:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462678+0000", 2024-09-15T21:47:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049939279999999999 2024-09-15T21:47:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464017+0000", 2024-09-15T21:47:05.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013397140000000001 2024-09-15T21:47:05.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464071+0000", 2024-09-15T21:47:05.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3735e-05 2024-09-15T21:47:05.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464110+0000", 2024-09-15T21:47:05.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8569000000000003e-05 2024-09-15T21:47:05.783 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 31.284331628, 2024-09-15T21:47:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091802889999999995, 2024-09-15T21:47:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455081+0000", 2024-09-15T21:47:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455084+0000", 2024-09-15T21:47:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7810000000000001e-06 2024-09-15T21:47:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455536+0000", 2024-09-15T21:47:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00045219100000000001 2024-09-15T21:47:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455583+0000", 2024-09-15T21:47:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7308999999999997e-05 2024-09-15T21:47:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457998+0000", 2024-09-15T21:47:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024154290000000002 2024-09-15T21:47:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458147+0000", 2024-09-15T21:47:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014834599999999999 2024-09-15T21:47:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458265+0000", 2024-09-15T21:47:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118458 2024-09-15T21:47:05.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464134+0000", 2024-09-15T21:47:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005869038 2024-09-15T21:47:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464187+0000", 2024-09-15T21:47:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2343999999999999e-05 2024-09-15T21:47:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464222+0000", 2024-09-15T21:47:05.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5447000000000001e-05 2024-09-15T21:47:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464261+0000", 2024-09-15T21:47:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8659999999999999e-05 2024-09-15T21:47:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 25.263709249000001, 2024-09-15T21:47:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048468290000000004, 2024-09-15T21:47:05.789 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.35e-07 2024-09-15T21:47:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475706+0000", 2024-09-15T21:47:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.994e-06 2024-09-15T21:47:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475770+0000", 2024-09-15T21:47:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3904000000000003e-05 2024-09-15T21:47:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475775+0000", 2024-09-15T21:47:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2490000000000001e-06 2024-09-15T21:47:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476502+0000", 2024-09-15T21:47:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072697300000000001 2024-09-15T21:47:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476611+0000", 2024-09-15T21:47:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0001091 2024-09-15T21:47:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476807+0000", 2024-09-15T21:47:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000195892 2024-09-15T21:47:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479678+0000", 2024-09-15T21:47:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028704709999999999 2024-09-15T21:47:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480444+0000", 2024-09-15T21:47:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00076602099999999998 2024-09-15T21:47:05.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480502+0000", 2024-09-15T21:47:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8313999999999998e-05 2024-09-15T21:47:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480550+0000", 2024-09-15T21:47:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7676000000000002e-05 2024-09-15T21:47:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.794 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 25.263702074000001, 2024-09-15T21:47:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051250929999999998, 2024-09-15T21:47:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1e-07 2024-09-15T21:47:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475723+0000", 2024-09-15T21:47:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2758000000000001e-05 2024-09-15T21:47:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475972+0000", 2024-09-15T21:47:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024886299999999999 2024-09-15T21:47:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475979+0000", 2024-09-15T21:47:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0230000000000004e-06 2024-09-15T21:47:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476560+0000", 2024-09-15T21:47:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00058097599999999998 2024-09-15T21:47:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476689+0000", 2024-09-15T21:47:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129486 2024-09-15T21:47:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480835+0000", 2024-09-15T21:47:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041457769999999998 2024-09-15T21:47:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.798 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.194786849, 2024-09-15T21:47:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054420570000000001, 2024-09-15T21:47:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:05.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544626+0000", 2024-09-15T21:47:05.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8100000000000003e-07 2024-09-15T21:47:05.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544629+0000", 2024-09-15T21:47:05.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9869999999999999e-06 2024-09-15T21:47:05.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544775+0000", 2024-09-15T21:47:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014610599999999999 2024-09-15T21:47:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544782+0000", 2024-09-15T21:47:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4109999999999999e-06 2024-09-15T21:47:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545617+0000", 2024-09-15T21:47:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00083468300000000002 2024-09-15T21:47:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545690+0000", 2024-09-15T21:47:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3545000000000001e-05 2024-09-15T21:47:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545803+0000", 2024-09-15T21:47:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000112083 2024-09-15T21:47:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548353+0000", 2024-09-15T21:47:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025507220000000001 2024-09-15T21:47:05.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549964+0000", 2024-09-15T21:47:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001610382 2024-09-15T21:47:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550021+0000", 2024-09-15T21:47:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6898999999999999e-05 2024-09-15T21:47:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550067+0000", 2024-09-15T21:47:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758000000000002e-05 2024-09-15T21:47:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.194781571, 2024-09-15T21:47:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056323399999999996, 2024-09-15T21:47:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:47:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544634+0000", 2024-09-15T21:47:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.58e-06 2024-09-15T21:47:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544800+0000", 2024-09-15T21:47:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016576899999999999 2024-09-15T21:47:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544808+0000", 2024-09-15T21:47:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5179999999999998e-06 2024-09-15T21:47:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546075+0000", 2024-09-15T21:47:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012676969999999999 2024-09-15T21:47:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546179+0000", 2024-09-15T21:47:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000103935 2024-09-15T21:47:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546286+0000", 2024-09-15T21:47:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106758 2024-09-15T21:47:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548528+0000", 2024-09-15T21:47:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022420510000000001 2024-09-15T21:47:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550119+0000", 2024-09-15T21:47:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015907079999999999 2024-09-15T21:47:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550202+0000", 2024-09-15T21:47:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3374000000000005e-05 2024-09-15T21:47:05.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550263+0000", 2024-09-15T21:47:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0643999999999997e-05 2024-09-15T21:47:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.194776600000001, 2024-09-15T21:47:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061905470000000002, 2024-09-15T21:47:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.813 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:05.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6899999999999999e-07 2024-09-15T21:47:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544638+0000", 2024-09-15T21:47:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3920000000000001e-06 2024-09-15T21:47:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544819+0000", 2024-09-15T21:47:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018065099999999999 2024-09-15T21:47:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544825+0000", 2024-09-15T21:47:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7239999999999997e-06 2024-09-15T21:47:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546583+0000", 2024-09-15T21:47:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017588219999999999 2024-09-15T21:47:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546665+0000", 2024-09-15T21:47:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1693000000000005e-05 2024-09-15T21:47:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546808+0000", 2024-09-15T21:47:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142993 2024-09-15T21:47:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548533+0000", 2024-09-15T21:47:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017246 2024-09-15T21:47:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550606+0000", 2024-09-15T21:47:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020733969999999998 2024-09-15T21:47:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550783+0000", 2024-09-15T21:47:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000176565 2024-09-15T21:47:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550826+0000", 2024-09-15T21:47:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3441e-05 2024-09-15T21:47:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.818 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.194768879000002, 2024-09-15T21:47:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052813010000000004, 2024-09-15T21:47:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3799999999999999e-07 2024-09-15T21:47:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544647+0000", 2024-09-15T21:47:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6569999999999999e-06 2024-09-15T21:47:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544858+0000", 2024-09-15T21:47:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021069900000000001 2024-09-15T21:47:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544862+0000", 2024-09-15T21:47:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4900000000000002e-06 2024-09-15T21:47:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545610+0000", 2024-09-15T21:47:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00074786999999999998 2024-09-15T21:47:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545706+0000", 2024-09-15T21:47:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6271e-05 2024-09-15T21:47:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545834+0000", 2024-09-15T21:47:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012759499999999999 2024-09-15T21:47:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548554+0000", 2024-09-15T21:47:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027199339999999998 2024-09-15T21:47:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549818+0000", 2024-09-15T21:47:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012642199999999999 2024-09-15T21:47:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549884+0000", 2024-09-15T21:47:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5843e-05 2024-09-15T21:47:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549925+0000", 2024-09-15T21:47:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0484000000000001e-05 2024-09-15T21:47:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.823 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 20.194760202000001, 2024-09-15T21:47:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051106110000000001, 2024-09-15T21:47:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:05.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2700000000000001e-07 2024-09-15T21:47:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544667+0000", 2024-09-15T21:47:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4746000000000001e-05 2024-09-15T21:47:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545021+0000", 2024-09-15T21:47:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035412900000000001 2024-09-15T21:47:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545052+0000", 2024-09-15T21:47:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1158000000000002e-05 2024-09-15T21:47:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546050+0000", 2024-09-15T21:47:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00099782800000000008 2024-09-15T21:47:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546291+0000", 2024-09-15T21:47:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024093500000000001 2024-09-15T21:47:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546434+0000", 2024-09-15T21:47:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014308600000000001 2024-09-15T21:47:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548583+0000", 2024-09-15T21:47:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021491029999999999 2024-09-15T21:47:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549695+0000", 2024-09-15T21:47:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011116419999999999 2024-09-15T21:47:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549726+0000", 2024-09-15T21:47:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1195000000000002e-05 2024-09-15T21:47:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549763+0000", 2024-09-15T21:47:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6562e-05 2024-09-15T21:47:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:17 2.14 2:2eebc8f4:::62f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.17433059, 2024-09-15T21:47:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071004800000000002, 2024-09-15T21:47:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.829 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565083+0000", 2024-09-15T21:47:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.689e-06 2024-09-15T21:47:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565090+0000", 2024-09-15T21:47:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5930000000000002e-06 2024-09-15T21:47:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565119+0000", 2024-09-15T21:47:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8785999999999998e-05 2024-09-15T21:47:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565126+0000", 2024-09-15T21:47:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0110000000000003e-06 2024-09-15T21:47:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565237+0000", 2024-09-15T21:47:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011161 2024-09-15T21:47:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565383+0000", 2024-09-15T21:47:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014626900000000001 2024-09-15T21:47:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565520+0000", 2024-09-15T21:47:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013676399999999999 2024-09-15T21:47:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569909+0000", 2024-09-15T21:47:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043890149999999996 2024-09-15T21:47:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572096+0000", 2024-09-15T21:47:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021866559999999999 2024-09-15T21:47:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572144+0000", 2024-09-15T21:47:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7725000000000001e-05 2024-09-15T21:47:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572182+0000", 2024-09-15T21:47:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8362000000000003e-05 2024-09-15T21:47:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.834 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.174319492, 2024-09-15T21:47:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072110280000000004, 2024-09-15T21:47:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4299999999999999e-07 2024-09-15T21:47:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565096+0000", 2024-09-15T21:47:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4319999999999999e-06 2024-09-15T21:47:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565152+0000", 2024-09-15T21:47:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5686999999999997e-05 2024-09-15T21:47:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565157+0000", 2024-09-15T21:47:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2700000000000004e-06 2024-09-15T21:47:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565847+0000", 2024-09-15T21:47:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068937299999999996 2024-09-15T21:47:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565940+0000", 2024-09-15T21:47:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3573999999999996e-05 2024-09-15T21:47:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566085+0000", 2024-09-15T21:47:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000144902 2024-09-15T21:47:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570595+0000", 2024-09-15T21:47:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045103469999999996 2024-09-15T21:47:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572224+0000", 2024-09-15T21:47:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016282930000000001 2024-09-15T21:47:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572260+0000", 2024-09-15T21:47:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6103999999999999e-05 2024-09-15T21:47:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572304+0000", 2024-09-15T21:47:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3803000000000001e-05 2024-09-15T21:47:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.839 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.174315027, 2024-09-15T21:47:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055995389999999997, 2024-09-15T21:47:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9999999999999999e-07 2024-09-15T21:47:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2399999999999999e-06 2024-09-15T21:47:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565280+0000", 2024-09-15T21:47:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017929400000000001 2024-09-15T21:47:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565292+0000", 2024-09-15T21:47:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2193e-05 2024-09-15T21:47:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565357+0000", 2024-09-15T21:47:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4684000000000006e-05 2024-09-15T21:47:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565499+0000", 2024-09-15T21:47:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142032 2024-09-15T21:47:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565634+0000", 2024-09-15T21:47:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000135613 2024-09-15T21:47:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569941+0000", 2024-09-15T21:47:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043066789999999999 2024-09-15T21:47:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570585+0000", 2024-09-15T21:47:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00064364700000000001 2024-09-15T21:47:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570647+0000", 2024-09-15T21:47:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2173000000000003e-05 2024-09-15T21:47:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570697+0000", 2024-09-15T21:47:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9784000000000003e-05 2024-09-15T21:47:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.174310797, 2024-09-15T21:47:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066548939999999997, 2024-09-15T21:47:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:05.845 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:05.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565102+0000", 2024-09-15T21:47:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-15T21:47:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565103+0000", 2024-09-15T21:47:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6700000000000001e-06 2024-09-15T21:47:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565608+0000", 2024-09-15T21:47:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050518299999999998 2024-09-15T21:47:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565616+0000", 2024-09-15T21:47:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8199999999999997e-06 2024-09-15T21:47:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566598+0000", 2024-09-15T21:47:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00098176300000000008 2024-09-15T21:47:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566871+0000", 2024-09-15T21:47:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027312799999999998 2024-09-15T21:47:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567015+0000", 2024-09-15T21:47:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014428499999999999 2024-09-15T21:47:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570729+0000", 2024-09-15T21:47:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037132459999999999 2024-09-15T21:47:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571667+0000", 2024-09-15T21:47:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00093862299999999997 2024-09-15T21:47:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571718+0000", 2024-09-15T21:47:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1146999999999997e-05 2024-09-15T21:47:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571756+0000", 2024-09-15T21:47:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7821000000000003e-05 2024-09-15T21:47:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:05.850 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:05.852 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:47:05.852 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:05.850+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:05.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:05.905+0000 7f9dfc21e640 1 -- 172.21.15.100:0/774470838 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f9df415b980 con 0x7f9df40cca30 2024-09-15T21:47:05.952 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:05.951+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:05.955 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:05.956 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:05.956 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:05.956 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:05.964 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:47:06.053 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:06.052+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:06.066 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:06.067 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:06.067 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:06.067 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:06.067 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:06.075 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:47:06.155 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:06.154+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:06.181 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:06.181 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:06.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:06.181 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:06.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.181 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:47:06.181 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:06.181 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 52.943382210000003, 2024-09-15T21:47:06.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0944283340000001, 2024-09-15T21:47:06.182 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.182 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.182 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.182 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:06.182 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:06.182 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:06.182 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2024-09-15T21:47:06.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.182 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:06.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:06.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234360+0000", 2024-09-15T21:47:06.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3710000000000001e-06 2024-09-15T21:47:06.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234374+0000", 2024-09-15T21:47:06.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4003e-05 2024-09-15T21:47:06.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234392+0000", 2024-09-15T21:47:06.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7294000000000001e-05 2024-09-15T21:47:06.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234405+0000", 2024-09-15T21:47:06.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2961e-05 2024-09-15T21:47:06.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234483+0000", 2024-09-15T21:47:06.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8850000000000006e-05 2024-09-15T21:47:06.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-15T21:47:06.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234491+0000", 2024-09-15T21:47:06.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3529999999999997e-06 2024-09-15T21:47:06.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327253+0000", 2024-09-15T21:47:06.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.092761962 2024-09-15T21:47:06.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327456+0000", 2024-09-15T21:47:06.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000202893 2024-09-15T21:47:06.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327652+0000", 2024-09-15T21:47:06.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019613000000000001 2024-09-15T21:47:06.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328368+0000", 2024-09-15T21:47:06.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000716582 2024-09-15T21:47:06.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328682+0000", 2024-09-15T21:47:06.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000313825 2024-09-15T21:47:06.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328743+0000", 2024-09-15T21:47:06.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0571e-05 2024-09-15T21:47:06.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328787+0000", 2024-09-15T21:47:06.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4539000000000002e-05 2024-09-15T21:47:06.188 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.188 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.188 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.188 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:06.188 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:06.189 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.822216808, 2024-09-15T21:47:06.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0089551270000000002, 2024-09-15T21:47:06.189 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.189 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.189 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.189 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:06.189 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:06.189 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:06.189 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 35 2024-09-15T21:47:06.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.189 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:06.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:06.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355525+0000", 2024-09-15T21:47:06.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:47:06.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355606+0000", 2024-09-15T21:47:06.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1723000000000003e-05 2024-09-15T21:47:06.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355624+0000", 2024-09-15T21:47:06.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7645000000000001e-05 2024-09-15T21:47:06.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355629+0000", 2024-09-15T21:47:06.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7319999999999998e-06 2024-09-15T21:47:06.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358315+0000", 2024-09-15T21:47:06.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026858659999999999 2024-09-15T21:47:06.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358373+0000", 2024-09-15T21:47:06.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8593e-05 2024-09-15T21:47:06.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358521+0000", 2024-09-15T21:47:06.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014817500000000001 2024-09-15T21:47:06.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360353+0000", 2024-09-15T21:47:06.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018317100000000001 2024-09-15T21:47:06.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364424+0000", 2024-09-15T21:47:06.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040707970000000001 2024-09-15T21:47:06.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364452+0000", 2024-09-15T21:47:06.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7937999999999999e-05 2024-09-15T21:47:06.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364479+0000", 2024-09-15T21:47:06.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7744999999999999e-05 2024-09-15T21:47:06.195 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.195 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.195 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.195 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:06.195 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:06.195 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.822132844999999, 2024-09-15T21:47:06.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090267869999999997, 2024-09-15T21:47:06.195 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.195 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.195 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.195 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:06.196 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:06.196 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:06.196 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 36 2024-09-15T21:47:06.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.196 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:06.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:06.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355609+0000", 2024-09-15T21:47:06.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3099999999999998e-07 2024-09-15T21:47:06.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355612+0000", 2024-09-15T21:47:06.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1489999999999998e-06 2024-09-15T21:47:06.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355637+0000", 2024-09-15T21:47:06.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.508e-05 2024-09-15T21:47:06.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355640+0000", 2024-09-15T21:47:06.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7930000000000002e-06 2024-09-15T21:47:06.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359195+0000", 2024-09-15T21:47:06.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035555529999999999 2024-09-15T21:47:06.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359283+0000", 2024-09-15T21:47:06.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8042999999999997e-05 2024-09-15T21:47:06.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359419+0000", 2024-09-15T21:47:06.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013576699999999999 2024-09-15T21:47:06.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360361+0000", 2024-09-15T21:47:06.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094171000000000001 2024-09-15T21:47:06.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364536+0000", 2024-09-15T21:47:06.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00417484 2024-09-15T21:47:06.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364590+0000", 2024-09-15T21:47:06.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4355999999999999e-05 2024-09-15T21:47:06.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364635+0000", 2024-09-15T21:47:06.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5065000000000003e-05 2024-09-15T21:47:06.201 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.201 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.201 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.202 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:06.202 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:06.202 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.822128239999998, 2024-09-15T21:47:06.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091162750000000001, 2024-09-15T21:47:06.202 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.202 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.202 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.202 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:06.202 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:06.202 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:06.202 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 37 2024-09-15T21:47:06.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.202 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:06.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:06.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:06.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:47:06.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355615+0000", 2024-09-15T21:47:06.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-15T21:47:06.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355646+0000", 2024-09-15T21:47:06.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0391000000000001e-05 2024-09-15T21:47:06.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355648+0000", 2024-09-15T21:47:06.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6309999999999999e-06 2024-09-15T21:47:06.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359770+0000", 2024-09-15T21:47:06.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041222899999999998 2024-09-15T21:47:06.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359854+0000", 2024-09-15T21:47:06.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3399000000000006e-05 2024-09-15T21:47:06.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359988+0000", 2024-09-15T21:47:06.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013442200000000001 2024-09-15T21:47:06.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360524+0000", 2024-09-15T21:47:06.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00053582900000000004 2024-09-15T21:47:06.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364673+0000", 2024-09-15T21:47:06.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041493199999999997 2024-09-15T21:47:06.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364704+0000", 2024-09-15T21:47:06.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0380999999999999e-05 2024-09-15T21:47:06.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364729+0000", 2024-09-15T21:47:06.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5335999999999999e-05 2024-09-15T21:47:06.207 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.207 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.207 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.208 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:06.208 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:06.208 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.753411002, 2024-09-15T21:47:06.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0026270020000001, 2024-09-15T21:47:06.208 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.208 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.208 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.208 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:06.208 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:06.208 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:06.208 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 88 2024-09-15T21:47:06.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.208 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:06.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:06.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424331+0000", 2024-09-15T21:47:06.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4299999999999998e-07 2024-09-15T21:47:06.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424333+0000", 2024-09-15T21:47:06.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2359999999999999e-06 2024-09-15T21:47:06.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422895+0000", 2024-09-15T21:47:06.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99856184199999998 2024-09-15T21:47:06.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422902+0000", 2024-09-15T21:47:06.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4440000000000002e-06 2024-09-15T21:47:06.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423053+0000", 2024-09-15T21:47:06.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000150958 2024-09-15T21:47:06.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423160+0000", 2024-09-15T21:47:06.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106774 2024-09-15T21:47:06.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423344+0000", 2024-09-15T21:47:06.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018424199999999999 2024-09-15T21:47:06.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.424730+0000", 2024-09-15T21:47:06.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001386155 2024-09-15T21:47:06.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426756+0000", 2024-09-15T21:47:06.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020261150000000002 2024-09-15T21:47:06.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426906+0000", 2024-09-15T21:47:06.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000149176 2024-09-15T21:47:06.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426957+0000", 2024-09-15T21:47:06.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1616999999999997e-05 2024-09-15T21:47:06.215 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.215 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.215 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.215 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:06.215 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:06.215 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.623337804000002, 2024-09-15T21:47:06.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095037790000000004, 2024-09-15T21:47:06.215 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.216 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.216 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.216 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:06.216 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:06.216 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:06.216 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 114 2024-09-15T21:47:06.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.216 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:06.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:06.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554404+0000", 2024-09-15T21:47:06.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2399999999999999e-07 2024-09-15T21:47:06.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554407+0000", 2024-09-15T21:47:06.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-15T21:47:06.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554632+0000", 2024-09-15T21:47:06.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022568999999999999 2024-09-15T21:47:06.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554640+0000", 2024-09-15T21:47:06.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7970000000000001e-06 2024-09-15T21:47:06.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561325+0000", 2024-09-15T21:47:06.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066848020000000001 2024-09-15T21:47:06.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561402+0000", 2024-09-15T21:47:06.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6851e-05 2024-09-15T21:47:06.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561530+0000", 2024-09-15T21:47:06.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012863999999999999 2024-09-15T21:47:06.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563144+0000", 2024-09-15T21:47:06.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016132849999999999 2024-09-15T21:47:06.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563800+0000", 2024-09-15T21:47:06.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00065586099999999999 2024-09-15T21:47:06.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563857+0000", 2024-09-15T21:47:06.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.702e-05 2024-09-15T21:47:06.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563907+0000", 2024-09-15T21:47:06.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0479000000000002e-05 2024-09-15T21:47:06.221 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.222 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.222 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.222 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:06.222 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:06.222 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.623332806999997, 2024-09-15T21:47:06.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011813423999999999, 2024-09-15T21:47:06.222 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.222 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.222 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.222 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:06.222 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:06.222 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:06.223 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 115 2024-09-15T21:47:06.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.223 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:06.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:06.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554409+0000", 2024-09-15T21:47:06.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1100000000000002e-07 2024-09-15T21:47:06.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554498+0000", 2024-09-15T21:47:06.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9642999999999995e-05 2024-09-15T21:47:06.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554656+0000", 2024-09-15T21:47:06.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000158114 2024-09-15T21:47:06.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554661+0000", 2024-09-15T21:47:06.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0730000000000004e-06 2024-09-15T21:47:06.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561862+0000", 2024-09-15T21:47:06.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072002999999999998 2024-09-15T21:47:06.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561939+0000", 2024-09-15T21:47:06.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7265e-05 2024-09-15T21:47:06.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562073+0000", 2024-09-15T21:47:06.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013371600000000001 2024-09-15T21:47:06.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563196+0000", 2024-09-15T21:47:06.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011231679999999999 2024-09-15T21:47:06.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566124+0000", 2024-09-15T21:47:06.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002928403 2024-09-15T21:47:06.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566183+0000", 2024-09-15T21:47:06.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8328000000000001e-05 2024-09-15T21:47:06.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566222+0000", 2024-09-15T21:47:06.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9103000000000002e-05 2024-09-15T21:47:06.227 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.228 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.228 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.228 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:06.228 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:06.228 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.623239366999996, 2024-09-15T21:47:06.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01181426, 2024-09-15T21:47:06.228 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.228 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.228 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.228 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:06.228 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:06.228 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:06.229 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 116 2024-09-15T21:47:06.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.229 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:06.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:06.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554503+0000", 2024-09-15T21:47:06.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8000000000000005e-07 2024-09-15T21:47:06.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554506+0000", 2024-09-15T21:47:06.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5899999999999999e-06 2024-09-15T21:47:06.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554674+0000", 2024-09-15T21:47:06.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016763600000000001 2024-09-15T21:47:06.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554678+0000", 2024-09-15T21:47:06.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5310000000000003e-06 2024-09-15T21:47:06.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562714+0000", 2024-09-15T21:47:06.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.008035258 2024-09-15T21:47:06.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562765+0000", 2024-09-15T21:47:06.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1093e-05 2024-09-15T21:47:06.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562845+0000", 2024-09-15T21:47:06.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0427999999999996e-05 2024-09-15T21:47:06.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563943+0000", 2024-09-15T21:47:06.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097647 2024-09-15T21:47:06.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566232+0000", 2024-09-15T21:47:06.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022890850000000002 2024-09-15T21:47:06.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566279+0000", 2024-09-15T21:47:06.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6692e-05 2024-09-15T21:47:06.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566316+0000", 2024-09-15T21:47:06.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.752e-05 2024-09-15T21:47:06.233 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.234 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.234 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.234 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:06.234 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:06.234 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.623233268, 2024-09-15T21:47:06.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012232125, 2024-09-15T21:47:06.234 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.234 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.234 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.234 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:06.234 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:06.234 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:06.235 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 117 2024-09-15T21:47:06.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.235 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:06.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:06.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:06.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7700000000000001e-07 2024-09-15T21:47:06.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554511+0000", 2024-09-15T21:47:06.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8430000000000001e-06 2024-09-15T21:47:06.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554690+0000", 2024-09-15T21:47:06.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017856299999999999 2024-09-15T21:47:06.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554698+0000", 2024-09-15T21:47:06.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6200000000000005e-06 2024-09-15T21:47:06.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563224+0000", 2024-09-15T21:47:06.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0085262640000000004 2024-09-15T21:47:06.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563342+0000", 2024-09-15T21:47:06.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117687 2024-09-15T21:47:06.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563472+0000", 2024-09-15T21:47:06.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012957999999999999 2024-09-15T21:47:06.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566367+0000", 2024-09-15T21:47:06.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028952280000000001 2024-09-15T21:47:06.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566680+0000", 2024-09-15T21:47:06.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031345100000000002 2024-09-15T21:47:06.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566714+0000", 2024-09-15T21:47:06.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3701000000000001e-05 2024-09-15T21:47:06.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566740+0000", 2024-09-15T21:47:06.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5911e-05 2024-09-15T21:47:06.240 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.240 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.240 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.240 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:06.240 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:06.240 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.280341186000001, 2024-09-15T21:47:06.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069806590000000002, 2024-09-15T21:47:06.241 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.241 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.241 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.241 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:06.241 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:06.241 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:06.241 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 134 2024-09-15T21:47:06.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.241 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:06.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:06.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897401+0000", 2024-09-15T21:47:06.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0500000000000003e-07 2024-09-15T21:47:06.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897492+0000", 2024-09-15T21:47:06.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1770999999999999e-05 2024-09-15T21:47:06.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897750+0000", 2024-09-15T21:47:06.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025778299999999998 2024-09-15T21:47:06.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897756+0000", 2024-09-15T21:47:06.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2140000000000001e-06 2024-09-15T21:47:06.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899312+0000", 2024-09-15T21:47:06.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001556144 2024-09-15T21:47:06.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899372+0000", 2024-09-15T21:47:06.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0015000000000001e-05 2024-09-15T21:47:06.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899444+0000", 2024-09-15T21:47:06.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1923999999999998e-05 2024-09-15T21:47:06.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901354+0000", 2024-09-15T21:47:06.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019092530000000001 2024-09-15T21:47:06.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904327+0000", 2024-09-15T21:47:06.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029734520000000001 2024-09-15T21:47:06.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904360+0000", 2024-09-15T21:47:06.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3216000000000002e-05 2024-09-15T21:47:06.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904381+0000", 2024-09-15T21:47:06.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0282e-05 2024-09-15T21:47:06.246 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.246 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.246 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.246 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:06.247 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:06.247 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.280245329000003, 2024-09-15T21:47:06.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070121059999999997, 2024-09-15T21:47:06.247 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.247 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.247 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.247 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:06.247 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:06.247 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:06.247 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 135 2024-09-15T21:47:06.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.247 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:06.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:06.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897497+0000", 2024-09-15T21:47:06.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4799999999999997e-07 2024-09-15T21:47:06.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897500+0000", 2024-09-15T21:47:06.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2949999999999998e-06 2024-09-15T21:47:06.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897781+0000", 2024-09-15T21:47:06.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028064300000000002 2024-09-15T21:47:06.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897786+0000", 2024-09-15T21:47:06.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5169999999999999e-06 2024-09-15T21:47:06.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899857+0000", 2024-09-15T21:47:06.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020705340000000002 2024-09-15T21:47:06.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899932+0000", 2024-09-15T21:47:06.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5840999999999994e-05 2024-09-15T21:47:06.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900005+0000", 2024-09-15T21:47:06.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2716e-05 2024-09-15T21:47:06.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902296+0000", 2024-09-15T21:47:06.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022912010000000001 2024-09-15T21:47:06.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904424+0000", 2024-09-15T21:47:06.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021278600000000001 2024-09-15T21:47:06.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904472+0000", 2024-09-15T21:47:06.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7586000000000001e-05 2024-09-15T21:47:06.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904508+0000", 2024-09-15T21:47:06.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6164999999999997e-05 2024-09-15T21:47:06.253 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.253 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.254 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.254 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:06.254 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:06.254 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.280220837999998, 2024-09-15T21:47:06.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070611429999999998, 2024-09-15T21:47:06.254 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.254 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.254 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.254 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:06.254 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:06.254 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:06.254 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 136 2024-09-15T21:47:06.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.255 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:06.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:06.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897521+0000", 2024-09-15T21:47:06.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2900000000000004e-07 2024-09-15T21:47:06.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897527+0000", 2024-09-15T21:47:06.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8490000000000002e-06 2024-09-15T21:47:06.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897799+0000", 2024-09-15T21:47:06.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027219499999999998 2024-09-15T21:47:06.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897804+0000", 2024-09-15T21:47:06.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4219999999999999e-06 2024-09-15T21:47:06.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900248+0000", 2024-09-15T21:47:06.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002443457 2024-09-15T21:47:06.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900306+0000", 2024-09-15T21:47:06.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8263000000000001e-05 2024-09-15T21:47:06.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900385+0000", 2024-09-15T21:47:06.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9161999999999999e-05 2024-09-15T21:47:06.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902300+0000", 2024-09-15T21:47:06.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001914947 2024-09-15T21:47:06.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904536+0000", 2024-09-15T21:47:06.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022354419999999998 2024-09-15T21:47:06.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904562+0000", 2024-09-15T21:47:06.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6378999999999999e-05 2024-09-15T21:47:06.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904581+0000", 2024-09-15T21:47:06.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9497999999999999e-05 2024-09-15T21:47:06.259 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.259 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.260 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.260 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:06.260 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:06.260 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.280212271000003, 2024-09-15T21:47:06.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007180306, 2024-09-15T21:47:06.260 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.260 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.260 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.260 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:06.260 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:06.260 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:06.261 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 137 2024-09-15T21:47:06.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.261 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:06.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:06.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:06.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1399999999999998e-07 2024-09-15T21:47:06.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897612+0000", 2024-09-15T21:47:06.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2607000000000004e-05 2024-09-15T21:47:06.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897815+0000", 2024-09-15T21:47:06.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020352400000000001 2024-09-15T21:47:06.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897821+0000", 2024-09-15T21:47:06.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4670000000000003e-06 2024-09-15T21:47:06.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900665+0000", 2024-09-15T21:47:06.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028443689999999998 2024-09-15T21:47:06.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900714+0000", 2024-09-15T21:47:06.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8812e-05 2024-09-15T21:47:06.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900799+0000", 2024-09-15T21:47:06.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5129000000000004e-05 2024-09-15T21:47:06.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903413+0000", 2024-09-15T21:47:06.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026142890000000001 2024-09-15T21:47:06.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904625+0000", 2024-09-15T21:47:06.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012121149999999999 2024-09-15T21:47:06.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904673+0000", 2024-09-15T21:47:06.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7540999999999998e-05 2024-09-15T21:47:06.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904709+0000", 2024-09-15T21:47:06.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6139000000000001e-05 2024-09-15T21:47:06.265 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.266 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.266 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.266 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:06.266 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:06.266 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.280126316, 2024-09-15T21:47:06.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071727450000000003, 2024-09-15T21:47:06.266 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.266 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.266 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.266 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:06.266 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:06.266 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:06.267 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 138 2024-09-15T21:47:06.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.267 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:06.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:06.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:06.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:47:06.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897619+0000", 2024-09-15T21:47:06.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3869999999999999e-06 2024-09-15T21:47:06.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.268 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.268 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897832+0000", 2024-09-15T21:47:06.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021329500000000001 2024-09-15T21:47:06.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897837+0000", 2024-09-15T21:47:06.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3730000000000001e-06 2024-09-15T21:47:06.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901391+0000", 2024-09-15T21:47:06.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003553601 2024-09-15T21:47:06.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901466+0000", 2024-09-15T21:47:06.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5130999999999996e-05 2024-09-15T21:47:06.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901601+0000", 2024-09-15T21:47:06.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134797 2024-09-15T21:47:06.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903417+0000", 2024-09-15T21:47:06.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001816156 2024-09-15T21:47:06.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904741+0000", 2024-09-15T21:47:06.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013241049999999999 2024-09-15T21:47:06.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904767+0000", 2024-09-15T21:47:06.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5876000000000001e-05 2024-09-15T21:47:06.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904788+0000", 2024-09-15T21:47:06.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0449999999999999e-05 2024-09-15T21:47:06.271 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.272 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.272 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.272 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:06.272 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:06.272 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.280120662999998, 2024-09-15T21:47:06.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072970250000000004, 2024-09-15T21:47:06.272 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.272 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.272 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.273 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:06.273 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:06.273 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:06.273 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 139 2024-09-15T21:47:06.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.273 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:06.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:06.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897621+0000", 2024-09-15T21:47:06.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3099999999999999e-07 2024-09-15T21:47:06.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897625+0000", 2024-09-15T21:47:06.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.698e-06 2024-09-15T21:47:06.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897848+0000", 2024-09-15T21:47:06.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022375699999999999 2024-09-15T21:47:06.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.275 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.275 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897853+0000", 2024-09-15T21:47:06.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7709999999999997e-06 2024-09-15T21:47:06.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901929+0000", 2024-09-15T21:47:06.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040757780000000004 2024-09-15T21:47:06.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901976+0000", 2024-09-15T21:47:06.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6751e-05 2024-09-15T21:47:06.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902054+0000", 2024-09-15T21:47:06.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8856999999999994e-05 2024-09-15T21:47:06.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903421+0000", 2024-09-15T21:47:06.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00136619 2024-09-15T21:47:06.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904827+0000", 2024-09-15T21:47:06.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0014067369999999999 2024-09-15T21:47:06.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904881+0000", 2024-09-15T21:47:06.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3520999999999998e-05 2024-09-15T21:47:06.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904918+0000", 2024-09-15T21:47:06.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6634000000000002e-05 2024-09-15T21:47:06.279 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.279 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.279 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.279 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:06.279 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:06.279 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.280114834000003, 2024-09-15T21:47:06.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097221730000000006, 2024-09-15T21:47:06.279 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.279 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.279 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.279 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:06.280 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:06.280 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:06.280 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 140 2024-09-15T21:47:06.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.280 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:06.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:06.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897627+0000", 2024-09-15T21:47:06.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-15T21:47:06.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897698+0000", 2024-09-15T21:47:06.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1722000000000002e-05 2024-09-15T21:47:06.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897863+0000", 2024-09-15T21:47:06.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016479099999999999 2024-09-15T21:47:06.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.282 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.282 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897868+0000", 2024-09-15T21:47:06.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4950000000000002e-06 2024-09-15T21:47:06.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902335+0000", 2024-09-15T21:47:06.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004467632 2024-09-15T21:47:06.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902414+0000", 2024-09-15T21:47:06.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8688000000000003e-05 2024-09-15T21:47:06.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902535+0000", 2024-09-15T21:47:06.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012066900000000001 2024-09-15T21:47:06.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903714+0000", 2024-09-15T21:47:06.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011794520000000001 2024-09-15T21:47:06.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907115+0000", 2024-09-15T21:47:06.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034008879999999999 2024-09-15T21:47:06.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907250+0000", 2024-09-15T21:47:06.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134657 2024-09-15T21:47:06.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907348+0000", 2024-09-15T21:47:06.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8838000000000005e-05 2024-09-15T21:47:06.286 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.286 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.286 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.286 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:06.286 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:06.286 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.280039752999997, 2024-09-15T21:47:06.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097618050000000001, 2024-09-15T21:47:06.287 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.287 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.287 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.287 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:06.287 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:06.287 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:06.287 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 141 2024-09-15T21:47:06.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.287 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:06.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:06.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.288 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897702+0000", 2024-09-15T21:47:06.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6199999999999998e-07 2024-09-15T21:47:06.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.288 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897705+0000", 2024-09-15T21:47:06.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0180000000000002e-06 2024-09-15T21:47:06.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897889+0000", 2024-09-15T21:47:06.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000183678 2024-09-15T21:47:06.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897894+0000", 2024-09-15T21:47:06.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8930000000000004e-06 2024-09-15T21:47:06.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903099+0000", 2024-09-15T21:47:06.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052056280000000003 2024-09-15T21:47:06.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903151+0000", 2024-09-15T21:47:06.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2163000000000002e-05 2024-09-15T21:47:06.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903220+0000", 2024-09-15T21:47:06.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8483000000000001e-05 2024-09-15T21:47:06.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904794+0000", 2024-09-15T21:47:06.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001574521 2024-09-15T21:47:06.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907405+0000", 2024-09-15T21:47:06.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026110339999999999 2024-09-15T21:47:06.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907440+0000", 2024-09-15T21:47:06.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4564999999999999e-05 2024-09-15T21:47:06.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907463+0000", 2024-09-15T21:47:06.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3260000000000001e-05 2024-09-15T21:47:06.292 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.292 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.292 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.292 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:22 2.8 2:1470b2f8:::613.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:06.292 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:06.293 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 31.722599425999999, 2024-09-15T21:47:06.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007454937, 2024-09-15T21:47:06.293 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.293 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.293 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.293 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:06.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:06.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.294 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.294 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:06.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6800000000000002e-07 2024-09-15T21:47:06.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455145+0000", 2024-09-15T21:47:06.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4470000000000002e-06 2024-09-15T21:47:06.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456425+0000", 2024-09-15T21:47:06.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012798200000000001 2024-09-15T21:47:06.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456433+0000", 2024-09-15T21:47:06.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7389999999999999e-06 2024-09-15T21:47:06.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:06.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097865 2024-09-15T21:47:06.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457614+0000", 2024-09-15T21:47:06.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3560000000000006e-05 2024-09-15T21:47:06.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457998+0000", 2024-09-15T21:47:06.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00038372699999999999 2024-09-15T21:47:06.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460488+0000", 2024-09-15T21:47:06.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002489855 2024-09-15T21:47:06.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462512+0000", 2024-09-15T21:47:06.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020235269999999998 2024-09-15T21:47:06.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462554+0000", 2024-09-15T21:47:06.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2472000000000003e-05 2024-09-15T21:47:06.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462597+0000", 2024-09-15T21:47:06.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2657000000000002e-05 2024-09-15T21:47:06.298 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.298 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.298 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.298 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:2 2.5 2:a6e36b3f:::401.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:06.299 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:06.299 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 31.722594457, 2024-09-15T21:47:06.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074484590000000002, 2024-09-15T21:47:06.299 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.299 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.299 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.299 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:06.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:06.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:06.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:47:06.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455150+0000", 2024-09-15T21:47:06.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7319999999999998e-06 2024-09-15T21:47:06.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456449+0000", 2024-09-15T21:47:06.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012991490000000001 2024-09-15T21:47:06.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456454+0000", 2024-09-15T21:47:06.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0390000000000004e-06 2024-09-15T21:47:06.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456513+0000", 2024-09-15T21:47:06.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8768999999999999e-05 2024-09-15T21:47:06.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456628+0000", 2024-09-15T21:47:06.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115641 2024-09-15T21:47:06.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456741+0000", 2024-09-15T21:47:06.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000113094 2024-09-15T21:47:06.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458520+0000", 2024-09-15T21:47:06.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001778993 2024-09-15T21:47:06.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462494+0000", 2024-09-15T21:47:06.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039737009999999996 2024-09-15T21:47:06.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462549+0000", 2024-09-15T21:47:06.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4537000000000003e-05 2024-09-15T21:47:06.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462595+0000", 2024-09-15T21:47:06.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6511000000000002e-05 2024-09-15T21:47:06.304 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.304 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.304 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.304 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:27 2.5 2:a5fa685f:::504.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:06.304 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:06.305 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 14.612902073000001, 2024-09-15T21:47:06.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071804599999999996, 2024-09-15T21:47:06.305 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:06.305 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:06.305 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.305 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:06.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:06.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:06.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.305 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:06.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:06.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:06.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:06.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:06.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-15T21:47:06.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:06.306 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564841+0000", 2024-09-15T21:47:06.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-15T21:47:06.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.306 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.306 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:06.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565126+0000", 2024-09-15T21:47:06.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028469900000000002 2024-09-15T21:47:06.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565133+0000", 2024-09-15T21:47:06.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9480000000000002e-06 2024-09-15T21:47:06.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.307 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:06.307 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568006+0000", 2024-09-15T21:47:06.307 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028730330000000001 2024-09-15T21:47:06.307 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:06.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568119+0000", 2024-09-15T21:47:06.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011325899999999999 2024-09-15T21:47:06.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:06.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568235+0000", 2024-09-15T21:47:06.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115868 2024-09-15T21:47:06.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:06.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569668+0000", 2024-09-15T21:47:06.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001433099 2024-09-15T21:47:06.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571926+0000", 2024-09-15T21:47:06.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022572410000000001 2024-09-15T21:47:06.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:06.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571981+0000", 2024-09-15T21:47:06.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5383000000000002e-05 2024-09-15T21:47:06.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:06.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:06.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:06.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572020+0000", 2024-09-15T21:47:06.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8574e-05 2024-09-15T21:47:06.310 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.310 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.310 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.310 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:06.310 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:06.310 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:06.311 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:47:06.312 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:06.310+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:06.412 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:06.411+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:06.419 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:06.420 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:06.420 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:06.420 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:06.430 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:47:06.514 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:06.513+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:06.573 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:06.573 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:06.573 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:06.573 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:06.573 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:06.582 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:47:06.614 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:06.613+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:06.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:06.625+0000 7f9dea7fc640 1 -- 172.21.15.100:0/774470838 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v26) ==== 97+0+0 (secure 0 0 0) 0x7f9de4058a30 con 0x7f9df40cca30 2024-09-15T21:47:06.627 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is set 2024-09-15T21:47:06.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:06.627+0000 7f9dfc21e640 1 -- 172.21.15.100:0/774470838 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9dd806c2b0 msgr2=0x7f9dd806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:06.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:06.627+0000 7f9dfc21e640 1 --2- 172.21.15.100:0/774470838 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9dd806c2b0 0x7f9dd806e770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9df0009b40 tx=0x7f9df0009190 comp rx=0 tx=0).stop 2024-09-15T21:47:06.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:06.627+0000 7f9dfc21e640 1 -- 172.21.15.100:0/774470838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9df40cca30 msgr2=0x7f9df40cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:06.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:06.627+0000 7f9dfc21e640 1 --2- 172.21.15.100:0/774470838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9df40cca30 0x7f9df40cb890 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9de400c400 tx=0x7f9de400c8c0 comp rx=0 tx=0).stop 2024-09-15T21:47:06.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:06.628+0000 7f9dfc21e640 1 -- 172.21.15.100:0/774470838 shutdown_connections 2024-09-15T21:47:06.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:06.628+0000 7f9dfc21e640 1 --2- 172.21.15.100:0/774470838 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9dd806c2b0 0x7f9dd806e770 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:06.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:06.628+0000 7f9dfc21e640 1 --2- 172.21.15.100:0/774470838 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9df40cce10 0x7f9df4156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:06.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:06.628+0000 7f9dfc21e640 1 --2- 172.21.15.100:0/774470838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9df40cca30 0x7f9df40cb890 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:06.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:06.628+0000 7f9dfc21e640 1 --2- 172.21.15.100:0/774470838 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9df4070ee0 0x7f9df41edb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:06.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:06.628+0000 7f9dfc21e640 1 -- 172.21.15.100:0/774470838 >> 172.21.15.100:0/774470838 conn(0x7f9df4067dd0 msgr2=0x7f9df406efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:06.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:06.628+0000 7f9dfc21e640 1 -- 172.21.15.100:0/774470838 shutdown_connections 2024-09-15T21:47:06.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:06.628+0000 7f9dfc21e640 1 -- 172.21.15.100:0/774470838 wait complete. 2024-09-15T21:47:06.716 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:06.714+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:06.728 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:06.728 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:06.728 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:06.728 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:06.728 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.728 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:32 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'18, mlcod=17'14)", 2024-09-15T21:47:06.728 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:06.728 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 52.368811211999997, 2024-09-15T21:47:06.728 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052297760000000002, 2024-09-15T21:47:06.729 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.729 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.729 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.729 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.729 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.729 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.729 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:06.729 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.729 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.729 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.729 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.729 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:06.729 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.729 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.730 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.730 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.730 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:06.730 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.06e-07 2024-09-15T21:47:06.730 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.730 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.730 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.730 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356780+0000", 2024-09-15T21:47:06.730 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4208999999999994e-05 2024-09-15T21:47:06.730 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.730 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.730 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.730 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356789+0000", 2024-09-15T21:47:06.731 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2800000000000003e-06 2024-09-15T21:47:06.731 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.731 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.731 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.731 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356793+0000", 2024-09-15T21:47:06.731 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:47:06.731 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.731 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.731 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.731 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356838+0000", 2024-09-15T21:47:06.731 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4851999999999997e-05 2024-09-15T21:47:06.731 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.731 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.731 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.732 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356854+0000", 2024-09-15T21:47:06.732 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6124999999999999e-05 2024-09-15T21:47:06.732 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.732 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.732 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:06.732 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.361892+0000", 2024-09-15T21:47:06.732 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050379140000000001 2024-09-15T21:47:06.732 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.732 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.732 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.733 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.361935+0000", 2024-09-15T21:47:06.733 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3217999999999999e-05 2024-09-15T21:47:06.733 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.733 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.733 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.733 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.733 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.733 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:35 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'21, mlcod=17'15)", 2024-09-15T21:47:06.733 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:06.733 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 52.366774081999999, 2024-09-15T21:47:06.733 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055652039999999998, 2024-09-15T21:47:06.733 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.733 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.734 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.734 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.734 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.734 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.734 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:06.734 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.734 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.734 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.734 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.734 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:06.734 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.734 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.734 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.734 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.735 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358744+0000", 2024-09-15T21:47:06.735 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4399999999999998e-07 2024-09-15T21:47:06.735 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.735 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.735 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.735 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358773+0000", 2024-09-15T21:47:06.735 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8847e-05 2024-09-15T21:47:06.735 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.735 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.735 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.735 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358783+0000", 2024-09-15T21:47:06.735 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0299000000000001e-05 2024-09-15T21:47:06.735 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.736 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.736 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.736 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358796+0000", 2024-09-15T21:47:06.736 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3499000000000001e-05 2024-09-15T21:47:06.736 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.736 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.736 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.736 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358850+0000", 2024-09-15T21:47:06.736 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3952000000000001e-05 2024-09-15T21:47:06.736 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.736 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.736 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.736 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358872+0000", 2024-09-15T21:47:06.737 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1424999999999999e-05 2024-09-15T21:47:06.737 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.737 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.737 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:06.737 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364273+0000", 2024-09-15T21:47:06.737 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054011299999999996 2024-09-15T21:47:06.737 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.737 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.737 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.737 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364308+0000", 2024-09-15T21:47:06.737 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5108000000000003e-05 2024-09-15T21:47:06.737 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.737 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.737 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.738 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.738 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.738 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:36 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'22, mlcod=17'15)", 2024-09-15T21:47:06.738 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:06.738 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 52.365979891999999, 2024-09-15T21:47:06.738 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047904530000000001, 2024-09-15T21:47:06.738 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.738 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.738 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.738 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.738 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.738 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.738 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:06.739 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.739 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.739 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.739 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.739 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:06.739 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.739 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.739 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.739 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.739 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:06.740 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6000000000000004e-07 2024-09-15T21:47:06.740 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.740 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.740 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.740 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359542+0000", 2024-09-15T21:47:06.740 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3510000000000004e-06 2024-09-15T21:47:06.740 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.740 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.740 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.740 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359552+0000", 2024-09-15T21:47:06.740 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0263e-05 2024-09-15T21:47:06.740 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.740 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.741 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.741 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359557+0000", 2024-09-15T21:47:06.741 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0479999999999998e-06 2024-09-15T21:47:06.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.741 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.741 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.741 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359593+0000", 2024-09-15T21:47:06.741 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6151e-05 2024-09-15T21:47:06.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.741 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.741 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.741 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359608+0000", 2024-09-15T21:47:06.741 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4609e-05 2024-09-15T21:47:06.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.742 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.742 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:06.742 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364310+0000", 2024-09-15T21:47:06.742 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047025510000000001 2024-09-15T21:47:06.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.742 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.742 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.742 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364327+0000", 2024-09-15T21:47:06.742 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6920000000000001e-05 2024-09-15T21:47:06.742 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.742 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.742 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.743 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.743 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=18'85)", 2024-09-15T21:47:06.743 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:06.743 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 50.827433956999997, 2024-09-15T21:47:06.743 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057443499999999996, 2024-09-15T21:47:06.743 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.743 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.743 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.743 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.743 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.744 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.744 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:06.744 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.744 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.744 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.744 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.744 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:06.744 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.744 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.744 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.744 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.744 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:06.744 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8899999999999999e-07 2024-09-15T21:47:06.745 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.745 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.745 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.745 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898167+0000", 2024-09-15T21:47:06.745 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3171999999999996e-05 2024-09-15T21:47:06.745 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.745 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.745 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.745 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898177+0000", 2024-09-15T21:47:06.745 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0866000000000001e-05 2024-09-15T21:47:06.746 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.746 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.746 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.746 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898185+0000", 2024-09-15T21:47:06.746 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2490000000000004e-06 2024-09-15T21:47:06.746 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.746 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.746 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.746 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898224+0000", 2024-09-15T21:47:06.746 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9237999999999999e-05 2024-09-15T21:47:06.746 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.746 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.746 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.746 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898238+0000", 2024-09-15T21:47:06.747 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.366e-05 2024-09-15T21:47:06.747 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.747 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.747 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:06.747 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.903803+0000", 2024-09-15T21:47:06.747 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005565291 2024-09-15T21:47:06.747 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.747 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.747 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.747 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.903827+0000", 2024-09-15T21:47:06.747 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4284999999999999e-05 2024-09-15T21:47:06.748 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.748 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.748 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.748 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.748 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.748 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:132 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'90, mlcod=18'85)", 2024-09-15T21:47:06.748 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:06.748 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 50.826942232999997, 2024-09-15T21:47:06.748 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055085469999999999, 2024-09-15T21:47:06.748 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.748 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.748 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.748 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.749 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.749 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.749 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:06.749 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.749 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.749 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.749 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.749 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:06.749 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.749 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.750 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.750 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.750 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:06.750 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9299999999999999e-07 2024-09-15T21:47:06.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.750 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.750 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.750 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898579+0000", 2024-09-15T21:47:06.750 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6650000000000001e-06 2024-09-15T21:47:06.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.750 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.750 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.750 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898586+0000", 2024-09-15T21:47:06.750 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8429999999999993e-06 2024-09-15T21:47:06.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.751 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.751 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.751 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898592+0000", 2024-09-15T21:47:06.751 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3990000000000003e-06 2024-09-15T21:47:06.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.751 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.751 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.751 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898621+0000", 2024-09-15T21:47:06.751 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9019000000000001e-05 2024-09-15T21:47:06.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.751 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.751 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.751 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898630+0000", 2024-09-15T21:47:06.752 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5540000000000003e-06 2024-09-15T21:47:06.752 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.752 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.752 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:06.752 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904047+0000", 2024-09-15T21:47:06.752 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054167169999999997 2024-09-15T21:47:06.752 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.752 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.752 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.752 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904083+0000", 2024-09-15T21:47:06.752 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5957000000000001e-05 2024-09-15T21:47:06.752 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.752 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.753 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.753 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.753 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.753 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-15T21:47:06.753 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:06.753 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 50.826353083999997, 2024-09-15T21:47:06.753 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049424780000000001, 2024-09-15T21:47:06.753 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.753 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.753 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.753 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.753 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.753 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.753 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:06.754 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.754 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.754 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.754 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:06.754 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.754 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.754 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.754 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899166+0000", 2024-09-15T21:47:06.754 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0779999999999998e-06 2024-09-15T21:47:06.755 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.755 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.755 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.755 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899174+0000", 2024-09-15T21:47:06.755 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.729e-06 2024-09-15T21:47:06.755 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.755 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.755 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.755 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899181+0000", 2024-09-15T21:47:06.755 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4080000000000001e-06 2024-09-15T21:47:06.756 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.756 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.756 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.756 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899185+0000", 2024-09-15T21:47:06.756 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0840000000000002e-06 2024-09-15T21:47:06.756 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.756 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.756 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.756 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899240+0000", 2024-09-15T21:47:06.756 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4874999999999999e-05 2024-09-15T21:47:06.757 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.757 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.757 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.757 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899257+0000", 2024-09-15T21:47:06.757 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7549000000000001e-05 2024-09-15T21:47:06.758 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.758 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.758 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:06.758 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904085+0000", 2024-09-15T21:47:06.758 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048272640000000004 2024-09-15T21:47:06.758 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.758 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.758 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.758 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904106+0000", 2024-09-15T21:47:06.758 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1491000000000001e-05 2024-09-15T21:47:06.758 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.759 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.759 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.759 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.759 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.759 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:06.759 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:06.759 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.312129808000002, 2024-09-15T21:47:06.759 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046608459999999997, 2024-09-15T21:47:06.759 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.759 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.759 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.759 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.759 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.760 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.760 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:06.760 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.760 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.760 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.760 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.760 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:06.760 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.760 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.760 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.760 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.760 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413391+0000", 2024-09-15T21:47:06.760 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9899999999999999e-06 2024-09-15T21:47:06.761 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.761 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.761 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.761 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413399+0000", 2024-09-15T21:47:06.761 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6310000000000003e-06 2024-09-15T21:47:06.761 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.761 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.761 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.761 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413444+0000", 2024-09-15T21:47:06.761 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5052000000000002e-05 2024-09-15T21:47:06.761 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.761 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.761 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.762 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413453+0000", 2024-09-15T21:47:06.762 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8869999999999998e-06 2024-09-15T21:47:06.762 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.762 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.762 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.762 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413576+0000", 2024-09-15T21:47:06.762 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012358700000000001 2024-09-15T21:47:06.762 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.762 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.762 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.762 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413679+0000", 2024-09-15T21:47:06.763 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000103318 2024-09-15T21:47:06.763 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.763 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.763 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:06.763 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413785+0000", 2024-09-15T21:47:06.763 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010561 2024-09-15T21:47:06.763 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.763 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.763 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:06.763 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416276+0000", 2024-09-15T21:47:06.763 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002490754 2024-09-15T21:47:06.763 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.763 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.763 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.764 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417949+0000", 2024-09-15T21:47:06.764 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016733519999999999 2024-09-15T21:47:06.764 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.764 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.764 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:06.764 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418005+0000", 2024-09-15T21:47:06.764 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5747e-05 2024-09-15T21:47:06.764 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.764 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.764 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.764 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418048+0000", 2024-09-15T21:47:06.764 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2917999999999998e-05 2024-09-15T21:47:06.765 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.765 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.765 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.765 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.765 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.765 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:06.765 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:06.765 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.312115337999998, 2024-09-15T21:47:06.765 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051419999999999999, 2024-09-15T21:47:06.765 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.765 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.765 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.765 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.765 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.766 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.766 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:06.766 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.766 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.766 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.766 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.766 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:06.766 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.766 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.766 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.766 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.766 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413402+0000", 2024-09-15T21:47:06.766 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6899999999999998e-07 2024-09-15T21:47:06.766 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.766 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.767 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.767 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413405+0000", 2024-09-15T21:47:06.767 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8109999999999999e-06 2024-09-15T21:47:06.767 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.767 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.767 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.767 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413517+0000", 2024-09-15T21:47:06.767 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000112592 2024-09-15T21:47:06.767 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.767 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.767 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.767 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413527+0000", 2024-09-15T21:47:06.767 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0144e-05 2024-09-15T21:47:06.767 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.768 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.768 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.768 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414155+0000", 2024-09-15T21:47:06.768 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00062767200000000002 2024-09-15T21:47:06.768 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.768 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.768 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.768 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414239+0000", 2024-09-15T21:47:06.768 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3607000000000001e-05 2024-09-15T21:47:06.768 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.768 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.769 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:06.769 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414368+0000", 2024-09-15T21:47:06.769 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000129049 2024-09-15T21:47:06.769 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.769 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.769 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:06.769 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416332+0000", 2024-09-15T21:47:06.769 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019645109999999999 2024-09-15T21:47:06.769 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.769 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.770 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.770 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418446+0000", 2024-09-15T21:47:06.770 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021137640000000002 2024-09-15T21:47:06.770 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.770 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.770 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:06.770 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418503+0000", 2024-09-15T21:47:06.770 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7166999999999997e-05 2024-09-15T21:47:06.770 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.770 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.770 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.770 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418543+0000", 2024-09-15T21:47:06.770 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0314000000000001e-05 2024-09-15T21:47:06.770 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.771 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.771 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.771 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.771 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.771 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:06.771 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:06.771 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.312102146999997, 2024-09-15T21:47:06.771 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057945979999999998, 2024-09-15T21:47:06.771 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.771 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.771 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.771 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.772 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.772 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.772 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:06.772 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.772 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.772 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.772 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.772 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:06.772 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.772 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.772 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.773 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.773 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:06.773 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9499999999999998e-07 2024-09-15T21:47:06.773 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.773 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.773 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.773 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413417+0000", 2024-09-15T21:47:06.773 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3910000000000001e-06 2024-09-15T21:47:06.773 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.773 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.773 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.773 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414570+0000", 2024-09-15T21:47:06.773 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011525890000000001 2024-09-15T21:47:06.773 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.774 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.774 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.774 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414577+0000", 2024-09-15T21:47:06.774 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1840000000000002e-06 2024-09-15T21:47:06.774 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.774 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.774 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.774 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414733+0000", 2024-09-15T21:47:06.774 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015563900000000001 2024-09-15T21:47:06.774 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.774 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.774 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.774 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414831+0000", 2024-09-15T21:47:06.774 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8138000000000001e-05 2024-09-15T21:47:06.775 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.775 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.775 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:06.775 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414956+0000", 2024-09-15T21:47:06.775 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000124604 2024-09-15T21:47:06.775 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.775 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.775 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.775 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418804+0000", 2024-09-15T21:47:06.775 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038484299999999999 2024-09-15T21:47:06.776 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.776 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.776 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:06.776 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419124+0000", 2024-09-15T21:47:06.776 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00032017699999999999 2024-09-15T21:47:06.776 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.776 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.776 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:06.776 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419171+0000", 2024-09-15T21:47:06.776 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6520000000000002e-05 2024-09-15T21:47:06.776 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.776 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.776 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.777 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419209+0000", 2024-09-15T21:47:06.777 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8531000000000001e-05 2024-09-15T21:47:06.777 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.777 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.777 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.777 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.777 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.777 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:06.777 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:06.777 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.312095051, 2024-09-15T21:47:06.777 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059183300000000003, 2024-09-15T21:47:06.777 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.777 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.778 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:06.778 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.778 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.778 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.778 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:06.778 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.778 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.778 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.778 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.778 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:06.778 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.778 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.778 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.778 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.779 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:06.779 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3000000000000002e-07 2024-09-15T21:47:06.779 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.779 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.779 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.779 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413426+0000", 2024-09-15T21:47:06.779 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7110000000000001e-06 2024-09-15T21:47:06.779 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.779 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.779 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.779 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414621+0000", 2024-09-15T21:47:06.779 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001195022 2024-09-15T21:47:06.779 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.780 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.780 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.780 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414625+0000", 2024-09-15T21:47:06.780 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6360000000000002e-06 2024-09-15T21:47:06.780 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.780 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.780 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.780 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415269+0000", 2024-09-15T21:47:06.780 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00064315399999999995 2024-09-15T21:47:06.780 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.780 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.780 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.781 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415356+0000", 2024-09-15T21:47:06.781 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7410999999999994e-05 2024-09-15T21:47:06.781 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.781 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.781 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.781 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419340+0000", 2024-09-15T21:47:06.781 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0039840659999999996 2024-09-15T21:47:06.781 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.781 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.782 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.782 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.782 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.782 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-15T21:47:06.782 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:06.782 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.311787459999998, 2024-09-15T21:47:06.782 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053379719999999999, 2024-09-15T21:47:06.782 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.782 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.782 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.782 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.782 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.782 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.783 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:06.783 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.783 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.783 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.783 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.783 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:06.783 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.783 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.783 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.783 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.783 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413730+0000", 2024-09-15T21:47:06.783 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1379999999999999e-06 2024-09-15T21:47:06.783 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.783 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.784 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.784 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413740+0000", 2024-09-15T21:47:06.784 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4830000000000005e-06 2024-09-15T21:47:06.784 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.784 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.784 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.784 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413751+0000", 2024-09-15T21:47:06.784 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1228e-05 2024-09-15T21:47:06.784 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.784 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.784 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.784 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413758+0000", 2024-09-15T21:47:06.784 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1289999999999998e-06 2024-09-15T21:47:06.784 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.785 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.785 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.785 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414038+0000", 2024-09-15T21:47:06.785 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00027935999999999998 2024-09-15T21:47:06.785 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.785 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.785 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.785 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414055+0000", 2024-09-15T21:47:06.785 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7325000000000001e-05 2024-09-15T21:47:06.785 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.785 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.785 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:06.785 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419028+0000", 2024-09-15T21:47:06.785 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049730720000000003 2024-09-15T21:47:06.786 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.786 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.786 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.786 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419067+0000", 2024-09-15T21:47:06.786 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9236999999999997e-05 2024-09-15T21:47:06.786 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.786 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.786 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.786 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.786 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.786 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:06.786 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:06.786 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 32.269864192999997, 2024-09-15T21:47:06.786 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.92695576499999999, 2024-09-15T21:47:06.787 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.787 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.787 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.787 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.787 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.787 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.787 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:06.787 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.787 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.787 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.787 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.787 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:06.787 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.787 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.788 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.788 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.788 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:06.788 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8500000000000002e-07 2024-09-15T21:47:06.788 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.788 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.788 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.788 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455655+0000", 2024-09-15T21:47:06.788 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.401e-06 2024-09-15T21:47:06.788 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.788 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.788 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.788 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.380965+0000", 2024-09-15T21:47:06.789 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.92530946000000003 2024-09-15T21:47:06.789 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.789 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.789 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.789 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.380981+0000", 2024-09-15T21:47:06.789 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6209e-05 2024-09-15T21:47:06.789 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.789 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.789 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.789 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381086+0000", 2024-09-15T21:47:06.789 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105011 2024-09-15T21:47:06.789 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.789 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.789 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381154+0000", 2024-09-15T21:47:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7596999999999997e-05 2024-09-15T21:47:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381249+0000", 2024-09-15T21:47:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5354999999999998e-05 2024-09-15T21:47:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381844+0000", 2024-09-15T21:47:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00059518600000000004 2024-09-15T21:47:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.790 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382531+0000", 2024-09-15T21:47:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00068671499999999996 2024-09-15T21:47:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382570+0000", 2024-09-15T21:47:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9373999999999999e-05 2024-09-15T21:47:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382608+0000", 2024-09-15T21:47:06.791 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8071999999999997e-05 2024-09-15T21:47:06.792 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.792 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.792 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.792 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.792 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.792 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:06.792 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:06.792 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 26.249954934000002, 2024-09-15T21:47:06.792 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047117319999999997, 2024-09-15T21:47:06.792 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.792 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.792 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.792 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.792 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.793 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.793 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:06.793 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.793 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.793 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.793 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.793 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:06.793 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.793 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.793 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.793 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.793 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:06.793 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4700000000000001e-07 2024-09-15T21:47:06.794 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.794 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.794 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.794 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475566+0000", 2024-09-15T21:47:06.794 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1669999999999999e-06 2024-09-15T21:47:06.794 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.794 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.794 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.794 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476316+0000", 2024-09-15T21:47:06.794 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075022999999999995 2024-09-15T21:47:06.794 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.794 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.794 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.794 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476324+0000", 2024-09-15T21:47:06.795 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.259e-06 2024-09-15T21:47:06.795 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.795 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.795 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.795 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476373+0000", 2024-09-15T21:47:06.795 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8807999999999998e-05 2024-09-15T21:47:06.795 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.795 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.795 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.795 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476446+0000", 2024-09-15T21:47:06.795 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3138000000000003e-05 2024-09-15T21:47:06.795 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.795 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.795 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:06.796 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476593+0000", 2024-09-15T21:47:06.796 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00014680199999999999 2024-09-15T21:47:06.796 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.796 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.796 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.796 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480189+0000", 2024-09-15T21:47:06.796 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035961370000000001 2024-09-15T21:47:06.796 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.796 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.796 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:06.796 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480198+0000", 2024-09-15T21:47:06.796 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.6649999999999992e-06 2024-09-15T21:47:06.796 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.796 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.797 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:06.797 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480235+0000", 2024-09-15T21:47:06.797 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6943999999999997e-05 2024-09-15T21:47:06.797 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.797 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.797 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.797 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480274+0000", 2024-09-15T21:47:06.797 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9035000000000001e-05 2024-09-15T21:47:06.797 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.797 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.797 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.797 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.797 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.797 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:06.798 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:06.798 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 26.249949931, 2024-09-15T21:47:06.798 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046830109999999999, 2024-09-15T21:47:06.798 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.798 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.798 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.798 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.798 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.798 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.798 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:06.798 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.798 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.798 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.798 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.799 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:06.799 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.799 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.799 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.799 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.799 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:06.799 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.53e-07 2024-09-15T21:47:06.799 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.799 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.799 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.799 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475570+0000", 2024-09-15T21:47:06.799 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7470000000000001e-06 2024-09-15T21:47:06.799 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.800 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.800 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.800 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476353+0000", 2024-09-15T21:47:06.800 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00078312300000000004 2024-09-15T21:47:06.800 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.800 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.800 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.800 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476359+0000", 2024-09-15T21:47:06.800 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.117e-06 2024-09-15T21:47:06.800 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.800 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.800 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.800 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476412+0000", 2024-09-15T21:47:06.800 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2398999999999998e-05 2024-09-15T21:47:06.801 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.801 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.801 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.801 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476524+0000", 2024-09-15T21:47:06.801 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011293 2024-09-15T21:47:06.801 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.801 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.801 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:06.801 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476909+0000", 2024-09-15T21:47:06.801 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00038490099999999999 2024-09-15T21:47:06.801 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.801 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.801 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.802 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479761+0000", 2024-09-15T21:47:06.802 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002851737 2024-09-15T21:47:06.802 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.802 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.802 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:06.802 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480154+0000", 2024-09-15T21:47:06.802 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00039307399999999998 2024-09-15T21:47:06.802 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.802 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.802 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:06.803 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480203+0000", 2024-09-15T21:47:06.803 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8418000000000003e-05 2024-09-15T21:47:06.803 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.803 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.803 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.803 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480250+0000", 2024-09-15T21:47:06.803 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7311999999999997e-05 2024-09-15T21:47:06.803 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.803 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.803 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.803 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.803 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.803 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:06.804 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:06.804 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 26.249945769, 2024-09-15T21:47:06.804 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005035398, 2024-09-15T21:47:06.804 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.804 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.804 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.804 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.804 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.804 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.804 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:06.804 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.804 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.805 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.805 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.805 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:06.805 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.805 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.805 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.805 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.805 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:06.805 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.22e-07 2024-09-15T21:47:06.805 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.805 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.805 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.805 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475574+0000", 2024-09-15T21:47:06.805 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.593e-06 2024-09-15T21:47:06.806 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.806 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.806 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.806 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476579+0000", 2024-09-15T21:47:06.806 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010053989999999999 2024-09-15T21:47:06.806 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.806 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.806 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.806 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476588+0000", 2024-09-15T21:47:06.806 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4530000000000002e-06 2024-09-15T21:47:06.806 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.806 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.806 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.806 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477723+0000", 2024-09-15T21:47:06.807 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001134962 2024-09-15T21:47:06.807 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.807 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.807 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.807 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477801+0000", 2024-09-15T21:47:06.807 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8228999999999999e-05 2024-09-15T21:47:06.807 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.807 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.807 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:06.807 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477946+0000", 2024-09-15T21:47:06.807 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00014553400000000001 2024-09-15T21:47:06.807 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.807 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.808 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:06.808 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480258+0000", 2024-09-15T21:47:06.808 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023112039999999999 2024-09-15T21:47:06.808 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.808 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.808 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.808 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480504+0000", 2024-09-15T21:47:06.808 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024667500000000002 2024-09-15T21:47:06.808 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.808 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.808 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:06.808 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480556+0000", 2024-09-15T21:47:06.808 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2114999999999998e-05 2024-09-15T21:47:06.808 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.809 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.809 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.809 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480606+0000", 2024-09-15T21:47:06.809 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0012000000000002e-05 2024-09-15T21:47:06.809 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.809 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.809 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.809 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.809 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.809 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:06.809 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:06.809 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 26.249941839000002, 2024-09-15T21:47:06.809 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050672490000000002, 2024-09-15T21:47:06.810 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.810 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.810 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.810 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.810 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.810 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.810 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:06.810 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.810 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.810 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.810 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.810 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:06.810 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.810 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.811 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.811 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.811 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:06.811 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:06.811 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.811 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.811 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.811 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475577+0000", 2024-09-15T21:47:06.811 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2290000000000002e-06 2024-09-15T21:47:06.811 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.811 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.811 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.811 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476605+0000", 2024-09-15T21:47:06.811 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010278780000000001 2024-09-15T21:47:06.812 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.812 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.812 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.812 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476611+0000", 2024-09-15T21:47:06.812 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5600000000000001e-06 2024-09-15T21:47:06.812 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.812 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.812 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.812 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477053+0000", 2024-09-15T21:47:06.812 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044162800000000001 2024-09-15T21:47:06.812 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.812 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.812 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.812 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477180+0000", 2024-09-15T21:47:06.813 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000127129 2024-09-15T21:47:06.813 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.813 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.813 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:06.813 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477288+0000", 2024-09-15T21:47:06.813 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010789999999999999 2024-09-15T21:47:06.813 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.813 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.813 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:06.813 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480288+0000", 2024-09-15T21:47:06.813 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003000052 2024-09-15T21:47:06.813 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.813 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.813 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.814 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480568+0000", 2024-09-15T21:47:06.814 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028026899999999998 2024-09-15T21:47:06.814 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.814 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.814 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:06.814 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480609+0000", 2024-09-15T21:47:06.814 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1437999999999999e-05 2024-09-15T21:47:06.814 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.814 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.814 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.814 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480642+0000", 2024-09-15T21:47:06.814 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2879999999999997e-05 2024-09-15T21:47:06.814 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.814 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.815 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.815 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.815 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.815 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:20 2.1f 2:fdd45aa0:::61c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:06.815 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:06.815 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 26.249938168, 2024-09-15T21:47:06.815 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053112589999999996, 2024-09-15T21:47:06.815 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.815 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.815 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.816 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.816 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.816 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.816 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:06.816 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.816 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.816 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.816 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.816 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:06.817 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.817 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.817 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.817 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.817 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:06.817 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:06.817 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.817 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.817 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.817 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475581+0000", 2024-09-15T21:47:06.817 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1009999999999999e-06 2024-09-15T21:47:06.817 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.817 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.818 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.818 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476628+0000", 2024-09-15T21:47:06.818 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010465380000000001 2024-09-15T21:47:06.818 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.818 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.818 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.818 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476633+0000", 2024-09-15T21:47:06.818 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9869999999999999e-06 2024-09-15T21:47:06.818 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.818 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.818 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.818 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477618+0000", 2024-09-15T21:47:06.818 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00098539299999999999 2024-09-15T21:47:06.818 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.819 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.819 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477696+0000", 2024-09-15T21:47:06.819 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8509000000000003e-05 2024-09-15T21:47:06.819 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.819 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:06.819 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477824+0000", 2024-09-15T21:47:06.819 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000127062 2024-09-15T21:47:06.819 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.819 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:06.819 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480294+0000", 2024-09-15T21:47:06.820 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024703939999999999 2024-09-15T21:47:06.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.820 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.820 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.820 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480788+0000", 2024-09-15T21:47:06.820 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00049443800000000004 2024-09-15T21:47:06.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.820 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.820 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:06.820 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480845+0000", 2024-09-15T21:47:06.820 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6762000000000003e-05 2024-09-15T21:47:06.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.820 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.821 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.821 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480890+0000", 2024-09-15T21:47:06.821 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4789000000000001e-05 2024-09-15T21:47:06.821 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.821 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.821 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.821 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.821 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.821 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:06.821 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:06.821 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 26.249934697, 2024-09-15T21:47:06.821 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007191546, 2024-09-15T21:47:06.821 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.821 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.822 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:06.822 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.822 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.822 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:06.822 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.822 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.822 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:06.822 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.823 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.823 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:06.823 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9999999999999999e-07 2024-09-15T21:47:06.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.823 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.823 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.823 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475585+0000", 2024-09-15T21:47:06.823 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.384e-06 2024-09-15T21:47:06.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.823 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.823 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.823 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476665+0000", 2024-09-15T21:47:06.823 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010797140000000001 2024-09-15T21:47:06.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.824 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.824 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.824 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476671+0000", 2024-09-15T21:47:06.824 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3940000000000001e-06 2024-09-15T21:47:06.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.824 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.824 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.824 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478048+0000", 2024-09-15T21:47:06.824 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013767530000000001 2024-09-15T21:47:06.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.824 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.824 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.824 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478177+0000", 2024-09-15T21:47:06.824 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012898099999999999 2024-09-15T21:47:06.825 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.825 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.825 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.825 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482774+0000", 2024-09-15T21:47:06.825 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045970200000000003 2024-09-15T21:47:06.825 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.825 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.825 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.825 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.825 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.825 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:06.825 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:06.825 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 21.180705031999999, 2024-09-15T21:47:06.826 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055444480000000004, 2024-09-15T21:47:06.826 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.826 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.826 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.826 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.826 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.826 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.826 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:06.826 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.826 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.826 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.826 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.826 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:06.827 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.827 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.827 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:06.827 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4200000000000002e-07 2024-09-15T21:47:06.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.827 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.827 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544814+0000", 2024-09-15T21:47:06.827 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9980000000000002e-06 2024-09-15T21:47:06.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.827 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.828 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544872+0000", 2024-09-15T21:47:06.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8405000000000001e-05 2024-09-15T21:47:06.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.828 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.828 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.828 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544877+0000", 2024-09-15T21:47:06.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.143e-06 2024-09-15T21:47:06.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.828 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.828 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.828 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545750+0000", 2024-09-15T21:47:06.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00087310000000000003 2024-09-15T21:47:06.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.829 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.829 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.829 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545850+0000", 2024-09-15T21:47:06.829 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9939999999999995e-05 2024-09-15T21:47:06.829 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.829 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.829 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:06.829 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546014+0000", 2024-09-15T21:47:06.829 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00016408500000000001 2024-09-15T21:47:06.829 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.829 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.829 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.829 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.548748+0000", 2024-09-15T21:47:06.829 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027339539999999998 2024-09-15T21:47:06.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.830 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:06.830 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550265+0000", 2024-09-15T21:47:06.830 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015170019999999999 2024-09-15T21:47:06.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.830 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:06.830 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550315+0000", 2024-09-15T21:47:06.830 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9867999999999998e-05 2024-09-15T21:47:06.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.830 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.830 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550356+0000", 2024-09-15T21:47:06.831 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1711000000000001e-05 2024-09-15T21:47:06.831 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.831 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.831 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.831 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.831 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.831 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:06.831 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:06.831 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 21.180701831, 2024-09-15T21:47:06.831 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058729619999999998, 2024-09-15T21:47:06.831 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:06.831 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:06.831 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:06.831 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:06.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:06.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:06.832 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:06.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:06.832 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:06.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.833 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:06.833 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:06.833 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0599999999999999e-07 2024-09-15T21:47:06.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.833 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:06.833 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544821+0000", 2024-09-15T21:47:06.833 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4079999999999997e-06 2024-09-15T21:47:06.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.833 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:06.833 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544939+0000", 2024-09-15T21:47:06.833 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118532 2024-09-15T21:47:06.836 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.836 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.837 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:06.837 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544946+0000", 2024-09-15T21:47:06.837 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9260000000000003e-06 2024-09-15T21:47:06.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.837 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:06.837 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545011+0000", 2024-09-15T21:47:06.837 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4572000000000005e-05 2024-09-15T21:47:06.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.837 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:06.837 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545134+0000", 2024-09-15T21:47:06.837 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012309899999999999 2024-09-15T21:47:06.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:06.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545271+0000", 2024-09-15T21:47:06.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013746599999999999 2024-09-15T21:47:06.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:06.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550243+0000", 2024-09-15T21:47:06.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049714529999999998 2024-09-15T21:47:06.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:06.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550605+0000", 2024-09-15T21:47:06.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00036260699999999998 2024-09-15T21:47:06.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.839 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:06.839 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550662+0000", 2024-09-15T21:47:06.839 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6573000000000002e-05 2024-09-15T21:47:06.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:06.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:06.839 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:06.839 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550688+0000", 2024-09-15T21:47:06.839 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6120000000000001e-05 2024-09-15T21:47:06.839 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.840 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.840 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.840 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:06.840 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:06.840 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:06.841 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:47:06.842 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:06.840+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:06.943 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:06.941+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:06.985 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:06.985 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:06.985 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:06.985 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:06.994 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:47:07.044 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:07.042+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:07.096 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:07.096 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:07.096 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:07.096 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:07.097 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:07.104 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:47:07.132 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:47:07.149 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:07.147+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:07.211 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:07.212 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:07.212 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:07.212 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:07.212 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.212 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:07.212 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:07.212 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.795832930000003, 2024-09-15T21:47:07.212 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006101291, 2024-09-15T21:47:07.212 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.212 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.212 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:07.213 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.213 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.213 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.213 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:07.213 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.213 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.213 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.213 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.213 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:07.213 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.213 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.213 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.213 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.213 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413116+0000", 2024-09-15T21:47:07.214 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9700000000000003e-07 2024-09-15T21:47:07.214 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.214 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.214 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.214 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413118+0000", 2024-09-15T21:47:07.214 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2910000000000002e-06 2024-09-15T21:47:07.214 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.214 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.214 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.214 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413220+0000", 2024-09-15T21:47:07.214 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102056 2024-09-15T21:47:07.214 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.214 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.214 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.215 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413225+0000", 2024-09-15T21:47:07.215 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0490000000000003e-06 2024-09-15T21:47:07.215 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.215 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.215 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.215 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414808+0000", 2024-09-15T21:47:07.215 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015826900000000001 2024-09-15T21:47:07.215 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.215 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.215 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.215 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414907+0000", 2024-09-15T21:47:07.215 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9600999999999997e-05 2024-09-15T21:47:07.215 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.215 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.216 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:07.216 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415054+0000", 2024-09-15T21:47:07.216 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000146532 2024-09-15T21:47:07.216 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.216 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.216 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:07.216 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418056+0000", 2024-09-15T21:47:07.216 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030020519999999998 2024-09-15T21:47:07.216 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.216 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.216 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:07.216 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419128+0000", 2024-09-15T21:47:07.217 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001071671 2024-09-15T21:47:07.217 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.217 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.217 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:07.217 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419184+0000", 2024-09-15T21:47:07.217 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6125000000000002e-05 2024-09-15T21:47:07.217 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.217 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.217 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.217 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419217+0000", 2024-09-15T21:47:07.217 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2926999999999998e-05 2024-09-15T21:47:07.217 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.217 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.218 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.218 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.218 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.218 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:07.218 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:07.218 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.795829404000003, 2024-09-15T21:47:07.218 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059975130000000003, 2024-09-15T21:47:07.218 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.218 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.219 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:07.219 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.219 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.219 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.219 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:07.220 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.220 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.220 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.220 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.220 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:07.220 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.221 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.221 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.221 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.221 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:07.221 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6199999999999999e-07 2024-09-15T21:47:07.221 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.221 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.221 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.221 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413121+0000", 2024-09-15T21:47:07.221 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2000000000000001e-06 2024-09-15T21:47:07.221 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.221 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.221 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.221 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413292+0000", 2024-09-15T21:47:07.221 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017034400000000001 2024-09-15T21:47:07.222 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.222 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.222 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.222 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413300+0000", 2024-09-15T21:47:07.222 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8539999999999997e-06 2024-09-15T21:47:07.222 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.222 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.222 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.222 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414299+0000", 2024-09-15T21:47:07.222 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0009992829999999999 2024-09-15T21:47:07.222 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.222 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.222 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.223 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414390+0000", 2024-09-15T21:47:07.223 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1237999999999996e-05 2024-09-15T21:47:07.223 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.223 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.223 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:07.223 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414490+0000", 2024-09-15T21:47:07.223 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9438999999999994e-05 2024-09-15T21:47:07.223 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.223 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.223 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:07.223 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417393+0000", 2024-09-15T21:47:07.223 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002903306 2024-09-15T21:47:07.223 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.224 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.224 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:07.224 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419023+0000", 2024-09-15T21:47:07.224 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001630292 2024-09-15T21:47:07.224 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.224 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.224 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:07.224 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419067+0000", 2024-09-15T21:47:07.224 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4017999999999998e-05 2024-09-15T21:47:07.224 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.224 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.224 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.224 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419116+0000", 2024-09-15T21:47:07.224 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9277000000000003e-05 2024-09-15T21:47:07.225 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.225 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.225 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.225 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.225 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.225 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:07.225 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:07.225 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.793131207000002, 2024-09-15T21:47:07.225 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060879280000000003, 2024-09-15T21:47:07.225 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.225 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.225 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:07.225 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.225 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.226 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.226 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:07.226 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.226 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.226 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.226 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.226 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:07.226 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.226 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.226 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.226 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.226 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415818+0000", 2024-09-15T21:47:07.226 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:47:07.226 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.227 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.227 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.227 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415854+0000", 2024-09-15T21:47:07.227 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6706999999999999e-05 2024-09-15T21:47:07.227 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.227 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.227 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.227 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415860+0000", 2024-09-15T21:47:07.227 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0789999999999997e-06 2024-09-15T21:47:07.227 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.227 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.227 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.227 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415865+0000", 2024-09-15T21:47:07.228 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4580000000000003e-06 2024-09-15T21:47:07.228 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.228 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.228 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.228 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416092+0000", 2024-09-15T21:47:07.228 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00022739400000000001 2024-09-15T21:47:07.228 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.228 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.228 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.228 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416108+0000", 2024-09-15T21:47:07.228 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.613e-05 2024-09-15T21:47:07.228 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.228 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.228 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:07.229 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421866+0000", 2024-09-15T21:47:07.229 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005757585 2024-09-15T21:47:07.229 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.229 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.229 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.229 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421905+0000", 2024-09-15T21:47:07.229 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9214000000000001e-05 2024-09-15T21:47:07.229 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.229 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.229 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.229 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.230 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.230 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-15T21:47:07.230 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:07.230 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.793071046999998, 2024-09-15T21:47:07.230 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060637570000000003, 2024-09-15T21:47:07.230 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.230 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.230 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:07.230 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.230 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.230 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.230 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:07.230 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.230 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.231 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.231 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.231 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:07.231 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.231 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.231 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.231 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.231 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415878+0000", 2024-09-15T21:47:07.231 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.34e-07 2024-09-15T21:47:07.231 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.231 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.231 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.231 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415881+0000", 2024-09-15T21:47:07.231 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.354e-06 2024-09-15T21:47:07.232 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.232 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.232 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.232 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415887+0000", 2024-09-15T21:47:07.232 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:47:07.232 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.232 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.232 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.232 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415891+0000", 2024-09-15T21:47:07.232 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2690000000000001e-06 2024-09-15T21:47:07.232 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.232 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.232 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.232 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416267+0000", 2024-09-15T21:47:07.233 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00037614799999999998 2024-09-15T21:47:07.233 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.233 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.233 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.233 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416288+0000", 2024-09-15T21:47:07.236 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0604000000000001e-05 2024-09-15T21:47:07.236 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.236 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.236 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:07.236 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421908+0000", 2024-09-15T21:47:07.236 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056203390000000002 2024-09-15T21:47:07.237 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.237 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.237 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.237 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421941+0000", 2024-09-15T21:47:07.237 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3079e-05 2024-09-15T21:47:07.237 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.237 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.237 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.237 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.237 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.237 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:07.237 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:07.237 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.793052619999997, 2024-09-15T21:47:07.237 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006080284, 2024-09-15T21:47:07.238 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.238 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.238 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:07.238 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.238 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.238 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.238 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:07.238 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.238 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.238 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.238 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.238 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:07.238 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.238 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.239 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.239 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.239 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:07.239 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6899999999999999e-07 2024-09-15T21:47:07.239 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.239 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.239 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.239 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415898+0000", 2024-09-15T21:47:07.239 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.951e-06 2024-09-15T21:47:07.239 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.239 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.239 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.239 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415920+0000", 2024-09-15T21:47:07.239 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1651999999999999e-05 2024-09-15T21:47:07.239 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.240 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.240 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.240 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415925+0000", 2024-09-15T21:47:07.240 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.463e-06 2024-09-15T21:47:07.240 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.240 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.240 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.240 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416449+0000", 2024-09-15T21:47:07.240 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052384000000000003 2024-09-15T21:47:07.240 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.240 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.240 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.240 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416466+0000", 2024-09-15T21:47:07.240 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7108999999999998e-05 2024-09-15T21:47:07.241 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.241 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.241 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:07.241 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421943+0000", 2024-09-15T21:47:07.241 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054771569999999999 2024-09-15T21:47:07.241 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.241 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.241 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.241 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421976+0000", 2024-09-15T21:47:07.241 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2943e-05 2024-09-15T21:47:07.241 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.241 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.241 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.241 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.242 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.242 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-15T21:47:07.242 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:07.242 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.793013383999998, 2024-09-15T21:47:07.242 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060643340000000002, 2024-09-15T21:47:07.242 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.242 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.242 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:07.242 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.242 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.242 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.243 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:07.243 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.243 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.243 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.243 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.243 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:07.243 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.243 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.243 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.243 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.243 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:07.243 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4200000000000002e-07 2024-09-15T21:47:07.243 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.244 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.244 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.244 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415939+0000", 2024-09-15T21:47:07.244 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.439e-06 2024-09-15T21:47:07.244 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.244 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.244 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.244 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415945+0000", 2024-09-15T21:47:07.244 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7740000000000001e-06 2024-09-15T21:47:07.244 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.245 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.245 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.245 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415948+0000", 2024-09-15T21:47:07.245 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.737e-06 2024-09-15T21:47:07.245 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.245 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.245 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.246 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416605+0000", 2024-09-15T21:47:07.246 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00065721699999999998 2024-09-15T21:47:07.246 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.246 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.246 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.246 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416620+0000", 2024-09-15T21:47:07.246 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4564e-05 2024-09-15T21:47:07.246 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.246 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.246 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:07.246 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421977+0000", 2024-09-15T21:47:07.247 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053572369999999999 2024-09-15T21:47:07.247 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.247 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.247 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.247 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421999+0000", 2024-09-15T21:47:07.247 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2024000000000001e-05 2024-09-15T21:47:07.247 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.247 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.247 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.247 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.248 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.248 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-15T21:47:07.248 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:07.248 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.792812724999997, 2024-09-15T21:47:07.248 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058922669999999996, 2024-09-15T21:47:07.248 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.248 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.248 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:07.248 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.248 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.248 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.249 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:07.249 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.249 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.249 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.249 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:07.249 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.249 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.249 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.249 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:07.249 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6499999999999999e-07 2024-09-15T21:47:07.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.250 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.250 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.250 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416142+0000", 2024-09-15T21:47:07.250 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3929999999999999e-06 2024-09-15T21:47:07.250 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.250 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.250 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.250 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416151+0000", 2024-09-15T21:47:07.250 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5840000000000001e-06 2024-09-15T21:47:07.250 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.250 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.250 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.250 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416157+0000", 2024-09-15T21:47:07.251 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7590000000000003e-06 2024-09-15T21:47:07.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.251 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.251 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.251 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416767+0000", 2024-09-15T21:47:07.251 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00061034900000000005 2024-09-15T21:47:07.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.251 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.251 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.251 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416782+0000", 2024-09-15T21:47:07.251 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.483e-05 2024-09-15T21:47:07.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.252 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.252 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:07.252 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422002+0000", 2024-09-15T21:47:07.252 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052202639999999996 2024-09-15T21:47:07.252 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.252 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.252 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.252 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422028+0000", 2024-09-15T21:47:07.252 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5423e-05 2024-09-15T21:47:07.252 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.252 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.252 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.253 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.253 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.253 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-15T21:47:07.253 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:07.253 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.792406223999997, 2024-09-15T21:47:07.253 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055107769999999997, 2024-09-15T21:47:07.253 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.253 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.253 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:07.253 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.253 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.253 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.254 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:07.254 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.254 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.254 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.254 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.254 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:07.254 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.254 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.254 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.254 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.254 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416543+0000", 2024-09-15T21:47:07.254 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5400000000000001e-07 2024-09-15T21:47:07.254 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.255 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.255 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.255 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416548+0000", 2024-09-15T21:47:07.255 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8659999999999997e-06 2024-09-15T21:47:07.255 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.255 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.255 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.255 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416555+0000", 2024-09-15T21:47:07.255 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7190000000000001e-06 2024-09-15T21:47:07.255 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.255 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.255 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.255 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416560+0000", 2024-09-15T21:47:07.255 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3270000000000002e-06 2024-09-15T21:47:07.256 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.256 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.256 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.256 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416916+0000", 2024-09-15T21:47:07.256 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00035661200000000002 2024-09-15T21:47:07.256 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.256 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.256 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.256 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416932+0000", 2024-09-15T21:47:07.256 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5824e-05 2024-09-15T21:47:07.256 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.256 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.256 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:07.257 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422030+0000", 2024-09-15T21:47:07.257 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050975229999999996 2024-09-15T21:47:07.257 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.257 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.257 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.257 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422053+0000", 2024-09-15T21:47:07.257 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3252e-05 2024-09-15T21:47:07.257 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.257 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.257 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.257 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.257 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.257 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-15T21:47:07.258 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:07.258 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.792007724999998, 2024-09-15T21:47:07.258 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051422289999999999, 2024-09-15T21:47:07.258 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.258 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.258 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:07.258 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.258 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.258 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.258 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:07.258 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.258 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.258 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.259 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.259 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:07.259 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.259 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.259 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.259 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.259 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:07.259 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2699999999999999e-07 2024-09-15T21:47:07.259 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.259 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.259 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.259 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416946+0000", 2024-09-15T21:47:07.259 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1390000000000003e-06 2024-09-15T21:47:07.259 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.260 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.260 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.260 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416957+0000", 2024-09-15T21:47:07.260 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.044e-05 2024-09-15T21:47:07.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.260 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.260 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.260 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416963+0000", 2024-09-15T21:47:07.260 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5320000000000003e-06 2024-09-15T21:47:07.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.260 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.260 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.260 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417090+0000", 2024-09-15T21:47:07.260 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126549 2024-09-15T21:47:07.261 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.261 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.261 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417114+0000", 2024-09-15T21:47:07.261 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3736000000000001e-05 2024-09-15T21:47:07.261 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.261 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:07.261 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422055+0000", 2024-09-15T21:47:07.261 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049410410000000002 2024-09-15T21:47:07.261 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.261 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.262 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422083+0000", 2024-09-15T21:47:07.262 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8164999999999999e-05 2024-09-15T21:47:07.262 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.262 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.262 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.262 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.262 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.262 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:07.262 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:07.262 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 32.747515753000002, 2024-09-15T21:47:07.262 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007521186, 2024-09-15T21:47:07.262 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.262 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.263 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:07.263 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.263 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.263 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.263 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:07.263 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.263 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.263 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.263 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.263 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:07.263 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.263 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.263 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.264 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.264 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:07.264 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9999999999999998e-07 2024-09-15T21:47:07.264 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.264 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.264 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.264 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461443+0000", 2024-09-15T21:47:07.264 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0448000000000001e-05 2024-09-15T21:47:07.264 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.264 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.264 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.264 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462146+0000", 2024-09-15T21:47:07.264 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00070223499999999995 2024-09-15T21:47:07.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.265 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.265 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462154+0000", 2024-09-15T21:47:07.265 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9060000000000002e-06 2024-09-15T21:47:07.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.265 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.265 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462254+0000", 2024-09-15T21:47:07.265 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9897999999999998e-05 2024-09-15T21:47:07.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.265 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.266 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462330+0000", 2024-09-15T21:47:07.266 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6551000000000006e-05 2024-09-15T21:47:07.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.266 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.266 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:07.266 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462454+0000", 2024-09-15T21:47:07.266 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000124213 2024-09-15T21:47:07.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.266 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.266 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:07.266 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463681+0000", 2024-09-15T21:47:07.266 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012265729999999999 2024-09-15T21:47:07.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.266 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.267 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:07.267 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468817+0000", 2024-09-15T21:47:07.267 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051361430000000001 2024-09-15T21:47:07.267 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.267 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.267 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:07.267 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468892+0000", 2024-09-15T21:47:07.267 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5480000000000002e-05 2024-09-15T21:47:07.267 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.267 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.267 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.267 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468954+0000", 2024-09-15T21:47:07.268 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1339000000000003e-05 2024-09-15T21:47:07.268 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.268 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.268 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.268 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.268 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.268 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:07.268 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:07.268 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 32.747499138000002, 2024-09-15T21:47:07.268 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077296149999999996, 2024-09-15T21:47:07.268 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.268 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.268 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:07.269 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.269 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.269 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.269 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:07.269 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.269 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.269 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.269 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:07.269 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.269 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.269 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.269 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461450+0000", 2024-09-15T21:47:07.270 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0499999999999999e-07 2024-09-15T21:47:07.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.270 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.270 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.270 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461452+0000", 2024-09-15T21:47:07.270 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.007e-06 2024-09-15T21:47:07.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.270 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.270 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.270 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462209+0000", 2024-09-15T21:47:07.270 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075781000000000004 2024-09-15T21:47:07.271 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.271 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.271 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.271 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462214+0000", 2024-09-15T21:47:07.271 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1420000000000001e-06 2024-09-15T21:47:07.271 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.271 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.271 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.271 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462681+0000", 2024-09-15T21:47:07.271 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046634200000000002 2024-09-15T21:47:07.271 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.271 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.271 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.272 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462772+0000", 2024-09-15T21:47:07.272 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1414000000000003e-05 2024-09-15T21:47:07.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.272 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.272 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:07.272 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462871+0000", 2024-09-15T21:47:07.272 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8455000000000005e-05 2024-09-15T21:47:07.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.272 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.272 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:07.272 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464138+0000", 2024-09-15T21:47:07.272 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012673910000000001 2024-09-15T21:47:07.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.272 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.273 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:07.273 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469079+0000", 2024-09-15T21:47:07.273 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049407799999999997 2024-09-15T21:47:07.273 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.273 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.273 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:07.273 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469141+0000", 2024-09-15T21:47:07.273 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2169999999999996e-05 2024-09-15T21:47:07.273 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.273 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.273 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.273 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469179+0000", 2024-09-15T21:47:07.273 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7799000000000002e-05 2024-09-15T21:47:07.273 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.274 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.274 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.274 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.274 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:07.274 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:07.274 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 26.733383329999999, 2024-09-15T21:47:07.274 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004990497, 2024-09-15T21:47:07.274 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.274 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.274 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:07.274 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.274 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.274 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.274 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:07.275 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.275 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.275 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.275 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.275 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:07.275 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.275 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.275 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.275 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.275 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:07.275 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4729999999999999e-06 2024-09-15T21:47:07.275 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.275 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.276 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.276 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475576+0000", 2024-09-15T21:47:07.276 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1749999999999994e-06 2024-09-15T21:47:07.276 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.276 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.276 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.276 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475619+0000", 2024-09-15T21:47:07.276 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3557000000000003e-05 2024-09-15T21:47:07.276 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.276 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.276 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.276 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475631+0000", 2024-09-15T21:47:07.276 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1739999999999999e-05 2024-09-15T21:47:07.277 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.277 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.277 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.277 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475770+0000", 2024-09-15T21:47:07.277 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013950599999999999 2024-09-15T21:47:07.277 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.277 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.277 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.277 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475887+0000", 2024-09-15T21:47:07.277 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011616 2024-09-15T21:47:07.277 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.277 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.277 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:07.278 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476018+0000", 2024-09-15T21:47:07.278 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013119499999999999 2024-09-15T21:47:07.278 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.278 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.278 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:07.278 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478442+0000", 2024-09-15T21:47:07.278 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024240870000000001 2024-09-15T21:47:07.278 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.278 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.278 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:07.278 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480457+0000", 2024-09-15T21:47:07.279 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020151779999999998 2024-09-15T21:47:07.279 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.279 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.279 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:07.279 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480511+0000", 2024-09-15T21:47:07.279 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3582000000000003e-05 2024-09-15T21:47:07.279 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.279 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.279 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.279 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480556+0000", 2024-09-15T21:47:07.280 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4844e-05 2024-09-15T21:47:07.280 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.280 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.280 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.280 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.280 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.280 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:07.280 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:07.280 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 26.733369012000001, 2024-09-15T21:47:07.280 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051124810000000003, 2024-09-15T21:47:07.280 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.280 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.280 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:07.280 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.281 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.281 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.281 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:07.281 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.281 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.281 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.281 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.281 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:07.281 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.281 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.281 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.281 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.282 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475580+0000", 2024-09-15T21:47:07.282 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7800000000000002e-07 2024-09-15T21:47:07.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.282 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.282 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475595+0000", 2024-09-15T21:47:07.282 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4919000000000001e-05 2024-09-15T21:47:07.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.282 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.282 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475662+0000", 2024-09-15T21:47:07.282 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7769999999999997e-05 2024-09-15T21:47:07.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.283 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.283 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475669+0000", 2024-09-15T21:47:07.283 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.703e-06 2024-09-15T21:47:07.283 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.283 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.283 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.283 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476781+0000", 2024-09-15T21:47:07.283 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011118300000000001 2024-09-15T21:47:07.283 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.283 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.283 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.283 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476863+0000", 2024-09-15T21:47:07.283 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2208999999999998e-05 2024-09-15T21:47:07.283 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.284 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.284 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:07.284 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477040+0000", 2024-09-15T21:47:07.284 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000176424 2024-09-15T21:47:07.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.284 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.284 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:07.284 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480154+0000", 2024-09-15T21:47:07.284 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031147589999999999 2024-09-15T21:47:07.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.284 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.284 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:07.284 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480596+0000", 2024-09-15T21:47:07.285 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044172699999999999 2024-09-15T21:47:07.285 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.285 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.285 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:07.285 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480652+0000", 2024-09-15T21:47:07.285 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6350000000000001e-05 2024-09-15T21:47:07.285 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.285 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.285 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.285 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480692+0000", 2024-09-15T21:47:07.285 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9412000000000001e-05 2024-09-15T21:47:07.285 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.285 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.286 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.286 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:6 2.9 2:941d59bc:::61f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:07.286 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:07.286 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 21.664158480000001, 2024-09-15T21:47:07.286 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057427529999999997, 2024-09-15T21:47:07.286 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.286 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.286 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:07.286 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.286 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.286 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:07.287 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.287 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.287 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.287 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:07.287 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.287 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.287 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.287 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:07.287 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.41e-07 2024-09-15T21:47:07.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.288 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.288 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544792+0000", 2024-09-15T21:47:07.288 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9929999999999998e-06 2024-09-15T21:47:07.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.288 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.288 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544885+0000", 2024-09-15T21:47:07.288 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2825000000000007e-05 2024-09-15T21:47:07.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.288 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.288 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544891+0000", 2024-09-15T21:47:07.289 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.93e-06 2024-09-15T21:47:07.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.289 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.289 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.289 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546766+0000", 2024-09-15T21:47:07.289 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018747569999999999 2024-09-15T21:47:07.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.289 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.289 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.289 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546864+0000", 2024-09-15T21:47:07.290 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8603000000000004e-05 2024-09-15T21:47:07.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.290 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.290 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:07.290 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547028+0000", 2024-09-15T21:47:07.290 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000163461 2024-09-15T21:47:07.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.290 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.290 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:07.290 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549499+0000", 2024-09-15T21:47:07.290 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024711749999999999 2024-09-15T21:47:07.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.290 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.291 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:07.291 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550417+0000", 2024-09-15T21:47:07.291 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00091817199999999998 2024-09-15T21:47:07.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.291 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.291 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:07.291 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550476+0000", 2024-09-15T21:47:07.291 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8635999999999999e-05 2024-09-15T21:47:07.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.291 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.291 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.291 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550533+0000", 2024-09-15T21:47:07.291 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6959999999999997e-05 2024-09-15T21:47:07.291 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.292 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.292 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.292 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.292 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.292 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:07.292 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:07.292 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 21.664144015000002, 2024-09-15T21:47:07.292 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004917121, 2024-09-15T21:47:07.292 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.292 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.292 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:07.292 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.293 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.293 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.293 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:07.293 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.293 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.293 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.293 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:07.293 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.293 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.293 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.294 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544805+0000", 2024-09-15T21:47:07.294 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4600000000000001e-07 2024-09-15T21:47:07.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.294 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.294 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.294 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544817+0000", 2024-09-15T21:47:07.294 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2023e-05 2024-09-15T21:47:07.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.294 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.294 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.294 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545000+0000", 2024-09-15T21:47:07.294 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018384400000000001 2024-09-15T21:47:07.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.295 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.295 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.295 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545007+0000", 2024-09-15T21:47:07.295 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1249999999999998e-06 2024-09-15T21:47:07.295 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.295 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.295 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.295 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545670+0000", 2024-09-15T21:47:07.295 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00066324999999999999 2024-09-15T21:47:07.295 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.295 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.295 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.296 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545749+0000", 2024-09-15T21:47:07.296 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9239000000000005e-05 2024-09-15T21:47:07.296 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.296 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.296 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:07.296 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545885+0000", 2024-09-15T21:47:07.296 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013586199999999999 2024-09-15T21:47:07.296 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.296 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.296 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:07.296 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547944+0000", 2024-09-15T21:47:07.296 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020594720000000001 2024-09-15T21:47:07.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.297 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.297 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:07.297 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549630+0000", 2024-09-15T21:47:07.297 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001685831 2024-09-15T21:47:07.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.297 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.297 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:07.297 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549681+0000", 2024-09-15T21:47:07.297 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1187000000000003e-05 2024-09-15T21:47:07.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.297 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.297 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.298 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549721+0000", 2024-09-15T21:47:07.298 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0042e-05 2024-09-15T21:47:07.298 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.298 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.298 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.298 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.298 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.298 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:07.298 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:07.298 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 21.664130267000001, 2024-09-15T21:47:07.298 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050615549999999997, 2024-09-15T21:47:07.298 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.299 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.299 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:07.299 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.299 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.299 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.299 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:07.299 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.299 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.299 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.299 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.299 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:07.299 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.299 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.299 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.300 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.300 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:07.300 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:47:07.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.300 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.300 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.300 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544821+0000", 2024-09-15T21:47:07.300 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4399999999999999e-06 2024-09-15T21:47:07.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.300 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.300 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.301 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545025+0000", 2024-09-15T21:47:07.301 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020379400000000001 2024-09-15T21:47:07.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.301 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.301 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545030+0000", 2024-09-15T21:47:07.301 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3109999999999996e-06 2024-09-15T21:47:07.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.301 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.301 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546196+0000", 2024-09-15T21:47:07.301 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001166261 2024-09-15T21:47:07.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.302 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.302 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546307+0000", 2024-09-15T21:47:07.302 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000110641 2024-09-15T21:47:07.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.302 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.302 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:07.302 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546422+0000", 2024-09-15T21:47:07.302 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000115399 2024-09-15T21:47:07.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.302 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.302 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:07.302 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547973+0000", 2024-09-15T21:47:07.302 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001551228 2024-09-15T21:47:07.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:07.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549815+0000", 2024-09-15T21:47:07.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001841685 2024-09-15T21:47:07.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:07.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549856+0000", 2024-09-15T21:47:07.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.091e-05 2024-09-15T21:47:07.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549880+0000", 2024-09-15T21:47:07.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3580000000000001e-05 2024-09-15T21:47:07.304 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.304 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.304 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.304 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.304 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:07.304 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:07.304 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 21.664126259, 2024-09-15T21:47:07.304 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056931389999999998, 2024-09-15T21:47:07.304 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.304 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.304 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:07.304 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.305 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.305 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:07.305 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.305 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.305 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:07.305 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.306 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.306 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:07.306 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:47:07.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.306 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.306 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544828+0000", 2024-09-15T21:47:07.306 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4369999999999998e-06 2024-09-15T21:47:07.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.306 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.306 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545041+0000", 2024-09-15T21:47:07.306 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000212695 2024-09-15T21:47:07.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.307 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545058+0000", 2024-09-15T21:47:07.307 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7717999999999999e-05 2024-09-15T21:47:07.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.307 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546958+0000", 2024-09-15T21:47:07.307 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018995539999999999 2024-09-15T21:47:07.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.307 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547057+0000", 2024-09-15T21:47:07.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9370999999999994e-05 2024-09-15T21:47:07.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.308 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:07.308 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547579+0000", 2024-09-15T21:47:07.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052198200000000004 2024-09-15T21:47:07.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.308 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:07.308 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549929+0000", 2024-09-15T21:47:07.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023494649999999998 2024-09-15T21:47:07.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.309 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:07.309 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550401+0000", 2024-09-15T21:47:07.309 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00047277300000000001 2024-09-15T21:47:07.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.309 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.309 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:07.309 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550458+0000", 2024-09-15T21:47:07.309 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6756999999999999e-05 2024-09-15T21:47:07.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.309 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.309 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.309 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550515+0000", 2024-09-15T21:47:07.309 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7093999999999999e-05 2024-09-15T21:47:07.310 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.310 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.310 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.310 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.310 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.310 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:26 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:07.310 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:07.310 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 21.664119602, 2024-09-15T21:47:07.310 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050112029999999997, 2024-09-15T21:47:07.310 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.310 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.310 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:07.310 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.310 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.311 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.311 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:07.311 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.311 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.311 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.311 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:07.311 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.311 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.311 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.311 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:07.311 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.36e-07 2024-09-15T21:47:07.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.312 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545086+0000", 2024-09-15T21:47:07.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025704699999999999 2024-09-15T21:47:07.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.312 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545119+0000", 2024-09-15T21:47:07.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2734999999999997e-05 2024-09-15T21:47:07.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545126+0000", 2024-09-15T21:47:07.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5599999999999996e-06 2024-09-15T21:47:07.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.313 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545253+0000", 2024-09-15T21:47:07.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012644500000000001 2024-09-15T21:47:07.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.313 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545366+0000", 2024-09-15T21:47:07.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011349999999999999 2024-09-15T21:47:07.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.314 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.314 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:07.314 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545673+0000", 2024-09-15T21:47:07.314 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030643599999999999 2024-09-15T21:47:07.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.314 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.314 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:07.314 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547924+0000", 2024-09-15T21:47:07.314 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0022516189999999998 2024-09-15T21:47:07.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.314 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.314 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:07.315 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549746+0000", 2024-09-15T21:47:07.315 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018217629999999999 2024-09-15T21:47:07.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.315 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:07.315 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549800+0000", 2024-09-15T21:47:07.315 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3412999999999998e-05 2024-09-15T21:47:07.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.315 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.315 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549840+0000", 2024-09-15T21:47:07.315 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0448999999999998e-05 2024-09-15T21:47:07.315 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.315 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.316 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.316 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.316 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.316 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:30 2.4 2:218afd09:::203.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:07.316 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:07.316 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 21.663858773000001, 2024-09-15T21:47:07.316 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055481230000000003, 2024-09-15T21:47:07.316 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.316 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.316 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:07.316 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.316 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.316 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.316 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:07.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:07.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545091+0000", 2024-09-15T21:47:07.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7000000000000003e-07 2024-09-15T21:47:07.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.318 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545094+0000", 2024-09-15T21:47:07.318 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9569999999999998e-06 2024-09-15T21:47:07.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.318 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.318 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545156+0000", 2024-09-15T21:47:07.318 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2277e-05 2024-09-15T21:47:07.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.318 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.318 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545162+0000", 2024-09-15T21:47:07.318 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9120000000000003e-06 2024-09-15T21:47:07.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.319 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.319 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545262+0000", 2024-09-15T21:47:07.319 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010001900000000001 2024-09-15T21:47:07.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.319 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.320 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545388+0000", 2024-09-15T21:47:07.320 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126243 2024-09-15T21:47:07.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.320 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.320 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.320 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550638+0000", 2024-09-15T21:47:07.320 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052497450000000001 2024-09-15T21:47:07.321 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.321 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.321 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.321 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.321 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:10 2.9 2:954a9a2b:::62b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:07.321 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:07.321 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 15.644175313, 2024-09-15T21:47:07.321 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048438860000000004, 2024-09-15T21:47:07.321 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:07.321 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:07.321 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:07.322 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:07.322 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.322 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:07.322 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:07.322 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.322 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.322 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.322 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:07.322 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:07.322 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:07.322 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.322 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.322 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:07.322 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:07.323 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0400000000000002e-07 2024-09-15T21:47:07.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.323 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:07.323 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564786+0000", 2024-09-15T21:47:07.323 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2412000000000001e-05 2024-09-15T21:47:07.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.323 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:07.323 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564991+0000", 2024-09-15T21:47:07.323 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020483900000000001 2024-09-15T21:47:07.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.323 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:07.324 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564998+0000", 2024-09-15T21:47:07.324 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2459999999999997e-06 2024-09-15T21:47:07.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.324 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.324 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:07.324 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565422+0000", 2024-09-15T21:47:07.324 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00042420100000000003 2024-09-15T21:47:07.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.324 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.324 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:07.324 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565515+0000", 2024-09-15T21:47:07.324 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2449999999999997e-05 2024-09-15T21:47:07.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.324 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.325 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:07.325 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565642+0000", 2024-09-15T21:47:07.325 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012743699999999999 2024-09-15T21:47:07.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.325 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:07.325 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566656+0000", 2024-09-15T21:47:07.325 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010138580000000001 2024-09-15T21:47:07.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.325 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:07.325 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569552+0000", 2024-09-15T21:47:07.325 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028966819999999998 2024-09-15T21:47:07.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.326 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.326 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:07.326 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569594+0000", 2024-09-15T21:47:07.326 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1235000000000001e-05 2024-09-15T21:47:07.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:07.326 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:07.326 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:07.326 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569617+0000", 2024-09-15T21:47:07.326 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3221999999999999e-05 2024-09-15T21:47:07.326 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.326 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.326 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.326 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:07.326 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:07.327 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:07.328 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:47:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.244+0000 7f8d56a01640 1 Processor -- start 2024-09-15T21:47:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.244+0000 7f8d56a01640 1 -- start start 2024-09-15T21:47:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.245+0000 7f8d56a01640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f8d501530b0 0x7f8d500c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.245+0000 7f8d56a01640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d500c7bf0 con 0x7f8d500c5720 2024-09-15T21:47:07.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.245+0000 7f8d56a01640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d500706c0 con 0x7f8d50152cd0 2024-09-15T21:47:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.245+0000 7f8d56a01640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f8d500c4970 con 0x7f8d501530b0 2024-09-15T21:47:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.245+0000 7f8d54f9e640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f8d500c5720 0x7f8d5006bbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:33990/0 (socket says 172.21.15.100:33990) 2024-09-15T21:47:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.245+0000 7f8d54f9e640 1 -- 172.21.15.100:0/1179663913 learned_addr learned my addr 172.21.15.100:0/1179663913 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.245+0000 7f8d4f7fe640 1 --2- 172.21.15.100:0/1179663913 >> v2:172.21.15.122:3300/0 conn(0x7f8d501530b0 0x7f8d500c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.245+0000 7f8d4effd640 1 -- 172.21.15.100:0/1179663913 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 380412466 0 0) 0x7f8d500c7bf0 con 0x7f8d500c5720 2024-09-15T21:47:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.245+0000 7f8d4effd640 1 -- 172.21.15.100:0/1179663913 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d38003520 con 0x7f8d500c5720 2024-09-15T21:47:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.246+0000 7f8d4f7fe640 1 -- 172.21.15.100:0/1179663913 >> v1:172.21.15.122:6789/0 conn(0x7f8d50152cd0 legacy=0x7f8d5006ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.246+0000 7f8d4f7fe640 1 -- 172.21.15.100:0/1179663913 >> v1:172.21.15.100:6789/0 conn(0x7f8d500c5720 legacy=0x7f8d5006bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.246+0000 7f8d4f7fe640 1 -- 172.21.15.100:0/1179663913 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d500c7070 con 0x7f8d501530b0 2024-09-15T21:47:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.246+0000 7f8d4f7fe640 1 --2- 172.21.15.100:0/1179663913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d501530b0 0x7f8d500c51e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8d500c4af0 tx=0x7f8d4402a2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c83ea19aeed065b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.246+0000 7f8d4effd640 1 -- 172.21.15.100:0/1179663913 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d44037070 con 0x7f8d501530b0 2024-09-15T21:47:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.247+0000 7f8d4effd640 1 -- 172.21.15.100:0/1179663913 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d44032640 con 0x7f8d501530b0 2024-09-15T21:47:07.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.247+0000 7f8d4effd640 1 -- 172.21.15.100:0/1179663913 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d440329f0 con 0x7f8d501530b0 2024-09-15T21:47:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.247+0000 7f8d56a01640 1 -- 172.21.15.100:0/1179663913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d501530b0 msgr2=0x7f8d500c51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.247+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1179663913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d501530b0 0x7f8d500c51e0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8d500c4af0 tx=0x7f8d4402a2c0 comp rx=0 tx=0).stop 2024-09-15T21:47:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.247+0000 7f8d56a01640 1 -- 172.21.15.100:0/1179663913 shutdown_connections 2024-09-15T21:47:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.247+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1179663913 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d501530b0 0x7f8d500c51e0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.247+0000 7f8d56a01640 1 -- 172.21.15.100:0/1179663913 >> 172.21.15.100:0/1179663913 conn(0x7f8d5006ccf0 msgr2=0x7f8d5006d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.247+0000 7f8d56a01640 1 -- 172.21.15.100:0/1179663913 shutdown_connections 2024-09-15T21:47:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.247+0000 7f8d56a01640 1 -- 172.21.15.100:0/1179663913 wait complete. 2024-09-15T21:47:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.248+0000 7f8d56a01640 1 Processor -- start 2024-09-15T21:47:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.248+0000 7f8d56a01640 1 -- start start 2024-09-15T21:47:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.249+0000 7f8d56a01640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d500c5720 0x7f8d5013c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.249+0000 7f8d56a01640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8d50152cd0 0x7f8d50140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.249+0000 7f8d56a01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d501411b0 0x7f8d501eedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:07.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.249+0000 7f8d56a01640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8d500c7910 con 0x7f8d500c5720 2024-09-15T21:47:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.249+0000 7f8d56a01640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8d500c7630 con 0x7f8d501411b0 2024-09-15T21:47:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.249+0000 7f8d56a01640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8d500c4cc0 con 0x7f8d50152cd0 2024-09-15T21:47:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.249+0000 7f8d4ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d500c5720 0x7f8d5013c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.249+0000 7f8d4ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d500c5720 0x7f8d5013c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49100/0 (socket says 172.21.15.100:49100) 2024-09-15T21:47:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.249+0000 7f8d4ffff640 1 -- 172.21.15.100:0/1664259874 learned_addr learned my addr 172.21.15.100:0/1664259874 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.249+0000 7f8d4f7fe640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8d50152cd0 0x7f8d50140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.249+0000 7f8d4ffff640 1 -- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8d50152cd0 msgr2=0x7f8d50140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.249+0000 7f8d54f9e640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d501411b0 0x7f8d501eedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:07.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.249+0000 7f8d4ffff640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8d50152cd0 0x7f8d50140c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.249+0000 7f8d4ffff640 1 -- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d501411b0 msgr2=0x7f8d501eedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.249+0000 7f8d4ffff640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8d501411b0 0x7f8d501eedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.249+0000 7f8d4ffff640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d501f2b60 con 0x7f8d500c5720 2024-09-15T21:47:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.250+0000 7f8d4ffff640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d500c5720 0x7f8d5013c390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000c400 tx=0x7f8d4000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.250+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d400052d0 con 0x7f8d500c5720 2024-09-15T21:47:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.250+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d40005490 con 0x7f8d500c5720 2024-09-15T21:47:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.250+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d501f1fe0 con 0x7f8d500c5720 2024-09-15T21:47:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.250+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d40007260 con 0x7f8d500c5720 2024-09-15T21:47:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.250+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d501f25a0 con 0x7f8d500c5720 2024-09-15T21:47:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.251+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f8d50158de0 con 0x7f8d500c5720 2024-09-15T21:47:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.251+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8d40007400 con 0x7f8d500c5720 2024-09-15T21:47:07.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.252+0000 7f8d4cff9640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8d3c06c2b0 0x7f8d3c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.252+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5854+0+0 (secure 0 0 0) 0x7f8d4008aa00 con 0x7f8d500c5720 2024-09-15T21:47:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.252+0000 7f8d4f7fe640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8d3c06c2b0 0x7f8d3c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.252+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v26) ==== 54+0+16 (secure 0 0 0) 0x7f8d4008f240 con 0x7f8d500c5720 2024-09-15T21:47:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.252+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f8d501530b0 0x7f8d501ee710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.252+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8d501e5020 con 0x7f8d501530b0 2024-09-15T21:47:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.252+0000 7f8d4f7fe640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8d3c06c2b0 0x7f8d3c06e770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8d44034000 tx=0x7f8d4402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.252+0000 7f8d54f9e640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f8d501530b0 0x7f8d501ee710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.252+0000 7f8d54f9e640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f8d501530b0 0x7f8d501ee710 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.253+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d501e5020 con 0x7f8d501530b0 2024-09-15T21:47:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.267+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8d501f1c80 con 0x7f8d501530b0 2024-09-15T21:47:07.338 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.269+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d501f1c80 con 0x7f8d501530b0 2024-09-15T21:47:07.339 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.269+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f8d50142600 0x7f8d50144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.269+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f8d50144c40 con 0x7f8d50142600 2024-09-15T21:47:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.269+0000 7f8d4ffff640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f8d50142600 0x7f8d50144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.269+0000 7f8d4ffff640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f8d50142600 0x7f8d50144700 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.270+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d50144c40 con 0x7f8d50142600 2024-09-15T21:47:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.284+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8d501450d0 con 0x7f8d50142600 2024-09-15T21:47:07.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.286+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d501450d0 con 0x7f8d50142600 2024-09-15T21:47:07.341 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.286+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f8d50062b40 0x7f8d50062f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.286+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f8d500634a0 con 0x7f8d50062b40 2024-09-15T21:47:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.286+0000 7f8d4f7fe640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f8d50062b40 0x7f8d50062f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.286+0000 7f8d4f7fe640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f8d50062b40 0x7f8d50062f60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.287+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d500634a0 con 0x7f8d50062b40 2024-09-15T21:47:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.302+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8d5002c910 con 0x7f8d50062b40 2024-09-15T21:47:07.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.303+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d5002c910 con 0x7f8d50062b40 2024-09-15T21:47:07.343 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.303+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f8d5002d3f0 0x7f8d501e25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.303+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f8d5002d830 con 0x7f8d5002d3f0 2024-09-15T21:47:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.304+0000 7f8d54f9e640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f8d5002d3f0 0x7f8d501e25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.304+0000 7f8d54f9e640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f8d5002d3f0 0x7f8d501e25e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.304+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d5002d830 con 0x7f8d5002d3f0 2024-09-15T21:47:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.318+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8d5002d830 con 0x7f8d5002d3f0 2024-09-15T21:47:07.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.319+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d5002d830 con 0x7f8d5002d3f0 2024-09-15T21:47:07.345 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:07.345 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:07.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.319+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f8d501e3030 0x7f8d501e3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:07.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.319+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f8d501e39d0 con 0x7f8d501e3030 2024-09-15T21:47:07.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.320+0000 7f8d4ffff640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f8d501e3030 0x7f8d501e3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:07.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.320+0000 7f8d4ffff640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f8d501e3030 0x7f8d501e3490 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:07.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.321+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d501e39d0 con 0x7f8d501e3030 2024-09-15T21:47:07.346 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:07.326+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:07.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.334+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8d501e39d0 con 0x7f8d501e3030 2024-09-15T21:47:07.347 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.336+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d501e39d0 con 0x7f8d501e3030 2024-09-15T21:47:07.347 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:07.348 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:07.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.337+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f8d50203000 0x7f8d502053f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:07.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.337+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f8d501e4050 con 0x7f8d50203000 2024-09-15T21:47:07.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.337+0000 7f8d4f7fe640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f8d50203000 0x7f8d502053f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:07.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.337+0000 7f8d4f7fe640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f8d50203000 0x7f8d502053f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:07.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.338+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== osd.5 v2:172.21.15.122:6824/2400668532 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d501e4050 con 0x7f8d50203000 2024-09-15T21:47:07.353 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.351+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8d501e4050 con 0x7f8d50203000 2024-09-15T21:47:07.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.354+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== osd.5 v2:172.21.15.122:6824/2400668532 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d501e4050 con 0x7f8d50203000 2024-09-15T21:47:07.355 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-15T21:47:07.356 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:47:07.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.354+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f8d50205e40 0x7f8d50208230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:07.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.354+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f8d5020b000 con 0x7f8d50205e40 2024-09-15T21:47:07.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.354+0000 7f8d54f9e640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f8d50205e40 0x7f8d50208230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:07.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.355+0000 7f8d54f9e640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f8d50205e40 0x7f8d50208230 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:07.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.355+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d5020b000 con 0x7f8d50205e40 2024-09-15T21:47:07.371 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.369+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8d50208770 con 0x7f8d50205e40 2024-09-15T21:47:07.372 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.371+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d50208770 con 0x7f8d50205e40 2024-09-15T21:47:07.373 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:07.373 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:07.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.371+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f8d50208df0 0x7f8d5020b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.371+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f8d5020dfb0 con 0x7f8d50208df0 2024-09-15T21:47:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.371+0000 7f8d4ffff640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f8d50208df0 0x7f8d5020b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.372+0000 7f8d4ffff640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f8d50208df0 0x7f8d5020b1e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:07.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.373+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8d5020dfb0 con 0x7f8d50208df0 2024-09-15T21:47:07.387 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.386+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f8d5020b720 con 0x7f8d50208df0 2024-09-15T21:47:07.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d4cff9640 1 -- 172.21.15.100:0/1664259874 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f8d5020b720 con 0x7f8d50208df0 2024-09-15T21:47:07.390 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:07.392 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:07.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f8d501530b0 msgr2=0x7f8d501ee710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:07.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f8d501530b0 0x7f8d501ee710 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:07.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f8d50142600 msgr2=0x7f8d50144700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:07.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f8d50142600 0x7f8d50144700 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:07.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f8d50062b40 msgr2=0x7f8d50062f60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:07.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f8d50062b40 0x7f8d50062f60 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:07.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f8d5002d3f0 msgr2=0x7f8d501e25e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:07.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f8d5002d3f0 0x7f8d501e25e0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:07.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f8d501e3030 msgr2=0x7f8d501e3490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:07.392 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f8d501e3030 0x7f8d501e3490 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f8d50203000 msgr2=0x7f8d502053f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f8d50203000 0x7f8d502053f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f8d50205e40 msgr2=0x7f8d50208230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d54f9e640 1 -- 172.21.15.100:0/1664259874 reap_dead start 2024-09-15T21:47:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f8d50205e40 0x7f8d50208230 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d54f9e640 1 -- 172.21.15.100:0/1664259874 reap_dead start 2024-09-15T21:47:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f8d50208df0 msgr2=0x7f8d5020b1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d54f9e640 1 -- 172.21.15.100:0/1664259874 reap_dead start 2024-09-15T21:47:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f8d50208df0 0x7f8d5020b1e0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8d3c06c2b0 msgr2=0x7f8d3c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:07.393 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8d3c06c2b0 0x7f8d3c06e770 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8d44034000 tx=0x7f8d4402e000 comp rx=0 tx=0).stop 2024-09-15T21:47:07.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d500c5720 msgr2=0x7f8d5013c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:07.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d500c5720 0x7f8d5013c390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8d4000c400 tx=0x7f8d4000c8c0 comp rx=0 tx=0).stop 2024-09-15T21:47:07.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 shutdown_connections 2024-09-15T21:47:07.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f8d50208df0 0x7f8d5020b1e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:07.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8d3c06c2b0 0x7f8d3c06e770 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:07.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.388+0000 7f8d56a01640 1 --2- 172.21.15.100:0/1664259874 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8d500c5720 0x7f8d5013c390 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:07.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.389+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 >> 172.21.15.100:0/1664259874 conn(0x7f8d5006ccf0 msgr2=0x7f8d50156a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:07.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.389+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 shutdown_connections 2024-09-15T21:47:07.394 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:07.389+0000 7f8d56a01640 1 -- 172.21.15.100:0/1664259874 wait complete. 2024-09-15T21:47:07.442 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:07.441+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:07.452 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:07.452 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:07.460 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:47:07.530 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:47:07.528+0000 7f74313af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37847) UID: 0 2024-09-15T21:47:07.605 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:07.605 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:07.613 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:47:07.631 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:07.629+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:07.717 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:07.717 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:07.726 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:47:07.732 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:07.731+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:07.829 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:07.830 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:07.833 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:07.831+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:07.839 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:47:07.934 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:07.933+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:07.982 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:07.982 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:07.991 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:47:08.035 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:08.034+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:08.134 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:08.134 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:08.136 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:47:08.135+0000 7f74313af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37847) UID: 0 2024-09-15T21:47:08.142 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:47:08.238 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:47:08.236+0000 7f74313af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37847) UID: 0 2024-09-15T21:47:08.261 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:08.262 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:08.271 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:47:08.338 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:08.337+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:08.415 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:08.416 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:08.423 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:47:08.468 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:08.466+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:08.526 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:08.526 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:08.534 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:47:08.576 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:47:08.574+0000 7f74313af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37847) UID: 0 2024-09-15T21:47:08.641 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:08.640+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:08.642 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:47:08.678 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:08.678 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:08.687 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:47:08.741 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:08.740+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:08.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.742+0000 7f995a1a2640 1 Processor -- start 2024-09-15T21:47:08.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.742+0000 7f995a1a2640 1 -- start start 2024-09-15T21:47:08.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.743+0000 7f995a1a2640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f99540cbf90 0x7f995415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:08.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.743+0000 7f995a1a2640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f995415c580 0x7f995415c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:08.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.743+0000 7f995a1a2640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99540cbcd0 con 0x7f99541550e0 2024-09-15T21:47:08.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.743+0000 7f995a1a2640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f995415cf30 con 0x7f99540cbf90 2024-09-15T21:47:08.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.743+0000 7f995a1a2640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f99541554c0 con 0x7f995415c580 2024-09-15T21:47:08.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.743+0000 7f99599a1640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f995415c580 0x7f995415c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:08.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.743+0000 7f995899f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f99540cbf90 0x7f995415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:08.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.743+0000 7f995899f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f99540cbf90 0x7f995415ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:37218/0 (socket says 172.21.15.100:37218) 2024-09-15T21:47:08.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.743+0000 7f99599a1640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f995415c580 0x7f995415c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:46212/0 (socket says 172.21.15.100:46212) 2024-09-15T21:47:08.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.743+0000 7f995899f640 1 -- 172.21.15.100:0/2688216482 learned_addr learned my addr 172.21.15.100:0/2688216482 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:08.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.743+0000 7f99591a0640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f99541550e0 0x7f99540d08b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.100:51562/0 (socket says 172.21.15.100:51562) 2024-09-15T21:47:08.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.743+0000 7f995899f640 1 -- 172.21.15.100:0/2688216482 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f995415c580 msgr2=0x7f995415c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:08.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.743+0000 7f995899f640 1 --2- 172.21.15.100:0/2688216482 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f995415c580 0x7f995415c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:08.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.743+0000 7f995899f640 1 -- 172.21.15.100:0/2688216482 >> v1:172.21.15.122:6789/0 conn(0x7f99541550e0 legacy=0x7f99540d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:08.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.743+0000 7f995899f640 1 -- 172.21.15.100:0/2688216482 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99540cce40 con 0x7f99540cbf90 2024-09-15T21:47:08.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.743+0000 7f99599a1640 1 --2- 172.21.15.100:0/2688216482 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f995415c580 0x7f995415c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:08.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.743+0000 7f994bfff640 1 -- 172.21.15.100:0/2688216482 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4193187893 0 0) 0x7f9950001110 con 0x7f99541550e0 2024-09-15T21:47:08.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.744+0000 7f995899f640 1 --2- 172.21.15.100:0/2688216482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99540cbf90 0x7f995415ba50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9944009870 tx=0x7f994402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62a394223817e540 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:08.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.744+0000 7f994bfff640 1 -- 172.21.15.100:0/2688216482 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944037070 con 0x7f99540cbf90 2024-09-15T21:47:08.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.744+0000 7f994bfff640 1 -- 172.21.15.100:0/2688216482 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9944032440 con 0x7f99540cbf90 2024-09-15T21:47:08.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.744+0000 7f994bfff640 1 -- 172.21.15.100:0/2688216482 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9944032810 con 0x7f99540cbf90 2024-09-15T21:47:08.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.745+0000 7f995a1a2640 1 -- 172.21.15.100:0/2688216482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99540cbf90 msgr2=0x7f995415ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:08.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.745+0000 7f995a1a2640 1 --2- 172.21.15.100:0/2688216482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99540cbf90 0x7f995415ba50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9944009870 tx=0x7f994402a270 comp rx=0 tx=0).stop 2024-09-15T21:47:08.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.745+0000 7f995a1a2640 1 -- 172.21.15.100:0/2688216482 shutdown_connections 2024-09-15T21:47:08.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.745+0000 7f995a1a2640 1 --2- 172.21.15.100:0/2688216482 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f995415c580 0x7f995415c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:08.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.745+0000 7f995a1a2640 1 --2- 172.21.15.100:0/2688216482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99540cbf90 0x7f995415ba50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:08.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.745+0000 7f995a1a2640 1 -- 172.21.15.100:0/2688216482 >> 172.21.15.100:0/2688216482 conn(0x7f9954067980 msgr2=0x7f9954067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:08.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.745+0000 7f995a1a2640 1 -- 172.21.15.100:0/2688216482 shutdown_connections 2024-09-15T21:47:08.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.745+0000 7f995a1a2640 1 -- 172.21.15.100:0/2688216482 wait complete. 2024-09-15T21:47:08.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.746+0000 7f995a1a2640 1 Processor -- start 2024-09-15T21:47:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.746+0000 7f995a1a2640 1 -- start start 2024-09-15T21:47:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f995a1a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99540cbf90 0x7f99540ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f995a1a2640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f99541550e0 0x7f9954159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f995a1a2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f995415c580 0x7f99540c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f995a1a2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f99540cd6c0 con 0x7f995415c580 2024-09-15T21:47:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f995a1a2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f99540cd3e0 con 0x7f99540cbf90 2024-09-15T21:47:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f995a1a2640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f99540cd9a0 con 0x7f99541550e0 2024-09-15T21:47:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f99591a0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99540cbf90 0x7f99540ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f99599a1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f995415c580 0x7f99540c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f995899f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f99541550e0 0x7f9954159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f99591a0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99540cbf90 0x7f99540ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:37234/0 (socket says 172.21.15.100:37234) 2024-09-15T21:47:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f99591a0640 1 -- 172.21.15.100:0/1026447390 learned_addr learned my addr 172.21.15.100:0/1026447390 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f99599a1640 1 -- 172.21.15.100:0/1026447390 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f99541550e0 msgr2=0x7f9954159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:08.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f99599a1640 1 --2- 172.21.15.100:0/1026447390 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f99541550e0 0x7f9954159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:08.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f99599a1640 1 -- 172.21.15.100:0/1026447390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99540cbf90 msgr2=0x7f99540ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:08.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f99599a1640 1 --2- 172.21.15.100:0/1026447390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99540cbf90 0x7f99540ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:08.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f99599a1640 1 -- 172.21.15.100:0/1026447390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995415ac00 con 0x7f995415c580 2024-09-15T21:47:08.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f99591a0640 1 --2- 172.21.15.100:0/1026447390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99540cbf90 0x7f99540ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:08.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f99599a1640 1 --2- 172.21.15.100:0/1026447390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f995415c580 0x7f99540c5a50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f994c00efc0 tx=0x7f994c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:08.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f9949ffb640 1 -- 172.21.15.100:0/1026447390 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c018030 con 0x7f995415c580 2024-09-15T21:47:08.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f9949ffb640 1 -- 172.21.15.100:0/1026447390 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f994c00cd30 con 0x7f995415c580 2024-09-15T21:47:08.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f995a1a2640 1 -- 172.21.15.100:0/1026447390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995415aee0 con 0x7f995415c580 2024-09-15T21:47:08.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.747+0000 7f9949ffb640 1 -- 172.21.15.100:0/1026447390 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c003bb0 con 0x7f995415c580 2024-09-15T21:47:08.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.748+0000 7f995a1a2640 1 -- 172.21.15.100:0/1026447390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99540b4e10 con 0x7f995415c580 2024-09-15T21:47:08.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.749+0000 7f995a1a2640 1 -- 172.21.15.100:0/1026447390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9954069420 con 0x7f995415c580 2024-09-15T21:47:08.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.749+0000 7f9949ffb640 1 -- 172.21.15.100:0/1026447390 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f994c01b070 con 0x7f995415c580 2024-09-15T21:47:08.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.749+0000 7f9949ffb640 1 --2- 172.21.15.100:0/1026447390 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f993806c2b0 0x7f993806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:08.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.750+0000 7f9949ffb640 1 -- 172.21.15.100:0/1026447390 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5854+0+0 (secure 0 0 0) 0x7f994c08bcc0 con 0x7f995415c580 2024-09-15T21:47:08.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.750+0000 7f99591a0640 1 --2- 172.21.15.100:0/1026447390 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f993806c2b0 0x7f993806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:08.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.750+0000 7f99591a0640 1 --2- 172.21.15.100:0/1026447390 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f993806c2b0 0x7f993806e770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9950000f60 tx=0x7f9950007c10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:08.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.754+0000 7f9949ffb640 1 -- 172.21.15.100:0/1026447390 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f994c010040 con 0x7f995415c580 2024-09-15T21:47:08.791 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:08.791 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:08.800 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:47:08.842 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:08.841+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:08.903 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:08.904 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:08.912 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:47:08.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:08.930+0000 7f995a1a2640 1 -- 172.21.15.100:0/1026447390 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f9954158e70 con 0x7f995415c580 2024-09-15T21:47:08.943 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:08.942+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:09.015 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:09.016 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:09.024 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:47:09.068 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:09.066+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:09.126 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:09.126 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:09.135 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:47:09.177 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:09.175+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:09.247 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:47:09.245+0000 7f74313af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37847) UID: 0 2024-09-15T21:47:09.277 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:09.277 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:09.286 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:47:09.370 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:09.369+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:09.402 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:47:09.430 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:09.430 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:09.439 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:47:09.481 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:47:09.479+0000 7f74313af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37847) UID: 0 2024-09-15T21:47:09.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.500+0000 7f31548c9640 1 Processor -- start 2024-09-15T21:47:09.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.500+0000 7f31548c9640 1 -- start start 2024-09-15T21:47:09.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.501+0000 7f31548c9640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f31500ced10 0x7f31500c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.501+0000 7f31548c9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f31500cf0f0 0x7f3150073a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.501+0000 7f31548c9640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f3150073f60 0x7f31501551d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.501+0000 7f31548c9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f315015f760 con 0x7f31500cf0f0 2024-09-15T21:47:09.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.501+0000 7f31548c9640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f315015e900 con 0x7f3150073f60 2024-09-15T21:47:09.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.501+0000 7f31548c9640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f315015ebe0 con 0x7f31500ced10 2024-09-15T21:47:09.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.501+0000 7f314dd9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f31500cf0f0 0x7f3150073a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.501+0000 7f314dd9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f31500cf0f0 0x7f3150073a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49122/0 (socket says 172.21.15.100:49122) 2024-09-15T21:47:09.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.501+0000 7f314dd9b640 1 -- 172.21.15.100:0/1194481481 learned_addr learned my addr 172.21.15.100:0/1194481481 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:09.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.501+0000 7f314e59c640 1 --2- 172.21.15.100:0/1194481481 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f31500ced10 0x7f31500c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.501+0000 7f314ed9d640 1 --2- 172.21.15.100:0/1194481481 >> v2:172.21.15.122:3300/0 conn(0x7f3150073f60 0x7f31501551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.501+0000 7f314dd9b640 1 -- 172.21.15.100:0/1194481481 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f31500ced10 msgr2=0x7f31500c48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:09.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.501+0000 7f314dd9b640 1 --2- 172.21.15.100:0/1194481481 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f31500ced10 0x7f31500c48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.501+0000 7f314dd9b640 1 -- 172.21.15.100:0/1194481481 >> v2:172.21.15.122:3300/0 conn(0x7f3150073f60 msgr2=0x7f31501551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:09.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.501+0000 7f314dd9b640 1 --2- 172.21.15.100:0/1194481481 >> v2:172.21.15.122:3300/0 conn(0x7f3150073f60 0x7f31501551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.501+0000 7f314dd9b640 1 -- 172.21.15.100:0/1194481481 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f315015eec0 con 0x7f31500cf0f0 2024-09-15T21:47:09.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.502+0000 7f314dd9b640 1 --2- 172.21.15.100:0/1194481481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31500cf0f0 0x7f3150073a20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3150071ac0 tx=0x7f314402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=252c2bf059412ccd server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:09.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.502+0000 7f314d59a640 1 -- 172.21.15.100:0/1194481481 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144037070 con 0x7f31500cf0f0 2024-09-15T21:47:09.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.502+0000 7f314d59a640 1 -- 172.21.15.100:0/1194481481 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3144032440 con 0x7f31500cf0f0 2024-09-15T21:47:09.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.502+0000 7f314d59a640 1 -- 172.21.15.100:0/1194481481 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144032810 con 0x7f31500cf0f0 2024-09-15T21:47:09.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.503+0000 7f31548c9640 1 -- 172.21.15.100:0/1194481481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31500cf0f0 msgr2=0x7f3150073a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.503+0000 7f31548c9640 1 --2- 172.21.15.100:0/1194481481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31500cf0f0 0x7f3150073a20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3150071ac0 tx=0x7f314402a1e0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.503+0000 7f31548c9640 1 -- 172.21.15.100:0/1194481481 shutdown_connections 2024-09-15T21:47:09.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.503+0000 7f31548c9640 1 --2- 172.21.15.100:0/1194481481 >> v2:172.21.15.122:3300/0 conn(0x7f3150073f60 0x7f31501551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.503+0000 7f31548c9640 1 --2- 172.21.15.100:0/1194481481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31500cf0f0 0x7f3150073a20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.503+0000 7f31548c9640 1 --2- 172.21.15.100:0/1194481481 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f31500ced10 0x7f31500c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.503+0000 7f31548c9640 1 -- 172.21.15.100:0/1194481481 >> 172.21.15.100:0/1194481481 conn(0x7f315006f9f0 msgr2=0x7f3150070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:09.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.503+0000 7f31548c9640 1 -- 172.21.15.100:0/1194481481 shutdown_connections 2024-09-15T21:47:09.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.503+0000 7f31548c9640 1 -- 172.21.15.100:0/1194481481 wait complete. 2024-09-15T21:47:09.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.504+0000 7f31548c9640 1 Processor -- start 2024-09-15T21:47:09.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.504+0000 7f31548c9640 1 -- start start 2024-09-15T21:47:09.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.505+0000 7f31548c9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31500ced10 0x7f315015d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.505+0000 7f31548c9640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f31500cf0f0 0x7f31500b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.505+0000 7f31548c9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31500b4530 0x7f31500b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.505+0000 7f31548c9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3150155710 con 0x7f31500ced10 2024-09-15T21:47:09.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.505+0000 7f31548c9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f315015f480 con 0x7f31500b4530 2024-09-15T21:47:09.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.505+0000 7f31548c9640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3150155890 con 0x7f31500cf0f0 2024-09-15T21:47:09.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.505+0000 7f314e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31500ced10 0x7f315015d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.505+0000 7f314e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31500ced10 0x7f315015d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49128/0 (socket says 172.21.15.100:49128) 2024-09-15T21:47:09.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.505+0000 7f314e59c640 1 -- 172.21.15.100:0/4284974880 learned_addr learned my addr 172.21.15.100:0/4284974880 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:09.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.505+0000 7f314dd9b640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f31500cf0f0 0x7f31500b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.505+0000 7f314ed9d640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31500b4530 0x7f31500b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.505+0000 7f314e59c640 1 -- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f31500cf0f0 msgr2=0x7f31500b0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.505+0000 7f314e59c640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f31500cf0f0 0x7f31500b0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.505+0000 7f314e59c640 1 -- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31500b4530 msgr2=0x7f31500b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.505+0000 7f314e59c640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f31500b4530 0x7f31500b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.505+0000 7f314e59c640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31501f1a40 con 0x7f31500ced10 2024-09-15T21:47:09.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.505+0000 7f314e59c640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31500ced10 0x7f315015d4b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f314000e9b0 tx=0x7f314000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:09.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.506+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3140017070 con 0x7f31500ced10 2024-09-15T21:47:09.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.506+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f314000c9a0 con 0x7f31500ced10 2024-09-15T21:47:09.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.506+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314000cd90 con 0x7f31500ced10 2024-09-15T21:47:09.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.506+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31501f1d00 con 0x7f31500ced10 2024-09-15T21:47:09.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.506+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3150068460 con 0x7f31500ced10 2024-09-15T21:47:09.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.506+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f31500c4450 con 0x7f31500ced10 2024-09-15T21:47:09.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.507+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f314001a070 con 0x7f31500ced10 2024-09-15T21:47:09.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.507+0000 7f313effd640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f313006c2b0 0x7f313006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.508+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5854+0+0 (secure 0 0 0) 0x7f314008b830 con 0x7f31500ced10 2024-09-15T21:47:09.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.508+0000 7f314dd9b640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f313006c2b0 0x7f313006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.508+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v26) ==== 54+0+16 (secure 0 0 0) 0x7f31500c4450 con 0x7f31500ced10 2024-09-15T21:47:09.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.508+0000 7f31548c9640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f3150073f60 0x7f31500c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.508+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f31501551d0 con 0x7f3150073f60 2024-09-15T21:47:09.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.508+0000 7f314dd9b640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f313006c2b0 0x7f313006e770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3144034000 tx=0x7f314402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:09.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.508+0000 7f314ed9d640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f3150073f60 0x7f31500c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.508+0000 7f314ed9d640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f3150073f60 0x7f31500c7590 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:09.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.509+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31501551d0 con 0x7f3150073f60 2024-09-15T21:47:09.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.523+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f31500c6ed0 con 0x7f3150073f60 2024-09-15T21:47:09.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.524+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f31500c6ed0 con 0x7f3150073f60 2024-09-15T21:47:09.526 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:09.527 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:09.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.524+0000 7f31548c9640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f31501f4510 0x7f31501f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.524+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f31500c9700 con 0x7f31501f4510 2024-09-15T21:47:09.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.525+0000 7f314e59c640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f31501f4510 0x7f31501f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.525+0000 7f314e59c640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f31501f4510 0x7f31501f48f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:09.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.525+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31500c9700 con 0x7f31501f4510 2024-09-15T21:47:09.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.540+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f3150153320 con 0x7f31501f4510 2024-09-15T21:47:09.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.541+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3150153320 con 0x7f31501f4510 2024-09-15T21:47:09.543 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:09.544 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:09.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.541+0000 7f31548c9640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f3150062630 0x7f3150062a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.541+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f31501f5340 con 0x7f3150062630 2024-09-15T21:47:09.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.542+0000 7f314dd9b640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f3150062630 0x7f3150062a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.542+0000 7f314dd9b640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f3150062630 0x7f3150062a30 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:09.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.543+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f31501f5340 con 0x7f3150062630 2024-09-15T21:47:09.550 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:09.548+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:09.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.556+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f3150152d60 con 0x7f3150062630 2024-09-15T21:47:09.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.557+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f3150152d60 con 0x7f3150062630 2024-09-15T21:47:09.559 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.558+0000 7f31548c9640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f31500ccf10 0x7f31500cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.558+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f31500c88e0 con 0x7f31500ccf10 2024-09-15T21:47:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.558+0000 7f314ed9d640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f31500ccf10 0x7f31500cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.558+0000 7f314ed9d640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f31500ccf10 0x7f31500cd350 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:09.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.559+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3148011b90 con 0x7f31500ccf10 2024-09-15T21:47:09.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.572+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f31500cd890 con 0x7f31500ccf10 2024-09-15T21:47:09.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.574+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f31500cd890 con 0x7f31500ccf10 2024-09-15T21:47:09.576 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:09.576 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:09.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.574+0000 7f31548c9640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f315002c640 0x7f315002caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.574+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f315002cfe0 con 0x7f315002c640 2024-09-15T21:47:09.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.574+0000 7f314e59c640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f315002c640 0x7f315002caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.574+0000 7f314e59c640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f315002c640 0x7f315002caa0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:09.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.575+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f315002cfe0 con 0x7f315002c640 2024-09-15T21:47:09.583 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:09.583 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:09.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.588+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f315002cfe0 con 0x7f315002c640 2024-09-15T21:47:09.592 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:47:09.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.590+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f315002cfe0 con 0x7f315002c640 2024-09-15T21:47:09.593 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:09.594 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:09.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.590+0000 7f31548c9640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f315002d4f0 0x7f3150207900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.590+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f315002d980 con 0x7f315002d4f0 2024-09-15T21:47:09.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.591+0000 7f314dd9b640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f315002d4f0 0x7f3150207900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.591+0000 7f314dd9b640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f315002d4f0 0x7f3150207900 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:09.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.592+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== osd.5 v2:172.21.15.122:6824/2400668532 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f315002d980 con 0x7f315002d4f0 2024-09-15T21:47:09.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.605+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f315002d980 con 0x7f315002d4f0 2024-09-15T21:47:09.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.607+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== osd.5 v2:172.21.15.122:6824/2400668532 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f315002d980 con 0x7f315002d4f0 2024-09-15T21:47:09.608 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-15T21:47:09.609 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:47:09.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.607+0000 7f31548c9640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f3150208350 0x7f315020a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.607+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f315002db70 con 0x7f3150208350 2024-09-15T21:47:09.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.607+0000 7f314ed9d640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f3150208350 0x7f315020a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.607+0000 7f314ed9d640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f3150208350 0x7f315020a740 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:09.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.608+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f315002db70 con 0x7f3150208350 2024-09-15T21:47:09.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.621+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f315020ac80 con 0x7f3150208350 2024-09-15T21:47:09.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.623+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f315020ac80 con 0x7f3150208350 2024-09-15T21:47:09.624 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:09.625 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:09.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.623+0000 7f31548c9640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f315020b760 0x7f315020bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.623+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f315020d300 con 0x7f315020b760 2024-09-15T21:47:09.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.623+0000 7f314e59c640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f315020b760 0x7f315020bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.623+0000 7f314e59c640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f315020b760 0x7f315020bbc0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:09.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.624+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f315020d300 con 0x7f315020b760 2024-09-15T21:47:09.631 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.629+0000 7f9949ffb640 1 -- 172.21.15.100:0/1026447390 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v27) ==== 91+0+0 (secure 0 0 0) 0x7f994c059d50 con 0x7f995415c580 2024-09-15T21:47:09.631 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is unset 2024-09-15T21:47:09.631 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:47:09.631 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:47:09.631 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-15T21:47:09.632 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-15T21:47:09.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.631+0000 7f995a1a2640 1 -- 172.21.15.100:0/1026447390 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f993806c2b0 msgr2=0x7f993806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.631+0000 7f995a1a2640 1 --2- 172.21.15.100:0/1026447390 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f993806c2b0 0x7f993806e770 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9950000f60 tx=0x7f9950007c10 comp rx=0 tx=0).stop 2024-09-15T21:47:09.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.631+0000 7f995a1a2640 1 -- 172.21.15.100:0/1026447390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f995415c580 msgr2=0x7f99540c5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.631+0000 7f995a1a2640 1 --2- 172.21.15.100:0/1026447390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f995415c580 0x7f99540c5a50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f994c00efc0 tx=0x7f994c00c490 comp rx=0 tx=0).stop 2024-09-15T21:47:09.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.631+0000 7f995a1a2640 1 -- 172.21.15.100:0/1026447390 shutdown_connections 2024-09-15T21:47:09.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.631+0000 7f995a1a2640 1 --2- 172.21.15.100:0/1026447390 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f993806c2b0 0x7f993806e770 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.631+0000 7f995a1a2640 1 --2- 172.21.15.100:0/1026447390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f995415c580 0x7f99540c5a50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.631+0000 7f995a1a2640 1 --2- 172.21.15.100:0/1026447390 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f99541550e0 0x7f9954159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.631+0000 7f995a1a2640 1 --2- 172.21.15.100:0/1026447390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99540cbf90 0x7f99540ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.631+0000 7f995a1a2640 1 -- 172.21.15.100:0/1026447390 >> 172.21.15.100:0/1026447390 conn(0x7f9954067980 msgr2=0x7f995406dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:09.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.631+0000 7f995a1a2640 1 -- 172.21.15.100:0/1026447390 shutdown_connections 2024-09-15T21:47:09.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.631+0000 7f995a1a2640 1 -- 172.21.15.100:0/1026447390 wait complete. 2024-09-15T21:47:09.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.639+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f315020c180 con 0x7f315020b760 2024-09-15T21:47:09.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.641+0000 7f313effd640 1 -- 172.21.15.100:0/4284974880 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f315020c180 con 0x7f315020b760 2024-09-15T21:47:09.643 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:09.644 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:09.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.641+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f3150073f60 msgr2=0x7f31500c7590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.641+0000 7f31548c9640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f3150073f60 0x7f31500c7590 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.641+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f31501f4510 msgr2=0x7f31501f48f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.641+0000 7f31548c9640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f31501f4510 0x7f31501f48f0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.641+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f3150062630 msgr2=0x7f3150062a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.641+0000 7f31548c9640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f3150062630 0x7f3150062a30 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.641+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f31500ccf10 msgr2=0x7f31500cd350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.641+0000 7f31548c9640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f31500ccf10 0x7f31500cd350 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.641+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f315002c640 msgr2=0x7f315002caa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.641+0000 7f31548c9640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f315002c640 0x7f315002caa0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.641+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f315002d4f0 msgr2=0x7f3150207900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.641+0000 7f31548c9640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f315002d4f0 0x7f3150207900 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.641+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f3150208350 msgr2=0x7f315020a740 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.641+0000 7f31548c9640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f3150208350 0x7f315020a740 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.641+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f315020b760 msgr2=0x7f315020bbc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.641+0000 7f31548c9640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f315020b760 0x7f315020bbc0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.642+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f313006c2b0 msgr2=0x7f313006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.642+0000 7f31548c9640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f313006c2b0 0x7f313006e770 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3144034000 tx=0x7f314402e000 comp rx=0 tx=0).stop 2024-09-15T21:47:09.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.642+0000 7f314ed9d640 1 -- 172.21.15.100:0/4284974880 reap_dead start 2024-09-15T21:47:09.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.642+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31500ced10 msgr2=0x7f315015d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.642+0000 7f31548c9640 1 --2- 172.21.15.100:0/4284974880 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f31500ced10 0x7f315015d4b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f314000e9b0 tx=0x7f314000ee70 comp rx=0 tx=0).stop 2024-09-15T21:47:09.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.642+0000 7f314ed9d640 1 -- 172.21.15.100:0/4284974880 reap_dead start 2024-09-15T21:47:09.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.642+0000 7f314ed9d640 1 -- 172.21.15.100:0/4284974880 reap_dead start 2024-09-15T21:47:09.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.642+0000 7f314ed9d640 1 -- 172.21.15.100:0/4284974880 reap_dead start 2024-09-15T21:47:09.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.642+0000 7f314ed9d640 1 -- 172.21.15.100:0/4284974880 reap_dead start 2024-09-15T21:47:09.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.642+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 shutdown_connections 2024-09-15T21:47:09.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.642+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 >> 172.21.15.100:0/4284974880 conn(0x7f315006f9f0 msgr2=0x7f31500752f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:09.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.642+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 shutdown_connections 2024-09-15T21:47:09.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.642+0000 7f31548c9640 1 -- 172.21.15.100:0/4284974880 wait complete. 2024-09-15T21:47:09.677 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:09.675+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:09.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.731+0000 7f03f74f1640 1 Processor -- start 2024-09-15T21:47:09.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.731+0000 7f03f74f1640 1 -- start start 2024-09-15T21:47:09.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.733+0000 7f03f74f1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f03f00cd2f0 0x7f03f0166aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.733+0000 7f03f74f1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03f0071ed0 con 0x7f03f006c2f0 2024-09-15T21:47:09.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.733+0000 7f03f74f1640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03f015e890 con 0x7f03f00ccf10 2024-09-15T21:47:09.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.733+0000 7f03f74f1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f03f015f650 con 0x7f03f00cd2f0 2024-09-15T21:47:09.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.733+0000 7f03f5a8e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f03f00cd2f0 0x7f03f0166aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.733+0000 7f03f4a8c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f03f006c2f0 0x7f03f015b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:34000/0 (socket says 172.21.15.100:34000) 2024-09-15T21:47:09.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.733+0000 7f03f4a8c640 1 -- 172.21.15.100:0/1646412536 learned_addr learned my addr 172.21.15.100:0/1646412536 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:09.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.733+0000 7f03f5a8e640 1 -- 172.21.15.100:0/1646412536 >> v1:172.21.15.122:6789/0 conn(0x7f03f00ccf10 legacy=0x7f03f0071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.733+0000 7f03f5a8e640 1 -- 172.21.15.100:0/1646412536 >> v1:172.21.15.100:6789/0 conn(0x7f03f006c2f0 legacy=0x7f03f015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.733+0000 7f03e7fff640 1 -- 172.21.15.100:0/1646412536 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1394604635 0 0) 0x7f03f0071ed0 con 0x7f03f006c2f0 2024-09-15T21:47:09.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.733+0000 7f03f5a8e640 1 -- 172.21.15.100:0/1646412536 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f015eb70 con 0x7f03f00cd2f0 2024-09-15T21:47:09.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.734+0000 7f03f5a8e640 1 --2- 172.21.15.100:0/1646412536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03f00cd2f0 0x7f03f0166aa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009dc0 tx=0x7f03e802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ad2dd0b883a5e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:09.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.734+0000 7f03e7fff640 1 -- 172.21.15.100:0/1646412536 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8007740 con 0x7f03f00cd2f0 2024-09-15T21:47:09.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.734+0000 7f03e7fff640 1 -- 172.21.15.100:0/1646412536 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03e80078e0 con 0x7f03f00cd2f0 2024-09-15T21:47:09.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.734+0000 7f03e7fff640 1 -- 172.21.15.100:0/1646412536 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8007cb0 con 0x7f03f00cd2f0 2024-09-15T21:47:09.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.735+0000 7f03f74f1640 1 -- 172.21.15.100:0/1646412536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03f00cd2f0 msgr2=0x7f03f0166aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.735+0000 7f03f74f1640 1 --2- 172.21.15.100:0/1646412536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03f00cd2f0 0x7f03f0166aa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009dc0 tx=0x7f03e802a730 comp rx=0 tx=0).stop 2024-09-15T21:47:09.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.735+0000 7f03f74f1640 1 -- 172.21.15.100:0/1646412536 shutdown_connections 2024-09-15T21:47:09.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.735+0000 7f03f74f1640 1 --2- 172.21.15.100:0/1646412536 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03f00cd2f0 0x7f03f0166aa0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.735+0000 7f03f74f1640 1 -- 172.21.15.100:0/1646412536 >> 172.21.15.100:0/1646412536 conn(0x7f03f006d890 msgr2=0x7f03f006bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:09.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.735+0000 7f03f74f1640 1 -- 172.21.15.100:0/1646412536 shutdown_connections 2024-09-15T21:47:09.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.735+0000 7f03f74f1640 1 -- 172.21.15.100:0/1646412536 wait complete. 2024-09-15T21:47:09.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.736+0000 7f03f74f1640 1 Processor -- start 2024-09-15T21:47:09.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.736+0000 7f03f74f1640 1 -- start start 2024-09-15T21:47:09.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.737+0000 7f03f74f1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03f006c2f0 0x7f03f0159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.737+0000 7f03f74f1640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f03f00ccf10 0x7f03f00c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.737+0000 7f03f74f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03f00cd2f0 0x7f03f01e4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.737+0000 7f03f74f1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f03f015f130 con 0x7f03f006c2f0 2024-09-15T21:47:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.737+0000 7f03f74f1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f03f00721b0 con 0x7f03f00cd2f0 2024-09-15T21:47:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.737+0000 7f03f74f1640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f03f00cd740 con 0x7f03f00ccf10 2024-09-15T21:47:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.737+0000 7f03f528d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03f006c2f0 0x7f03f0159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.737+0000 7f03f528d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03f006c2f0 0x7f03f0159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49150/0 (socket says 172.21.15.100:49150) 2024-09-15T21:47:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.737+0000 7f03f528d640 1 -- 172.21.15.100:0/2044823479 learned_addr learned my addr 172.21.15.100:0/2044823479 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:09.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.737+0000 7f03f5a8e640 1 --2- 172.21.15.100:0/2044823479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03f00cd2f0 0x7f03f01e4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.737+0000 7f03f4a8c640 1 --2- 172.21.15.100:0/2044823479 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f03f00ccf10 0x7f03f00c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.737+0000 7f03f528d640 1 -- 172.21.15.100:0/2044823479 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f03f00ccf10 msgr2=0x7f03f00c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.737+0000 7f03f528d640 1 --2- 172.21.15.100:0/2044823479 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f03f00ccf10 0x7f03f00c64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.737+0000 7f03f528d640 1 -- 172.21.15.100:0/2044823479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03f00cd2f0 msgr2=0x7f03f01e4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.737+0000 7f03f528d640 1 --2- 172.21.15.100:0/2044823479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03f00cd2f0 0x7f03f01e4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.737+0000 7f03f528d640 1 -- 172.21.15.100:0/2044823479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f01e89f0 con 0x7f03f006c2f0 2024-09-15T21:47:09.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.737+0000 7f03f4a8c640 1 --2- 172.21.15.100:0/2044823479 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f03f00ccf10 0x7f03f00c64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:47:09.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.737+0000 7f03f528d640 1 --2- 172.21.15.100:0/2044823479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03f006c2f0 0x7f03f0159310 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00c450 tx=0x7f03ec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:09.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.738+0000 7f03e5ffb640 1 -- 172.21.15.100:0/2044823479 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec017070 con 0x7f03f006c2f0 2024-09-15T21:47:09.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.738+0000 7f03e5ffb640 1 -- 172.21.15.100:0/2044823479 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ec0052d0 con 0x7f03f006c2f0 2024-09-15T21:47:09.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.738+0000 7f03f74f1640 1 -- 172.21.15.100:0/2044823479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f01f22c0 con 0x7f03f006c2f0 2024-09-15T21:47:09.743 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.738+0000 7f03e5ffb640 1 -- 172.21.15.100:0/2044823479 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec003830 con 0x7f03f006c2f0 2024-09-15T21:47:09.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.738+0000 7f03f74f1640 1 -- 172.21.15.100:0/2044823479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f00c56f0 con 0x7f03f006c2f0 2024-09-15T21:47:09.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.739+0000 7f03f74f1640 1 -- 172.21.15.100:0/2044823479 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f03f00769a0 con 0x7f03f006c2f0 2024-09-15T21:47:09.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.739+0000 7f03e5ffb640 1 -- 172.21.15.100:0/2044823479 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f03ec01a070 con 0x7f03f006c2f0 2024-09-15T21:47:09.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.740+0000 7f03e5ffb640 1 --2- 172.21.15.100:0/2044823479 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f03d406c2e0 0x7f03d406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.740+0000 7f03e5ffb640 1 -- 172.21.15.100:0/2044823479 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5854+0+0 (secure 0 0 0) 0x7f03ec08a790 con 0x7f03f006c2f0 2024-09-15T21:47:09.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.740+0000 7f03f4a8c640 1 --2- 172.21.15.100:0/2044823479 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f03d406c2e0 0x7f03d406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.740+0000 7f03e5ffb640 1 --2- 172.21.15.100:0/2044823479 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f03d4074660 0x7f03d4076b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:09.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.740+0000 7f03e5ffb640 1 -- 172.21.15.100:0/2044823479 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f03d40771d0 con 0x7f03d4074660 2024-09-15T21:47:09.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.740+0000 7f03e5ffb640 1 -- 172.21.15.100:0/2044823479 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f03ec091050 con 0x7f03f006c2f0 2024-09-15T21:47:09.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.740+0000 7f03f5a8e640 1 --2- 172.21.15.100:0/2044823479 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f03d4074660 0x7f03d4076b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:09.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.740+0000 7f03f4a8c640 1 --2- 172.21.15.100:0/2044823479 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f03d406c2e0 0x7f03d406e7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f03e0007fd0 tx=0x7f03e0007dc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:09.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.740+0000 7f03f5a8e640 1 --2- 172.21.15.100:0/2044823479 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f03d4074660 0x7f03d4076b00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:09.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.741+0000 7f03e5ffb640 1 -- 172.21.15.100:0/2044823479 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f03d40771d0 con 0x7f03d4074660 2024-09-15T21:47:09.750 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:09.750 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:09.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:09.750 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:09.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.750 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:47:09.750 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 56.500262157999998, 2024-09-15T21:47:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0944283340000001, 2024-09-15T21:47:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2024-09-15T21:47:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234360+0000", 2024-09-15T21:47:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3710000000000001e-06 2024-09-15T21:47:09.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234374+0000", 2024-09-15T21:47:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4003e-05 2024-09-15T21:47:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234392+0000", 2024-09-15T21:47:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7294000000000001e-05 2024-09-15T21:47:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234405+0000", 2024-09-15T21:47:09.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2961e-05 2024-09-15T21:47:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234483+0000", 2024-09-15T21:47:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8850000000000006e-05 2024-09-15T21:47:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-15T21:47:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234491+0000", 2024-09-15T21:47:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3529999999999997e-06 2024-09-15T21:47:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327253+0000", 2024-09-15T21:47:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.092761962 2024-09-15T21:47:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327456+0000", 2024-09-15T21:47:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000202893 2024-09-15T21:47:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327652+0000", 2024-09-15T21:47:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019613000000000001 2024-09-15T21:47:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328368+0000", 2024-09-15T21:47:09.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000716582 2024-09-15T21:47:09.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328682+0000", 2024-09-15T21:47:09.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000313825 2024-09-15T21:47:09.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328743+0000", 2024-09-15T21:47:09.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0571e-05 2024-09-15T21:47:09.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328787+0000", 2024-09-15T21:47:09.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4539000000000002e-05 2024-09-15T21:47:09.758 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.758 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.758 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.758 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:09.759 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:09.759 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.379096756000003, 2024-09-15T21:47:09.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0089551270000000002, 2024-09-15T21:47:09.759 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.759 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.759 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.759 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:09.759 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:09.759 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:09.759 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 35 2024-09-15T21:47:09.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.759 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:09.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:09.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355525+0000", 2024-09-15T21:47:09.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:47:09.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355606+0000", 2024-09-15T21:47:09.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1723000000000003e-05 2024-09-15T21:47:09.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355624+0000", 2024-09-15T21:47:09.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7645000000000001e-05 2024-09-15T21:47:09.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355629+0000", 2024-09-15T21:47:09.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7319999999999998e-06 2024-09-15T21:47:09.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358315+0000", 2024-09-15T21:47:09.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026858659999999999 2024-09-15T21:47:09.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358373+0000", 2024-09-15T21:47:09.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8593e-05 2024-09-15T21:47:09.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358521+0000", 2024-09-15T21:47:09.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014817500000000001 2024-09-15T21:47:09.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360353+0000", 2024-09-15T21:47:09.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018317100000000001 2024-09-15T21:47:09.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364424+0000", 2024-09-15T21:47:09.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040707970000000001 2024-09-15T21:47:09.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364452+0000", 2024-09-15T21:47:09.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7937999999999999e-05 2024-09-15T21:47:09.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364479+0000", 2024-09-15T21:47:09.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7744999999999999e-05 2024-09-15T21:47:09.765 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.765 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.765 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.766 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:09.766 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:09.766 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.379012793000001, 2024-09-15T21:47:09.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090267869999999997, 2024-09-15T21:47:09.766 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.766 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.766 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.766 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:09.766 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:09.767 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:09.767 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 36 2024-09-15T21:47:09.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.767 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:09.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:09.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355609+0000", 2024-09-15T21:47:09.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3099999999999998e-07 2024-09-15T21:47:09.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355612+0000", 2024-09-15T21:47:09.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1489999999999998e-06 2024-09-15T21:47:09.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355637+0000", 2024-09-15T21:47:09.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.508e-05 2024-09-15T21:47:09.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355640+0000", 2024-09-15T21:47:09.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7930000000000002e-06 2024-09-15T21:47:09.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359195+0000", 2024-09-15T21:47:09.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035555529999999999 2024-09-15T21:47:09.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359283+0000", 2024-09-15T21:47:09.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8042999999999997e-05 2024-09-15T21:47:09.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359419+0000", 2024-09-15T21:47:09.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013576699999999999 2024-09-15T21:47:09.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360361+0000", 2024-09-15T21:47:09.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094171000000000001 2024-09-15T21:47:09.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364536+0000", 2024-09-15T21:47:09.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00417484 2024-09-15T21:47:09.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364590+0000", 2024-09-15T21:47:09.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4355999999999999e-05 2024-09-15T21:47:09.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364635+0000", 2024-09-15T21:47:09.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5065000000000003e-05 2024-09-15T21:47:09.774 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.775 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.775 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.775 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:09.775 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:09.775 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.379008188, 2024-09-15T21:47:09.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091162750000000001, 2024-09-15T21:47:09.775 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.775 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.775 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.776 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:09.776 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:09.776 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:09.776 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 37 2024-09-15T21:47:09.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.776 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:09.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:09.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:09.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:47:09.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355615+0000", 2024-09-15T21:47:09.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-15T21:47:09.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355646+0000", 2024-09-15T21:47:09.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0391000000000001e-05 2024-09-15T21:47:09.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355648+0000", 2024-09-15T21:47:09.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6309999999999999e-06 2024-09-15T21:47:09.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359770+0000", 2024-09-15T21:47:09.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041222899999999998 2024-09-15T21:47:09.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359854+0000", 2024-09-15T21:47:09.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3399000000000006e-05 2024-09-15T21:47:09.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359988+0000", 2024-09-15T21:47:09.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013442200000000001 2024-09-15T21:47:09.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360524+0000", 2024-09-15T21:47:09.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00053582900000000004 2024-09-15T21:47:09.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364673+0000", 2024-09-15T21:47:09.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041493199999999997 2024-09-15T21:47:09.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364704+0000", 2024-09-15T21:47:09.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0380999999999999e-05 2024-09-15T21:47:09.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364729+0000", 2024-09-15T21:47:09.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5335999999999999e-05 2024-09-15T21:47:09.781 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.781 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.781 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.781 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:09.781 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:09.781 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.310290950000002, 2024-09-15T21:47:09.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0026270020000001, 2024-09-15T21:47:09.782 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.782 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.782 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.782 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:09.782 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:09.782 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:09.782 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 88 2024-09-15T21:47:09.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.782 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:09.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:09.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424331+0000", 2024-09-15T21:47:09.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4299999999999998e-07 2024-09-15T21:47:09.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424333+0000", 2024-09-15T21:47:09.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2359999999999999e-06 2024-09-15T21:47:09.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422895+0000", 2024-09-15T21:47:09.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99856184199999998 2024-09-15T21:47:09.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422902+0000", 2024-09-15T21:47:09.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4440000000000002e-06 2024-09-15T21:47:09.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423053+0000", 2024-09-15T21:47:09.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000150958 2024-09-15T21:47:09.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423160+0000", 2024-09-15T21:47:09.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106774 2024-09-15T21:47:09.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423344+0000", 2024-09-15T21:47:09.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018424199999999999 2024-09-15T21:47:09.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.424730+0000", 2024-09-15T21:47:09.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001386155 2024-09-15T21:47:09.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426756+0000", 2024-09-15T21:47:09.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020261150000000002 2024-09-15T21:47:09.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426906+0000", 2024-09-15T21:47:09.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000149176 2024-09-15T21:47:09.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426957+0000", 2024-09-15T21:47:09.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1616999999999997e-05 2024-09-15T21:47:09.787 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.788 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.788 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.788 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:09.788 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:09.788 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.180217751999997, 2024-09-15T21:47:09.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095037790000000004, 2024-09-15T21:47:09.788 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.788 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.788 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.788 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:09.788 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:09.788 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:09.789 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 114 2024-09-15T21:47:09.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.789 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:09.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:09.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554404+0000", 2024-09-15T21:47:09.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2399999999999999e-07 2024-09-15T21:47:09.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554407+0000", 2024-09-15T21:47:09.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-15T21:47:09.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554632+0000", 2024-09-15T21:47:09.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022568999999999999 2024-09-15T21:47:09.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554640+0000", 2024-09-15T21:47:09.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7970000000000001e-06 2024-09-15T21:47:09.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561325+0000", 2024-09-15T21:47:09.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066848020000000001 2024-09-15T21:47:09.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561402+0000", 2024-09-15T21:47:09.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6851e-05 2024-09-15T21:47:09.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561530+0000", 2024-09-15T21:47:09.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012863999999999999 2024-09-15T21:47:09.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563144+0000", 2024-09-15T21:47:09.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016132849999999999 2024-09-15T21:47:09.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563800+0000", 2024-09-15T21:47:09.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00065586099999999999 2024-09-15T21:47:09.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563857+0000", 2024-09-15T21:47:09.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.702e-05 2024-09-15T21:47:09.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563907+0000", 2024-09-15T21:47:09.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0479000000000002e-05 2024-09-15T21:47:09.797 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.798 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.798 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.798 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:09.798 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:09.799 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.180212754999999, 2024-09-15T21:47:09.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011813423999999999, 2024-09-15T21:47:09.799 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.799 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.799 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.799 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:09.799 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:09.799 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:09.799 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 115 2024-09-15T21:47:09.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.800 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:09.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:09.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554409+0000", 2024-09-15T21:47:09.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1100000000000002e-07 2024-09-15T21:47:09.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554498+0000", 2024-09-15T21:47:09.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9642999999999995e-05 2024-09-15T21:47:09.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554656+0000", 2024-09-15T21:47:09.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000158114 2024-09-15T21:47:09.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554661+0000", 2024-09-15T21:47:09.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0730000000000004e-06 2024-09-15T21:47:09.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561862+0000", 2024-09-15T21:47:09.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072002999999999998 2024-09-15T21:47:09.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561939+0000", 2024-09-15T21:47:09.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7265e-05 2024-09-15T21:47:09.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562073+0000", 2024-09-15T21:47:09.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013371600000000001 2024-09-15T21:47:09.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563196+0000", 2024-09-15T21:47:09.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011231679999999999 2024-09-15T21:47:09.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566124+0000", 2024-09-15T21:47:09.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002928403 2024-09-15T21:47:09.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566183+0000", 2024-09-15T21:47:09.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8328000000000001e-05 2024-09-15T21:47:09.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566222+0000", 2024-09-15T21:47:09.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9103000000000002e-05 2024-09-15T21:47:09.805 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.805 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.805 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.806 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:09.806 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:09.806 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.180119314999999, 2024-09-15T21:47:09.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01181426, 2024-09-15T21:47:09.806 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.806 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.806 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.806 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:09.806 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:09.806 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:09.806 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 116 2024-09-15T21:47:09.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.807 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:09.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:09.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554503+0000", 2024-09-15T21:47:09.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8000000000000005e-07 2024-09-15T21:47:09.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554506+0000", 2024-09-15T21:47:09.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5899999999999999e-06 2024-09-15T21:47:09.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554674+0000", 2024-09-15T21:47:09.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016763600000000001 2024-09-15T21:47:09.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554678+0000", 2024-09-15T21:47:09.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5310000000000003e-06 2024-09-15T21:47:09.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562714+0000", 2024-09-15T21:47:09.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.008035258 2024-09-15T21:47:09.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562765+0000", 2024-09-15T21:47:09.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1093e-05 2024-09-15T21:47:09.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562845+0000", 2024-09-15T21:47:09.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0427999999999996e-05 2024-09-15T21:47:09.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563943+0000", 2024-09-15T21:47:09.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097647 2024-09-15T21:47:09.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566232+0000", 2024-09-15T21:47:09.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022890850000000002 2024-09-15T21:47:09.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566279+0000", 2024-09-15T21:47:09.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6692e-05 2024-09-15T21:47:09.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566316+0000", 2024-09-15T21:47:09.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.752e-05 2024-09-15T21:47:09.811 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.811 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.811 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.811 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:09.812 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:09.812 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.180113216000002, 2024-09-15T21:47:09.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012232125, 2024-09-15T21:47:09.812 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.812 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.812 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.812 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:09.812 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:09.812 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:09.812 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 117 2024-09-15T21:47:09.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.812 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:09.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:09.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:09.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7700000000000001e-07 2024-09-15T21:47:09.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554511+0000", 2024-09-15T21:47:09.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8430000000000001e-06 2024-09-15T21:47:09.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554690+0000", 2024-09-15T21:47:09.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017856299999999999 2024-09-15T21:47:09.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554698+0000", 2024-09-15T21:47:09.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6200000000000005e-06 2024-09-15T21:47:09.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563224+0000", 2024-09-15T21:47:09.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0085262640000000004 2024-09-15T21:47:09.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563342+0000", 2024-09-15T21:47:09.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117687 2024-09-15T21:47:09.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563472+0000", 2024-09-15T21:47:09.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012957999999999999 2024-09-15T21:47:09.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566367+0000", 2024-09-15T21:47:09.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028952280000000001 2024-09-15T21:47:09.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566680+0000", 2024-09-15T21:47:09.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031345100000000002 2024-09-15T21:47:09.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566714+0000", 2024-09-15T21:47:09.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3701000000000001e-05 2024-09-15T21:47:09.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566740+0000", 2024-09-15T21:47:09.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5911e-05 2024-09-15T21:47:09.819 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.819 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.819 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.819 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:09.820 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:09.820 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.837221134000004, 2024-09-15T21:47:09.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069806590000000002, 2024-09-15T21:47:09.820 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.820 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.820 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.820 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:09.820 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:09.820 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:09.820 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 134 2024-09-15T21:47:09.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.820 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:09.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:09.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897401+0000", 2024-09-15T21:47:09.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0500000000000003e-07 2024-09-15T21:47:09.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897492+0000", 2024-09-15T21:47:09.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1770999999999999e-05 2024-09-15T21:47:09.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897750+0000", 2024-09-15T21:47:09.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025778299999999998 2024-09-15T21:47:09.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897756+0000", 2024-09-15T21:47:09.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2140000000000001e-06 2024-09-15T21:47:09.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899312+0000", 2024-09-15T21:47:09.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001556144 2024-09-15T21:47:09.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899372+0000", 2024-09-15T21:47:09.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0015000000000001e-05 2024-09-15T21:47:09.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899444+0000", 2024-09-15T21:47:09.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1923999999999998e-05 2024-09-15T21:47:09.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901354+0000", 2024-09-15T21:47:09.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019092530000000001 2024-09-15T21:47:09.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904327+0000", 2024-09-15T21:47:09.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029734520000000001 2024-09-15T21:47:09.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904360+0000", 2024-09-15T21:47:09.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3216000000000002e-05 2024-09-15T21:47:09.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904381+0000", 2024-09-15T21:47:09.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0282e-05 2024-09-15T21:47:09.826 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.826 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.826 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.826 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:09.826 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:09.826 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.837125276999998, 2024-09-15T21:47:09.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070121059999999997, 2024-09-15T21:47:09.827 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.827 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.827 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.827 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:09.827 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:09.827 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:09.827 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 135 2024-09-15T21:47:09.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.827 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:09.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:09.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897497+0000", 2024-09-15T21:47:09.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4799999999999997e-07 2024-09-15T21:47:09.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897500+0000", 2024-09-15T21:47:09.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2949999999999998e-06 2024-09-15T21:47:09.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897781+0000", 2024-09-15T21:47:09.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028064300000000002 2024-09-15T21:47:09.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897786+0000", 2024-09-15T21:47:09.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5169999999999999e-06 2024-09-15T21:47:09.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899857+0000", 2024-09-15T21:47:09.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020705340000000002 2024-09-15T21:47:09.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899932+0000", 2024-09-15T21:47:09.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5840999999999994e-05 2024-09-15T21:47:09.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900005+0000", 2024-09-15T21:47:09.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2716e-05 2024-09-15T21:47:09.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902296+0000", 2024-09-15T21:47:09.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022912010000000001 2024-09-15T21:47:09.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904424+0000", 2024-09-15T21:47:09.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021278600000000001 2024-09-15T21:47:09.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904472+0000", 2024-09-15T21:47:09.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7586000000000001e-05 2024-09-15T21:47:09.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904508+0000", 2024-09-15T21:47:09.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6164999999999997e-05 2024-09-15T21:47:09.833 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.833 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.833 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.833 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:09.833 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:09.833 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.837100786000001, 2024-09-15T21:47:09.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070611429999999998, 2024-09-15T21:47:09.834 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.834 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.834 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.834 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:09.834 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:09.834 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:09.834 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 136 2024-09-15T21:47:09.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.834 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:09.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:09.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897521+0000", 2024-09-15T21:47:09.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2900000000000004e-07 2024-09-15T21:47:09.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897527+0000", 2024-09-15T21:47:09.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8490000000000002e-06 2024-09-15T21:47:09.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897799+0000", 2024-09-15T21:47:09.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027219499999999998 2024-09-15T21:47:09.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897804+0000", 2024-09-15T21:47:09.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4219999999999999e-06 2024-09-15T21:47:09.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900248+0000", 2024-09-15T21:47:09.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002443457 2024-09-15T21:47:09.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900306+0000", 2024-09-15T21:47:09.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8263000000000001e-05 2024-09-15T21:47:09.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900385+0000", 2024-09-15T21:47:09.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9161999999999999e-05 2024-09-15T21:47:09.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902300+0000", 2024-09-15T21:47:09.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001914947 2024-09-15T21:47:09.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904536+0000", 2024-09-15T21:47:09.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022354419999999998 2024-09-15T21:47:09.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904562+0000", 2024-09-15T21:47:09.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6378999999999999e-05 2024-09-15T21:47:09.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904581+0000", 2024-09-15T21:47:09.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9497999999999999e-05 2024-09-15T21:47:09.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.840 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.840 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.841 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:09.841 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:09.841 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.837092218999999, 2024-09-15T21:47:09.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007180306, 2024-09-15T21:47:09.841 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.841 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.841 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:09.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:09.841 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:09.842 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 137 2024-09-15T21:47:09.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.842 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:09.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:09.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:09.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1399999999999998e-07 2024-09-15T21:47:09.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897612+0000", 2024-09-15T21:47:09.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2607000000000004e-05 2024-09-15T21:47:09.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897815+0000", 2024-09-15T21:47:09.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020352400000000001 2024-09-15T21:47:09.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897821+0000", 2024-09-15T21:47:09.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4670000000000003e-06 2024-09-15T21:47:09.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900665+0000", 2024-09-15T21:47:09.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028443689999999998 2024-09-15T21:47:09.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900714+0000", 2024-09-15T21:47:09.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8812e-05 2024-09-15T21:47:09.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900799+0000", 2024-09-15T21:47:09.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5129000000000004e-05 2024-09-15T21:47:09.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903413+0000", 2024-09-15T21:47:09.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026142890000000001 2024-09-15T21:47:09.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904625+0000", 2024-09-15T21:47:09.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012121149999999999 2024-09-15T21:47:09.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904673+0000", 2024-09-15T21:47:09.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7540999999999998e-05 2024-09-15T21:47:09.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904709+0000", 2024-09-15T21:47:09.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6139000000000001e-05 2024-09-15T21:47:09.848 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.848 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.848 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.848 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:09.848 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:09.848 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.837006264000003, 2024-09-15T21:47:09.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071727450000000003, 2024-09-15T21:47:09.849 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.849 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.849 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.849 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:09.849 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:09.849 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:09.849 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 138 2024-09-15T21:47:09.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.849 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:09.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:09.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:09.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:47:09.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897619+0000", 2024-09-15T21:47:09.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3869999999999999e-06 2024-09-15T21:47:09.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897832+0000", 2024-09-15T21:47:09.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021329500000000001 2024-09-15T21:47:09.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897837+0000", 2024-09-15T21:47:09.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3730000000000001e-06 2024-09-15T21:47:09.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901391+0000", 2024-09-15T21:47:09.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003553601 2024-09-15T21:47:09.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901466+0000", 2024-09-15T21:47:09.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5130999999999996e-05 2024-09-15T21:47:09.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901601+0000", 2024-09-15T21:47:09.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134797 2024-09-15T21:47:09.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903417+0000", 2024-09-15T21:47:09.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001816156 2024-09-15T21:47:09.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904741+0000", 2024-09-15T21:47:09.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013241049999999999 2024-09-15T21:47:09.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904767+0000", 2024-09-15T21:47:09.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5876000000000001e-05 2024-09-15T21:47:09.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904788+0000", 2024-09-15T21:47:09.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0449999999999999e-05 2024-09-15T21:47:09.855 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.855 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.855 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.855 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:09.856 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:09.856 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.837000611000001, 2024-09-15T21:47:09.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072970250000000004, 2024-09-15T21:47:09.856 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.856 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.856 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.856 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:09.856 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:09.856 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:09.856 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 139 2024-09-15T21:47:09.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.856 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:09.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:09.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897621+0000", 2024-09-15T21:47:09.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3099999999999999e-07 2024-09-15T21:47:09.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897625+0000", 2024-09-15T21:47:09.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.698e-06 2024-09-15T21:47:09.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897848+0000", 2024-09-15T21:47:09.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022375699999999999 2024-09-15T21:47:09.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897853+0000", 2024-09-15T21:47:09.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7709999999999997e-06 2024-09-15T21:47:09.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901929+0000", 2024-09-15T21:47:09.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040757780000000004 2024-09-15T21:47:09.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901976+0000", 2024-09-15T21:47:09.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6751e-05 2024-09-15T21:47:09.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902054+0000", 2024-09-15T21:47:09.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8856999999999994e-05 2024-09-15T21:47:09.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903421+0000", 2024-09-15T21:47:09.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00136619 2024-09-15T21:47:09.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904827+0000", 2024-09-15T21:47:09.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0014067369999999999 2024-09-15T21:47:09.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904881+0000", 2024-09-15T21:47:09.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3520999999999998e-05 2024-09-15T21:47:09.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904918+0000", 2024-09-15T21:47:09.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6634000000000002e-05 2024-09-15T21:47:09.862 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.862 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.862 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.862 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:09.862 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:09.862 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.836994781999998, 2024-09-15T21:47:09.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097221730000000006, 2024-09-15T21:47:09.863 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.863 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.863 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.863 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:09.863 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:09.863 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:09.863 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 140 2024-09-15T21:47:09.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.863 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:09.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:09.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897627+0000", 2024-09-15T21:47:09.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-15T21:47:09.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897698+0000", 2024-09-15T21:47:09.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1722000000000002e-05 2024-09-15T21:47:09.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897863+0000", 2024-09-15T21:47:09.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016479099999999999 2024-09-15T21:47:09.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897868+0000", 2024-09-15T21:47:09.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4950000000000002e-06 2024-09-15T21:47:09.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902335+0000", 2024-09-15T21:47:09.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004467632 2024-09-15T21:47:09.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902414+0000", 2024-09-15T21:47:09.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8688000000000003e-05 2024-09-15T21:47:09.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902535+0000", 2024-09-15T21:47:09.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012066900000000001 2024-09-15T21:47:09.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903714+0000", 2024-09-15T21:47:09.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011794520000000001 2024-09-15T21:47:09.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907115+0000", 2024-09-15T21:47:09.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034008879999999999 2024-09-15T21:47:09.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907250+0000", 2024-09-15T21:47:09.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134657 2024-09-15T21:47:09.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907348+0000", 2024-09-15T21:47:09.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8838000000000005e-05 2024-09-15T21:47:09.869 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.869 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.869 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.869 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:09.869 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:09.869 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.836919700999999, 2024-09-15T21:47:09.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097618050000000001, 2024-09-15T21:47:09.869 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.870 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.870 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.870 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:09.870 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:09.870 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:09.870 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 141 2024-09-15T21:47:09.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.870 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:09.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:09.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897702+0000", 2024-09-15T21:47:09.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6199999999999998e-07 2024-09-15T21:47:09.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897705+0000", 2024-09-15T21:47:09.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0180000000000002e-06 2024-09-15T21:47:09.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897889+0000", 2024-09-15T21:47:09.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000183678 2024-09-15T21:47:09.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897894+0000", 2024-09-15T21:47:09.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8930000000000004e-06 2024-09-15T21:47:09.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903099+0000", 2024-09-15T21:47:09.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052056280000000003 2024-09-15T21:47:09.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903151+0000", 2024-09-15T21:47:09.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2163000000000002e-05 2024-09-15T21:47:09.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903220+0000", 2024-09-15T21:47:09.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8483000000000001e-05 2024-09-15T21:47:09.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904794+0000", 2024-09-15T21:47:09.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001574521 2024-09-15T21:47:09.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907405+0000", 2024-09-15T21:47:09.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026110339999999999 2024-09-15T21:47:09.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907440+0000", 2024-09-15T21:47:09.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4564999999999999e-05 2024-09-15T21:47:09.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907463+0000", 2024-09-15T21:47:09.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3260000000000001e-05 2024-09-15T21:47:09.875 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.876 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.876 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.876 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:22 2.8 2:1470b2f8:::613.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:09.876 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:09.876 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.279479373999997, 2024-09-15T21:47:09.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007454937, 2024-09-15T21:47:09.876 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.876 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.876 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.877 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:09.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:09.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:09.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6800000000000002e-07 2024-09-15T21:47:09.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455145+0000", 2024-09-15T21:47:09.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4470000000000002e-06 2024-09-15T21:47:09.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456425+0000", 2024-09-15T21:47:09.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012798200000000001 2024-09-15T21:47:09.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456433+0000", 2024-09-15T21:47:09.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7389999999999999e-06 2024-09-15T21:47:09.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:09.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097865 2024-09-15T21:47:09.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457614+0000", 2024-09-15T21:47:09.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3560000000000006e-05 2024-09-15T21:47:09.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457998+0000", 2024-09-15T21:47:09.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00038372699999999999 2024-09-15T21:47:09.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460488+0000", 2024-09-15T21:47:09.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002489855 2024-09-15T21:47:09.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462512+0000", 2024-09-15T21:47:09.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020235269999999998 2024-09-15T21:47:09.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462554+0000", 2024-09-15T21:47:09.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2472000000000003e-05 2024-09-15T21:47:09.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462597+0000", 2024-09-15T21:47:09.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2657000000000002e-05 2024-09-15T21:47:09.882 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.882 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.882 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.882 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:2 2.5 2:a6e36b3f:::401.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:09.882 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:09.882 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.279474405000002, 2024-09-15T21:47:09.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074484590000000002, 2024-09-15T21:47:09.882 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.882 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.882 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.883 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:09.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:09.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:09.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:47:09.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455150+0000", 2024-09-15T21:47:09.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7319999999999998e-06 2024-09-15T21:47:09.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456449+0000", 2024-09-15T21:47:09.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012991490000000001 2024-09-15T21:47:09.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456454+0000", 2024-09-15T21:47:09.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0390000000000004e-06 2024-09-15T21:47:09.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456513+0000", 2024-09-15T21:47:09.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8768999999999999e-05 2024-09-15T21:47:09.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.886 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456628+0000", 2024-09-15T21:47:09.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115641 2024-09-15T21:47:09.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.886 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456741+0000", 2024-09-15T21:47:09.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000113094 2024-09-15T21:47:09.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.886 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458520+0000", 2024-09-15T21:47:09.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001778993 2024-09-15T21:47:09.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462494+0000", 2024-09-15T21:47:09.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039737009999999996 2024-09-15T21:47:09.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462549+0000", 2024-09-15T21:47:09.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4537000000000003e-05 2024-09-15T21:47:09.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462595+0000", 2024-09-15T21:47:09.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6511000000000002e-05 2024-09-15T21:47:09.888 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.888 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.888 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.888 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:27 2.5 2:a5fa685f:::504.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:09.889 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:09.889 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 18.169782021, 2024-09-15T21:47:09.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071804599999999996, 2024-09-15T21:47:09.889 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:09.889 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:09.889 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:09.889 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:09.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:09.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:09.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:09.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:09.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:09.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:09.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:09.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-15T21:47:09.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:09.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564841+0000", 2024-09-15T21:47:09.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-15T21:47:09.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:09.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565126+0000", 2024-09-15T21:47:09.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028469900000000002 2024-09-15T21:47:09.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:09.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565133+0000", 2024-09-15T21:47:09.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9480000000000002e-06 2024-09-15T21:47:09.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:09.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568006+0000", 2024-09-15T21:47:09.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028730330000000001 2024-09-15T21:47:09.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:09.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568119+0000", 2024-09-15T21:47:09.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011325899999999999 2024-09-15T21:47:09.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:09.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568235+0000", 2024-09-15T21:47:09.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115868 2024-09-15T21:47:09.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:09.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569668+0000", 2024-09-15T21:47:09.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001433099 2024-09-15T21:47:09.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:09.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571926+0000", 2024-09-15T21:47:09.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022572410000000001 2024-09-15T21:47:09.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:09.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571981+0000", 2024-09-15T21:47:09.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5383000000000002e-05 2024-09-15T21:47:09.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:09.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:09.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:09.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572020+0000", 2024-09-15T21:47:09.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8574e-05 2024-09-15T21:47:09.895 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.895 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.895 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.895 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:09.895 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:09.895 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:09.896 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:47:09.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.757+0000 7f03f74f1640 1 -- 172.21.15.100:0/2044823479 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f03f00b2400 con 0x7f03d4074660 2024-09-15T21:47:09.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.757+0000 7f03e5ffb640 1 -- 172.21.15.100:0/2044823479 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f03f00b2400 con 0x7f03d4074660 2024-09-15T21:47:09.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.757+0000 7f03f74f1640 1 -- 172.21.15.100:0/2044823479 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f03d4074660 msgr2=0x7f03d4076b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.757+0000 7f03f74f1640 1 --2- 172.21.15.100:0/2044823479 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f03d4074660 0x7f03d4076b00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.758+0000 7f03f74f1640 1 -- 172.21.15.100:0/2044823479 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f03d406c2e0 msgr2=0x7f03d406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.758+0000 7f03f74f1640 1 --2- 172.21.15.100:0/2044823479 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f03d406c2e0 0x7f03d406e7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f03e0007fd0 tx=0x7f03e0007dc0 comp rx=0 tx=0).stop 2024-09-15T21:47:09.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.758+0000 7f03f74f1640 1 -- 172.21.15.100:0/2044823479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03f006c2f0 msgr2=0x7f03f0159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:09.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.758+0000 7f03f74f1640 1 --2- 172.21.15.100:0/2044823479 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03f006c2f0 0x7f03f0159310 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f03ec00c450 tx=0x7f03ec00c910 comp rx=0 tx=0).stop 2024-09-15T21:47:09.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.758+0000 7f03f5a8e640 1 -- 172.21.15.100:0/2044823479 reap_dead start 2024-09-15T21:47:09.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.758+0000 7f03f74f1640 1 -- 172.21.15.100:0/2044823479 shutdown_connections 2024-09-15T21:47:09.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.758+0000 7f03f74f1640 1 -- 172.21.15.100:0/2044823479 >> 172.21.15.100:0/2044823479 conn(0x7f03f006d890 msgr2=0x7f03f00cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:09.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.758+0000 7f03f74f1640 1 -- 172.21.15.100:0/2044823479 shutdown_connections 2024-09-15T21:47:09.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:09.758+0000 7f03f74f1640 1 -- 172.21.15.100:0/2044823479 wait complete. 2024-09-15T21:47:09.900 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-09-15T21:47:09.901 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:09.896+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:09.998 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:47:09.997+0000 7f74313af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37847) UID: 0 2024-09-15T21:47:10.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.038+0000 7f68d291f640 1 Processor -- start 2024-09-15T21:47:10.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.038+0000 7f68d291f640 1 -- start start 2024-09-15T21:47:10.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.039+0000 7f68d291f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f68cc14e9c0 0x7f68cc06df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.039+0000 7f68d291f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f68cc0c4600 0x7f68cc06ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.039+0000 7f68d291f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68cc157e30 con 0x7f68cc06d230 2024-09-15T21:47:10.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.039+0000 7f68d291f640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f68cc06e4c0 con 0x7f68cc14e9c0 2024-09-15T21:47:10.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.039+0000 7f68d291f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f68cc1580d0 con 0x7f68cc0c4600 2024-09-15T21:47:10.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.039+0000 7f68d0ebc640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f68cc06d230 0x7f68cc072f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:34016/0 (socket says 172.21.15.100:34016) 2024-09-15T21:47:10.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.039+0000 7f68d0ebc640 1 -- 172.21.15.100:0/3900983481 learned_addr learned my addr 172.21.15.100:0/3900983481 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:10.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.039+0000 7f68cbfff640 1 --2- 172.21.15.100:0/3900983481 >> v2:172.21.15.122:3300/0 conn(0x7f68cc14e9c0 0x7f68cc06df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.039+0000 7f68cb7fe640 1 --2- 172.21.15.100:0/3900983481 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f68cc0c4600 0x7f68cc06ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.039+0000 7f68caffd640 1 -- 172.21.15.100:0/3900983481 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4140583156 0 0) 0x7f68cc157e30 con 0x7f68cc06d230 2024-09-15T21:47:10.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.039+0000 7f68caffd640 1 -- 172.21.15.100:0/3900983481 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f68b8003520 con 0x7f68cc06d230 2024-09-15T21:47:10.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.039+0000 7f68caffd640 1 -- 172.21.15.100:0/3900983481 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1161711138 0 0) 0x7f68b8003520 con 0x7f68cc06d230 2024-09-15T21:47:10.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.039+0000 7f68cbfff640 1 -- 172.21.15.100:0/3900983481 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f68cc0c4600 msgr2=0x7f68cc06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:10.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.039+0000 7f68cbfff640 1 --2- 172.21.15.100:0/3900983481 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f68cc0c4600 0x7f68cc06ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.039+0000 7f68cbfff640 1 -- 172.21.15.100:0/3900983481 >> v1:172.21.15.100:6789/0 conn(0x7f68cc06d230 legacy=0x7f68cc072f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.039+0000 7f68cbfff640 1 -- 172.21.15.100:0/3900983481 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68cc1583b0 con 0x7f68cc14e9c0 2024-09-15T21:47:10.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.039+0000 7f68cb7fe640 1 --2- 172.21.15.100:0/3900983481 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f68cc0c4600 0x7f68cc06ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:10.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.040+0000 7f68cbfff640 1 --2- 172.21.15.100:0/3900983481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68cc14e9c0 0x7f68cc06df80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f68bc00ba40 tx=0x7f68bc02c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbc182414b9affcf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.040+0000 7f68caffd640 1 -- 172.21.15.100:0/3900983481 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c40019f0 con 0x7f68cc14e9c0 2024-09-15T21:47:10.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.040+0000 7f68caffd640 1 -- 172.21.15.100:0/3900983481 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68bc031510 con 0x7f68cc14e9c0 2024-09-15T21:47:10.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.040+0000 7f68caffd640 1 -- 172.21.15.100:0/3900983481 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68bc0318e0 con 0x7f68cc14e9c0 2024-09-15T21:47:10.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.041+0000 7f68d291f640 1 -- 172.21.15.100:0/3900983481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68cc14e9c0 msgr2=0x7f68cc06df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.041+0000 7f68d291f640 1 --2- 172.21.15.100:0/3900983481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68cc14e9c0 0x7f68cc06df80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f68bc00ba40 tx=0x7f68bc02c870 comp rx=0 tx=0).stop 2024-09-15T21:47:10.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.041+0000 7f68d291f640 1 -- 172.21.15.100:0/3900983481 shutdown_connections 2024-09-15T21:47:10.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.041+0000 7f68d291f640 1 --2- 172.21.15.100:0/3900983481 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f68cc0c4600 0x7f68cc06ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.041+0000 7f68d291f640 1 --2- 172.21.15.100:0/3900983481 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68cc14e9c0 0x7f68cc06df80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.041+0000 7f68d291f640 1 -- 172.21.15.100:0/3900983481 >> 172.21.15.100:0/3900983481 conn(0x7f68cc0ca430 msgr2=0x7f68cc0ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:10.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.041+0000 7f68d291f640 1 -- 172.21.15.100:0/3900983481 shutdown_connections 2024-09-15T21:47:10.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.041+0000 7f68d291f640 1 -- 172.21.15.100:0/3900983481 wait complete. 2024-09-15T21:47:10.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.042+0000 7f68d291f640 1 Processor -- start 2024-09-15T21:47:10.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.042+0000 7f68d291f640 1 -- start start 2024-09-15T21:47:10.046 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:10.046 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:10.046 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:10.046 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:10.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.043+0000 7f68d291f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68cc06d230 0x7f68cc1ecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.043+0000 7f68d291f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68cc0c4600 0x7f68cc148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.043+0000 7f68d291f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f68cc14e9c0 0x7f68cc152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.043+0000 7f68d291f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc0caf20 con 0x7f68cc0c4600 2024-09-15T21:47:10.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.043+0000 7f68d291f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f68cc158970 con 0x7f68cc06d230 2024-09-15T21:47:10.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.043+0000 7f68d291f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f68cc14eda0 con 0x7f68cc14e9c0 2024-09-15T21:47:10.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.043+0000 7f68cb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68cc0c4600 0x7f68cc148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.043+0000 7f68cb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68cc0c4600 0x7f68cc148590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49154/0 (socket says 172.21.15.100:49154) 2024-09-15T21:47:10.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.043+0000 7f68cb7fe640 1 -- 172.21.15.100:0/3804678137 learned_addr learned my addr 172.21.15.100:0/3804678137 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:10.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.043+0000 7f68cbfff640 1 --2- 172.21.15.100:0/3804678137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68cc06d230 0x7f68cc1ecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.043+0000 7f68d0ebc640 1 --2- 172.21.15.100:0/3804678137 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f68cc14e9c0 0x7f68cc152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.044+0000 7f68cb7fe640 1 -- 172.21.15.100:0/3804678137 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f68cc14e9c0 msgr2=0x7f68cc152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.044+0000 7f68cb7fe640 1 --2- 172.21.15.100:0/3804678137 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f68cc14e9c0 0x7f68cc152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.044+0000 7f68cb7fe640 1 -- 172.21.15.100:0/3804678137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68cc06d230 msgr2=0x7f68cc1ecfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.044+0000 7f68cb7fe640 1 --2- 172.21.15.100:0/3804678137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68cc06d230 0x7f68cc1ecfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.044+0000 7f68cb7fe640 1 -- 172.21.15.100:0/3804678137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68cc1ed4e0 con 0x7f68cc0c4600 2024-09-15T21:47:10.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.044+0000 7f68cb7fe640 1 --2- 172.21.15.100:0/3804678137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68cc0c4600 0x7f68cc148590 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f68b400ec10 tx=0x7f68b400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.044+0000 7f68c8ff9640 1 -- 172.21.15.100:0/3804678137 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4017070 con 0x7f68cc0c4600 2024-09-15T21:47:10.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.044+0000 7f68c8ff9640 1 -- 172.21.15.100:0/3804678137 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68b400cc30 con 0x7f68cc0c4600 2024-09-15T21:47:10.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.044+0000 7f68d291f640 1 -- 172.21.15.100:0/3804678137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68cc0c6130 con 0x7f68cc0c4600 2024-09-15T21:47:10.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.044+0000 7f68c8ff9640 1 -- 172.21.15.100:0/3804678137 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4013470 con 0x7f68cc0c4600 2024-09-15T21:47:10.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.044+0000 7f68d291f640 1 -- 172.21.15.100:0/3804678137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68cc0c6a00 con 0x7f68cc0c4600 2024-09-15T21:47:10.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.045+0000 7f68d291f640 1 -- 172.21.15.100:0/3804678137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f68cc06df80 con 0x7f68cc0c4600 2024-09-15T21:47:10.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.046+0000 7f68c8ff9640 1 -- 172.21.15.100:0/3804678137 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f68b401a070 con 0x7f68cc0c4600 2024-09-15T21:47:10.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.046+0000 7f68c8ff9640 1 --2- 172.21.15.100:0/3804678137 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f68ac06c2b0 0x7f68ac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.046+0000 7f68cbfff640 1 --2- 172.21.15.100:0/3804678137 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f68ac06c2b0 0x7f68ac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.046+0000 7f68c8ff9640 1 -- 172.21.15.100:0/3804678137 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5854+0+0 (secure 0 0 0) 0x7f68b408b3a0 con 0x7f68cc0c4600 2024-09-15T21:47:10.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.047+0000 7f68c8ff9640 1 --2- 172.21.15.100:0/3804678137 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f68ac074630 0x7f68ac076ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.047+0000 7f68c8ff9640 1 -- 172.21.15.100:0/3804678137 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f68b40093e0 con 0x7f68ac074630 2024-09-15T21:47:10.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.047+0000 7f68c8ff9640 1 -- 172.21.15.100:0/3804678137 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f68b4092050 con 0x7f68cc0c4600 2024-09-15T21:47:10.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.047+0000 7f68d0ebc640 1 --2- 172.21.15.100:0/3804678137 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f68ac074630 0x7f68ac076ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.047+0000 7f68cbfff640 1 --2- 172.21.15.100:0/3804678137 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f68ac06c2b0 0x7f68ac06e770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0ca3e0 tx=0x7f68bc031000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.047+0000 7f68d0ebc640 1 --2- 172.21.15.100:0/3804678137 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f68ac074630 0x7f68ac076ad0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.048+0000 7f68c8ff9640 1 -- 172.21.15.100:0/3804678137 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f68b40093e0 con 0x7f68ac074630 2024-09-15T21:47:10.057 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:47:10.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.066+0000 7f68d291f640 1 -- 172.21.15.100:0/3804678137 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f68cc1e2150 con 0x7f68ac074630 2024-09-15T21:47:10.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.067+0000 7f68c8ff9640 1 -- 172.21.15.100:0/3804678137 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f68cc1e2150 con 0x7f68ac074630 2024-09-15T21:47:10.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.067+0000 7f68d291f640 1 -- 172.21.15.100:0/3804678137 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f68ac074630 msgr2=0x7f68ac076ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.067+0000 7f68d291f640 1 --2- 172.21.15.100:0/3804678137 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f68ac074630 0x7f68ac076ad0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.067+0000 7f68d291f640 1 -- 172.21.15.100:0/3804678137 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f68ac06c2b0 msgr2=0x7f68ac06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.067+0000 7f68d291f640 1 --2- 172.21.15.100:0/3804678137 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f68ac06c2b0 0x7f68ac06e770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0ca3e0 tx=0x7f68bc031000 comp rx=0 tx=0).stop 2024-09-15T21:47:10.069 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.067+0000 7f68d291f640 1 -- 172.21.15.100:0/3804678137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68cc0c4600 msgr2=0x7f68cc148590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.067+0000 7f68d291f640 1 --2- 172.21.15.100:0/3804678137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68cc0c4600 0x7f68cc148590 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f68b400ec10 tx=0x7f68b400c520 comp rx=0 tx=0).stop 2024-09-15T21:47:10.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.067+0000 7f68d0ebc640 1 -- 172.21.15.100:0/3804678137 reap_dead start 2024-09-15T21:47:10.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.067+0000 7f68d291f640 1 -- 172.21.15.100:0/3804678137 shutdown_connections 2024-09-15T21:47:10.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.067+0000 7f68d291f640 1 -- 172.21.15.100:0/3804678137 >> 172.21.15.100:0/3804678137 conn(0x7f68cc0ca430 msgr2=0x7f68cc149670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:10.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.067+0000 7f68d291f640 1 -- 172.21.15.100:0/3804678137 shutdown_connections 2024-09-15T21:47:10.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.067+0000 7f68d291f640 1 -- 172.21.15.100:0/3804678137 wait complete. 2024-09-15T21:47:10.079 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-15T21:47:10.121 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:10.119+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:10.162 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:10.162 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:10.162 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:10.162 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:10.163 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:10.171 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:47:10.214 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:10.212+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:10.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.219+0000 7f7ed46d0640 1 Processor -- start 2024-09-15T21:47:10.221 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.219+0000 7f7ed46d0640 1 -- start start 2024-09-15T21:47:10.222 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.220+0000 7f7ed46d0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7ecc0c6ff0 0x7f7ecc0ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.220+0000 7f7ed46d0640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7ecc0718b0 0x7f7ecc06f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.220+0000 7f7ed46d0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ecc0cd0c0 con 0x7f7ecc071ce0 2024-09-15T21:47:10.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.220+0000 7f7ed46d0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f7ecc0cd360 con 0x7f7ecc0c6ff0 2024-09-15T21:47:10.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.220+0000 7f7ed46d0640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f7ecc0cd640 con 0x7f7ecc0718b0 2024-09-15T21:47:10.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.220+0000 7f7ed2c6d640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f7ecc071ce0 0x7f7ecc15b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:34018/0 (socket says 172.21.15.100:34018) 2024-09-15T21:47:10.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.220+0000 7f7ed2c6d640 1 -- 172.21.15.100:0/3679573071 learned_addr learned my addr 172.21.15.100:0/3679573071 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:10.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.220+0000 7f7ed246c640 1 --2- 172.21.15.100:0/3679573071 >> v2:172.21.15.100:3300/0 conn(0x7f7ecc0c6ff0 0x7f7ecc0ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.221+0000 7f7ed1c6b640 1 --2- 172.21.15.100:0/3679573071 >> v2:172.21.15.122:3300/0 conn(0x7f7ecc0718b0 0x7f7ecc06f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.221+0000 7f7ed246c640 1 -- 172.21.15.100:0/3679573071 >> v2:172.21.15.122:3300/0 conn(0x7f7ecc0718b0 msgr2=0x7f7ecc06f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:10.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.221+0000 7f7ed246c640 1 --2- 172.21.15.100:0/3679573071 >> v2:172.21.15.122:3300/0 conn(0x7f7ecc0718b0 0x7f7ecc06f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.223 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.221+0000 7f7ed246c640 1 -- 172.21.15.100:0/3679573071 >> v1:172.21.15.100:6789/0 conn(0x7f7ecc071ce0 legacy=0x7f7ecc15b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.221+0000 7f7ed146a640 1 -- 172.21.15.100:0/3679573071 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 224402888 0 0) 0x7f7ecc0cd0c0 con 0x7f7ecc071ce0 2024-09-15T21:47:10.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.221+0000 7f7ed246c640 1 -- 172.21.15.100:0/3679573071 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc0cd920 con 0x7f7ecc0c6ff0 2024-09-15T21:47:10.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.221+0000 7f7ed246c640 1 --2- 172.21.15.100:0/3679573071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ecc0c6ff0 0x7f7ecc0ca090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0737f0 tx=0x7f7ebc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385a2c4c6ebcb0b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.225 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.222+0000 7f7ed146a640 1 -- 172.21.15.100:0/3679573071 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc037070 con 0x7f7ecc0c6ff0 2024-09-15T21:47:10.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.222+0000 7f7ed146a640 1 -- 172.21.15.100:0/3679573071 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ebc032440 con 0x7f7ecc0c6ff0 2024-09-15T21:47:10.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.222+0000 7f7ed146a640 1 -- 172.21.15.100:0/3679573071 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc02ae60 con 0x7f7ecc0c6ff0 2024-09-15T21:47:10.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.222+0000 7f7ed46d0640 1 -- 172.21.15.100:0/3679573071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ecc0c6ff0 msgr2=0x7f7ecc0ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.222+0000 7f7ed46d0640 1 --2- 172.21.15.100:0/3679573071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ecc0c6ff0 0x7f7ecc0ca090 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0737f0 tx=0x7f7ebc02a110 comp rx=0 tx=0).stop 2024-09-15T21:47:10.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.222+0000 7f7ed46d0640 1 -- 172.21.15.100:0/3679573071 shutdown_connections 2024-09-15T21:47:10.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.222+0000 7f7ed46d0640 1 --2- 172.21.15.100:0/3679573071 >> v2:172.21.15.122:3300/0 conn(0x7f7ecc0718b0 0x7f7ecc06f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.222+0000 7f7ed46d0640 1 --2- 172.21.15.100:0/3679573071 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ecc0c6ff0 0x7f7ecc0ca090 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.222+0000 7f7ed46d0640 1 -- 172.21.15.100:0/3679573071 >> 172.21.15.100:0/3679573071 conn(0x7f7ecc073820 msgr2=0x7f7ecc073c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:10.226 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.223+0000 7f7ed46d0640 1 -- 172.21.15.100:0/3679573071 shutdown_connections 2024-09-15T21:47:10.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.223+0000 7f7ed46d0640 1 -- 172.21.15.100:0/3679573071 wait complete. 2024-09-15T21:47:10.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.224+0000 7f7ed46d0640 1 Processor -- start 2024-09-15T21:47:10.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.224+0000 7f7ed46d0640 1 -- start start 2024-09-15T21:47:10.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.224+0000 7f7ed46d0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7ecc0718b0 0x7f7ecc1f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.224+0000 7f7ed46d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ecc071ce0 0x7f7ecc1e70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.224+0000 7f7ed46d0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ecc0c6ff0 0x7f7ecc1e44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.224+0000 7f7ed46d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc0c73d0 con 0x7f7ecc071ce0 2024-09-15T21:47:10.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.224+0000 7f7ed46d0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc0c8630 con 0x7f7ecc0c6ff0 2024-09-15T21:47:10.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.224+0000 7f7ed46d0640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7ecc0c7550 con 0x7f7ecc0718b0 2024-09-15T21:47:10.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.225+0000 7f7ed1c6b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ecc071ce0 0x7f7ecc1e70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.225+0000 7f7ed1c6b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ecc071ce0 0x7f7ecc1e70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49174/0 (socket says 172.21.15.100:49174) 2024-09-15T21:47:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.225+0000 7f7ed1c6b640 1 -- 172.21.15.100:0/1114636454 learned_addr learned my addr 172.21.15.100:0/1114636454 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.225+0000 7f7ed2c6d640 1 --2- 172.21.15.100:0/1114636454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ecc0c6ff0 0x7f7ecc1e44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.225+0000 7f7ed246c640 1 --2- 172.21.15.100:0/1114636454 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7ecc0718b0 0x7f7ecc1f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.225+0000 7f7ed1c6b640 1 -- 172.21.15.100:0/1114636454 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7ecc0718b0 msgr2=0x7f7ecc1f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.225+0000 7f7ed1c6b640 1 --2- 172.21.15.100:0/1114636454 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7ecc0718b0 0x7f7ecc1f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.225+0000 7f7ed1c6b640 1 -- 172.21.15.100:0/1114636454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ecc0c6ff0 msgr2=0x7f7ecc1e44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.225+0000 7f7ed1c6b640 1 --2- 172.21.15.100:0/1114636454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ecc0c6ff0 0x7f7ecc1e44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.225+0000 7f7ed1c6b640 1 -- 172.21.15.100:0/1114636454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc13e0b0 con 0x7f7ecc071ce0 2024-09-15T21:47:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.225+0000 7f7ed246c640 1 --2- 172.21.15.100:0/1114636454 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7ecc0718b0 0x7f7ecc1f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:47:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.225+0000 7f7ed2c6d640 1 --2- 172.21.15.100:0/1114636454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ecc0c6ff0 0x7f7ecc1e44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:47:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.225+0000 7f7ed1c6b640 1 --2- 172.21.15.100:0/1114636454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ecc071ce0 0x7f7ecc1e70e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800e870 tx=0x7f7ec800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.225+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/1114636454 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8017070 con 0x7f7ecc071ce0 2024-09-15T21:47:10.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.225+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/1114636454 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ec800c940 con 0x7f7ecc071ce0 2024-09-15T21:47:10.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.225+0000 7f7ed46d0640 1 -- 172.21.15.100:0/1114636454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ecc13e600 con 0x7f7ecc071ce0 2024-09-15T21:47:10.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.225+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/1114636454 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec800cd30 con 0x7f7ecc071ce0 2024-09-15T21:47:10.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.225+0000 7f7ed46d0640 1 -- 172.21.15.100:0/1114636454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ecc13b6a0 con 0x7f7ecc071ce0 2024-09-15T21:47:10.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.226+0000 7f7ed46d0640 1 -- 172.21.15.100:0/1114636454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7ecc0ca090 con 0x7f7ecc071ce0 2024-09-15T21:47:10.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.227+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/1114636454 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7ec801a070 con 0x7f7ecc071ce0 2024-09-15T21:47:10.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.227+0000 7f7ec2ffd640 1 --2- 172.21.15.100:0/1114636454 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7eb006c2b0 0x7f7eb006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.228+0000 7f7ed246c640 1 --2- 172.21.15.100:0/1114636454 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7eb006c2b0 0x7f7eb006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.228+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/1114636454 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5854+0+0 (secure 0 0 0) 0x7f7ec808b780 con 0x7f7ecc071ce0 2024-09-15T21:47:10.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.228+0000 7f7ec2ffd640 1 --2- 172.21.15.100:0/1114636454 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f7eb0074630 0x7f7eb0076ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.228+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/1114636454 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7eb00771a0 con 0x7f7eb0074630 2024-09-15T21:47:10.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.228+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/1114636454 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7ec808bc10 con 0x7f7ecc071ce0 2024-09-15T21:47:10.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.228+0000 7f7ed2c6d640 1 --2- 172.21.15.100:0/1114636454 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f7eb0074630 0x7f7eb0076ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.228+0000 7f7ed246c640 1 --2- 172.21.15.100:0/1114636454 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7eb006c2b0 0x7f7eb006e770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc034000 tx=0x7f7ebc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.228+0000 7f7ed2c6d640 1 --2- 172.21.15.100:0/1114636454 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f7eb0074630 0x7f7eb0076ad0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.229+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/1114636454 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7eb00771a0 con 0x7f7eb0074630 2024-09-15T21:47:10.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.245+0000 7f7ed46d0640 1 -- 172.21.15.100:0/1114636454 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f7ecc0ca840 con 0x7f7eb0074630 2024-09-15T21:47:10.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.246+0000 7f7ec2ffd640 1 -- 172.21.15.100:0/1114636454 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f7ecc0ca840 con 0x7f7eb0074630 2024-09-15T21:47:10.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.246+0000 7f7ed46d0640 1 -- 172.21.15.100:0/1114636454 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f7eb0074630 msgr2=0x7f7eb0076ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.246+0000 7f7ed46d0640 1 --2- 172.21.15.100:0/1114636454 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f7eb0074630 0x7f7eb0076ad0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.246+0000 7f7ed46d0640 1 -- 172.21.15.100:0/1114636454 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7eb006c2b0 msgr2=0x7f7eb006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.246+0000 7f7ed46d0640 1 --2- 172.21.15.100:0/1114636454 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7eb006c2b0 0x7f7eb006e770 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc034000 tx=0x7f7ebc02e000 comp rx=0 tx=0).stop 2024-09-15T21:47:10.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.246+0000 7f7ed46d0640 1 -- 172.21.15.100:0/1114636454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ecc071ce0 msgr2=0x7f7ecc1e70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.246+0000 7f7ed46d0640 1 --2- 172.21.15.100:0/1114636454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7ecc071ce0 0x7f7ecc1e70e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800e870 tx=0x7f7ec800ed30 comp rx=0 tx=0).stop 2024-09-15T21:47:10.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.246+0000 7f7ed2c6d640 1 -- 172.21.15.100:0/1114636454 reap_dead start 2024-09-15T21:47:10.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.246+0000 7f7ed46d0640 1 -- 172.21.15.100:0/1114636454 shutdown_connections 2024-09-15T21:47:10.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.246+0000 7f7ed46d0640 1 -- 172.21.15.100:0/1114636454 >> 172.21.15.100:0/1114636454 conn(0x7f7ecc073820 msgr2=0x7f7ecc1439a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:10.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.246+0000 7f7ed46d0640 1 -- 172.21.15.100:0/1114636454 shutdown_connections 2024-09-15T21:47:10.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.246+0000 7f7ed46d0640 1 -- 172.21.15.100:0/1114636454 wait complete. 2024-09-15T21:47:10.258 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-15T21:47:10.302 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:10.300+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:10.323 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:10.323 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:10.323 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:10.323 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:10.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.324 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:32 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'18, mlcod=17'14)", 2024-09-15T21:47:10.324 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:10.324 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.962645510999998, 2024-09-15T21:47:10.324 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052297760000000002, 2024-09-15T21:47:10.324 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.324 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.325 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.325 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.325 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.329 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:10.329 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.06e-07 2024-09-15T21:47:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.330 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356780+0000", 2024-09-15T21:47:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4208999999999994e-05 2024-09-15T21:47:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356789+0000", 2024-09-15T21:47:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2800000000000003e-06 2024-09-15T21:47:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356793+0000", 2024-09-15T21:47:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:47:10.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356838+0000", 2024-09-15T21:47:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4851999999999997e-05 2024-09-15T21:47:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356854+0000", 2024-09-15T21:47:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6124999999999999e-05 2024-09-15T21:47:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.361892+0000", 2024-09-15T21:47:10.332 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050379140000000001 2024-09-15T21:47:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.361935+0000", 2024-09-15T21:47:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3217999999999999e-05 2024-09-15T21:47:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:35 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'21, mlcod=17'15)", 2024-09-15T21:47:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.960608381, 2024-09-15T21:47:10.333 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055652039999999998, 2024-09-15T21:47:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358744+0000", 2024-09-15T21:47:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4399999999999998e-07 2024-09-15T21:47:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358773+0000", 2024-09-15T21:47:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8847e-05 2024-09-15T21:47:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358783+0000", 2024-09-15T21:47:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0299000000000001e-05 2024-09-15T21:47:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358796+0000", 2024-09-15T21:47:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3499000000000001e-05 2024-09-15T21:47:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358850+0000", 2024-09-15T21:47:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3952000000000001e-05 2024-09-15T21:47:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.336 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358872+0000", 2024-09-15T21:47:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1424999999999999e-05 2024-09-15T21:47:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364273+0000", 2024-09-15T21:47:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054011299999999996 2024-09-15T21:47:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364308+0000", 2024-09-15T21:47:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5108000000000003e-05 2024-09-15T21:47:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.337 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:36 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'22, mlcod=17'15)", 2024-09-15T21:47:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.959814191, 2024-09-15T21:47:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047904530000000001, 2024-09-15T21:47:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.338 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6000000000000004e-07 2024-09-15T21:47:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.339 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359542+0000", 2024-09-15T21:47:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3510000000000004e-06 2024-09-15T21:47:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359552+0000", 2024-09-15T21:47:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0263e-05 2024-09-15T21:47:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359557+0000", 2024-09-15T21:47:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0479999999999998e-06 2024-09-15T21:47:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359593+0000", 2024-09-15T21:47:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6151e-05 2024-09-15T21:47:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359608+0000", 2024-09-15T21:47:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4609e-05 2024-09-15T21:47:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364310+0000", 2024-09-15T21:47:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047025510000000001 2024-09-15T21:47:10.341 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364327+0000", 2024-09-15T21:47:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6920000000000001e-05 2024-09-15T21:47:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=18'85)", 2024-09-15T21:47:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 54.421268255999998, 2024-09-15T21:47:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057443499999999996, 2024-09-15T21:47:10.342 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.343 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8899999999999999e-07 2024-09-15T21:47:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898167+0000", 2024-09-15T21:47:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3171999999999996e-05 2024-09-15T21:47:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898177+0000", 2024-09-15T21:47:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0866000000000001e-05 2024-09-15T21:47:10.344 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898185+0000", 2024-09-15T21:47:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2490000000000004e-06 2024-09-15T21:47:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898224+0000", 2024-09-15T21:47:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9237999999999999e-05 2024-09-15T21:47:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898238+0000", 2024-09-15T21:47:10.345 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.366e-05 2024-09-15T21:47:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.903803+0000", 2024-09-15T21:47:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005565291 2024-09-15T21:47:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.903827+0000", 2024-09-15T21:47:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4284999999999999e-05 2024-09-15T21:47:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.346 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:132 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'90, mlcod=18'85)", 2024-09-15T21:47:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 54.420776531999998, 2024-09-15T21:47:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055085469999999999, 2024-09-15T21:47:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.347 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9299999999999999e-07 2024-09-15T21:47:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898579+0000", 2024-09-15T21:47:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6650000000000001e-06 2024-09-15T21:47:10.348 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898586+0000", 2024-09-15T21:47:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8429999999999993e-06 2024-09-15T21:47:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898592+0000", 2024-09-15T21:47:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3990000000000003e-06 2024-09-15T21:47:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898621+0000", 2024-09-15T21:47:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9019000000000001e-05 2024-09-15T21:47:10.349 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898630+0000", 2024-09-15T21:47:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5540000000000003e-06 2024-09-15T21:47:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904047+0000", 2024-09-15T21:47:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054167169999999997 2024-09-15T21:47:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904083+0000", 2024-09-15T21:47:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5957000000000001e-05 2024-09-15T21:47:10.350 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-15T21:47:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 54.420187382999998, 2024-09-15T21:47:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049424780000000001, 2024-09-15T21:47:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.351 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899166+0000", 2024-09-15T21:47:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0779999999999998e-06 2024-09-15T21:47:10.352 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899174+0000", 2024-09-15T21:47:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.729e-06 2024-09-15T21:47:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899181+0000", 2024-09-15T21:47:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4080000000000001e-06 2024-09-15T21:47:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899185+0000", 2024-09-15T21:47:10.353 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0840000000000002e-06 2024-09-15T21:47:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899240+0000", 2024-09-15T21:47:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4874999999999999e-05 2024-09-15T21:47:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899257+0000", 2024-09-15T21:47:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7549000000000001e-05 2024-09-15T21:47:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904085+0000", 2024-09-15T21:47:10.354 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048272640000000004 2024-09-15T21:47:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904106+0000", 2024-09-15T21:47:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1491000000000001e-05 2024-09-15T21:47:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 41.905964107000003, 2024-09-15T21:47:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046608459999999997, 2024-09-15T21:47:10.355 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.356 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.356 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.356 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.356 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.356 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.356 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:10.356 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.356 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.356 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413391+0000", 2024-09-15T21:47:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9899999999999999e-06 2024-09-15T21:47:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413399+0000", 2024-09-15T21:47:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6310000000000003e-06 2024-09-15T21:47:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.357 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.358 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.358 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413444+0000", 2024-09-15T21:47:10.358 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5052000000000002e-05 2024-09-15T21:47:10.358 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.358 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.358 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.358 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413453+0000", 2024-09-15T21:47:10.358 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8869999999999998e-06 2024-09-15T21:47:10.358 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.358 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.358 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413576+0000", 2024-09-15T21:47:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012358700000000001 2024-09-15T21:47:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413679+0000", 2024-09-15T21:47:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000103318 2024-09-15T21:47:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413785+0000", 2024-09-15T21:47:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010561 2024-09-15T21:47:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.359 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416276+0000", 2024-09-15T21:47:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002490754 2024-09-15T21:47:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417949+0000", 2024-09-15T21:47:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016733519999999999 2024-09-15T21:47:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418005+0000", 2024-09-15T21:47:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5747e-05 2024-09-15T21:47:10.360 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418048+0000", 2024-09-15T21:47:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2917999999999998e-05 2024-09-15T21:47:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 41.905949636999999, 2024-09-15T21:47:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051419999999999999, 2024-09-15T21:47:10.361 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.362 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413402+0000", 2024-09-15T21:47:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6899999999999998e-07 2024-09-15T21:47:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413405+0000", 2024-09-15T21:47:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8109999999999999e-06 2024-09-15T21:47:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413517+0000", 2024-09-15T21:47:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000112592 2024-09-15T21:47:10.363 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413527+0000", 2024-09-15T21:47:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0144e-05 2024-09-15T21:47:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414155+0000", 2024-09-15T21:47:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00062767200000000002 2024-09-15T21:47:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414239+0000", 2024-09-15T21:47:10.364 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3607000000000001e-05 2024-09-15T21:47:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414368+0000", 2024-09-15T21:47:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000129049 2024-09-15T21:47:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416332+0000", 2024-09-15T21:47:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019645109999999999 2024-09-15T21:47:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.365 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418446+0000", 2024-09-15T21:47:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021137640000000002 2024-09-15T21:47:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418503+0000", 2024-09-15T21:47:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7166999999999997e-05 2024-09-15T21:47:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418543+0000", 2024-09-15T21:47:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0314000000000001e-05 2024-09-15T21:47:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.366 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 41.905936445999998, 2024-09-15T21:47:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057945979999999998, 2024-09-15T21:47:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.367 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9499999999999998e-07 2024-09-15T21:47:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413417+0000", 2024-09-15T21:47:10.368 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3910000000000001e-06 2024-09-15T21:47:10.369 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.369 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.369 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.369 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414570+0000", 2024-09-15T21:47:10.369 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011525890000000001 2024-09-15T21:47:10.369 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.369 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.369 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.369 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414577+0000", 2024-09-15T21:47:10.369 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1840000000000002e-06 2024-09-15T21:47:10.369 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414733+0000", 2024-09-15T21:47:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015563900000000001 2024-09-15T21:47:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414831+0000", 2024-09-15T21:47:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8138000000000001e-05 2024-09-15T21:47:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:10.370 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414956+0000", 2024-09-15T21:47:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000124604 2024-09-15T21:47:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418804+0000", 2024-09-15T21:47:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038484299999999999 2024-09-15T21:47:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419124+0000", 2024-09-15T21:47:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00032017699999999999 2024-09-15T21:47:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.371 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419171+0000", 2024-09-15T21:47:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6520000000000002e-05 2024-09-15T21:47:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419209+0000", 2024-09-15T21:47:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8531000000000001e-05 2024-09-15T21:47:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.372 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 41.905929350000001, 2024-09-15T21:47:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059183300000000003, 2024-09-15T21:47:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.373 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3000000000000002e-07 2024-09-15T21:47:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413426+0000", 2024-09-15T21:47:10.374 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7110000000000001e-06 2024-09-15T21:47:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414621+0000", 2024-09-15T21:47:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001195022 2024-09-15T21:47:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414625+0000", 2024-09-15T21:47:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6360000000000002e-06 2024-09-15T21:47:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.375 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415269+0000", 2024-09-15T21:47:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00064315399999999995 2024-09-15T21:47:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415356+0000", 2024-09-15T21:47:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7410999999999994e-05 2024-09-15T21:47:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419340+0000", 2024-09-15T21:47:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0039840659999999996 2024-09-15T21:47:10.376 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-15T21:47:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 41.905621758999999, 2024-09-15T21:47:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053379719999999999, 2024-09-15T21:47:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.377 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.378 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:10.378 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.378 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.378 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.378 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.378 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:10.378 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.378 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.378 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.378 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.378 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413730+0000", 2024-09-15T21:47:10.378 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1379999999999999e-06 2024-09-15T21:47:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413740+0000", 2024-09-15T21:47:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4830000000000005e-06 2024-09-15T21:47:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413751+0000", 2024-09-15T21:47:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1228e-05 2024-09-15T21:47:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.379 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413758+0000", 2024-09-15T21:47:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1289999999999998e-06 2024-09-15T21:47:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414038+0000", 2024-09-15T21:47:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00027935999999999998 2024-09-15T21:47:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414055+0000", 2024-09-15T21:47:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7325000000000001e-05 2024-09-15T21:47:10.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419028+0000", 2024-09-15T21:47:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049730720000000003 2024-09-15T21:47:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419067+0000", 2024-09-15T21:47:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9236999999999997e-05 2024-09-15T21:47:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.381 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 35.863698491999997, 2024-09-15T21:47:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.92695576499999999, 2024-09-15T21:47:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.382 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8500000000000002e-07 2024-09-15T21:47:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455655+0000", 2024-09-15T21:47:10.383 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.401e-06 2024-09-15T21:47:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.380965+0000", 2024-09-15T21:47:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.92530946000000003 2024-09-15T21:47:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.380981+0000", 2024-09-15T21:47:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6209e-05 2024-09-15T21:47:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.384 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381086+0000", 2024-09-15T21:47:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105011 2024-09-15T21:47:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381154+0000", 2024-09-15T21:47:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7596999999999997e-05 2024-09-15T21:47:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381249+0000", 2024-09-15T21:47:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5354999999999998e-05 2024-09-15T21:47:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.385 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381844+0000", 2024-09-15T21:47:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00059518600000000004 2024-09-15T21:47:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382531+0000", 2024-09-15T21:47:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00068671499999999996 2024-09-15T21:47:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382570+0000", 2024-09-15T21:47:10.386 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9373999999999999e-05 2024-09-15T21:47:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382608+0000", 2024-09-15T21:47:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8071999999999997e-05 2024-09-15T21:47:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:10.387 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 29.843789232999999, 2024-09-15T21:47:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047117319999999997, 2024-09-15T21:47:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.388 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4700000000000001e-07 2024-09-15T21:47:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475566+0000", 2024-09-15T21:47:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1669999999999999e-06 2024-09-15T21:47:10.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476316+0000", 2024-09-15T21:47:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075022999999999995 2024-09-15T21:47:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476324+0000", 2024-09-15T21:47:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.259e-06 2024-09-15T21:47:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.390 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476373+0000", 2024-09-15T21:47:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8807999999999998e-05 2024-09-15T21:47:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476446+0000", 2024-09-15T21:47:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3138000000000003e-05 2024-09-15T21:47:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476593+0000", 2024-09-15T21:47:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00014680199999999999 2024-09-15T21:47:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.391 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480189+0000", 2024-09-15T21:47:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035961370000000001 2024-09-15T21:47:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480198+0000", 2024-09-15T21:47:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.6649999999999992e-06 2024-09-15T21:47:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480235+0000", 2024-09-15T21:47:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6943999999999997e-05 2024-09-15T21:47:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.392 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480274+0000", 2024-09-15T21:47:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9035000000000001e-05 2024-09-15T21:47:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 29.843784230000001, 2024-09-15T21:47:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046830109999999999, 2024-09-15T21:47:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.393 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.394 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.53e-07 2024-09-15T21:47:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475570+0000", 2024-09-15T21:47:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7470000000000001e-06 2024-09-15T21:47:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476353+0000", 2024-09-15T21:47:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00078312300000000004 2024-09-15T21:47:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.395 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476359+0000", 2024-09-15T21:47:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.117e-06 2024-09-15T21:47:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476412+0000", 2024-09-15T21:47:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2398999999999998e-05 2024-09-15T21:47:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476524+0000", 2024-09-15T21:47:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011293 2024-09-15T21:47:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.396 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476909+0000", 2024-09-15T21:47:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00038490099999999999 2024-09-15T21:47:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479761+0000", 2024-09-15T21:47:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002851737 2024-09-15T21:47:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480154+0000", 2024-09-15T21:47:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00039307399999999998 2024-09-15T21:47:10.397 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480203+0000", 2024-09-15T21:47:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8418000000000003e-05 2024-09-15T21:47:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480250+0000", 2024-09-15T21:47:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7311999999999997e-05 2024-09-15T21:47:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.398 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 29.843780068000001, 2024-09-15T21:47:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005035398, 2024-09-15T21:47:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.399 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.22e-07 2024-09-15T21:47:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475574+0000", 2024-09-15T21:47:10.400 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.593e-06 2024-09-15T21:47:10.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.401 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.401 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.401 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476579+0000", 2024-09-15T21:47:10.401 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010053989999999999 2024-09-15T21:47:10.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.401 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.401 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.401 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476588+0000", 2024-09-15T21:47:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4530000000000002e-06 2024-09-15T21:47:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477723+0000", 2024-09-15T21:47:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001134962 2024-09-15T21:47:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477801+0000", 2024-09-15T21:47:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8228999999999999e-05 2024-09-15T21:47:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.402 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477946+0000", 2024-09-15T21:47:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00014553400000000001 2024-09-15T21:47:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480258+0000", 2024-09-15T21:47:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023112039999999999 2024-09-15T21:47:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480504+0000", 2024-09-15T21:47:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024667500000000002 2024-09-15T21:47:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.403 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480556+0000", 2024-09-15T21:47:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2114999999999998e-05 2024-09-15T21:47:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480606+0000", 2024-09-15T21:47:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0012000000000002e-05 2024-09-15T21:47:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.404 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 29.843776137999999, 2024-09-15T21:47:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050672490000000002, 2024-09-15T21:47:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.405 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475577+0000", 2024-09-15T21:47:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2290000000000002e-06 2024-09-15T21:47:10.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476605+0000", 2024-09-15T21:47:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010278780000000001 2024-09-15T21:47:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476611+0000", 2024-09-15T21:47:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5600000000000001e-06 2024-09-15T21:47:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477053+0000", 2024-09-15T21:47:10.407 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044162800000000001 2024-09-15T21:47:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477180+0000", 2024-09-15T21:47:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000127129 2024-09-15T21:47:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477288+0000", 2024-09-15T21:47:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010789999999999999 2024-09-15T21:47:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.408 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480288+0000", 2024-09-15T21:47:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003000052 2024-09-15T21:47:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480568+0000", 2024-09-15T21:47:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028026899999999998 2024-09-15T21:47:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480609+0000", 2024-09-15T21:47:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1437999999999999e-05 2024-09-15T21:47:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.409 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480642+0000", 2024-09-15T21:47:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2879999999999997e-05 2024-09-15T21:47:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:20 2.1f 2:fdd45aa0:::61c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 29.843772467000001, 2024-09-15T21:47:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053112589999999996, 2024-09-15T21:47:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.410 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:10.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475581+0000", 2024-09-15T21:47:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1009999999999999e-06 2024-09-15T21:47:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476628+0000", 2024-09-15T21:47:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010465380000000001 2024-09-15T21:47:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476633+0000", 2024-09-15T21:47:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9869999999999999e-06 2024-09-15T21:47:10.412 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477618+0000", 2024-09-15T21:47:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00098539299999999999 2024-09-15T21:47:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477696+0000", 2024-09-15T21:47:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8509000000000003e-05 2024-09-15T21:47:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:10.413 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477824+0000", 2024-09-15T21:47:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000127062 2024-09-15T21:47:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480294+0000", 2024-09-15T21:47:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024703939999999999 2024-09-15T21:47:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480788+0000", 2024-09-15T21:47:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00049443800000000004 2024-09-15T21:47:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.414 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480845+0000", 2024-09-15T21:47:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6762000000000003e-05 2024-09-15T21:47:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480890+0000", 2024-09-15T21:47:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4789000000000001e-05 2024-09-15T21:47:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.415 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 29.843768996000001, 2024-09-15T21:47:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007191546, 2024-09-15T21:47:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:10.416 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9999999999999999e-07 2024-09-15T21:47:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475585+0000", 2024-09-15T21:47:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.384e-06 2024-09-15T21:47:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.417 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476665+0000", 2024-09-15T21:47:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010797140000000001 2024-09-15T21:47:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476671+0000", 2024-09-15T21:47:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3940000000000001e-06 2024-09-15T21:47:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478048+0000", 2024-09-15T21:47:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013767530000000001 2024-09-15T21:47:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.418 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478177+0000", 2024-09-15T21:47:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012898099999999999 2024-09-15T21:47:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482774+0000", 2024-09-15T21:47:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045970200000000003 2024-09-15T21:47:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.419 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 24.774539331, 2024-09-15T21:47:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055444480000000004, 2024-09-15T21:47:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.420 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4200000000000002e-07 2024-09-15T21:47:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544814+0000", 2024-09-15T21:47:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9980000000000002e-06 2024-09-15T21:47:10.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544872+0000", 2024-09-15T21:47:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8405000000000001e-05 2024-09-15T21:47:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544877+0000", 2024-09-15T21:47:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.143e-06 2024-09-15T21:47:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545750+0000", 2024-09-15T21:47:10.422 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00087310000000000003 2024-09-15T21:47:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545850+0000", 2024-09-15T21:47:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9939999999999995e-05 2024-09-15T21:47:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546014+0000", 2024-09-15T21:47:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00016408500000000001 2024-09-15T21:47:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.548748+0000", 2024-09-15T21:47:10.423 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027339539999999998 2024-09-15T21:47:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550265+0000", 2024-09-15T21:47:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015170019999999999 2024-09-15T21:47:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550315+0000", 2024-09-15T21:47:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9867999999999998e-05 2024-09-15T21:47:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550356+0000", 2024-09-15T21:47:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1711000000000001e-05 2024-09-15T21:47:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 24.774536130000001, 2024-09-15T21:47:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058729619999999998, 2024-09-15T21:47:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.425 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0599999999999999e-07 2024-09-15T21:47:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544821+0000", 2024-09-15T21:47:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4079999999999997e-06 2024-09-15T21:47:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544939+0000", 2024-09-15T21:47:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118532 2024-09-15T21:47:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544946+0000", 2024-09-15T21:47:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9260000000000003e-06 2024-09-15T21:47:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545011+0000", 2024-09-15T21:47:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4572000000000005e-05 2024-09-15T21:47:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545134+0000", 2024-09-15T21:47:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012309899999999999 2024-09-15T21:47:10.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.429 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.429 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:10.429 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545271+0000", 2024-09-15T21:47:10.429 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013746599999999999 2024-09-15T21:47:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550243+0000", 2024-09-15T21:47:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049714529999999998 2024-09-15T21:47:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550605+0000", 2024-09-15T21:47:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00036260699999999998 2024-09-15T21:47:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550662+0000", 2024-09-15T21:47:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6573000000000002e-05 2024-09-15T21:47:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550688+0000", 2024-09-15T21:47:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6120000000000001e-05 2024-09-15T21:47:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.431 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.431 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:10.433 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:47:10.437 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:10.431+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:10.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.397+0000 7f9caea53640 1 Processor -- start 2024-09-15T21:47:10.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.397+0000 7f9caea53640 1 -- start start 2024-09-15T21:47:10.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9caea53640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9ca80cca30 0x7f9ca8071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9caea53640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ca8070ee0 0x7f9ca8151450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9caea53640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ca80c91f0 con 0x7f9ca80cce10 2024-09-15T21:47:10.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9caea53640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f9ca80c94d0 con 0x7f9ca80cca30 2024-09-15T21:47:10.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9caea53640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca80c97b0 con 0x7f9ca8070ee0 2024-09-15T21:47:10.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9cacff0640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f9ca80cce10 0x7f9ca815af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:34022/0 (socket says 172.21.15.100:34022) 2024-09-15T21:47:10.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9cacff0640 1 -- 172.21.15.100:0/3333594707 learned_addr learned my addr 172.21.15.100:0/3333594707 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:10.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca77fe640 1 --2- 172.21.15.100:0/3333594707 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ca8070ee0 0x7f9ca8151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca7fff640 1 --2- 172.21.15.100:0/3333594707 >> v2:172.21.15.122:3300/0 conn(0x7f9ca80cca30 0x7f9ca8071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca6ffd640 1 -- 172.21.15.100:0/3333594707 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3707080784 0 0) 0x7f9ca80c91f0 con 0x7f9ca80cce10 2024-09-15T21:47:10.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca6ffd640 1 -- 172.21.15.100:0/3333594707 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c90003520 con 0x7f9ca80cce10 2024-09-15T21:47:10.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca6ffd640 1 -- 172.21.15.100:0/3333594707 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4277531686 0 0) 0x7f9c90003520 con 0x7f9ca80cce10 2024-09-15T21:47:10.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca6ffd640 1 -- 172.21.15.100:0/3333594707 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ca8070ee0 msgr2=0x7f9ca8151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:10.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca6ffd640 1 --2- 172.21.15.100:0/3333594707 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ca8070ee0 0x7f9ca8151450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca6ffd640 1 -- 172.21.15.100:0/3333594707 >> v2:172.21.15.122:3300/0 conn(0x7f9ca80cca30 msgr2=0x7f9ca8071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:10.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca6ffd640 1 --2- 172.21.15.100:0/3333594707 >> v2:172.21.15.122:3300/0 conn(0x7f9ca80cca30 0x7f9ca8071340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca6ffd640 1 -- 172.21.15.100:0/3333594707 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca80c9a90 con 0x7f9ca80cce10 2024-09-15T21:47:10.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca6ffd640 1 -- 172.21.15.100:0/3333594707 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7f9ca00019f0 con 0x7f9ca80cce10 2024-09-15T21:47:10.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca6ffd640 1 -- 172.21.15.100:0/3333594707 >> v1:172.21.15.100:6789/0 conn(0x7f9ca80cce10 legacy=0x7f9ca815af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca6ffd640 1 --2- 172.21.15.100:0/3333594707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c900039b0 0x7f9c90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca6ffd640 1 --2- 172.21.15.100:0/3333594707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c90004800 0x7f9c90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca6ffd640 1 --2- 172.21.15.100:0/3333594707 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ca8070ee0 0x7f9c900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca6ffd640 1 -- 172.21.15.100:0/3333594707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca80c94d0 con 0x7f9c900039b0 2024-09-15T21:47:10.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca6ffd640 1 -- 172.21.15.100:0/3333594707 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca80c97b0 con 0x7f9c90004800 2024-09-15T21:47:10.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca6ffd640 1 -- 172.21.15.100:0/3333594707 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9c9000ba70 con 0x7f9ca8070ee0 2024-09-15T21:47:10.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca7fff640 1 --2- 172.21.15.100:0/3333594707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c900039b0 0x7f9c90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9ca77fe640 1 --2- 172.21.15.100:0/3333594707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c90004800 0x7f9c90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.398+0000 7f9cacff0640 1 --2- 172.21.15.100:0/3333594707 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ca8070ee0 0x7f9c900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.399+0000 7f9ca7fff640 1 -- 172.21.15.100:0/3333594707 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ca8070ee0 msgr2=0x7f9c900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:10.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.399+0000 7f9ca7fff640 1 --2- 172.21.15.100:0/3333594707 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ca8070ee0 0x7f9c900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.399+0000 7f9ca7fff640 1 -- 172.21.15.100:0/3333594707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c90004800 msgr2=0x7f9c90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:10.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.399+0000 7f9ca7fff640 1 --2- 172.21.15.100:0/3333594707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c90004800 0x7f9c90004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.399+0000 7f9ca7fff640 1 -- 172.21.15.100:0/3333594707 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c90003520 con 0x7f9c900039b0 2024-09-15T21:47:10.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.399+0000 7f9ca7fff640 1 --2- 172.21.15.100:0/3333594707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c900039b0 0x7f9c90003dd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c009ac0 tx=0x7f9c90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c1ef47bd82605a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.399+0000 7f9ca6ffd640 1 -- 172.21.15.100:0/3333594707 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c036070 con 0x7f9c900039b0 2024-09-15T21:47:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.399+0000 7f9ca6ffd640 1 -- 172.21.15.100:0/3333594707 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c9c031400 con 0x7f9c900039b0 2024-09-15T21:47:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.399+0000 7f9ca6ffd640 1 -- 172.21.15.100:0/3333594707 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c009690 con 0x7f9c900039b0 2024-09-15T21:47:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.400+0000 7f9caea53640 1 -- 172.21.15.100:0/3333594707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c900039b0 msgr2=0x7f9c90003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.400+0000 7f9caea53640 1 --2- 172.21.15.100:0/3333594707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c900039b0 0x7f9c90003dd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c009ac0 tx=0x7f9c90005830 comp rx=0 tx=0).stop 2024-09-15T21:47:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.400+0000 7f9cacff0640 1 -- 172.21.15.100:0/3333594707 reap_dead start 2024-09-15T21:47:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.400+0000 7f9caea53640 1 -- 172.21.15.100:0/3333594707 shutdown_connections 2024-09-15T21:47:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.400+0000 7f9caea53640 1 -- 172.21.15.100:0/3333594707 >> 172.21.15.100:0/3333594707 conn(0x7f9ca8067dd0 msgr2=0x7f9ca806bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.400+0000 7f9caea53640 1 -- 172.21.15.100:0/3333594707 shutdown_connections 2024-09-15T21:47:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.400+0000 7f9caea53640 1 -- 172.21.15.100:0/3333594707 wait complete. 2024-09-15T21:47:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.401+0000 7f9caea53640 1 Processor -- start 2024-09-15T21:47:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.401+0000 7f9caea53640 1 -- start start 2024-09-15T21:47:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.402+0000 7f9caea53640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c900039b0 0x7f9ca8158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.402+0000 7f9caea53640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ca8158c90 0x7f9ca8156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.402+0000 7f9caea53640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ca8159310 0x7f9ca8159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.402+0000 7f9caea53640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca815b7e0 con 0x7f9c900039b0 2024-09-15T21:47:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.402+0000 7f9caea53640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ca815b660 con 0x7f9ca8159310 2024-09-15T21:47:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.402+0000 7f9caea53640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca8154e80 con 0x7f9ca8158c90 2024-09-15T21:47:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.402+0000 7f9ca7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c900039b0 0x7f9ca8158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.402+0000 7f9ca7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c900039b0 0x7f9ca8158420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49190/0 (socket says 172.21.15.100:49190) 2024-09-15T21:47:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.402+0000 7f9ca7fff640 1 -- 172.21.15.100:0/2446374175 learned_addr learned my addr 172.21.15.100:0/2446374175 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.402+0000 7f9cacff0640 1 --2- 172.21.15.100:0/2446374175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ca8159310 0x7f9ca8159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.402+0000 7f9ca77fe640 1 --2- 172.21.15.100:0/2446374175 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ca8158c90 0x7f9ca8156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.402+0000 7f9ca7fff640 1 -- 172.21.15.100:0/2446374175 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ca8158c90 msgr2=0x7f9ca8156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.402+0000 7f9ca7fff640 1 --2- 172.21.15.100:0/2446374175 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ca8158c90 0x7f9ca8156cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.402+0000 7f9ca7fff640 1 -- 172.21.15.100:0/2446374175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ca8159310 msgr2=0x7f9ca8159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.402+0000 7f9ca7fff640 1 --2- 172.21.15.100:0/2446374175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ca8159310 0x7f9ca8159790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.402+0000 7f9ca7fff640 1 -- 172.21.15.100:0/2446374175 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca815a6f0 con 0x7f9c900039b0 2024-09-15T21:47:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.403+0000 7f9ca7fff640 1 --2- 172.21.15.100:0/2446374175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c900039b0 0x7f9ca8158420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c009ac0 tx=0x7f9c9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.403+0000 7f9ca4ff9640 1 -- 172.21.15.100:0/2446374175 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c041020 con 0x7f9c900039b0 2024-09-15T21:47:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.403+0000 7f9ca4ff9640 1 -- 172.21.15.100:0/2446374175 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c9c03c470 con 0x7f9c900039b0 2024-09-15T21:47:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.403+0000 7f9ca4ff9640 1 -- 172.21.15.100:0/2446374175 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c9c036040 con 0x7f9c900039b0 2024-09-15T21:47:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.403+0000 7f9caea53640 1 -- 172.21.15.100:0/2446374175 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ca81571f0 con 0x7f9c900039b0 2024-09-15T21:47:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.403+0000 7f9caea53640 1 -- 172.21.15.100:0/2446374175 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ca81f1bf0 con 0x7f9c900039b0 2024-09-15T21:47:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.404+0000 7f9caea53640 1 -- 172.21.15.100:0/2446374175 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9ca8071990 con 0x7f9c900039b0 2024-09-15T21:47:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.405+0000 7f9ca4ff9640 1 -- 172.21.15.100:0/2446374175 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9c9c044020 con 0x7f9c900039b0 2024-09-15T21:47:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.405+0000 7f9ca4ff9640 1 --2- 172.21.15.100:0/2446374175 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9c9406c2e0 0x7f9c9406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.405+0000 7f9ca4ff9640 1 -- 172.21.15.100:0/2446374175 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5854+0+0 (secure 0 0 0) 0x7f9c9c0abbf0 con 0x7f9c900039b0 2024-09-15T21:47:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.405+0000 7f9ca77fe640 1 --2- 172.21.15.100:0/2446374175 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9c9406c2e0 0x7f9c9406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.405+0000 7f9ca4ff9640 1 --2- 172.21.15.100:0/2446374175 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f9c94074660 0x7f9c94076b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.405+0000 7f9ca4ff9640 1 -- 172.21.15.100:0/2446374175 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9c9c031400 con 0x7f9c94074660 2024-09-15T21:47:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.405+0000 7f9ca4ff9640 1 -- 172.21.15.100:0/2446374175 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f9c9c0ae270 con 0x7f9c900039b0 2024-09-15T21:47:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.405+0000 7f9cacff0640 1 --2- 172.21.15.100:0/2446374175 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f9c94074660 0x7f9c94076b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.405+0000 7f9ca77fe640 1 --2- 172.21.15.100:0/2446374175 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9c9406c2e0 0x7f9c9406e7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ca814d270 tx=0x7f9c98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.406+0000 7f9cacff0640 1 --2- 172.21.15.100:0/2446374175 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f9c94074660 0x7f9c94076b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.406+0000 7f9ca4ff9640 1 -- 172.21.15.100:0/2446374175 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9c9c031400 con 0x7f9c94074660 2024-09-15T21:47:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.422+0000 7f9caea53640 1 -- 172.21.15.100:0/2446374175 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f9ca8155870 con 0x7f9c94074660 2024-09-15T21:47:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.423+0000 7f9ca4ff9640 1 -- 172.21.15.100:0/2446374175 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f9ca8155870 con 0x7f9c94074660 2024-09-15T21:47:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.423+0000 7f9caea53640 1 -- 172.21.15.100:0/2446374175 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f9c94074660 msgr2=0x7f9c94076b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.423+0000 7f9caea53640 1 --2- 172.21.15.100:0/2446374175 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f9c94074660 0x7f9c94076b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.423+0000 7f9caea53640 1 -- 172.21.15.100:0/2446374175 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9c9406c2e0 msgr2=0x7f9c9406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.423+0000 7f9caea53640 1 --2- 172.21.15.100:0/2446374175 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9c9406c2e0 0x7f9c9406e7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9ca814d270 tx=0x7f9c98008040 comp rx=0 tx=0).stop 2024-09-15T21:47:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.423+0000 7f9caea53640 1 -- 172.21.15.100:0/2446374175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c900039b0 msgr2=0x7f9ca8158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.423+0000 7f9caea53640 1 --2- 172.21.15.100:0/2446374175 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9c900039b0 0x7f9ca8158420 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c009ac0 tx=0x7f9c9c004730 comp rx=0 tx=0).stop 2024-09-15T21:47:10.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.423+0000 7f9cacff0640 1 -- 172.21.15.100:0/2446374175 reap_dead start 2024-09-15T21:47:10.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.423+0000 7f9caea53640 1 -- 172.21.15.100:0/2446374175 shutdown_connections 2024-09-15T21:47:10.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.423+0000 7f9caea53640 1 -- 172.21.15.100:0/2446374175 >> 172.21.15.100:0/2446374175 conn(0x7f9ca8067dd0 msgr2=0x7f9ca80c5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:10.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.423+0000 7f9caea53640 1 -- 172.21.15.100:0/2446374175 shutdown_connections 2024-09-15T21:47:10.448 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.423+0000 7f9caea53640 1 -- 172.21.15.100:0/2446374175 wait complete. 2024-09-15T21:47:10.448 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-09-15T21:47:10.533 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:10.531+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:10.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.580+0000 7f83f4206640 1 Processor -- start 2024-09-15T21:47:10.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.580+0000 7f83f4206640 1 -- start start 2024-09-15T21:47:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.581+0000 7f83f4206640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f83ec0c6ff0 0x7f83ec0ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.581+0000 7f83f4206640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f83ec0718b0 0x7f83ec06f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.581+0000 7f83f4206640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83ec0cd0c0 con 0x7f83ec071ce0 2024-09-15T21:47:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.581+0000 7f83f4206640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f83ec0cd360 con 0x7f83ec0c6ff0 2024-09-15T21:47:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.581+0000 7f83f4206640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f83ec0cd640 con 0x7f83ec0718b0 2024-09-15T21:47:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.581+0000 7f83f1fa2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f83ec0c6ff0 0x7f83ec0ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.581+0000 7f83f1fa2640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f83ec0c6ff0 0x7f83ec0ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49202/0 (socket says 172.21.15.100:49202) 2024-09-15T21:47:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.581+0000 7f83f1fa2640 1 -- 172.21.15.100:0/3302440305 learned_addr learned my addr 172.21.15.100:0/3302440305 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.581+0000 7f83f17a1640 1 --2- 172.21.15.100:0/3302440305 >> v2:172.21.15.122:3300/0 conn(0x7f83ec0718b0 0x7f83ec06f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.581+0000 7f83f1fa2640 1 -- 172.21.15.100:0/3302440305 >> v2:172.21.15.122:3300/0 conn(0x7f83ec0718b0 msgr2=0x7f83ec06f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.581+0000 7f83f1fa2640 1 --2- 172.21.15.100:0/3302440305 >> v2:172.21.15.122:3300/0 conn(0x7f83ec0718b0 0x7f83ec06f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.582+0000 7f83f1fa2640 1 -- 172.21.15.100:0/3302440305 >> v1:172.21.15.122:6789/0 conn(0x7f83ec071ce0 legacy=0x7f83ec15b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.582+0000 7f83f1fa2640 1 -- 172.21.15.100:0/3302440305 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ec0cd920 con 0x7f83ec0c6ff0 2024-09-15T21:47:10.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.582+0000 7f83f1fa2640 1 --2- 172.21.15.100:0/3302440305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ec0c6ff0 0x7f83ec0ca090 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0737f0 tx=0x7f83dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7915ee8d396b21da server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.582+0000 7f83f0fa0640 1 -- 172.21.15.100:0/3302440305 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc037070 con 0x7f83ec0c6ff0 2024-09-15T21:47:10.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.582+0000 7f83f0fa0640 1 -- 172.21.15.100:0/3302440305 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83dc032440 con 0x7f83ec0c6ff0 2024-09-15T21:47:10.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.582+0000 7f83f0fa0640 1 -- 172.21.15.100:0/3302440305 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc032810 con 0x7f83ec0c6ff0 2024-09-15T21:47:10.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.583+0000 7f83f4206640 1 -- 172.21.15.100:0/3302440305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ec0c6ff0 msgr2=0x7f83ec0ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.583+0000 7f83f4206640 1 --2- 172.21.15.100:0/3302440305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ec0c6ff0 0x7f83ec0ca090 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0737f0 tx=0x7f83dc02a1e0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.583+0000 7f83f4206640 1 -- 172.21.15.100:0/3302440305 shutdown_connections 2024-09-15T21:47:10.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.583+0000 7f83f4206640 1 --2- 172.21.15.100:0/3302440305 >> v2:172.21.15.122:3300/0 conn(0x7f83ec0718b0 0x7f83ec06f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.583+0000 7f83f4206640 1 --2- 172.21.15.100:0/3302440305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ec0c6ff0 0x7f83ec0ca090 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.583+0000 7f83f4206640 1 -- 172.21.15.100:0/3302440305 >> 172.21.15.100:0/3302440305 conn(0x7f83ec073820 msgr2=0x7f83ec073c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:10.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.583+0000 7f83f4206640 1 -- 172.21.15.100:0/3302440305 shutdown_connections 2024-09-15T21:47:10.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.583+0000 7f83f4206640 1 -- 172.21.15.100:0/3302440305 wait complete. 2024-09-15T21:47:10.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.584+0000 7f83f4206640 1 Processor -- start 2024-09-15T21:47:10.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.584+0000 7f83f4206640 1 -- start start 2024-09-15T21:47:10.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.585+0000 7f83f4206640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83ec0718b0 0x7f83ec155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.585+0000 7f83f4206640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83ec071ce0 0x7f83ec15ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.585+0000 7f83f4206640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ec0c6ff0 0x7f83ec15d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.585+0000 7f83f4206640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec0c73d0 con 0x7f83ec0c6ff0 2024-09-15T21:47:10.588 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:10.588 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:10.588 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:10.588 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:10.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.585+0000 7f83f4206640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec0c8630 con 0x7f83ec0718b0 2024-09-15T21:47:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.585+0000 7f83f4206640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f83ec0c7550 con 0x7f83ec071ce0 2024-09-15T21:47:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.585+0000 7f83f27a3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ec0c6ff0 0x7f83ec15d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.585+0000 7f83f27a3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ec0c6ff0 0x7f83ec15d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49208/0 (socket says 172.21.15.100:49208) 2024-09-15T21:47:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.585+0000 7f83f27a3640 1 -- 172.21.15.100:0/1384152483 learned_addr learned my addr 172.21.15.100:0/1384152483 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.585+0000 7f83f17a1640 1 --2- 172.21.15.100:0/1384152483 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83ec071ce0 0x7f83ec15ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.585+0000 7f83f1fa2640 1 --2- 172.21.15.100:0/1384152483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83ec0718b0 0x7f83ec155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.585+0000 7f83f27a3640 1 -- 172.21.15.100:0/1384152483 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83ec071ce0 msgr2=0x7f83ec15ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.585+0000 7f83f27a3640 1 --2- 172.21.15.100:0/1384152483 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83ec071ce0 0x7f83ec15ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.585+0000 7f83f27a3640 1 -- 172.21.15.100:0/1384152483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83ec0718b0 msgr2=0x7f83ec155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.585+0000 7f83f27a3640 1 --2- 172.21.15.100:0/1384152483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83ec0718b0 0x7f83ec155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.585+0000 7f83f27a3640 1 -- 172.21.15.100:0/1384152483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ec15c590 con 0x7f83ec0c6ff0 2024-09-15T21:47:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.586+0000 7f83f27a3640 1 --2- 172.21.15.100:0/1384152483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ec0c6ff0 0x7f83ec15d530 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f83e400c760 tx=0x7f83e400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.586+0000 7f83e27fc640 1 -- 172.21.15.100:0/1384152483 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4003b90 con 0x7f83ec0c6ff0 2024-09-15T21:47:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.586+0000 7f83e27fc640 1 -- 172.21.15.100:0/1384152483 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e4003d30 con 0x7f83ec0c6ff0 2024-09-15T21:47:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.586+0000 7f83e27fc640 1 -- 172.21.15.100:0/1384152483 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e400aa90 con 0x7f83ec0c6ff0 2024-09-15T21:47:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.586+0000 7f83f4206640 1 -- 172.21.15.100:0/1384152483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83ec0730c0 con 0x7f83ec0c6ff0 2024-09-15T21:47:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.586+0000 7f83f4206640 1 -- 172.21.15.100:0/1384152483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83ec15cb10 con 0x7f83ec0c6ff0 2024-09-15T21:47:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.587+0000 7f83f4206640 1 -- 172.21.15.100:0/1384152483 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f83ec140520 con 0x7f83ec0c6ff0 2024-09-15T21:47:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.587+0000 7f83e27fc640 1 -- 172.21.15.100:0/1384152483 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f83e400ac30 con 0x7f83ec0c6ff0 2024-09-15T21:47:10.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.588+0000 7f83e27fc640 1 --2- 172.21.15.100:0/1384152483 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f83d006c2b0 0x7f83d006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.588+0000 7f83e27fc640 1 -- 172.21.15.100:0/1384152483 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5854+0+0 (secure 0 0 0) 0x7f83e408b250 con 0x7f83ec0c6ff0 2024-09-15T21:47:10.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.588+0000 7f83f1fa2640 1 --2- 172.21.15.100:0/1384152483 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f83d006c2b0 0x7f83d006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.588+0000 7f83e27fc640 1 --2- 172.21.15.100:0/1384152483 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f83d0074690 0x7f83d0076b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.588+0000 7f83e27fc640 1 -- 172.21.15.100:0/1384152483 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f83e4005820 con 0x7f83d0074690 2024-09-15T21:47:10.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.588+0000 7f83e27fc640 1 -- 172.21.15.100:0/1384152483 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f83e4092050 con 0x7f83ec0c6ff0 2024-09-15T21:47:10.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.588+0000 7f83f17a1640 1 --2- 172.21.15.100:0/1384152483 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f83d0074690 0x7f83d0076b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.588+0000 7f83f1fa2640 1 --2- 172.21.15.100:0/1384152483 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f83d006c2b0 0x7f83d006e770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f83dc034000 tx=0x7f83dc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.589+0000 7f83f17a1640 1 --2- 172.21.15.100:0/1384152483 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f83d0074690 0x7f83d0076b30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.589+0000 7f83e27fc640 1 -- 172.21.15.100:0/1384152483 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f83e4005820 con 0x7f83d0074690 2024-09-15T21:47:10.597 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:47:10.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.605+0000 7f83f4206640 1 -- 172.21.15.100:0/1384152483 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f83ec0735e0 con 0x7f83d0074690 2024-09-15T21:47:10.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.606+0000 7f83e27fc640 1 -- 172.21.15.100:0/1384152483 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f83ec0735e0 con 0x7f83d0074690 2024-09-15T21:47:10.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.606+0000 7f83f4206640 1 -- 172.21.15.100:0/1384152483 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f83d0074690 msgr2=0x7f83d0076b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.606+0000 7f83f4206640 1 --2- 172.21.15.100:0/1384152483 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f83d0074690 0x7f83d0076b30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.606+0000 7f83f4206640 1 -- 172.21.15.100:0/1384152483 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f83d006c2b0 msgr2=0x7f83d006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.606+0000 7f83f4206640 1 --2- 172.21.15.100:0/1384152483 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f83d006c2b0 0x7f83d006e770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f83dc034000 tx=0x7f83dc02e000 comp rx=0 tx=0).stop 2024-09-15T21:47:10.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.606+0000 7f83f4206640 1 -- 172.21.15.100:0/1384152483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ec0c6ff0 msgr2=0x7f83ec15d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.606+0000 7f83f4206640 1 --2- 172.21.15.100:0/1384152483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83ec0c6ff0 0x7f83ec15d530 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f83e400c760 tx=0x7f83e400cc20 comp rx=0 tx=0).stop 2024-09-15T21:47:10.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.606+0000 7f83f27a3640 1 -- 172.21.15.100:0/1384152483 reap_dead start 2024-09-15T21:47:10.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.607+0000 7f83f4206640 1 -- 172.21.15.100:0/1384152483 shutdown_connections 2024-09-15T21:47:10.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.607+0000 7f83f4206640 1 -- 172.21.15.100:0/1384152483 >> 172.21.15.100:0/1384152483 conn(0x7f83ec073820 msgr2=0x7f83ec0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:10.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.607+0000 7f83f4206640 1 -- 172.21.15.100:0/1384152483 shutdown_connections 2024-09-15T21:47:10.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.607+0000 7f83f4206640 1 -- 172.21.15.100:0/1384152483 wait complete. 2024-09-15T21:47:10.619 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-15T21:47:10.661 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:10.659+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:10.711 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:10.711 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:10.712 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:10.712 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:10.712 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:10.720 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:47:10.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.760+0000 7fba48817640 1 Processor -- start 2024-09-15T21:47:10.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.760+0000 7fba48817640 1 -- start start 2024-09-15T21:47:10.762 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:10.760+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.761+0000 7fba48817640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fba44073320 0x7fba44158e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.761+0000 7fba48817640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fba440c8a10 0x7fba440c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.761+0000 7fba48817640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba440c91a0 con 0x7fba44073700 2024-09-15T21:47:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.761+0000 7fba48817640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fba440c9440 con 0x7fba440c8a10 2024-09-15T21:47:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.761+0000 7fba48817640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fba440c9cc0 con 0x7fba44073320 2024-09-15T21:47:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.761+0000 7fba42d9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fba440c8a10 0x7fba440c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.761+0000 7fba42d9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fba440c8a10 0x7fba440c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49216/0 (socket says 172.21.15.100:49216) 2024-09-15T21:47:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.761+0000 7fba42d9d640 1 -- 172.21.15.100:0/3292676156 learned_addr learned my addr 172.21.15.100:0/3292676156 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.761+0000 7fba4259c640 1 --2- 172.21.15.100:0/3292676156 >> v2:172.21.15.122:3300/0 conn(0x7fba44073320 0x7fba44158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.761+0000 7fba42d9d640 1 -- 172.21.15.100:0/3292676156 >> v2:172.21.15.122:3300/0 conn(0x7fba44073320 msgr2=0x7fba44158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:10.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.761+0000 7fba42d9d640 1 --2- 172.21.15.100:0/3292676156 >> v2:172.21.15.122:3300/0 conn(0x7fba44073320 0x7fba44158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.761+0000 7fba42d9d640 1 -- 172.21.15.100:0/3292676156 >> v1:172.21.15.122:6789/0 conn(0x7fba44073700 legacy=0x7fba44152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.761+0000 7fba42d9d640 1 -- 172.21.15.100:0/3292676156 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba440c9700 con 0x7fba440c8a10 2024-09-15T21:47:10.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.762+0000 7fba42d9d640 1 --2- 172.21.15.100:0/3292676156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba440c8a10 0x7fba440c6540 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fba4406ffc0 tx=0x7fba3c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f17d9569127b03 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.762+0000 7fba4159a640 1 -- 172.21.15.100:0/3292676156 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c036070 con 0x7fba440c8a10 2024-09-15T21:47:10.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.762+0000 7fba4159a640 1 -- 172.21.15.100:0/3292676156 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba3c031600 con 0x7fba440c8a10 2024-09-15T21:47:10.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.762+0000 7fba4159a640 1 -- 172.21.15.100:0/3292676156 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c0319d0 con 0x7fba440c8a10 2024-09-15T21:47:10.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.763+0000 7fba48817640 1 -- 172.21.15.100:0/3292676156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba440c8a10 msgr2=0x7fba440c6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.763+0000 7fba48817640 1 --2- 172.21.15.100:0/3292676156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba440c8a10 0x7fba440c6540 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fba4406ffc0 tx=0x7fba3c02a730 comp rx=0 tx=0).stop 2024-09-15T21:47:10.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.763+0000 7fba48817640 1 -- 172.21.15.100:0/3292676156 shutdown_connections 2024-09-15T21:47:10.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.763+0000 7fba48817640 1 --2- 172.21.15.100:0/3292676156 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba440c8a10 0x7fba440c6540 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.763+0000 7fba48817640 1 --2- 172.21.15.100:0/3292676156 >> v2:172.21.15.122:3300/0 conn(0x7fba44073320 0x7fba44158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.763+0000 7fba48817640 1 -- 172.21.15.100:0/3292676156 >> 172.21.15.100:0/3292676156 conn(0x7fba44070ca0 msgr2=0x7fba44071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:10.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.763+0000 7fba48817640 1 -- 172.21.15.100:0/3292676156 shutdown_connections 2024-09-15T21:47:10.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.763+0000 7fba48817640 1 -- 172.21.15.100:0/3292676156 wait complete. 2024-09-15T21:47:10.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.764+0000 7fba48817640 1 Processor -- start 2024-09-15T21:47:10.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.764+0000 7fba48817640 1 -- start start 2024-09-15T21:47:10.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.765+0000 7fba48817640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba44073320 0x7fba440ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.765+0000 7fba48817640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba44073700 0x7fba441e3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.765+0000 7fba48817640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba440c8a10 0x7fba441e44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.765+0000 7fba48817640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fba440c6020 con 0x7fba440c8a10 2024-09-15T21:47:10.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.765+0000 7fba48817640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fba440c5d40 con 0x7fba44073320 2024-09-15T21:47:10.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.765+0000 7fba48817640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fba440c78c0 con 0x7fba44073700 2024-09-15T21:47:10.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.765+0000 7fba42d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba440c8a10 0x7fba441e44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.765+0000 7fba42d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba440c8a10 0x7fba441e44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49228/0 (socket says 172.21.15.100:49228) 2024-09-15T21:47:10.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.765+0000 7fba42d9d640 1 -- 172.21.15.100:0/3111803094 learned_addr learned my addr 172.21.15.100:0/3111803094 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:10.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.765+0000 7fba4259c640 1 --2- 172.21.15.100:0/3111803094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba44073320 0x7fba440ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.765+0000 7fba41d9b640 1 --2- 172.21.15.100:0/3111803094 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba44073700 0x7fba441e3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.765+0000 7fba42d9d640 1 -- 172.21.15.100:0/3111803094 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba44073700 msgr2=0x7fba441e3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.765+0000 7fba42d9d640 1 --2- 172.21.15.100:0/3111803094 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fba44073700 0x7fba441e3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.765+0000 7fba42d9d640 1 -- 172.21.15.100:0/3111803094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba44073320 msgr2=0x7fba440ca6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.765+0000 7fba42d9d640 1 --2- 172.21.15.100:0/3111803094 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fba44073320 0x7fba440ca6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.765+0000 7fba42d9d640 1 -- 172.21.15.100:0/3111803094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba440caea0 con 0x7fba440c8a10 2024-09-15T21:47:10.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.766+0000 7fba42d9d640 1 --2- 172.21.15.100:0/3111803094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba440c8a10 0x7fba441e44d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fba3c02ad20 tx=0x7fba3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.766+0000 7fba32ffd640 1 -- 172.21.15.100:0/3111803094 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c03f020 con 0x7fba440c8a10 2024-09-15T21:47:10.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.766+0000 7fba32ffd640 1 -- 172.21.15.100:0/3111803094 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba3c037040 con 0x7fba440c8a10 2024-09-15T21:47:10.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.766+0000 7fba48817640 1 -- 172.21.15.100:0/3111803094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba440cb180 con 0x7fba440c8a10 2024-09-15T21:47:10.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.766+0000 7fba32ffd640 1 -- 172.21.15.100:0/3111803094 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c036070 con 0x7fba440c8a10 2024-09-15T21:47:10.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.766+0000 7fba48817640 1 -- 172.21.15.100:0/3111803094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba441e3790 con 0x7fba440c8a10 2024-09-15T21:47:10.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.767+0000 7fba48817640 1 -- 172.21.15.100:0/3111803094 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fba441e32f0 con 0x7fba440c8a10 2024-09-15T21:47:10.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.767+0000 7fba32ffd640 1 -- 172.21.15.100:0/3111803094 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fba3c042020 con 0x7fba440c8a10 2024-09-15T21:47:10.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.768+0000 7fba32ffd640 1 --2- 172.21.15.100:0/3111803094 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fba2406c2b0 0x7fba2406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.768+0000 7fba32ffd640 1 -- 172.21.15.100:0/3111803094 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5854+0+0 (secure 0 0 0) 0x7fba3c0ac250 con 0x7fba440c8a10 2024-09-15T21:47:10.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.768+0000 7fba4259c640 1 --2- 172.21.15.100:0/3111803094 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fba2406c2b0 0x7fba2406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.768+0000 7fba32ffd640 1 --2- 172.21.15.100:0/3111803094 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fba24074630 0x7fba24076ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:10.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.768+0000 7fba32ffd640 1 -- 172.21.15.100:0/3111803094 --> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fba240771a0 con 0x7fba24074630 2024-09-15T21:47:10.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.768+0000 7fba32ffd640 1 -- 172.21.15.100:0/3111803094 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fba3c0ac6e0 con 0x7fba440c8a10 2024-09-15T21:47:10.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.768+0000 7fba4259c640 1 --2- 172.21.15.100:0/3111803094 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fba2406c2b0 0x7fba2406e770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fba34009b40 tx=0x7fba34009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.768+0000 7fba41d9b640 1 --2- 172.21.15.100:0/3111803094 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fba24074630 0x7fba24076ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:10.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.769+0000 7fba41d9b640 1 --2- 172.21.15.100:0/3111803094 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fba24074630 0x7fba24076ad0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:10.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.769+0000 7fba32ffd640 1 -- 172.21.15.100:0/3111803094 <== osd.5 v2:172.21.15.122:6824/2400668532 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fba240771a0 con 0x7fba24074630 2024-09-15T21:47:10.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.785+0000 7fba48817640 1 -- 172.21.15.100:0/3111803094 --> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7fba440ced90 con 0x7fba24074630 2024-09-15T21:47:10.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.786+0000 7fba32ffd640 1 -- 172.21.15.100:0/3111803094 <== osd.5 v2:172.21.15.122:6824/2400668532 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7fba440ced90 con 0x7fba24074630 2024-09-15T21:47:10.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.786+0000 7fba48817640 1 -- 172.21.15.100:0/3111803094 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fba24074630 msgr2=0x7fba24076ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.786+0000 7fba48817640 1 --2- 172.21.15.100:0/3111803094 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fba24074630 0x7fba24076ad0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:10.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.786+0000 7fba48817640 1 -- 172.21.15.100:0/3111803094 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fba2406c2b0 msgr2=0x7fba2406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.786+0000 7fba48817640 1 --2- 172.21.15.100:0/3111803094 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fba2406c2b0 0x7fba2406e770 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fba34009b40 tx=0x7fba34009190 comp rx=0 tx=0).stop 2024-09-15T21:47:10.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.786+0000 7fba48817640 1 -- 172.21.15.100:0/3111803094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba440c8a10 msgr2=0x7fba441e44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:10.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.786+0000 7fba48817640 1 --2- 172.21.15.100:0/3111803094 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fba440c8a10 0x7fba441e44d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fba3c02ad20 tx=0x7fba3c004a40 comp rx=0 tx=0).stop 2024-09-15T21:47:10.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.786+0000 7fba42d9d640 1 -- 172.21.15.100:0/3111803094 reap_dead start 2024-09-15T21:47:10.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.786+0000 7fba48817640 1 -- 172.21.15.100:0/3111803094 shutdown_connections 2024-09-15T21:47:10.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.786+0000 7fba48817640 1 -- 172.21.15.100:0/3111803094 >> 172.21.15.100:0/3111803094 conn(0x7fba44070ca0 msgr2=0x7fba44150af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:10.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.787+0000 7fba48817640 1 -- 172.21.15.100:0/3111803094 shutdown_connections 2024-09-15T21:47:10.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:10.787+0000 7fba48817640 1 -- 172.21.15.100:0/3111803094 wait complete. 2024-09-15T21:47:10.799 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-15T21:47:10.828 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.412938711999999, 2024-09-15T21:47:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006101291, 2024-09-15T21:47:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.829 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.830 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413116+0000", 2024-09-15T21:47:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9700000000000003e-07 2024-09-15T21:47:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413118+0000", 2024-09-15T21:47:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2910000000000002e-06 2024-09-15T21:47:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413220+0000", 2024-09-15T21:47:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102056 2024-09-15T21:47:10.831 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413225+0000", 2024-09-15T21:47:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0490000000000003e-06 2024-09-15T21:47:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414808+0000", 2024-09-15T21:47:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015826900000000001 2024-09-15T21:47:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414907+0000", 2024-09-15T21:47:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9600999999999997e-05 2024-09-15T21:47:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415054+0000", 2024-09-15T21:47:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000146532 2024-09-15T21:47:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418056+0000", 2024-09-15T21:47:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030020519999999998 2024-09-15T21:47:10.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.834 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.834 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.834 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419128+0000", 2024-09-15T21:47:10.834 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001071671 2024-09-15T21:47:10.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.834 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.834 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.834 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419184+0000", 2024-09-15T21:47:10.834 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6125000000000002e-05 2024-09-15T21:47:10.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419217+0000", 2024-09-15T21:47:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2926999999999998e-05 2024-09-15T21:47:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.412935185999999, 2024-09-15T21:47:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059975130000000003, 2024-09-15T21:47:10.835 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:10.836 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6199999999999999e-07 2024-09-15T21:47:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413121+0000", 2024-09-15T21:47:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2000000000000001e-06 2024-09-15T21:47:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.837 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413292+0000", 2024-09-15T21:47:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017034400000000001 2024-09-15T21:47:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413300+0000", 2024-09-15T21:47:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8539999999999997e-06 2024-09-15T21:47:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414299+0000", 2024-09-15T21:47:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0009992829999999999 2024-09-15T21:47:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.839 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.839 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414390+0000", 2024-09-15T21:47:10.839 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1237999999999996e-05 2024-09-15T21:47:10.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.839 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:10.839 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414490+0000", 2024-09-15T21:47:10.839 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9438999999999994e-05 2024-09-15T21:47:10.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.839 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.840 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417393+0000", 2024-09-15T21:47:10.840 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002903306 2024-09-15T21:47:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419023+0000", 2024-09-15T21:47:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001630292 2024-09-15T21:47:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419067+0000", 2024-09-15T21:47:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4017999999999998e-05 2024-09-15T21:47:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.841 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419116+0000", 2024-09-15T21:47:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9277000000000003e-05 2024-09-15T21:47:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.410236988999998, 2024-09-15T21:47:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060879280000000003, 2024-09-15T21:47:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.842 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.843 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415818+0000", 2024-09-15T21:47:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:47:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415854+0000", 2024-09-15T21:47:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6706999999999999e-05 2024-09-15T21:47:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415860+0000", 2024-09-15T21:47:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0789999999999997e-06 2024-09-15T21:47:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415865+0000", 2024-09-15T21:47:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4580000000000003e-06 2024-09-15T21:47:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416092+0000", 2024-09-15T21:47:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00022739400000000001 2024-09-15T21:47:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416108+0000", 2024-09-15T21:47:10.845 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.613e-05 2024-09-15T21:47:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421866+0000", 2024-09-15T21:47:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005757585 2024-09-15T21:47:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421905+0000", 2024-09-15T21:47:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9214000000000001e-05 2024-09-15T21:47:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-15T21:47:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.410176829000001, 2024-09-15T21:47:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060637570000000003, 2024-09-15T21:47:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:10.847 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415878+0000", 2024-09-15T21:47:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.34e-07 2024-09-15T21:47:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.848 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415881+0000", 2024-09-15T21:47:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.354e-06 2024-09-15T21:47:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415887+0000", 2024-09-15T21:47:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:47:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415891+0000", 2024-09-15T21:47:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2690000000000001e-06 2024-09-15T21:47:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.849 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416267+0000", 2024-09-15T21:47:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00037614799999999998 2024-09-15T21:47:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416288+0000", 2024-09-15T21:47:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0604000000000001e-05 2024-09-15T21:47:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421908+0000", 2024-09-15T21:47:10.850 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056203390000000002 2024-09-15T21:47:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421941+0000", 2024-09-15T21:47:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3079e-05 2024-09-15T21:47:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:10.851 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.410158402, 2024-09-15T21:47:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006080284, 2024-09-15T21:47:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.852 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6899999999999999e-07 2024-09-15T21:47:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415898+0000", 2024-09-15T21:47:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.951e-06 2024-09-15T21:47:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.853 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415920+0000", 2024-09-15T21:47:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1651999999999999e-05 2024-09-15T21:47:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415925+0000", 2024-09-15T21:47:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.463e-06 2024-09-15T21:47:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416449+0000", 2024-09-15T21:47:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052384000000000003 2024-09-15T21:47:10.854 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416466+0000", 2024-09-15T21:47:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7108999999999998e-05 2024-09-15T21:47:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421943+0000", 2024-09-15T21:47:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054771569999999999 2024-09-15T21:47:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421976+0000", 2024-09-15T21:47:10.855 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2943e-05 2024-09-15T21:47:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-15T21:47:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.410119166000001, 2024-09-15T21:47:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060643340000000002, 2024-09-15T21:47:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.856 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:10.857 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4200000000000002e-07 2024-09-15T21:47:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415939+0000", 2024-09-15T21:47:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.439e-06 2024-09-15T21:47:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415945+0000", 2024-09-15T21:47:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7740000000000001e-06 2024-09-15T21:47:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.858 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415948+0000", 2024-09-15T21:47:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.737e-06 2024-09-15T21:47:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416605+0000", 2024-09-15T21:47:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00065721699999999998 2024-09-15T21:47:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416620+0000", 2024-09-15T21:47:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4564e-05 2024-09-15T21:47:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.859 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421977+0000", 2024-09-15T21:47:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053572369999999999 2024-09-15T21:47:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421999+0000", 2024-09-15T21:47:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2024000000000001e-05 2024-09-15T21:47:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.860 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-15T21:47:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.409918507, 2024-09-15T21:47:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058922669999999996, 2024-09-15T21:47:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.861 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6499999999999999e-07 2024-09-15T21:47:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416142+0000", 2024-09-15T21:47:10.862 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3929999999999999e-06 2024-09-15T21:47:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416151+0000", 2024-09-15T21:47:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5840000000000001e-06 2024-09-15T21:47:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416157+0000", 2024-09-15T21:47:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7590000000000003e-06 2024-09-15T21:47:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.863 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416767+0000", 2024-09-15T21:47:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00061034900000000005 2024-09-15T21:47:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416782+0000", 2024-09-15T21:47:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.483e-05 2024-09-15T21:47:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422002+0000", 2024-09-15T21:47:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052202639999999996 2024-09-15T21:47:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.864 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422028+0000", 2024-09-15T21:47:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5423e-05 2024-09-15T21:47:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-15T21:47:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.409512006, 2024-09-15T21:47:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055107769999999997, 2024-09-15T21:47:10.865 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.866 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416543+0000", 2024-09-15T21:47:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5400000000000001e-07 2024-09-15T21:47:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416548+0000", 2024-09-15T21:47:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8659999999999997e-06 2024-09-15T21:47:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416555+0000", 2024-09-15T21:47:10.867 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7190000000000001e-06 2024-09-15T21:47:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416560+0000", 2024-09-15T21:47:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3270000000000002e-06 2024-09-15T21:47:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416916+0000", 2024-09-15T21:47:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00035661200000000002 2024-09-15T21:47:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.868 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.869 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.869 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416932+0000", 2024-09-15T21:47:10.869 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5824e-05 2024-09-15T21:47:10.869 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.869 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.869 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.869 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422030+0000", 2024-09-15T21:47:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050975229999999996 2024-09-15T21:47:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422053+0000", 2024-09-15T21:47:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3252e-05 2024-09-15T21:47:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-15T21:47:10.870 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.409113507000001, 2024-09-15T21:47:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051422289999999999, 2024-09-15T21:47:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.871 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2699999999999999e-07 2024-09-15T21:47:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416946+0000", 2024-09-15T21:47:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1390000000000003e-06 2024-09-15T21:47:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416957+0000", 2024-09-15T21:47:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.044e-05 2024-09-15T21:47:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416963+0000", 2024-09-15T21:47:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5320000000000003e-06 2024-09-15T21:47:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417090+0000", 2024-09-15T21:47:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126549 2024-09-15T21:47:10.873 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417114+0000", 2024-09-15T21:47:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3736000000000001e-05 2024-09-15T21:47:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422055+0000", 2024-09-15T21:47:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049410410000000002 2024-09-15T21:47:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422083+0000", 2024-09-15T21:47:10.874 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8164999999999999e-05 2024-09-15T21:47:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 36.364621534999998, 2024-09-15T21:47:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007521186, 2024-09-15T21:47:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.875 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.876 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9999999999999998e-07 2024-09-15T21:47:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461443+0000", 2024-09-15T21:47:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0448000000000001e-05 2024-09-15T21:47:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462146+0000", 2024-09-15T21:47:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00070223499999999995 2024-09-15T21:47:10.877 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462154+0000", 2024-09-15T21:47:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9060000000000002e-06 2024-09-15T21:47:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462254+0000", 2024-09-15T21:47:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9897999999999998e-05 2024-09-15T21:47:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.878 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462330+0000", 2024-09-15T21:47:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6551000000000006e-05 2024-09-15T21:47:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462454+0000", 2024-09-15T21:47:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000124213 2024-09-15T21:47:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463681+0000", 2024-09-15T21:47:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012265729999999999 2024-09-15T21:47:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.879 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468817+0000", 2024-09-15T21:47:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051361430000000001 2024-09-15T21:47:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468892+0000", 2024-09-15T21:47:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5480000000000002e-05 2024-09-15T21:47:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468954+0000", 2024-09-15T21:47:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1339000000000003e-05 2024-09-15T21:47:10.880 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 36.364604919999998, 2024-09-15T21:47:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077296149999999996, 2024-09-15T21:47:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.881 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461450+0000", 2024-09-15T21:47:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0499999999999999e-07 2024-09-15T21:47:10.882 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.883 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.883 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.883 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461452+0000", 2024-09-15T21:47:10.883 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.007e-06 2024-09-15T21:47:10.883 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.883 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.883 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.883 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462209+0000", 2024-09-15T21:47:10.883 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075781000000000004 2024-09-15T21:47:10.883 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.883 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462214+0000", 2024-09-15T21:47:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1420000000000001e-06 2024-09-15T21:47:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462681+0000", 2024-09-15T21:47:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046634200000000002 2024-09-15T21:47:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462772+0000", 2024-09-15T21:47:10.884 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1414000000000003e-05 2024-09-15T21:47:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462871+0000", 2024-09-15T21:47:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8455000000000005e-05 2024-09-15T21:47:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464138+0000", 2024-09-15T21:47:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012673910000000001 2024-09-15T21:47:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.885 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469079+0000", 2024-09-15T21:47:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049407799999999997 2024-09-15T21:47:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469141+0000", 2024-09-15T21:47:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2169999999999996e-05 2024-09-15T21:47:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469179+0000", 2024-09-15T21:47:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7799000000000002e-05 2024-09-15T21:47:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.886 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 30.350489111999998, 2024-09-15T21:47:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004990497, 2024-09-15T21:47:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.887 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4729999999999999e-06 2024-09-15T21:47:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.888 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475576+0000", 2024-09-15T21:47:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1749999999999994e-06 2024-09-15T21:47:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475619+0000", 2024-09-15T21:47:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3557000000000003e-05 2024-09-15T21:47:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475631+0000", 2024-09-15T21:47:10.889 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1739999999999999e-05 2024-09-15T21:47:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475770+0000", 2024-09-15T21:47:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013950599999999999 2024-09-15T21:47:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475887+0000", 2024-09-15T21:47:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011616 2024-09-15T21:47:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476018+0000", 2024-09-15T21:47:10.890 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013119499999999999 2024-09-15T21:47:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478442+0000", 2024-09-15T21:47:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024240870000000001 2024-09-15T21:47:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480457+0000", 2024-09-15T21:47:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020151779999999998 2024-09-15T21:47:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.891 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480511+0000", 2024-09-15T21:47:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3582000000000003e-05 2024-09-15T21:47:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480556+0000", 2024-09-15T21:47:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4844e-05 2024-09-15T21:47:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.892 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 30.350474794, 2024-09-15T21:47:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051124810000000003, 2024-09-15T21:47:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.893 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475580+0000", 2024-09-15T21:47:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7800000000000002e-07 2024-09-15T21:47:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475595+0000", 2024-09-15T21:47:10.894 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4919000000000001e-05 2024-09-15T21:47:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475662+0000", 2024-09-15T21:47:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7769999999999997e-05 2024-09-15T21:47:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475669+0000", 2024-09-15T21:47:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.703e-06 2024-09-15T21:47:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.895 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476781+0000", 2024-09-15T21:47:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011118300000000001 2024-09-15T21:47:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476863+0000", 2024-09-15T21:47:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2208999999999998e-05 2024-09-15T21:47:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477040+0000", 2024-09-15T21:47:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000176424 2024-09-15T21:47:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.896 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480154+0000", 2024-09-15T21:47:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031147589999999999 2024-09-15T21:47:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480596+0000", 2024-09-15T21:47:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044172699999999999 2024-09-15T21:47:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480652+0000", 2024-09-15T21:47:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6350000000000001e-05 2024-09-15T21:47:10.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480692+0000", 2024-09-15T21:47:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9412000000000001e-05 2024-09-15T21:47:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:6 2.9 2:941d59bc:::61f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 25.281264262000001, 2024-09-15T21:47:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057427529999999997, 2024-09-15T21:47:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.898 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.899 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.41e-07 2024-09-15T21:47:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544792+0000", 2024-09-15T21:47:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9929999999999998e-06 2024-09-15T21:47:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544885+0000", 2024-09-15T21:47:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2825000000000007e-05 2024-09-15T21:47:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.900 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544891+0000", 2024-09-15T21:47:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.93e-06 2024-09-15T21:47:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546766+0000", 2024-09-15T21:47:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018747569999999999 2024-09-15T21:47:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546864+0000", 2024-09-15T21:47:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8603000000000004e-05 2024-09-15T21:47:10.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547028+0000", 2024-09-15T21:47:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000163461 2024-09-15T21:47:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549499+0000", 2024-09-15T21:47:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024711749999999999 2024-09-15T21:47:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550417+0000", 2024-09-15T21:47:10.902 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00091817199999999998 2024-09-15T21:47:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550476+0000", 2024-09-15T21:47:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8635999999999999e-05 2024-09-15T21:47:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550533+0000", 2024-09-15T21:47:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6959999999999997e-05 2024-09-15T21:47:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 25.281249797000001, 2024-09-15T21:47:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004917121, 2024-09-15T21:47:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.904 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544805+0000", 2024-09-15T21:47:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4600000000000001e-07 2024-09-15T21:47:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544817+0000", 2024-09-15T21:47:10.905 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2023e-05 2024-09-15T21:47:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545000+0000", 2024-09-15T21:47:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018384400000000001 2024-09-15T21:47:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545007+0000", 2024-09-15T21:47:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1249999999999998e-06 2024-09-15T21:47:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.906 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545670+0000", 2024-09-15T21:47:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00066324999999999999 2024-09-15T21:47:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545749+0000", 2024-09-15T21:47:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9239000000000005e-05 2024-09-15T21:47:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545885+0000", 2024-09-15T21:47:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013586199999999999 2024-09-15T21:47:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.907 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547944+0000", 2024-09-15T21:47:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020594720000000001 2024-09-15T21:47:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549630+0000", 2024-09-15T21:47:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001685831 2024-09-15T21:47:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549681+0000", 2024-09-15T21:47:10.908 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1187000000000003e-05 2024-09-15T21:47:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549721+0000", 2024-09-15T21:47:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0042e-05 2024-09-15T21:47:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 25.281236049, 2024-09-15T21:47:10.909 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050615549999999997, 2024-09-15T21:47:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.910 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:47:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544821+0000", 2024-09-15T21:47:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4399999999999999e-06 2024-09-15T21:47:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545025+0000", 2024-09-15T21:47:10.911 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020379400000000001 2024-09-15T21:47:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545030+0000", 2024-09-15T21:47:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3109999999999996e-06 2024-09-15T21:47:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546196+0000", 2024-09-15T21:47:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001166261 2024-09-15T21:47:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.912 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546307+0000", 2024-09-15T21:47:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000110641 2024-09-15T21:47:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546422+0000", 2024-09-15T21:47:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000115399 2024-09-15T21:47:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547973+0000", 2024-09-15T21:47:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001551228 2024-09-15T21:47:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.913 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549815+0000", 2024-09-15T21:47:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001841685 2024-09-15T21:47:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549856+0000", 2024-09-15T21:47:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.091e-05 2024-09-15T21:47:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549880+0000", 2024-09-15T21:47:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3580000000000001e-05 2024-09-15T21:47:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.914 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 25.281232040999999, 2024-09-15T21:47:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056931389999999998, 2024-09-15T21:47:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.915 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:10.916 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.916 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.916 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.916 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:10.916 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.916 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.916 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:47:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544828+0000", 2024-09-15T21:47:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4369999999999998e-06 2024-09-15T21:47:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545041+0000", 2024-09-15T21:47:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000212695 2024-09-15T21:47:10.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.918 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.918 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.918 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545058+0000", 2024-09-15T21:47:10.918 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7717999999999999e-05 2024-09-15T21:47:10.918 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.918 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.918 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.918 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546958+0000", 2024-09-15T21:47:10.918 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018995539999999999 2024-09-15T21:47:10.918 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.918 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.918 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547057+0000", 2024-09-15T21:47:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9370999999999994e-05 2024-09-15T21:47:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547579+0000", 2024-09-15T21:47:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052198200000000004 2024-09-15T21:47:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549929+0000", 2024-09-15T21:47:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023494649999999998 2024-09-15T21:47:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550401+0000", 2024-09-15T21:47:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00047277300000000001 2024-09-15T21:47:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550458+0000", 2024-09-15T21:47:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6756999999999999e-05 2024-09-15T21:47:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550515+0000", 2024-09-15T21:47:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7093999999999999e-05 2024-09-15T21:47:10.920 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:26 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 25.281225383999999, 2024-09-15T21:47:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050112029999999997, 2024-09-15T21:47:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.921 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.36e-07 2024-09-15T21:47:10.922 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545086+0000", 2024-09-15T21:47:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025704699999999999 2024-09-15T21:47:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545119+0000", 2024-09-15T21:47:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2734999999999997e-05 2024-09-15T21:47:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.923 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545126+0000", 2024-09-15T21:47:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5599999999999996e-06 2024-09-15T21:47:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545253+0000", 2024-09-15T21:47:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012644500000000001 2024-09-15T21:47:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545366+0000", 2024-09-15T21:47:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011349999999999999 2024-09-15T21:47:10.924 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545673+0000", 2024-09-15T21:47:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030643599999999999 2024-09-15T21:47:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547924+0000", 2024-09-15T21:47:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0022516189999999998 2024-09-15T21:47:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.925 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549746+0000", 2024-09-15T21:47:10.926 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018217629999999999 2024-09-15T21:47:10.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.926 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.926 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.926 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549800+0000", 2024-09-15T21:47:10.926 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3412999999999998e-05 2024-09-15T21:47:10.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.926 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.926 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549840+0000", 2024-09-15T21:47:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0448999999999998e-05 2024-09-15T21:47:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:30 2.4 2:218afd09:::203.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 25.280964555000001, 2024-09-15T21:47:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055481230000000003, 2024-09-15T21:47:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.927 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.928 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545091+0000", 2024-09-15T21:47:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7000000000000003e-07 2024-09-15T21:47:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545094+0000", 2024-09-15T21:47:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9569999999999998e-06 2024-09-15T21:47:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545156+0000", 2024-09-15T21:47:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2277e-05 2024-09-15T21:47:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.929 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545162+0000", 2024-09-15T21:47:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9120000000000003e-06 2024-09-15T21:47:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545262+0000", 2024-09-15T21:47:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010001900000000001 2024-09-15T21:47:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545388+0000", 2024-09-15T21:47:10.930 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126243 2024-09-15T21:47:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550638+0000", 2024-09-15T21:47:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052497450000000001 2024-09-15T21:47:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:10 2.9 2:954a9a2b:::62b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.261281095000001, 2024-09-15T21:47:10.931 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048438860000000004, 2024-09-15T21:47:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:10.932 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0400000000000002e-07 2024-09-15T21:47:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564786+0000", 2024-09-15T21:47:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2412000000000001e-05 2024-09-15T21:47:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564991+0000", 2024-09-15T21:47:10.933 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020483900000000001 2024-09-15T21:47:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564998+0000", 2024-09-15T21:47:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2459999999999997e-06 2024-09-15T21:47:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565422+0000", 2024-09-15T21:47:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00042420100000000003 2024-09-15T21:47:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.934 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565515+0000", 2024-09-15T21:47:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2449999999999997e-05 2024-09-15T21:47:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565642+0000", 2024-09-15T21:47:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012743699999999999 2024-09-15T21:47:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566656+0000", 2024-09-15T21:47:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010138580000000001 2024-09-15T21:47:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.935 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569552+0000", 2024-09-15T21:47:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028966819999999998 2024-09-15T21:47:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569594+0000", 2024-09-15T21:47:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1235000000000001e-05 2024-09-15T21:47:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569617+0000", 2024-09-15T21:47:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3221999999999999e-05 2024-09-15T21:47:10.936 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.937 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.937 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.937 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:10.937 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:10.937 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:10.938 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:47:10.939 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:47:10.937+0000 7f74313af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37847) UID: 0 2024-09-15T21:47:11.040 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:11.038+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:11.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.038+0000 7f5910074640 1 Processor -- start 2024-09-15T21:47:11.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.038+0000 7f5910074640 1 -- start start 2024-09-15T21:47:11.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.039+0000 7f5910074640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5908073790 0x7f59080c4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.039+0000 7f5910074640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f590806ccf0 0x7f590806d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.039+0000 7f5910074640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5908157ce0 con 0x7f5908152160 2024-09-15T21:47:11.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.039+0000 7f5910074640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f5908072bf0 con 0x7f590806ccf0 2024-09-15T21:47:11.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.039+0000 7f5910074640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5908157fa0 con 0x7f5908073790 2024-09-15T21:47:11.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.039+0000 7f590e611640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f590806ccf0 0x7f590806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.039+0000 7f590e611640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f590806ccf0 0x7f590806d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49234/0 (socket says 172.21.15.100:49234) 2024-09-15T21:47:11.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.039+0000 7f590e611640 1 -- 172.21.15.100:0/2016632836 learned_addr learned my addr 172.21.15.100:0/2016632836 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:11.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.039+0000 7f590d60f640 1 --2- 172.21.15.100:0/2016632836 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5908073790 0x7f59080c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.039+0000 7f590e611640 1 -- 172.21.15.100:0/2016632836 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5908073790 msgr2=0x7f59080c4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:11.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.039+0000 7f590e611640 1 --2- 172.21.15.100:0/2016632836 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5908073790 0x7f59080c4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.039+0000 7f590e611640 1 -- 172.21.15.100:0/2016632836 >> v1:172.21.15.122:6789/0 conn(0x7f5908152160 legacy=0x7f5908073080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.039+0000 7f590e611640 1 -- 172.21.15.100:0/2016632836 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5908158280 con 0x7f590806ccf0 2024-09-15T21:47:11.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.040+0000 7f590e611640 1 --2- 172.21.15.100:0/2016632836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f590806ccf0 0x7f590806d160 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5900009e50 tx=0x7f590002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b567c7db0b479d92 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.041+0000 7f590ce0e640 1 -- 172.21.15.100:0/2016632836 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900036070 con 0x7f590806ccf0 2024-09-15T21:47:11.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.041+0000 7f590ce0e640 1 -- 172.21.15.100:0/2016632836 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5900031600 con 0x7f590806ccf0 2024-09-15T21:47:11.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.041+0000 7f590ce0e640 1 -- 172.21.15.100:0/2016632836 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59000319d0 con 0x7f590806ccf0 2024-09-15T21:47:11.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.041+0000 7f5910074640 1 -- 172.21.15.100:0/2016632836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f590806ccf0 msgr2=0x7f590806d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.041+0000 7f5910074640 1 --2- 172.21.15.100:0/2016632836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f590806ccf0 0x7f590806d160 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5900009e50 tx=0x7f590002a710 comp rx=0 tx=0).stop 2024-09-15T21:47:11.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.041+0000 7f5910074640 1 -- 172.21.15.100:0/2016632836 shutdown_connections 2024-09-15T21:47:11.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.041+0000 7f5910074640 1 --2- 172.21.15.100:0/2016632836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f590806ccf0 0x7f590806d160 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.041+0000 7f5910074640 1 --2- 172.21.15.100:0/2016632836 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5908073790 0x7f59080c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.041+0000 7f5910074640 1 -- 172.21.15.100:0/2016632836 >> 172.21.15.100:0/2016632836 conn(0x7f59080ca390 msgr2=0x7f59080ca770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:11.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.041+0000 7f5910074640 1 -- 172.21.15.100:0/2016632836 shutdown_connections 2024-09-15T21:47:11.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.041+0000 7f5910074640 1 -- 172.21.15.100:0/2016632836 wait complete. 2024-09-15T21:47:11.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.042+0000 7f5910074640 1 Processor -- start 2024-09-15T21:47:11.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.043+0000 7f5910074640 1 -- start start 2024-09-15T21:47:11.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.043+0000 7f5910074640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f590806ccf0 0x7f5908143070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.043+0000 7f5910074640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5908073790 0x7f5908160780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.043+0000 7f5910074640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5908152160 0x7f59081511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.043+0000 7f5910074640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5908158b20 con 0x7f5908152160 2024-09-15T21:47:11.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.043+0000 7f5910074640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5908158840 con 0x7f590806ccf0 2024-09-15T21:47:11.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.043+0000 7f5910074640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f590806efd0 con 0x7f5908073790 2024-09-15T21:47:11.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.043+0000 7f590e611640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5908152160 0x7f59081511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.043+0000 7f590e611640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5908152160 0x7f59081511b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49244/0 (socket says 172.21.15.100:49244) 2024-09-15T21:47:11.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.043+0000 7f590e611640 1 -- 172.21.15.100:0/560990746 learned_addr learned my addr 172.21.15.100:0/560990746 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:11.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.043+0000 7f590d60f640 1 --2- 172.21.15.100:0/560990746 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5908073790 0x7f5908160780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.043+0000 7f590de10640 1 --2- 172.21.15.100:0/560990746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f590806ccf0 0x7f5908143070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.044+0000 7f590e611640 1 -- 172.21.15.100:0/560990746 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5908073790 msgr2=0x7f5908160780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.044+0000 7f590e611640 1 --2- 172.21.15.100:0/560990746 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5908073790 0x7f5908160780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.044+0000 7f590e611640 1 -- 172.21.15.100:0/560990746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f590806ccf0 msgr2=0x7f5908143070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.044+0000 7f590e611640 1 --2- 172.21.15.100:0/560990746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f590806ccf0 0x7f5908143070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.044+0000 7f590e611640 1 -- 172.21.15.100:0/560990746 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59081f4b70 con 0x7f5908152160 2024-09-15T21:47:11.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.044+0000 7f590e611640 1 --2- 172.21.15.100:0/560990746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5908152160 0x7f59081511b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5900034040 tx=0x7f5900004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.044+0000 7f58fe7fc640 1 -- 172.21.15.100:0/560990746 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590003f020 con 0x7f5908152160 2024-09-15T21:47:11.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.044+0000 7f58fe7fc640 1 -- 172.21.15.100:0/560990746 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5900037040 con 0x7f5908152160 2024-09-15T21:47:11.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.044+0000 7f5910074640 1 -- 172.21.15.100:0/560990746 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59081f4e30 con 0x7f5908152160 2024-09-15T21:47:11.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.044+0000 7f58fe7fc640 1 -- 172.21.15.100:0/560990746 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900036070 con 0x7f5908152160 2024-09-15T21:47:11.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.044+0000 7f5910074640 1 -- 172.21.15.100:0/560990746 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59081f53f0 con 0x7f5908152160 2024-09-15T21:47:11.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.045+0000 7f5910074640 1 -- 172.21.15.100:0/560990746 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f59081f5a40 con 0x7f5908152160 2024-09-15T21:47:11.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.046+0000 7f58fe7fc640 1 -- 172.21.15.100:0/560990746 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5900042020 con 0x7f5908152160 2024-09-15T21:47:11.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.046+0000 7f58fe7fc640 1 --2- 172.21.15.100:0/560990746 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f58ec06c2b0 0x7f58ec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.046+0000 7f58fe7fc640 1 -- 172.21.15.100:0/560990746 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5854+0+0 (secure 0 0 0) 0x7f59000abe10 con 0x7f5908152160 2024-09-15T21:47:11.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.046+0000 7f590de10640 1 --2- 172.21.15.100:0/560990746 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f58ec06c2b0 0x7f58ec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.046+0000 7f58fe7fc640 1 --2- 172.21.15.100:0/560990746 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f58ec074690 0x7f58ec076b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.046+0000 7f58fe7fc640 1 -- 172.21.15.100:0/560990746 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f58ec077200 con 0x7f58ec074690 2024-09-15T21:47:11.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.046+0000 7f58fe7fc640 1 -- 172.21.15.100:0/560990746 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f59000ac2a0 con 0x7f5908152160 2024-09-15T21:47:11.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.046+0000 7f590d60f640 1 --2- 172.21.15.100:0/560990746 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f58ec074690 0x7f58ec076b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.046+0000 7f590de10640 1 --2- 172.21.15.100:0/560990746 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f58ec06c2b0 0x7f58ec06e770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f59040016e0 tx=0x7f5904009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.047+0000 7f590d60f640 1 --2- 172.21.15.100:0/560990746 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f58ec074690 0x7f58ec076b30 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.048+0000 7f58fe7fc640 1 -- 172.21.15.100:0/560990746 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f58ec077200 con 0x7f58ec074690 2024-09-15T21:47:11.065 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.063+0000 7f5910074640 1 -- 172.21.15.100:0/560990746 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f59081f1600 con 0x7f58ec074690 2024-09-15T21:47:11.066 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.064+0000 7f58fe7fc640 1 -- 172.21.15.100:0/560990746 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f58f8009870 con 0x7f58ec074690 2024-09-15T21:47:11.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.064+0000 7f5910074640 1 -- 172.21.15.100:0/560990746 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f58ec074690 msgr2=0x7f58ec076b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.064+0000 7f5910074640 1 --2- 172.21.15.100:0/560990746 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f58ec074690 0x7f58ec076b30 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.064+0000 7f5910074640 1 -- 172.21.15.100:0/560990746 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f58ec06c2b0 msgr2=0x7f58ec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.064+0000 7f5910074640 1 --2- 172.21.15.100:0/560990746 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f58ec06c2b0 0x7f58ec06e770 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f59040016e0 tx=0x7f5904009040 comp rx=0 tx=0).stop 2024-09-15T21:47:11.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.064+0000 7f5910074640 1 -- 172.21.15.100:0/560990746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5908152160 msgr2=0x7f59081511b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.064+0000 7f5910074640 1 --2- 172.21.15.100:0/560990746 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5908152160 0x7f59081511b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5900034040 tx=0x7f5900004a40 comp rx=0 tx=0).stop 2024-09-15T21:47:11.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.064+0000 7f590e611640 1 -- 172.21.15.100:0/560990746 reap_dead start 2024-09-15T21:47:11.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.065+0000 7f5910074640 1 -- 172.21.15.100:0/560990746 shutdown_connections 2024-09-15T21:47:11.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.065+0000 7f5910074640 1 -- 172.21.15.100:0/560990746 >> 172.21.15.100:0/560990746 conn(0x7f59080ca390 msgr2=0x7f590806d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:11.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.065+0000 7f5910074640 1 -- 172.21.15.100:0/560990746 shutdown_connections 2024-09-15T21:47:11.067 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.065+0000 7f5910074640 1 -- 172.21.15.100:0/560990746 wait complete. 2024-09-15T21:47:11.077 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-15T21:47:11.088 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:11.088 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:11.088 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:11.088 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:11.099 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:47:11.141 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:11.139+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:11.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.177+0000 7f55240eb640 1 Processor -- start 2024-09-15T21:47:11.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.177+0000 7f55240eb640 1 -- start start 2024-09-15T21:47:11.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.178+0000 7f55240eb640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f551c0c7ec0 0x7f551c06df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.178+0000 7f55240eb640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f551c0c2150 0x7f551c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.178+0000 7f55240eb640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f551c06d4c0 con 0x7f551c0c82a0 2024-09-15T21:47:11.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.178+0000 7f55240eb640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f551c06dad0 con 0x7f551c0c2150 2024-09-15T21:47:11.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.178+0000 7f55240eb640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f551c0b1250 con 0x7f551c0c7ec0 2024-09-15T21:47:11.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.178+0000 7f5522688640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f551c0c2150 0x7f551c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.178+0000 7f5522688640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f551c0c2150 0x7f551c06e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49246/0 (socket says 172.21.15.100:49246) 2024-09-15T21:47:11.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.178+0000 7f5522688640 1 -- 172.21.15.100:0/3898247970 learned_addr learned my addr 172.21.15.100:0/3898247970 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:11.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.178+0000 7f5521e87640 1 --2- 172.21.15.100:0/3898247970 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f551c0c7ec0 0x7f551c06df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.178+0000 7f5522688640 1 -- 172.21.15.100:0/3898247970 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f551c0c7ec0 msgr2=0x7f551c06df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.178+0000 7f5522688640 1 --2- 172.21.15.100:0/3898247970 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f551c0c7ec0 0x7f551c06df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.178+0000 7f5522688640 1 -- 172.21.15.100:0/3898247970 >> v1:172.21.15.122:6789/0 conn(0x7f551c0c82a0 legacy=0x7f551c15ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.178+0000 7f5522688640 1 -- 172.21.15.100:0/3898247970 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551c0b1530 con 0x7f551c0c2150 2024-09-15T21:47:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.178+0000 7f5521e87640 1 --2- 172.21.15.100:0/3898247970 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f551c0c7ec0 0x7f551c06df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:47:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.179+0000 7f5522688640 1 --2- 172.21.15.100:0/3898247970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0c2150 0x7f551c06e4a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f551c06dc50 tx=0x7f551402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676d55875860805b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.179+0000 7f5520e85640 1 -- 172.21.15.100:0/3898247970 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5514036070 con 0x7f551c0c2150 2024-09-15T21:47:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.179+0000 7f5520e85640 1 -- 172.21.15.100:0/3898247970 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5514031600 con 0x7f551c0c2150 2024-09-15T21:47:11.182 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.179+0000 7f5520e85640 1 -- 172.21.15.100:0/3898247970 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55140319d0 con 0x7f551c0c2150 2024-09-15T21:47:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.180+0000 7f55240eb640 1 -- 172.21.15.100:0/3898247970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0c2150 msgr2=0x7f551c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.180+0000 7f55240eb640 1 --2- 172.21.15.100:0/3898247970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0c2150 0x7f551c06e4a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f551c06dc50 tx=0x7f551402a6c0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.180+0000 7f55240eb640 1 -- 172.21.15.100:0/3898247970 shutdown_connections 2024-09-15T21:47:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.180+0000 7f55240eb640 1 --2- 172.21.15.100:0/3898247970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0c2150 0x7f551c06e4a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.180+0000 7f55240eb640 1 --2- 172.21.15.100:0/3898247970 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f551c0c7ec0 0x7f551c06df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.180+0000 7f55240eb640 1 -- 172.21.15.100:0/3898247970 >> 172.21.15.100:0/3898247970 conn(0x7f551c0cd130 msgr2=0x7f551c0cd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.180+0000 7f55240eb640 1 -- 172.21.15.100:0/3898247970 shutdown_connections 2024-09-15T21:47:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.180+0000 7f55240eb640 1 -- 172.21.15.100:0/3898247970 wait complete. 2024-09-15T21:47:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.181+0000 7f55240eb640 1 Processor -- start 2024-09-15T21:47:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.181+0000 7f55240eb640 1 -- start start 2024-09-15T21:47:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f55240eb640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f551c0c2150 0x7f551c0c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f55240eb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0c7ec0 0x7f551c0c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f55240eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f551c0c82a0 0x7f551c152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f55240eb640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f551c0b1810 con 0x7f551c0c7ec0 2024-09-15T21:47:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f55240eb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f551c0b0a40 con 0x7f551c0c82a0 2024-09-15T21:47:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f55240eb640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f551c0c98a0 con 0x7f551c0c2150 2024-09-15T21:47:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f5521686640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0c7ec0 0x7f551c0c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f5521686640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0c7ec0 0x7f551c0c6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49252/0 (socket says 172.21.15.100:49252) 2024-09-15T21:47:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f5521686640 1 -- 172.21.15.100:0/2745126348 learned_addr learned my addr 172.21.15.100:0/2745126348 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f5521e87640 1 --2- 172.21.15.100:0/2745126348 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f551c0c2150 0x7f551c0c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f5522688640 1 --2- 172.21.15.100:0/2745126348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f551c0c82a0 0x7f551c152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f5521686640 1 -- 172.21.15.100:0/2745126348 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f551c0c2150 msgr2=0x7f551c0c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f5521686640 1 --2- 172.21.15.100:0/2745126348 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f551c0c2150 0x7f551c0c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f5521686640 1 -- 172.21.15.100:0/2745126348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f551c0c82a0 msgr2=0x7f551c152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f5521686640 1 --2- 172.21.15.100:0/2745126348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f551c0c82a0 0x7f551c152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f5521686640 1 -- 172.21.15.100:0/2745126348 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f551c1548b0 con 0x7f551c0c7ec0 2024-09-15T21:47:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f5522688640 1 --2- 172.21.15.100:0/2745126348 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f551c0c82a0 0x7f551c152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:47:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f5521686640 1 --2- 172.21.15.100:0/2745126348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0c7ec0 0x7f551c0c6d20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f550c00b3c0 tx=0x7f550c00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f55127fc640 1 -- 172.21.15.100:0/2745126348 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c011830 con 0x7f551c0c7ec0 2024-09-15T21:47:11.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f55127fc640 1 -- 172.21.15.100:0/2745126348 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f550c016070 con 0x7f551c0c7ec0 2024-09-15T21:47:11.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f55240eb640 1 -- 172.21.15.100:0/2745126348 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f551c1f4730 con 0x7f551c0c7ec0 2024-09-15T21:47:11.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f55127fc640 1 -- 172.21.15.100:0/2745126348 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f550c011c50 con 0x7f551c0c7ec0 2024-09-15T21:47:11.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.182+0000 7f55240eb640 1 -- 172.21.15.100:0/2745126348 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f551c1f4cf0 con 0x7f551c0c7ec0 2024-09-15T21:47:11.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.183+0000 7f55240eb640 1 -- 172.21.15.100:0/2745126348 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f551c0b0ed0 con 0x7f551c0c7ec0 2024-09-15T21:47:11.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.185+0000 7f55127fc640 1 -- 172.21.15.100:0/2745126348 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f550c01b460 con 0x7f551c0c7ec0 2024-09-15T21:47:11.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.185+0000 7f55127fc640 1 --2- 172.21.15.100:0/2745126348 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f550006c2e0 0x7f550006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.185+0000 7f55127fc640 1 -- 172.21.15.100:0/2745126348 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5854+0+0 (secure 0 0 0) 0x7f550c08ae70 con 0x7f551c0c7ec0 2024-09-15T21:47:11.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.185+0000 7f5521e87640 1 --2- 172.21.15.100:0/2745126348 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f550006c2e0 0x7f550006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.185+0000 7f55127fc640 1 --2- 172.21.15.100:0/2745126348 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f55000746c0 0x7f5500076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.185+0000 7f55127fc640 1 -- 172.21.15.100:0/2745126348 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5500077230 con 0x7f55000746c0 2024-09-15T21:47:11.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.185+0000 7f55127fc640 1 -- 172.21.15.100:0/2745126348 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f550c08d570 con 0x7f551c0c7ec0 2024-09-15T21:47:11.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.186+0000 7f5522688640 1 --2- 172.21.15.100:0/2745126348 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f55000746c0 0x7f5500076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.186+0000 7f5521e87640 1 --2- 172.21.15.100:0/2745126348 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f550006c2e0 0x7f550006e7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5518006fd0 tx=0x7f5518008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.186+0000 7f5522688640 1 --2- 172.21.15.100:0/2745126348 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f55000746c0 0x7f5500076b60 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.187+0000 7f55127fc640 1 -- 172.21.15.100:0/2745126348 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5500077230 con 0x7f55000746c0 2024-09-15T21:47:11.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.202+0000 7f55240eb640 1 -- 172.21.15.100:0/2745126348 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 2: {"prefix": "reset_purged_snaps_last"}) -- 0x7f551c0c8790 con 0x7f55000746c0 2024-09-15T21:47:11.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.203+0000 7f55127fc640 1 -- 172.21.15.100:0/2745126348 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 2: 0 ) ==== 8+0+0 (crc 0 0 0) 0x7f551c0c8790 con 0x7f55000746c0 2024-09-15T21:47:11.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.203+0000 7f55240eb640 1 -- 172.21.15.100:0/2745126348 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f55000746c0 msgr2=0x7f5500076b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.203+0000 7f55240eb640 1 --2- 172.21.15.100:0/2745126348 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f55000746c0 0x7f5500076b60 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.206 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.203+0000 7f55240eb640 1 -- 172.21.15.100:0/2745126348 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f550006c2e0 msgr2=0x7f550006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.203+0000 7f55240eb640 1 --2- 172.21.15.100:0/2745126348 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f550006c2e0 0x7f550006e7a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5518006fd0 tx=0x7f5518008040 comp rx=0 tx=0).stop 2024-09-15T21:47:11.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.203+0000 7f55240eb640 1 -- 172.21.15.100:0/2745126348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0c7ec0 msgr2=0x7f551c0c6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.203+0000 7f55240eb640 1 --2- 172.21.15.100:0/2745126348 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f551c0c7ec0 0x7f551c0c6d20 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f550c00b3c0 tx=0x7f550c00b880 comp rx=0 tx=0).stop 2024-09-15T21:47:11.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.203+0000 7f5522688640 1 -- 172.21.15.100:0/2745126348 reap_dead start 2024-09-15T21:47:11.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.204+0000 7f55240eb640 1 -- 172.21.15.100:0/2745126348 shutdown_connections 2024-09-15T21:47:11.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.204+0000 7f55240eb640 1 -- 172.21.15.100:0/2745126348 >> 172.21.15.100:0/2745126348 conn(0x7f551c0cd130 msgr2=0x7f551c0c2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:11.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.204+0000 7f55240eb640 1 -- 172.21.15.100:0/2745126348 shutdown_connections 2024-09-15T21:47:11.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.204+0000 7f55240eb640 1 -- 172.21.15.100:0/2745126348 wait complete. 2024-09-15T21:47:11.248 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:11.248 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:11.248 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:11.249 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:11.249 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:11.249 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:11.247+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:11.257 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:47:11.342 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:11.341+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:11.403 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:11.403 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:11.403 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:11.403 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:11.403 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.403 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:11.403 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:11.403 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.986825068000002, 2024-09-15T21:47:11.403 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00598016, 2024-09-15T21:47:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:11.404 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413339+0000", 2024-09-15T21:47:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6279999999999998e-06 2024-09-15T21:47:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413354+0000", 2024-09-15T21:47:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4877e-05 2024-09-15T21:47:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.405 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413418+0000", 2024-09-15T21:47:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4274000000000002e-05 2024-09-15T21:47:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413428+0000", 2024-09-15T21:47:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5899999999999997e-06 2024-09-15T21:47:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413563+0000", 2024-09-15T21:47:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013521900000000001 2024-09-15T21:47:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413678+0000", 2024-09-15T21:47:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000114904 2024-09-15T21:47:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413789+0000", 2024-09-15T21:47:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011069300000000001 2024-09-15T21:47:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417603+0000", 2024-09-15T21:47:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038138460000000001 2024-09-15T21:47:11.407 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419220+0000", 2024-09-15T21:47:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001617678 2024-09-15T21:47:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419275+0000", 2024-09-15T21:47:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4391999999999997e-05 2024-09-15T21:47:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419316+0000", 2024-09-15T21:47:11.408 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1059000000000001e-05 2024-09-15T21:47:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.986802939999997, 2024-09-15T21:47:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0087926470000000007, 2024-09-15T21:47:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:11.409 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.410 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4099999999999999e-07 2024-09-15T21:47:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413361+0000", 2024-09-15T21:47:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6189999999999998e-06 2024-09-15T21:47:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413461+0000", 2024-09-15T21:47:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000100053 2024-09-15T21:47:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.411 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413466+0000", 2024-09-15T21:47:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3850000000000001e-06 2024-09-15T21:47:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414073+0000", 2024-09-15T21:47:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000606724 2024-09-15T21:47:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414143+0000", 2024-09-15T21:47:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9727000000000005e-05 2024-09-15T21:47:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.412 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414259+0000", 2024-09-15T21:47:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000116178 2024-09-15T21:47:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417610+0000", 2024-09-15T21:47:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003351176 2024-09-15T21:47:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422058+0000", 2024-09-15T21:47:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004447599 2024-09-15T21:47:11.413 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422111+0000", 2024-09-15T21:47:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3606999999999997e-05 2024-09-15T21:47:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422150+0000", 2024-09-15T21:47:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9137999999999997e-05 2024-09-15T21:47:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.414 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.98679851, 2024-09-15T21:47:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0089009769999999992, 2024-09-15T21:47:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.415 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413363+0000", 2024-09-15T21:47:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3999999999999997e-07 2024-09-15T21:47:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413365+0000", 2024-09-15T21:47:11.416 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.216e-06 2024-09-15T21:47:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413481+0000", 2024-09-15T21:47:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000116311 2024-09-15T21:47:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413488+0000", 2024-09-15T21:47:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0110000000000003e-06 2024-09-15T21:47:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.417 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414452+0000", 2024-09-15T21:47:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00096385399999999999 2024-09-15T21:47:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414545+0000", 2024-09-15T21:47:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.3024999999999998e-05 2024-09-15T21:47:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414644+0000", 2024-09-15T21:47:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9098000000000006e-05 2024-09-15T21:47:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.418 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417667+0000", 2024-09-15T21:47:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030229409999999999 2024-09-15T21:47:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422200+0000", 2024-09-15T21:47:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045327880000000003 2024-09-15T21:47:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422228+0000", 2024-09-15T21:47:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8109000000000001e-05 2024-09-15T21:47:11.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422263+0000", 2024-09-15T21:47:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5284000000000003e-05 2024-09-15T21:47:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.986794924000002, 2024-09-15T21:47:11.420 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0089846820000000008, 2024-09-15T21:47:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7599999999999998e-07 2024-09-15T21:47:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413368+0000", 2024-09-15T21:47:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.058e-06 2024-09-15T21:47:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413832+0000", 2024-09-15T21:47:11.422 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046398700000000001 2024-09-15T21:47:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413841+0000", 2024-09-15T21:47:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1710000000000008e-06 2024-09-15T21:47:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414869+0000", 2024-09-15T21:47:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001027584 2024-09-15T21:47:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.423 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414946+0000", 2024-09-15T21:47:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7177999999999999e-05 2024-09-15T21:47:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415054+0000", 2024-09-15T21:47:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010807 2024-09-15T21:47:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417672+0000", 2024-09-15T21:47:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002618015 2024-09-15T21:47:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422295+0000", 2024-09-15T21:47:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046233300000000001 2024-09-15T21:47:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422330+0000", 2024-09-15T21:47:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4476e-05 2024-09-15T21:47:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422351+0000", 2024-09-15T21:47:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0537e-05 2024-09-15T21:47:11.425 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.986791564000001, 2024-09-15T21:47:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0090873700000000009, 2024-09-15T21:47:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6100000000000002e-07 2024-09-15T21:47:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413371+0000", 2024-09-15T21:47:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.764e-06 2024-09-15T21:47:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413859+0000", 2024-09-15T21:47:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00048751900000000001 2024-09-15T21:47:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413864+0000", 2024-09-15T21:47:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6200000000000004e-06 2024-09-15T21:47:11.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415253+0000", 2024-09-15T21:47:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013884050000000001 2024-09-15T21:47:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415364+0000", 2024-09-15T21:47:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000111119 2024-09-15T21:47:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:11.429 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415453+0000", 2024-09-15T21:47:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8641999999999996e-05 2024-09-15T21:47:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417676+0000", 2024-09-15T21:47:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00222368 2024-09-15T21:47:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422395+0000", 2024-09-15T21:47:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047191150000000003 2024-09-15T21:47:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422434+0000", 2024-09-15T21:47:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8525999999999997e-05 2024-09-15T21:47:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422457+0000", 2024-09-15T21:47:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2719000000000001e-05 2024-09-15T21:47:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.431 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.986788541999999, 2024-09-15T21:47:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0092139749999999992, 2024-09-15T21:47:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.432 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3799999999999999e-07 2024-09-15T21:47:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413375+0000", 2024-09-15T21:47:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1959999999999998e-06 2024-09-15T21:47:11.433 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413876+0000", 2024-09-15T21:47:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00050134500000000005 2024-09-15T21:47:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413881+0000", 2024-09-15T21:47:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3580000000000002e-06 2024-09-15T21:47:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415669+0000", 2024-09-15T21:47:11.434 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0017875090000000001 2024-09-15T21:47:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415737+0000", 2024-09-15T21:47:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8386000000000006e-05 2024-09-15T21:47:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415857+0000", 2024-09-15T21:47:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119906 2024-09-15T21:47:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:11.435 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421848+0000", 2024-09-15T21:47:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059903880000000001 2024-09-15T21:47:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422513+0000", 2024-09-15T21:47:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00066575599999999999 2024-09-15T21:47:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422549+0000", 2024-09-15T21:47:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5719999999999997e-05 2024-09-15T21:47:11.436 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422586+0000", 2024-09-15T21:47:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7172999999999998e-05 2024-09-15T21:47:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.986782720000001, 2024-09-15T21:47:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.009324087, 2024-09-15T21:47:11.437 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.438 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8900000000000001e-07 2024-09-15T21:47:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413381+0000", 2024-09-15T21:47:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1959999999999998e-06 2024-09-15T21:47:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413895+0000", 2024-09-15T21:47:11.439 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051466000000000001 2024-09-15T21:47:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413900+0000", 2024-09-15T21:47:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6090000000000003e-06 2024-09-15T21:47:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416081+0000", 2024-09-15T21:47:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021816090000000002 2024-09-15T21:47:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.440 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416173+0000", 2024-09-15T21:47:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1334000000000003e-05 2024-09-15T21:47:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416264+0000", 2024-09-15T21:47:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0864000000000003e-05 2024-09-15T21:47:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421854+0000", 2024-09-15T21:47:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055902240000000004 2024-09-15T21:47:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.441 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422629+0000", 2024-09-15T21:47:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00077550700000000004 2024-09-15T21:47:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422678+0000", 2024-09-15T21:47:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8810999999999997e-05 2024-09-15T21:47:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422702+0000", 2024-09-15T21:47:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3983999999999999e-05 2024-09-15T21:47:11.442 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.986779237999997, 2024-09-15T21:47:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0094270580000000003, 2024-09-15T21:47:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.443 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:11.444 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7500000000000001e-07 2024-09-15T21:47:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413386+0000", 2024-09-15T21:47:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5619999999999997e-06 2024-09-15T21:47:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413918+0000", 2024-09-15T21:47:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00053162499999999998 2024-09-15T21:47:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.445 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413923+0000", 2024-09-15T21:47:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9280000000000001e-06 2024-09-15T21:47:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416492+0000", 2024-09-15T21:47:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0025687140000000002 2024-09-15T21:47:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416558+0000", 2024-09-15T21:47:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6699000000000006e-05 2024-09-15T21:47:11.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416691+0000", 2024-09-15T21:47:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013263199999999999 2024-09-15T21:47:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421879+0000", 2024-09-15T21:47:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051881089999999998 2024-09-15T21:47:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:11.447 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422742+0000", 2024-09-15T21:47:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000862845 2024-09-15T21:47:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422786+0000", 2024-09-15T21:47:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4332000000000002e-05 2024-09-15T21:47:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422809+0000", 2024-09-15T21:47:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2337e-05 2024-09-15T21:47:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.448 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:4 2.10 2:0f65c9b8:::60a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 36.944493069000004, 2024-09-15T21:47:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058710300000000002, 2024-09-15T21:47:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.449 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0800000000000005e-07 2024-09-15T21:47:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.450 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455672+0000", 2024-09-15T21:47:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3900000000000002e-06 2024-09-15T21:47:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456538+0000", 2024-09-15T21:47:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00086613100000000002 2024-09-15T21:47:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456545+0000", 2024-09-15T21:47:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4830000000000002e-06 2024-09-15T21:47:11.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456594+0000", 2024-09-15T21:47:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8429e-05 2024-09-15T21:47:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456675+0000", 2024-09-15T21:47:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1516999999999999e-05 2024-09-15T21:47:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456772+0000", 2024-09-15T21:47:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6523999999999999e-05 2024-09-15T21:47:11.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.460299+0000", 2024-09-15T21:47:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035274859999999998 2024-09-15T21:47:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461451+0000", 2024-09-15T21:47:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011518100000000001 2024-09-15T21:47:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:11.453 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461498+0000", 2024-09-15T21:47:11.454 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7406e-05 2024-09-15T21:47:11.454 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.454 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.454 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.454 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461539+0000", 2024-09-15T21:47:11.454 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0346000000000003e-05 2024-09-15T21:47:11.454 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.454 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.454 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:31 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 36.944487807000002, 2024-09-15T21:47:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0078158979999999999, 2024-09-15T21:47:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.455 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0499999999999999e-07 2024-09-15T21:47:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.456 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.457 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455675+0000", 2024-09-15T21:47:11.457 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.376e-06 2024-09-15T21:47:11.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.457 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.457 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456964+0000", 2024-09-15T21:47:11.457 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001289204 2024-09-15T21:47:11.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.457 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456970+0000", 2024-09-15T21:47:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2129999999999996e-06 2024-09-15T21:47:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457032+0000", 2024-09-15T21:47:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2209e-05 2024-09-15T21:47:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457134+0000", 2024-09-15T21:47:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010204000000000001 2024-09-15T21:47:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.458 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463489+0000", 2024-09-15T21:47:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0063545509999999999 2024-09-15T21:47:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 30.924728043000002, 2024-09-15T21:47:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057594079999999997, 2024-09-15T21:47:11.459 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:11.460 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1999999999999999e-07 2024-09-15T21:47:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475437+0000", 2024-09-15T21:47:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5429999999999998e-06 2024-09-15T21:47:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.461 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475686+0000", 2024-09-15T21:47:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000248908 2024-09-15T21:47:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475695+0000", 2024-09-15T21:47:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.9740000000000008e-06 2024-09-15T21:47:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476291+0000", 2024-09-15T21:47:11.462 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00059651500000000004 2024-09-15T21:47:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476366+0000", 2024-09-15T21:47:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4531999999999997e-05 2024-09-15T21:47:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476491+0000", 2024-09-15T21:47:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012527299999999999 2024-09-15T21:47:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.463 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480600+0000", 2024-09-15T21:47:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0041085870000000003 2024-09-15T21:47:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481089+0000", 2024-09-15T21:47:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000489037 2024-09-15T21:47:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481152+0000", 2024-09-15T21:47:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3136e-05 2024-09-15T21:47:11.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481192+0000", 2024-09-15T21:47:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0182999999999998e-05 2024-09-15T21:47:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:17 2.10 2:0e9da45e:::61b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 30.924722381999999, 2024-09-15T21:47:11.465 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058311830000000002, 2024-09-15T21:47:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.466 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475439+0000", 2024-09-15T21:47:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1300000000000001e-07 2024-09-15T21:47:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475441+0000", 2024-09-15T21:47:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.419e-06 2024-09-15T21:47:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000269187 2024-09-15T21:47:11.467 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475715+0000", 2024-09-15T21:47:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3660000000000002e-06 2024-09-15T21:47:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476671+0000", 2024-09-15T21:47:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000956507 2024-09-15T21:47:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477500+0000", 2024-09-15T21:47:11.468 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00082897400000000001 2024-09-15T21:47:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477625+0000", 2024-09-15T21:47:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012513100000000001 2024-09-15T21:47:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480520+0000", 2024-09-15T21:47:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002894715 2024-09-15T21:47:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:11.469 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481184+0000", 2024-09-15T21:47:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000663801 2024-09-15T21:47:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481232+0000", 2024-09-15T21:47:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8457e-05 2024-09-15T21:47:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481270+0000", 2024-09-15T21:47:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7313e-05 2024-09-15T21:47:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.470 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 30.924718506000001, 2024-09-15T21:47:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0075605530000000002, 2024-09-15T21:47:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.471 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.53e-07 2024-09-15T21:47:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.472 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.473 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.473 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475445+0000", 2024-09-15T21:47:11.473 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7360000000000001e-06 2024-09-15T21:47:11.473 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.473 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.473 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.473 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475735+0000", 2024-09-15T21:47:11.473 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028975799999999999 2024-09-15T21:47:11.473 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.474 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.474 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.474 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475740+0000", 2024-09-15T21:47:11.474 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8250000000000004e-06 2024-09-15T21:47:11.474 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.474 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.474 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.474 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478479+0000", 2024-09-15T21:47:11.474 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002739525 2024-09-15T21:47:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478556+0000", 2024-09-15T21:47:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6213999999999999e-05 2024-09-15T21:47:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478672+0000", 2024-09-15T21:47:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011605 2024-09-15T21:47:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:11.475 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481323+0000", 2024-09-15T21:47:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0026518819999999999 2024-09-15T21:47:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482918+0000", 2024-09-15T21:47:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001594105 2024-09-15T21:47:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482965+0000", 2024-09-15T21:47:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6950000000000003e-05 2024-09-15T21:47:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.476 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.477 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.477 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483003+0000", 2024-09-15T21:47:11.477 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8254999999999999e-05 2024-09-15T21:47:11.477 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.477 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.477 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.477 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.478 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:11.478 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:11.478 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 30.92471445, 2024-09-15T21:47:11.478 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076582890000000004, 2024-09-15T21:47:11.478 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.478 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.479 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475447+0000", 2024-09-15T21:47:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4699999999999998e-07 2024-09-15T21:47:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475449+0000", 2024-09-15T21:47:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.176e-06 2024-09-15T21:47:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475752+0000", 2024-09-15T21:47:11.480 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030374200000000002 2024-09-15T21:47:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475757+0000", 2024-09-15T21:47:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2209999999999999e-06 2024-09-15T21:47:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477441+0000", 2024-09-15T21:47:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016843489999999999 2024-09-15T21:47:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.481 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477567+0000", 2024-09-15T21:47:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012552099999999999 2024-09-15T21:47:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477712+0000", 2024-09-15T21:47:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000145427 2024-09-15T21:47:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482935+0000", 2024-09-15T21:47:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052231200000000004 2024-09-15T21:47:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.482 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483015+0000", 2024-09-15T21:47:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9634000000000004e-05 2024-09-15T21:47:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483065+0000", 2024-09-15T21:47:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9869e-05 2024-09-15T21:47:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483105+0000", 2024-09-15T21:47:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9983e-05 2024-09-15T21:47:11.483 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 30.924707908999999, 2024-09-15T21:47:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076459379999999997, 2024-09-15T21:47:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.484 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.58e-07 2024-09-15T21:47:11.485 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475455+0000", 2024-09-15T21:47:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.075e-06 2024-09-15T21:47:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475769+0000", 2024-09-15T21:47:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00031336999999999999 2024-09-15T21:47:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.486 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475773+0000", 2024-09-15T21:47:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8080000000000004e-06 2024-09-15T21:47:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478950+0000", 2024-09-15T21:47:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031763870000000001 2024-09-15T21:47:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479067+0000", 2024-09-15T21:47:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000117543 2024-09-15T21:47:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.487 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479196+0000", 2024-09-15T21:47:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000128987 2024-09-15T21:47:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481366+0000", 2024-09-15T21:47:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021699089999999998 2024-09-15T21:47:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483015+0000", 2024-09-15T21:47:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016486719999999999 2024-09-15T21:47:11.488 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483061+0000", 2024-09-15T21:47:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5729999999999998e-05 2024-09-15T21:47:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483099+0000", 2024-09-15T21:47:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8099000000000002e-05 2024-09-15T21:47:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.489 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 30.924660800000002, 2024-09-15T21:47:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077290859999999996, 2024-09-15T21:47:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.490 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475501+0000", 2024-09-15T21:47:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1900000000000002e-07 2024-09-15T21:47:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.491 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475504+0000", 2024-09-15T21:47:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.168e-06 2024-09-15T21:47:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477037+0000", 2024-09-15T21:47:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001533202 2024-09-15T21:47:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477050+0000", 2024-09-15T21:47:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3132999999999999e-05 2024-09-15T21:47:11.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479477+0000", 2024-09-15T21:47:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002426489 2024-09-15T21:47:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479553+0000", 2024-09-15T21:47:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6583999999999997e-05 2024-09-15T21:47:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:11.493 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479637+0000", 2024-09-15T21:47:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4172999999999995e-05 2024-09-15T21:47:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483108+0000", 2024-09-15T21:47:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003470356 2024-09-15T21:47:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483160+0000", 2024-09-15T21:47:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2507999999999999e-05 2024-09-15T21:47:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.494 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483202+0000", 2024-09-15T21:47:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1758999999999998e-05 2024-09-15T21:47:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483229+0000", 2024-09-15T21:47:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6995000000000002e-05 2024-09-15T21:47:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:11.495 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 30.924655843, 2024-09-15T21:47:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.55145717800000005, 2024-09-15T21:47:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.496 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.65e-07 2024-09-15T21:47:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475507+0000", 2024-09-15T21:47:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4699999999999999e-06 2024-09-15T21:47:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.497 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.025935+0000", 2024-09-15T21:47:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.55042854200000002 2024-09-15T21:47:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.025965+0000", 2024-09-15T21:47:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9995999999999999e-05 2024-09-15T21:47:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026016+0000", 2024-09-15T21:47:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1310000000000002e-05 2024-09-15T21:47:11.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.499 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.499 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026122+0000", 2024-09-15T21:47:11.499 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105605 2024-09-15T21:47:11.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.499 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.499 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026962+0000", 2024-09-15T21:47:11.499 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00083998999999999999 2024-09-15T21:47:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:5 2.e 2:75ee5fd8:::629.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.835449478000001, 2024-09-15T21:47:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0071955470000000001, 2024-09-15T21:47:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:11.500 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.501 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564713+0000", 2024-09-15T21:47:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.866e-06 2024-09-15T21:47:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564721+0000", 2024-09-15T21:47:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3299999999999999e-06 2024-09-15T21:47:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564762+0000", 2024-09-15T21:47:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0707999999999997e-05 2024-09-15T21:47:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.502 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564774+0000", 2024-09-15T21:47:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1883e-05 2024-09-15T21:47:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564877+0000", 2024-09-15T21:47:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010254700000000001 2024-09-15T21:47:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565066+0000", 2024-09-15T21:47:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018933700000000001 2024-09-15T21:47:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.503 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565164+0000", 2024-09-15T21:47:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7633999999999994e-05 2024-09-15T21:47:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569030+0000", 2024-09-15T21:47:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038669070000000002 2024-09-15T21:47:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:11.504 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571818+0000", 2024-09-15T21:47:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027878930000000001 2024-09-15T21:47:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571868+0000", 2024-09-15T21:47:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9880999999999999e-05 2024-09-15T21:47:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571907+0000", 2024-09-15T21:47:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8560999999999999e-05 2024-09-15T21:47:11.505 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:6 2.e 2:75ee5fd8:::629.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.835435351000001, 2024-09-15T21:47:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073433220000000002, 2024-09-15T21:47:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.506 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564726+0000", 2024-09-15T21:47:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7399999999999999e-07 2024-09-15T21:47:11.507 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564728+0000", 2024-09-15T21:47:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.447e-06 2024-09-15T21:47:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565446+0000", 2024-09-15T21:47:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00071774999999999996 2024-09-15T21:47:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.508 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565453+0000", 2024-09-15T21:47:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3239999999999997e-06 2024-09-15T21:47:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565515+0000", 2024-09-15T21:47:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1768999999999998e-05 2024-09-15T21:47:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565591+0000", 2024-09-15T21:47:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5873999999999999e-05 2024-09-15T21:47:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:11.509 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565709+0000", 2024-09-15T21:47:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011838899999999999 2024-09-15T21:47:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569039+0000", 2024-09-15T21:47:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0033292170000000002 2024-09-15T21:47:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571977+0000", 2024-09-15T21:47:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0029387829999999999 2024-09-15T21:47:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.510 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.572032+0000", 2024-09-15T21:47:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4540999999999998e-05 2024-09-15T21:47:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.572069+0000", 2024-09-15T21:47:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6854000000000003e-05 2024-09-15T21:47:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:2 2.1 2:82971f97:::404.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:11.511 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 19.835431384, 2024-09-15T21:47:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007245357, 2024-09-15T21:47:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:11.512 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564730+0000", 2024-09-15T21:47:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2999999999999999e-07 2024-09-15T21:47:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564731+0000", 2024-09-15T21:47:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.367e-06 2024-09-15T21:47:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565484+0000", 2024-09-15T21:47:11.513 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075311599999999996 2024-09-15T21:47:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565490+0000", 2024-09-15T21:47:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8100000000000003e-06 2024-09-15T21:47:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565828+0000", 2024-09-15T21:47:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000337836 2024-09-15T21:47:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565964+0000", 2024-09-15T21:47:11.514 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013641599999999999 2024-09-15T21:47:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566071+0000", 2024-09-15T21:47:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000107126 2024-09-15T21:47:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571361+0000", 2024-09-15T21:47:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052896210000000004 2024-09-15T21:47:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.515 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571874+0000", 2024-09-15T21:47:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051320599999999999 2024-09-15T21:47:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571936+0000", 2024-09-15T21:47:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1650999999999996e-05 2024-09-15T21:47:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571975+0000", 2024-09-15T21:47:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8977999999999998e-05 2024-09-15T21:47:11.516 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.517 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.517 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.517 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:11.517 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:11.517 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:11.518 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:47:11.519 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:47:11.517+0000 7f74313af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37847) UID: 0 2024-09-15T21:47:11.619 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:11.618+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:11.649 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:47:11.655 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:47:11.665 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:11.665 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:11.665 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:11.665 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:11.675 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:47:11.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.750+0000 7f4a0cce6640 1 Processor -- start 2024-09-15T21:47:11.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.750+0000 7f4a0cce6640 1 -- start start 2024-09-15T21:47:11.752 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:11.750+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:11.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.751+0000 7f4a0cce6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4a080cb7d0 0x7f4a080cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.751+0000 7f4a0cce6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4a0806d0e0 0x7f4a080cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.751+0000 7f4a0cce6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a0815e7a0 con 0x7f4a08155440 2024-09-15T21:47:11.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.751+0000 7f4a0cce6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f4a0815b470 con 0x7f4a0806d0e0 2024-09-15T21:47:11.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.751+0000 7f4a0cce6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4a08155820 con 0x7f4a080cb7d0 2024-09-15T21:47:11.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.751+0000 7f4a06d9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4a0806d0e0 0x7f4a080cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.751+0000 7f4a0659c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f4a08155440 0x7f4a080cd420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:34038/0 (socket says 172.21.15.100:34038) 2024-09-15T21:47:11.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.751+0000 7f4a0659c640 1 -- 172.21.15.100:0/501470826 learned_addr learned my addr 172.21.15.100:0/501470826 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:11.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.751+0000 7f4a05d9b640 1 --2- 172.21.15.100:0/501470826 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4a080cb7d0 0x7f4a080cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.751+0000 7f4a0559a640 1 -- 172.21.15.100:0/501470826 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4009463892 0 0) 0x7f4a0815e7a0 con 0x7f4a08155440 2024-09-15T21:47:11.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.751+0000 7f4a0559a640 1 -- 172.21.15.100:0/501470826 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f49f0003520 con 0x7f4a08155440 2024-09-15T21:47:11.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.751+0000 7f4a06d9d640 1 -- 172.21.15.100:0/501470826 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4a080cb7d0 msgr2=0x7f4a080cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:11.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.751+0000 7f4a06d9d640 1 --2- 172.21.15.100:0/501470826 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4a080cb7d0 0x7f4a080cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.751+0000 7f4a06d9d640 1 -- 172.21.15.100:0/501470826 >> v1:172.21.15.100:6789/0 conn(0x7f4a08155440 legacy=0x7f4a080cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.751+0000 7f4a06d9d640 1 -- 172.21.15.100:0/501470826 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0815ea40 con 0x7f4a0806d0e0 2024-09-15T21:47:11.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.752+0000 7f4a06d9d640 1 --2- 172.21.15.100:0/501470826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a0806d0e0 0x7f4a080cc720 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009d90 tx=0x7f4a0002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cdb63d2db8095b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.752+0000 7f4a0559a640 1 -- 172.21.15.100:0/501470826 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00031600 con 0x7f4a0806d0e0 2024-09-15T21:47:11.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.752+0000 7f4a0559a640 1 -- 172.21.15.100:0/501470826 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a000317a0 con 0x7f4a0806d0e0 2024-09-15T21:47:11.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.752+0000 7f4a0559a640 1 -- 172.21.15.100:0/501470826 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00031b70 con 0x7f4a0806d0e0 2024-09-15T21:47:11.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.753+0000 7f4a0cce6640 1 -- 172.21.15.100:0/501470826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a0806d0e0 msgr2=0x7f4a080cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.753+0000 7f4a0cce6640 1 --2- 172.21.15.100:0/501470826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a0806d0e0 0x7f4a080cc720 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009d90 tx=0x7f4a0002a5d0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.753+0000 7f4a0cce6640 1 -- 172.21.15.100:0/501470826 shutdown_connections 2024-09-15T21:47:11.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.753+0000 7f4a0cce6640 1 --2- 172.21.15.100:0/501470826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a0806d0e0 0x7f4a080cc720 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.753+0000 7f4a0cce6640 1 --2- 172.21.15.100:0/501470826 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4a080cb7d0 0x7f4a080cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.753+0000 7f4a0cce6640 1 -- 172.21.15.100:0/501470826 >> 172.21.15.100:0/501470826 conn(0x7f4a080680d0 msgr2=0x7f4a0806d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:11.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.753+0000 7f4a0cce6640 1 -- 172.21.15.100:0/501470826 shutdown_connections 2024-09-15T21:47:11.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.753+0000 7f4a0cce6640 1 -- 172.21.15.100:0/501470826 wait complete. 2024-09-15T21:47:11.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.755+0000 7f4a0cce6640 1 Processor -- start 2024-09-15T21:47:11.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.755+0000 7f4a0cce6640 1 -- start start 2024-09-15T21:47:11.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.755+0000 7f4a0cce6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4a0806d0e0 0x7f4a08150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.755+0000 7f4a0cce6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a080cb7d0 0x7f4a0815c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.755+0000 7f4a0cce6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a08155440 0x7f4a080b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.755+0000 7f4a0cce6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0815f2e0 con 0x7f4a080cb7d0 2024-09-15T21:47:11.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.755+0000 7f4a0cce6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0815f000 con 0x7f4a08155440 2024-09-15T21:47:11.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.755+0000 7f4a0cce6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4a0815b070 con 0x7f4a0806d0e0 2024-09-15T21:47:11.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.755+0000 7f4a05d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a080cb7d0 0x7f4a0815c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.755+0000 7f4a05d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a080cb7d0 0x7f4a0815c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49278/0 (socket says 172.21.15.100:49278) 2024-09-15T21:47:11.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.755+0000 7f4a05d9b640 1 -- 172.21.15.100:0/2733988999 learned_addr learned my addr 172.21.15.100:0/2733988999 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:11.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.756+0000 7f4a0659c640 1 --2- 172.21.15.100:0/2733988999 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4a0806d0e0 0x7f4a08150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.756+0000 7f4a06d9d640 1 --2- 172.21.15.100:0/2733988999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a08155440 0x7f4a080b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.756+0000 7f4a05d9b640 1 -- 172.21.15.100:0/2733988999 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4a0806d0e0 msgr2=0x7f4a08150770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.756+0000 7f4a05d9b640 1 --2- 172.21.15.100:0/2733988999 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4a0806d0e0 0x7f4a08150770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.756+0000 7f4a05d9b640 1 -- 172.21.15.100:0/2733988999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a08155440 msgr2=0x7f4a080b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.756+0000 7f4a05d9b640 1 --2- 172.21.15.100:0/2733988999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a08155440 0x7f4a080b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.756+0000 7f4a05d9b640 1 -- 172.21.15.100:0/2733988999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a080cf5d0 con 0x7f4a080cb7d0 2024-09-15T21:47:11.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.756+0000 7f4a05d9b640 1 --2- 172.21.15.100:0/2733988999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a080cb7d0 0x7f4a0815c590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00e9e0 tx=0x7f49fc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.756+0000 7f49f6ffd640 1 -- 172.21.15.100:0/2733988999 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc017070 con 0x7f4a080cb7d0 2024-09-15T21:47:11.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.756+0000 7f49f6ffd640 1 -- 172.21.15.100:0/2733988999 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49fc00c9d0 con 0x7f4a080cb7d0 2024-09-15T21:47:11.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.756+0000 7f49f6ffd640 1 -- 172.21.15.100:0/2733988999 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc00cdc0 con 0x7f4a080cb7d0 2024-09-15T21:47:11.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.756+0000 7f4a0cce6640 1 -- 172.21.15.100:0/2733988999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a080b4000 con 0x7f4a080cb7d0 2024-09-15T21:47:11.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.756+0000 7f4a0cce6640 1 -- 172.21.15.100:0/2733988999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a080b48a0 con 0x7f4a080cb7d0 2024-09-15T21:47:11.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.757+0000 7f4a0cce6640 1 -- 172.21.15.100:0/2733988999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a080b3940 con 0x7f4a080cb7d0 2024-09-15T21:47:11.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.758+0000 7f49f6ffd640 1 -- 172.21.15.100:0/2733988999 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f49fc01a070 con 0x7f4a080cb7d0 2024-09-15T21:47:11.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.759+0000 7f49f6ffd640 1 --2- 172.21.15.100:0/2733988999 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f49e806c2b0 0x7f49e806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.759+0000 7f49f6ffd640 1 -- 172.21.15.100:0/2733988999 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5854+0+0 (secure 0 0 0) 0x7f49fc08b7b0 con 0x7f4a080cb7d0 2024-09-15T21:47:11.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.759+0000 7f4a0659c640 1 --2- 172.21.15.100:0/2733988999 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f49e806c2b0 0x7f49e806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.759+0000 7f4a0659c640 1 --2- 172.21.15.100:0/2733988999 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f49e806c2b0 0x7f49e806e770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f49f8005880 tx=0x7f49f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.763+0000 7f90f3de4640 1 Processor -- start 2024-09-15T21:47:11.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.763+0000 7f49f6ffd640 1 -- 172.21.15.100:0/2733988999 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f49fc059f20 con 0x7f4a080cb7d0 2024-09-15T21:47:11.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.763+0000 7f90f3de4640 1 -- start start 2024-09-15T21:47:11.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.764+0000 7f90f3de4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90ec155290 0x7f90ec15e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.764+0000 7f90f3de4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90ec15edb0 con 0x7f90ec15bb60 2024-09-15T21:47:11.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.764+0000 7f90f3de4640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90ec15f090 con 0x7f90ec155670 2024-09-15T21:47:11.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.764+0000 7f90f3de4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f90ec15f370 con 0x7f90ec155290 2024-09-15T21:47:11.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.764+0000 7f90f2381640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f90ec15bb60 0x7f90ec06d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:34054/0 (socket says 172.21.15.100:34054) 2024-09-15T21:47:11.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.764+0000 7f90f2381640 1 -- 172.21.15.100:0/296868331 learned_addr learned my addr 172.21.15.100:0/296868331 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:11.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.764+0000 7f90f1b80640 1 --2- 172.21.15.100:0/296868331 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90ec155290 0x7f90ec15e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.765+0000 7f90f0b7e640 1 -- 172.21.15.100:0/296868331 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 784068324 0 0) 0x7f90ec15edb0 con 0x7f90ec15bb60 2024-09-15T21:47:11.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.765+0000 7f90f0b7e640 1 -- 172.21.15.100:0/296868331 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90d8003520 con 0x7f90ec15bb60 2024-09-15T21:47:11.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.765+0000 7f90f0b7e640 1 -- 172.21.15.100:0/296868331 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2615971873 0 0) 0x7f90ec15f090 con 0x7f90ec155670 2024-09-15T21:47:11.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.765+0000 7f90f0b7e640 1 -- 172.21.15.100:0/296868331 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90ec15edb0 con 0x7f90ec155670 2024-09-15T21:47:11.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.765+0000 7f90f1b80640 1 -- 172.21.15.100:0/296868331 >> v1:172.21.15.122:6789/0 conn(0x7f90ec155670 legacy=0x7f90ec0c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.765+0000 7f90f1b80640 1 -- 172.21.15.100:0/296868331 >> v1:172.21.15.100:6789/0 conn(0x7f90ec15bb60 legacy=0x7f90ec06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.765+0000 7f90f1b80640 1 -- 172.21.15.100:0/296868331 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ec0c5db0 con 0x7f90ec155290 2024-09-15T21:47:11.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.766+0000 7f90f1b80640 1 --2- 172.21.15.100:0/296868331 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90ec155290 0x7f90ec15e690 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f90e80097d0 tx=0x7f90e802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b706926f44db812 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.766+0000 7f90f0b7e640 1 -- 172.21.15.100:0/296868331 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8037070 con 0x7f90ec155290 2024-09-15T21:47:11.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.766+0000 7f90f0b7e640 1 -- 172.21.15.100:0/296868331 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e8032440 con 0x7f90ec155290 2024-09-15T21:47:11.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.766+0000 7f90f0b7e640 1 -- 172.21.15.100:0/296868331 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e80327f0 con 0x7f90ec155290 2024-09-15T21:47:11.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.766+0000 7f90f3de4640 1 -- 172.21.15.100:0/296868331 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90ec155290 msgr2=0x7f90ec15e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.766+0000 7f90f3de4640 1 --2- 172.21.15.100:0/296868331 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90ec155290 0x7f90ec15e690 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f90e80097d0 tx=0x7f90e802a210 comp rx=0 tx=0).stop 2024-09-15T21:47:11.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.767+0000 7f90f3de4640 1 -- 172.21.15.100:0/296868331 shutdown_connections 2024-09-15T21:47:11.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.767+0000 7f90f3de4640 1 --2- 172.21.15.100:0/296868331 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90ec155290 0x7f90ec15e690 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.767+0000 7f90f3de4640 1 -- 172.21.15.100:0/296868331 >> 172.21.15.100:0/296868331 conn(0x7f90ec073310 msgr2=0x7f90ec073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:11.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.767+0000 7f90f3de4640 1 -- 172.21.15.100:0/296868331 shutdown_connections 2024-09-15T21:47:11.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.767+0000 7f90f3de4640 1 -- 172.21.15.100:0/296868331 wait complete. 2024-09-15T21:47:11.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.768+0000 7f90f3de4640 1 Processor -- start 2024-09-15T21:47:11.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.768+0000 7f90f3de4640 1 -- start start 2024-09-15T21:47:11.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.768+0000 7f90f3de4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90ec155290 0x7f90ec151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.768+0000 7f90f3de4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90ec155670 0x7f90ec1f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.768+0000 7f90f3de4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90ec15bb60 0x7f90ec1f11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.768+0000 7f90f3de4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec15bf90 con 0x7f90ec15bb60 2024-09-15T21:47:11.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.768+0000 7f90f3de4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec0b39e0 con 0x7f90ec155290 2024-09-15T21:47:11.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.768+0000 7f90f3de4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f90ec15c590 con 0x7f90ec155670 2024-09-15T21:47:11.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.769+0000 7f90f2381640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90ec15bb60 0x7f90ec1f11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.769+0000 7f90f2381640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90ec15bb60 0x7f90ec1f11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49288/0 (socket says 172.21.15.100:49288) 2024-09-15T21:47:11.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.769+0000 7f90f2381640 1 -- 172.21.15.100:0/2922993342 learned_addr learned my addr 172.21.15.100:0/2922993342 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:11.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.769+0000 7f90f1b80640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90ec155290 0x7f90ec151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.769+0000 7f90f137f640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90ec155670 0x7f90ec1f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.769+0000 7f90f2381640 1 -- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90ec155670 msgr2=0x7f90ec1f0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.769+0000 7f90f2381640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f90ec155670 0x7f90ec1f0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.769+0000 7f90f2381640 1 -- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90ec155290 msgr2=0x7f90ec151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.769+0000 7f90f2381640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f90ec155290 0x7f90ec151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.769+0000 7f90f2381640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ec1f4d60 con 0x7f90ec15bb60 2024-09-15T21:47:11.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.769+0000 7f90f2381640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90ec15bb60 0x7f90ec1f11d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f90e400cb40 tx=0x7f90e400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.769+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4017070 con 0x7f90ec15bb60 2024-09-15T21:47:11.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.769+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90e400acc0 con 0x7f90ec15bb60 2024-09-15T21:47:11.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.769+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ec14e840 con 0x7f90ec15bb60 2024-09-15T21:47:11.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.769+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e4012640 con 0x7f90ec15bb60 2024-09-15T21:47:11.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.769+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ec1f4a80 con 0x7f90ec15bb60 2024-09-15T21:47:11.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.770+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f90ec1f4870 con 0x7f90ec15bb60 2024-09-15T21:47:11.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.771+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f90e401a070 con 0x7f90ec15bb60 2024-09-15T21:47:11.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.772+0000 7f90e27fc640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f90d006c2e0 0x7f90d006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.772+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5854+0+0 (secure 0 0 0) 0x7f90e408b1c0 con 0x7f90ec15bb60 2024-09-15T21:47:11.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.772+0000 7f90f1b80640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f90d006c2e0 0x7f90d006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.772+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v27) ==== 54+0+16 (secure 0 0 0) 0x7f90e408d840 con 0x7f90ec15bb60 2024-09-15T21:47:11.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.772+0000 7f90f3de4640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f90ec0cea80 0x7f90ec14da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.772+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f90ec153050 con 0x7f90ec0cea80 2024-09-15T21:47:11.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.773+0000 7f90f1b80640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f90d006c2e0 0x7f90d006e7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f90e8034000 tx=0x7f90e802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.773+0000 7f90f137f640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f90ec0cea80 0x7f90ec14da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.773+0000 7f90f137f640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f90ec0cea80 0x7f90ec14da30 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.774+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f90ec153050 con 0x7f90ec0cea80 2024-09-15T21:47:11.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.788+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f90ec15e690 con 0x7f90ec0cea80 2024-09-15T21:47:11.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.789+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f90ec15e690 con 0x7f90ec0cea80 2024-09-15T21:47:11.791 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:11.792 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:11.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.789+0000 7f90f3de4640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f90ec15aec0 0x7f90ec159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.789+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f90ec14e480 con 0x7f90ec15aec0 2024-09-15T21:47:11.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.790+0000 7f90f1b80640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f90ec15aec0 0x7f90ec159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.790+0000 7f90f1b80640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f90ec15aec0 0x7f90ec159d90 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.791+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f90ec14e480 con 0x7f90ec15aec0 2024-09-15T21:47:11.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.805+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f90ec0cc660 con 0x7f90ec15aec0 2024-09-15T21:47:11.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.807+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f90ec0cc660 con 0x7f90ec15aec0 2024-09-15T21:47:11.809 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:11.809 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:11.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.807+0000 7f90f3de4640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f90ec15a7e0 0x7f90ec062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.807+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f90ec15abc0 con 0x7f90ec15a7e0 2024-09-15T21:47:11.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.807+0000 7f90f137f640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f90ec15a7e0 0x7f90ec062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.807+0000 7f90f137f640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f90ec15a7e0 0x7f90ec062630 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.809+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f90ec15abc0 con 0x7f90ec15a7e0 2024-09-15T21:47:11.822 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:47:11.820+0000 7f74313af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37847) UID: 0 2024-09-15T21:47:11.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.822+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f90ec062dc0 con 0x7f90ec15a7e0 2024-09-15T21:47:11.825 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:11.826 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:11.826 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:11.826 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:11.826 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:11.827 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.825+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f90ec062dc0 con 0x7f90ec15a7e0 2024-09-15T21:47:11.827 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.825+0000 7f90f3de4640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f90ec063390 0x7f90ec0b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.825+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f90ec0b48c0 con 0x7f90ec063390 2024-09-15T21:47:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.825+0000 7f90f2381640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f90ec063390 0x7f90ec0b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.826+0000 7f90f2381640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f90ec063390 0x7f90ec0b4380 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.826+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f90ec0b48c0 con 0x7f90ec063390 2024-09-15T21:47:11.835 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:47:11.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.840+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f90ec0b4dd0 con 0x7f90ec063390 2024-09-15T21:47:11.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.842+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f90ec0b4dd0 con 0x7f90ec063390 2024-09-15T21:47:11.845 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:11.845 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:11.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.842+0000 7f90f3de4640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f90ec02cb50 0x7f90ec02cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.842+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f90ec02d4f0 con 0x7f90ec02cb50 2024-09-15T21:47:11.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.843+0000 7f90f1b80640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f90ec02cb50 0x7f90ec02cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.843+0000 7f90f1b80640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f90ec02cb50 0x7f90ec02cfb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.844+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f90ec02d4f0 con 0x7f90ec02cb50 2024-09-15T21:47:11.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.859+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f90ec02d4f0 con 0x7f90ec02cb50 2024-09-15T21:47:11.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.861+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f90ec02d4f0 con 0x7f90ec02cb50 2024-09-15T21:47:11.864 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:11.864 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:11.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.862+0000 7f90f3de4640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f90ec2051e0 0x7f90ec2075d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.862+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7f90ec20a3a0 con 0x7f90ec2051e0 2024-09-15T21:47:11.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.862+0000 7f90f137f640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f90ec2051e0 0x7f90ec2075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.863+0000 7f90f137f640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f90ec2051e0 0x7f90ec2075d0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.863+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== osd.5 v2:172.21.15.122:6824/2400668532 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f90ec20a3a0 con 0x7f90ec2051e0 2024-09-15T21:47:11.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.892+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f90ec207b10 con 0x7f90ec2051e0 2024-09-15T21:47:11.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.894+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== osd.5 v2:172.21.15.122:6824/2400668532 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f90ec207b10 con 0x7f90ec2051e0 2024-09-15T21:47:11.896 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-15T21:47:11.897 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:47:11.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.894+0000 7f90f3de4640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f90ec208190 0x7f90ec20a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.894+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7f90ec20d350 con 0x7f90ec208190 2024-09-15T21:47:11.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.895+0000 7f90f2381640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f90ec208190 0x7f90ec20a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.895+0000 7f90f2381640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f90ec208190 0x7f90ec20a580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.896+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f90ec20d350 con 0x7f90ec208190 2024-09-15T21:47:11.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.909+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f90ec20aac0 con 0x7f90ec208190 2024-09-15T21:47:11.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.911+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f90ec20aac0 con 0x7f90ec208190 2024-09-15T21:47:11.913 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:11.914 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:11.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.912+0000 7f90f3de4640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f90ec20b140 0x7f90ec20d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:11.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.912+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7f90ec2103d0 con 0x7f90ec20b140 2024-09-15T21:47:11.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.912+0000 7f90f1b80640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f90ec20b140 0x7f90ec20d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:11.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.912+0000 7f90f1b80640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f90ec20b140 0x7f90ec20d530 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:11.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.913+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f90ec2103d0 con 0x7f90ec20b140 2024-09-15T21:47:11.922 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:11.920+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:11.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.927+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f90ec000950 con 0x7f90ec20b140 2024-09-15T21:47:11.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90e27fc640 1 -- 172.21.15.100:0/2922993342 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f90ec000950 con 0x7f90ec20b140 2024-09-15T21:47:11.931 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:11.932 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:11.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f90ec0cea80 msgr2=0x7f90ec14da30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f90ec0cea80 0x7f90ec14da30 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f90ec15aec0 msgr2=0x7f90ec159d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f90ec15aec0 0x7f90ec159d90 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f90ec15a7e0 msgr2=0x7f90ec062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f90ec15a7e0 0x7f90ec062630 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f90ec063390 msgr2=0x7f90ec0b4380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f90ec063390 0x7f90ec0b4380 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f2381640 1 -- 172.21.15.100:0/2922993342 reap_dead start 2024-09-15T21:47:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f90ec02cb50 msgr2=0x7f90ec02cfb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f90ec02cb50 0x7f90ec02cfb0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f2381640 1 -- 172.21.15.100:0/2922993342 reap_dead start 2024-09-15T21:47:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f90ec2051e0 msgr2=0x7f90ec2075d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7f90ec2051e0 0x7f90ec2075d0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f90ec208190 msgr2=0x7f90ec20a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f90ec208190 0x7f90ec20a580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f90ec20b140 msgr2=0x7f90ec20d530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f90ec20b140 0x7f90ec20d530 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f90d006c2e0 msgr2=0x7f90d006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f90d006c2e0 0x7f90d006e7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f90e8034000 tx=0x7f90e802e000 comp rx=0 tx=0).stop 2024-09-15T21:47:11.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90ec15bb60 msgr2=0x7f90ec1f11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:11.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f3de4640 1 --2- 172.21.15.100:0/2922993342 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f90ec15bb60 0x7f90ec1f11d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f90e400cb40 tx=0x7f90e400a060 comp rx=0 tx=0).stop 2024-09-15T21:47:11.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.929+0000 7f90f2381640 1 -- 172.21.15.100:0/2922993342 reap_dead start 2024-09-15T21:47:11.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.930+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 shutdown_connections 2024-09-15T21:47:11.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.930+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 >> 172.21.15.100:0/2922993342 conn(0x7f90ec073310 msgr2=0x7f90ec0cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:11.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.930+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 shutdown_connections 2024-09-15T21:47:11.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.930+0000 7f90f3de4640 1 -- 172.21.15.100:0/2922993342 wait complete. 2024-09-15T21:47:11.954 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:11.955 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:11.963 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:47:11.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:11.969+0000 7f4a0cce6640 1 -- 172.21.15.100:0/2733988999 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f4a080caed0 con 0x7f4a080cb7d0 2024-09-15T21:47:12.023 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:12.021+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:12.077 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:12.078 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:12.087 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:47:12.123 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:12.122+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:12.190 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:12.190 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:12.198 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:47:12.243 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:12.241+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:12.301 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:12.301 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:12.309 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:47:12.351 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:12.349+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:12.427 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:12.425+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:12.453 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:12.453 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:12.462 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:47:12.528 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:12.526+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:12.565 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:12.566 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:12.574 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:47:12.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:12.648+0000 7f49f6ffd640 1 -- 172.21.15.100:0/2733988999 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v28) ==== 101+0+0 (secure 0 0 0) 0x7f49fc059820 con 0x7f4a080cb7d0 2024-09-15T21:47:12.650 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is unset 2024-09-15T21:47:12.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:12.650+0000 7f4a0cce6640 1 -- 172.21.15.100:0/2733988999 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f49e806c2b0 msgr2=0x7f49e806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:12.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:12.650+0000 7f4a0cce6640 1 --2- 172.21.15.100:0/2733988999 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f49e806c2b0 0x7f49e806e770 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f49f8005880 tx=0x7f49f8009040 comp rx=0 tx=0).stop 2024-09-15T21:47:12.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:12.650+0000 7f4a0cce6640 1 -- 172.21.15.100:0/2733988999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a080cb7d0 msgr2=0x7f4a0815c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:12.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:12.650+0000 7f4a0cce6640 1 --2- 172.21.15.100:0/2733988999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a080cb7d0 0x7f4a0815c590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00e9e0 tx=0x7f49fc00eea0 comp rx=0 tx=0).stop 2024-09-15T21:47:12.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:12.650+0000 7f4a0cce6640 1 -- 172.21.15.100:0/2733988999 shutdown_connections 2024-09-15T21:47:12.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:12.650+0000 7f4a0cce6640 1 --2- 172.21.15.100:0/2733988999 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f49e806c2b0 0x7f49e806e770 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:12.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:12.651+0000 7f4a0cce6640 1 --2- 172.21.15.100:0/2733988999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a08155440 0x7f4a080b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:12.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:12.651+0000 7f4a0cce6640 1 --2- 172.21.15.100:0/2733988999 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4a080cb7d0 0x7f4a0815c590 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:12.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:12.651+0000 7f4a0cce6640 1 --2- 172.21.15.100:0/2733988999 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4a0806d0e0 0x7f4a08150770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:12.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:12.651+0000 7f4a0cce6640 1 -- 172.21.15.100:0/2733988999 >> 172.21.15.100:0/2733988999 conn(0x7f4a080680d0 msgr2=0x7f4a080cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:12.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:12.651+0000 7f4a0cce6640 1 -- 172.21.15.100:0/2733988999 shutdown_connections 2024-09-15T21:47:12.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:12.651+0000 7f4a0cce6640 1 -- 172.21.15.100:0/2733988999 wait complete. 2024-09-15T21:47:12.658 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:12.657+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:12.720 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:12.720 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:12.729 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:47:12.730 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:12.728+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:12.829 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:12.828+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:12.873 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:12.873 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:12.881 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:47:12.930 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:12.929+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:12.983 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:12.984 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:12.992 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:47:13.031 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:13.030+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:13.094 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:13.094 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:13.103 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:47:13.131 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:13.130+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:13.205 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:13.206 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:13.214 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:47:13.232 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:13.231+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:13.319 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:13.319 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:13.327 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:47:13.371 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:47:13.370+0000 7f74313af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37847) UID: 0 2024-09-15T21:47:13.431 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:13.431 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:13.434 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:47:13.432+0000 7f74313af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37847) UID: 0 2024-09-15T21:47:13.439 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:47:13.535 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:13.533+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:13.556 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:13.556 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:13.565 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:47:13.636 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:13.634+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:13.709 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:13.710 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:13.718 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:47:13.761 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:13.760+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:13.825 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:13.825 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:13.832 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:47:13.874 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:13.872+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:13.939 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:13.938+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:13.947 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:47:13.978 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:13.978 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:13.987 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:47:14.040 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:14.039+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:14.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.043+0000 7fb0b7ef1640 1 Processor -- start 2024-09-15T21:47:14.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.043+0000 7fb0b7ef1640 1 -- start start 2024-09-15T21:47:14.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.044+0000 7fb0b7ef1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb0b0073420 0x7fb0b015be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.044+0000 7fb0b7ef1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0b00cab00 con 0x7fb0b0071420 2024-09-15T21:47:14.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.044+0000 7fb0b7ef1640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0b00718d0 con 0x7fb0b0073800 2024-09-15T21:47:14.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.044+0000 7fb0b7ef1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fb0b015c360 con 0x7fb0b0073420 2024-09-15T21:47:14.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.044+0000 7fb0b5c8d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb0b0073420 0x7fb0b015be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.044+0000 7fb0b548c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb0b0071420 0x7fb0b00ca3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:34060/0 (socket says 172.21.15.100:34060) 2024-09-15T21:47:14.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.044+0000 7fb0b548c640 1 -- 172.21.15.100:0/358282143 learned_addr learned my addr 172.21.15.100:0/358282143 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:14.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.044+0000 7fb0b5c8d640 1 -- 172.21.15.100:0/358282143 >> v1:172.21.15.122:6789/0 conn(0x7fb0b0073800 legacy=0x7fb0b0155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:14.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.044+0000 7fb0b5c8d640 1 -- 172.21.15.100:0/358282143 >> v1:172.21.15.100:6789/0 conn(0x7fb0b0071420 legacy=0x7fb0b00ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:14.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.044+0000 7fb0b5c8d640 1 -- 172.21.15.100:0/358282143 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b0071bb0 con 0x7fb0b0073420 2024-09-15T21:47:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.045+0000 7fb0b5c8d640 1 --2- 172.21.15.100:0/358282143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b0073420 0x7fb0b015be20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0097a0 tx=0x7fb0ac02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d0e57eddd56e7d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.045+0000 7fb0b4c8b640 1 -- 172.21.15.100:0/358282143 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac037070 con 0x7fb0b0073420 2024-09-15T21:47:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.045+0000 7fb0b4c8b640 1 -- 172.21.15.100:0/358282143 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0ac032640 con 0x7fb0b0073420 2024-09-15T21:47:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.045+0000 7fb0b4c8b640 1 -- 172.21.15.100:0/358282143 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac032a10 con 0x7fb0b0073420 2024-09-15T21:47:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.046+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/358282143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b0073420 msgr2=0x7fb0b015be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.046+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/358282143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b0073420 0x7fb0b015be20 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0097a0 tx=0x7fb0ac02a2c0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.046+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/358282143 shutdown_connections 2024-09-15T21:47:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.046+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/358282143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b0073420 0x7fb0b015be20 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.046+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/358282143 >> 172.21.15.100:0/358282143 conn(0x7fb0b0067980 msgr2=0x7fb0b0067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:14.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.046+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/358282143 shutdown_connections 2024-09-15T21:47:14.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.046+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/358282143 wait complete. 2024-09-15T21:47:14.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.047+0000 7fb0b7ef1640 1 Processor -- start 2024-09-15T21:47:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.047+0000 7fb0b7ef1640 1 -- start start 2024-09-15T21:47:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.048+0000 7fb0b7ef1640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0b0071420 0x7fb0b015d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.048+0000 7fb0b7ef1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b0073420 0x7fb0b0153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.048+0000 7fb0b7ef1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b0073800 0x7fb0b0153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.048+0000 7fb0b7ef1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b015efb0 con 0x7fb0b0073800 2024-09-15T21:47:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.048+0000 7fb0b7ef1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b015ecd0 con 0x7fb0b0073420 2024-09-15T21:47:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.048+0000 7fb0b7ef1640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb0b015f290 con 0x7fb0b0071420 2024-09-15T21:47:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.048+0000 7fb0b648e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b0073800 0x7fb0b0153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.048+0000 7fb0b5c8d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0b0071420 0x7fb0b015d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.048+0000 7fb0b648e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b0073800 0x7fb0b0153b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49296/0 (socket says 172.21.15.100:49296) 2024-09-15T21:47:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.048+0000 7fb0b548c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b0073420 0x7fb0b0153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.048+0000 7fb0b648e640 1 -- 172.21.15.100:0/3932957398 learned_addr learned my addr 172.21.15.100:0/3932957398 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.048+0000 7fb0b648e640 1 -- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0b0071420 msgr2=0x7fb0b015d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.048+0000 7fb0b648e640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb0b0071420 0x7fb0b015d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.048+0000 7fb0b648e640 1 -- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b0073420 msgr2=0x7fb0b0153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.048+0000 7fb0b648e640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b0073420 0x7fb0b0153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.048+0000 7fb0b648e640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b01f4f80 con 0x7fb0b0073800 2024-09-15T21:47:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.048+0000 7fb0b548c640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0b0073420 0x7fb0b0153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.048+0000 7fb0b648e640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b0073800 0x7fb0b0153b60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800c6b0 tx=0x7fb0a800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.049+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8003b70 con 0x7fb0b0073800 2024-09-15T21:47:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.049+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0a8003d30 con 0x7fb0b0073800 2024-09-15T21:47:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.049+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b01f5220 con 0x7fb0b0073800 2024-09-15T21:47:14.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.049+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8005310 con 0x7fb0b0073800 2024-09-15T21:47:14.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.049+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b01e63d0 con 0x7fb0b0073800 2024-09-15T21:47:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.049+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fb0b015f7b0 con 0x7fb0b0073800 2024-09-15T21:47:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.050+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb0a80054b0 con 0x7fb0b0073800 2024-09-15T21:47:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.051+0000 7fb0a67fc640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb09406c2e0 0x7fb09406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.051+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5854+0+0 (secure 0 0 0) 0x7fb0a808b150 con 0x7fb0b0073800 2024-09-15T21:47:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.051+0000 7fb0b5c8d640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb09406c2e0 0x7fb09406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.051+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v28) ==== 54+0+16 (secure 0 0 0) 0x7fb0a808d7d0 con 0x7fb0b0073800 2024-09-15T21:47:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.051+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fb0b01e46d0 0x7fb0b01e4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.051+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb0b01598d0 con 0x7fb0b01e46d0 2024-09-15T21:47:14.054 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.051+0000 7fb0b5c8d640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb09406c2e0 0x7fb09406e7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac034000 tx=0x7fb0ac02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:14.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.051+0000 7fb0b548c640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fb0b01e46d0 0x7fb0b01e4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.052+0000 7fb0b548c640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fb0b01e46d0 0x7fb0b01e4bf0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:14.055 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.052+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0b01598d0 con 0x7fb0b01e46d0 2024-09-15T21:47:14.068 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.066+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb0b00ceeb0 con 0x7fb0b01e46d0 2024-09-15T21:47:14.070 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.068+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb0b00ceeb0 con 0x7fb0b01e46d0 2024-09-15T21:47:14.070 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:14.071 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:14.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.068+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fb0b01f1030 0x7fb0b01f1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.068+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fb0b00c96e0 con 0x7fb0b01f1030 2024-09-15T21:47:14.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.068+0000 7fb0b5c8d640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fb0b01f1030 0x7fb0b01f1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.069+0000 7fb0b5c8d640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fb0b01f1030 0x7fb0b01f1410 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:14.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.069+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0b00c96e0 con 0x7fb0b01f1030 2024-09-15T21:47:14.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.084+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb0b01e7430 con 0x7fb0b01f1030 2024-09-15T21:47:14.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.085+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb0b01e7430 con 0x7fb0b01f1030 2024-09-15T21:47:14.087 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:14.088 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:14.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.085+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fb0b0062630 0x7fb0b0062a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.085+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fb0b01e6880 con 0x7fb0b0062630 2024-09-15T21:47:14.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.086+0000 7fb0b548c640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fb0b0062630 0x7fb0b0062a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.086+0000 7fb0b548c640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fb0b0062630 0x7fb0b0062a10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:14.089 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.087+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0b01e6880 con 0x7fb0b0062630 2024-09-15T21:47:14.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.100+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb0b01e6ab0 con 0x7fb0b0062630 2024-09-15T21:47:14.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.102+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb0b01e6ab0 con 0x7fb0b0062630 2024-09-15T21:47:14.104 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:14.104 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:14.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.102+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fb0b006d110 0x7fb0b006d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.102+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fb0b01e6ab0 con 0x7fb0b006d110 2024-09-15T21:47:14.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.102+0000 7fb0b648e640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fb0b006d110 0x7fb0b006d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.102+0000 7fb0b648e640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fb0b006d110 0x7fb0b006d590 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:14.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.103+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0b01e6ab0 con 0x7fb0b006d110 2024-09-15T21:47:14.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.116+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb0b01e6ab0 con 0x7fb0b006d110 2024-09-15T21:47:14.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.118+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb0b01e6ab0 con 0x7fb0b006d110 2024-09-15T21:47:14.120 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:14.121 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:14.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.118+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fb0b002c640 0x7fb0b002ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.118+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fb0b006dfe0 con 0x7fb0b002c640 2024-09-15T21:47:14.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.119+0000 7fb0b5c8d640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fb0b002c640 0x7fb0b002ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.119+0000 7fb0b5c8d640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fb0b002c640 0x7fb0b002ca60 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:14.121 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.120+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0b006dfe0 con 0x7fb0b002c640 2024-09-15T21:47:14.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.133+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb0b006dfe0 con 0x7fb0b002c640 2024-09-15T21:47:14.135 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:14.135 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:14.135 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:14.135 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:14.135 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:14.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.135+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb0b006dfe0 con 0x7fb0b002c640 2024-09-15T21:47:14.137 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:14.137 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:14.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.135+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fb0b002d4b0 0x7fb0b02079b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.135+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fb0b002d8f0 con 0x7fb0b002d4b0 2024-09-15T21:47:14.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.135+0000 7fb0b548c640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fb0b002d4b0 0x7fb0b02079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.135+0000 7fb0b548c640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fb0b002d4b0 0x7fb0b02079b0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:14.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.136+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== osd.5 v2:172.21.15.122:6824/2400668532 1 ==== command_reply(tid 11: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0b002d8f0 con 0x7fb0b002d4b0 2024-09-15T21:47:14.141 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:47:14.139+0000 7f74313af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37847) UID: 0 2024-09-15T21:47:14.144 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:47:14.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.150+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] -- command(tid 12: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb0b002d8f0 con 0x7fb0b002d4b0 2024-09-15T21:47:14.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.152+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== osd.5 v2:172.21.15.122:6824/2400668532 2 ==== command_reply(tid 12: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb0b002d8f0 con 0x7fb0b002d4b0 2024-09-15T21:47:14.154 INFO:teuthology.orchestra.run.smithi100.stdout:osd.5: {} 2024-09-15T21:47:14.155 INFO:teuthology.orchestra.run.smithi100.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:47:14.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.152+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fb0b0208400 0x7fb0b020a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.152+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "get_command_descriptions"}) -- 0x7fb0b020d5c0 con 0x7fb0b0208400 2024-09-15T21:47:14.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.153+0000 7fb0b648e640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fb0b0208400 0x7fb0b020a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.153+0000 7fb0b648e640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fb0b0208400 0x7fb0b020a7f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:14.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.154+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 13: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0b020d5c0 con 0x7fb0b0208400 2024-09-15T21:47:14.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.167+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 14: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb0b020ad30 con 0x7fb0b0208400 2024-09-15T21:47:14.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.169+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 14: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb0b020ad30 con 0x7fb0b0208400 2024-09-15T21:47:14.171 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:14.171 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:14.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.169+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fb0b020b3b0 0x7fb0b020d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.169+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "get_command_descriptions"}) -- 0x7fb0b0210570 con 0x7fb0b020b3b0 2024-09-15T21:47:14.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.169+0000 7fb0b5c8d640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fb0b020b3b0 0x7fb0b020d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.170+0000 7fb0b5c8d640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fb0b020b3b0 0x7fb0b020d7a0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:14.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.170+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 15: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb0b0210570 con 0x7fb0b020b3b0 2024-09-15T21:47:14.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.184+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 16: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb0b020dd60 con 0x7fb0b020b3b0 2024-09-15T21:47:14.187 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0a67fc640 1 -- 172.21.15.100:0/3932957398 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 16: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb0b020dd60 con 0x7fb0b020b3b0 2024-09-15T21:47:14.188 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:14.189 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:14.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fb0b01e46d0 msgr2=0x7fb0b01e4bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:14.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fb0b01e46d0 0x7fb0b01e4bf0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.189 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fb0b01f1030 msgr2=0x7fb0b01f1410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:14.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fb0b01f1030 0x7fb0b01f1410 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fb0b0062630 msgr2=0x7fb0b0062a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:14.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fb0b0062630 0x7fb0b0062a10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fb0b006d110 msgr2=0x7fb0b006d590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:14.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fb0b006d110 0x7fb0b006d590 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fb0b002c640 msgr2=0x7fb0b002ca60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:14.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fb0b002c640 0x7fb0b002ca60 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fb0b002d4b0 msgr2=0x7fb0b02079b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:14.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fb0b002d4b0 0x7fb0b02079b0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b648e640 1 -- 172.21.15.100:0/3932957398 reap_dead start 2024-09-15T21:47:14.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fb0b0208400 msgr2=0x7fb0b020a7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:14.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fb0b0208400 0x7fb0b020a7f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b648e640 1 -- 172.21.15.100:0/3932957398 reap_dead start 2024-09-15T21:47:14.190 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b648e640 1 -- 172.21.15.100:0/3932957398 reap_dead start 2024-09-15T21:47:14.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b648e640 1 -- 172.21.15.100:0/3932957398 reap_dead start 2024-09-15T21:47:14.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fb0b020b3b0 msgr2=0x7fb0b020d7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:14.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fb0b020b3b0 0x7fb0b020d7a0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb09406c2e0 msgr2=0x7fb09406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:14.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb09406c2e0 0x7fb09406e7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac034000 tx=0x7fb0ac02e000 comp rx=0 tx=0).stop 2024-09-15T21:47:14.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b0073800 msgr2=0x7fb0b0153b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:14.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.186+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b0073800 0x7fb0b0153b60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb0a800c6b0 tx=0x7fb0a800cb70 comp rx=0 tx=0).stop 2024-09-15T21:47:14.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.187+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 shutdown_connections 2024-09-15T21:47:14.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.187+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fb0b0208400 0x7fb0b020a7f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.187+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fb0b020b3b0 0x7fb0b020d7a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.187+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb09406c2e0 0x7fb09406e7a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.187+0000 7fb0b7ef1640 1 --2- 172.21.15.100:0/3932957398 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb0b0073800 0x7fb0b0153b60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.187+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 >> 172.21.15.100:0/3932957398 conn(0x7fb0b0067980 msgr2=0x7fb0b00cacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:14.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.187+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 shutdown_connections 2024-09-15T21:47:14.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.187+0000 7fb0b7ef1640 1 -- 172.21.15.100:0/3932957398 wait complete. 2024-09-15T21:47:14.242 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:14.241+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:14.258 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:14.258 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:14.258 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:14.258 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:14.258 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.258 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:14.258 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:14.258 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 45.842173406000001, 2024-09-15T21:47:14.258 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006101291, 2024-09-15T21:47:14.258 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.258 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.259 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.259 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.259 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.259 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.259 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:14.259 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.259 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.259 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.259 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.259 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:14.259 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.259 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.259 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.259 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.260 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413116+0000", 2024-09-15T21:47:14.260 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9700000000000003e-07 2024-09-15T21:47:14.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.260 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.260 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.260 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413118+0000", 2024-09-15T21:47:14.260 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2910000000000002e-06 2024-09-15T21:47:14.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.260 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.260 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.260 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413220+0000", 2024-09-15T21:47:14.260 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102056 2024-09-15T21:47:14.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.261 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.261 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413225+0000", 2024-09-15T21:47:14.261 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0490000000000003e-06 2024-09-15T21:47:14.261 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.261 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.261 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414808+0000", 2024-09-15T21:47:14.261 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015826900000000001 2024-09-15T21:47:14.261 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.261 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.261 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414907+0000", 2024-09-15T21:47:14.262 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9600999999999997e-05 2024-09-15T21:47:14.262 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.262 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.262 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:14.262 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415054+0000", 2024-09-15T21:47:14.262 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000146532 2024-09-15T21:47:14.262 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.262 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.262 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.262 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418056+0000", 2024-09-15T21:47:14.262 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030020519999999998 2024-09-15T21:47:14.262 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.262 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.262 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.262 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419128+0000", 2024-09-15T21:47:14.263 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001071671 2024-09-15T21:47:14.263 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.263 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.263 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.263 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419184+0000", 2024-09-15T21:47:14.263 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6125000000000002e-05 2024-09-15T21:47:14.263 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.263 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.263 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.263 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419217+0000", 2024-09-15T21:47:14.263 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2926999999999998e-05 2024-09-15T21:47:14.263 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.263 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.263 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.264 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.264 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.264 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:14.264 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:14.264 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 45.84216988, 2024-09-15T21:47:14.264 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059975130000000003, 2024-09-15T21:47:14.264 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.264 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.264 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.264 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.264 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.264 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.264 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:14.264 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.264 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.265 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.265 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:14.265 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.265 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.265 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:14.265 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6199999999999999e-07 2024-09-15T21:47:14.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.265 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.265 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413121+0000", 2024-09-15T21:47:14.266 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2000000000000001e-06 2024-09-15T21:47:14.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.266 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.266 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.266 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413292+0000", 2024-09-15T21:47:14.266 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017034400000000001 2024-09-15T21:47:14.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.266 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.266 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.266 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413300+0000", 2024-09-15T21:47:14.266 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8539999999999997e-06 2024-09-15T21:47:14.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.266 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.266 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414299+0000", 2024-09-15T21:47:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0009992829999999999 2024-09-15T21:47:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414390+0000", 2024-09-15T21:47:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1237999999999996e-05 2024-09-15T21:47:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414490+0000", 2024-09-15T21:47:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9438999999999994e-05 2024-09-15T21:47:14.267 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417393+0000", 2024-09-15T21:47:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002903306 2024-09-15T21:47:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419023+0000", 2024-09-15T21:47:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001630292 2024-09-15T21:47:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419067+0000", 2024-09-15T21:47:14.268 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4017999999999998e-05 2024-09-15T21:47:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419116+0000", 2024-09-15T21:47:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9277000000000003e-05 2024-09-15T21:47:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 45.839471682999999, 2024-09-15T21:47:14.269 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060879280000000003, 2024-09-15T21:47:14.270 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.270 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.270 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.270 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.270 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.270 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.270 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:14.270 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.270 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.270 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.270 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:14.270 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.270 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.271 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.271 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415818+0000", 2024-09-15T21:47:14.271 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:47:14.271 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.271 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.271 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.271 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415854+0000", 2024-09-15T21:47:14.271 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6706999999999999e-05 2024-09-15T21:47:14.271 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.271 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.271 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.271 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415860+0000", 2024-09-15T21:47:14.271 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0789999999999997e-06 2024-09-15T21:47:14.271 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.271 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.272 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.272 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415865+0000", 2024-09-15T21:47:14.272 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4580000000000003e-06 2024-09-15T21:47:14.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.272 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.272 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.273 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416092+0000", 2024-09-15T21:47:14.273 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00022739400000000001 2024-09-15T21:47:14.273 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.273 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.273 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.273 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416108+0000", 2024-09-15T21:47:14.273 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.613e-05 2024-09-15T21:47:14.273 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.273 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.273 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.273 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421866+0000", 2024-09-15T21:47:14.273 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005757585 2024-09-15T21:47:14.273 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.273 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.274 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.274 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421905+0000", 2024-09-15T21:47:14.274 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9214000000000001e-05 2024-09-15T21:47:14.274 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.274 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.274 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.274 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.274 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-15T21:47:14.274 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:14.274 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 45.839411523000003, 2024-09-15T21:47:14.274 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060637570000000003, 2024-09-15T21:47:14.274 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.274 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.275 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.275 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.275 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.275 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.275 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:14.275 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.275 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.275 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.275 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.275 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:14.275 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.275 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.275 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.276 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.276 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415878+0000", 2024-09-15T21:47:14.276 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.34e-07 2024-09-15T21:47:14.276 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.276 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.276 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.276 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415881+0000", 2024-09-15T21:47:14.276 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.354e-06 2024-09-15T21:47:14.276 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.276 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.276 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.276 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415887+0000", 2024-09-15T21:47:14.276 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:47:14.277 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.277 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.277 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.277 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415891+0000", 2024-09-15T21:47:14.277 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2690000000000001e-06 2024-09-15T21:47:14.277 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.277 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.277 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.277 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416267+0000", 2024-09-15T21:47:14.277 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00037614799999999998 2024-09-15T21:47:14.277 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.277 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.277 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.277 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416288+0000", 2024-09-15T21:47:14.277 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0604000000000001e-05 2024-09-15T21:47:14.278 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.278 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.278 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.278 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421908+0000", 2024-09-15T21:47:14.278 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056203390000000002 2024-09-15T21:47:14.278 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.278 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.278 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.278 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421941+0000", 2024-09-15T21:47:14.278 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3079e-05 2024-09-15T21:47:14.278 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.278 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.278 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.279 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.279 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.279 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:14.279 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:14.279 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 45.839393096000002, 2024-09-15T21:47:14.279 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006080284, 2024-09-15T21:47:14.279 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.279 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.279 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.279 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.279 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.279 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.279 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:14.279 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.279 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.280 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.280 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.280 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:14.280 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.280 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.280 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.280 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.280 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:14.280 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6899999999999999e-07 2024-09-15T21:47:14.280 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.280 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.280 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.280 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415898+0000", 2024-09-15T21:47:14.280 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.951e-06 2024-09-15T21:47:14.281 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.281 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.281 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.281 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415920+0000", 2024-09-15T21:47:14.281 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1651999999999999e-05 2024-09-15T21:47:14.281 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.281 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.281 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.281 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415925+0000", 2024-09-15T21:47:14.281 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.463e-06 2024-09-15T21:47:14.281 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.281 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.281 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.281 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416449+0000", 2024-09-15T21:47:14.282 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052384000000000003 2024-09-15T21:47:14.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.282 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.282 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416466+0000", 2024-09-15T21:47:14.282 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7108999999999998e-05 2024-09-15T21:47:14.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.282 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.282 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421943+0000", 2024-09-15T21:47:14.282 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054771569999999999 2024-09-15T21:47:14.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.282 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.282 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421976+0000", 2024-09-15T21:47:14.283 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2943e-05 2024-09-15T21:47:14.283 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.283 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.283 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.283 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.283 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.283 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-15T21:47:14.283 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:14.283 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 45.839353860000003, 2024-09-15T21:47:14.283 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060643340000000002, 2024-09-15T21:47:14.283 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.283 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.283 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.283 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.283 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.284 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.284 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:14.284 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.284 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.284 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.284 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:14.284 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.284 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.284 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.284 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:14.284 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4200000000000002e-07 2024-09-15T21:47:14.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.285 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.285 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.285 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415939+0000", 2024-09-15T21:47:14.285 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.439e-06 2024-09-15T21:47:14.285 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.285 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.285 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.285 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415945+0000", 2024-09-15T21:47:14.285 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7740000000000001e-06 2024-09-15T21:47:14.285 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.285 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.285 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.285 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415948+0000", 2024-09-15T21:47:14.285 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.737e-06 2024-09-15T21:47:14.285 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.286 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.286 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416605+0000", 2024-09-15T21:47:14.286 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00065721699999999998 2024-09-15T21:47:14.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.286 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.286 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416620+0000", 2024-09-15T21:47:14.286 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4564e-05 2024-09-15T21:47:14.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.286 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.286 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421977+0000", 2024-09-15T21:47:14.286 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053572369999999999 2024-09-15T21:47:14.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.287 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.287 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.287 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421999+0000", 2024-09-15T21:47:14.287 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2024000000000001e-05 2024-09-15T21:47:14.287 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.287 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.287 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.287 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.287 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-15T21:47:14.287 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:14.287 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 45.839153201000002, 2024-09-15T21:47:14.287 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058922669999999996, 2024-09-15T21:47:14.288 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.288 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.288 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.288 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.288 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.288 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:14.288 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.288 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.289 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:14.289 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.289 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.289 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.289 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:14.289 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6499999999999999e-07 2024-09-15T21:47:14.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.289 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.289 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.289 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416142+0000", 2024-09-15T21:47:14.289 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3929999999999999e-06 2024-09-15T21:47:14.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.290 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.290 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.290 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416151+0000", 2024-09-15T21:47:14.290 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5840000000000001e-06 2024-09-15T21:47:14.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.290 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.290 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.290 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416157+0000", 2024-09-15T21:47:14.290 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7590000000000003e-06 2024-09-15T21:47:14.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.290 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.290 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.290 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416767+0000", 2024-09-15T21:47:14.290 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00061034900000000005 2024-09-15T21:47:14.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.291 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.291 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.291 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416782+0000", 2024-09-15T21:47:14.291 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.483e-05 2024-09-15T21:47:14.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.291 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.291 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.291 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422002+0000", 2024-09-15T21:47:14.291 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052202639999999996 2024-09-15T21:47:14.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.291 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.291 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.291 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422028+0000", 2024-09-15T21:47:14.291 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5423e-05 2024-09-15T21:47:14.292 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.292 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.292 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.292 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.292 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.292 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-15T21:47:14.292 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:14.292 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 45.838746700000002, 2024-09-15T21:47:14.292 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055107769999999997, 2024-09-15T21:47:14.292 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.292 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.292 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.292 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.292 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.293 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.293 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:14.293 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.293 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.293 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.293 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:14.293 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.293 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.293 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.293 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416543+0000", 2024-09-15T21:47:14.293 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5400000000000001e-07 2024-09-15T21:47:14.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.293 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.294 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.294 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416548+0000", 2024-09-15T21:47:14.294 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8659999999999997e-06 2024-09-15T21:47:14.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.294 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.294 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.294 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416555+0000", 2024-09-15T21:47:14.294 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7190000000000001e-06 2024-09-15T21:47:14.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.294 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.294 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.294 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416560+0000", 2024-09-15T21:47:14.294 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3270000000000002e-06 2024-09-15T21:47:14.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416916+0000", 2024-09-15T21:47:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00035661200000000002 2024-09-15T21:47:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416932+0000", 2024-09-15T21:47:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5824e-05 2024-09-15T21:47:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422030+0000", 2024-09-15T21:47:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050975229999999996 2024-09-15T21:47:14.295 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422053+0000", 2024-09-15T21:47:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3252e-05 2024-09-15T21:47:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-15T21:47:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 45.838348201000002, 2024-09-15T21:47:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051422289999999999, 2024-09-15T21:47:14.296 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.297 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.297 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.297 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.297 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.297 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.297 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:14.297 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.297 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.297 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.297 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:14.297 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.297 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.298 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.298 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:14.298 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2699999999999999e-07 2024-09-15T21:47:14.298 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.298 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.298 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.298 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416946+0000", 2024-09-15T21:47:14.298 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1390000000000003e-06 2024-09-15T21:47:14.298 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.298 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.298 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.298 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416957+0000", 2024-09-15T21:47:14.298 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.044e-05 2024-09-15T21:47:14.298 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.298 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.299 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.299 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416963+0000", 2024-09-15T21:47:14.299 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5320000000000003e-06 2024-09-15T21:47:14.299 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.299 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.299 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.299 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417090+0000", 2024-09-15T21:47:14.299 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126549 2024-09-15T21:47:14.299 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.299 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.299 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.299 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417114+0000", 2024-09-15T21:47:14.299 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3736000000000001e-05 2024-09-15T21:47:14.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.300 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.300 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.300 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422055+0000", 2024-09-15T21:47:14.300 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049410410000000002 2024-09-15T21:47:14.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.300 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.300 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.300 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422083+0000", 2024-09-15T21:47:14.300 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8164999999999999e-05 2024-09-15T21:47:14.300 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.300 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.300 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.301 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.301 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:14.301 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 39.793856228999999, 2024-09-15T21:47:14.301 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007521186, 2024-09-15T21:47:14.301 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.301 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.301 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.301 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.301 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.301 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:14.301 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.302 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.302 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.302 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:14.302 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.302 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.302 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.302 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:14.302 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9999999999999998e-07 2024-09-15T21:47:14.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.302 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.302 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.302 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461443+0000", 2024-09-15T21:47:14.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0448000000000001e-05 2024-09-15T21:47:14.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462146+0000", 2024-09-15T21:47:14.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00070223499999999995 2024-09-15T21:47:14.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462154+0000", 2024-09-15T21:47:14.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9060000000000002e-06 2024-09-15T21:47:14.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462254+0000", 2024-09-15T21:47:14.304 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9897999999999998e-05 2024-09-15T21:47:14.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.304 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.304 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.304 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462330+0000", 2024-09-15T21:47:14.304 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6551000000000006e-05 2024-09-15T21:47:14.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.304 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.304 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:14.304 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462454+0000", 2024-09-15T21:47:14.304 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000124213 2024-09-15T21:47:14.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.304 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.304 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.305 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463681+0000", 2024-09-15T21:47:14.305 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012265729999999999 2024-09-15T21:47:14.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.305 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.305 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468817+0000", 2024-09-15T21:47:14.305 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051361430000000001 2024-09-15T21:47:14.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.305 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.305 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468892+0000", 2024-09-15T21:47:14.305 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5480000000000002e-05 2024-09-15T21:47:14.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.306 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.306 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468954+0000", 2024-09-15T21:47:14.306 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1339000000000003e-05 2024-09-15T21:47:14.306 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.306 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.306 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.306 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.306 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:14.306 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 39.793839613999999, 2024-09-15T21:47:14.306 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077296149999999996, 2024-09-15T21:47:14.306 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.306 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.306 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.307 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.307 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:14.307 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.307 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:14.307 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.307 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461450+0000", 2024-09-15T21:47:14.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0499999999999999e-07 2024-09-15T21:47:14.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.308 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.308 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461452+0000", 2024-09-15T21:47:14.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.007e-06 2024-09-15T21:47:14.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.308 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.308 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462209+0000", 2024-09-15T21:47:14.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075781000000000004 2024-09-15T21:47:14.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.308 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.308 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462214+0000", 2024-09-15T21:47:14.309 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1420000000000001e-06 2024-09-15T21:47:14.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.309 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.309 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.309 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462681+0000", 2024-09-15T21:47:14.309 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046634200000000002 2024-09-15T21:47:14.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.309 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.309 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.309 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462772+0000", 2024-09-15T21:47:14.309 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1414000000000003e-05 2024-09-15T21:47:14.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.309 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.309 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:14.310 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462871+0000", 2024-09-15T21:47:14.310 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8455000000000005e-05 2024-09-15T21:47:14.310 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.310 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.310 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.310 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464138+0000", 2024-09-15T21:47:14.310 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012673910000000001 2024-09-15T21:47:14.310 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.310 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.310 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.310 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469079+0000", 2024-09-15T21:47:14.310 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049407799999999997 2024-09-15T21:47:14.310 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.310 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.310 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.311 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469141+0000", 2024-09-15T21:47:14.311 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2169999999999996e-05 2024-09-15T21:47:14.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.311 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.311 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.311 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469179+0000", 2024-09-15T21:47:14.311 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7799000000000002e-05 2024-09-15T21:47:14.311 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.311 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.311 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.311 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.311 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.311 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:14.312 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 33.779723806, 2024-09-15T21:47:14.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004990497, 2024-09-15T21:47:14.312 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.312 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.312 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.312 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.312 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:14.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.312 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:14.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.313 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:14.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4729999999999999e-06 2024-09-15T21:47:14.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.313 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475576+0000", 2024-09-15T21:47:14.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1749999999999994e-06 2024-09-15T21:47:14.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.313 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475619+0000", 2024-09-15T21:47:14.314 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3557000000000003e-05 2024-09-15T21:47:14.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.314 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.314 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.314 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475631+0000", 2024-09-15T21:47:14.314 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1739999999999999e-05 2024-09-15T21:47:14.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.314 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.314 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.314 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475770+0000", 2024-09-15T21:47:14.314 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013950599999999999 2024-09-15T21:47:14.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.314 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.314 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.315 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475887+0000", 2024-09-15T21:47:14.315 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011616 2024-09-15T21:47:14.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.315 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:14.315 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476018+0000", 2024-09-15T21:47:14.315 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013119499999999999 2024-09-15T21:47:14.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.315 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.315 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478442+0000", 2024-09-15T21:47:14.315 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024240870000000001 2024-09-15T21:47:14.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.315 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480457+0000", 2024-09-15T21:47:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020151779999999998 2024-09-15T21:47:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480511+0000", 2024-09-15T21:47:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3582000000000003e-05 2024-09-15T21:47:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480556+0000", 2024-09-15T21:47:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4844e-05 2024-09-15T21:47:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.316 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 33.779709488000002, 2024-09-15T21:47:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051124810000000003, 2024-09-15T21:47:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475580+0000", 2024-09-15T21:47:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7800000000000002e-07 2024-09-15T21:47:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.318 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475595+0000", 2024-09-15T21:47:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4919000000000001e-05 2024-09-15T21:47:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475662+0000", 2024-09-15T21:47:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7769999999999997e-05 2024-09-15T21:47:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475669+0000", 2024-09-15T21:47:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.703e-06 2024-09-15T21:47:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.319 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476781+0000", 2024-09-15T21:47:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011118300000000001 2024-09-15T21:47:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476863+0000", 2024-09-15T21:47:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2208999999999998e-05 2024-09-15T21:47:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477040+0000", 2024-09-15T21:47:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000176424 2024-09-15T21:47:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.320 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480154+0000", 2024-09-15T21:47:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031147589999999999 2024-09-15T21:47:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480596+0000", 2024-09-15T21:47:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044172699999999999 2024-09-15T21:47:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480652+0000", 2024-09-15T21:47:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6350000000000001e-05 2024-09-15T21:47:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.321 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480692+0000", 2024-09-15T21:47:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9412000000000001e-05 2024-09-15T21:47:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:6 2.9 2:941d59bc:::61f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 28.710498955999999, 2024-09-15T21:47:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057427529999999997, 2024-09-15T21:47:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.322 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:14.323 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.41e-07 2024-09-15T21:47:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544792+0000", 2024-09-15T21:47:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9929999999999998e-06 2024-09-15T21:47:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544885+0000", 2024-09-15T21:47:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2825000000000007e-05 2024-09-15T21:47:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.324 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544891+0000", 2024-09-15T21:47:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.93e-06 2024-09-15T21:47:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546766+0000", 2024-09-15T21:47:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018747569999999999 2024-09-15T21:47:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546864+0000", 2024-09-15T21:47:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8603000000000004e-05 2024-09-15T21:47:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:14.325 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547028+0000", 2024-09-15T21:47:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000163461 2024-09-15T21:47:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549499+0000", 2024-09-15T21:47:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024711749999999999 2024-09-15T21:47:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550417+0000", 2024-09-15T21:47:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00091817199999999998 2024-09-15T21:47:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.326 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550476+0000", 2024-09-15T21:47:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8635999999999999e-05 2024-09-15T21:47:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550533+0000", 2024-09-15T21:47:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6959999999999997e-05 2024-09-15T21:47:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:14.327 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 28.710484490999999, 2024-09-15T21:47:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004917121, 2024-09-15T21:47:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:14.328 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544805+0000", 2024-09-15T21:47:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4600000000000001e-07 2024-09-15T21:47:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544817+0000", 2024-09-15T21:47:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2023e-05 2024-09-15T21:47:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545000+0000", 2024-09-15T21:47:14.329 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018384400000000001 2024-09-15T21:47:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545007+0000", 2024-09-15T21:47:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1249999999999998e-06 2024-09-15T21:47:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545670+0000", 2024-09-15T21:47:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00066324999999999999 2024-09-15T21:47:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.330 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545749+0000", 2024-09-15T21:47:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9239000000000005e-05 2024-09-15T21:47:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545885+0000", 2024-09-15T21:47:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013586199999999999 2024-09-15T21:47:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547944+0000", 2024-09-15T21:47:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020594720000000001 2024-09-15T21:47:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.331 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549630+0000", 2024-09-15T21:47:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001685831 2024-09-15T21:47:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549681+0000", 2024-09-15T21:47:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1187000000000003e-05 2024-09-15T21:47:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549721+0000", 2024-09-15T21:47:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0042e-05 2024-09-15T21:47:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.332 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 28.710470742999998, 2024-09-15T21:47:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050615549999999997, 2024-09-15T21:47:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:14.333 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:47:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.334 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544821+0000", 2024-09-15T21:47:14.335 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4399999999999999e-06 2024-09-15T21:47:14.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545025+0000", 2024-09-15T21:47:14.335 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020379400000000001 2024-09-15T21:47:14.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545030+0000", 2024-09-15T21:47:14.335 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3109999999999996e-06 2024-09-15T21:47:14.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546196+0000", 2024-09-15T21:47:14.336 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001166261 2024-09-15T21:47:14.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.336 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.336 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546307+0000", 2024-09-15T21:47:14.336 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000110641 2024-09-15T21:47:14.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.336 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:14.336 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546422+0000", 2024-09-15T21:47:14.336 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000115399 2024-09-15T21:47:14.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.337 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.337 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547973+0000", 2024-09-15T21:47:14.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001551228 2024-09-15T21:47:14.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.337 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.337 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.337 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549815+0000", 2024-09-15T21:47:14.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001841685 2024-09-15T21:47:14.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.337 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.337 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.337 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549856+0000", 2024-09-15T21:47:14.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.091e-05 2024-09-15T21:47:14.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.338 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.338 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.338 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549880+0000", 2024-09-15T21:47:14.338 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3580000000000001e-05 2024-09-15T21:47:14.338 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.338 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.338 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.338 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.338 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.338 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.338 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:14.338 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 28.710466735000001, 2024-09-15T21:47:14.338 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056931389999999998, 2024-09-15T21:47:14.338 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.338 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.339 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.339 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.339 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.339 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.339 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:14.339 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.339 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.339 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.339 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:14.339 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.339 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.339 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.340 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:14.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:47:14.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.340 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.340 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544828+0000", 2024-09-15T21:47:14.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4369999999999998e-06 2024-09-15T21:47:14.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.340 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.340 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545041+0000", 2024-09-15T21:47:14.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000212695 2024-09-15T21:47:14.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.341 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.341 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.341 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545058+0000", 2024-09-15T21:47:14.341 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7717999999999999e-05 2024-09-15T21:47:14.341 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.341 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.341 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.341 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546958+0000", 2024-09-15T21:47:14.341 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018995539999999999 2024-09-15T21:47:14.341 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.341 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.341 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.341 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547057+0000", 2024-09-15T21:47:14.341 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9370999999999994e-05 2024-09-15T21:47:14.342 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.342 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.342 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:14.342 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547579+0000", 2024-09-15T21:47:14.342 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052198200000000004 2024-09-15T21:47:14.342 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.342 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.342 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.342 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549929+0000", 2024-09-15T21:47:14.342 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023494649999999998 2024-09-15T21:47:14.342 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.342 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.342 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.342 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550401+0000", 2024-09-15T21:47:14.343 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00047277300000000001 2024-09-15T21:47:14.343 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.343 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.343 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.343 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550458+0000", 2024-09-15T21:47:14.343 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6756999999999999e-05 2024-09-15T21:47:14.343 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.343 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.343 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.343 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550515+0000", 2024-09-15T21:47:14.343 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7093999999999999e-05 2024-09-15T21:47:14.343 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.343 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.343 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.344 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.344 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.344 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:26 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.344 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:14.344 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 28.710460078000001, 2024-09-15T21:47:14.344 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050112029999999997, 2024-09-15T21:47:14.344 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.344 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.344 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.344 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.345 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.345 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.345 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:14.345 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.345 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.345 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.345 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.345 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:14.345 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.345 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.345 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.345 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.345 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:14.345 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.36e-07 2024-09-15T21:47:14.345 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.346 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.346 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.346 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545086+0000", 2024-09-15T21:47:14.346 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025704699999999999 2024-09-15T21:47:14.346 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.346 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.346 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.346 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545119+0000", 2024-09-15T21:47:14.346 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2734999999999997e-05 2024-09-15T21:47:14.346 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.346 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.346 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.346 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545126+0000", 2024-09-15T21:47:14.346 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5599999999999996e-06 2024-09-15T21:47:14.347 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.347 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.347 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.347 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545253+0000", 2024-09-15T21:47:14.347 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012644500000000001 2024-09-15T21:47:14.347 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.347 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.347 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.347 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545366+0000", 2024-09-15T21:47:14.347 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011349999999999999 2024-09-15T21:47:14.347 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.347 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.347 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:14.347 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545673+0000", 2024-09-15T21:47:14.348 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030643599999999999 2024-09-15T21:47:14.348 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.348 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.348 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.348 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547924+0000", 2024-09-15T21:47:14.348 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0022516189999999998 2024-09-15T21:47:14.348 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.348 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.348 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.348 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549746+0000", 2024-09-15T21:47:14.348 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018217629999999999 2024-09-15T21:47:14.348 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.348 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.348 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.349 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549800+0000", 2024-09-15T21:47:14.349 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3412999999999998e-05 2024-09-15T21:47:14.349 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.349 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.349 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.349 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549840+0000", 2024-09-15T21:47:14.349 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0448999999999998e-05 2024-09-15T21:47:14.349 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.349 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.349 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.349 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.349 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.349 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:30 2.4 2:218afd09:::203.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.350 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:14.350 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 28.710199248999999, 2024-09-15T21:47:14.350 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055481230000000003, 2024-09-15T21:47:14.350 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.350 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.350 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:14.350 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.350 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.350 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.350 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:14.350 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.350 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.350 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.350 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.351 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:14.351 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.351 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.351 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.351 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.351 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545091+0000", 2024-09-15T21:47:14.351 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7000000000000003e-07 2024-09-15T21:47:14.351 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.351 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.351 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.351 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545094+0000", 2024-09-15T21:47:14.351 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9569999999999998e-06 2024-09-15T21:47:14.351 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.351 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.351 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.352 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545156+0000", 2024-09-15T21:47:14.352 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2277e-05 2024-09-15T21:47:14.352 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.352 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.352 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.352 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545162+0000", 2024-09-15T21:47:14.352 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9120000000000003e-06 2024-09-15T21:47:14.352 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.352 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.352 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.352 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545262+0000", 2024-09-15T21:47:14.352 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010001900000000001 2024-09-15T21:47:14.352 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.352 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.352 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.353 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545388+0000", 2024-09-15T21:47:14.353 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126243 2024-09-15T21:47:14.353 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.353 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.353 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.353 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550638+0000", 2024-09-15T21:47:14.353 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052497450000000001 2024-09-15T21:47:14.353 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.353 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.353 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.353 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.353 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.353 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:10 2.9 2:954a9a2b:::62b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.353 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:14.354 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 22.690515788999999, 2024-09-15T21:47:14.354 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048438860000000004, 2024-09-15T21:47:14.354 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.354 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.354 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.354 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.354 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.354 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.354 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:14.354 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.354 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.354 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.354 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.354 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:14.355 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.355 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.355 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.355 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.355 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:14.355 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0400000000000002e-07 2024-09-15T21:47:14.355 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.355 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.355 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.355 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564786+0000", 2024-09-15T21:47:14.355 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2412000000000001e-05 2024-09-15T21:47:14.355 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.355 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.355 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.356 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564991+0000", 2024-09-15T21:47:14.356 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020483900000000001 2024-09-15T21:47:14.356 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.356 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.356 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.356 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564998+0000", 2024-09-15T21:47:14.356 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2459999999999997e-06 2024-09-15T21:47:14.356 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.356 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.356 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.356 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565422+0000", 2024-09-15T21:47:14.356 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00042420100000000003 2024-09-15T21:47:14.356 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.356 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.357 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.357 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565515+0000", 2024-09-15T21:47:14.357 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2449999999999997e-05 2024-09-15T21:47:14.357 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.357 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.357 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:14.357 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565642+0000", 2024-09-15T21:47:14.357 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012743699999999999 2024-09-15T21:47:14.357 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.357 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.357 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.357 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566656+0000", 2024-09-15T21:47:14.357 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010138580000000001 2024-09-15T21:47:14.357 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.357 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.358 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.358 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569552+0000", 2024-09-15T21:47:14.358 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028966819999999998 2024-09-15T21:47:14.358 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.358 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.358 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.358 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569594+0000", 2024-09-15T21:47:14.358 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1235000000000001e-05 2024-09-15T21:47:14.358 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.358 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.358 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.358 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569617+0000", 2024-09-15T21:47:14.358 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3221999999999999e-05 2024-09-15T21:47:14.358 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.359 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.359 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.359 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.359 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.359 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:14.360 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:47:14.361 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:47:14.359+0000 7f74313af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37847) UID: 0 2024-09-15T21:47:14.461 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:14.459+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:14.505 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:14.505 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:14.505 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:14.505 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:14.514 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:47:14.561 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:14.560+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:14.657 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:14.657 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:14.657 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:14.657 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:14.657 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:14.662 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:14.661+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:14.666 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:47:14.668 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:47:14.764 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:14.763+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:14.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.807+0000 7f56bf2e6640 1 Processor -- start 2024-09-15T21:47:14.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.807+0000 7f56bf2e6640 1 -- start start 2024-09-15T21:47:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.808+0000 7f56bf2e6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56b8155440 0x7f56b80cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.808+0000 7f56bf2e6640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f56b80cb7d0 0x7f56b80cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.808+0000 7f56bf2e6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f56b806d0e0 0x7f56b80cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.808+0000 7f56bf2e6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f56b815b470 con 0x7f56b806d0e0 2024-09-15T21:47:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.808+0000 7f56bf2e6640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f56b8155820 con 0x7f56b80cb7d0 2024-09-15T21:47:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.808+0000 7f56bf2e6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f56b815e7a0 con 0x7f56b8155440 2024-09-15T21:47:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.808+0000 7f56bd883640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f56b806d0e0 0x7f56b80cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.808+0000 7f56bd883640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f56b806d0e0 0x7f56b80cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49308/0 (socket says 172.21.15.100:49308) 2024-09-15T21:47:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.808+0000 7f56bd883640 1 -- 172.21.15.100:0/2961870037 learned_addr learned my addr 172.21.15.100:0/2961870037 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.808+0000 7f56bd082640 1 --2- 172.21.15.100:0/2961870037 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56b8155440 0x7f56b80cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.808+0000 7f56bc881640 1 --2- 172.21.15.100:0/2961870037 >> v2:172.21.15.122:3300/0 conn(0x7f56b80cb7d0 0x7f56b80cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.808+0000 7f56bd883640 1 -- 172.21.15.100:0/2961870037 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56b8155440 msgr2=0x7f56b80cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.808+0000 7f56bd883640 1 --2- 172.21.15.100:0/2961870037 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56b8155440 0x7f56b80cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.808+0000 7f56bd883640 1 -- 172.21.15.100:0/2961870037 >> v2:172.21.15.122:3300/0 conn(0x7f56b80cb7d0 msgr2=0x7f56b80cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.808+0000 7f56bd883640 1 --2- 172.21.15.100:0/2961870037 >> v2:172.21.15.122:3300/0 conn(0x7f56b80cb7d0 0x7f56b80cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.808+0000 7f56bd883640 1 -- 172.21.15.100:0/2961870037 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b815ea40 con 0x7f56b806d0e0 2024-09-15T21:47:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.808+0000 7f56bc881640 1 --2- 172.21.15.100:0/2961870037 >> v2:172.21.15.122:3300/0 conn(0x7f56b80cb7d0 0x7f56b80cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:47:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.809+0000 7f56bd883640 1 --2- 172.21.15.100:0/2961870037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56b806d0e0 0x7f56b80cc1e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f56b80ce410 tx=0x7f56b002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c7153269674ff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.809+0000 7f56affff640 1 -- 172.21.15.100:0/2961870037 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0036070 con 0x7f56b806d0e0 2024-09-15T21:47:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.809+0000 7f56affff640 1 -- 172.21.15.100:0/2961870037 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56b0031600 con 0x7f56b806d0e0 2024-09-15T21:47:14.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.809+0000 7f56affff640 1 -- 172.21.15.100:0/2961870037 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b00319d0 con 0x7f56b806d0e0 2024-09-15T21:47:14.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.810+0000 7f56bf2e6640 1 -- 172.21.15.100:0/2961870037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56b806d0e0 msgr2=0x7f56b80cc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:14.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.810+0000 7f56bf2e6640 1 --2- 172.21.15.100:0/2961870037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56b806d0e0 0x7f56b80cc1e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f56b80ce410 tx=0x7f56b002a6c0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.810+0000 7f56bf2e6640 1 -- 172.21.15.100:0/2961870037 shutdown_connections 2024-09-15T21:47:14.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.810+0000 7f56bf2e6640 1 --2- 172.21.15.100:0/2961870037 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56b806d0e0 0x7f56b80cc1e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.810+0000 7f56bf2e6640 1 --2- 172.21.15.100:0/2961870037 >> v2:172.21.15.122:3300/0 conn(0x7f56b80cb7d0 0x7f56b80cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.810+0000 7f56bf2e6640 1 --2- 172.21.15.100:0/2961870037 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56b8155440 0x7f56b80cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.810+0000 7f56bf2e6640 1 -- 172.21.15.100:0/2961870037 >> 172.21.15.100:0/2961870037 conn(0x7f56b80680d0 msgr2=0x7f56b806d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:14.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.810+0000 7f56bf2e6640 1 -- 172.21.15.100:0/2961870037 shutdown_connections 2024-09-15T21:47:14.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.810+0000 7f56bf2e6640 1 -- 172.21.15.100:0/2961870037 wait complete. 2024-09-15T21:47:14.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.811+0000 7f56bf2e6640 1 Processor -- start 2024-09-15T21:47:14.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.811+0000 7f56bf2e6640 1 -- start start 2024-09-15T21:47:14.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56bf2e6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56b806d0e0 0x7f56b81559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56bf2e6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56b80cb7d0 0x7f56b81584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56bf2e6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56b8155440 0x7f56b8153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56bf2e6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f56b815f2e0 con 0x7f56b80cb7d0 2024-09-15T21:47:14.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56bf2e6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56b815f000 con 0x7f56b8155440 2024-09-15T21:47:14.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56bf2e6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f56b815b070 con 0x7f56b806d0e0 2024-09-15T21:47:14.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56bc881640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56b80cb7d0 0x7f56b81584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56bc881640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56b80cb7d0 0x7f56b81584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49312/0 (socket says 172.21.15.100:49312) 2024-09-15T21:47:14.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56bc881640 1 -- 172.21.15.100:0/2734004719 learned_addr learned my addr 172.21.15.100:0/2734004719 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:14.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56bd082640 1 --2- 172.21.15.100:0/2734004719 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56b806d0e0 0x7f56b81559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56bd883640 1 --2- 172.21.15.100:0/2734004719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56b8155440 0x7f56b8153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56bc881640 1 -- 172.21.15.100:0/2734004719 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56b806d0e0 msgr2=0x7f56b81559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:14.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56bc881640 1 --2- 172.21.15.100:0/2734004719 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56b806d0e0 0x7f56b81559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56bc881640 1 -- 172.21.15.100:0/2734004719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56b8155440 msgr2=0x7f56b8153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:14.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56bc881640 1 --2- 172.21.15.100:0/2734004719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56b8155440 0x7f56b8153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:14.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56bc881640 1 -- 172.21.15.100:0/2734004719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56b81589e0 con 0x7f56b80cb7d0 2024-09-15T21:47:14.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56bc881640 1 --2- 172.21.15.100:0/2734004719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56b80cb7d0 0x7f56b81584a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f56a800e810 tx=0x7f56a800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:14.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56adffb640 1 -- 172.21.15.100:0/2734004719 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a8017070 con 0x7f56b80cb7d0 2024-09-15T21:47:14.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56adffb640 1 -- 172.21.15.100:0/2734004719 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56a800c6d0 con 0x7f56b80cb7d0 2024-09-15T21:47:14.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56adffb640 1 -- 172.21.15.100:0/2734004719 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a800f040 con 0x7f56b80cb7d0 2024-09-15T21:47:14.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.812+0000 7f56bf2e6640 1 -- 172.21.15.100:0/2734004719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56b8150240 con 0x7f56b80cb7d0 2024-09-15T21:47:14.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.813+0000 7f56bf2e6640 1 -- 172.21.15.100:0/2734004719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56b814f860 con 0x7f56b80cb7d0 2024-09-15T21:47:14.819 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:14.819 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:14.819 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:14.819 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:14.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.819 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:14.819 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:14.819 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.399445489999998, 2024-09-15T21:47:14.819 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00598016, 2024-09-15T21:47:14.820 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.820 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.820 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.820 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.820 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.820 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.820 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:14.820 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.820 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.820 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.820 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:14.820 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.821 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.821 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.821 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.821 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413339+0000", 2024-09-15T21:47:14.821 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6279999999999998e-06 2024-09-15T21:47:14.821 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.821 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.821 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.821 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413354+0000", 2024-09-15T21:47:14.821 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4877e-05 2024-09-15T21:47:14.821 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.821 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.821 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.821 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413418+0000", 2024-09-15T21:47:14.822 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4274000000000002e-05 2024-09-15T21:47:14.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.822 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.822 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413428+0000", 2024-09-15T21:47:14.822 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5899999999999997e-06 2024-09-15T21:47:14.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.822 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.822 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413563+0000", 2024-09-15T21:47:14.822 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013521900000000001 2024-09-15T21:47:14.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.822 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.823 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413678+0000", 2024-09-15T21:47:14.823 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000114904 2024-09-15T21:47:14.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.823 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.823 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:14.823 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413789+0000", 2024-09-15T21:47:14.823 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011069300000000001 2024-09-15T21:47:14.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.823 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.823 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.823 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417603+0000", 2024-09-15T21:47:14.824 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038138460000000001 2024-09-15T21:47:14.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.824 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.824 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.824 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419220+0000", 2024-09-15T21:47:14.824 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001617678 2024-09-15T21:47:14.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.824 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.824 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.824 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419275+0000", 2024-09-15T21:47:14.824 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4391999999999997e-05 2024-09-15T21:47:14.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.825 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.825 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.825 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419316+0000", 2024-09-15T21:47:14.825 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1059000000000001e-05 2024-09-15T21:47:14.825 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.825 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.825 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.825 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.825 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.825 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:14.825 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:14.825 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.399423362, 2024-09-15T21:47:14.825 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0087926470000000007, 2024-09-15T21:47:14.825 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.826 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.826 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.826 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.826 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.826 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.826 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:14.826 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.826 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.826 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.826 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.826 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:14.826 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.827 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.827 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:14.827 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4099999999999999e-07 2024-09-15T21:47:14.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.827 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.827 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413361+0000", 2024-09-15T21:47:14.827 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6189999999999998e-06 2024-09-15T21:47:14.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.827 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.828 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413461+0000", 2024-09-15T21:47:14.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000100053 2024-09-15T21:47:14.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.828 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.828 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.828 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413466+0000", 2024-09-15T21:47:14.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3850000000000001e-06 2024-09-15T21:47:14.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.828 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.828 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.828 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414073+0000", 2024-09-15T21:47:14.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000606724 2024-09-15T21:47:14.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.829 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.829 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.829 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414143+0000", 2024-09-15T21:47:14.829 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9727000000000005e-05 2024-09-15T21:47:14.829 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.829 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.829 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:14.829 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414259+0000", 2024-09-15T21:47:14.829 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000116178 2024-09-15T21:47:14.829 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.829 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.829 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.829 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417610+0000", 2024-09-15T21:47:14.829 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003351176 2024-09-15T21:47:14.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.830 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.830 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422058+0000", 2024-09-15T21:47:14.830 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004447599 2024-09-15T21:47:14.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.830 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.830 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422111+0000", 2024-09-15T21:47:14.830 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3606999999999997e-05 2024-09-15T21:47:14.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.830 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.831 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422150+0000", 2024-09-15T21:47:14.831 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9137999999999997e-05 2024-09-15T21:47:14.831 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.831 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.831 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.831 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.831 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.831 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:14.831 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:14.831 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.399418932000003, 2024-09-15T21:47:14.831 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0089009769999999992, 2024-09-15T21:47:14.831 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.831 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.831 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.832 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:14.832 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:14.832 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413363+0000", 2024-09-15T21:47:14.833 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3999999999999997e-07 2024-09-15T21:47:14.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.833 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.833 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413365+0000", 2024-09-15T21:47:14.833 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.216e-06 2024-09-15T21:47:14.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.833 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.833 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413481+0000", 2024-09-15T21:47:14.833 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000116311 2024-09-15T21:47:14.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.833 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.834 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413488+0000", 2024-09-15T21:47:14.834 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0110000000000003e-06 2024-09-15T21:47:14.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.834 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.834 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.834 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414452+0000", 2024-09-15T21:47:14.834 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00096385399999999999 2024-09-15T21:47:14.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.834 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.834 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.834 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414545+0000", 2024-09-15T21:47:14.834 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.3024999999999998e-05 2024-09-15T21:47:14.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.835 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.835 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:14.835 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414644+0000", 2024-09-15T21:47:14.835 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9098000000000006e-05 2024-09-15T21:47:14.835 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.835 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.835 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.835 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417667+0000", 2024-09-15T21:47:14.835 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030229409999999999 2024-09-15T21:47:14.835 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.835 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.835 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.835 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422200+0000", 2024-09-15T21:47:14.835 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045327880000000003 2024-09-15T21:47:14.835 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.836 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.836 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.836 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422228+0000", 2024-09-15T21:47:14.836 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8109000000000001e-05 2024-09-15T21:47:14.836 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.836 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.836 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.836 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422263+0000", 2024-09-15T21:47:14.837 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5284000000000003e-05 2024-09-15T21:47:14.837 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.837 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.837 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.837 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:14.837 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:14.837 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.399415345999998, 2024-09-15T21:47:14.837 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0089846820000000008, 2024-09-15T21:47:14.837 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.837 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.837 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.837 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:14.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:14.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:14.839 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7599999999999998e-07 2024-09-15T21:47:14.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.839 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.839 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413368+0000", 2024-09-15T21:47:14.839 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.058e-06 2024-09-15T21:47:14.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.839 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.839 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413832+0000", 2024-09-15T21:47:14.839 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046398700000000001 2024-09-15T21:47:14.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.839 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.840 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413841+0000", 2024-09-15T21:47:14.840 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1710000000000008e-06 2024-09-15T21:47:14.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.840 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.840 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.840 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414869+0000", 2024-09-15T21:47:14.840 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001027584 2024-09-15T21:47:14.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.840 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.840 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.840 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414946+0000", 2024-09-15T21:47:14.840 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7177999999999999e-05 2024-09-15T21:47:14.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.840 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.841 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:14.841 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415054+0000", 2024-09-15T21:47:14.841 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010807 2024-09-15T21:47:14.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.841 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.841 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.841 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417672+0000", 2024-09-15T21:47:14.841 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002618015 2024-09-15T21:47:14.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.841 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.841 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.841 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422295+0000", 2024-09-15T21:47:14.841 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046233300000000001 2024-09-15T21:47:14.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.842 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.842 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.842 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422330+0000", 2024-09-15T21:47:14.842 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4476e-05 2024-09-15T21:47:14.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.842 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.842 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.842 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422351+0000", 2024-09-15T21:47:14.842 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0537e-05 2024-09-15T21:47:14.842 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.842 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.842 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.842 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.842 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:14.843 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:14.843 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.399411985999997, 2024-09-15T21:47:14.843 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0090873700000000009, 2024-09-15T21:47:14.843 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.843 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.843 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.843 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.843 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.843 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.843 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:14.843 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.843 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.843 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.843 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.844 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:14.844 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.844 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.844 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:14.844 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6100000000000002e-07 2024-09-15T21:47:14.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.844 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.844 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413371+0000", 2024-09-15T21:47:14.844 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.764e-06 2024-09-15T21:47:14.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.845 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.845 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413859+0000", 2024-09-15T21:47:14.845 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00048751900000000001 2024-09-15T21:47:14.845 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.845 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.845 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413864+0000", 2024-09-15T21:47:14.845 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6200000000000004e-06 2024-09-15T21:47:14.845 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.846 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.846 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415253+0000", 2024-09-15T21:47:14.846 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013884050000000001 2024-09-15T21:47:14.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.846 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.846 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415364+0000", 2024-09-15T21:47:14.846 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000111119 2024-09-15T21:47:14.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.846 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:14.846 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415453+0000", 2024-09-15T21:47:14.846 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8641999999999996e-05 2024-09-15T21:47:14.847 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.847 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.847 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.847 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417676+0000", 2024-09-15T21:47:14.847 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00222368 2024-09-15T21:47:14.847 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.847 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.847 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.847 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422395+0000", 2024-09-15T21:47:14.847 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047191150000000003 2024-09-15T21:47:14.847 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.847 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.847 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.847 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422434+0000", 2024-09-15T21:47:14.848 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8525999999999997e-05 2024-09-15T21:47:14.848 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.848 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.848 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.848 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422457+0000", 2024-09-15T21:47:14.848 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2719000000000001e-05 2024-09-15T21:47:14.848 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.848 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.848 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.848 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.848 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.848 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:14.848 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:14.849 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.399408964000003, 2024-09-15T21:47:14.849 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0092139749999999992, 2024-09-15T21:47:14.849 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.849 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.849 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.849 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.849 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.849 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.849 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:14.849 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.849 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.849 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.849 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.849 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:14.850 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.850 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.850 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.850 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.850 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:14.850 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3799999999999999e-07 2024-09-15T21:47:14.850 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.850 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.850 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.850 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413375+0000", 2024-09-15T21:47:14.850 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1959999999999998e-06 2024-09-15T21:47:14.850 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.850 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.850 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.851 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413876+0000", 2024-09-15T21:47:14.851 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00050134500000000005 2024-09-15T21:47:14.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.851 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.851 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.851 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413881+0000", 2024-09-15T21:47:14.851 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3580000000000002e-06 2024-09-15T21:47:14.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.851 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.851 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.851 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415669+0000", 2024-09-15T21:47:14.851 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0017875090000000001 2024-09-15T21:47:14.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.851 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.852 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.852 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415737+0000", 2024-09-15T21:47:14.852 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8386000000000006e-05 2024-09-15T21:47:14.852 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.852 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.852 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:14.852 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415857+0000", 2024-09-15T21:47:14.852 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119906 2024-09-15T21:47:14.852 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.852 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.852 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.852 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421848+0000", 2024-09-15T21:47:14.852 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059903880000000001 2024-09-15T21:47:14.852 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.853 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.853 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.853 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422513+0000", 2024-09-15T21:47:14.853 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00066575599999999999 2024-09-15T21:47:14.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.853 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.853 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.853 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422549+0000", 2024-09-15T21:47:14.853 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5719999999999997e-05 2024-09-15T21:47:14.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.853 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.853 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.853 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422586+0000", 2024-09-15T21:47:14.853 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7172999999999998e-05 2024-09-15T21:47:14.854 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.854 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.854 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.854 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.854 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.854 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:14.854 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:14.854 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.399403141999997, 2024-09-15T21:47:14.854 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.009324087, 2024-09-15T21:47:14.854 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.854 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.854 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.854 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.854 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.854 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.855 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:14.855 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.855 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.855 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.855 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:14.855 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.855 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.855 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.855 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:14.855 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8900000000000001e-07 2024-09-15T21:47:14.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.855 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.856 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.856 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413381+0000", 2024-09-15T21:47:14.856 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1959999999999998e-06 2024-09-15T21:47:14.856 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.856 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.856 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.856 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413895+0000", 2024-09-15T21:47:14.856 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051466000000000001 2024-09-15T21:47:14.856 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.856 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.856 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.856 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413900+0000", 2024-09-15T21:47:14.856 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6090000000000003e-06 2024-09-15T21:47:14.857 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.857 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.857 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.857 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416081+0000", 2024-09-15T21:47:14.857 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021816090000000002 2024-09-15T21:47:14.857 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.857 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.857 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.857 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416173+0000", 2024-09-15T21:47:14.857 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1334000000000003e-05 2024-09-15T21:47:14.857 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.857 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.857 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:14.857 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416264+0000", 2024-09-15T21:47:14.858 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0864000000000003e-05 2024-09-15T21:47:14.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.858 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.858 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421854+0000", 2024-09-15T21:47:14.858 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055902240000000004 2024-09-15T21:47:14.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.858 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.858 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422629+0000", 2024-09-15T21:47:14.858 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00077550700000000004 2024-09-15T21:47:14.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.858 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.859 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422678+0000", 2024-09-15T21:47:14.859 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8810999999999997e-05 2024-09-15T21:47:14.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.859 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.859 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.859 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422702+0000", 2024-09-15T21:47:14.859 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3983999999999999e-05 2024-09-15T21:47:14.859 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.859 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.859 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.859 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.859 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:14.859 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:14.859 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.39939966, 2024-09-15T21:47:14.860 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0094270580000000003, 2024-09-15T21:47:14.860 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.860 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.860 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.860 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.860 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.860 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.860 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:14.860 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.860 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.860 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.860 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.860 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:14.860 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.861 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.861 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.861 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.861 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:14.861 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7500000000000001e-07 2024-09-15T21:47:14.861 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.861 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.861 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.861 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413386+0000", 2024-09-15T21:47:14.861 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5619999999999997e-06 2024-09-15T21:47:14.861 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.861 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.861 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.861 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413918+0000", 2024-09-15T21:47:14.862 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00053162499999999998 2024-09-15T21:47:14.862 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.862 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.862 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.862 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413923+0000", 2024-09-15T21:47:14.862 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9280000000000001e-06 2024-09-15T21:47:14.862 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.862 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.862 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.862 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416492+0000", 2024-09-15T21:47:14.862 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0025687140000000002 2024-09-15T21:47:14.862 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.862 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.862 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.863 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416558+0000", 2024-09-15T21:47:14.863 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6699000000000006e-05 2024-09-15T21:47:14.863 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.863 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.863 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:14.863 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416691+0000", 2024-09-15T21:47:14.863 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013263199999999999 2024-09-15T21:47:14.863 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.863 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.863 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.863 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421879+0000", 2024-09-15T21:47:14.863 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051881089999999998 2024-09-15T21:47:14.863 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.863 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.864 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.864 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422742+0000", 2024-09-15T21:47:14.864 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000862845 2024-09-15T21:47:14.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.864 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.864 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.864 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422786+0000", 2024-09-15T21:47:14.864 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4332000000000002e-05 2024-09-15T21:47:14.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.864 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.864 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.864 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422809+0000", 2024-09-15T21:47:14.864 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2337e-05 2024-09-15T21:47:14.864 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.865 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.865 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.865 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.865 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.865 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:4 2.10 2:0f65c9b8:::60a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.865 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:14.865 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 40.357113491, 2024-09-15T21:47:14.865 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058710300000000002, 2024-09-15T21:47:14.865 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.865 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.865 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.865 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.865 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.866 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.866 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:14.866 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.866 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.866 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.866 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.866 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:14.866 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.866 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.866 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.866 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.866 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:14.866 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0800000000000005e-07 2024-09-15T21:47:14.866 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.866 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.867 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.867 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455672+0000", 2024-09-15T21:47:14.867 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3900000000000002e-06 2024-09-15T21:47:14.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.867 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.867 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.867 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456538+0000", 2024-09-15T21:47:14.867 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00086613100000000002 2024-09-15T21:47:14.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.867 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.867 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.867 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456545+0000", 2024-09-15T21:47:14.867 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4830000000000002e-06 2024-09-15T21:47:14.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.868 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.868 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.868 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456594+0000", 2024-09-15T21:47:14.868 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8429e-05 2024-09-15T21:47:14.868 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.868 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.868 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.868 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456675+0000", 2024-09-15T21:47:14.868 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1516999999999999e-05 2024-09-15T21:47:14.868 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.868 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.868 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:14.868 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456772+0000", 2024-09-15T21:47:14.868 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6523999999999999e-05 2024-09-15T21:47:14.869 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.869 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.869 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.869 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.460299+0000", 2024-09-15T21:47:14.869 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035274859999999998 2024-09-15T21:47:14.869 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.869 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.869 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.869 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461451+0000", 2024-09-15T21:47:14.869 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011518100000000001 2024-09-15T21:47:14.869 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.869 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.869 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.869 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461498+0000", 2024-09-15T21:47:14.870 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7406e-05 2024-09-15T21:47:14.870 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.870 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.870 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.870 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461539+0000", 2024-09-15T21:47:14.870 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0346000000000003e-05 2024-09-15T21:47:14.870 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.870 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.870 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.870 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.870 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.870 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:31 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.870 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:14.870 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 40.357108228999998, 2024-09-15T21:47:14.871 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0078158979999999999, 2024-09-15T21:47:14.871 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.871 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.871 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:14.871 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.871 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.871 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.871 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:14.871 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.871 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.871 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.871 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.871 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:14.871 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.872 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.872 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:14.872 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0499999999999999e-07 2024-09-15T21:47:14.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.872 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.872 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455675+0000", 2024-09-15T21:47:14.872 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.376e-06 2024-09-15T21:47:14.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.872 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.872 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456964+0000", 2024-09-15T21:47:14.873 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001289204 2024-09-15T21:47:14.873 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.873 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.873 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.873 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456970+0000", 2024-09-15T21:47:14.873 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2129999999999996e-06 2024-09-15T21:47:14.873 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.873 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.873 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.873 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457032+0000", 2024-09-15T21:47:14.873 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2209e-05 2024-09-15T21:47:14.873 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.873 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.873 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.874 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457134+0000", 2024-09-15T21:47:14.874 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010204000000000001 2024-09-15T21:47:14.874 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.874 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.874 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.874 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463489+0000", 2024-09-15T21:47:14.874 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0063545509999999999 2024-09-15T21:47:14.874 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.874 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.874 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.874 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.874 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.874 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.874 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:14.874 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 34.337348464999998, 2024-09-15T21:47:14.875 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057594079999999997, 2024-09-15T21:47:14.875 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.875 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.875 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.875 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.875 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.875 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.875 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:14.875 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.875 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.875 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.875 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.875 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:14.875 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.876 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.876 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.876 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.876 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:14.876 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1999999999999999e-07 2024-09-15T21:47:14.876 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.876 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.876 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.876 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475437+0000", 2024-09-15T21:47:14.876 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5429999999999998e-06 2024-09-15T21:47:14.876 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.876 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.876 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.877 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475686+0000", 2024-09-15T21:47:14.877 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000248908 2024-09-15T21:47:14.877 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.878 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.878 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.878 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475695+0000", 2024-09-15T21:47:14.879 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.9740000000000008e-06 2024-09-15T21:47:14.879 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.879 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.879 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.879 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476291+0000", 2024-09-15T21:47:14.879 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00059651500000000004 2024-09-15T21:47:14.879 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.879 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.879 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.879 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476366+0000", 2024-09-15T21:47:14.880 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4531999999999997e-05 2024-09-15T21:47:14.880 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.880 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.880 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:14.880 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476491+0000", 2024-09-15T21:47:14.880 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012527299999999999 2024-09-15T21:47:14.880 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.880 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.880 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.880 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480600+0000", 2024-09-15T21:47:14.880 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0041085870000000003 2024-09-15T21:47:14.880 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.880 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.880 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.881 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481089+0000", 2024-09-15T21:47:14.881 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000489037 2024-09-15T21:47:14.881 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.881 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.881 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.881 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481152+0000", 2024-09-15T21:47:14.881 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3136e-05 2024-09-15T21:47:14.881 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.881 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.881 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.881 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481192+0000", 2024-09-15T21:47:14.881 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0182999999999998e-05 2024-09-15T21:47:14.881 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.881 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.882 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.882 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.882 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.882 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:17 2.10 2:0e9da45e:::61b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.882 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:14.882 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 34.337342804000002, 2024-09-15T21:47:14.882 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058311830000000002, 2024-09-15T21:47:14.882 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.882 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.882 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.882 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.882 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.882 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.882 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:14.882 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.883 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.883 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.883 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.883 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:14.883 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.883 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.883 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.883 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.883 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475439+0000", 2024-09-15T21:47:14.883 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1300000000000001e-07 2024-09-15T21:47:14.883 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.883 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.883 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.883 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475441+0000", 2024-09-15T21:47:14.883 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.419e-06 2024-09-15T21:47:14.884 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.884 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.884 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.884 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:14.884 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000269187 2024-09-15T21:47:14.884 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.884 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.884 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.884 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475715+0000", 2024-09-15T21:47:14.884 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3660000000000002e-06 2024-09-15T21:47:14.884 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.884 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.884 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.884 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476671+0000", 2024-09-15T21:47:14.885 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000956507 2024-09-15T21:47:14.885 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.885 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.885 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.885 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477500+0000", 2024-09-15T21:47:14.885 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00082897400000000001 2024-09-15T21:47:14.885 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.885 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.885 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:14.885 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477625+0000", 2024-09-15T21:47:14.885 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012513100000000001 2024-09-15T21:47:14.885 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.885 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.886 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.886 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480520+0000", 2024-09-15T21:47:14.886 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002894715 2024-09-15T21:47:14.886 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.886 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.886 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.886 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481184+0000", 2024-09-15T21:47:14.886 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000663801 2024-09-15T21:47:14.886 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.886 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.886 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.886 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481232+0000", 2024-09-15T21:47:14.886 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8457e-05 2024-09-15T21:47:14.886 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.887 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.887 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.887 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481270+0000", 2024-09-15T21:47:14.887 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7313e-05 2024-09-15T21:47:14.887 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.887 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.887 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.887 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.887 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.887 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.887 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:14.887 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 34.337338928000001, 2024-09-15T21:47:14.887 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0075605530000000002, 2024-09-15T21:47:14.887 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.887 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.888 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.888 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.888 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.888 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.888 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:14.888 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.888 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.888 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.888 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:14.888 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.888 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.888 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.889 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:14.889 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.53e-07 2024-09-15T21:47:14.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.889 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.889 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.889 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475445+0000", 2024-09-15T21:47:14.889 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7360000000000001e-06 2024-09-15T21:47:14.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.889 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.889 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.889 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475735+0000", 2024-09-15T21:47:14.889 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028975799999999999 2024-09-15T21:47:14.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.889 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.889 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.890 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475740+0000", 2024-09-15T21:47:14.890 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8250000000000004e-06 2024-09-15T21:47:14.890 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.890 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.890 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.890 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478479+0000", 2024-09-15T21:47:14.890 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002739525 2024-09-15T21:47:14.890 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.890 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.890 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.890 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478556+0000", 2024-09-15T21:47:14.890 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6213999999999999e-05 2024-09-15T21:47:14.890 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.891 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.891 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:14.891 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478672+0000", 2024-09-15T21:47:14.891 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011605 2024-09-15T21:47:14.891 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.891 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.891 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.891 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481323+0000", 2024-09-15T21:47:14.892 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0026518819999999999 2024-09-15T21:47:14.892 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.892 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.892 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.892 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482918+0000", 2024-09-15T21:47:14.892 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001594105 2024-09-15T21:47:14.892 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.892 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.892 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.892 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482965+0000", 2024-09-15T21:47:14.892 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6950000000000003e-05 2024-09-15T21:47:14.892 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.892 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.892 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.893 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483003+0000", 2024-09-15T21:47:14.893 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8254999999999999e-05 2024-09-15T21:47:14.893 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.893 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.893 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.893 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.893 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.893 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.893 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:14.893 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 34.337334872, 2024-09-15T21:47:14.893 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076582890000000004, 2024-09-15T21:47:14.893 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.893 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.893 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.893 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.894 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.894 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.894 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:14.894 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.894 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.894 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.894 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.894 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:14.894 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.894 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.894 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.894 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.894 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475447+0000", 2024-09-15T21:47:14.894 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4699999999999998e-07 2024-09-15T21:47:14.895 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.895 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.895 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.895 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475449+0000", 2024-09-15T21:47:14.895 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.176e-06 2024-09-15T21:47:14.895 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.895 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.895 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.895 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475752+0000", 2024-09-15T21:47:14.895 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030374200000000002 2024-09-15T21:47:14.895 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.895 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.895 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.895 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475757+0000", 2024-09-15T21:47:14.896 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2209999999999999e-06 2024-09-15T21:47:14.896 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.896 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.896 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.896 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477441+0000", 2024-09-15T21:47:14.896 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016843489999999999 2024-09-15T21:47:14.896 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.896 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.896 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.896 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477567+0000", 2024-09-15T21:47:14.896 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012552099999999999 2024-09-15T21:47:14.896 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.896 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.896 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:14.897 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477712+0000", 2024-09-15T21:47:14.897 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000145427 2024-09-15T21:47:14.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.897 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.897 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.897 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482935+0000", 2024-09-15T21:47:14.897 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052231200000000004 2024-09-15T21:47:14.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.897 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.897 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.897 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483015+0000", 2024-09-15T21:47:14.897 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9634000000000004e-05 2024-09-15T21:47:14.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.897 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.898 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.898 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483065+0000", 2024-09-15T21:47:14.898 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9869e-05 2024-09-15T21:47:14.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.898 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.898 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.898 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483105+0000", 2024-09-15T21:47:14.898 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9983e-05 2024-09-15T21:47:14.898 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.898 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.898 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.898 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.898 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.898 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:14.899 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 34.337328331000002, 2024-09-15T21:47:14.899 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076459379999999997, 2024-09-15T21:47:14.899 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.899 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.899 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.899 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.899 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.899 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.899 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:14.899 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.899 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.899 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.899 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.899 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:14.900 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.900 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.900 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.900 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.900 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:14.900 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.58e-07 2024-09-15T21:47:14.900 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.900 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.900 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.900 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475455+0000", 2024-09-15T21:47:14.900 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.075e-06 2024-09-15T21:47:14.900 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.900 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.900 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.901 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475769+0000", 2024-09-15T21:47:14.901 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00031336999999999999 2024-09-15T21:47:14.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.901 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.901 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.901 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475773+0000", 2024-09-15T21:47:14.901 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8080000000000004e-06 2024-09-15T21:47:14.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.901 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.901 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.901 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478950+0000", 2024-09-15T21:47:14.901 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031763870000000001 2024-09-15T21:47:14.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.901 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.902 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.902 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479067+0000", 2024-09-15T21:47:14.902 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000117543 2024-09-15T21:47:14.902 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.902 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.902 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:14.902 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479196+0000", 2024-09-15T21:47:14.902 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000128987 2024-09-15T21:47:14.902 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.902 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.902 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.902 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481366+0000", 2024-09-15T21:47:14.902 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021699089999999998 2024-09-15T21:47:14.902 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.903 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.903 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.903 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483015+0000", 2024-09-15T21:47:14.903 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016486719999999999 2024-09-15T21:47:14.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.903 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.903 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.903 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483061+0000", 2024-09-15T21:47:14.903 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5729999999999998e-05 2024-09-15T21:47:14.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.903 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.903 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.903 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483099+0000", 2024-09-15T21:47:14.903 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8099000000000002e-05 2024-09-15T21:47:14.904 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.904 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.904 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.904 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.904 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.904 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.904 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:14.904 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 34.337281222000001, 2024-09-15T21:47:14.904 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077290859999999996, 2024-09-15T21:47:14.904 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.904 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.904 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.904 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.905 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.905 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.905 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:14.905 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.905 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.905 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.905 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.905 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:14.905 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.905 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.905 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.905 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.905 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475501+0000", 2024-09-15T21:47:14.905 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1900000000000002e-07 2024-09-15T21:47:14.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.906 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.906 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.906 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475504+0000", 2024-09-15T21:47:14.906 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.168e-06 2024-09-15T21:47:14.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.906 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.906 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.906 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477037+0000", 2024-09-15T21:47:14.906 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001533202 2024-09-15T21:47:14.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.906 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.906 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.906 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477050+0000", 2024-09-15T21:47:14.906 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3132999999999999e-05 2024-09-15T21:47:14.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.907 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.907 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.907 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479477+0000", 2024-09-15T21:47:14.907 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002426489 2024-09-15T21:47:14.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.907 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.907 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.907 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479553+0000", 2024-09-15T21:47:14.907 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6583999999999997e-05 2024-09-15T21:47:14.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.907 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.907 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:14.907 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479637+0000", 2024-09-15T21:47:14.908 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4172999999999995e-05 2024-09-15T21:47:14.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.908 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.908 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483108+0000", 2024-09-15T21:47:14.908 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003470356 2024-09-15T21:47:14.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.908 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.908 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483160+0000", 2024-09-15T21:47:14.908 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2507999999999999e-05 2024-09-15T21:47:14.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.908 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.909 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483202+0000", 2024-09-15T21:47:14.909 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1758999999999998e-05 2024-09-15T21:47:14.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.909 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.909 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.909 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483229+0000", 2024-09-15T21:47:14.909 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6995000000000002e-05 2024-09-15T21:47:14.909 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.909 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.909 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.909 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.909 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.909 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:14.909 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 34.337276265, 2024-09-15T21:47:14.910 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.55145717800000005, 2024-09-15T21:47:14.910 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.910 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.910 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:14.910 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.910 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.910 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.910 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:14.910 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.910 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.910 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.910 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.910 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:14.910 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.911 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.911 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.911 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:14.911 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.65e-07 2024-09-15T21:47:14.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.911 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.911 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.911 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475507+0000", 2024-09-15T21:47:14.911 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4699999999999999e-06 2024-09-15T21:47:14.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.911 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.911 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.911 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.025935+0000", 2024-09-15T21:47:14.911 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.55042854200000002 2024-09-15T21:47:14.912 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.912 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.912 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.912 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.025965+0000", 2024-09-15T21:47:14.912 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9995999999999999e-05 2024-09-15T21:47:14.912 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.912 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.912 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.912 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026016+0000", 2024-09-15T21:47:14.912 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1310000000000002e-05 2024-09-15T21:47:14.912 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.912 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.912 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.913 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026122+0000", 2024-09-15T21:47:14.913 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105605 2024-09-15T21:47:14.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.913 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.913 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.913 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026962+0000", 2024-09-15T21:47:14.913 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00083998999999999999 2024-09-15T21:47:14.913 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.913 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.913 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.913 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.913 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:5 2.e 2:75ee5fd8:::629.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.913 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:14.913 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 23.248069900000001, 2024-09-15T21:47:14.914 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0071955470000000001, 2024-09-15T21:47:14.914 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.914 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.914 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.914 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.914 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.914 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.914 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:14.914 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.914 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.914 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.914 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:14.914 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.915 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.915 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564713+0000", 2024-09-15T21:47:14.915 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.866e-06 2024-09-15T21:47:14.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.915 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.915 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564721+0000", 2024-09-15T21:47:14.915 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3299999999999999e-06 2024-09-15T21:47:14.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.915 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.915 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564762+0000", 2024-09-15T21:47:14.915 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0707999999999997e-05 2024-09-15T21:47:14.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.916 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.916 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.916 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564774+0000", 2024-09-15T21:47:14.916 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1883e-05 2024-09-15T21:47:14.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.916 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.916 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.916 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564877+0000", 2024-09-15T21:47:14.916 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010254700000000001 2024-09-15T21:47:14.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.917 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.917 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565066+0000", 2024-09-15T21:47:14.917 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018933700000000001 2024-09-15T21:47:14.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.917 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:14.917 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565164+0000", 2024-09-15T21:47:14.917 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7633999999999994e-05 2024-09-15T21:47:14.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.918 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.918 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569030+0000", 2024-09-15T21:47:14.918 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038669070000000002 2024-09-15T21:47:14.918 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.918 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.918 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.918 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571818+0000", 2024-09-15T21:47:14.918 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027878930000000001 2024-09-15T21:47:14.918 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.918 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.918 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.918 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571868+0000", 2024-09-15T21:47:14.918 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9880999999999999e-05 2024-09-15T21:47:14.918 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.919 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.919 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571907+0000", 2024-09-15T21:47:14.919 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8560999999999999e-05 2024-09-15T21:47:14.919 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.919 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.919 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.919 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.919 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:6 2.e 2:75ee5fd8:::629.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.919 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:14.919 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 23.248055773000001, 2024-09-15T21:47:14.919 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073433220000000002, 2024-09-15T21:47:14.919 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.920 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.920 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.920 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.920 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.920 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.920 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:14.920 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.920 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.920 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.920 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:14.920 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.920 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.921 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.921 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564726+0000", 2024-09-15T21:47:14.921 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7399999999999999e-07 2024-09-15T21:47:14.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.921 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.921 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.921 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564728+0000", 2024-09-15T21:47:14.921 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.447e-06 2024-09-15T21:47:14.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.921 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.921 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.921 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565446+0000", 2024-09-15T21:47:14.921 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00071774999999999996 2024-09-15T21:47:14.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.921 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.922 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.922 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565453+0000", 2024-09-15T21:47:14.922 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3239999999999997e-06 2024-09-15T21:47:14.922 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.922 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.922 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.922 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565515+0000", 2024-09-15T21:47:14.922 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1768999999999998e-05 2024-09-15T21:47:14.922 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.922 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.922 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.922 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565591+0000", 2024-09-15T21:47:14.922 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5873999999999999e-05 2024-09-15T21:47:14.922 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.923 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.923 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:14.923 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565709+0000", 2024-09-15T21:47:14.923 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011838899999999999 2024-09-15T21:47:14.923 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.923 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.923 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.923 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569039+0000", 2024-09-15T21:47:14.923 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0033292170000000002 2024-09-15T21:47:14.923 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.923 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.923 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.923 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571977+0000", 2024-09-15T21:47:14.923 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0029387829999999999 2024-09-15T21:47:14.924 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.924 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.924 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.924 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.572032+0000", 2024-09-15T21:47:14.924 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4540999999999998e-05 2024-09-15T21:47:14.924 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.924 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.924 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.924 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.572069+0000", 2024-09-15T21:47:14.924 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6854000000000003e-05 2024-09-15T21:47:14.924 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.924 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.924 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.924 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.924 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.925 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:2 2.1 2:82971f97:::404.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:14.925 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:14.925 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 23.248051805999999, 2024-09-15T21:47:14.925 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007245357, 2024-09-15T21:47:14.925 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:14.925 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:14.925 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:14.925 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:14.925 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.925 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:14.925 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:14.925 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.926 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.926 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:14.926 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:14.926 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:14.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.926 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.926 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:14.926 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564730+0000", 2024-09-15T21:47:14.926 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2999999999999999e-07 2024-09-15T21:47:14.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.926 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.926 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:14.926 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564731+0000", 2024-09-15T21:47:14.926 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.367e-06 2024-09-15T21:47:14.927 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.927 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.927 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:14.927 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565484+0000", 2024-09-15T21:47:14.927 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075311599999999996 2024-09-15T21:47:14.927 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.927 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.927 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:14.927 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565490+0000", 2024-09-15T21:47:14.927 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8100000000000003e-06 2024-09-15T21:47:14.927 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.927 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.927 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:14.927 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565828+0000", 2024-09-15T21:47:14.927 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000337836 2024-09-15T21:47:14.928 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.928 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.928 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:14.928 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565964+0000", 2024-09-15T21:47:14.928 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013641599999999999 2024-09-15T21:47:14.928 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.928 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.928 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:14.928 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566071+0000", 2024-09-15T21:47:14.928 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000107126 2024-09-15T21:47:14.928 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.928 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.928 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:14.928 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571361+0000", 2024-09-15T21:47:14.929 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052896210000000004 2024-09-15T21:47:14.929 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.929 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.929 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:14.929 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571874+0000", 2024-09-15T21:47:14.929 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051320599999999999 2024-09-15T21:47:14.929 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.929 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.929 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:14.929 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571936+0000", 2024-09-15T21:47:14.929 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1650999999999996e-05 2024-09-15T21:47:14.929 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:14.929 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:14.929 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:14.930 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571975+0000", 2024-09-15T21:47:14.930 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8977999999999998e-05 2024-09-15T21:47:14.930 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.930 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.930 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.930 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:14.930 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:14.930 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:14.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.814+0000 7f56bf2e6640 1 -- 172.21.15.100:0/2734004719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56b80b3940 con 0x7f56b80cb7d0 2024-09-15T21:47:14.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.814+0000 7f56adffb640 1 -- 172.21.15.100:0/2734004719 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f56a801a070 con 0x7f56b80cb7d0 2024-09-15T21:47:14.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.815+0000 7f56adffb640 1 --2- 172.21.15.100:0/2734004719 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f569c06c2b0 0x7f569c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:14.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.815+0000 7f56adffb640 1 -- 172.21.15.100:0/2734004719 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 5854+0+0 (secure 0 0 0) 0x7f56a808b080 con 0x7f56b80cb7d0 2024-09-15T21:47:14.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.815+0000 7f56bd082640 1 --2- 172.21.15.100:0/2734004719 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f569c06c2b0 0x7f569c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:14.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.815+0000 7f56bd082640 1 --2- 172.21.15.100:0/2734004719 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f569c06c2b0 0x7f569c06e770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f56b4006fd0 tx=0x7f56b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:14.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.819+0000 7f56adffb640 1 -- 172.21.15.100:0/2734004719 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f56a8059760 con 0x7f56b80cb7d0 2024-09-15T21:47:14.932 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:47:14.934 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:14.932+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:14.999 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:14.997+0000 7f56bf2e6640 1 -- 172.21.15.100:0/2734004719 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f56b815e4a0 con 0x7f56b80cb7d0 2024-09-15T21:47:15.034 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:15.033+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:15.035 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:15.036 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:15.036 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:15.036 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:15.044 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:47:15.135 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:15.134+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:15.188 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:15.188 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:15.188 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:15.188 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:15.189 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:15.197 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:47:15.236 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:47:15.235+0000 7f74313af640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 37847) UID: 0 2024-09-15T21:47:15.301 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:15.302 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:15.302 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:15.302 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:15.302 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.302 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:15.302 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:47:15.302 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.885412180000003, 2024-09-15T21:47:15.302 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047486860000000002, 2024-09-15T21:47:15.302 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.302 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.303 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:15.303 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:47:15.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:47:15.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.304 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.304 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413816+0000", 2024-09-15T21:47:15.304 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4810000000000003e-06 2024-09-15T21:47:15.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.304 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.304 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.304 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413827+0000", 2024-09-15T21:47:15.304 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.187e-05 2024-09-15T21:47:15.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.304 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.304 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.304 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413862+0000", 2024-09-15T21:47:15.304 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4041000000000002e-05 2024-09-15T21:47:15.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.305 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.305 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413871+0000", 2024-09-15T21:47:15.305 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8919999999999995e-06 2024-09-15T21:47:15.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.305 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.305 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413997+0000", 2024-09-15T21:47:15.305 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012601299999999999 2024-09-15T21:47:15.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.305 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:15.305 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414100+0000", 2024-09-15T21:47:15.306 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102555 2024-09-15T21:47:15.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.306 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.306 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418558+0000", 2024-09-15T21:47:15.306 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044578339999999999 2024-09-15T21:47:15.306 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.306 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.306 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.306 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:15.306 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:47:15.307 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.881767365999998, 2024-09-15T21:47:15.307 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045423590000000002, 2024-09-15T21:47:15.307 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.307 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.307 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.307 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.307 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:47:15.307 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.307 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:47:15.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.308 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.308 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417455+0000", 2024-09-15T21:47:15.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9100000000000003e-07 2024-09-15T21:47:15.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.308 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.308 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417463+0000", 2024-09-15T21:47:15.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3750000000000003e-06 2024-09-15T21:47:15.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.308 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.309 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417475+0000", 2024-09-15T21:47:15.309 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2473000000000001e-05 2024-09-15T21:47:15.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.309 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.309 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.309 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417482+0000", 2024-09-15T21:47:15.309 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4010000000000002e-06 2024-09-15T21:47:15.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.309 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.310 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.310 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417524+0000", 2024-09-15T21:47:15.310 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1870999999999999e-05 2024-09-15T21:47:15.310 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.310 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.310 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:15.310 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417542+0000", 2024-09-15T21:47:15.310 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8224e-05 2024-09-15T21:47:15.310 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.310 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.310 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:15.310 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421955+0000", 2024-09-15T21:47:15.310 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044130740000000003 2024-09-15T21:47:15.310 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.311 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.311 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.311 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421996+0000", 2024-09-15T21:47:15.311 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.125e-05 2024-09-15T21:47:15.311 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.311 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.311 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.311 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.311 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 7 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:15.311 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:15.311 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.881506561000002, 2024-09-15T21:47:15.311 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043086519999999996, 2024-09-15T21:47:15.311 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.312 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.312 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:47:15.312 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.312 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:15.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.312 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:15.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.313 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:15.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.58e-07 2024-09-15T21:47:15.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.313 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417722+0000", 2024-09-15T21:47:15.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4729999999999997e-06 2024-09-15T21:47:15.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.313 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417737+0000", 2024-09-15T21:47:15.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5155e-05 2024-09-15T21:47:15.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.314 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.314 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417752+0000", 2024-09-15T21:47:15.314 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4863e-05 2024-09-15T21:47:15.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.314 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.314 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.314 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417819+0000", 2024-09-15T21:47:15.314 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7680000000000003e-05 2024-09-15T21:47:15.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.314 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.314 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.314 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422023+0000", 2024-09-15T21:47:15.314 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042040230000000003 2024-09-15T21:47:15.314 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.315 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.315 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.315 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:25 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:15.315 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:47:15.315 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 40.842017531000003, 2024-09-15T21:47:15.315 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070572830000000001, 2024-09-15T21:47:15.315 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.315 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.315 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.315 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.315 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.316 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:47:15.316 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.316 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.316 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.316 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.316 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:47:15.316 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.316 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.316 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.316 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.316 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457205+0000", 2024-09-15T21:47:15.316 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6899999999999999e-06 2024-09-15T21:47:15.316 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457211+0000", 2024-09-15T21:47:15.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.626e-06 2024-09-15T21:47:15.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457244+0000", 2024-09-15T21:47:15.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2691999999999998e-05 2024-09-15T21:47:15.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457251+0000", 2024-09-15T21:47:15.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3839999999999999e-06 2024-09-15T21:47:15.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.318 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.318 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457309+0000", 2024-09-15T21:47:15.318 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8023999999999999e-05 2024-09-15T21:47:15.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.318 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:15.318 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457413+0000", 2024-09-15T21:47:15.318 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010331999999999999 2024-09-15T21:47:15.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.318 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:15.318 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:15.319 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118439 2024-09-15T21:47:15.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.319 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:15.319 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463246+0000", 2024-09-15T21:47:15.319 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057148219999999996 2024-09-15T21:47:15.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.319 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:15.319 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463428+0000", 2024-09-15T21:47:15.319 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018218299999999999 2024-09-15T21:47:15.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.319 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:15.319 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464212+0000", 2024-09-15T21:47:15.320 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00078402799999999998 2024-09-15T21:47:15.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.320 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.320 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.320 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464261+0000", 2024-09-15T21:47:15.320 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9075e-05 2024-09-15T21:47:15.320 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.320 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.320 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.320 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.320 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:28 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:15.320 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:15.321 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 40.842007504000001, 2024-09-15T21:47:15.321 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0072141449999999999, 2024-09-15T21:47:15.321 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.321 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.321 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:15.321 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.321 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.321 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.321 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:15.321 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.321 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.321 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.321 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:15.322 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.322 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.322 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.322 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.322 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:15.322 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:47:15.322 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.322 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.322 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.322 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457217+0000", 2024-09-15T21:47:15.322 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7259999999999998e-06 2024-09-15T21:47:15.322 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.322 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.322 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.323 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457842+0000", 2024-09-15T21:47:15.323 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00062482599999999998 2024-09-15T21:47:15.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.323 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.323 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457848+0000", 2024-09-15T21:47:15.323 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7089999999999996e-06 2024-09-15T21:47:15.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.323 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.323 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457895+0000", 2024-09-15T21:47:15.323 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6317999999999999e-05 2024-09-15T21:47:15.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.324 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:15.324 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458013+0000", 2024-09-15T21:47:15.324 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011848100000000001 2024-09-15T21:47:15.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.324 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.324 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.324 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464428+0000", 2024-09-15T21:47:15.324 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0064146990000000003 2024-09-15T21:47:15.324 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.324 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.324 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.324 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.324 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:29 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:15.325 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:47:15.325 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 40.842003061, 2024-09-15T21:47:15.325 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073658980000000001, 2024-09-15T21:47:15.325 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.325 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.325 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:15.325 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.325 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.325 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:47:15.325 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.326 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.326 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:47:15.326 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.326 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.326 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.326 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457219+0000", 2024-09-15T21:47:15.326 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2000000000000001e-07 2024-09-15T21:47:15.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.326 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.326 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.326 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457226+0000", 2024-09-15T21:47:15.326 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.956e-06 2024-09-15T21:47:15.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.327 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.327 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.327 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458266+0000", 2024-09-15T21:47:15.327 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001040476 2024-09-15T21:47:15.327 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.327 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.327 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.327 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458273+0000", 2024-09-15T21:47:15.327 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0690000000000004e-06 2024-09-15T21:47:15.327 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.327 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.327 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.327 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458319+0000", 2024-09-15T21:47:15.327 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5668999999999999e-05 2024-09-15T21:47:15.327 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.328 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.328 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:15.328 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458415+0000", 2024-09-15T21:47:15.328 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6701999999999997e-05 2024-09-15T21:47:15.328 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.328 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.328 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.328 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464584+0000", 2024-09-15T21:47:15.328 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061687060000000004 2024-09-15T21:47:15.328 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.328 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.328 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.328 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.329 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.329 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.1 epoch 22/19 rep_tid 12 entries 22'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.10:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:15.329 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:47:15.329 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 40.840573055, 2024-09-15T21:47:15.329 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0039881550000000002, 2024-09-15T21:47:15.329 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.329 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.329 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:47:15.329 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.329 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.329 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.329 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:47:15.329 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.330 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.330 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:47:15.330 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.330 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.330 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458649+0000", 2024-09-15T21:47:15.330 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9599999999999999e-07 2024-09-15T21:47:15.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.330 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.331 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458656+0000", 2024-09-15T21:47:15.331 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2629999999999997e-06 2024-09-15T21:47:15.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.331 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.331 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.331 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458670+0000", 2024-09-15T21:47:15.331 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4166e-05 2024-09-15T21:47:15.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.331 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.331 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.331 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458682+0000", 2024-09-15T21:47:15.331 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1865000000000001e-05 2024-09-15T21:47:15.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.331 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.332 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.332 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458749+0000", 2024-09-15T21:47:15.332 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6507000000000005e-05 2024-09-15T21:47:15.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.332 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.332 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.332 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462636+0000", 2024-09-15T21:47:15.332 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038877579999999998 2024-09-15T21:47:15.332 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.332 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.332 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.332 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.332 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.1.10:12 2.12 e22/19 2:48cd7578:::60e.00000000:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:15.333 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:15.333 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 40.840258722999998, 2024-09-15T21:47:15.333 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042479500000000003, 2024-09-15T21:47:15.333 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.333 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.333 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.333 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.333 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.333 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.333 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:15.333 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.333 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.333 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.333 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.334 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:15.334 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.334 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.334 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.334 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:15.334 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2900000000000004e-07 2024-09-15T21:47:15.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.334 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.334 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.334 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458968+0000", 2024-09-15T21:47:15.334 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2039999999999996e-06 2024-09-15T21:47:15.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458977+0000", 2024-09-15T21:47:15.335 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.6179999999999995e-06 2024-09-15T21:47:15.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458983+0000", 2024-09-15T21:47:15.335 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9510000000000003e-06 2024-09-15T21:47:15.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459479+0000", 2024-09-15T21:47:15.336 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000496274 2024-09-15T21:47:15.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.336 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:15.336 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459496+0000", 2024-09-15T21:47:15.336 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6538e-05 2024-09-15T21:47:15.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.336 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:15.336 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463169+0000", 2024-09-15T21:47:15.336 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003672832 2024-09-15T21:47:15.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.336 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.337 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463211+0000", 2024-09-15T21:47:15.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2004e-05 2024-09-15T21:47:15.337 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.337 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.337 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.337 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.337 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.12 epoch 22/19 rep_tid 2 entries 22'3 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.10:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:15.337 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:47:15.337 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 40.839579168999997, 2024-09-15T21:47:15.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035859609999999999, 2024-09-15T21:47:15.337 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.338 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.338 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:47:15.338 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.338 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.338 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.338 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:47:15.338 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.338 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.338 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.338 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.338 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:47:15.338 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.338 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.338 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.339 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.339 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459643+0000", 2024-09-15T21:47:15.339 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2200000000000004e-07 2024-09-15T21:47:15.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.339 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.339 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.339 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459650+0000", 2024-09-15T21:47:15.339 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4479999999999997e-06 2024-09-15T21:47:15.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.339 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.339 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.339 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459665+0000", 2024-09-15T21:47:15.339 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4586999999999999e-05 2024-09-15T21:47:15.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.340 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.340 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459676+0000", 2024-09-15T21:47:15.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1673e-05 2024-09-15T21:47:15.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.340 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.340 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459732+0000", 2024-09-15T21:47:15.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5612999999999998e-05 2024-09-15T21:47:15.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.340 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.340 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463228+0000", 2024-09-15T21:47:15.341 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0034961179999999999 2024-09-15T21:47:15.341 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.341 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.341 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.341 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.341 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.341 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:34 2.1b 2:db457a8a:::201.00000001:head [write 0~588] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:15.341 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:47:15.341 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 40.834367532999998, 2024-09-15T21:47:15.341 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070946129999999996, 2024-09-15T21:47:15.341 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.341 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.341 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.342 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.342 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.342 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.342 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:47:15.342 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.342 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.342 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.342 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.342 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:47:15.342 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.342 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.342 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.342 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.342 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464855+0000", 2024-09-15T21:47:15.343 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5880000000000001e-06 2024-09-15T21:47:15.343 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.343 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.343 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.343 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464863+0000", 2024-09-15T21:47:15.343 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0879999999999995e-06 2024-09-15T21:47:15.343 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.343 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.343 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.343 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464889+0000", 2024-09-15T21:47:15.343 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5729e-05 2024-09-15T21:47:15.343 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.343 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.344 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.344 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464898+0000", 2024-09-15T21:47:15.344 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0389999999999993e-06 2024-09-15T21:47:15.344 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.344 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.344 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.344 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464962+0000", 2024-09-15T21:47:15.344 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3567999999999999e-05 2024-09-15T21:47:15.344 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.344 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.344 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:15.344 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.465041+0000", 2024-09-15T21:47:15.344 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8889999999999999e-05 2024-09-15T21:47:15.345 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.345 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.345 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:15.345 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.465173+0000", 2024-09-15T21:47:15.345 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013226899999999999 2024-09-15T21:47:15.345 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.345 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.345 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:15.345 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.466319+0000", 2024-09-15T21:47:15.345 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011463039999999999 2024-09-15T21:47:15.345 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.345 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.345 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:15.345 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471825+0000", 2024-09-15T21:47:15.346 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055062160000000004 2024-09-15T21:47:15.346 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.346 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.346 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:15.346 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471886+0000", 2024-09-15T21:47:15.346 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0503e-05 2024-09-15T21:47:15.346 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.346 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.346 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.346 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471948+0000", 2024-09-15T21:47:15.346 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2419e-05 2024-09-15T21:47:15.346 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.346 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.347 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.347 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.347 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.347 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:15.347 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:47:15.347 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 34.823709368999999, 2024-09-15T21:47:15.347 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047360500000000003, 2024-09-15T21:47:15.347 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.347 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.347 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.347 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.347 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.347 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.348 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:47:15.348 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.348 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.348 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.348 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.348 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:47:15.348 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.348 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.348 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.348 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.348 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475514+0000", 2024-09-15T21:47:15.348 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0990000000000001e-06 2024-09-15T21:47:15.348 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.349 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.349 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.349 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475524+0000", 2024-09-15T21:47:15.349 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5850000000000006e-06 2024-09-15T21:47:15.349 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.349 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.349 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.349 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475557+0000", 2024-09-15T21:47:15.349 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3090000000000003e-05 2024-09-15T21:47:15.349 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.349 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.349 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.349 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475568+0000", 2024-09-15T21:47:15.350 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1116e-05 2024-09-15T21:47:15.350 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.350 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.350 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.350 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475649+0000", 2024-09-15T21:47:15.350 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0815999999999993e-05 2024-09-15T21:47:15.350 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.350 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.350 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:15.350 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475805+0000", 2024-09-15T21:47:15.350 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015667200000000001 2024-09-15T21:47:15.350 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.350 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.350 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:15.351 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475944+0000", 2024-09-15T21:47:15.351 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000138651 2024-09-15T21:47:15.351 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.351 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.351 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:15.351 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476900+0000", 2024-09-15T21:47:15.351 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00095588699999999999 2024-09-15T21:47:15.351 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.351 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.351 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:15.351 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480153+0000", 2024-09-15T21:47:15.351 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0032535200000000002 2024-09-15T21:47:15.351 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.352 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.352 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:15.352 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480203+0000", 2024-09-15T21:47:15.352 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9932999999999998e-05 2024-09-15T21:47:15.352 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.352 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.352 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.352 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480248+0000", 2024-09-15T21:47:15.352 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4681000000000001e-05 2024-09-15T21:47:15.352 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.352 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.352 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.352 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.353 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.353 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:15.353 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:15.353 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 34.823693427999999, 2024-09-15T21:47:15.353 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049206709999999997, 2024-09-15T21:47:15.353 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.353 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.353 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.353 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.353 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.353 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.353 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:15.353 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.353 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.354 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.354 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.354 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:15.354 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.354 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.354 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.354 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.354 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:15.354 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1699999999999999e-07 2024-09-15T21:47:15.354 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.354 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.354 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.354 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475531+0000", 2024-09-15T21:47:15.354 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.05e-06 2024-09-15T21:47:15.355 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.355 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.355 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.355 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475743+0000", 2024-09-15T21:47:15.355 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021138099999999999 2024-09-15T21:47:15.355 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.355 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.355 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.355 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475751+0000", 2024-09-15T21:47:15.355 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7770000000000002e-06 2024-09-15T21:47:15.355 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.355 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.355 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.355 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476272+0000", 2024-09-15T21:47:15.356 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052158399999999996 2024-09-15T21:47:15.356 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.356 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.356 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:15.356 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476348+0000", 2024-09-15T21:47:15.356 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5963000000000004e-05 2024-09-15T21:47:15.356 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.356 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.356 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:15.356 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476474+0000", 2024-09-15T21:47:15.356 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012638800000000001 2024-09-15T21:47:15.356 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.356 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.357 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:15.357 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478194+0000", 2024-09-15T21:47:15.357 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001719768 2024-09-15T21:47:15.357 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.357 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.357 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:15.357 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480358+0000", 2024-09-15T21:47:15.357 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021637340000000001 2024-09-15T21:47:15.357 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.357 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.357 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:15.357 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480411+0000", 2024-09-15T21:47:15.358 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2720999999999999e-05 2024-09-15T21:47:15.358 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.358 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.358 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.358 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480449+0000", 2024-09-15T21:47:15.358 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7888e-05 2024-09-15T21:47:15.358 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.358 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.358 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.358 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.358 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.358 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:15.358 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:15.359 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 34.823688638999997, 2024-09-15T21:47:15.359 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005215478, 2024-09-15T21:47:15.359 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.359 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.359 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:15.359 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.359 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.359 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.359 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:15.359 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.359 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.359 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.359 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.359 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:15.360 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.360 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.360 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.360 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.360 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:15.360 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5199999999999998e-07 2024-09-15T21:47:15.360 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.360 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.360 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.360 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475535+0000", 2024-09-15T21:47:15.360 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9520000000000001e-06 2024-09-15T21:47:15.360 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.361 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.361 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.361 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475785+0000", 2024-09-15T21:47:15.361 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024966099999999998 2024-09-15T21:47:15.361 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.361 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.361 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.361 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475799+0000", 2024-09-15T21:47:15.361 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4649e-05 2024-09-15T21:47:15.361 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.361 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.361 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.361 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475856+0000", 2024-09-15T21:47:15.361 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6956000000000002e-05 2024-09-15T21:47:15.361 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.362 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.362 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:15.362 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476016+0000", 2024-09-15T21:47:15.362 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000159457 2024-09-15T21:47:15.362 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.362 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.362 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.362 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480748+0000", 2024-09-15T21:47:15.362 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047324510000000004 2024-09-15T21:47:15.362 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.362 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.362 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.362 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.362 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.363 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.2.12:1 2.1 e22/19 2:85bbe569:::202.00000000:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:15.363 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:47:15.363 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 34.820886168000001, 2024-09-15T21:47:15.363 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045423649999999996, 2024-09-15T21:47:15.363 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.363 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.363 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.363 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.363 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.363 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.363 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:47:15.363 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.363 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.364 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.364 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.364 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:47:15.364 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.364 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.364 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.364 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.364 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478336+0000", 2024-09-15T21:47:15.364 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4499999999999996e-07 2024-09-15T21:47:15.364 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.364 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.364 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.365 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478342+0000", 2024-09-15T21:47:15.365 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5579999999999997e-06 2024-09-15T21:47:15.365 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.365 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.365 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.365 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478356+0000", 2024-09-15T21:47:15.365 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3567000000000001e-05 2024-09-15T21:47:15.365 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.365 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.365 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.365 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478364+0000", 2024-09-15T21:47:15.365 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7889999999999995e-06 2024-09-15T21:47:15.365 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.365 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.365 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.366 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478416+0000", 2024-09-15T21:47:15.366 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2188000000000002e-05 2024-09-15T21:47:15.366 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.366 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.366 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:15.366 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478433+0000", 2024-09-15T21:47:15.366 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6674e-05 2024-09-15T21:47:15.366 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.366 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.366 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:15.366 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482832+0000", 2024-09-15T21:47:15.366 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043996110000000003 2024-09-15T21:47:15.366 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.366 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.367 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.367 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482878+0000", 2024-09-15T21:47:15.367 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5232999999999999e-05 2024-09-15T21:47:15.367 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.367 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.367 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.367 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.367 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.367 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:25 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:15.367 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:47:15.367 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 23.734026129, 2024-09-15T21:47:15.367 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058459619999999997, 2024-09-15T21:47:15.367 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.368 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.368 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.368 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.368 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.368 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.368 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:47:15.368 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.368 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.368 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.368 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.368 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:47:15.368 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.368 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.368 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.369 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.369 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565197+0000", 2024-09-15T21:47:15.369 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7889999999999999e-06 2024-09-15T21:47:15.369 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.369 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.369 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.369 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565208+0000", 2024-09-15T21:47:15.369 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1358999999999999e-05 2024-09-15T21:47:15.369 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.369 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.369 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.369 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565237+0000", 2024-09-15T21:47:15.370 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8626e-05 2024-09-15T21:47:15.370 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.370 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.370 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.370 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565249+0000", 2024-09-15T21:47:15.370 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1897e-05 2024-09-15T21:47:15.370 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.370 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.370 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.370 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565332+0000", 2024-09-15T21:47:15.370 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2943999999999997e-05 2024-09-15T21:47:15.370 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.370 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.370 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:15.370 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565452+0000", 2024-09-15T21:47:15.371 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119992 2024-09-15T21:47:15.371 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.371 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.371 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:15.371 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565610+0000", 2024-09-15T21:47:15.371 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015845099999999999 2024-09-15T21:47:15.371 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.371 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.371 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:15.371 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.568999+0000", 2024-09-15T21:47:15.371 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0033887190000000001 2024-09-15T21:47:15.371 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.371 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.372 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:15.372 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.570979+0000", 2024-09-15T21:47:15.372 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019804919999999999 2024-09-15T21:47:15.372 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.372 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.372 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:15.372 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571014+0000", 2024-09-15T21:47:15.372 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4331e-05 2024-09-15T21:47:15.372 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.372 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.372 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.372 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571041+0000", 2024-09-15T21:47:15.372 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7362e-05 2024-09-15T21:47:15.373 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.373 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.373 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.373 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.373 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.373 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:5 2.e e22/19 2:75ee5fd8:::629.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:47:15.373 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:47:15.373 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 23.733350482999999, 2024-09-15T21:47:15.373 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057947119999999996, 2024-09-15T21:47:15.373 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.373 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.373 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.373 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.373 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.374 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.374 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:47:15.374 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.374 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.374 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.374 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.374 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:47:15.374 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.374 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.374 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.374 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.374 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565872+0000", 2024-09-15T21:47:15.374 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9800000000000002e-07 2024-09-15T21:47:15.374 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.375 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.375 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.375 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565880+0000", 2024-09-15T21:47:15.375 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7619999999999995e-06 2024-09-15T21:47:15.375 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.375 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.375 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.375 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565890+0000", 2024-09-15T21:47:15.375 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0648999999999999e-05 2024-09-15T21:47:15.375 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.375 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.375 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.375 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565897+0000", 2024-09-15T21:47:15.375 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.153e-06 2024-09-15T21:47:15.376 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.376 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.376 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.376 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565966+0000", 2024-09-15T21:47:15.376 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9046000000000003e-05 2024-09-15T21:47:15.376 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.376 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.376 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:15.376 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565985+0000", 2024-09-15T21:47:15.376 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8695e-05 2024-09-15T21:47:15.376 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.376 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.376 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:15.376 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571625+0000", 2024-09-15T21:47:15.377 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056397139999999997 2024-09-15T21:47:15.377 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.377 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.377 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.377 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571666+0000", 2024-09-15T21:47:15.377 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0695000000000003e-05 2024-09-15T21:47:15.377 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.377 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.377 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.377 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.377 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.377 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:6 2.e e22/19 2:75ee5fd8:::629.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:47:15.377 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:47:15.378 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 23.733288235, 2024-09-15T21:47:15.378 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057607429999999996, 2024-09-15T21:47:15.378 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.378 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.378 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.378 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.378 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.378 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.378 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:47:15.378 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.378 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.378 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.378 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.379 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:47:15.379 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.379 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.379 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.379 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.379 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565934+0000", 2024-09-15T21:47:15.379 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6199999999999998e-07 2024-09-15T21:47:15.379 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.379 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.379 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.379 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565938+0000", 2024-09-15T21:47:15.379 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4970000000000003e-06 2024-09-15T21:47:15.379 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.379 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.380 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.380 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565945+0000", 2024-09-15T21:47:15.380 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5069999999999997e-06 2024-09-15T21:47:15.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.380 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.380 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.380 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565948+0000", 2024-09-15T21:47:15.380 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:47:15.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.380 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.380 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.380 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566200+0000", 2024-09-15T21:47:15.381 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025170200000000002 2024-09-15T21:47:15.381 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.381 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.381 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:15.381 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566218+0000", 2024-09-15T21:47:15.381 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7543000000000001e-05 2024-09-15T21:47:15.381 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.381 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.381 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:15.381 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571668+0000", 2024-09-15T21:47:15.381 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054502999999999999 2024-09-15T21:47:15.381 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.381 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.381 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.382 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571694+0000", 2024-09-15T21:47:15.382 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5979e-05 2024-09-15T21:47:15.382 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.382 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.382 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.382 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.382 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.382 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:2 2.1 e22/19 2:82971f97:::404.00000000:head v 22'5, mlcod=22'3)", 2024-09-15T21:47:15.382 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:47:15.382 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 23.733080299000001, 2024-09-15T21:47:15.382 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056016019999999998, 2024-09-15T21:47:15.382 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.382 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.382 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.382 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.383 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.383 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.383 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:47:15.383 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.383 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.383 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.383 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.383 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:47:15.383 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.383 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.383 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.383 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.383 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566142+0000", 2024-09-15T21:47:15.383 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:47:15.383 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.384 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.384 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.384 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566149+0000", 2024-09-15T21:47:15.384 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7510000000000008e-06 2024-09-15T21:47:15.384 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.384 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.384 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.384 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566161+0000", 2024-09-15T21:47:15.384 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1259e-05 2024-09-15T21:47:15.384 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.384 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.384 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.384 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566168+0000", 2024-09-15T21:47:15.385 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.255e-06 2024-09-15T21:47:15.385 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.385 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.385 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.385 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566227+0000", 2024-09-15T21:47:15.385 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8961e-05 2024-09-15T21:47:15.385 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.385 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.385 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:15.385 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566247+0000", 2024-09-15T21:47:15.385 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0536000000000001e-05 2024-09-15T21:47:15.385 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.385 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.386 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:15.386 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571694+0000", 2024-09-15T21:47:15.386 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005446762 2024-09-15T21:47:15.386 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.386 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.386 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.386 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571743+0000", 2024-09-15T21:47:15.386 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8396999999999998e-05 2024-09-15T21:47:15.386 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.386 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.386 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.386 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.386 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.386 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:17 2.14 e22/19 2:2eebc8f4:::62f.00000000:head v 22'9, mlcod=22'7)", 2024-09-15T21:47:15.387 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:47:15.387 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 23.733064202000001, 2024-09-15T21:47:15.387 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056532860000000004, 2024-09-15T21:47:15.387 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.387 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.387 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.387 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.387 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.387 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.387 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:47:15.387 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.387 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.387 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.388 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.388 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:47:15.388 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.388 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.388 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.388 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.388 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566158+0000", 2024-09-15T21:47:15.388 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.92e-07 2024-09-15T21:47:15.388 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.388 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.388 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.388 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566172+0000", 2024-09-15T21:47:15.389 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3976e-05 2024-09-15T21:47:15.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.389 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.389 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.389 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566407+0000", 2024-09-15T21:47:15.389 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023483900000000001 2024-09-15T21:47:15.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.389 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.389 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.389 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566416+0000", 2024-09-15T21:47:15.389 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0780000000000002e-06 2024-09-15T21:47:15.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.389 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.390 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.390 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566476+0000", 2024-09-15T21:47:15.390 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9908999999999998e-05 2024-09-15T21:47:15.390 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.390 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.390 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:15.390 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566493+0000", 2024-09-15T21:47:15.390 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7567e-05 2024-09-15T21:47:15.390 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.390 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.390 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:15.390 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571762+0000", 2024-09-15T21:47:15.390 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052682959999999996 2024-09-15T21:47:15.390 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.391 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.391 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.391 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571810+0000", 2024-09-15T21:47:15.391 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8829000000000003e-05 2024-09-15T21:47:15.391 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.391 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.391 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.391 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.391 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.391 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:18 2.14 e22/19 2:2eebc8f4:::62f.00000000:head v 22'10, mlcod=22'7)", 2024-09-15T21:47:15.391 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:47:15.391 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 23.732775127, 2024-09-15T21:47:15.391 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053992579999999997, 2024-09-15T21:47:15.391 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:15.392 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:15.392 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.392 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:15.392 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.392 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:15.392 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:47:15.392 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.392 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.392 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:15.392 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:47:15.392 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:15.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.393 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.393 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:15.393 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566447+0000", 2024-09-15T21:47:15.393 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2e-07 2024-09-15T21:47:15.393 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.393 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.393 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:15.393 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566452+0000", 2024-09-15T21:47:15.393 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0390000000000004e-06 2024-09-15T21:47:15.393 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.393 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.393 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:15.393 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566460+0000", 2024-09-15T21:47:15.394 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1389999999999995e-06 2024-09-15T21:47:15.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.394 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.394 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566473+0000", 2024-09-15T21:47:15.394 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3093e-05 2024-09-15T21:47:15.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.394 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:15.394 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566688+0000", 2024-09-15T21:47:15.394 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021537100000000001 2024-09-15T21:47:15.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.395 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:15.395 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566706+0000", 2024-09-15T21:47:15.395 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7802999999999999e-05 2024-09-15T21:47:15.395 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.395 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.395 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:15.395 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571813+0000", 2024-09-15T21:47:15.395 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051065140000000004 2024-09-15T21:47:15.395 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:15.395 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:15.395 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:15.395 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571845+0000", 2024-09-15T21:47:15.395 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2879000000000001e-05 2024-09-15T21:47:15.395 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.396 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.396 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.396 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:15.396 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:15.396 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:15.397 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:47:15.398 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:15.396+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:15.498 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:15.497+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:15.518 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:15.518 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:15.518 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:15.519 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:15.528 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:47:15.600 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:15.598+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:15.634 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:15.634 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:15.634 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:15.634 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:15.634 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:15.643 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:47:15.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:15.656+0000 7f56adffb640 1 -- 172.21.15.100:0/2734004719 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v29) ==== 87+0+0 (secure 0 0 0) 0x7f56a8059060 con 0x7f56b80cb7d0 2024-09-15T21:47:15.658 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is set 2024-09-15T21:47:15.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:15.658+0000 7f56bf2e6640 1 -- 172.21.15.100:0/2734004719 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f569c06c2b0 msgr2=0x7f569c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:15.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:15.658+0000 7f56bf2e6640 1 --2- 172.21.15.100:0/2734004719 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f569c06c2b0 0x7f569c06e770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f56b4006fd0 tx=0x7f56b4008040 comp rx=0 tx=0).stop 2024-09-15T21:47:15.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:15.658+0000 7f56bf2e6640 1 -- 172.21.15.100:0/2734004719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56b80cb7d0 msgr2=0x7f56b81584a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:15.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:15.658+0000 7f56bf2e6640 1 --2- 172.21.15.100:0/2734004719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56b80cb7d0 0x7f56b81584a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f56a800e810 tx=0x7f56a800ecd0 comp rx=0 tx=0).stop 2024-09-15T21:47:15.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:15.659+0000 7f56bf2e6640 1 -- 172.21.15.100:0/2734004719 shutdown_connections 2024-09-15T21:47:15.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:15.659+0000 7f56bf2e6640 1 --2- 172.21.15.100:0/2734004719 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f569c06c2b0 0x7f569c06e770 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:15.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:15.659+0000 7f56bf2e6640 1 --2- 172.21.15.100:0/2734004719 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56b8155440 0x7f56b8153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:15.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:15.659+0000 7f56bf2e6640 1 --2- 172.21.15.100:0/2734004719 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56b80cb7d0 0x7f56b81584a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:15.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:15.659+0000 7f56bf2e6640 1 --2- 172.21.15.100:0/2734004719 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56b806d0e0 0x7f56b81559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:15.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:15.659+0000 7f56bf2e6640 1 -- 172.21.15.100:0/2734004719 >> 172.21.15.100:0/2734004719 conn(0x7f56b80680d0 msgr2=0x7f56b806d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:15.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:15.659+0000 7f56bf2e6640 1 -- 172.21.15.100:0/2734004719 shutdown_connections 2024-09-15T21:47:15.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:15.659+0000 7f56bf2e6640 1 -- 172.21.15.100:0/2734004719 wait complete. 2024-09-15T21:47:15.715 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:15.713+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:15.750 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:15.750 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:15.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:15.750 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:15.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.750 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:15.750 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:47:15.751 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 47.333770461, 2024-09-15T21:47:15.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061533810000000003, 2024-09-15T21:47:15.751 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.751 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.751 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.751 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:47:15.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:47:15.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413085+0000", 2024-09-15T21:47:15.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8390000000000003e-06 2024-09-15T21:47:15.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413092+0000", 2024-09-15T21:47:15.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3479999999999998e-06 2024-09-15T21:47:15.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413109+0000", 2024-09-15T21:47:15.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7192000000000001e-05 2024-09-15T21:47:15.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413118+0000", 2024-09-15T21:47:15.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.935e-06 2024-09-15T21:47:15.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413219+0000", 2024-09-15T21:47:15.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010121 2024-09-15T21:47:15.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:15.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413381+0000", 2024-09-15T21:47:15.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016148399999999999 2024-09-15T21:47:15.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:15.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413525+0000", 2024-09-15T21:47:15.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014428100000000001 2024-09-15T21:47:15.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:15.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415866+0000", 2024-09-15T21:47:15.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023409300000000002 2024-09-15T21:47:15.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:15.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419156+0000", 2024-09-15T21:47:15.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032903580000000002 2024-09-15T21:47:15.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:15.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419199+0000", 2024-09-15T21:47:15.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2171e-05 2024-09-15T21:47:15.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419232+0000", 2024-09-15T21:47:15.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3633000000000001e-05 2024-09-15T21:47:15.756 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.756 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.756 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.756 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.4:28 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-15T21:47:15.757 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:47:15.757 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 47.332506952000003, 2024-09-15T21:47:15.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002614594, 2024-09-15T21:47:15.757 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.757 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.757 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.757 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:47:15.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:47:15.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414344+0000", 2024-09-15T21:47:15.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6589999999999999e-06 2024-09-15T21:47:15.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414355+0000", 2024-09-15T21:47:15.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0604999999999999e-05 2024-09-15T21:47:15.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414370+0000", 2024-09-15T21:47:15.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-15T21:47:15.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414379+0000", 2024-09-15T21:47:15.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2449999999999998e-06 2024-09-15T21:47:15.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414477+0000", 2024-09-15T21:47:15.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8968000000000005e-05 2024-09-15T21:47:15.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:15.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414504+0000", 2024-09-15T21:47:15.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6899000000000001e-05 2024-09-15T21:47:15.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:15.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416837+0000", 2024-09-15T21:47:15.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023327130000000001 2024-09-15T21:47:15.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416957+0000", 2024-09-15T21:47:15.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000119796 2024-09-15T21:47:15.762 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.762 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.762 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.762 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.1.10:8 2.b e22/19 2:d70a01c6:::60c.00000000:head v 22'4, mlcod=22'1)", 2024-09-15T21:47:15.762 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:47:15.762 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.2889275, 2024-09-15T21:47:15.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027633890000000002, 2024-09-15T21:47:15.762 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.762 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.762 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.762 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:47:15.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:47:15.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457923+0000", 2024-09-15T21:47:15.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.136e-06 2024-09-15T21:47:15.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457931+0000", 2024-09-15T21:47:15.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1529999999999997e-06 2024-09-15T21:47:15.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457946+0000", 2024-09-15T21:47:15.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4482e-05 2024-09-15T21:47:15.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457956+0000", 2024-09-15T21:47:15.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0774999999999999e-05 2024-09-15T21:47:15.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458016+0000", 2024-09-15T21:47:15.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9247999999999999e-05 2024-09-15T21:47:15.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:15.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458036+0000", 2024-09-15T21:47:15.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0920999999999998e-05 2024-09-15T21:47:15.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:15.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460636+0000", 2024-09-15T21:47:15.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025992089999999999 2024-09-15T21:47:15.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460685+0000", 2024-09-15T21:47:15.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9465000000000002e-05 2024-09-15T21:47:15.766 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.766 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.766 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.767 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.18 epoch 22/19 rep_tid 5 entries 22'1 (0'0) error 2:1d00e16c:::201.00000005:head by mds.1.10:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:15.767 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:47:15.767 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 41.287235369000001, 2024-09-15T21:47:15.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003509599, 2024-09-15T21:47:15.767 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.767 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.767 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:15.767 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:47:15.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:47:15.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459615+0000", 2024-09-15T21:47:15.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4200000000000005e-07 2024-09-15T21:47:15.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459626+0000", 2024-09-15T21:47:15.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1456e-05 2024-09-15T21:47:15.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459650+0000", 2024-09-15T21:47:15.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.429e-05 2024-09-15T21:47:15.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459664+0000", 2024-09-15T21:47:15.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3805000000000001e-05 2024-09-15T21:47:15.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459784+0000", 2024-09-15T21:47:15.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000119614 2024-09-15T21:47:15.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463123+0000", 2024-09-15T21:47:15.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033395920000000002 2024-09-15T21:47:15.771 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.771 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.771 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.771 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:7 2.13 e22/19 2:cdf3ac81:::616.00000000:head v 22'5, mlcod=22'3)", 2024-09-15T21:47:15.771 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:47:15.771 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.269454590999999, 2024-09-15T21:47:15.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002138615, 2024-09-15T21:47:15.771 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.771 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.771 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.771 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:47:15.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:47:15.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477396+0000", 2024-09-15T21:47:15.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4190000000000001e-06 2024-09-15T21:47:15.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477405+0000", 2024-09-15T21:47:15.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2320000000000007e-06 2024-09-15T21:47:15.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477419+0000", 2024-09-15T21:47:15.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3589e-05 2024-09-15T21:47:15.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477428+0000", 2024-09-15T21:47:15.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4779999999999997e-06 2024-09-15T21:47:15.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477493+0000", 2024-09-15T21:47:15.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4546000000000002e-05 2024-09-15T21:47:15.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:15.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477511+0000", 2024-09-15T21:47:15.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7793000000000001e-05 2024-09-15T21:47:15.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:15.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479481+0000", 2024-09-15T21:47:15.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019701660000000002 2024-09-15T21:47:15.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479533+0000", 2024-09-15T21:47:15.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2392000000000003e-05 2024-09-15T21:47:15.775 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.775 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.775 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.775 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:26 2.11 e22/19 2:893cd530:::mds2_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-15T21:47:15.776 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:15.776 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.269307664999999, 2024-09-15T21:47:15.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020450450000000001, 2024-09-15T21:47:15.776 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.776 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.776 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.776 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:15.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:15.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:15.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9699999999999995e-07 2024-09-15T21:47:15.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477555+0000", 2024-09-15T21:47:15.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3144e-05 2024-09-15T21:47:15.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477565+0000", 2024-09-15T21:47:15.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8579999999999995e-06 2024-09-15T21:47:15.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477573+0000", 2024-09-15T21:47:15.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4869999999999996e-06 2024-09-15T21:47:15.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477632+0000", 2024-09-15T21:47:15.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9178999999999997e-05 2024-09-15T21:47:15.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:15.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477654+0000", 2024-09-15T21:47:15.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2234000000000001e-05 2024-09-15T21:47:15.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:15.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479526+0000", 2024-09-15T21:47:15.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018717600000000001 2024-09-15T21:47:15.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479587+0000", 2024-09-15T21:47:15.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0686000000000001e-05 2024-09-15T21:47:15.781 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.781 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.781 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.781 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:22 2.1a e22/19 2:5fd68f60:::61d.00000000:head v 22'3, mlcod=0'0)", 2024-09-15T21:47:15.781 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:15.781 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.268947609999998, 2024-09-15T21:47:15.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023694380000000002, 2024-09-15T21:47:15.781 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.781 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.782 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.782 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:15.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:15.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:15.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4899999999999995e-07 2024-09-15T21:47:15.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477910+0000", 2024-09-15T21:47:15.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:47:15.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477920+0000", 2024-09-15T21:47:15.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9250000000000007e-06 2024-09-15T21:47:15.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477927+0000", 2024-09-15T21:47:15.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0940000000000002e-06 2024-09-15T21:47:15.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477985+0000", 2024-09-15T21:47:15.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8451e-05 2024-09-15T21:47:15.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:15.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478005+0000", 2024-09-15T21:47:15.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0064e-05 2024-09-15T21:47:15.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:15.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480223+0000", 2024-09-15T21:47:15.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022177910000000002 2024-09-15T21:47:15.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480271+0000", 2024-09-15T21:47:15.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8164999999999997e-05 2024-09-15T21:47:15.786 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.786 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.786 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.786 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:8 2.13 e22/19 2:cdf3ac81:::616.00000000:head v 22'6, mlcod=22'3)", 2024-09-15T21:47:15.786 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:47:15.786 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.268748651000003, 2024-09-15T21:47:15.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021760249999999998, 2024-09-15T21:47:15.786 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.787 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.787 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.787 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:47:15.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:47:15.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478102+0000", 2024-09-15T21:47:15.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3099999999999996e-07 2024-09-15T21:47:15.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478124+0000", 2024-09-15T21:47:15.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2790999999999999e-05 2024-09-15T21:47:15.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478136+0000", 2024-09-15T21:47:15.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1922999999999999e-05 2024-09-15T21:47:15.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478144+0000", 2024-09-15T21:47:15.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9300000000000003e-06 2024-09-15T21:47:15.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478192+0000", 2024-09-15T21:47:15.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8155999999999998e-05 2024-09-15T21:47:15.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:15.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478209+0000", 2024-09-15T21:47:15.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6816e-05 2024-09-15T21:47:15.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:15.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480243+0000", 2024-09-15T21:47:15.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020336199999999999 2024-09-15T21:47:15.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480277+0000", 2024-09-15T21:47:15.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3858e-05 2024-09-15T21:47:15.790 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.790 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.791 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.791 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.11 epoch 22/19 rep_tid 19 entries 22'2 (0'0) error 2:8975f766:::202.00000001:head by mds.2.12:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:15.791 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:15.791 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 35.268622712999999, 2024-09-15T21:47:15.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023882299999999999, 2024-09-15T21:47:15.791 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.791 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.791 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:15.791 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:15.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:15.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:15.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2399999999999998e-07 2024-09-15T21:47:15.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478236+0000", 2024-09-15T21:47:15.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5299999999999996e-06 2024-09-15T21:47:15.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478248+0000", 2024-09-15T21:47:15.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2806000000000001e-05 2024-09-15T21:47:15.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478264+0000", 2024-09-15T21:47:15.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5468000000000002e-05 2024-09-15T21:47:15.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478339+0000", 2024-09-15T21:47:15.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4851000000000006e-05 2024-09-15T21:47:15.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480615+0000", 2024-09-15T21:47:15.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022760509999999999 2024-09-15T21:47:15.795 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.795 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.795 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.795 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:3 2.1a e22/19 2:5fdc322d:::61e.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:47:15.795 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:47:15.795 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.200438111, 2024-09-15T21:47:15.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023961220000000001, 2024-09-15T21:47:15.795 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.795 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.795 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.795 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:47:15.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:47:15.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546413+0000", 2024-09-15T21:47:15.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.812e-06 2024-09-15T21:47:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546424+0000", 2024-09-15T21:47:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1313e-05 2024-09-15T21:47:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546444+0000", 2024-09-15T21:47:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9706000000000001e-05 2024-09-15T21:47:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546453+0000", 2024-09-15T21:47:15.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:47:15.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546519+0000", 2024-09-15T21:47:15.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5710999999999995e-05 2024-09-15T21:47:15.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:15.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546539+0000", 2024-09-15T21:47:15.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0275000000000002e-05 2024-09-15T21:47:15.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:15.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548763+0000", 2024-09-15T21:47:15.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002223226 2024-09-15T21:47:15.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548807+0000", 2024-09-15T21:47:15.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4598000000000002e-05 2024-09-15T21:47:15.799 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.799 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.799 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.799 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:19 2.11 e22/19 2:890f50f7:::626.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:47:15.800 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:47:15.800 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.200140951000002, 2024-09-15T21:47:15.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022671280000000002, 2024-09-15T21:47:15.800 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.800 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.800 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.800 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:47:15.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:47:15.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546709+0000", 2024-09-15T21:47:15.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4499999999999997e-07 2024-09-15T21:47:15.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546723+0000", 2024-09-15T21:47:15.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4623999999999999e-05 2024-09-15T21:47:15.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546734+0000", 2024-09-15T21:47:15.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0331e-05 2024-09-15T21:47:15.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546741+0000", 2024-09-15T21:47:15.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.075e-06 2024-09-15T21:47:15.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546954+0000", 2024-09-15T21:47:15.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021275600000000001 2024-09-15T21:47:15.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:15.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546971+0000", 2024-09-15T21:47:15.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7503000000000002e-05 2024-09-15T21:47:15.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:15.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548926+0000", 2024-09-15T21:47:15.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019550129999999998 2024-09-15T21:47:15.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548975+0000", 2024-09-15T21:47:15.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9280999999999998e-05 2024-09-15T21:47:15.804 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.804 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.804 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.804 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.d epoch 22/19 rep_tid 18 entries 22'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.14:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:15.804 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:47:15.804 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.200134600999998, 2024-09-15T21:47:15.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021738489999999998, 2024-09-15T21:47:15.804 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.804 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.805 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:15.805 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:47:15.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:47:15.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546716+0000", 2024-09-15T21:47:15.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.615e-06 2024-09-15T21:47:15.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546730+0000", 2024-09-15T21:47:15.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3519999999999999e-05 2024-09-15T21:47:15.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546753+0000", 2024-09-15T21:47:15.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3643999999999999e-05 2024-09-15T21:47:15.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546770+0000", 2024-09-15T21:47:15.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6518999999999999e-05 2024-09-15T21:47:15.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546900+0000", 2024-09-15T21:47:15.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013049099999999999 2024-09-15T21:47:15.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548888+0000", 2024-09-15T21:47:15.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019880599999999998 2024-09-15T21:47:15.808 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.808 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.808 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.808 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:4 2.1a e22/19 2:5fdc322d:::61e.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:47:15.808 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:47:15.808 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.200025953000001, 2024-09-15T21:47:15.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002968769, 2024-09-15T21:47:15.808 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.808 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.808 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.808 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:47:15.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:47:15.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546824+0000", 2024-09-15T21:47:15.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6499999999999999e-07 2024-09-15T21:47:15.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546831+0000", 2024-09-15T21:47:15.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9940000000000003e-06 2024-09-15T21:47:15.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546840+0000", 2024-09-15T21:47:15.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3689999999999996e-06 2024-09-15T21:47:15.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546847+0000", 2024-09-15T21:47:15.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0589999999999997e-06 2024-09-15T21:47:15.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547709+0000", 2024-09-15T21:47:15.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00086144699999999999 2024-09-15T21:47:15.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:15.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547730+0000", 2024-09-15T21:47:15.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1542000000000001e-05 2024-09-15T21:47:15.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:15.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549750+0000", 2024-09-15T21:47:15.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002020165 2024-09-15T21:47:15.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549792+0000", 2024-09-15T21:47:15.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1628000000000002e-05 2024-09-15T21:47:15.812 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.812 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.813 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.813 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:20 2.11 e22/19 2:890f50f7:::626.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:47:15.813 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:15.813 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.199001344999999, 2024-09-15T21:47:15.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017734579999999999, 2024-09-15T21:47:15.813 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.813 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.813 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.813 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:15.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:15.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:15.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:47:15.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547856+0000", 2024-09-15T21:47:15.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7999999999999999e-06 2024-09-15T21:47:15.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547867+0000", 2024-09-15T21:47:15.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0971999999999999e-05 2024-09-15T21:47:15.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547881+0000", 2024-09-15T21:47:15.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4168999999999999e-05 2024-09-15T21:47:15.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547935+0000", 2024-09-15T21:47:15.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3721999999999998e-05 2024-09-15T21:47:15.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:15.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547976+0000", 2024-09-15T21:47:15.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1248999999999998e-05 2024-09-15T21:47:15.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:15.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549559+0000", 2024-09-15T21:47:15.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001582815 2024-09-15T21:47:15.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549621+0000", 2024-09-15T21:47:15.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2156999999999995e-05 2024-09-15T21:47:15.817 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.817 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.817 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.817 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:13 2.1a e22/19 2:594e21d0:::623.00000000:head v 22'6, mlcod=22'2)", 2024-09-15T21:47:15.817 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:47:15.817 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.198936349, 2024-09-15T21:47:15.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019085300000000001, 2024-09-15T21:47:15.818 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.818 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.818 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.818 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:47:15.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:47:15.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547914+0000", 2024-09-15T21:47:15.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1500000000000004e-07 2024-09-15T21:47:15.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547920+0000", 2024-09-15T21:47:15.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5239999999999997e-06 2024-09-15T21:47:15.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547930+0000", 2024-09-15T21:47:15.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5109999999999992e-06 2024-09-15T21:47:15.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547937+0000", 2024-09-15T21:47:15.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5410000000000003e-06 2024-09-15T21:47:15.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548518+0000", 2024-09-15T21:47:15.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00058074299999999999 2024-09-15T21:47:15.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:15.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548538+0000", 2024-09-15T21:47:15.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9959e-05 2024-09-15T21:47:15.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:15.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549795+0000", 2024-09-15T21:47:15.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012570120000000001 2024-09-15T21:47:15.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549821+0000", 2024-09-15T21:47:15.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6625e-05 2024-09-15T21:47:15.821 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.822 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.822 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.822 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:23 2.18 e22/19 2:1c1db442:::103.00000000.inode:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:15.822 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:47:15.822 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.198874930999999, 2024-09-15T21:47:15.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002713198, 2024-09-15T21:47:15.822 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.822 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.822 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.822 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:47:15.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:47:15.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547975+0000", 2024-09-15T21:47:15.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3900000000000004e-07 2024-09-15T21:47:15.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547983+0000", 2024-09-15T21:47:15.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6690000000000006e-06 2024-09-15T21:47:15.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547994+0000", 2024-09-15T21:47:15.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1503e-05 2024-09-15T21:47:15.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548002+0000", 2024-09-15T21:47:15.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1340000000000004e-06 2024-09-15T21:47:15.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548098+0000", 2024-09-15T21:47:15.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5791000000000005e-05 2024-09-15T21:47:15.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:15.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548118+0000", 2024-09-15T21:47:15.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0129e-05 2024-09-15T21:47:15.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:15.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550643+0000", 2024-09-15T21:47:15.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002524728 2024-09-15T21:47:15.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550688+0000", 2024-09-15T21:47:15.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4404999999999999e-05 2024-09-15T21:47:15.826 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.826 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.826 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.826 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:14 2.1a e22/19 2:594e21d0:::623.00000000:head v 22'7, mlcod=22'2)", 2024-09-15T21:47:15.827 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:47:15.827 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.198859971000001, 2024-09-15T21:47:15.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024358579999999999, 2024-09-15T21:47:15.827 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.827 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.827 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.827 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:47:15.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:47:15.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547990+0000", 2024-09-15T21:47:15.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5799999999999999e-07 2024-09-15T21:47:15.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548011+0000", 2024-09-15T21:47:15.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0942999999999999e-05 2024-09-15T21:47:15.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548020+0000", 2024-09-15T21:47:15.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2380000000000003e-06 2024-09-15T21:47:15.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548027+0000", 2024-09-15T21:47:15.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6429999999999998e-06 2024-09-15T21:47:15.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548848+0000", 2024-09-15T21:47:15.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082167999999999998 2024-09-15T21:47:15.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:15.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548868+0000", 2024-09-15T21:47:15.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9791000000000001e-05 2024-09-15T21:47:15.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:15.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550387+0000", 2024-09-15T21:47:15.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001519322 2024-09-15T21:47:15.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550425+0000", 2024-09-15T21:47:15.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7682999999999998e-05 2024-09-15T21:47:15.831 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.831 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.831 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.831 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:35 2.f 2:f426ad4f:::mds2_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:15.831 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:47:15.831 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 30.008748285999999, 2024-09-15T21:47:15.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017328949999999999, 2024-09-15T21:47:15.831 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.832 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.832 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.832 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:47:15.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:47:15.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738103+0000", 2024-09-15T21:47:15.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0020000000000001e-06 2024-09-15T21:47:15.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738112+0000", 2024-09-15T21:47:15.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2220000000000008e-06 2024-09-15T21:47:15.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738127+0000", 2024-09-15T21:47:15.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5016e-05 2024-09-15T21:47:15.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738137+0000", 2024-09-15T21:47:15.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0301e-05 2024-09-15T21:47:15.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738216+0000", 2024-09-15T21:47:15.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8356000000000007e-05 2024-09-15T21:47:15.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:15.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738340+0000", 2024-09-15T21:47:15.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012372800000000001 2024-09-15T21:47:15.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:15.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738453+0000", 2024-09-15T21:47:15.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000113785 2024-09-15T21:47:15.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:15.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739115+0000", 2024-09-15T21:47:15.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00066178500000000002 2024-09-15T21:47:15.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:15.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739745+0000", 2024-09-15T21:47:15.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00063017900000000003 2024-09-15T21:47:15.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:15.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739783+0000", 2024-09-15T21:47:15.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7941000000000001e-05 2024-09-15T21:47:15.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739834+0000", 2024-09-15T21:47:15.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.058e-05 2024-09-15T21:47:15.837 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.837 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.837 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.837 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.4.16:13 2.13 e22/19 2:c81eaac8:::62d.00000000:head v 22'8, mlcod=22'6)", 2024-09-15T21:47:15.837 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:47:15.837 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 24.181343226999999, 2024-09-15T21:47:15.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002151778, 2024-09-15T21:47:15.837 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.837 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.837 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:15.838 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:47:15.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:47:15.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565507+0000", 2024-09-15T21:47:15.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3340000000000001e-06 2024-09-15T21:47:15.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565518+0000", 2024-09-15T21:47:15.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0339999999999999e-05 2024-09-15T21:47:15.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565536+0000", 2024-09-15T21:47:15.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8647e-05 2024-09-15T21:47:15.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565545+0000", 2024-09-15T21:47:15.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1430000000000004e-06 2024-09-15T21:47:15.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565611+0000", 2024-09-15T21:47:15.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5129000000000006e-05 2024-09-15T21:47:15.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:15.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565634+0000", 2024-09-15T21:47:15.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3173e-05 2024-09-15T21:47:15.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:15.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567561+0000", 2024-09-15T21:47:15.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019268180000000001 2024-09-15T21:47:15.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567658+0000", 2024-09-15T21:47:15.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7194000000000005e-05 2024-09-15T21:47:15.841 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.841 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.841 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.842 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.d epoch 22/19 rep_tid 24 entries 22'3 (0'0) error 2:b2057508:::204.00000002:head by mds.4.16:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 22'1)", 2024-09-15T21:47:15.842 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:47:15.842 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 24.181182365000002, 2024-09-15T21:47:15.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026915049999999999, 2024-09-15T21:47:15.842 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:15.842 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:15.842 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:15.842 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:15.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:15.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:47:15.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:15.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:47:15.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:15.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:15.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565668+0000", 2024-09-15T21:47:15.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.113e-06 2024-09-15T21:47:15.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:15.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565684+0000", 2024-09-15T21:47:15.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5943999999999998e-05 2024-09-15T21:47:15.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:15.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565705+0000", 2024-09-15T21:47:15.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0761999999999999e-05 2024-09-15T21:47:15.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:15.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565724+0000", 2024-09-15T21:47:15.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9208e-05 2024-09-15T21:47:15.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:15.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565817+0000", 2024-09-15T21:47:15.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3218000000000001e-05 2024-09-15T21:47:15.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:15.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:15.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:15.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568358+0000", 2024-09-15T21:47:15.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00254126 2024-09-15T21:47:15.845 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.845 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.845 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.845 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:15.846 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:15.846 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:15.847 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:47:15.848 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:15.845+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:15.947 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:15.946+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:15.990 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:15.991 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:15.991 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:15.991 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:15.999 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:47:16.083 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:16.082+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:16.143 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:16.143 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:16.143 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:16.143 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:16.143 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:16.149 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:16.148+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:16.152 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:47:16.199 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:47:16.217 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:47:16.218 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:47:16.218 INFO:tasks.thrashosds.thrasher:Killing osd 5, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:47:16.218 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-15T21:47:16.218 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:47:16.251 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:16.249+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:16.269 INFO:tasks.ceph.osd.5:Stopped 2024-09-15T21:47:16.269 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.5 2024-09-15T21:47:16.269 INFO:tasks.thrashosds.thrasher:remote for osd 5 is ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T21:47:16.269 DEBUG:teuthology.orchestra.run.smithi122:> type ceph-objectstore-tool 2024-09-15T21:47:16.284 INFO:teuthology.orchestra.run.smithi122.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-15T21:47:16.287 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T21:47:16.288 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:47:16.299 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T21:47:16.299 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op list-pgs 2024-09-15T21:47:16.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.307+0000 7fc49ea65640 1 Processor -- start 2024-09-15T21:47:16.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.307+0000 7fc49ea65640 1 -- start start 2024-09-15T21:47:16.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.308+0000 7fc49ea65640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc498152280 0x7fc4981588a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:16.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.308+0000 7fc49ea65640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc4980c3880 0x7fc4980c8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:16.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.308+0000 7fc49ea65640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc498071580 con 0x7fc498072fa0 2024-09-15T21:47:16.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.308+0000 7fc49ea65640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fc498152660 con 0x7fc498152280 2024-09-15T21:47:16.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.308+0000 7fc49ea65640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc4980c4890 con 0x7fc4980c3880 2024-09-15T21:47:16.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.308+0000 7fc49d002640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc498072fa0 0x7fc49815b480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:34064/0 (socket says 172.21.15.100:34064) 2024-09-15T21:47:16.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.308+0000 7fc49d002640 1 -- 172.21.15.100:0/3903375928 learned_addr learned my addr 172.21.15.100:0/3903375928 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:16.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.308+0000 7fc48ffff640 1 --2- 172.21.15.100:0/3903375928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc4980c3880 0x7fc4980c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:16.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.308+0000 7fc49c801640 1 --2- 172.21.15.100:0/3903375928 >> v2:172.21.15.122:3300/0 conn(0x7fc498152280 0x7fc4981588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.308+0000 7fc48f7fe640 1 -- 172.21.15.100:0/3903375928 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1810823706 0 0) 0x7fc498071580 con 0x7fc498072fa0 2024-09-15T21:47:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.308+0000 7fc48f7fe640 1 -- 172.21.15.100:0/3903375928 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc480003520 con 0x7fc498072fa0 2024-09-15T21:47:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.308+0000 7fc49c801640 1 -- 172.21.15.100:0/3903375928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc4980c3880 msgr2=0x7fc4980c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.308+0000 7fc49c801640 1 --2- 172.21.15.100:0/3903375928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc4980c3880 0x7fc4980c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:16.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.308+0000 7fc49c801640 1 -- 172.21.15.100:0/3903375928 >> v1:172.21.15.100:6789/0 conn(0x7fc498072fa0 legacy=0x7fc49815b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.309+0000 7fc49c801640 1 -- 172.21.15.100:0/3903375928 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc498071860 con 0x7fc498152280 2024-09-15T21:47:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.309+0000 7fc48ffff640 1 --2- 172.21.15.100:0/3903375928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc4980c3880 0x7fc4980c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.309+0000 7fc49c801640 1 --2- 172.21.15.100:0/3903375928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc498152280 0x7fc4981588a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc494009d90 tx=0x7fc49402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a07834e5e5f89bb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.309+0000 7fc48f7fe640 1 -- 172.21.15.100:0/3903375928 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494007740 con 0x7fc498152280 2024-09-15T21:47:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.309+0000 7fc48f7fe640 1 -- 172.21.15.100:0/3903375928 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4940078e0 con 0x7fc498152280 2024-09-15T21:47:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.310+0000 7fc48f7fe640 1 -- 172.21.15.100:0/3903375928 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc494007c90 con 0x7fc498152280 2024-09-15T21:47:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.310+0000 7fc49ea65640 1 -- 172.21.15.100:0/3903375928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc498152280 msgr2=0x7fc4981588a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.310+0000 7fc49ea65640 1 --2- 172.21.15.100:0/3903375928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc498152280 0x7fc4981588a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc494009d90 tx=0x7fc49402a6f0 comp rx=0 tx=0).stop 2024-09-15T21:47:16.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.310+0000 7fc49ea65640 1 -- 172.21.15.100:0/3903375928 shutdown_connections 2024-09-15T21:47:16.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.310+0000 7fc49ea65640 1 --2- 172.21.15.100:0/3903375928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc4980c3880 0x7fc4980c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:16.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.310+0000 7fc49ea65640 1 --2- 172.21.15.100:0/3903375928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc498152280 0x7fc4981588a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:16.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.310+0000 7fc49ea65640 1 -- 172.21.15.100:0/3903375928 >> 172.21.15.100:0/3903375928 conn(0x7fc4980c9380 msgr2=0x7fc4980c9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:16.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.310+0000 7fc49ea65640 1 -- 172.21.15.100:0/3903375928 shutdown_connections 2024-09-15T21:47:16.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.310+0000 7fc49ea65640 1 -- 172.21.15.100:0/3903375928 wait complete. 2024-09-15T21:47:16.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.311+0000 7fc49ea65640 1 Processor -- start 2024-09-15T21:47:16.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.311+0000 7fc49ea65640 1 -- start start 2024-09-15T21:47:16.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.312+0000 7fc49ea65640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc498072fa0 0x7fc49814c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:16.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.312+0000 7fc49ea65640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4980c3880 0x7fc4981ee520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:16.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.312+0000 7fc49ea65640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc498152280 0x7fc498149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:16.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.312+0000 7fc49ea65640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc498158380 con 0x7fc4980c3880 2024-09-15T21:47:16.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.312+0000 7fc49ea65640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4981580a0 con 0x7fc498072fa0 2024-09-15T21:47:16.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.312+0000 7fc49ea65640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc498158de0 con 0x7fc498152280 2024-09-15T21:47:16.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.313+0000 7fc48ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4980c3880 0x7fc4981ee520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:16.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.313+0000 7fc49c801640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc498072fa0 0x7fc49814c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:16.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.313+0000 7fc48ffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4980c3880 0x7fc4981ee520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49324/0 (socket says 172.21.15.100:49324) 2024-09-15T21:47:16.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.313+0000 7fc48ffff640 1 -- 172.21.15.100:0/3559481978 learned_addr learned my addr 172.21.15.100:0/3559481978 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:16.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.313+0000 7fc49d002640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc498152280 0x7fc498149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:16.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.313+0000 7fc48ffff640 1 -- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc498152280 msgr2=0x7fc498149f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:16.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.313+0000 7fc48ffff640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc498152280 0x7fc498149f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:16.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.313+0000 7fc48ffff640 1 -- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc498072fa0 msgr2=0x7fc49814c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:16.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.313+0000 7fc48ffff640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc498072fa0 0x7fc49814c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:16.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.313+0000 7fc48ffff640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4981eda60 con 0x7fc4980c3880 2024-09-15T21:47:16.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.313+0000 7fc48ffff640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc4980c3880 0x7fc4981ee520 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc488010df0 tx=0x7fc48800e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:16.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.313+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc48801e070 con 0x7fc4980c3880 2024-09-15T21:47:16.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.313+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc488009e20 con 0x7fc4980c3880 2024-09-15T21:47:16.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.313+0000 7fc49ea65640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4981f1a80 con 0x7fc4980c3880 2024-09-15T21:47:16.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.313+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488011880 con 0x7fc4980c3880 2024-09-15T21:47:16.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.313+0000 7fc49ea65640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4981edd20 con 0x7fc4980c3880 2024-09-15T21:47:16.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.314+0000 7fc49ea65640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fc498069420 con 0x7fc4980c3880 2024-09-15T21:47:16.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.315+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc488021070 con 0x7fc4980c3880 2024-09-15T21:47:16.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.315+0000 7fc48d7fa640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc48406c2e0 0x7fc48406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:16.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.315+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5854+0+0 (secure 0 0 0) 0x7fc48808d660 con 0x7fc4980c3880 2024-09-15T21:47:16.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.315+0000 7fc49c801640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc48406c2e0 0x7fc48406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:16.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.316+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v29) ==== 54+0+16 (secure 0 0 0) 0x7fc488093170 con 0x7fc4980c3880 2024-09-15T21:47:16.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.316+0000 7fc49ea65640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fc498157300 0x7fc4980cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:16.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.316+0000 7fc49ea65640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc4980c9090 con 0x7fc498157300 2024-09-15T21:47:16.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.316+0000 7fc49c801640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc48406c2e0 0x7fc48406e7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc494034000 tx=0x7fc49402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:16.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.316+0000 7fc49d002640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fc498157300 0x7fc4980cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:16.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.316+0000 7fc49d002640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fc498157300 0x7fc4980cb580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:16.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.317+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4980c9090 con 0x7fc498157300 2024-09-15T21:47:16.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.331+0000 7fc49ea65640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc4980ca1b0 con 0x7fc498157300 2024-09-15T21:47:16.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.332+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4980ca1b0 con 0x7fc498157300 2024-09-15T21:47:16.334 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:16.335 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:16.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.333+0000 7fc49ea65640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fc498147520 0x7fc498155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:16.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.333+0000 7fc49ea65640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fc4980c9ae0 con 0x7fc498147520 2024-09-15T21:47:16.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.333+0000 7fc49c801640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fc498147520 0x7fc498155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:16.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.333+0000 7fc49c801640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fc498147520 0x7fc498155af0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:16.336 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.334+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4980c9ae0 con 0x7fc498147520 2024-09-15T21:47:16.350 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.348+0000 7fc49ea65640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc4981ee090 con 0x7fc498147520 2024-09-15T21:47:16.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.349+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4981ee090 con 0x7fc498147520 2024-09-15T21:47:16.352 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:16.353 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:16.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.350+0000 7fc49ea65640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fc498147900 0x7fc498156540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:16.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.350+0000 7fc49ea65640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fc4981577d0 con 0x7fc498147900 2024-09-15T21:47:16.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.350+0000 7fc48ffff640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fc498147900 0x7fc498156540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:16.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.350+0000 7fc48ffff640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fc498147900 0x7fc498156540 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:16.355 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:16.350+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:16.355 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.351+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4981577d0 con 0x7fc498147900 2024-09-15T21:47:16.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.364+0000 7fc49ea65640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc4980c74d0 con 0x7fc498147900 2024-09-15T21:47:16.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.365+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4980c74d0 con 0x7fc498147900 2024-09-15T21:47:16.368 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:16.369 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:16.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.366+0000 7fc49ea65640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fc4980c7fb0 0x7fc498062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:16.369 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.366+0000 7fc49ea65640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fc4980c74d0 con 0x7fc4980c7fb0 2024-09-15T21:47:16.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.366+0000 7fc49d002640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fc4980c7fb0 0x7fc498062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:16.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.366+0000 7fc49d002640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fc4980c7fb0 0x7fc498062630 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:16.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.367+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4980c74d0 con 0x7fc4980c7fb0 2024-09-15T21:47:16.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.381+0000 7fc49ea65640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc4980c74d0 con 0x7fc4980c7fb0 2024-09-15T21:47:16.384 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.382+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4980c74d0 con 0x7fc4980c7fb0 2024-09-15T21:47:16.384 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:16.385 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:16.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.383+0000 7fc49ea65640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fc49806bbe0 0x7fc49806c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:16.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.383+0000 7fc49ea65640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fc4980c74d0 con 0x7fc49806bbe0 2024-09-15T21:47:16.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.383+0000 7fc49c801640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fc49806bbe0 0x7fc49806c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:16.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.383+0000 7fc49c801640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fc49806bbe0 0x7fc49806c020 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:16.386 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.384+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4980c74d0 con 0x7fc49806bbe0 2024-09-15T21:47:16.399 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.397+0000 7fc49ea65640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc4980c74d0 con 0x7fc49806bbe0 2024-09-15T21:47:16.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.400+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4980c74d0 con 0x7fc49806bbe0 2024-09-15T21:47:16.402 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:16.403 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:16.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.400+0000 7fc49ea65640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fc49802c640 0x7fc49802caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:16.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.400+0000 7fc49ea65640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] -- command(tid 11: {"prefix": "get_command_descriptions"}) -- 0x7fc49806ca70 con 0x7fc49802c640 2024-09-15T21:47:16.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.400+0000 7fc48ffff640 1 -- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fc49802c640 msgr2=0x7fc49802caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6824/2400668532 2024-09-15T21:47:16.403 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.401+0000 7fc48ffff640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fc49802c640 0x7fc49802caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:47:16.443 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:16.443 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:16.443 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:16.443 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:16.453 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:47:16.453 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:16.451+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:16.554 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:16.552+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:16.556 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:16.556 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:16.556 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:16.556 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:16.556 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:16.565 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:47:16.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.602+0000 7fc48ffff640 1 -- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fc49802c640 msgr2=0x7fc49802caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6824/2400668532 2024-09-15T21:47:16.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:16.602+0000 7fc48ffff640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fc49802c640 0x7fc49802caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-15T21:47:16.654 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:16.653+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:16.671 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:16.671 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:16.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:16.671 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:16.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.671 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:47:16.671 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:16.671 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.433567345999997, 2024-09-15T21:47:16.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0944283340000001, 2024-09-15T21:47:16.672 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.672 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.672 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.672 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:16.672 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:16.672 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:16.672 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2024-09-15T21:47:16.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.672 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:16.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:16.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234360+0000", 2024-09-15T21:47:16.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3710000000000001e-06 2024-09-15T21:47:16.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.674 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.674 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234374+0000", 2024-09-15T21:47:16.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4003e-05 2024-09-15T21:47:16.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234392+0000", 2024-09-15T21:47:16.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7294000000000001e-05 2024-09-15T21:47:16.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234405+0000", 2024-09-15T21:47:16.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2961e-05 2024-09-15T21:47:16.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234483+0000", 2024-09-15T21:47:16.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8850000000000006e-05 2024-09-15T21:47:16.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-15T21:47:16.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234491+0000", 2024-09-15T21:47:16.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3529999999999997e-06 2024-09-15T21:47:16.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327253+0000", 2024-09-15T21:47:16.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.092761962 2024-09-15T21:47:16.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327456+0000", 2024-09-15T21:47:16.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000202893 2024-09-15T21:47:16.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327652+0000", 2024-09-15T21:47:16.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019613000000000001 2024-09-15T21:47:16.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.678 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328368+0000", 2024-09-15T21:47:16.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000716582 2024-09-15T21:47:16.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.678 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328682+0000", 2024-09-15T21:47:16.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000313825 2024-09-15T21:47:16.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328743+0000", 2024-09-15T21:47:16.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0571e-05 2024-09-15T21:47:16.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328787+0000", 2024-09-15T21:47:16.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4539000000000002e-05 2024-09-15T21:47:16.680 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.680 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.680 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.680 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:16.680 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:16.680 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.312401944000001, 2024-09-15T21:47:16.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0089551270000000002, 2024-09-15T21:47:16.680 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.681 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.681 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.681 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:16.681 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:16.681 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:16.681 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 35 2024-09-15T21:47:16.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.681 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:16.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:16.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355525+0000", 2024-09-15T21:47:16.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:47:16.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355606+0000", 2024-09-15T21:47:16.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1723000000000003e-05 2024-09-15T21:47:16.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355624+0000", 2024-09-15T21:47:16.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7645000000000001e-05 2024-09-15T21:47:16.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355629+0000", 2024-09-15T21:47:16.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7319999999999998e-06 2024-09-15T21:47:16.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.684 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358315+0000", 2024-09-15T21:47:16.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026858659999999999 2024-09-15T21:47:16.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358373+0000", 2024-09-15T21:47:16.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8593e-05 2024-09-15T21:47:16.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358521+0000", 2024-09-15T21:47:16.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014817500000000001 2024-09-15T21:47:16.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360353+0000", 2024-09-15T21:47:16.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018317100000000001 2024-09-15T21:47:16.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364424+0000", 2024-09-15T21:47:16.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040707970000000001 2024-09-15T21:47:16.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364452+0000", 2024-09-15T21:47:16.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7937999999999999e-05 2024-09-15T21:47:16.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364479+0000", 2024-09-15T21:47:16.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7744999999999999e-05 2024-09-15T21:47:16.687 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.687 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.687 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.687 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:16.687 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:16.688 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.312317981, 2024-09-15T21:47:16.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090267869999999997, 2024-09-15T21:47:16.688 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.688 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.688 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.688 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:16.688 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:16.688 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:16.688 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 36 2024-09-15T21:47:16.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.689 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:16.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:16.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355609+0000", 2024-09-15T21:47:16.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3099999999999998e-07 2024-09-15T21:47:16.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355612+0000", 2024-09-15T21:47:16.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1489999999999998e-06 2024-09-15T21:47:16.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355637+0000", 2024-09-15T21:47:16.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.508e-05 2024-09-15T21:47:16.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355640+0000", 2024-09-15T21:47:16.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7930000000000002e-06 2024-09-15T21:47:16.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359195+0000", 2024-09-15T21:47:16.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035555529999999999 2024-09-15T21:47:16.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359283+0000", 2024-09-15T21:47:16.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8042999999999997e-05 2024-09-15T21:47:16.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359419+0000", 2024-09-15T21:47:16.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013576699999999999 2024-09-15T21:47:16.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360361+0000", 2024-09-15T21:47:16.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094171000000000001 2024-09-15T21:47:16.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364536+0000", 2024-09-15T21:47:16.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00417484 2024-09-15T21:47:16.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364590+0000", 2024-09-15T21:47:16.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4355999999999999e-05 2024-09-15T21:47:16.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364635+0000", 2024-09-15T21:47:16.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5065000000000003e-05 2024-09-15T21:47:16.694 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.694 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.694 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.695 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:16.695 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:16.695 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.312313375999999, 2024-09-15T21:47:16.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091162750000000001, 2024-09-15T21:47:16.695 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.695 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.695 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.695 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:16.695 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:16.695 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:16.696 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 37 2024-09-15T21:47:16.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.696 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:16.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:16.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:16.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:47:16.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355615+0000", 2024-09-15T21:47:16.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-15T21:47:16.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355646+0000", 2024-09-15T21:47:16.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0391000000000001e-05 2024-09-15T21:47:16.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355648+0000", 2024-09-15T21:47:16.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6309999999999999e-06 2024-09-15T21:47:16.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359770+0000", 2024-09-15T21:47:16.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041222899999999998 2024-09-15T21:47:16.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359854+0000", 2024-09-15T21:47:16.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3399000000000006e-05 2024-09-15T21:47:16.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359988+0000", 2024-09-15T21:47:16.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013442200000000001 2024-09-15T21:47:16.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360524+0000", 2024-09-15T21:47:16.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00053582900000000004 2024-09-15T21:47:16.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364673+0000", 2024-09-15T21:47:16.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041493199999999997 2024-09-15T21:47:16.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364704+0000", 2024-09-15T21:47:16.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0380999999999999e-05 2024-09-15T21:47:16.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364729+0000", 2024-09-15T21:47:16.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5335999999999999e-05 2024-09-15T21:47:16.701 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.701 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.701 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.702 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:16.702 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:16.702 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.243596138000001, 2024-09-15T21:47:16.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0026270020000001, 2024-09-15T21:47:16.702 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.702 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.702 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.702 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:16.702 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:16.702 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:16.702 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 88 2024-09-15T21:47:16.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.703 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:16.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:16.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424331+0000", 2024-09-15T21:47:16.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4299999999999998e-07 2024-09-15T21:47:16.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424333+0000", 2024-09-15T21:47:16.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2359999999999999e-06 2024-09-15T21:47:16.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422895+0000", 2024-09-15T21:47:16.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99856184199999998 2024-09-15T21:47:16.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422902+0000", 2024-09-15T21:47:16.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4440000000000002e-06 2024-09-15T21:47:16.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423053+0000", 2024-09-15T21:47:16.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000150958 2024-09-15T21:47:16.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423160+0000", 2024-09-15T21:47:16.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106774 2024-09-15T21:47:16.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423344+0000", 2024-09-15T21:47:16.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018424199999999999 2024-09-15T21:47:16.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.424730+0000", 2024-09-15T21:47:16.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001386155 2024-09-15T21:47:16.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426756+0000", 2024-09-15T21:47:16.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020261150000000002 2024-09-15T21:47:16.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426906+0000", 2024-09-15T21:47:16.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000149176 2024-09-15T21:47:16.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426957+0000", 2024-09-15T21:47:16.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1616999999999997e-05 2024-09-15T21:47:16.709 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.709 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.709 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.709 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:16.709 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:16.709 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 61.113522940000003, 2024-09-15T21:47:16.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095037790000000004, 2024-09-15T21:47:16.710 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.710 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.710 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.710 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:16.710 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:16.710 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:16.710 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 114 2024-09-15T21:47:16.710 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.710 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.710 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:16.710 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:16.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554404+0000", 2024-09-15T21:47:16.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2399999999999999e-07 2024-09-15T21:47:16.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554407+0000", 2024-09-15T21:47:16.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-15T21:47:16.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554632+0000", 2024-09-15T21:47:16.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022568999999999999 2024-09-15T21:47:16.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554640+0000", 2024-09-15T21:47:16.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7970000000000001e-06 2024-09-15T21:47:16.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561325+0000", 2024-09-15T21:47:16.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066848020000000001 2024-09-15T21:47:16.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561402+0000", 2024-09-15T21:47:16.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6851e-05 2024-09-15T21:47:16.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561530+0000", 2024-09-15T21:47:16.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012863999999999999 2024-09-15T21:47:16.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563144+0000", 2024-09-15T21:47:16.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016132849999999999 2024-09-15T21:47:16.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563800+0000", 2024-09-15T21:47:16.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00065586099999999999 2024-09-15T21:47:16.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563857+0000", 2024-09-15T21:47:16.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.702e-05 2024-09-15T21:47:16.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563907+0000", 2024-09-15T21:47:16.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0479000000000002e-05 2024-09-15T21:47:16.716 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.716 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.716 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.716 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:16.716 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:16.716 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 61.113517942999998, 2024-09-15T21:47:16.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011813423999999999, 2024-09-15T21:47:16.717 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.717 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.717 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.717 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:16.717 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:16.717 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:16.717 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 115 2024-09-15T21:47:16.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.717 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:16.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:16.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554409+0000", 2024-09-15T21:47:16.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1100000000000002e-07 2024-09-15T21:47:16.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554498+0000", 2024-09-15T21:47:16.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9642999999999995e-05 2024-09-15T21:47:16.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554656+0000", 2024-09-15T21:47:16.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000158114 2024-09-15T21:47:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554661+0000", 2024-09-15T21:47:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0730000000000004e-06 2024-09-15T21:47:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561862+0000", 2024-09-15T21:47:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072002999999999998 2024-09-15T21:47:16.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561939+0000", 2024-09-15T21:47:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7265e-05 2024-09-15T21:47:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562073+0000", 2024-09-15T21:47:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013371600000000001 2024-09-15T21:47:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563196+0000", 2024-09-15T21:47:16.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011231679999999999 2024-09-15T21:47:16.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566124+0000", 2024-09-15T21:47:16.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002928403 2024-09-15T21:47:16.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566183+0000", 2024-09-15T21:47:16.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8328000000000001e-05 2024-09-15T21:47:16.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566222+0000", 2024-09-15T21:47:16.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9103000000000002e-05 2024-09-15T21:47:16.723 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.723 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.724 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.724 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:16.724 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:16.724 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 61.113424502999997, 2024-09-15T21:47:16.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01181426, 2024-09-15T21:47:16.724 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.724 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.724 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.724 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:16.725 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:16.725 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:16.725 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 116 2024-09-15T21:47:16.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.725 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:16.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:16.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554503+0000", 2024-09-15T21:47:16.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8000000000000005e-07 2024-09-15T21:47:16.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554506+0000", 2024-09-15T21:47:16.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5899999999999999e-06 2024-09-15T21:47:16.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554674+0000", 2024-09-15T21:47:16.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016763600000000001 2024-09-15T21:47:16.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554678+0000", 2024-09-15T21:47:16.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5310000000000003e-06 2024-09-15T21:47:16.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562714+0000", 2024-09-15T21:47:16.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.008035258 2024-09-15T21:47:16.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562765+0000", 2024-09-15T21:47:16.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1093e-05 2024-09-15T21:47:16.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562845+0000", 2024-09-15T21:47:16.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0427999999999996e-05 2024-09-15T21:47:16.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563943+0000", 2024-09-15T21:47:16.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097647 2024-09-15T21:47:16.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566232+0000", 2024-09-15T21:47:16.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022890850000000002 2024-09-15T21:47:16.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566279+0000", 2024-09-15T21:47:16.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6692e-05 2024-09-15T21:47:16.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566316+0000", 2024-09-15T21:47:16.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.752e-05 2024-09-15T21:47:16.731 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.731 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.731 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.731 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:16.731 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:16.731 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 61.113418404000001, 2024-09-15T21:47:16.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012232125, 2024-09-15T21:47:16.732 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.732 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.732 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.732 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:16.732 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:16.732 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:16.732 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 117 2024-09-15T21:47:16.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.732 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:16.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:16.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:16.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7700000000000001e-07 2024-09-15T21:47:16.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554511+0000", 2024-09-15T21:47:16.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8430000000000001e-06 2024-09-15T21:47:16.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554690+0000", 2024-09-15T21:47:16.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017856299999999999 2024-09-15T21:47:16.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554698+0000", 2024-09-15T21:47:16.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6200000000000005e-06 2024-09-15T21:47:16.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563224+0000", 2024-09-15T21:47:16.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0085262640000000004 2024-09-15T21:47:16.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563342+0000", 2024-09-15T21:47:16.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117687 2024-09-15T21:47:16.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563472+0000", 2024-09-15T21:47:16.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012957999999999999 2024-09-15T21:47:16.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566367+0000", 2024-09-15T21:47:16.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028952280000000001 2024-09-15T21:47:16.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566680+0000", 2024-09-15T21:47:16.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031345100000000002 2024-09-15T21:47:16.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566714+0000", 2024-09-15T21:47:16.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3701000000000001e-05 2024-09-15T21:47:16.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566740+0000", 2024-09-15T21:47:16.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5911e-05 2024-09-15T21:47:16.738 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.738 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.738 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.739 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:16.739 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:16.739 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.770526322000002, 2024-09-15T21:47:16.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069806590000000002, 2024-09-15T21:47:16.739 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.739 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.739 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.739 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:16.739 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:16.739 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:16.740 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 134 2024-09-15T21:47:16.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.740 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:16.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:16.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897401+0000", 2024-09-15T21:47:16.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0500000000000003e-07 2024-09-15T21:47:16.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897492+0000", 2024-09-15T21:47:16.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1770999999999999e-05 2024-09-15T21:47:16.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897750+0000", 2024-09-15T21:47:16.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025778299999999998 2024-09-15T21:47:16.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897756+0000", 2024-09-15T21:47:16.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2140000000000001e-06 2024-09-15T21:47:16.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899312+0000", 2024-09-15T21:47:16.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001556144 2024-09-15T21:47:16.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899372+0000", 2024-09-15T21:47:16.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0015000000000001e-05 2024-09-15T21:47:16.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899444+0000", 2024-09-15T21:47:16.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1923999999999998e-05 2024-09-15T21:47:16.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901354+0000", 2024-09-15T21:47:16.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019092530000000001 2024-09-15T21:47:16.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904327+0000", 2024-09-15T21:47:16.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029734520000000001 2024-09-15T21:47:16.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904360+0000", 2024-09-15T21:47:16.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3216000000000002e-05 2024-09-15T21:47:16.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904381+0000", 2024-09-15T21:47:16.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0282e-05 2024-09-15T21:47:16.746 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.746 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.746 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.746 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:16.746 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:16.746 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.770430464999997, 2024-09-15T21:47:16.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070121059999999997, 2024-09-15T21:47:16.746 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.746 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.747 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.747 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:16.747 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:16.747 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:16.747 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 135 2024-09-15T21:47:16.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.747 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:16.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:16.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897497+0000", 2024-09-15T21:47:16.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4799999999999997e-07 2024-09-15T21:47:16.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897500+0000", 2024-09-15T21:47:16.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2949999999999998e-06 2024-09-15T21:47:16.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897781+0000", 2024-09-15T21:47:16.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028064300000000002 2024-09-15T21:47:16.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897786+0000", 2024-09-15T21:47:16.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5169999999999999e-06 2024-09-15T21:47:16.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899857+0000", 2024-09-15T21:47:16.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020705340000000002 2024-09-15T21:47:16.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899932+0000", 2024-09-15T21:47:16.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5840999999999994e-05 2024-09-15T21:47:16.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900005+0000", 2024-09-15T21:47:16.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2716e-05 2024-09-15T21:47:16.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902296+0000", 2024-09-15T21:47:16.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022912010000000001 2024-09-15T21:47:16.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904424+0000", 2024-09-15T21:47:16.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021278600000000001 2024-09-15T21:47:16.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904472+0000", 2024-09-15T21:47:16.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7586000000000001e-05 2024-09-15T21:47:16.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904508+0000", 2024-09-15T21:47:16.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6164999999999997e-05 2024-09-15T21:47:16.753 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.753 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.754 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.754 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:16.754 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:16.754 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.770405973999999, 2024-09-15T21:47:16.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070611429999999998, 2024-09-15T21:47:16.754 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.754 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.754 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.754 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:16.755 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:16.755 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:16.755 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 136 2024-09-15T21:47:16.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.755 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:16.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:16.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897521+0000", 2024-09-15T21:47:16.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2900000000000004e-07 2024-09-15T21:47:16.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897527+0000", 2024-09-15T21:47:16.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8490000000000002e-06 2024-09-15T21:47:16.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897799+0000", 2024-09-15T21:47:16.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027219499999999998 2024-09-15T21:47:16.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897804+0000", 2024-09-15T21:47:16.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4219999999999999e-06 2024-09-15T21:47:16.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900248+0000", 2024-09-15T21:47:16.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002443457 2024-09-15T21:47:16.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900306+0000", 2024-09-15T21:47:16.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8263000000000001e-05 2024-09-15T21:47:16.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900385+0000", 2024-09-15T21:47:16.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9161999999999999e-05 2024-09-15T21:47:16.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902300+0000", 2024-09-15T21:47:16.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001914947 2024-09-15T21:47:16.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904536+0000", 2024-09-15T21:47:16.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022354419999999998 2024-09-15T21:47:16.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904562+0000", 2024-09-15T21:47:16.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6378999999999999e-05 2024-09-15T21:47:16.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904581+0000", 2024-09-15T21:47:16.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9497999999999999e-05 2024-09-15T21:47:16.761 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.761 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.761 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.761 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:16.762 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:16.762 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.770397406999997, 2024-09-15T21:47:16.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007180306, 2024-09-15T21:47:16.762 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.762 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.762 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.762 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:16.762 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:16.762 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:16.762 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 137 2024-09-15T21:47:16.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.763 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:16.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:16.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:16.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1399999999999998e-07 2024-09-15T21:47:16.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897612+0000", 2024-09-15T21:47:16.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2607000000000004e-05 2024-09-15T21:47:16.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897815+0000", 2024-09-15T21:47:16.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020352400000000001 2024-09-15T21:47:16.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897821+0000", 2024-09-15T21:47:16.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4670000000000003e-06 2024-09-15T21:47:16.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900665+0000", 2024-09-15T21:47:16.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028443689999999998 2024-09-15T21:47:16.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900714+0000", 2024-09-15T21:47:16.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8812e-05 2024-09-15T21:47:16.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900799+0000", 2024-09-15T21:47:16.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5129000000000004e-05 2024-09-15T21:47:16.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903413+0000", 2024-09-15T21:47:16.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026142890000000001 2024-09-15T21:47:16.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904625+0000", 2024-09-15T21:47:16.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012121149999999999 2024-09-15T21:47:16.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904673+0000", 2024-09-15T21:47:16.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7540999999999998e-05 2024-09-15T21:47:16.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904709+0000", 2024-09-15T21:47:16.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6139000000000001e-05 2024-09-15T21:47:16.768 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.768 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.768 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.769 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:16.769 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:16.769 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.770311452000001, 2024-09-15T21:47:16.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071727450000000003, 2024-09-15T21:47:16.769 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.769 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.769 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.769 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:16.769 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:16.770 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:16.770 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 138 2024-09-15T21:47:16.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.770 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:16.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:16.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:16.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:47:16.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897619+0000", 2024-09-15T21:47:16.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3869999999999999e-06 2024-09-15T21:47:16.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897832+0000", 2024-09-15T21:47:16.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021329500000000001 2024-09-15T21:47:16.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897837+0000", 2024-09-15T21:47:16.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3730000000000001e-06 2024-09-15T21:47:16.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901391+0000", 2024-09-15T21:47:16.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003553601 2024-09-15T21:47:16.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901466+0000", 2024-09-15T21:47:16.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5130999999999996e-05 2024-09-15T21:47:16.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901601+0000", 2024-09-15T21:47:16.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134797 2024-09-15T21:47:16.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903417+0000", 2024-09-15T21:47:16.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001816156 2024-09-15T21:47:16.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904741+0000", 2024-09-15T21:47:16.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013241049999999999 2024-09-15T21:47:16.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904767+0000", 2024-09-15T21:47:16.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5876000000000001e-05 2024-09-15T21:47:16.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904788+0000", 2024-09-15T21:47:16.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0449999999999999e-05 2024-09-15T21:47:16.776 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.776 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.776 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.776 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:16.776 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:16.776 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.770305798999999, 2024-09-15T21:47:16.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072970250000000004, 2024-09-15T21:47:16.777 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.777 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.777 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.777 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:16.777 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:16.777 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:16.777 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 139 2024-09-15T21:47:16.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.777 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:16.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:16.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897621+0000", 2024-09-15T21:47:16.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3099999999999999e-07 2024-09-15T21:47:16.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897625+0000", 2024-09-15T21:47:16.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.698e-06 2024-09-15T21:47:16.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897848+0000", 2024-09-15T21:47:16.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022375699999999999 2024-09-15T21:47:16.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897853+0000", 2024-09-15T21:47:16.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7709999999999997e-06 2024-09-15T21:47:16.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901929+0000", 2024-09-15T21:47:16.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040757780000000004 2024-09-15T21:47:16.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901976+0000", 2024-09-15T21:47:16.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6751e-05 2024-09-15T21:47:16.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902054+0000", 2024-09-15T21:47:16.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8856999999999994e-05 2024-09-15T21:47:16.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903421+0000", 2024-09-15T21:47:16.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00136619 2024-09-15T21:47:16.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904827+0000", 2024-09-15T21:47:16.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0014067369999999999 2024-09-15T21:47:16.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904881+0000", 2024-09-15T21:47:16.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3520999999999998e-05 2024-09-15T21:47:16.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904918+0000", 2024-09-15T21:47:16.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6634000000000002e-05 2024-09-15T21:47:16.783 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.783 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.783 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.783 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:16.783 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:16.783 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.770299969999996, 2024-09-15T21:47:16.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097221730000000006, 2024-09-15T21:47:16.784 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.784 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.784 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.784 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:16.784 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:16.784 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:16.784 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 140 2024-09-15T21:47:16.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.784 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:16.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:16.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897627+0000", 2024-09-15T21:47:16.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-15T21:47:16.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897698+0000", 2024-09-15T21:47:16.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1722000000000002e-05 2024-09-15T21:47:16.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897863+0000", 2024-09-15T21:47:16.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016479099999999999 2024-09-15T21:47:16.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897868+0000", 2024-09-15T21:47:16.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4950000000000002e-06 2024-09-15T21:47:16.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902335+0000", 2024-09-15T21:47:16.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004467632 2024-09-15T21:47:16.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902414+0000", 2024-09-15T21:47:16.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8688000000000003e-05 2024-09-15T21:47:16.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902535+0000", 2024-09-15T21:47:16.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012066900000000001 2024-09-15T21:47:16.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903714+0000", 2024-09-15T21:47:16.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011794520000000001 2024-09-15T21:47:16.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907115+0000", 2024-09-15T21:47:16.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034008879999999999 2024-09-15T21:47:16.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907250+0000", 2024-09-15T21:47:16.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134657 2024-09-15T21:47:16.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907348+0000", 2024-09-15T21:47:16.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8838000000000005e-05 2024-09-15T21:47:16.791 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.791 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.791 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.791 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:16.791 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:16.791 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.770224888999998, 2024-09-15T21:47:16.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097618050000000001, 2024-09-15T21:47:16.791 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.791 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.792 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.792 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:16.792 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:16.792 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:16.792 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 141 2024-09-15T21:47:16.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.792 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:16.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:16.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897702+0000", 2024-09-15T21:47:16.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6199999999999998e-07 2024-09-15T21:47:16.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897705+0000", 2024-09-15T21:47:16.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0180000000000002e-06 2024-09-15T21:47:16.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897889+0000", 2024-09-15T21:47:16.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000183678 2024-09-15T21:47:16.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897894+0000", 2024-09-15T21:47:16.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8930000000000004e-06 2024-09-15T21:47:16.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.795 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903099+0000", 2024-09-15T21:47:16.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052056280000000003 2024-09-15T21:47:16.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.795 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903151+0000", 2024-09-15T21:47:16.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2163000000000002e-05 2024-09-15T21:47:16.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903220+0000", 2024-09-15T21:47:16.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8483000000000001e-05 2024-09-15T21:47:16.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904794+0000", 2024-09-15T21:47:16.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001574521 2024-09-15T21:47:16.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907405+0000", 2024-09-15T21:47:16.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026110339999999999 2024-09-15T21:47:16.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907440+0000", 2024-09-15T21:47:16.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4564999999999999e-05 2024-09-15T21:47:16.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907463+0000", 2024-09-15T21:47:16.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3260000000000001e-05 2024-09-15T21:47:16.798 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.798 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.798 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.798 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:22 2.8 2:1470b2f8:::613.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:16.798 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:16.799 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 42.212784562000003, 2024-09-15T21:47:16.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007454937, 2024-09-15T21:47:16.799 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.799 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.799 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.799 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:16.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:16.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:16.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6800000000000002e-07 2024-09-15T21:47:16.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455145+0000", 2024-09-15T21:47:16.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4470000000000002e-06 2024-09-15T21:47:16.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456425+0000", 2024-09-15T21:47:16.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012798200000000001 2024-09-15T21:47:16.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456433+0000", 2024-09-15T21:47:16.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7389999999999999e-06 2024-09-15T21:47:16.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:16.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097865 2024-09-15T21:47:16.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457614+0000", 2024-09-15T21:47:16.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3560000000000006e-05 2024-09-15T21:47:16.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457998+0000", 2024-09-15T21:47:16.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00038372699999999999 2024-09-15T21:47:16.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460488+0000", 2024-09-15T21:47:16.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002489855 2024-09-15T21:47:16.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462512+0000", 2024-09-15T21:47:16.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020235269999999998 2024-09-15T21:47:16.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462554+0000", 2024-09-15T21:47:16.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2472000000000003e-05 2024-09-15T21:47:16.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462597+0000", 2024-09-15T21:47:16.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2657000000000002e-05 2024-09-15T21:47:16.805 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.805 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.805 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.805 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:2 2.5 2:a6e36b3f:::401.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:16.805 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:16.805 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 42.212779593, 2024-09-15T21:47:16.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074484590000000002, 2024-09-15T21:47:16.806 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.806 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.806 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.806 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:16.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:16.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:16.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:47:16.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455150+0000", 2024-09-15T21:47:16.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7319999999999998e-06 2024-09-15T21:47:16.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456449+0000", 2024-09-15T21:47:16.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012991490000000001 2024-09-15T21:47:16.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456454+0000", 2024-09-15T21:47:16.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0390000000000004e-06 2024-09-15T21:47:16.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456513+0000", 2024-09-15T21:47:16.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8768999999999999e-05 2024-09-15T21:47:16.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456628+0000", 2024-09-15T21:47:16.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115641 2024-09-15T21:47:16.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456741+0000", 2024-09-15T21:47:16.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000113094 2024-09-15T21:47:16.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458520+0000", 2024-09-15T21:47:16.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001778993 2024-09-15T21:47:16.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462494+0000", 2024-09-15T21:47:16.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039737009999999996 2024-09-15T21:47:16.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462549+0000", 2024-09-15T21:47:16.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4537000000000003e-05 2024-09-15T21:47:16.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462595+0000", 2024-09-15T21:47:16.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6511000000000002e-05 2024-09-15T21:47:16.811 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.812 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.812 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.812 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:27 2.5 2:a5fa685f:::504.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:16.812 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:16.812 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 25.103087209000002, 2024-09-15T21:47:16.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071804599999999996, 2024-09-15T21:47:16.812 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:16.812 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:16.812 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:16.813 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:16.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:16.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:16.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:16.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:16.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:16.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:16.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:16.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-15T21:47:16.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:16.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564841+0000", 2024-09-15T21:47:16.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-15T21:47:16.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:16.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565126+0000", 2024-09-15T21:47:16.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028469900000000002 2024-09-15T21:47:16.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:16.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565133+0000", 2024-09-15T21:47:16.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9480000000000002e-06 2024-09-15T21:47:16.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:16.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568006+0000", 2024-09-15T21:47:16.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028730330000000001 2024-09-15T21:47:16.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:16.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568119+0000", 2024-09-15T21:47:16.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011325899999999999 2024-09-15T21:47:16.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:16.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568235+0000", 2024-09-15T21:47:16.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115868 2024-09-15T21:47:16.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:16.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569668+0000", 2024-09-15T21:47:16.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001433099 2024-09-15T21:47:16.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:16.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571926+0000", 2024-09-15T21:47:16.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022572410000000001 2024-09-15T21:47:16.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:16.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571981+0000", 2024-09-15T21:47:16.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5383000000000002e-05 2024-09-15T21:47:16.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:16.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:16.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:16.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572020+0000", 2024-09-15T21:47:16.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8574e-05 2024-09-15T21:47:16.818 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.819 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.819 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.819 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:16.819 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:16.819 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:16.820 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T21:47:16.821 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:16.819+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:16.822 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:47:16.823 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T21:47:16.921 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:16.920+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:16.926 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:16.926 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:16.935 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:47:17.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.003+0000 7fc48ffff640 1 -- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fc49802c640 msgr2=0x7fc49802caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6824/2400668532 2024-09-15T21:47:17.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.003+0000 7fc48ffff640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fc49802c640 0x7fc49802caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-15T21:47:17.022 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:17.021+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:17.077 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:17.077 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:17.085 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:47:17.127 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:17.125+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:17.224 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:17.223+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:17.229 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:17.229 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:17.238 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:47:17.325 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:17.323+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:17.341 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:17.342 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:17.351 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:47:17.434 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:17.433+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:17.495 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:17.495 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:17.504 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:47:17.544 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:17.543+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:17.628 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:17.627+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:17.647 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:17.647 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:17.656 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:47:17.674 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:47:17.746 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:17.745+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:17.766 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:17.766 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:17.775 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:47:17.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.799+0000 7f2618aaf640 1 Processor -- start 2024-09-15T21:47:17.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.799+0000 7f2618aaf640 1 -- start start 2024-09-15T21:47:17.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.800+0000 7f2618aaf640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f26140c4600 0x7f261406d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:17.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.800+0000 7f2618aaf640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26140c4a10 con 0x7f2614152160 2024-09-15T21:47:17.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.800+0000 7f2618aaf640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2614157ce0 con 0x7f2614073790 2024-09-15T21:47:17.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.800+0000 7f2618aaf640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f2614072bf0 con 0x7f26140c4600 2024-09-15T21:47:17.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.800+0000 7f261259c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f2614152160 0x7f2614073080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:49974/0 (socket says 172.21.15.100:49974) 2024-09-15T21:47:17.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.800+0000 7f261259c640 1 -- 172.21.15.100:0/1403093047 learned_addr learned my addr 172.21.15.100:0/1403093047 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:17.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.800+0000 7f2612d9d640 1 --2- 172.21.15.100:0/1403093047 >> v2:172.21.15.122:3300/0 conn(0x7f26140c4600 0x7f261406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:17.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.801+0000 7f2611d9b640 1 -- 172.21.15.100:0/1403093047 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2033797973 0 0) 0x7f2604001110 con 0x7f2614152160 2024-09-15T21:47:17.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.801+0000 7f2611d9b640 1 -- 172.21.15.100:0/1403093047 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2600003520 con 0x7f2614152160 2024-09-15T21:47:17.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.801+0000 7f2611d9b640 1 -- 172.21.15.100:0/1403093047 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1653245181 0 0) 0x7f2614157ce0 con 0x7f2614073790 2024-09-15T21:47:17.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.801+0000 7f2611d9b640 1 -- 172.21.15.100:0/1403093047 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2604001110 con 0x7f2614073790 2024-09-15T21:47:17.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.801+0000 7f2612d9d640 1 -- 172.21.15.100:0/1403093047 >> v1:172.21.15.122:6789/0 conn(0x7f2614073790 legacy=0x7f261406ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:17.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.801+0000 7f2612d9d640 1 -- 172.21.15.100:0/1403093047 >> v1:172.21.15.100:6789/0 conn(0x7f2614152160 legacy=0x7f2614073080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:17.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.801+0000 7f2612d9d640 1 -- 172.21.15.100:0/1403093047 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2614157fa0 con 0x7f26140c4600 2024-09-15T21:47:17.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.802+0000 7f2612d9d640 1 --2- 172.21.15.100:0/1403093047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26140c4600 0x7f261406d400 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f260c009d90 tx=0x7f260c02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37052270881c41c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:17.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.802+0000 7f2611d9b640 1 -- 172.21.15.100:0/1403093047 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c007740 con 0x7f26140c4600 2024-09-15T21:47:17.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.802+0000 7f2611d9b640 1 -- 172.21.15.100:0/1403093047 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f260c0078e0 con 0x7f26140c4600 2024-09-15T21:47:17.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.802+0000 7f2611d9b640 1 -- 172.21.15.100:0/1403093047 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260c007c90 con 0x7f26140c4600 2024-09-15T21:47:17.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.803+0000 7f2618aaf640 1 -- 172.21.15.100:0/1403093047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26140c4600 msgr2=0x7f261406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:17.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.803+0000 7f2618aaf640 1 --2- 172.21.15.100:0/1403093047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26140c4600 0x7f261406d400 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f260c009d90 tx=0x7f260c02a6f0 comp rx=0 tx=0).stop 2024-09-15T21:47:17.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.803+0000 7f2618aaf640 1 -- 172.21.15.100:0/1403093047 shutdown_connections 2024-09-15T21:47:17.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.803+0000 7f2618aaf640 1 --2- 172.21.15.100:0/1403093047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26140c4600 0x7f261406d400 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:17.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.803+0000 7f2618aaf640 1 -- 172.21.15.100:0/1403093047 >> 172.21.15.100:0/1403093047 conn(0x7f26140ca390 msgr2=0x7f26140ca770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:17.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.803+0000 7f2618aaf640 1 -- 172.21.15.100:0/1403093047 shutdown_connections 2024-09-15T21:47:17.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.803+0000 7f2618aaf640 1 -- 172.21.15.100:0/1403093047 wait complete. 2024-09-15T21:47:17.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.804+0000 7fc48ffff640 1 -- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fc49802c640 msgr2=0x7fc49802caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6824/2400668532 2024-09-15T21:47:17.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.804+0000 7fc48ffff640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fc49802c640 0x7fc49802caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-15T21:47:17.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.804+0000 7f2618aaf640 1 Processor -- start 2024-09-15T21:47:17.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.804+0000 7f2618aaf640 1 -- start start 2024-09-15T21:47:17.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.805+0000 7f2618aaf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2614073790 0x7f2614154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:17.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.805+0000 7f2618aaf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26140c4600 0x7f2614072330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:17.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.805+0000 7f2618aaf640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2614152160 0x7f26141525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:17.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.805+0000 7f2618aaf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2614158840 con 0x7f2614073790 2024-09-15T21:47:17.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.805+0000 7f2618aaf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2614158560 con 0x7f26140c4600 2024-09-15T21:47:17.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.805+0000 7f2618aaf640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2614158b20 con 0x7f2614152160 2024-09-15T21:47:17.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.805+0000 7f261259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2614073790 0x7f2614154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:17.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.805+0000 7f261259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2614073790 0x7f2614154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51660/0 (socket says 172.21.15.100:51660) 2024-09-15T21:47:17.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.805+0000 7f261259c640 1 -- 172.21.15.100:0/1103733724 learned_addr learned my addr 172.21.15.100:0/1103733724 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:17.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.805+0000 7f2609d9b640 1 --2- 172.21.15.100:0/1103733724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26140c4600 0x7f2614072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:17.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.805+0000 7f2612d9d640 1 --2- 172.21.15.100:0/1103733724 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2614152160 0x7f26141525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:17.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.806+0000 7f261259c640 1 -- 172.21.15.100:0/1103733724 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2614152160 msgr2=0x7f26141525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:17.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.806+0000 7f261259c640 1 --2- 172.21.15.100:0/1103733724 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2614152160 0x7f26141525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:17.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.806+0000 7f261259c640 1 -- 172.21.15.100:0/1103733724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26140c4600 msgr2=0x7f2614072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:17.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.806+0000 7f261259c640 1 --2- 172.21.15.100:0/1103733724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26140c4600 0x7f2614072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:17.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.806+0000 7f261259c640 1 -- 172.21.15.100:0/1103733724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261414c960 con 0x7f2614073790 2024-09-15T21:47:17.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.806+0000 7f2609d9b640 1 --2- 172.21.15.100:0/1103733724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26140c4600 0x7f2614072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:17.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.806+0000 7f261259c640 1 --2- 172.21.15.100:0/1103733724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2614073790 0x7f2614154250 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f260400c760 tx=0x7f260400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:17.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.806+0000 7f260b7fe640 1 -- 172.21.15.100:0/1103733724 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604017070 con 0x7f2614073790 2024-09-15T21:47:17.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.806+0000 7f260b7fe640 1 -- 172.21.15.100:0/1103733724 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2604005ab0 con 0x7f2614073790 2024-09-15T21:47:17.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.806+0000 7f2618aaf640 1 -- 172.21.15.100:0/1103733724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26141efa10 con 0x7f2614073790 2024-09-15T21:47:17.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.806+0000 7f260b7fe640 1 -- 172.21.15.100:0/1103733724 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604007070 con 0x7f2614073790 2024-09-15T21:47:17.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.806+0000 7f2618aaf640 1 -- 172.21.15.100:0/1103733724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26141effd0 con 0x7f2614073790 2024-09-15T21:47:17.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.807+0000 7f2618aaf640 1 -- 172.21.15.100:0/1103733724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f261406efd0 con 0x7f2614073790 2024-09-15T21:47:17.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.808+0000 7f260b7fe640 1 -- 172.21.15.100:0/1103733724 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f260401a070 con 0x7f2614073790 2024-09-15T21:47:17.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.808+0000 7f260b7fe640 1 --2- 172.21.15.100:0/1103733724 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f25f406c2b0 0x7f25f406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:17.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.809+0000 7f2609d9b640 1 --2- 172.21.15.100:0/1103733724 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f25f406c2b0 0x7f25f406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:17.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.809+0000 7f260b7fe640 1 -- 172.21.15.100:0/1103733724 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5854+0+0 (secure 0 0 0) 0x7f260408a9f0 con 0x7f2614073790 2024-09-15T21:47:17.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.809+0000 7f2609d9b640 1 --2- 172.21.15.100:0/1103733724 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f25f406c2b0 0x7f25f406e770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f25fc007fd0 tx=0x7f25fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:17.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.813+0000 7f260b7fe640 1 -- 172.21.15.100:0/1103733724 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f26140c4a10 con 0x7f2614073790 2024-09-15T21:47:17.830 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:17.829+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:17.909 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:17.909 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:17.915 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:47:17.931 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:17.930+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:17.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:17.994+0000 7f2618aaf640 1 -- 172.21.15.100:0/1103733724 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f261406dd40 con 0x7f2614073790 2024-09-15T21:47:18.005 INFO:teuthology.orchestra.run.smithi122.stdout:3.c 2024-09-15T21:47:18.005 INFO:teuthology.orchestra.run.smithi122.stdout:3.d 2024-09-15T21:47:18.006 INFO:teuthology.orchestra.run.smithi122.stdout:3.3c 2024-09-15T21:47:18.006 INFO:teuthology.orchestra.run.smithi122.stdout:3.5 2024-09-15T21:47:18.006 INFO:teuthology.orchestra.run.smithi122.stdout:3.3a 2024-09-15T21:47:18.006 INFO:teuthology.orchestra.run.smithi122.stdout:3.38 2024-09-15T21:47:18.006 INFO:teuthology.orchestra.run.smithi122.stdout:3.37 2024-09-15T21:47:18.006 INFO:teuthology.orchestra.run.smithi122.stdout:3.36 2024-09-15T21:47:18.006 INFO:teuthology.orchestra.run.smithi122.stdout:3.30 2024-09-15T21:47:18.006 INFO:teuthology.orchestra.run.smithi122.stdout:3.2f 2024-09-15T21:47:18.006 INFO:teuthology.orchestra.run.smithi122.stdout:3.2 2024-09-15T21:47:18.006 INFO:teuthology.orchestra.run.smithi122.stdout:3.28 2024-09-15T21:47:18.006 INFO:teuthology.orchestra.run.smithi122.stdout:3.22 2024-09-15T21:47:18.006 INFO:teuthology.orchestra.run.smithi122.stdout:3.21 2024-09-15T21:47:18.006 INFO:teuthology.orchestra.run.smithi122.stdout:3.1c 2024-09-15T21:47:18.007 INFO:teuthology.orchestra.run.smithi122.stdout:3.16 2024-09-15T21:47:18.007 INFO:teuthology.orchestra.run.smithi122.stdout:2.c 2024-09-15T21:47:18.007 INFO:teuthology.orchestra.run.smithi122.stdout:2.8 2024-09-15T21:47:18.007 INFO:teuthology.orchestra.run.smithi122.stdout:3.1f 2024-09-15T21:47:18.007 INFO:teuthology.orchestra.run.smithi122.stdout:2.3 2024-09-15T21:47:18.007 INFO:teuthology.orchestra.run.smithi122.stdout:2.2 2024-09-15T21:47:18.007 INFO:teuthology.orchestra.run.smithi122.stdout:2.1c 2024-09-15T21:47:18.007 INFO:teuthology.orchestra.run.smithi122.stdout:3.2a 2024-09-15T21:47:18.007 INFO:teuthology.orchestra.run.smithi122.stdout:2.18 2024-09-15T21:47:18.007 INFO:teuthology.orchestra.run.smithi122.stdout:3.17 2024-09-15T21:47:18.007 INFO:teuthology.orchestra.run.smithi122.stdout:2.17 2024-09-15T21:47:18.007 INFO:teuthology.orchestra.run.smithi122.stdout:2.16 2024-09-15T21:47:18.007 INFO:teuthology.orchestra.run.smithi122.stdout:2.15 2024-09-15T21:47:18.007 INFO:teuthology.orchestra.run.smithi122.stdout:2.12 2024-09-15T21:47:18.033 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:18.031+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:18.059 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:18.059 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:18.067 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:47:18.154 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:18.153+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:18.212 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:18.212 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:18.220 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:47:18.262 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:18.260+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:18.336 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:18.334+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:18.364 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:18.364 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:18.373 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:47:18.436 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:18.435+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:18.517 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:18.517 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:18.525 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op export --pgid 3.22 --file /var/log/ceph/exp.3.22.5 2024-09-15T21:47:18.526 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:47:18.567 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:18.564+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:18.637 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:18.636+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:18.672 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:18.673 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:18.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:18.675+0000 7f260b7fe640 1 -- 172.21.15.100:0/1103733724 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v32) ==== 97+0+0 (secure 0 0 0) 0x7f26040589f0 con 0x7f2614073790 2024-09-15T21:47:18.676 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is set 2024-09-15T21:47:18.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:18.677+0000 7f2618aaf640 1 -- 172.21.15.100:0/1103733724 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f25f406c2b0 msgr2=0x7f25f406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:18.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:18.677+0000 7f2618aaf640 1 --2- 172.21.15.100:0/1103733724 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f25f406c2b0 0x7f25f406e770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f25fc007fd0 tx=0x7f25fc009040 comp rx=0 tx=0).stop 2024-09-15T21:47:18.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:18.677+0000 7f2618aaf640 1 -- 172.21.15.100:0/1103733724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2614073790 msgr2=0x7f2614154250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:18.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:18.677+0000 7f2618aaf640 1 --2- 172.21.15.100:0/1103733724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2614073790 0x7f2614154250 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f260400c760 tx=0x7f260400cc20 comp rx=0 tx=0).stop 2024-09-15T21:47:18.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:18.677+0000 7f2618aaf640 1 -- 172.21.15.100:0/1103733724 shutdown_connections 2024-09-15T21:47:18.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:18.677+0000 7f2618aaf640 1 --2- 172.21.15.100:0/1103733724 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f25f406c2b0 0x7f25f406e770 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:18.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:18.677+0000 7f2618aaf640 1 --2- 172.21.15.100:0/1103733724 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f26140c4600 0x7f2614072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:18.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:18.677+0000 7f2618aaf640 1 --2- 172.21.15.100:0/1103733724 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2614152160 0x7f26141525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:18.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:18.677+0000 7f2618aaf640 1 --2- 172.21.15.100:0/1103733724 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2614073790 0x7f2614154250 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:18.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:18.677+0000 7f2618aaf640 1 -- 172.21.15.100:0/1103733724 >> 172.21.15.100:0/1103733724 conn(0x7f26140ca390 msgr2=0x7f26140c83c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:18.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:18.677+0000 7f2618aaf640 1 -- 172.21.15.100:0/1103733724 shutdown_connections 2024-09-15T21:47:18.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:18.677+0000 7f2618aaf640 1 -- 172.21.15.100:0/1103733724 wait complete. 2024-09-15T21:47:18.684 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:47:18.739 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:18.738+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:18.832 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:18.833 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:18.840 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:18.839+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:18.841 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:47:18.941 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:18.940+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:18.945 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:18.945 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:18.954 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:47:19.042 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:19.042+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:19.097 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:19.097 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:19.097 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:19.097 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:19.106 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:47:19.143 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:19.142+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:19.212 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:19.212 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:19.212 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:19.212 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:19.212 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:19.222 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:47:19.266 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:19.264+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:19.333 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:19.334 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:19.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:19.334 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:19.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.334 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:19.334 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:19.334 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.916864853, 2024-09-15T21:47:19.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0050344960000000003, 2024-09-15T21:47:19.334 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.334 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.334 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.334 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:19.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:19.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:19.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0199999999999998e-07 2024-09-15T21:47:19.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413470+0000", 2024-09-15T21:47:19.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4329999999999998e-06 2024-09-15T21:47:19.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413579+0000", 2024-09-15T21:47:19.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108593 2024-09-15T21:47:19.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413587+0000", 2024-09-15T21:47:19.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3100000000000001e-06 2024-09-15T21:47:19.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413691+0000", 2024-09-15T21:47:19.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010357 2024-09-15T21:47:19.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413887+0000", 2024-09-15T21:47:19.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019666799999999999 2024-09-15T21:47:19.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:19.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414027+0000", 2024-09-15T21:47:19.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013996700000000001 2024-09-15T21:47:19.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416360+0000", 2024-09-15T21:47:19.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002332204 2024-09-15T21:47:19.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418437+0000", 2024-09-15T21:47:19.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002077279 2024-09-15T21:47:19.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418469+0000", 2024-09-15T21:47:19.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2359e-05 2024-09-15T21:47:19.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418502+0000", 2024-09-15T21:47:19.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2811000000000001e-05 2024-09-15T21:47:19.340 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.340 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.340 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.341 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:19.341 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:19.341 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.916860813, 2024-09-15T21:47:19.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0053538780000000003, 2024-09-15T21:47:19.341 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.341 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.341 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.341 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:19.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:19.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:19.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2000000000000001e-07 2024-09-15T21:47:19.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413474+0000", 2024-09-15T21:47:19.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-15T21:47:19.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413612+0000", 2024-09-15T21:47:19.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000137727 2024-09-15T21:47:19.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413618+0000", 2024-09-15T21:47:19.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.241e-06 2024-09-15T21:47:19.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414443+0000", 2024-09-15T21:47:19.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082447 2024-09-15T21:47:19.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414523+0000", 2024-09-15T21:47:19.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0059e-05 2024-09-15T21:47:19.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:19.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414653+0000", 2024-09-15T21:47:19.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013031699999999999 2024-09-15T21:47:19.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.345 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416366+0000", 2024-09-15T21:47:19.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001712917 2024-09-15T21:47:19.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.345 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418769+0000", 2024-09-15T21:47:19.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024031529999999999 2024-09-15T21:47:19.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418802+0000", 2024-09-15T21:47:19.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2925000000000001e-05 2024-09-15T21:47:19.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.346 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418825+0000", 2024-09-15T21:47:19.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3204e-05 2024-09-15T21:47:19.346 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.346 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.346 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.346 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:19.347 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:19.347 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.916852792, 2024-09-15T21:47:19.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059708640000000002, 2024-09-15T21:47:19.347 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.347 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.347 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.347 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:19.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:19.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:19.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7300000000000002e-07 2024-09-15T21:47:19.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413482+0000", 2024-09-15T21:47:19.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2359999999999999e-06 2024-09-15T21:47:19.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414323+0000", 2024-09-15T21:47:19.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00084095399999999999 2024-09-15T21:47:19.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414332+0000", 2024-09-15T21:47:19.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0880000000000001e-06 2024-09-15T21:47:19.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415439+0000", 2024-09-15T21:47:19.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011071869999999999 2024-09-15T21:47:19.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415554+0000", 2024-09-15T21:47:19.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114499 2024-09-15T21:47:19.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:19.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415682+0000", 2024-09-15T21:47:19.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012802500000000001 2024-09-15T21:47:19.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.417957+0000", 2024-09-15T21:47:19.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022746720000000002 2024-09-15T21:47:19.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419385+0000", 2024-09-15T21:47:19.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001428826 2024-09-15T21:47:19.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419420+0000", 2024-09-15T21:47:19.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4875000000000001e-05 2024-09-15T21:47:19.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419450+0000", 2024-09-15T21:47:19.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0128999999999999e-05 2024-09-15T21:47:19.353 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.353 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.353 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.353 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:19.353 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:19.353 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.916848973999997, 2024-09-15T21:47:19.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0087186220000000005, 2024-09-15T21:47:19.353 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.353 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.353 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.353 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:19.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:19.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413484+0000", 2024-09-15T21:47:19.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-15T21:47:19.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413487+0000", 2024-09-15T21:47:19.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.534e-06 2024-09-15T21:47:19.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414353+0000", 2024-09-15T21:47:19.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00086616999999999998 2024-09-15T21:47:19.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414359+0000", 2024-09-15T21:47:19.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3279999999999996e-06 2024-09-15T21:47:19.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416041+0000", 2024-09-15T21:47:19.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001682467 2024-09-15T21:47:19.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416158+0000", 2024-09-15T21:47:19.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117242 2024-09-15T21:47:19.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:19.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416284+0000", 2024-09-15T21:47:19.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012509000000000001 2024-09-15T21:47:19.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.417992+0000", 2024-09-15T21:47:19.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017087059999999999 2024-09-15T21:47:19.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422142+0000", 2024-09-15T21:47:19.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041501180000000004 2024-09-15T21:47:19.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422176+0000", 2024-09-15T21:47:19.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4109999999999997e-05 2024-09-15T21:47:19.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422202+0000", 2024-09-15T21:47:19.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5585e-05 2024-09-15T21:47:19.358 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.358 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.358 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.359 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:19.359 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:19.359 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.916843548999999, 2024-09-15T21:47:19.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0088000370000000001, 2024-09-15T21:47:19.359 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.359 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.359 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:19.359 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:19.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:19.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:19.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:19.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413491+0000", 2024-09-15T21:47:19.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.773e-06 2024-09-15T21:47:19.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414372+0000", 2024-09-15T21:47:19.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00088122000000000005 2024-09-15T21:47:19.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414378+0000", 2024-09-15T21:47:19.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4380000000000003e-06 2024-09-15T21:47:19.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416631+0000", 2024-09-15T21:47:19.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022530319999999999 2024-09-15T21:47:19.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416765+0000", 2024-09-15T21:47:19.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013423499999999999 2024-09-15T21:47:19.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422289+0000", 2024-09-15T21:47:19.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055240530000000001 2024-09-15T21:47:19.362 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.363 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.363 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.363 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:19.363 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:19.363 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.87527523, 2024-09-15T21:47:19.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056900609999999997, 2024-09-15T21:47:19.363 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.363 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.363 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.363 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:19.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:19.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455058+0000", 2024-09-15T21:47:19.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.51e-07 2024-09-15T21:47:19.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455061+0000", 2024-09-15T21:47:19.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.145e-06 2024-09-15T21:47:19.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455158+0000", 2024-09-15T21:47:19.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7101000000000004e-05 2024-09-15T21:47:19.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455165+0000", 2024-09-15T21:47:19.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1019999999999999e-06 2024-09-15T21:47:19.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456433+0000", 2024-09-15T21:47:19.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012678259999999999 2024-09-15T21:47:19.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456517+0000", 2024-09-15T21:47:19.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4373e-05 2024-09-15T21:47:19.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:19.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456677+0000", 2024-09-15T21:47:19.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015963300000000001 2024-09-15T21:47:19.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458636+0000", 2024-09-15T21:47:19.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019594030000000002 2024-09-15T21:47:19.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460665+0000", 2024-09-15T21:47:19.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020283549999999999 2024-09-15T21:47:19.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460707+0000", 2024-09-15T21:47:19.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2398000000000003e-05 2024-09-15T21:47:19.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460747+0000", 2024-09-15T21:47:19.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0274000000000001e-05 2024-09-15T21:47:19.369 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.369 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.369 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.369 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:19.369 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:19.369 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.875268036000001, 2024-09-15T21:47:19.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086420030000000005, 2024-09-15T21:47:19.370 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.370 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.370 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.370 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:19.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:19.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455065+0000", 2024-09-15T21:47:19.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3700000000000001e-07 2024-09-15T21:47:19.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455068+0000", 2024-09-15T21:47:19.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.915e-06 2024-09-15T21:47:19.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455366+0000", 2024-09-15T21:47:19.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00029825400000000002 2024-09-15T21:47:19.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455374+0000", 2024-09-15T21:47:19.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6240000000000008e-06 2024-09-15T21:47:19.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456199+0000", 2024-09-15T21:47:19.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082448500000000002 2024-09-15T21:47:19.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457034+0000", 2024-09-15T21:47:19.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00083553399999999997 2024-09-15T21:47:19.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:19.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457171+0000", 2024-09-15T21:47:19.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013653600000000001 2024-09-15T21:47:19.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462663+0000", 2024-09-15T21:47:19.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054924420000000002 2024-09-15T21:47:19.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463613+0000", 2024-09-15T21:47:19.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094971499999999996 2024-09-15T21:47:19.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463663+0000", 2024-09-15T21:47:19.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9636999999999999e-05 2024-09-15T21:47:19.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463706+0000", 2024-09-15T21:47:19.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3523999999999999e-05 2024-09-15T21:47:19.375 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.375 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.375 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.375 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:19.375 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:19.375 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.875263508000003, 2024-09-15T21:47:19.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090408420000000003, 2024-09-15T21:47:19.375 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.375 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.376 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.376 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:19.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:19.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:19.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.79e-07 2024-09-15T21:47:19.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455072+0000", 2024-09-15T21:47:19.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6189999999999998e-06 2024-09-15T21:47:19.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455403+0000", 2024-09-15T21:47:19.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033167299999999999 2024-09-15T21:47:19.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455420+0000", 2024-09-15T21:47:19.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6518999999999999e-05 2024-09-15T21:47:19.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457448+0000", 2024-09-15T21:47:19.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020275710000000001 2024-09-15T21:47:19.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:19.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3844999999999994e-05 2024-09-15T21:47:19.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:19.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457684+0000", 2024-09-15T21:47:19.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015239 2024-09-15T21:47:19.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462678+0000", 2024-09-15T21:47:19.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049939279999999999 2024-09-15T21:47:19.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464017+0000", 2024-09-15T21:47:19.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013397140000000001 2024-09-15T21:47:19.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464071+0000", 2024-09-15T21:47:19.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3735e-05 2024-09-15T21:47:19.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464110+0000", 2024-09-15T21:47:19.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8569000000000003e-05 2024-09-15T21:47:19.381 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.381 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.381 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.381 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:19.381 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:19.381 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.875251894999998, 2024-09-15T21:47:19.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091802889999999995, 2024-09-15T21:47:19.381 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.381 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.382 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.382 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:19.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:19.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455081+0000", 2024-09-15T21:47:19.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:19.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455084+0000", 2024-09-15T21:47:19.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7810000000000001e-06 2024-09-15T21:47:19.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455536+0000", 2024-09-15T21:47:19.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00045219100000000001 2024-09-15T21:47:19.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455583+0000", 2024-09-15T21:47:19.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7308999999999997e-05 2024-09-15T21:47:19.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457998+0000", 2024-09-15T21:47:19.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024154290000000002 2024-09-15T21:47:19.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458147+0000", 2024-09-15T21:47:19.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014834599999999999 2024-09-15T21:47:19.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:19.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458265+0000", 2024-09-15T21:47:19.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118458 2024-09-15T21:47:19.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464134+0000", 2024-09-15T21:47:19.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005869038 2024-09-15T21:47:19.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464187+0000", 2024-09-15T21:47:19.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2343999999999999e-05 2024-09-15T21:47:19.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464222+0000", 2024-09-15T21:47:19.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5447000000000001e-05 2024-09-15T21:47:19.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464261+0000", 2024-09-15T21:47:19.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8659999999999999e-05 2024-09-15T21:47:19.387 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.387 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.387 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.387 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:19.387 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:19.387 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 38.854629516000003, 2024-09-15T21:47:19.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048468290000000004, 2024-09-15T21:47:19.387 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.388 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.388 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.388 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:19.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:19.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:19.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.35e-07 2024-09-15T21:47:19.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475706+0000", 2024-09-15T21:47:19.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.994e-06 2024-09-15T21:47:19.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475770+0000", 2024-09-15T21:47:19.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3904000000000003e-05 2024-09-15T21:47:19.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475775+0000", 2024-09-15T21:47:19.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2490000000000001e-06 2024-09-15T21:47:19.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476502+0000", 2024-09-15T21:47:19.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072697300000000001 2024-09-15T21:47:19.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476611+0000", 2024-09-15T21:47:19.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0001091 2024-09-15T21:47:19.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:19.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476807+0000", 2024-09-15T21:47:19.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000195892 2024-09-15T21:47:19.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479678+0000", 2024-09-15T21:47:19.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028704709999999999 2024-09-15T21:47:19.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480444+0000", 2024-09-15T21:47:19.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00076602099999999998 2024-09-15T21:47:19.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480502+0000", 2024-09-15T21:47:19.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8313999999999998e-05 2024-09-15T21:47:19.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480550+0000", 2024-09-15T21:47:19.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7676000000000002e-05 2024-09-15T21:47:19.393 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.393 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.393 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.393 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:19.393 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:19.393 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 38.854622341000002, 2024-09-15T21:47:19.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051250929999999998, 2024-09-15T21:47:19.393 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.393 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.394 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:19.394 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:19.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:19.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:19.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1e-07 2024-09-15T21:47:19.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475723+0000", 2024-09-15T21:47:19.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2758000000000001e-05 2024-09-15T21:47:19.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475972+0000", 2024-09-15T21:47:19.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024886299999999999 2024-09-15T21:47:19.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475979+0000", 2024-09-15T21:47:19.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0230000000000004e-06 2024-09-15T21:47:19.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476560+0000", 2024-09-15T21:47:19.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00058097599999999998 2024-09-15T21:47:19.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476689+0000", 2024-09-15T21:47:19.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129486 2024-09-15T21:47:19.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480835+0000", 2024-09-15T21:47:19.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041457769999999998 2024-09-15T21:47:19.397 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.397 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.397 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.397 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:19.398 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:19.398 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.785707115999998, 2024-09-15T21:47:19.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054420570000000001, 2024-09-15T21:47:19.398 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.398 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.398 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.398 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:19.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:19.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544626+0000", 2024-09-15T21:47:19.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8100000000000003e-07 2024-09-15T21:47:19.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544629+0000", 2024-09-15T21:47:19.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9869999999999999e-06 2024-09-15T21:47:19.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544775+0000", 2024-09-15T21:47:19.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014610599999999999 2024-09-15T21:47:19.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544782+0000", 2024-09-15T21:47:19.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4109999999999999e-06 2024-09-15T21:47:19.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545617+0000", 2024-09-15T21:47:19.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00083468300000000002 2024-09-15T21:47:19.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545690+0000", 2024-09-15T21:47:19.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3545000000000001e-05 2024-09-15T21:47:19.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:19.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545803+0000", 2024-09-15T21:47:19.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000112083 2024-09-15T21:47:19.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548353+0000", 2024-09-15T21:47:19.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025507220000000001 2024-09-15T21:47:19.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549964+0000", 2024-09-15T21:47:19.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001610382 2024-09-15T21:47:19.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550021+0000", 2024-09-15T21:47:19.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6898999999999999e-05 2024-09-15T21:47:19.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550067+0000", 2024-09-15T21:47:19.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758000000000002e-05 2024-09-15T21:47:19.403 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.403 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.403 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.403 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:19.403 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:19.403 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.785701838000001, 2024-09-15T21:47:19.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056323399999999996, 2024-09-15T21:47:19.404 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.404 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.404 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.404 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:19.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:19.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:19.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:47:19.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544634+0000", 2024-09-15T21:47:19.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.58e-06 2024-09-15T21:47:19.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544800+0000", 2024-09-15T21:47:19.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016576899999999999 2024-09-15T21:47:19.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544808+0000", 2024-09-15T21:47:19.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5179999999999998e-06 2024-09-15T21:47:19.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546075+0000", 2024-09-15T21:47:19.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012676969999999999 2024-09-15T21:47:19.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546179+0000", 2024-09-15T21:47:19.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000103935 2024-09-15T21:47:19.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:19.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546286+0000", 2024-09-15T21:47:19.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106758 2024-09-15T21:47:19.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548528+0000", 2024-09-15T21:47:19.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022420510000000001 2024-09-15T21:47:19.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550119+0000", 2024-09-15T21:47:19.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015907079999999999 2024-09-15T21:47:19.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550202+0000", 2024-09-15T21:47:19.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3374000000000005e-05 2024-09-15T21:47:19.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550263+0000", 2024-09-15T21:47:19.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0643999999999997e-05 2024-09-15T21:47:19.409 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.409 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.409 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.409 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:19.409 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:19.409 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.785696866999999, 2024-09-15T21:47:19.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061905470000000002, 2024-09-15T21:47:19.409 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.409 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.410 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.410 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:19.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:19.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:19.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6899999999999999e-07 2024-09-15T21:47:19.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544638+0000", 2024-09-15T21:47:19.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3920000000000001e-06 2024-09-15T21:47:19.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544819+0000", 2024-09-15T21:47:19.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018065099999999999 2024-09-15T21:47:19.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544825+0000", 2024-09-15T21:47:19.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7239999999999997e-06 2024-09-15T21:47:19.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546583+0000", 2024-09-15T21:47:19.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017588219999999999 2024-09-15T21:47:19.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546665+0000", 2024-09-15T21:47:19.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1693000000000005e-05 2024-09-15T21:47:19.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:19.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546808+0000", 2024-09-15T21:47:19.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142993 2024-09-15T21:47:19.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548533+0000", 2024-09-15T21:47:19.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017246 2024-09-15T21:47:19.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550606+0000", 2024-09-15T21:47:19.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020733969999999998 2024-09-15T21:47:19.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550783+0000", 2024-09-15T21:47:19.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000176565 2024-09-15T21:47:19.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550826+0000", 2024-09-15T21:47:19.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3441e-05 2024-09-15T21:47:19.415 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.415 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.415 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.415 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:19.415 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:19.415 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.785689146000003, 2024-09-15T21:47:19.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052813010000000004, 2024-09-15T21:47:19.415 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.415 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.415 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.415 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:19.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:19.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:19.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3799999999999999e-07 2024-09-15T21:47:19.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544647+0000", 2024-09-15T21:47:19.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6569999999999999e-06 2024-09-15T21:47:19.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544858+0000", 2024-09-15T21:47:19.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021069900000000001 2024-09-15T21:47:19.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544862+0000", 2024-09-15T21:47:19.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4900000000000002e-06 2024-09-15T21:47:19.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545610+0000", 2024-09-15T21:47:19.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00074786999999999998 2024-09-15T21:47:19.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545706+0000", 2024-09-15T21:47:19.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6271e-05 2024-09-15T21:47:19.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545834+0000", 2024-09-15T21:47:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012759499999999999 2024-09-15T21:47:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548554+0000", 2024-09-15T21:47:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027199339999999998 2024-09-15T21:47:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549818+0000", 2024-09-15T21:47:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012642199999999999 2024-09-15T21:47:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549884+0000", 2024-09-15T21:47:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5843e-05 2024-09-15T21:47:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549925+0000", 2024-09-15T21:47:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0484000000000001e-05 2024-09-15T21:47:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.420 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.421 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.785680468999999, 2024-09-15T21:47:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051106110000000001, 2024-09-15T21:47:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2700000000000001e-07 2024-09-15T21:47:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544667+0000", 2024-09-15T21:47:19.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4746000000000001e-05 2024-09-15T21:47:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545021+0000", 2024-09-15T21:47:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035412900000000001 2024-09-15T21:47:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545052+0000", 2024-09-15T21:47:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1158000000000002e-05 2024-09-15T21:47:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546050+0000", 2024-09-15T21:47:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00099782800000000008 2024-09-15T21:47:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546291+0000", 2024-09-15T21:47:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024093500000000001 2024-09-15T21:47:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546434+0000", 2024-09-15T21:47:19.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014308600000000001 2024-09-15T21:47:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548583+0000", 2024-09-15T21:47:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021491029999999999 2024-09-15T21:47:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549695+0000", 2024-09-15T21:47:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011116419999999999 2024-09-15T21:47:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549726+0000", 2024-09-15T21:47:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1195000000000002e-05 2024-09-15T21:47:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549763+0000", 2024-09-15T21:47:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6562e-05 2024-09-15T21:47:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.428 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:17 2.14 2:2eebc8f4:::62f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:19.428 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:19.428 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.765250857000002, 2024-09-15T21:47:19.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071004800000000002, 2024-09-15T21:47:19.428 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.428 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.428 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.428 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:19.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:19.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565083+0000", 2024-09-15T21:47:19.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.689e-06 2024-09-15T21:47:19.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565090+0000", 2024-09-15T21:47:19.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5930000000000002e-06 2024-09-15T21:47:19.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565119+0000", 2024-09-15T21:47:19.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8785999999999998e-05 2024-09-15T21:47:19.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565126+0000", 2024-09-15T21:47:19.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0110000000000003e-06 2024-09-15T21:47:19.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565237+0000", 2024-09-15T21:47:19.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011161 2024-09-15T21:47:19.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565383+0000", 2024-09-15T21:47:19.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014626900000000001 2024-09-15T21:47:19.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:19.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565520+0000", 2024-09-15T21:47:19.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013676399999999999 2024-09-15T21:47:19.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569909+0000", 2024-09-15T21:47:19.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043890149999999996 2024-09-15T21:47:19.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572096+0000", 2024-09-15T21:47:19.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021866559999999999 2024-09-15T21:47:19.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572144+0000", 2024-09-15T21:47:19.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7725000000000001e-05 2024-09-15T21:47:19.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572182+0000", 2024-09-15T21:47:19.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8362000000000003e-05 2024-09-15T21:47:19.433 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.433 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.433 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.433 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:19.433 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:19.433 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.765239759, 2024-09-15T21:47:19.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072110280000000004, 2024-09-15T21:47:19.434 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.434 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.434 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.434 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:19.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:19.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:19.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4299999999999999e-07 2024-09-15T21:47:19.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565096+0000", 2024-09-15T21:47:19.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4319999999999999e-06 2024-09-15T21:47:19.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565152+0000", 2024-09-15T21:47:19.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5686999999999997e-05 2024-09-15T21:47:19.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565157+0000", 2024-09-15T21:47:19.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2700000000000004e-06 2024-09-15T21:47:19.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565847+0000", 2024-09-15T21:47:19.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068937299999999996 2024-09-15T21:47:19.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565940+0000", 2024-09-15T21:47:19.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3573999999999996e-05 2024-09-15T21:47:19.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:19.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566085+0000", 2024-09-15T21:47:19.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000144902 2024-09-15T21:47:19.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570595+0000", 2024-09-15T21:47:19.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045103469999999996 2024-09-15T21:47:19.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572224+0000", 2024-09-15T21:47:19.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016282930000000001 2024-09-15T21:47:19.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572260+0000", 2024-09-15T21:47:19.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6103999999999999e-05 2024-09-15T21:47:19.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572304+0000", 2024-09-15T21:47:19.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3803000000000001e-05 2024-09-15T21:47:19.439 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.439 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.439 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.439 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:19.439 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:19.439 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.765235294, 2024-09-15T21:47:19.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055995389999999997, 2024-09-15T21:47:19.439 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.439 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.440 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.440 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:19.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:19.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:19.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9999999999999999e-07 2024-09-15T21:47:19.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:19.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2399999999999999e-06 2024-09-15T21:47:19.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565280+0000", 2024-09-15T21:47:19.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017929400000000001 2024-09-15T21:47:19.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565292+0000", 2024-09-15T21:47:19.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2193e-05 2024-09-15T21:47:19.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565357+0000", 2024-09-15T21:47:19.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4684000000000006e-05 2024-09-15T21:47:19.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565499+0000", 2024-09-15T21:47:19.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142032 2024-09-15T21:47:19.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:19.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565634+0000", 2024-09-15T21:47:19.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000135613 2024-09-15T21:47:19.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569941+0000", 2024-09-15T21:47:19.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043066789999999999 2024-09-15T21:47:19.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570585+0000", 2024-09-15T21:47:19.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00064364700000000001 2024-09-15T21:47:19.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570647+0000", 2024-09-15T21:47:19.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2173000000000003e-05 2024-09-15T21:47:19.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570697+0000", 2024-09-15T21:47:19.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9784000000000003e-05 2024-09-15T21:47:19.444 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.445 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.445 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.445 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:19.445 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:19.445 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 27.765231063999998, 2024-09-15T21:47:19.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066548939999999997, 2024-09-15T21:47:19.445 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.445 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.445 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.445 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:19.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:19.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565102+0000", 2024-09-15T21:47:19.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-15T21:47:19.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565103+0000", 2024-09-15T21:47:19.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6700000000000001e-06 2024-09-15T21:47:19.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565608+0000", 2024-09-15T21:47:19.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050518299999999998 2024-09-15T21:47:19.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565616+0000", 2024-09-15T21:47:19.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8199999999999997e-06 2024-09-15T21:47:19.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566598+0000", 2024-09-15T21:47:19.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00098176300000000008 2024-09-15T21:47:19.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566871+0000", 2024-09-15T21:47:19.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027312799999999998 2024-09-15T21:47:19.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:19.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567015+0000", 2024-09-15T21:47:19.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014428499999999999 2024-09-15T21:47:19.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570729+0000", 2024-09-15T21:47:19.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037132459999999999 2024-09-15T21:47:19.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571667+0000", 2024-09-15T21:47:19.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00093862299999999997 2024-09-15T21:47:19.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571718+0000", 2024-09-15T21:47:19.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1146999999999997e-05 2024-09-15T21:47:19.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571756+0000", 2024-09-15T21:47:19.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7821000000000003e-05 2024-09-15T21:47:19.450 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.450 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.450 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.451 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.451 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.451 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:19.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:19.405+0000 7fc48ffff640 1 -- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fc49802c640 msgr2=0x7fc49802caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.122:6824/2400668532 2024-09-15T21:47:19.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:19.405+0000 7fc48ffff640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fc49802c640 0x7fc49802caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-15T21:47:19.453 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:47:19.453 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:19.450+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:19.553 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:19.552+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:19.597 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:19.597 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:19.597 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:19.597 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:19.605 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:47:19.654 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:19.653+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:19.749 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:19.749 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:19.749 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:19.749 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:19.749 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:19.758 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:47:19.758 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:19.756+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:19.858 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:19.857+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:19.863 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:19.863 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:19.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:19.863 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:19.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.864 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:47:19.864 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:19.864 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 66.626268851000006, 2024-09-15T21:47:19.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0944283340000001, 2024-09-15T21:47:19.864 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.864 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.864 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.864 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:19.864 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:19.864 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:19.864 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2024-09-15T21:47:19.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.865 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:19.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:19.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234360+0000", 2024-09-15T21:47:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3710000000000001e-06 2024-09-15T21:47:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234374+0000", 2024-09-15T21:47:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4003e-05 2024-09-15T21:47:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234392+0000", 2024-09-15T21:47:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7294000000000001e-05 2024-09-15T21:47:19.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234405+0000", 2024-09-15T21:47:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2961e-05 2024-09-15T21:47:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234483+0000", 2024-09-15T21:47:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8850000000000006e-05 2024-09-15T21:47:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-15T21:47:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234491+0000", 2024-09-15T21:47:19.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3529999999999997e-06 2024-09-15T21:47:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327253+0000", 2024-09-15T21:47:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.092761962 2024-09-15T21:47:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327456+0000", 2024-09-15T21:47:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000202893 2024-09-15T21:47:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327652+0000", 2024-09-15T21:47:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019613000000000001 2024-09-15T21:47:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328368+0000", 2024-09-15T21:47:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000716582 2024-09-15T21:47:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328682+0000", 2024-09-15T21:47:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000313825 2024-09-15T21:47:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328743+0000", 2024-09-15T21:47:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0571e-05 2024-09-15T21:47:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328787+0000", 2024-09-15T21:47:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4539000000000002e-05 2024-09-15T21:47:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.870 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 65.505103449000003, 2024-09-15T21:47:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0089551270000000002, 2024-09-15T21:47:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 35 2024-09-15T21:47:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355525+0000", 2024-09-15T21:47:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:47:19.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355606+0000", 2024-09-15T21:47:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1723000000000003e-05 2024-09-15T21:47:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355624+0000", 2024-09-15T21:47:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7645000000000001e-05 2024-09-15T21:47:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355629+0000", 2024-09-15T21:47:19.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7319999999999998e-06 2024-09-15T21:47:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358315+0000", 2024-09-15T21:47:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026858659999999999 2024-09-15T21:47:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358373+0000", 2024-09-15T21:47:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8593e-05 2024-09-15T21:47:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358521+0000", 2024-09-15T21:47:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014817500000000001 2024-09-15T21:47:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360353+0000", 2024-09-15T21:47:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018317100000000001 2024-09-15T21:47:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364424+0000", 2024-09-15T21:47:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040707970000000001 2024-09-15T21:47:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364452+0000", 2024-09-15T21:47:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7937999999999999e-05 2024-09-15T21:47:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364479+0000", 2024-09-15T21:47:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7744999999999999e-05 2024-09-15T21:47:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:19.876 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 65.505019485999995, 2024-09-15T21:47:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090267869999999997, 2024-09-15T21:47:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 36 2024-09-15T21:47:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355609+0000", 2024-09-15T21:47:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3099999999999998e-07 2024-09-15T21:47:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355612+0000", 2024-09-15T21:47:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1489999999999998e-06 2024-09-15T21:47:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355637+0000", 2024-09-15T21:47:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.508e-05 2024-09-15T21:47:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355640+0000", 2024-09-15T21:47:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7930000000000002e-06 2024-09-15T21:47:19.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359195+0000", 2024-09-15T21:47:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035555529999999999 2024-09-15T21:47:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359283+0000", 2024-09-15T21:47:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8042999999999997e-05 2024-09-15T21:47:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359419+0000", 2024-09-15T21:47:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013576699999999999 2024-09-15T21:47:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360361+0000", 2024-09-15T21:47:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094171000000000001 2024-09-15T21:47:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364536+0000", 2024-09-15T21:47:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00417484 2024-09-15T21:47:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364590+0000", 2024-09-15T21:47:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4355999999999999e-05 2024-09-15T21:47:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364635+0000", 2024-09-15T21:47:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5065000000000003e-05 2024-09-15T21:47:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 65.505014880999994, 2024-09-15T21:47:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091162750000000001, 2024-09-15T21:47:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 37 2024-09-15T21:47:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:47:19.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355615+0000", 2024-09-15T21:47:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-15T21:47:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355646+0000", 2024-09-15T21:47:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0391000000000001e-05 2024-09-15T21:47:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355648+0000", 2024-09-15T21:47:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6309999999999999e-06 2024-09-15T21:47:19.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359770+0000", 2024-09-15T21:47:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041222899999999998 2024-09-15T21:47:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359854+0000", 2024-09-15T21:47:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3399000000000006e-05 2024-09-15T21:47:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359988+0000", 2024-09-15T21:47:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013442200000000001 2024-09-15T21:47:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360524+0000", 2024-09-15T21:47:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00053582900000000004 2024-09-15T21:47:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364673+0000", 2024-09-15T21:47:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041493199999999997 2024-09-15T21:47:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364704+0000", 2024-09-15T21:47:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0380999999999999e-05 2024-09-15T21:47:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364729+0000", 2024-09-15T21:47:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5335999999999999e-05 2024-09-15T21:47:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.888 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 65.436297643000003, 2024-09-15T21:47:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0026270020000001, 2024-09-15T21:47:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 88 2024-09-15T21:47:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424331+0000", 2024-09-15T21:47:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4299999999999998e-07 2024-09-15T21:47:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424333+0000", 2024-09-15T21:47:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2359999999999999e-06 2024-09-15T21:47:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422895+0000", 2024-09-15T21:47:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99856184199999998 2024-09-15T21:47:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422902+0000", 2024-09-15T21:47:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4440000000000002e-06 2024-09-15T21:47:19.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423053+0000", 2024-09-15T21:47:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000150958 2024-09-15T21:47:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423160+0000", 2024-09-15T21:47:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106774 2024-09-15T21:47:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423344+0000", 2024-09-15T21:47:19.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018424199999999999 2024-09-15T21:47:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.424730+0000", 2024-09-15T21:47:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001386155 2024-09-15T21:47:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426756+0000", 2024-09-15T21:47:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020261150000000002 2024-09-15T21:47:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426906+0000", 2024-09-15T21:47:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000149176 2024-09-15T21:47:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426957+0000", 2024-09-15T21:47:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1616999999999997e-05 2024-09-15T21:47:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:19.894 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.306224444999998, 2024-09-15T21:47:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095037790000000004, 2024-09-15T21:47:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 114 2024-09-15T21:47:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554404+0000", 2024-09-15T21:47:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2399999999999999e-07 2024-09-15T21:47:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554407+0000", 2024-09-15T21:47:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-15T21:47:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554632+0000", 2024-09-15T21:47:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022568999999999999 2024-09-15T21:47:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554640+0000", 2024-09-15T21:47:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7970000000000001e-06 2024-09-15T21:47:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.898 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.898 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561325+0000", 2024-09-15T21:47:19.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066848020000000001 2024-09-15T21:47:19.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.898 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.898 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.898 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561402+0000", 2024-09-15T21:47:19.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6851e-05 2024-09-15T21:47:19.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561530+0000", 2024-09-15T21:47:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012863999999999999 2024-09-15T21:47:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563144+0000", 2024-09-15T21:47:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016132849999999999 2024-09-15T21:47:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563800+0000", 2024-09-15T21:47:19.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00065586099999999999 2024-09-15T21:47:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563857+0000", 2024-09-15T21:47:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.702e-05 2024-09-15T21:47:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563907+0000", 2024-09-15T21:47:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0479000000000002e-05 2024-09-15T21:47:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.900 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.306219447999993, 2024-09-15T21:47:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011813423999999999, 2024-09-15T21:47:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:19.901 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 115 2024-09-15T21:47:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554409+0000", 2024-09-15T21:47:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1100000000000002e-07 2024-09-15T21:47:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554498+0000", 2024-09-15T21:47:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9642999999999995e-05 2024-09-15T21:47:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554656+0000", 2024-09-15T21:47:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000158114 2024-09-15T21:47:19.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554661+0000", 2024-09-15T21:47:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0730000000000004e-06 2024-09-15T21:47:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561862+0000", 2024-09-15T21:47:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072002999999999998 2024-09-15T21:47:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561939+0000", 2024-09-15T21:47:19.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7265e-05 2024-09-15T21:47:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562073+0000", 2024-09-15T21:47:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013371600000000001 2024-09-15T21:47:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563196+0000", 2024-09-15T21:47:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011231679999999999 2024-09-15T21:47:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566124+0000", 2024-09-15T21:47:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002928403 2024-09-15T21:47:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566183+0000", 2024-09-15T21:47:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8328000000000001e-05 2024-09-15T21:47:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566222+0000", 2024-09-15T21:47:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9103000000000002e-05 2024-09-15T21:47:19.906 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.306126008000007, 2024-09-15T21:47:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01181426, 2024-09-15T21:47:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:19.907 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 116 2024-09-15T21:47:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554503+0000", 2024-09-15T21:47:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8000000000000005e-07 2024-09-15T21:47:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554506+0000", 2024-09-15T21:47:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5899999999999999e-06 2024-09-15T21:47:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554674+0000", 2024-09-15T21:47:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016763600000000001 2024-09-15T21:47:19.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554678+0000", 2024-09-15T21:47:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5310000000000003e-06 2024-09-15T21:47:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562714+0000", 2024-09-15T21:47:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.008035258 2024-09-15T21:47:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562765+0000", 2024-09-15T21:47:19.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1093e-05 2024-09-15T21:47:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562845+0000", 2024-09-15T21:47:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0427999999999996e-05 2024-09-15T21:47:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563943+0000", 2024-09-15T21:47:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097647 2024-09-15T21:47:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566232+0000", 2024-09-15T21:47:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022890850000000002 2024-09-15T21:47:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566279+0000", 2024-09-15T21:47:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6692e-05 2024-09-15T21:47:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566316+0000", 2024-09-15T21:47:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.752e-05 2024-09-15T21:47:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.912 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.306119909000003, 2024-09-15T21:47:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012232125, 2024-09-15T21:47:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:19.913 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 117 2024-09-15T21:47:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.914 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7700000000000001e-07 2024-09-15T21:47:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554511+0000", 2024-09-15T21:47:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8430000000000001e-06 2024-09-15T21:47:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554690+0000", 2024-09-15T21:47:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017856299999999999 2024-09-15T21:47:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.915 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554698+0000", 2024-09-15T21:47:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6200000000000005e-06 2024-09-15T21:47:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563224+0000", 2024-09-15T21:47:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0085262640000000004 2024-09-15T21:47:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563342+0000", 2024-09-15T21:47:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117687 2024-09-15T21:47:19.916 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563472+0000", 2024-09-15T21:47:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012957999999999999 2024-09-15T21:47:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566367+0000", 2024-09-15T21:47:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028952280000000001 2024-09-15T21:47:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566680+0000", 2024-09-15T21:47:19.917 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031345100000000002 2024-09-15T21:47:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566714+0000", 2024-09-15T21:47:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3701000000000001e-05 2024-09-15T21:47:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566740+0000", 2024-09-15T21:47:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5911e-05 2024-09-15T21:47:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.963227826999997, 2024-09-15T21:47:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069806590000000002, 2024-09-15T21:47:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 134 2024-09-15T21:47:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.919 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897401+0000", 2024-09-15T21:47:19.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0500000000000003e-07 2024-09-15T21:47:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897492+0000", 2024-09-15T21:47:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1770999999999999e-05 2024-09-15T21:47:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897750+0000", 2024-09-15T21:47:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025778299999999998 2024-09-15T21:47:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.921 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897756+0000", 2024-09-15T21:47:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2140000000000001e-06 2024-09-15T21:47:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899312+0000", 2024-09-15T21:47:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001556144 2024-09-15T21:47:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899372+0000", 2024-09-15T21:47:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0015000000000001e-05 2024-09-15T21:47:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899444+0000", 2024-09-15T21:47:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1923999999999998e-05 2024-09-15T21:47:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901354+0000", 2024-09-15T21:47:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019092530000000001 2024-09-15T21:47:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904327+0000", 2024-09-15T21:47:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029734520000000001 2024-09-15T21:47:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904360+0000", 2024-09-15T21:47:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3216000000000002e-05 2024-09-15T21:47:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904381+0000", 2024-09-15T21:47:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0282e-05 2024-09-15T21:47:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.963131969999999, 2024-09-15T21:47:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070121059999999997, 2024-09-15T21:47:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 135 2024-09-15T21:47:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.925 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897497+0000", 2024-09-15T21:47:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4799999999999997e-07 2024-09-15T21:47:19.926 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897500+0000", 2024-09-15T21:47:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2949999999999998e-06 2024-09-15T21:47:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897781+0000", 2024-09-15T21:47:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028064300000000002 2024-09-15T21:47:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897786+0000", 2024-09-15T21:47:19.927 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5169999999999999e-06 2024-09-15T21:47:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899857+0000", 2024-09-15T21:47:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020705340000000002 2024-09-15T21:47:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899932+0000", 2024-09-15T21:47:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5840999999999994e-05 2024-09-15T21:47:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.928 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900005+0000", 2024-09-15T21:47:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2716e-05 2024-09-15T21:47:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902296+0000", 2024-09-15T21:47:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022912010000000001 2024-09-15T21:47:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904424+0000", 2024-09-15T21:47:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021278600000000001 2024-09-15T21:47:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.929 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904472+0000", 2024-09-15T21:47:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7586000000000001e-05 2024-09-15T21:47:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904508+0000", 2024-09-15T21:47:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6164999999999997e-05 2024-09-15T21:47:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.930 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.963107479000001, 2024-09-15T21:47:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070611429999999998, 2024-09-15T21:47:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 136 2024-09-15T21:47:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.931 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897521+0000", 2024-09-15T21:47:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2900000000000004e-07 2024-09-15T21:47:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897527+0000", 2024-09-15T21:47:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8490000000000002e-06 2024-09-15T21:47:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897799+0000", 2024-09-15T21:47:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027219499999999998 2024-09-15T21:47:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897804+0000", 2024-09-15T21:47:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4219999999999999e-06 2024-09-15T21:47:19.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900248+0000", 2024-09-15T21:47:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002443457 2024-09-15T21:47:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900306+0000", 2024-09-15T21:47:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8263000000000001e-05 2024-09-15T21:47:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900385+0000", 2024-09-15T21:47:19.934 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9161999999999999e-05 2024-09-15T21:47:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902300+0000", 2024-09-15T21:47:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001914947 2024-09-15T21:47:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904536+0000", 2024-09-15T21:47:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022354419999999998 2024-09-15T21:47:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.935 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904562+0000", 2024-09-15T21:47:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6378999999999999e-05 2024-09-15T21:47:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904581+0000", 2024-09-15T21:47:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9497999999999999e-05 2024-09-15T21:47:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:19.936 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.963098912, 2024-09-15T21:47:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007180306, 2024-09-15T21:47:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 137 2024-09-15T21:47:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.937 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1399999999999998e-07 2024-09-15T21:47:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.938 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897612+0000", 2024-09-15T21:47:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2607000000000004e-05 2024-09-15T21:47:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897815+0000", 2024-09-15T21:47:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020352400000000001 2024-09-15T21:47:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897821+0000", 2024-09-15T21:47:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4670000000000003e-06 2024-09-15T21:47:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.939 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900665+0000", 2024-09-15T21:47:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028443689999999998 2024-09-15T21:47:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900714+0000", 2024-09-15T21:47:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8812e-05 2024-09-15T21:47:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900799+0000", 2024-09-15T21:47:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5129000000000004e-05 2024-09-15T21:47:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.940 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903413+0000", 2024-09-15T21:47:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026142890000000001 2024-09-15T21:47:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904625+0000", 2024-09-15T21:47:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012121149999999999 2024-09-15T21:47:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904673+0000", 2024-09-15T21:47:19.941 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7540999999999998e-05 2024-09-15T21:47:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904709+0000", 2024-09-15T21:47:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6139000000000001e-05 2024-09-15T21:47:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:19.942 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.963012956999997, 2024-09-15T21:47:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071727450000000003, 2024-09-15T21:47:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 138 2024-09-15T21:47:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:19.943 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:47:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.944 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897619+0000", 2024-09-15T21:47:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3869999999999999e-06 2024-09-15T21:47:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897832+0000", 2024-09-15T21:47:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021329500000000001 2024-09-15T21:47:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897837+0000", 2024-09-15T21:47:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3730000000000001e-06 2024-09-15T21:47:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.945 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901391+0000", 2024-09-15T21:47:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003553601 2024-09-15T21:47:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901466+0000", 2024-09-15T21:47:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5130999999999996e-05 2024-09-15T21:47:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901601+0000", 2024-09-15T21:47:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134797 2024-09-15T21:47:19.946 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903417+0000", 2024-09-15T21:47:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001816156 2024-09-15T21:47:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904741+0000", 2024-09-15T21:47:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013241049999999999 2024-09-15T21:47:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904767+0000", 2024-09-15T21:47:19.947 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5876000000000001e-05 2024-09-15T21:47:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904788+0000", 2024-09-15T21:47:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0449999999999999e-05 2024-09-15T21:47:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.963007304000001, 2024-09-15T21:47:19.948 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072970250000000004, 2024-09-15T21:47:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 139 2024-09-15T21:47:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897621+0000", 2024-09-15T21:47:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3099999999999999e-07 2024-09-15T21:47:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.950 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897625+0000", 2024-09-15T21:47:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.698e-06 2024-09-15T21:47:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897848+0000", 2024-09-15T21:47:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022375699999999999 2024-09-15T21:47:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897853+0000", 2024-09-15T21:47:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7709999999999997e-06 2024-09-15T21:47:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.951 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901929+0000", 2024-09-15T21:47:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040757780000000004 2024-09-15T21:47:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901976+0000", 2024-09-15T21:47:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6751e-05 2024-09-15T21:47:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902054+0000", 2024-09-15T21:47:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8856999999999994e-05 2024-09-15T21:47:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.952 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903421+0000", 2024-09-15T21:47:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00136619 2024-09-15T21:47:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904827+0000", 2024-09-15T21:47:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0014067369999999999 2024-09-15T21:47:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904881+0000", 2024-09-15T21:47:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3520999999999998e-05 2024-09-15T21:47:19.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.954 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.954 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.954 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904918+0000", 2024-09-15T21:47:19.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6634000000000002e-05 2024-09-15T21:47:19.954 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.954 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.954 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.956 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.956 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:19.956 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:19.956 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.963001474999999, 2024-09-15T21:47:19.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097221730000000006, 2024-09-15T21:47:19.957 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.957 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.957 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.957 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:19.957 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:19.957 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:19.957 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 140 2024-09-15T21:47:19.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.957 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897627+0000", 2024-09-15T21:47:19.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-15T21:47:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897698+0000", 2024-09-15T21:47:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1722000000000002e-05 2024-09-15T21:47:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897863+0000", 2024-09-15T21:47:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016479099999999999 2024-09-15T21:47:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897868+0000", 2024-09-15T21:47:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4950000000000002e-06 2024-09-15T21:47:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902335+0000", 2024-09-15T21:47:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004467632 2024-09-15T21:47:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902414+0000", 2024-09-15T21:47:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8688000000000003e-05 2024-09-15T21:47:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.960 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902535+0000", 2024-09-15T21:47:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012066900000000001 2024-09-15T21:47:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903714+0000", 2024-09-15T21:47:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011794520000000001 2024-09-15T21:47:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907115+0000", 2024-09-15T21:47:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034008879999999999 2024-09-15T21:47:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907250+0000", 2024-09-15T21:47:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134657 2024-09-15T21:47:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907348+0000", 2024-09-15T21:47:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8838000000000005e-05 2024-09-15T21:47:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.962 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 63.962926394, 2024-09-15T21:47:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097618050000000001, 2024-09-15T21:47:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 141 2024-09-15T21:47:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897702+0000", 2024-09-15T21:47:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6199999999999998e-07 2024-09-15T21:47:19.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897705+0000", 2024-09-15T21:47:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0180000000000002e-06 2024-09-15T21:47:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897889+0000", 2024-09-15T21:47:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000183678 2024-09-15T21:47:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897894+0000", 2024-09-15T21:47:19.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8930000000000004e-06 2024-09-15T21:47:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903099+0000", 2024-09-15T21:47:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052056280000000003 2024-09-15T21:47:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903151+0000", 2024-09-15T21:47:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2163000000000002e-05 2024-09-15T21:47:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903220+0000", 2024-09-15T21:47:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8483000000000001e-05 2024-09-15T21:47:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904794+0000", 2024-09-15T21:47:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001574521 2024-09-15T21:47:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907405+0000", 2024-09-15T21:47:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026110339999999999 2024-09-15T21:47:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.967 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907440+0000", 2024-09-15T21:47:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4564999999999999e-05 2024-09-15T21:47:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907463+0000", 2024-09-15T21:47:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3260000000000001e-05 2024-09-15T21:47:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:22 2.8 2:1470b2f8:::613.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:19.968 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 45.405486066999998, 2024-09-15T21:47:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007454937, 2024-09-15T21:47:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:19.969 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6800000000000002e-07 2024-09-15T21:47:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455145+0000", 2024-09-15T21:47:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4470000000000002e-06 2024-09-15T21:47:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.970 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456425+0000", 2024-09-15T21:47:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012798200000000001 2024-09-15T21:47:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456433+0000", 2024-09-15T21:47:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7389999999999999e-06 2024-09-15T21:47:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097865 2024-09-15T21:47:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.971 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457614+0000", 2024-09-15T21:47:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3560000000000006e-05 2024-09-15T21:47:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457998+0000", 2024-09-15T21:47:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00038372699999999999 2024-09-15T21:47:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460488+0000", 2024-09-15T21:47:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002489855 2024-09-15T21:47:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.972 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462512+0000", 2024-09-15T21:47:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020235269999999998 2024-09-15T21:47:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462554+0000", 2024-09-15T21:47:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2472000000000003e-05 2024-09-15T21:47:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462597+0000", 2024-09-15T21:47:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2657000000000002e-05 2024-09-15T21:47:19.973 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:2 2.5 2:a6e36b3f:::401.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 45.405481098000003, 2024-09-15T21:47:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074484590000000002, 2024-09-15T21:47:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.974 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:47:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.975 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455150+0000", 2024-09-15T21:47:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7319999999999998e-06 2024-09-15T21:47:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456449+0000", 2024-09-15T21:47:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012991490000000001 2024-09-15T21:47:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456454+0000", 2024-09-15T21:47:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0390000000000004e-06 2024-09-15T21:47:19.976 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456513+0000", 2024-09-15T21:47:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8768999999999999e-05 2024-09-15T21:47:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456628+0000", 2024-09-15T21:47:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115641 2024-09-15T21:47:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456741+0000", 2024-09-15T21:47:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000113094 2024-09-15T21:47:19.977 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458520+0000", 2024-09-15T21:47:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001778993 2024-09-15T21:47:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462494+0000", 2024-09-15T21:47:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039737009999999996 2024-09-15T21:47:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462549+0000", 2024-09-15T21:47:19.978 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4537000000000003e-05 2024-09-15T21:47:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462595+0000", 2024-09-15T21:47:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6511000000000002e-05 2024-09-15T21:47:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:27 2.5 2:a5fa685f:::504.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 28.295788714, 2024-09-15T21:47:19.979 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071804599999999996, 2024-09-15T21:47:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:19.980 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-15T21:47:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564841+0000", 2024-09-15T21:47:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-15T21:47:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565126+0000", 2024-09-15T21:47:19.981 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028469900000000002 2024-09-15T21:47:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565133+0000", 2024-09-15T21:47:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9480000000000002e-06 2024-09-15T21:47:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568006+0000", 2024-09-15T21:47:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028730330000000001 2024-09-15T21:47:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:19.982 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568119+0000", 2024-09-15T21:47:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011325899999999999 2024-09-15T21:47:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568235+0000", 2024-09-15T21:47:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115868 2024-09-15T21:47:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569668+0000", 2024-09-15T21:47:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001433099 2024-09-15T21:47:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.983 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571926+0000", 2024-09-15T21:47:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022572410000000001 2024-09-15T21:47:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571981+0000", 2024-09-15T21:47:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5383000000000002e-05 2024-09-15T21:47:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572020+0000", 2024-09-15T21:47:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8574e-05 2024-09-15T21:47:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.984 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.985 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.985 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:19.985 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:19.985 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:19.986 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:19.984+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:19.986 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:47:20.087 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:20.086+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:20.095 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:20.095 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:20.105 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:47:20.188 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:20.187+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:20.209 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:20.210 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:20.221 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:47:20.248 INFO:teuthology.orchestra.run.smithi122.stderr:Exporting 3.22 info 3.22( empty local-lis/les=20/21 n=0 ec=20/20 lis/c=20/20 les/c/f=21/21/0 sis=20) 2024-09-15T21:47:20.253 INFO:teuthology.orchestra.run.smithi122.stderr:Export successful 2024-09-15T21:47:20.290 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:20.288+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:20.329 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:20.329 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:20.338 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:47:20.423 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:20.421+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:20.484 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:20.484 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:20.491 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:20.489+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:20.494 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:47:20.591 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:20.590+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:20.642 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:20.642 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:20.652 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:47:20.691 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:47:20.693 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:20.691+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:20.766 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --force --op remove --pgid 3.22 2024-09-15T21:47:20.794 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:20.793+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:20.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.795+0000 7f5fbeee6640 1 Processor -- start 2024-09-15T21:47:20.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.795+0000 7f5fbeee6640 1 -- start start 2024-09-15T21:47:20.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.796+0000 7f5fbeee6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5fb80cc8f0 0x7f5fb81669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:20.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.796+0000 7f5fbeee6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fb815e7a0 con 0x7f5fb8155440 2024-09-15T21:47:20.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.796+0000 7f5fbeee6640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fb815ea40 con 0x7f5fb80cb7d0 2024-09-15T21:47:20.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.796+0000 7f5fbeee6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb815b470 con 0x7f5fb80cc8f0 2024-09-15T21:47:20.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.796+0000 7f5fbcc82640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f5fb8155440 0x7f5fb80cd420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:49982/0 (socket says 172.21.15.100:49982) 2024-09-15T21:47:20.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.796+0000 7f5fbcc82640 1 -- 172.21.15.100:0/550718141 learned_addr learned my addr 172.21.15.100:0/550718141 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:20.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.796+0000 7f5fbd483640 1 --2- 172.21.15.100:0/550718141 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5fb80cc8f0 0x7f5fb81669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:20.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.796+0000 7f5fb77fe640 1 -- 172.21.15.100:0/550718141 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2802803811 0 0) 0x7f5fb815e7a0 con 0x7f5fb8155440 2024-09-15T21:47:20.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.796+0000 7f5fb77fe640 1 -- 172.21.15.100:0/550718141 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5fa8003520 con 0x7f5fb8155440 2024-09-15T21:47:20.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fb77fe640 1 -- 172.21.15.100:0/550718141 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3763036075 0 0) 0x7f5fa8003520 con 0x7f5fb8155440 2024-09-15T21:47:20.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fb77fe640 1 -- 172.21.15.100:0/550718141 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5fb80cc8f0 msgr2=0x7f5fb81669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:20.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fb77fe640 1 --2- 172.21.15.100:0/550718141 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5fb80cc8f0 0x7f5fb81669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:20.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fb77fe640 1 -- 172.21.15.100:0/550718141 >> v1:172.21.15.122:6789/0 conn(0x7f5fb80cb7d0 legacy=0x7f5fb80cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:20.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fb77fe640 1 -- 172.21.15.100:0/550718141 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb815ed20 con 0x7f5fb8155440 2024-09-15T21:47:20.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fb77fe640 1 -- 172.21.15.100:0/550718141 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7f5fac001670 con 0x7f5fb8155440 2024-09-15T21:47:20.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fb77fe640 1 -- 172.21.15.100:0/550718141 >> v1:172.21.15.100:6789/0 conn(0x7f5fb8155440 legacy=0x7f5fb80cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:20.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fb77fe640 1 --2- 172.21.15.100:0/550718141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fa80039e0 0x7f5fa8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:20.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fb77fe640 1 --2- 172.21.15.100:0/550718141 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5fb80cc8f0 0x7f5fa8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:20.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fb77fe640 1 --2- 172.21.15.100:0/550718141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa8004d40 0x7f5fa80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:20.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fb77fe640 1 -- 172.21.15.100:0/550718141 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb815b470 con 0x7f5fa8004d40 2024-09-15T21:47:20.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fb77fe640 1 -- 172.21.15.100:0/550718141 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5fa800b9c0 con 0x7f5fa80039e0 2024-09-15T21:47:20.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fb77fe640 1 -- 172.21.15.100:0/550718141 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5fa800bbc0 con 0x7f5fb80cc8f0 2024-09-15T21:47:20.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fbd483640 1 --2- 172.21.15.100:0/550718141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa8004d40 0x7f5fa80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:20.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fbcc82640 1 --2- 172.21.15.100:0/550718141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fa80039e0 0x7f5fa8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:20.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fb7fff640 1 --2- 172.21.15.100:0/550718141 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5fb80cc8f0 0x7f5fa8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:20.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fbd483640 1 -- 172.21.15.100:0/550718141 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5fb80cc8f0 msgr2=0x7f5fa8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:20.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fbd483640 1 --2- 172.21.15.100:0/550718141 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5fb80cc8f0 0x7f5fa8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:20.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fbd483640 1 -- 172.21.15.100:0/550718141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fa80039e0 msgr2=0x7f5fa8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:20.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fbd483640 1 --2- 172.21.15.100:0/550718141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fa80039e0 0x7f5fa8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:20.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.797+0000 7f5fbd483640 1 -- 172.21.15.100:0/550718141 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fa8003520 con 0x7f5fa8004d40 2024-09-15T21:47:20.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.798+0000 7f5fbd483640 1 --2- 172.21.15.100:0/550718141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa8004d40 0x7f5fa80051c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009fc0 tx=0x7f5fa8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=468630a20dae43c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:20.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.798+0000 7f5fb77fe640 1 -- 172.21.15.100:0/550718141 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0009c10 con 0x7f5fa8004d40 2024-09-15T21:47:20.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.798+0000 7f5fb77fe640 1 -- 172.21.15.100:0/550718141 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb0036070 con 0x7f5fa8004d40 2024-09-15T21:47:20.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.798+0000 7f5fb77fe640 1 -- 172.21.15.100:0/550718141 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0031880 con 0x7f5fa8004d40 2024-09-15T21:47:20.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.799+0000 7f5fbeee6640 1 -- 172.21.15.100:0/550718141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa8004d40 msgr2=0x7f5fa80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:20.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.799+0000 7f5fbeee6640 1 --2- 172.21.15.100:0/550718141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fa8004d40 0x7f5fa80051c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0009fc0 tx=0x7f5fa8005830 comp rx=0 tx=0).stop 2024-09-15T21:47:20.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.799+0000 7f5fbd483640 1 -- 172.21.15.100:0/550718141 reap_dead start 2024-09-15T21:47:20.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.799+0000 7f5fbeee6640 1 -- 172.21.15.100:0/550718141 shutdown_connections 2024-09-15T21:47:20.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.799+0000 7f5fbeee6640 1 -- 172.21.15.100:0/550718141 >> 172.21.15.100:0/550718141 conn(0x7f5fb80680d0 msgr2=0x7f5fb806d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:20.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.799+0000 7f5fbeee6640 1 -- 172.21.15.100:0/550718141 shutdown_connections 2024-09-15T21:47:20.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.799+0000 7f5fbeee6640 1 -- 172.21.15.100:0/550718141 wait complete. 2024-09-15T21:47:20.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.801+0000 7f5fbeee6640 1 Processor -- start 2024-09-15T21:47:20.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.801+0000 7f5fbeee6640 1 -- start start 2024-09-15T21:47:20.805 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:20.805 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:20.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fbeee6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fa8004d40 0x7f5fb81f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:20.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fbeee6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5fb8158d60 0x7f5fb80cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:20.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fbeee6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fb814fc30 0x7f5fb8146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:20.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fbeee6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb815f2e0 con 0x7f5fb814fc30 2024-09-15T21:47:20.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fbeee6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb8155820 con 0x7f5fa8004d40 2024-09-15T21:47:20.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fbeee6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5fb815b070 con 0x7f5fb8158d60 2024-09-15T21:47:20.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fbd483640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fb814fc30 0x7f5fb8146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:20.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fbcc82640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fa8004d40 0x7f5fb81f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:20.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fbd483640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fb814fc30 0x7f5fb8146390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51678/0 (socket says 172.21.15.100:51678) 2024-09-15T21:47:20.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fb7fff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5fb8158d60 0x7f5fb80cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:20.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fbd483640 1 -- 172.21.15.100:0/3537681677 learned_addr learned my addr 172.21.15.100:0/3537681677 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:20.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fbcc82640 1 -- 172.21.15.100:0/3537681677 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5fb8158d60 msgr2=0x7f5fb80cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:20.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fbcc82640 1 --2- 172.21.15.100:0/3537681677 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5fb8158d60 0x7f5fb80cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:20.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fbcc82640 1 -- 172.21.15.100:0/3537681677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fb814fc30 msgr2=0x7f5fb8146390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:20.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fbcc82640 1 --2- 172.21.15.100:0/3537681677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fb814fc30 0x7f5fb8146390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:20.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fbcc82640 1 -- 172.21.15.100:0/3537681677 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb81f5020 con 0x7f5fa8004d40 2024-09-15T21:47:20.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fb7fff640 1 --2- 172.21.15.100:0/3537681677 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5fb8158d60 0x7f5fb80cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:20.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fbd483640 1 --2- 172.21.15.100:0/3537681677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fb814fc30 0x7f5fb8146390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:20.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fbcc82640 1 --2- 172.21.15.100:0/3537681677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fa8004d40 0x7f5fb81f4ae0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5fac007290 tx=0x7f5fac007750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:20.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.803+0000 7f5fb57fa640 1 -- 172.21.15.100:0/3537681677 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac00f9c0 con 0x7f5fa8004d40 2024-09-15T21:47:20.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.804+0000 7f5fb57fa640 1 -- 172.21.15.100:0/3537681677 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fac018070 con 0x7f5fa8004d40 2024-09-15T21:47:20.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.804+0000 7f5fbeee6640 1 -- 172.21.15.100:0/3537681677 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb81504a0 con 0x7f5fa8004d40 2024-09-15T21:47:20.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.804+0000 7f5fb57fa640 1 -- 172.21.15.100:0/3537681677 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fac00fd80 con 0x7f5fa8004d40 2024-09-15T21:47:20.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.804+0000 7f5fbeee6640 1 -- 172.21.15.100:0/3537681677 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb8150a60 con 0x7f5fa8004d40 2024-09-15T21:47:20.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.805+0000 7f5fbeee6640 1 -- 172.21.15.100:0/3537681677 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb80b3940 con 0x7f5fa8004d40 2024-09-15T21:47:20.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.805+0000 7f5fb57fa640 1 -- 172.21.15.100:0/3537681677 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5fac01d460 con 0x7f5fa8004d40 2024-09-15T21:47:20.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.805+0000 7f5fb57fa640 1 --2- 172.21.15.100:0/3537681677 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5f9c06c2e0 0x7f5f9c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:20.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.806+0000 7f5fb57fa640 1 -- 172.21.15.100:0/3537681677 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5854+0+0 (secure 0 0 0) 0x7f5fac08b340 con 0x7f5fa8004d40 2024-09-15T21:47:20.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.806+0000 7f5fb7fff640 1 --2- 172.21.15.100:0/3537681677 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5f9c06c2e0 0x7f5f9c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:20.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.806+0000 7f5fb7fff640 1 --2- 172.21.15.100:0/3537681677 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5f9c06c2e0 0x7f5f9c06e7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5fb815e6d0 tx=0x7f5fa4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:20.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:20.810+0000 7f5fb57fa640 1 -- 172.21.15.100:0/3537681677 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f5fb81f5020 con 0x7f5fa8004d40 2024-09-15T21:47:20.814 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:47:20.897 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:20.896+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:20.960 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:20.960 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:20.969 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:47:21.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.001+0000 7f5fbeee6640 1 -- 172.21.15.100:0/3537681677 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f5fb80c6a00 con 0x7f5fa8004d40 2024-09-15T21:47:21.005 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:21.002+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:21.097 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:21.095+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:21.111 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:21.111 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:21.121 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:47:21.206 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:21.204+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:21.268 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:21.268 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:21.277 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:47:21.299 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:21.298+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:21.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.313+0000 7fc48dffb640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=30}) -- 0x7fc4740034d0 con 0x7fc4980c3880 2024-09-15T21:47:21.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.313+0000 7fc48dffb640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] -- ping magic: 0 -- 0x7fc4740038a0 con 0x7fc49802c640 2024-09-15T21:47:21.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.313+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== mon.0 v2:172.21.15.100:3300/0 7 ==== osd_map(30..32 src has 1..32) ==== 1515+0+0 (secure 0 0 0) 0x7fc48808bc10 con 0x7fc4980c3880 2024-09-15T21:47:21.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.313+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc4981577d0 con 0x7fc4980c3880 2024-09-15T21:47:21.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.313+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fc49802c640 msgr2=0x7fc49802caa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:47:21.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.313+0000 7fc48d7fa640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6824/2400668532,v1:172.21.15.122:6825/2400668532] conn(0x7fc49802c640 0x7fc49802caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:21.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.313+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== mon.0 v2:172.21.15.100:3300/0 8 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc488094070 con 0x7fc4980c3880 2024-09-15T21:47:21.317 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:47:21.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.313+0000 7fc49ea65640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fc49802cfe0 0x7fc49802d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:21.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.313+0000 7fc49ea65640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fc49802da00 con 0x7fc49802cfe0 2024-09-15T21:47:21.317 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.314+0000 7fc48ffff640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fc49802cfe0 0x7fc49802d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:21.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.314+0000 7fc48ffff640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fc49802cfe0 0x7fc49802d440 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:21.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.315+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc49802da00 con 0x7fc49802cfe0 2024-09-15T21:47:21.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.329+0000 7fc49ea65640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc4980774b0 con 0x7fc49802cfe0 2024-09-15T21:47:21.334 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.331+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4980774b0 con 0x7fc49802cfe0 2024-09-15T21:47:21.334 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.331+0000 7fc49ea65640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fc498206880 0x7fc498208c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.331+0000 7fc49ea65640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fc49820bac0 con 0x7fc498206880 2024-09-15T21:47:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.332+0000 7fc49d002640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fc498206880 0x7fc498208c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.332+0000 7fc49d002640 1 --2- 172.21.15.100:0/3559481978 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fc498206880 0x7fc498208c70 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:21.335 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.333+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc49820bac0 con 0x7fc498206880 2024-09-15T21:47:21.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.347+0000 7fc49ea65640 1 -- 172.21.15.100:0/3559481978 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc4982091b0 con 0x7fc498206880 2024-09-15T21:47:21.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.349+0000 7fc48d7fa640 1 -- 172.21.15.100:0/3559481978 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4982091b0 con 0x7fc498206880 2024-09-15T21:47:21.351 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:21.352 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:21.356 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:47:21.356 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:47:21.400 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:21.398+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:21.423 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:21.423 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:21.432 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:47:21.502 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:21.500+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:21.541 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:21.541 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:21.549 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:47:21.602 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:21.599+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:21.654 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:21.654 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:21.664 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:47:21.707 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:21.706+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:21.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.730+0000 7f5fb57fa640 1 -- 172.21.15.100:0/3537681677 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v33) ==== 91+0+0 (secure 0 0 0) 0x7f5fac0593d0 con 0x7f5fa8004d40 2024-09-15T21:47:21.733 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is unset 2024-09-15T21:47:21.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.733+0000 7f5fbeee6640 1 -- 172.21.15.100:0/3537681677 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5f9c06c2e0 msgr2=0x7f5f9c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:21.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.733+0000 7f5fbeee6640 1 --2- 172.21.15.100:0/3537681677 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5f9c06c2e0 0x7f5f9c06e7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5fb815e6d0 tx=0x7f5fa4009040 comp rx=0 tx=0).stop 2024-09-15T21:47:21.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.733+0000 7f5fbeee6640 1 -- 172.21.15.100:0/3537681677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fa8004d40 msgr2=0x7f5fb81f4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:21.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.733+0000 7f5fbeee6640 1 --2- 172.21.15.100:0/3537681677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fa8004d40 0x7f5fb81f4ae0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5fac007290 tx=0x7f5fac007750 comp rx=0 tx=0).stop 2024-09-15T21:47:21.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.733+0000 7f5fbeee6640 1 -- 172.21.15.100:0/3537681677 shutdown_connections 2024-09-15T21:47:21.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.733+0000 7f5fbeee6640 1 --2- 172.21.15.100:0/3537681677 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5f9c06c2e0 0x7f5f9c06e7a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:21.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.733+0000 7f5fbeee6640 1 --2- 172.21.15.100:0/3537681677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fb814fc30 0x7f5fb8146390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:21.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.733+0000 7f5fbeee6640 1 --2- 172.21.15.100:0/3537681677 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5fb8158d60 0x7f5fb80cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:21.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.733+0000 7f5fbeee6640 1 --2- 172.21.15.100:0/3537681677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5fa8004d40 0x7f5fb81f4ae0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:21.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.733+0000 7f5fbeee6640 1 -- 172.21.15.100:0/3537681677 >> 172.21.15.100:0/3537681677 conn(0x7f5fb80680d0 msgr2=0x7f5fb815a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:21.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.733+0000 7f5fbeee6640 1 -- 172.21.15.100:0/3537681677 shutdown_connections 2024-09-15T21:47:21.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:21.733+0000 7f5fbeee6640 1 -- 172.21.15.100:0/3537681677 wait complete. 2024-09-15T21:47:21.769 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:21.769 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:21.778 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:47:21.803 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:21.801+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:21.904 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:21.903+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:21.922 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:21.923 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:21.931 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:47:22.016 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:22.014+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:22.075 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:22.075 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:22.083 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:47:22.125 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:22.123+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:22.207 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:22.205+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:22.230 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:22.230 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:22.238 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:47:22.308 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:22.305+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:22.341 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:22.341 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:22.349 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:47:22.409 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:22.407+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:22.451 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:22.451 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:22.460 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:47:22.487 INFO:teuthology.orchestra.run.smithi122.stdout: marking collection for removal 2024-09-15T21:47:22.487 INFO:teuthology.orchestra.run.smithi122.stdout:setting '_remove' omap key 2024-09-15T21:47:22.488 INFO:teuthology.orchestra.run.smithi122.stdout:finish_remove_pgs 3.22_head removing 3.22 2024-09-15T21:47:22.488 INFO:teuthology.orchestra.run.smithi122.stdout:Remove successful 2024-09-15T21:47:22.510 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:22.507+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:22.565 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:22.566 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:22.574 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:47:22.616 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:22.614+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:22.715 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:22.712+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:22.718 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:22.718 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:22.726 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:47:22.816 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:22.815+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:22.831 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:22.831 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:22.841 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:47:22.917 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:22.916+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:22.946 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:22.946 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:22.955 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:47:23.001 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.22.5 2024-09-15T21:47:23.018 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:23.016+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:23.059 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:23.059 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:23.068 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:47:23.152 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:23.150+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:23.213 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:23.214 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:23.220 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:23.218+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:23.222 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:47:23.321 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:23.320+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:23.361 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:47:23.373 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:23.373 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:23.384 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:47:23.422 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:23.421+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:23.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.459+0000 7f94c0cf8640 1 Processor -- start 2024-09-15T21:47:23.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.459+0000 7f94c0cf8640 1 -- start start 2024-09-15T21:47:23.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.460+0000 7f94c0cf8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f94bc155670 0x7f94bc15e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.460+0000 7f94c0cf8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f94bc15bb60 0x7f94bc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.460+0000 7f94c0cf8640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94bc15edb0 con 0x7f94bc155290 2024-09-15T21:47:23.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.460+0000 7f94c0cf8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f94bc15f090 con 0x7f94bc155670 2024-09-15T21:47:23.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.460+0000 7f94c0cf8640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f94bc15f370 con 0x7f94bc15bb60 2024-09-15T21:47:23.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.460+0000 7f94b9d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f94bc155670 0x7f94bc15e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.460+0000 7f94b9d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f94bc155670 0x7f94bc15e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51688/0 (socket says 172.21.15.100:51688) 2024-09-15T21:47:23.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.460+0000 7f94b9d9b640 1 -- 172.21.15.100:0/876602301 learned_addr learned my addr 172.21.15.100:0/876602301 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:23.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.460+0000 7f94bad9d640 1 --2- 172.21.15.100:0/876602301 >> v2:172.21.15.122:3300/0 conn(0x7f94bc15bb60 0x7f94bc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.460+0000 7f94b9d9b640 1 -- 172.21.15.100:0/876602301 >> v2:172.21.15.122:3300/0 conn(0x7f94bc15bb60 msgr2=0x7f94bc06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:23.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.460+0000 7f94b9d9b640 1 --2- 172.21.15.100:0/876602301 >> v2:172.21.15.122:3300/0 conn(0x7f94bc15bb60 0x7f94bc06d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:23.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.460+0000 7f94b9d9b640 1 -- 172.21.15.100:0/876602301 >> v1:172.21.15.122:6789/0 conn(0x7f94bc155290 legacy=0x7f94bc0c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:23.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.460+0000 7f94b9d9b640 1 -- 172.21.15.100:0/876602301 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94bc0c5db0 con 0x7f94bc155670 2024-09-15T21:47:23.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.460+0000 7f94bad9d640 1 --2- 172.21.15.100:0/876602301 >> v2:172.21.15.122:3300/0 conn(0x7f94bc15bb60 0x7f94bc06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:47:23.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.462+0000 7f94b9d9b640 1 --2- 172.21.15.100:0/876602301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94bc155670 0x7f94bc15e690 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0cde70 tx=0x7f94b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d258e5394b021760 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:23.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.462+0000 7f94b959a640 1 -- 172.21.15.100:0/876602301 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0037070 con 0x7f94bc155670 2024-09-15T21:47:23.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.462+0000 7f94b959a640 1 -- 172.21.15.100:0/876602301 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b0032440 con 0x7f94bc155670 2024-09-15T21:47:23.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.462+0000 7f94b959a640 1 -- 172.21.15.100:0/876602301 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0032810 con 0x7f94bc155670 2024-09-15T21:47:23.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.463+0000 7f94c0cf8640 1 -- 172.21.15.100:0/876602301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94bc155670 msgr2=0x7f94bc15e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:23.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.463+0000 7f94c0cf8640 1 --2- 172.21.15.100:0/876602301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94bc155670 0x7f94bc15e690 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0cde70 tx=0x7f94b002a270 comp rx=0 tx=0).stop 2024-09-15T21:47:23.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.463+0000 7f94c0cf8640 1 -- 172.21.15.100:0/876602301 shutdown_connections 2024-09-15T21:47:23.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.463+0000 7f94c0cf8640 1 --2- 172.21.15.100:0/876602301 >> v2:172.21.15.122:3300/0 conn(0x7f94bc15bb60 0x7f94bc06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:23.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.463+0000 7f94c0cf8640 1 --2- 172.21.15.100:0/876602301 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94bc155670 0x7f94bc15e690 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:23.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.463+0000 7f94c0cf8640 1 -- 172.21.15.100:0/876602301 >> 172.21.15.100:0/876602301 conn(0x7f94bc073310 msgr2=0x7f94bc073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:23.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.463+0000 7f94c0cf8640 1 -- 172.21.15.100:0/876602301 shutdown_connections 2024-09-15T21:47:23.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.463+0000 7f94c0cf8640 1 -- 172.21.15.100:0/876602301 wait complete. 2024-09-15T21:47:23.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.465+0000 7f94c0cf8640 1 Processor -- start 2024-09-15T21:47:23.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.465+0000 7f94c0cf8640 1 -- start start 2024-09-15T21:47:23.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.466+0000 7f94c0cf8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94bc155290 0x7f94bc153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.466+0000 7f94c0cf8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94bc155670 0x7f94bc156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.466+0000 7f94c0cf8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f94bc15bb60 0x7f94bc1574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.466+0000 7f94c0cf8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc15c590 con 0x7f94bc155670 2024-09-15T21:47:23.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.466+0000 7f94c0cf8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc0b39e0 con 0x7f94bc155290 2024-09-15T21:47:23.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.466+0000 7f94c0cf8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f94bc0c6cc0 con 0x7f94bc15bb60 2024-09-15T21:47:23.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.466+0000 7f94b9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94bc155670 0x7f94bc156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.466+0000 7f94b9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94bc155670 0x7f94bc156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51700/0 (socket says 172.21.15.100:51700) 2024-09-15T21:47:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.466+0000 7f94b9d9b640 1 -- 172.21.15.100:0/2756433998 learned_addr learned my addr 172.21.15.100:0/2756433998 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.466+0000 7f94bad9d640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f94bc15bb60 0x7f94bc1574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.466+0000 7f94ba59c640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94bc155290 0x7f94bc153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.467+0000 7f94b9d9b640 1 -- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f94bc15bb60 msgr2=0x7f94bc1574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.467+0000 7f94b9d9b640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f94bc15bb60 0x7f94bc1574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.467+0000 7f94b9d9b640 1 -- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94bc155290 msgr2=0x7f94bc153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.467+0000 7f94b9d9b640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94bc155290 0x7f94bc153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.467+0000 7f94b9d9b640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94bc15abb0 con 0x7f94bc155670 2024-09-15T21:47:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.467+0000 7f94b9d9b640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94bc155670 0x7f94bc156f90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f94b0000c00 tx=0x7f94b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.467+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0041020 con 0x7f94bc155670 2024-09-15T21:47:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.467+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94b00306c0 con 0x7f94bc155670 2024-09-15T21:47:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.467+0000 7f94c0cf8640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94bc15a410 con 0x7f94bc155670 2024-09-15T21:47:23.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.467+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b0037040 con 0x7f94bc155670 2024-09-15T21:47:23.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.467+0000 7f94c0cf8640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94bc159bd0 con 0x7f94bc155670 2024-09-15T21:47:23.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.468+0000 7f94c0cf8640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f94bc159700 con 0x7f94bc155670 2024-09-15T21:47:23.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.469+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f94b0044020 con 0x7f94bc155670 2024-09-15T21:47:23.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.469+0000 7f94aaffd640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f949c06c2e0 0x7f949c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.469+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5854+0+0 (secure 0 0 0) 0x7f94b00abb10 con 0x7f94bc155670 2024-09-15T21:47:23.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.470+0000 7f94ba59c640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f949c06c2e0 0x7f949c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.470+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v33) ==== 54+0+16 (secure 0 0 0) 0x7f94b004b030 con 0x7f94bc155670 2024-09-15T21:47:23.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.470+0000 7f94c0cf8640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f94bc14d980 0x7f94bc14dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.470+0000 7f94c0cf8640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94bc15bf70 con 0x7f94bc14d980 2024-09-15T21:47:23.473 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.470+0000 7f94bad9d640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f94bc14d980 0x7f94bc14dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.470+0000 7f94ba59c640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f949c06c2e0 0x7f949c06e7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f94ac000f60 tx=0x7f94ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:23.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.470+0000 7f94bad9d640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f94bc14d980 0x7f94bc14dd60 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:23.474 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.471+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94bc15bf70 con 0x7f94bc14d980 2024-09-15T21:47:23.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.486+0000 7f94c0cf8640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f94bc15a5e0 con 0x7f94bc14d980 2024-09-15T21:47:23.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.488+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f94bc15a5e0 con 0x7f94bc14d980 2024-09-15T21:47:23.491 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:23.492 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:23.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.488+0000 7f94c0cf8640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f94bc1f4250 0x7f94bc1f4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.488+0000 7f94c0cf8640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f94bc1f4b70 con 0x7f94bc1f4250 2024-09-15T21:47:23.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.489+0000 7f94ba59c640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f94bc1f4250 0x7f94bc1f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.489+0000 7f94ba59c640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f94bc1f4250 0x7f94bc1f4630 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:23.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.490+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94bc1f4b70 con 0x7f94bc1f4250 2024-09-15T21:47:23.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.505+0000 7f94c0cf8640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f94bc1523a0 con 0x7f94bc1f4250 2024-09-15T21:47:23.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.507+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f94bc1523a0 con 0x7f94bc1f4250 2024-09-15T21:47:23.510 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:23.510 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:23.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.507+0000 7f94c0cf8640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f94bc062b40 0x7f94bc062f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.507+0000 7f94c0cf8640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f94bc154b00 con 0x7f94bc062b40 2024-09-15T21:47:23.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.507+0000 7f94b9d9b640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f94bc062b40 0x7f94bc062f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.508+0000 7f94b9d9b640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f94bc062b40 0x7f94bc062f80 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:23.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.509+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94bc154b00 con 0x7f94bc062b40 2024-09-15T21:47:23.523 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:23.522+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:23.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.523+0000 7f94c0cf8640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f94bc0cb050 con 0x7f94bc062b40 2024-09-15T21:47:23.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.526+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f94bc0cb050 con 0x7f94bc062b40 2024-09-15T21:47:23.528 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:23.529 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:23.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.526+0000 7f94c0cf8640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f94bc02cf10 0x7f94bc02d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.526+0000 7f94c0cf8640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f94bc0b5970 con 0x7f94bc02cf10 2024-09-15T21:47:23.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.526+0000 7f94bad9d640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f94bc02cf10 0x7f94bc02d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.526+0000 7f94bad9d640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f94bc02cf10 0x7f94bc02d2f0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:23.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.527+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94bc0b5970 con 0x7f94bc02cf10 2024-09-15T21:47:23.536 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:23.536 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:23.536 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:23.536 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:23.536 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:23.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.542+0000 7f94c0cf8640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f94bc02db00 con 0x7f94bc02cf10 2024-09-15T21:47:23.545 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:47:23.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.544+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f94bc02db00 con 0x7f94bc02cf10 2024-09-15T21:47:23.546 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:23.548 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:23.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.544+0000 7f94c0cf8640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f94bc0b4770 0x7f94bc0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.544+0000 7f94c0cf8640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f94bc02da80 con 0x7f94bc0b4770 2024-09-15T21:47:23.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.544+0000 7f94ba59c640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f94bc0b4770 0x7f94bc0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.545+0000 7f94ba59c640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f94bc0b4770 0x7f94bc0b4bb0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:23.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.545+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94bc02da80 con 0x7f94bc0b4770 2024-09-15T21:47:23.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.560+0000 7f94c0cf8640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f94bc0b50f0 con 0x7f94bc0b4770 2024-09-15T21:47:23.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.562+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f94bc0b50f0 con 0x7f94bc0b4770 2024-09-15T21:47:23.565 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:23.566 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:23.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.562+0000 7f94c0cf8640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=34}) -- 0x7f94bc0b5300 con 0x7f94bc155670 2024-09-15T21:47:23.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.562+0000 7f94c0cf8640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f94bc0b50f0 con 0x7f94bc155670 2024-09-15T21:47:23.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.562+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f94b0079510 con 0x7f94bc155670 2024-09-15T21:47:23.566 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:47:23.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.563+0000 7f94c0cf8640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f94bc2055d0 0x7f94bc2079c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.563+0000 7f94c0cf8640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f94bc20a790 con 0x7f94bc2055d0 2024-09-15T21:47:23.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.563+0000 7f94b9d9b640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f94bc2055d0 0x7f94bc2079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.563+0000 7f94b9d9b640 1 -- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f94bc2055d0 msgr2=0x7f94bc2079c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 26 2024-09-15T21:47:23.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.563+0000 7f94b9d9b640 1 -- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f94bc2055d0 msgr2=0x7f94bc2079c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:47:23.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.563+0000 7f94b9d9b640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f94bc2055d0 0x7f94bc2079c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:47:23.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.563+0000 7f94b9d9b640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f94bc2055d0 0x7f94bc2079c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:47:23.624 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:23.622+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:23.656 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:23.656 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:23.656 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:23.656 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:23.656 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.656 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:23.656 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:23.657 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.239671287999997, 2024-09-15T21:47:23.657 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006101291, 2024-09-15T21:47:23.657 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.657 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.657 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:23.657 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.657 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.657 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.657 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:23.657 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.657 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.657 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.657 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.657 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:23.658 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.658 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.658 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.658 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.658 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413116+0000", 2024-09-15T21:47:23.658 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9700000000000003e-07 2024-09-15T21:47:23.658 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.658 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.658 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.658 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413118+0000", 2024-09-15T21:47:23.658 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2910000000000002e-06 2024-09-15T21:47:23.658 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.658 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.658 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.659 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413220+0000", 2024-09-15T21:47:23.659 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102056 2024-09-15T21:47:23.659 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.659 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.659 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.659 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413225+0000", 2024-09-15T21:47:23.659 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0490000000000003e-06 2024-09-15T21:47:23.659 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.659 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.659 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.659 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414808+0000", 2024-09-15T21:47:23.659 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015826900000000001 2024-09-15T21:47:23.659 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.659 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.660 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.660 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414907+0000", 2024-09-15T21:47:23.660 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9600999999999997e-05 2024-09-15T21:47:23.660 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.660 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.660 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:23.660 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415054+0000", 2024-09-15T21:47:23.660 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000146532 2024-09-15T21:47:23.660 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.660 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.660 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:23.660 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418056+0000", 2024-09-15T21:47:23.660 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030020519999999998 2024-09-15T21:47:23.660 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.661 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.661 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:23.661 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419128+0000", 2024-09-15T21:47:23.661 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001071671 2024-09-15T21:47:23.661 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.661 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.661 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:23.661 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419184+0000", 2024-09-15T21:47:23.661 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6125000000000002e-05 2024-09-15T21:47:23.661 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.661 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.661 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.661 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419217+0000", 2024-09-15T21:47:23.661 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2926999999999998e-05 2024-09-15T21:47:23.661 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.662 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.662 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.662 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.662 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.662 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:23.662 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:23.662 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.239667762000003, 2024-09-15T21:47:23.662 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059975130000000003, 2024-09-15T21:47:23.662 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.662 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.662 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:23.662 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.662 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.662 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.663 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:23.663 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.663 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.663 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.663 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.663 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:23.663 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.663 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.663 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.663 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.663 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:23.663 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6199999999999999e-07 2024-09-15T21:47:23.663 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.663 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.664 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.664 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413121+0000", 2024-09-15T21:47:23.664 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2000000000000001e-06 2024-09-15T21:47:23.664 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.664 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.664 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.664 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413292+0000", 2024-09-15T21:47:23.664 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017034400000000001 2024-09-15T21:47:23.664 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.664 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.664 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.664 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413300+0000", 2024-09-15T21:47:23.664 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8539999999999997e-06 2024-09-15T21:47:23.664 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.665 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.665 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.665 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414299+0000", 2024-09-15T21:47:23.665 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0009992829999999999 2024-09-15T21:47:23.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.665 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.665 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.665 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414390+0000", 2024-09-15T21:47:23.665 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1237999999999996e-05 2024-09-15T21:47:23.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.665 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.665 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:23.665 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414490+0000", 2024-09-15T21:47:23.665 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9438999999999994e-05 2024-09-15T21:47:23.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.666 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.666 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:23.666 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417393+0000", 2024-09-15T21:47:23.666 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002903306 2024-09-15T21:47:23.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.666 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.666 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:23.666 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419023+0000", 2024-09-15T21:47:23.666 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001630292 2024-09-15T21:47:23.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.666 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.666 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:23.666 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419067+0000", 2024-09-15T21:47:23.666 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4017999999999998e-05 2024-09-15T21:47:23.667 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.667 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.667 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.667 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419116+0000", 2024-09-15T21:47:23.667 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9277000000000003e-05 2024-09-15T21:47:23.667 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.667 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.667 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.667 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.667 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.667 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:23.667 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:23.667 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.236969565000003, 2024-09-15T21:47:23.667 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060879280000000003, 2024-09-15T21:47:23.667 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.668 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.668 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:23.668 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.668 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.668 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.668 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:23.668 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.668 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.668 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.668 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.668 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:23.668 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.668 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.668 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.669 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.669 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415818+0000", 2024-09-15T21:47:23.669 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:47:23.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.669 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.669 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.669 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415854+0000", 2024-09-15T21:47:23.669 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6706999999999999e-05 2024-09-15T21:47:23.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.669 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.669 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.669 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415860+0000", 2024-09-15T21:47:23.669 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0789999999999997e-06 2024-09-15T21:47:23.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.670 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.670 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.670 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415865+0000", 2024-09-15T21:47:23.670 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4580000000000003e-06 2024-09-15T21:47:23.670 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.670 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.670 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.670 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416092+0000", 2024-09-15T21:47:23.670 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00022739400000000001 2024-09-15T21:47:23.670 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.670 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.670 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.670 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416108+0000", 2024-09-15T21:47:23.670 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.613e-05 2024-09-15T21:47:23.671 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.671 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.671 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:23.671 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421866+0000", 2024-09-15T21:47:23.671 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005757585 2024-09-15T21:47:23.671 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.671 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.671 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.671 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421905+0000", 2024-09-15T21:47:23.671 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9214000000000001e-05 2024-09-15T21:47:23.671 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.671 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.671 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.671 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.672 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.672 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-15T21:47:23.672 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:23.672 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.236909404999999, 2024-09-15T21:47:23.672 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060637570000000003, 2024-09-15T21:47:23.672 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.672 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.672 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:23.672 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.672 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.672 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.672 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:23.672 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.672 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.673 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.673 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.673 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:23.673 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.673 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.673 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.673 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.673 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415878+0000", 2024-09-15T21:47:23.673 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.34e-07 2024-09-15T21:47:23.673 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.673 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.673 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.673 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415881+0000", 2024-09-15T21:47:23.673 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.354e-06 2024-09-15T21:47:23.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.674 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.674 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.674 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415887+0000", 2024-09-15T21:47:23.674 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:47:23.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.674 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.674 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.674 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415891+0000", 2024-09-15T21:47:23.674 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2690000000000001e-06 2024-09-15T21:47:23.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.674 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.674 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.674 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416267+0000", 2024-09-15T21:47:23.675 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00037614799999999998 2024-09-15T21:47:23.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.675 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.675 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.675 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416288+0000", 2024-09-15T21:47:23.675 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0604000000000001e-05 2024-09-15T21:47:23.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.675 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.675 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:23.675 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421908+0000", 2024-09-15T21:47:23.675 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056203390000000002 2024-09-15T21:47:23.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.675 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.675 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.675 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421941+0000", 2024-09-15T21:47:23.676 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3079e-05 2024-09-15T21:47:23.676 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.676 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.676 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.676 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.676 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.676 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:23.676 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:23.676 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.236890977999998, 2024-09-15T21:47:23.676 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006080284, 2024-09-15T21:47:23.676 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.676 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.676 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:23.676 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.677 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.677 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.677 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:23.677 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.677 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.677 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.677 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.677 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:23.677 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.677 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.677 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.677 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.677 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:23.677 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6899999999999999e-07 2024-09-15T21:47:23.678 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.678 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.678 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.678 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415898+0000", 2024-09-15T21:47:23.678 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.951e-06 2024-09-15T21:47:23.678 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.678 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.678 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.678 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415920+0000", 2024-09-15T21:47:23.678 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1651999999999999e-05 2024-09-15T21:47:23.678 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.678 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.678 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.678 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415925+0000", 2024-09-15T21:47:23.679 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.463e-06 2024-09-15T21:47:23.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.679 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.679 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.679 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416449+0000", 2024-09-15T21:47:23.679 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052384000000000003 2024-09-15T21:47:23.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.679 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.679 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.679 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416466+0000", 2024-09-15T21:47:23.679 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7108999999999998e-05 2024-09-15T21:47:23.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.679 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.679 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:23.680 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421943+0000", 2024-09-15T21:47:23.680 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054771569999999999 2024-09-15T21:47:23.680 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.680 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.680 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.680 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421976+0000", 2024-09-15T21:47:23.680 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2943e-05 2024-09-15T21:47:23.680 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.680 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.680 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.680 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.680 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.680 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-15T21:47:23.680 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:23.680 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.236851741999999, 2024-09-15T21:47:23.681 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060643340000000002, 2024-09-15T21:47:23.681 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.681 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.681 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:23.681 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.681 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.681 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.681 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:23.681 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.681 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.681 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.681 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.681 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:23.681 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.682 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.682 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.682 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.682 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:23.682 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4200000000000002e-07 2024-09-15T21:47:23.682 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.682 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.682 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.682 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415939+0000", 2024-09-15T21:47:23.682 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.439e-06 2024-09-15T21:47:23.682 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.682 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.682 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.682 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415945+0000", 2024-09-15T21:47:23.683 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7740000000000001e-06 2024-09-15T21:47:23.683 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.683 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.683 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.683 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415948+0000", 2024-09-15T21:47:23.683 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.737e-06 2024-09-15T21:47:23.683 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.683 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.683 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.683 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416605+0000", 2024-09-15T21:47:23.683 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00065721699999999998 2024-09-15T21:47:23.683 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.683 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.683 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.684 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416620+0000", 2024-09-15T21:47:23.684 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4564e-05 2024-09-15T21:47:23.684 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.684 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.684 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:23.684 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421977+0000", 2024-09-15T21:47:23.684 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053572369999999999 2024-09-15T21:47:23.684 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.684 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.684 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.684 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421999+0000", 2024-09-15T21:47:23.684 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2024000000000001e-05 2024-09-15T21:47:23.684 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.685 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.685 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.685 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.685 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.685 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-15T21:47:23.685 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:23.685 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.236651082999998, 2024-09-15T21:47:23.685 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058922669999999996, 2024-09-15T21:47:23.685 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.685 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.685 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:23.685 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.685 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.685 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.685 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:23.686 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.686 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.686 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.686 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.686 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:23.686 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.686 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.686 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.686 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.686 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:23.686 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6499999999999999e-07 2024-09-15T21:47:23.686 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.686 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.686 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.687 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416142+0000", 2024-09-15T21:47:23.687 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3929999999999999e-06 2024-09-15T21:47:23.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.687 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.687 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.687 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416151+0000", 2024-09-15T21:47:23.687 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5840000000000001e-06 2024-09-15T21:47:23.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.687 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.687 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.687 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416157+0000", 2024-09-15T21:47:23.687 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7590000000000003e-06 2024-09-15T21:47:23.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.687 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.688 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.688 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416767+0000", 2024-09-15T21:47:23.688 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00061034900000000005 2024-09-15T21:47:23.688 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.688 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.688 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.688 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416782+0000", 2024-09-15T21:47:23.688 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.483e-05 2024-09-15T21:47:23.688 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.688 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.688 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:23.688 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422002+0000", 2024-09-15T21:47:23.688 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052202639999999996 2024-09-15T21:47:23.688 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.689 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.689 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.689 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422028+0000", 2024-09-15T21:47:23.689 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5423e-05 2024-09-15T21:47:23.689 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.689 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.689 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.689 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.689 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.689 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-15T21:47:23.689 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:23.689 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.236244581999998, 2024-09-15T21:47:23.689 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055107769999999997, 2024-09-15T21:47:23.689 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.690 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.690 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:23.690 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.690 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.690 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.690 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:23.690 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.690 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.690 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.690 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.690 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:23.690 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.690 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.690 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.691 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.691 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416543+0000", 2024-09-15T21:47:23.691 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5400000000000001e-07 2024-09-15T21:47:23.691 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.691 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.691 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.691 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416548+0000", 2024-09-15T21:47:23.691 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8659999999999997e-06 2024-09-15T21:47:23.691 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.691 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.691 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.691 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416555+0000", 2024-09-15T21:47:23.691 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7190000000000001e-06 2024-09-15T21:47:23.691 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416560+0000", 2024-09-15T21:47:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3270000000000002e-06 2024-09-15T21:47:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416916+0000", 2024-09-15T21:47:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00035661200000000002 2024-09-15T21:47:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.692 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416932+0000", 2024-09-15T21:47:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5824e-05 2024-09-15T21:47:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422030+0000", 2024-09-15T21:47:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050975229999999996 2024-09-15T21:47:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422053+0000", 2024-09-15T21:47:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3252e-05 2024-09-15T21:47:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.693 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-15T21:47:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.235846082999998, 2024-09-15T21:47:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051422289999999999, 2024-09-15T21:47:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.694 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2699999999999999e-07 2024-09-15T21:47:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.695 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416946+0000", 2024-09-15T21:47:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1390000000000003e-06 2024-09-15T21:47:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416957+0000", 2024-09-15T21:47:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.044e-05 2024-09-15T21:47:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416963+0000", 2024-09-15T21:47:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5320000000000003e-06 2024-09-15T21:47:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.696 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417090+0000", 2024-09-15T21:47:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126549 2024-09-15T21:47:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417114+0000", 2024-09-15T21:47:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3736000000000001e-05 2024-09-15T21:47:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422055+0000", 2024-09-15T21:47:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049410410000000002 2024-09-15T21:47:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.697 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422083+0000", 2024-09-15T21:47:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8164999999999999e-05 2024-09-15T21:47:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 49.191354111000003, 2024-09-15T21:47:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007521186, 2024-09-15T21:47:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.698 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.699 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9999999999999998e-07 2024-09-15T21:47:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461443+0000", 2024-09-15T21:47:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0448000000000001e-05 2024-09-15T21:47:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462146+0000", 2024-09-15T21:47:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00070223499999999995 2024-09-15T21:47:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.700 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462154+0000", 2024-09-15T21:47:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9060000000000002e-06 2024-09-15T21:47:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462254+0000", 2024-09-15T21:47:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9897999999999998e-05 2024-09-15T21:47:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462330+0000", 2024-09-15T21:47:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6551000000000006e-05 2024-09-15T21:47:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.701 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462454+0000", 2024-09-15T21:47:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000124213 2024-09-15T21:47:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463681+0000", 2024-09-15T21:47:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012265729999999999 2024-09-15T21:47:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468817+0000", 2024-09-15T21:47:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051361430000000001 2024-09-15T21:47:23.702 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.703 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.703 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:23.703 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468892+0000", 2024-09-15T21:47:23.703 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5480000000000002e-05 2024-09-15T21:47:23.703 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.703 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.703 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.703 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468954+0000", 2024-09-15T21:47:23.703 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1339000000000003e-05 2024-09-15T21:47:23.703 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.703 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.703 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.703 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.703 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.703 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:23.704 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:23.704 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 49.191337496000003, 2024-09-15T21:47:23.704 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077296149999999996, 2024-09-15T21:47:23.704 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.704 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.704 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:23.704 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.704 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.704 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.704 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:23.704 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.704 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.704 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.705 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.705 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:23.705 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.705 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.705 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.705 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.705 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461450+0000", 2024-09-15T21:47:23.705 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0499999999999999e-07 2024-09-15T21:47:23.705 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.705 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.705 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.705 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461452+0000", 2024-09-15T21:47:23.705 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.007e-06 2024-09-15T21:47:23.705 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.705 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.706 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.706 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462209+0000", 2024-09-15T21:47:23.706 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075781000000000004 2024-09-15T21:47:23.706 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.706 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.706 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.706 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462214+0000", 2024-09-15T21:47:23.706 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1420000000000001e-06 2024-09-15T21:47:23.706 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.706 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.706 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.706 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462681+0000", 2024-09-15T21:47:23.706 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046634200000000002 2024-09-15T21:47:23.706 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.706 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.707 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.707 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462772+0000", 2024-09-15T21:47:23.707 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1414000000000003e-05 2024-09-15T21:47:23.707 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.707 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.707 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:23.707 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462871+0000", 2024-09-15T21:47:23.707 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8455000000000005e-05 2024-09-15T21:47:23.707 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.707 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.707 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:23.707 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464138+0000", 2024-09-15T21:47:23.707 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012673910000000001 2024-09-15T21:47:23.707 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.708 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.708 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:23.708 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469079+0000", 2024-09-15T21:47:23.708 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049407799999999997 2024-09-15T21:47:23.708 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.708 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.708 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:23.708 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469141+0000", 2024-09-15T21:47:23.708 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2169999999999996e-05 2024-09-15T21:47:23.708 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.708 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.708 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.708 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469179+0000", 2024-09-15T21:47:23.708 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7799000000000002e-05 2024-09-15T21:47:23.708 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.709 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.709 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.709 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.709 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.709 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:23.709 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:23.709 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.177221688000003, 2024-09-15T21:47:23.709 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004990497, 2024-09-15T21:47:23.709 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.709 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.709 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:23.709 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.709 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.709 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.710 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:23.710 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.710 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.710 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.710 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.710 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:23.710 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.710 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.710 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.710 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.710 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:23.710 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4729999999999999e-06 2024-09-15T21:47:23.710 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.710 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.710 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.711 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475576+0000", 2024-09-15T21:47:23.711 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1749999999999994e-06 2024-09-15T21:47:23.711 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.711 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.711 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.711 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475619+0000", 2024-09-15T21:47:23.711 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3557000000000003e-05 2024-09-15T21:47:23.711 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.711 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.711 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.711 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475631+0000", 2024-09-15T21:47:23.711 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1739999999999999e-05 2024-09-15T21:47:23.711 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.711 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.712 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.712 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475770+0000", 2024-09-15T21:47:23.712 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013950599999999999 2024-09-15T21:47:23.712 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.712 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.712 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.712 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475887+0000", 2024-09-15T21:47:23.712 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011616 2024-09-15T21:47:23.712 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.712 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.712 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:23.712 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476018+0000", 2024-09-15T21:47:23.712 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013119499999999999 2024-09-15T21:47:23.713 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.713 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.713 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:23.713 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478442+0000", 2024-09-15T21:47:23.713 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024240870000000001 2024-09-15T21:47:23.713 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.713 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.713 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:23.713 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480457+0000", 2024-09-15T21:47:23.713 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020151779999999998 2024-09-15T21:47:23.713 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.713 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.713 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:23.713 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480511+0000", 2024-09-15T21:47:23.714 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3582000000000003e-05 2024-09-15T21:47:23.714 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.714 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.714 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.714 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480556+0000", 2024-09-15T21:47:23.714 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4844e-05 2024-09-15T21:47:23.714 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.714 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.714 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.714 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.714 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.714 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:23.714 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:23.714 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.177207369999998, 2024-09-15T21:47:23.714 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051124810000000003, 2024-09-15T21:47:23.715 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.715 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.715 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:23.715 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.715 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.715 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.715 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:23.715 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.715 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.715 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.715 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:23.715 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.716 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.716 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.716 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475580+0000", 2024-09-15T21:47:23.716 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7800000000000002e-07 2024-09-15T21:47:23.716 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.716 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.716 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.716 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475595+0000", 2024-09-15T21:47:23.716 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4919000000000001e-05 2024-09-15T21:47:23.716 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.716 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.716 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.716 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475662+0000", 2024-09-15T21:47:23.716 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7769999999999997e-05 2024-09-15T21:47:23.717 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.717 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.717 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.717 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475669+0000", 2024-09-15T21:47:23.717 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.703e-06 2024-09-15T21:47:23.717 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.717 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.717 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.717 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476781+0000", 2024-09-15T21:47:23.717 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011118300000000001 2024-09-15T21:47:23.717 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.717 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.717 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.717 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476863+0000", 2024-09-15T21:47:23.718 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2208999999999998e-05 2024-09-15T21:47:23.718 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.718 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.718 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:23.718 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477040+0000", 2024-09-15T21:47:23.718 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000176424 2024-09-15T21:47:23.718 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.718 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.718 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:23.718 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480154+0000", 2024-09-15T21:47:23.718 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031147589999999999 2024-09-15T21:47:23.718 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.718 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.718 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:23.718 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480596+0000", 2024-09-15T21:47:23.719 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044172699999999999 2024-09-15T21:47:23.719 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.719 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.719 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:23.719 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480652+0000", 2024-09-15T21:47:23.719 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6350000000000001e-05 2024-09-15T21:47:23.719 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.719 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.719 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.719 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480692+0000", 2024-09-15T21:47:23.719 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9412000000000001e-05 2024-09-15T21:47:23.719 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.719 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.719 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.719 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.720 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.720 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:6 2.9 2:941d59bc:::61f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:23.720 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:23.720 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.107996837999998, 2024-09-15T21:47:23.720 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057427529999999997, 2024-09-15T21:47:23.720 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.720 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.720 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:23.720 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.720 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.720 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.720 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:23.720 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.721 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.721 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.721 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.721 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:23.721 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.721 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.721 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.721 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.721 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:23.721 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.41e-07 2024-09-15T21:47:23.721 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.721 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.721 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.721 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544792+0000", 2024-09-15T21:47:23.722 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9929999999999998e-06 2024-09-15T21:47:23.722 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.722 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.722 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.722 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544885+0000", 2024-09-15T21:47:23.722 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2825000000000007e-05 2024-09-15T21:47:23.722 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.722 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.722 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.722 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544891+0000", 2024-09-15T21:47:23.722 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.93e-06 2024-09-15T21:47:23.722 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.722 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.722 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.723 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546766+0000", 2024-09-15T21:47:23.723 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018747569999999999 2024-09-15T21:47:23.723 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.723 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.723 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.723 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546864+0000", 2024-09-15T21:47:23.723 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8603000000000004e-05 2024-09-15T21:47:23.723 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.723 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.723 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:23.723 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547028+0000", 2024-09-15T21:47:23.723 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000163461 2024-09-15T21:47:23.723 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.723 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.723 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:23.724 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549499+0000", 2024-09-15T21:47:23.724 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024711749999999999 2024-09-15T21:47:23.724 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.724 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.724 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:23.724 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550417+0000", 2024-09-15T21:47:23.724 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00091817199999999998 2024-09-15T21:47:23.724 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.724 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.724 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:23.724 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550476+0000", 2024-09-15T21:47:23.724 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8635999999999999e-05 2024-09-15T21:47:23.724 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.725 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.725 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.725 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550533+0000", 2024-09-15T21:47:23.725 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6959999999999997e-05 2024-09-15T21:47:23.725 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.725 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.725 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.725 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.725 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.725 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:23.725 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:23.725 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.107982372999999, 2024-09-15T21:47:23.725 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004917121, 2024-09-15T21:47:23.725 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.726 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.726 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:23.726 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.726 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.726 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.726 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:23.726 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.726 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.726 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.726 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.726 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:23.726 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.726 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.726 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.726 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.727 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544805+0000", 2024-09-15T21:47:23.727 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4600000000000001e-07 2024-09-15T21:47:23.727 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.727 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.727 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.727 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544817+0000", 2024-09-15T21:47:23.727 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2023e-05 2024-09-15T21:47:23.727 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.727 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.727 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.727 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545000+0000", 2024-09-15T21:47:23.727 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018384400000000001 2024-09-15T21:47:23.727 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.727 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.728 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.728 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545007+0000", 2024-09-15T21:47:23.728 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1249999999999998e-06 2024-09-15T21:47:23.728 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.728 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.728 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.728 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545670+0000", 2024-09-15T21:47:23.728 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00066324999999999999 2024-09-15T21:47:23.728 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.728 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.728 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.728 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545749+0000", 2024-09-15T21:47:23.728 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9239000000000005e-05 2024-09-15T21:47:23.728 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.729 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.729 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:23.729 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545885+0000", 2024-09-15T21:47:23.729 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013586199999999999 2024-09-15T21:47:23.729 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.729 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.729 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:23.729 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547944+0000", 2024-09-15T21:47:23.729 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020594720000000001 2024-09-15T21:47:23.729 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.729 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.729 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:23.729 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549630+0000", 2024-09-15T21:47:23.729 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001685831 2024-09-15T21:47:23.730 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.730 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.730 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:23.730 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549681+0000", 2024-09-15T21:47:23.730 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1187000000000003e-05 2024-09-15T21:47:23.730 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.730 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.730 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.730 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549721+0000", 2024-09-15T21:47:23.730 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0042e-05 2024-09-15T21:47:23.730 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.730 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.730 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.731 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.731 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.731 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:23.731 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:23.731 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.107968624999998, 2024-09-15T21:47:23.731 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050615549999999997, 2024-09-15T21:47:23.731 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.731 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.731 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:23.731 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.732 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.732 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.732 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:23.732 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.732 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.732 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.732 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.732 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:23.732 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.732 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.732 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.732 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.732 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:23.732 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:47:23.733 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.733 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.733 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.733 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544821+0000", 2024-09-15T21:47:23.733 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4399999999999999e-06 2024-09-15T21:47:23.733 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.733 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.733 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.733 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545025+0000", 2024-09-15T21:47:23.733 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020379400000000001 2024-09-15T21:47:23.733 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.733 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.733 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.733 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545030+0000", 2024-09-15T21:47:23.734 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3109999999999996e-06 2024-09-15T21:47:23.734 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.734 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.734 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.734 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546196+0000", 2024-09-15T21:47:23.734 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001166261 2024-09-15T21:47:23.734 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.734 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.734 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.734 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546307+0000", 2024-09-15T21:47:23.734 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000110641 2024-09-15T21:47:23.734 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.734 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.734 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:23.734 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546422+0000", 2024-09-15T21:47:23.735 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000115399 2024-09-15T21:47:23.735 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.735 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.735 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:23.735 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547973+0000", 2024-09-15T21:47:23.735 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001551228 2024-09-15T21:47:23.735 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.735 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.735 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:23.735 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549815+0000", 2024-09-15T21:47:23.735 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001841685 2024-09-15T21:47:23.735 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.735 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.735 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:23.736 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549856+0000", 2024-09-15T21:47:23.736 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.091e-05 2024-09-15T21:47:23.736 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.736 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.736 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.736 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549880+0000", 2024-09-15T21:47:23.736 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3580000000000001e-05 2024-09-15T21:47:23.736 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.736 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.736 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.736 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.736 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.736 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:23.737 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:23.737 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.107964617, 2024-09-15T21:47:23.737 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056931389999999998, 2024-09-15T21:47:23.737 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.737 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.737 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:23.737 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.737 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.737 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.737 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:23.737 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.737 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.737 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.738 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.738 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:23.738 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.738 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.738 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.738 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.738 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:23.738 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:47:23.738 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.738 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.738 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.738 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544828+0000", 2024-09-15T21:47:23.738 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4369999999999998e-06 2024-09-15T21:47:23.738 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.739 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.739 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.739 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545041+0000", 2024-09-15T21:47:23.739 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000212695 2024-09-15T21:47:23.739 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.739 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.739 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.739 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545058+0000", 2024-09-15T21:47:23.739 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7717999999999999e-05 2024-09-15T21:47:23.739 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.739 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.739 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.739 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546958+0000", 2024-09-15T21:47:23.739 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018995539999999999 2024-09-15T21:47:23.740 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.740 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.740 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.740 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547057+0000", 2024-09-15T21:47:23.740 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9370999999999994e-05 2024-09-15T21:47:23.740 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.740 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.740 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:23.740 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547579+0000", 2024-09-15T21:47:23.740 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052198200000000004 2024-09-15T21:47:23.740 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.740 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.740 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:23.740 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549929+0000", 2024-09-15T21:47:23.741 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023494649999999998 2024-09-15T21:47:23.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.741 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.741 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:23.741 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550401+0000", 2024-09-15T21:47:23.741 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00047277300000000001 2024-09-15T21:47:23.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.741 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.741 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:23.741 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550458+0000", 2024-09-15T21:47:23.741 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6756999999999999e-05 2024-09-15T21:47:23.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.741 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.742 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.742 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550515+0000", 2024-09-15T21:47:23.742 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7093999999999999e-05 2024-09-15T21:47:23.742 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.742 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.742 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.742 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.742 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:26 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:23.742 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:23.742 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.10795796, 2024-09-15T21:47:23.742 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050112029999999997, 2024-09-15T21:47:23.742 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.742 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.743 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:23.743 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.743 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.743 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.743 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:23.743 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.743 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.743 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.743 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.743 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:23.743 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.743 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.743 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.743 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.744 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:23.744 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.36e-07 2024-09-15T21:47:23.744 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.744 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.744 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.744 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545086+0000", 2024-09-15T21:47:23.744 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025704699999999999 2024-09-15T21:47:23.744 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.744 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.744 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.744 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545119+0000", 2024-09-15T21:47:23.744 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2734999999999997e-05 2024-09-15T21:47:23.744 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.745 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.745 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.745 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545126+0000", 2024-09-15T21:47:23.745 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5599999999999996e-06 2024-09-15T21:47:23.745 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.745 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.745 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.745 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545253+0000", 2024-09-15T21:47:23.745 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012644500000000001 2024-09-15T21:47:23.745 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.745 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.745 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.745 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545366+0000", 2024-09-15T21:47:23.745 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011349999999999999 2024-09-15T21:47:23.746 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.746 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.746 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:23.746 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545673+0000", 2024-09-15T21:47:23.746 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030643599999999999 2024-09-15T21:47:23.746 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.746 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.746 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:23.746 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547924+0000", 2024-09-15T21:47:23.746 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0022516189999999998 2024-09-15T21:47:23.746 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.746 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.746 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:23.746 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549746+0000", 2024-09-15T21:47:23.747 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018217629999999999 2024-09-15T21:47:23.747 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.747 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.747 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:23.747 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549800+0000", 2024-09-15T21:47:23.747 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3412999999999998e-05 2024-09-15T21:47:23.747 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.747 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.747 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.747 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549840+0000", 2024-09-15T21:47:23.747 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0448999999999998e-05 2024-09-15T21:47:23.747 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.747 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.747 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.747 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.748 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.748 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:30 2.4 2:218afd09:::203.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:23.748 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:23.748 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.107697131000002, 2024-09-15T21:47:23.748 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055481230000000003, 2024-09-15T21:47:23.748 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.748 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.748 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:23.748 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.748 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.748 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.748 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:23.748 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.748 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.748 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.749 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.749 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:23.749 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.749 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.749 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.749 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.749 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545091+0000", 2024-09-15T21:47:23.749 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7000000000000003e-07 2024-09-15T21:47:23.749 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.749 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.749 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.749 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545094+0000", 2024-09-15T21:47:23.749 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9569999999999998e-06 2024-09-15T21:47:23.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.750 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.750 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.750 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545156+0000", 2024-09-15T21:47:23.750 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2277e-05 2024-09-15T21:47:23.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.750 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.750 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.750 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545162+0000", 2024-09-15T21:47:23.750 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9120000000000003e-06 2024-09-15T21:47:23.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.750 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.750 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.750 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545262+0000", 2024-09-15T21:47:23.751 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010001900000000001 2024-09-15T21:47:23.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.751 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.751 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.751 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545388+0000", 2024-09-15T21:47:23.751 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126243 2024-09-15T21:47:23.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.751 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.751 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.751 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550638+0000", 2024-09-15T21:47:23.751 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052497450000000001 2024-09-15T21:47:23.751 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.751 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.751 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.752 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.752 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:10 2.9 2:954a9a2b:::62b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:23.752 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:23.752 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 32.088013670999999, 2024-09-15T21:47:23.752 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048438860000000004, 2024-09-15T21:47:23.752 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:23.752 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:23.752 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:23.752 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:23.752 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.752 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:23.752 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:23.752 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.752 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.752 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.753 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:23.753 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:23.753 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:23.753 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.753 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.753 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:23.753 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:23.753 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0400000000000002e-07 2024-09-15T21:47:23.753 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.753 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.753 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:23.753 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564786+0000", 2024-09-15T21:47:23.753 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2412000000000001e-05 2024-09-15T21:47:23.753 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.754 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.754 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:23.754 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564991+0000", 2024-09-15T21:47:23.754 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020483900000000001 2024-09-15T21:47:23.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.754 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.754 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:23.754 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564998+0000", 2024-09-15T21:47:23.754 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2459999999999997e-06 2024-09-15T21:47:23.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.754 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565422+0000", 2024-09-15T21:47:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00042420100000000003 2024-09-15T21:47:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565515+0000", 2024-09-15T21:47:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2449999999999997e-05 2024-09-15T21:47:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:23.755 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565642+0000", 2024-09-15T21:47:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012743699999999999 2024-09-15T21:47:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566656+0000", 2024-09-15T21:47:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010138580000000001 2024-09-15T21:47:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569552+0000", 2024-09-15T21:47:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028966819999999998 2024-09-15T21:47:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.756 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:23.757 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569594+0000", 2024-09-15T21:47:23.757 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1235000000000001e-05 2024-09-15T21:47:23.757 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:23.757 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:23.757 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:23.757 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569617+0000", 2024-09-15T21:47:23.757 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3221999999999999e-05 2024-09-15T21:47:23.757 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.757 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.757 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.757 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:23.757 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:23.757 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:23.758 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:47:23.759 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:23.757+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:23.760 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:47:23.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.764+0000 7f94b9d9b640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f94bc2055d0 0x7f94bc2079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.764+0000 7f94b9d9b640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f94bc2055d0 0x7f94bc2079c0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.765+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94bc20a790 con 0x7f94bc2055d0 2024-09-15T21:47:23.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.781+0000 7f94c0cf8640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f94bc207f00 con 0x7f94bc2055d0 2024-09-15T21:47:23.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.783+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f94bc207f00 con 0x7f94bc2055d0 2024-09-15T21:47:23.785 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:23.786 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:23.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.783+0000 7f94c0cf8640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f94bc208580 0x7f94bc2089e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.783+0000 7f94c0cf8640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f94bc20db60 con 0x7f94bc208580 2024-09-15T21:47:23.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.783+0000 7f94bad9d640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f94bc208580 0x7f94bc2089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.784+0000 7f94bad9d640 1 --2- 172.21.15.100:0/2756433998 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f94bc208580 0x7f94bc2089e0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:23.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.785+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94bc20db60 con 0x7f94bc208580 2024-09-15T21:47:23.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.800+0000 7f94c0cf8640 1 -- 172.21.15.100:0/2756433998 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f94bc208f70 con 0x7f94bc208580 2024-09-15T21:47:23.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.802+0000 7f94aaffd640 1 -- 172.21.15.100:0/2756433998 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f94bc208f70 con 0x7f94bc208580 2024-09-15T21:47:23.805 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:23.805 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:23.809 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:47:23.809 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:47:23.860 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:23.858+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:23.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.871+0000 7f4e8570a640 1 Processor -- start 2024-09-15T21:47:23.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.871+0000 7f4e8570a640 1 -- start start 2024-09-15T21:47:23.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.872+0000 7f4e8570a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4e800d0df0 0x7f4e80155060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.872+0000 7f4e8570a640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f4e800ccef0 0x7f4e801555a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.872+0000 7f4e8570a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e800d11d0 0x7f4e800cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.872+0000 7f4e8570a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f4e8015b3d0 con 0x7f4e800d0df0 2024-09-15T21:47:23.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.872+0000 7f4e8570a640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f4e8015b670 con 0x7f4e800ccef0 2024-09-15T21:47:23.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.872+0000 7f4e8570a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4e8015b950 con 0x7f4e800d11d0 2024-09-15T21:47:23.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.872+0000 7f4e7effd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4e800d0df0 0x7f4e80155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.872+0000 7f4e7effd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4e800d0df0 0x7f4e80155060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51706/0 (socket says 172.21.15.100:51706) 2024-09-15T21:47:23.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.872+0000 7f4e7effd640 1 -- 172.21.15.100:0/3390691060 learned_addr learned my addr 172.21.15.100:0/3390691060 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:23.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.873+0000 7f4e7f7fe640 1 --2- 172.21.15.100:0/3390691060 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e800d11d0 0x7f4e800cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.873+0000 7f4e7e7fc640 1 --2- 172.21.15.100:0/3390691060 >> v2:172.21.15.122:3300/0 conn(0x7f4e800ccef0 0x7f4e801555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.873+0000 7f4e7effd640 1 -- 172.21.15.100:0/3390691060 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e800d11d0 msgr2=0x7f4e800cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:23.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.873+0000 7f4e7effd640 1 --2- 172.21.15.100:0/3390691060 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e800d11d0 0x7f4e800cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:23.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.873+0000 7f4e7effd640 1 -- 172.21.15.100:0/3390691060 >> v2:172.21.15.122:3300/0 conn(0x7f4e800ccef0 msgr2=0x7f4e801555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:23.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.873+0000 7f4e7effd640 1 --2- 172.21.15.100:0/3390691060 >> v2:172.21.15.122:3300/0 conn(0x7f4e800ccef0 0x7f4e801555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:23.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.873+0000 7f4e7effd640 1 -- 172.21.15.100:0/3390691060 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e8015bc30 con 0x7f4e800d0df0 2024-09-15T21:47:23.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.873+0000 7f4e7e7fc640 1 --2- 172.21.15.100:0/3390691060 >> v2:172.21.15.122:3300/0 conn(0x7f4e800ccef0 0x7f4e801555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:47:23.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.874+0000 7f4e7effd640 1 --2- 172.21.15.100:0/3390691060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e800d0df0 0x7f4e80155060 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4e800cd560 tx=0x7f4e7002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e226ff0f53e67e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:23.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.874+0000 7f4e7dffb640 1 -- 172.21.15.100:0/3390691060 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70037070 con 0x7f4e800d0df0 2024-09-15T21:47:23.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.874+0000 7f4e7dffb640 1 -- 172.21.15.100:0/3390691060 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e70032440 con 0x7f4e800d0df0 2024-09-15T21:47:23.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.874+0000 7f4e7dffb640 1 -- 172.21.15.100:0/3390691060 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70032810 con 0x7f4e800d0df0 2024-09-15T21:47:23.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.875+0000 7f4e8570a640 1 -- 172.21.15.100:0/3390691060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e800d0df0 msgr2=0x7f4e80155060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:23.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.875+0000 7f4e8570a640 1 --2- 172.21.15.100:0/3390691060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e800d0df0 0x7f4e80155060 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4e800cd560 tx=0x7f4e7002a1e0 comp rx=0 tx=0).stop 2024-09-15T21:47:23.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.875+0000 7f4e8570a640 1 -- 172.21.15.100:0/3390691060 shutdown_connections 2024-09-15T21:47:23.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.875+0000 7f4e8570a640 1 --2- 172.21.15.100:0/3390691060 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e800d11d0 0x7f4e800cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:23.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.875+0000 7f4e8570a640 1 --2- 172.21.15.100:0/3390691060 >> v2:172.21.15.122:3300/0 conn(0x7f4e800ccef0 0x7f4e801555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:23.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.875+0000 7f4e8570a640 1 --2- 172.21.15.100:0/3390691060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e800d0df0 0x7f4e80155060 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:23.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.875+0000 7f4e8570a640 1 -- 172.21.15.100:0/3390691060 >> 172.21.15.100:0/3390691060 conn(0x7f4e80067980 msgr2=0x7f4e80067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:23.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.875+0000 7f4e8570a640 1 -- 172.21.15.100:0/3390691060 shutdown_connections 2024-09-15T21:47:23.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.875+0000 7f4e8570a640 1 -- 172.21.15.100:0/3390691060 wait complete. 2024-09-15T21:47:23.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.876+0000 7f4e8570a640 1 Processor -- start 2024-09-15T21:47:23.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.876+0000 7f4e8570a640 1 -- start start 2024-09-15T21:47:23.880 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:23.880 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:23.881 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:23.881 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:23.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.877+0000 7f4e8570a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e800ccef0 0x7f4e801e12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.877+0000 7f4e8570a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e800d0df0 0x7f4e801f1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.877+0000 7f4e8570a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e800d11d0 0x7f4e8014e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.877+0000 7f4e8570a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4e800cc200 con 0x7f4e800ccef0 2024-09-15T21:47:23.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.877+0000 7f4e8570a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4e800cc080 con 0x7f4e800d11d0 2024-09-15T21:47:23.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.877+0000 7f4e8570a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4e800cc380 con 0x7f4e800d0df0 2024-09-15T21:47:23.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.877+0000 7f4e7effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e800ccef0 0x7f4e801e12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.877+0000 7f4e7effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e800ccef0 0x7f4e801e12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51722/0 (socket says 172.21.15.100:51722) 2024-09-15T21:47:23.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.877+0000 7f4e7effd640 1 -- 172.21.15.100:0/2789721966 learned_addr learned my addr 172.21.15.100:0/2789721966 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:23.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.877+0000 7f4e7f7fe640 1 --2- 172.21.15.100:0/2789721966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e800d11d0 0x7f4e8014e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.877+0000 7f4e7e7fc640 1 --2- 172.21.15.100:0/2789721966 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e800d0df0 0x7f4e801f1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.878+0000 7f4e7effd640 1 -- 172.21.15.100:0/2789721966 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e800d0df0 msgr2=0x7f4e801f1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:23.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.878+0000 7f4e7effd640 1 --2- 172.21.15.100:0/2789721966 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e800d0df0 0x7f4e801f1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:23.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.878+0000 7f4e7effd640 1 -- 172.21.15.100:0/2789721966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e800d11d0 msgr2=0x7f4e8014e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:23.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.878+0000 7f4e7effd640 1 --2- 172.21.15.100:0/2789721966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e800d11d0 0x7f4e8014e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:23.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.878+0000 7f4e7effd640 1 -- 172.21.15.100:0/2789721966 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e800c93c0 con 0x7f4e800ccef0 2024-09-15T21:47:23.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.878+0000 7f4e7f7fe640 1 --2- 172.21.15.100:0/2789721966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e800d11d0 0x7f4e8014e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:23.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.878+0000 7f4e7effd640 1 --2- 172.21.15.100:0/2789721966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e800ccef0 0x7f4e801e12c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4e7002a220 tx=0x7f4e700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:23.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.878+0000 7f4e677fe640 1 -- 172.21.15.100:0/2789721966 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70041020 con 0x7f4e800ccef0 2024-09-15T21:47:23.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.878+0000 7f4e8570a640 1 -- 172.21.15.100:0/2789721966 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e800c96a0 con 0x7f4e800ccef0 2024-09-15T21:47:23.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.878+0000 7f4e677fe640 1 -- 172.21.15.100:0/2789721966 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e700306c0 con 0x7f4e800ccef0 2024-09-15T21:47:23.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.878+0000 7f4e677fe640 1 -- 172.21.15.100:0/2789721966 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70037040 con 0x7f4e800ccef0 2024-09-15T21:47:23.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.878+0000 7f4e8570a640 1 -- 172.21.15.100:0/2789721966 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e800c9c60 con 0x7f4e800ccef0 2024-09-15T21:47:23.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.879+0000 7f4e8570a640 1 -- 172.21.15.100:0/2789721966 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e800cfad0 con 0x7f4e800ccef0 2024-09-15T21:47:23.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.880+0000 7f4e677fe640 1 -- 172.21.15.100:0/2789721966 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4e70044020 con 0x7f4e800ccef0 2024-09-15T21:47:23.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.880+0000 7f4e677fe640 1 --2- 172.21.15.100:0/2789721966 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4e6c06c2b0 0x7f4e6c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:23.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.880+0000 7f4e677fe640 1 -- 172.21.15.100:0/2789721966 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5854+0+0 (secure 0 0 0) 0x7f4e700abb30 con 0x7f4e800ccef0 2024-09-15T21:47:23.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.880+0000 7f4e7e7fc640 1 --2- 172.21.15.100:0/2789721966 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4e6c06c2b0 0x7f4e6c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:23.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.881+0000 7f4e7e7fc640 1 --2- 172.21.15.100:0/2789721966 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4e6c06c2b0 0x7f4e6c06e770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4e801eed00 tx=0x7f4e74008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:23.890 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:47:23.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:23.889+0000 7f4e677fe640 1 -- 172.21.15.100:0/2789721966 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4e800cfad0 con 0x7f4e800ccef0 2024-09-15T21:47:23.961 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:23.958+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:24.002 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:24.003 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:24.003 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:24.003 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:24.003 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:24.011 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:47:24.062 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:24.060+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:24.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:24.097+0000 7f4e8570a640 1 -- 172.21.15.100:0/2789721966 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f4e8015c3a0 con 0x7f4e800ccef0 2024-09-15T21:47:24.158 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:24.158 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:24.158 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:24.158 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:24.159 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.159 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:24.159 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:24.159 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.742427239999998, 2024-09-15T21:47:24.159 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00598016, 2024-09-15T21:47:24.159 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.159 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.159 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.159 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.159 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.159 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.159 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:24.159 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413339+0000", 2024-09-15T21:47:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6279999999999998e-06 2024-09-15T21:47:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.160 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413354+0000", 2024-09-15T21:47:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4877e-05 2024-09-15T21:47:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413418+0000", 2024-09-15T21:47:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4274000000000002e-05 2024-09-15T21:47:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413428+0000", 2024-09-15T21:47:24.161 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5899999999999997e-06 2024-09-15T21:47:24.162 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.162 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.162 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.162 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413563+0000", 2024-09-15T21:47:24.162 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013521900000000001 2024-09-15T21:47:24.162 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.162 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.162 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.162 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413678+0000", 2024-09-15T21:47:24.162 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000114904 2024-09-15T21:47:24.162 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.162 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.162 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413789+0000", 2024-09-15T21:47:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011069300000000001 2024-09-15T21:47:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417603+0000", 2024-09-15T21:47:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038138460000000001 2024-09-15T21:47:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419220+0000", 2024-09-15T21:47:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001617678 2024-09-15T21:47:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.163 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.164 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.164 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419275+0000", 2024-09-15T21:47:24.164 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4391999999999997e-05 2024-09-15T21:47:24.164 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.164 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.164 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.164 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419316+0000", 2024-09-15T21:47:24.164 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1059000000000001e-05 2024-09-15T21:47:24.164 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.164 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.164 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.164 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.164 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.165 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:24.165 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:24.165 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.742405112, 2024-09-15T21:47:24.165 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0087926470000000007, 2024-09-15T21:47:24.165 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.165 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.165 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.165 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.165 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.165 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.165 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:24.165 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.165 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.166 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.166 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.166 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:24.166 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.166 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.166 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.166 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.166 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:24.166 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4099999999999999e-07 2024-09-15T21:47:24.166 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.166 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.166 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.166 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413361+0000", 2024-09-15T21:47:24.166 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6189999999999998e-06 2024-09-15T21:47:24.167 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.167 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.167 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.167 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413461+0000", 2024-09-15T21:47:24.167 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000100053 2024-09-15T21:47:24.167 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.167 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.167 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.167 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413466+0000", 2024-09-15T21:47:24.167 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3850000000000001e-06 2024-09-15T21:47:24.167 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.167 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.167 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.167 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414073+0000", 2024-09-15T21:47:24.168 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000606724 2024-09-15T21:47:24.168 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.168 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.168 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.168 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414143+0000", 2024-09-15T21:47:24.168 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9727000000000005e-05 2024-09-15T21:47:24.168 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.168 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.168 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:24.168 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414259+0000", 2024-09-15T21:47:24.168 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000116178 2024-09-15T21:47:24.168 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.168 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.168 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.169 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417610+0000", 2024-09-15T21:47:24.169 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003351176 2024-09-15T21:47:24.169 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.169 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.169 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.169 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422058+0000", 2024-09-15T21:47:24.169 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004447599 2024-09-15T21:47:24.169 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.169 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.169 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.169 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422111+0000", 2024-09-15T21:47:24.169 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3606999999999997e-05 2024-09-15T21:47:24.169 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.170 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.170 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.170 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422150+0000", 2024-09-15T21:47:24.170 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9137999999999997e-05 2024-09-15T21:47:24.170 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.170 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.170 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.170 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.170 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.170 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:24.170 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:24.170 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.742400682000003, 2024-09-15T21:47:24.170 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0089009769999999992, 2024-09-15T21:47:24.170 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.170 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.171 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.171 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.171 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.171 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.171 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:24.171 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.171 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.171 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.171 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.171 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:24.171 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.171 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.171 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.171 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.172 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413363+0000", 2024-09-15T21:47:24.172 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3999999999999997e-07 2024-09-15T21:47:24.172 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.172 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.172 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.172 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413365+0000", 2024-09-15T21:47:24.172 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.216e-06 2024-09-15T21:47:24.172 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.172 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.172 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.172 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413481+0000", 2024-09-15T21:47:24.172 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000116311 2024-09-15T21:47:24.172 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.173 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.173 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.173 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413488+0000", 2024-09-15T21:47:24.173 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0110000000000003e-06 2024-09-15T21:47:24.173 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.173 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.173 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.173 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414452+0000", 2024-09-15T21:47:24.173 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00096385399999999999 2024-09-15T21:47:24.173 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.173 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.173 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.173 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414545+0000", 2024-09-15T21:47:24.173 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.3024999999999998e-05 2024-09-15T21:47:24.174 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.174 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.174 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:24.174 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414644+0000", 2024-09-15T21:47:24.174 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9098000000000006e-05 2024-09-15T21:47:24.174 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.174 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.174 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.174 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417667+0000", 2024-09-15T21:47:24.174 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030229409999999999 2024-09-15T21:47:24.174 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.174 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.174 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.174 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422200+0000", 2024-09-15T21:47:24.175 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045327880000000003 2024-09-15T21:47:24.175 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.175 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.175 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.175 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422228+0000", 2024-09-15T21:47:24.175 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8109000000000001e-05 2024-09-15T21:47:24.175 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.175 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.175 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.175 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422263+0000", 2024-09-15T21:47:24.175 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5284000000000003e-05 2024-09-15T21:47:24.175 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.175 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.175 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.176 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.176 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.176 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:24.176 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:24.176 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.742397095999998, 2024-09-15T21:47:24.176 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0089846820000000008, 2024-09-15T21:47:24.176 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.176 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.176 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.176 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.176 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.176 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.177 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:24.177 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.177 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.177 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.177 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.177 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:24.177 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.177 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.177 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.177 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.177 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:24.177 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7599999999999998e-07 2024-09-15T21:47:24.177 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.178 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.178 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.178 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413368+0000", 2024-09-15T21:47:24.178 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.058e-06 2024-09-15T21:47:24.178 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.178 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.178 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.178 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413832+0000", 2024-09-15T21:47:24.178 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046398700000000001 2024-09-15T21:47:24.178 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.178 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.178 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.178 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413841+0000", 2024-09-15T21:47:24.178 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1710000000000008e-06 2024-09-15T21:47:24.179 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.179 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.179 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.179 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414869+0000", 2024-09-15T21:47:24.179 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001027584 2024-09-15T21:47:24.179 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.179 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.179 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.179 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414946+0000", 2024-09-15T21:47:24.179 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7177999999999999e-05 2024-09-15T21:47:24.179 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.179 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.179 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:24.179 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415054+0000", 2024-09-15T21:47:24.179 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010807 2024-09-15T21:47:24.180 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.180 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.180 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.180 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417672+0000", 2024-09-15T21:47:24.180 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002618015 2024-09-15T21:47:24.180 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.180 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.180 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.180 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422295+0000", 2024-09-15T21:47:24.180 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046233300000000001 2024-09-15T21:47:24.180 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.180 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.180 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.181 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422330+0000", 2024-09-15T21:47:24.181 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4476e-05 2024-09-15T21:47:24.181 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.181 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.181 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.181 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422351+0000", 2024-09-15T21:47:24.181 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0537e-05 2024-09-15T21:47:24.181 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.181 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.181 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.181 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.181 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.181 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:24.181 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:24.182 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.742393735999997, 2024-09-15T21:47:24.182 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0090873700000000009, 2024-09-15T21:47:24.182 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.182 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.182 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.182 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.182 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.182 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.182 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:24.182 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.182 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.182 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.182 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.182 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:24.182 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.183 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.183 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.183 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.183 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:24.183 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6100000000000002e-07 2024-09-15T21:47:24.183 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.183 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.183 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.183 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413371+0000", 2024-09-15T21:47:24.183 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.764e-06 2024-09-15T21:47:24.183 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.183 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.183 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.183 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413859+0000", 2024-09-15T21:47:24.184 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00048751900000000001 2024-09-15T21:47:24.184 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.184 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.184 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.184 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413864+0000", 2024-09-15T21:47:24.184 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6200000000000004e-06 2024-09-15T21:47:24.184 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.184 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.184 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.184 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415253+0000", 2024-09-15T21:47:24.184 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013884050000000001 2024-09-15T21:47:24.184 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.184 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.184 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.185 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415364+0000", 2024-09-15T21:47:24.185 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000111119 2024-09-15T21:47:24.185 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.185 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.185 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:24.185 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415453+0000", 2024-09-15T21:47:24.185 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8641999999999996e-05 2024-09-15T21:47:24.185 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.185 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.185 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.185 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417676+0000", 2024-09-15T21:47:24.185 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00222368 2024-09-15T21:47:24.185 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.185 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.186 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.186 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422395+0000", 2024-09-15T21:47:24.186 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047191150000000003 2024-09-15T21:47:24.186 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.186 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.186 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.186 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422434+0000", 2024-09-15T21:47:24.186 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8525999999999997e-05 2024-09-15T21:47:24.186 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.186 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.186 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.186 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422457+0000", 2024-09-15T21:47:24.186 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2719000000000001e-05 2024-09-15T21:47:24.186 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.187 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.187 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.187 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.187 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.187 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:24.187 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:24.187 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.742390714000003, 2024-09-15T21:47:24.187 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0092139749999999992, 2024-09-15T21:47:24.187 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.187 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.187 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.187 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.187 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.188 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.188 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:24.188 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.188 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.188 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.188 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.188 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:24.188 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.188 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.188 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.188 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.188 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:24.189 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3799999999999999e-07 2024-09-15T21:47:24.189 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.189 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.189 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.189 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413375+0000", 2024-09-15T21:47:24.189 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1959999999999998e-06 2024-09-15T21:47:24.189 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.189 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.189 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.189 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413876+0000", 2024-09-15T21:47:24.189 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00050134500000000005 2024-09-15T21:47:24.189 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.190 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.190 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.190 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413881+0000", 2024-09-15T21:47:24.190 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3580000000000002e-06 2024-09-15T21:47:24.190 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.190 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.190 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.190 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415669+0000", 2024-09-15T21:47:24.190 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0017875090000000001 2024-09-15T21:47:24.190 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.190 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.190 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.190 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415737+0000", 2024-09-15T21:47:24.190 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8386000000000006e-05 2024-09-15T21:47:24.191 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.191 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.191 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:24.191 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415857+0000", 2024-09-15T21:47:24.191 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119906 2024-09-15T21:47:24.191 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.191 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.191 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.191 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421848+0000", 2024-09-15T21:47:24.191 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059903880000000001 2024-09-15T21:47:24.191 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.191 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.191 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.192 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422513+0000", 2024-09-15T21:47:24.192 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00066575599999999999 2024-09-15T21:47:24.192 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.192 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.192 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.192 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422549+0000", 2024-09-15T21:47:24.192 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5719999999999997e-05 2024-09-15T21:47:24.192 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.192 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.192 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.192 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422586+0000", 2024-09-15T21:47:24.192 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7172999999999998e-05 2024-09-15T21:47:24.192 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.192 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.193 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.193 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.193 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.193 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:24.193 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:24.193 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.742384891999997, 2024-09-15T21:47:24.193 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.009324087, 2024-09-15T21:47:24.193 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.193 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.193 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.193 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.193 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.193 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.193 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:24.193 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.194 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.194 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.194 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.194 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:24.194 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.194 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.194 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.194 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.194 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:24.194 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8900000000000001e-07 2024-09-15T21:47:24.194 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.194 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.194 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.195 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413381+0000", 2024-09-15T21:47:24.195 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1959999999999998e-06 2024-09-15T21:47:24.195 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.195 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.195 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.195 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413895+0000", 2024-09-15T21:47:24.195 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051466000000000001 2024-09-15T21:47:24.195 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.195 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.195 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.195 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413900+0000", 2024-09-15T21:47:24.195 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6090000000000003e-06 2024-09-15T21:47:24.195 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.195 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.195 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.196 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416081+0000", 2024-09-15T21:47:24.196 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021816090000000002 2024-09-15T21:47:24.196 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.196 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.196 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.196 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416173+0000", 2024-09-15T21:47:24.196 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1334000000000003e-05 2024-09-15T21:47:24.196 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.196 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.196 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:24.196 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416264+0000", 2024-09-15T21:47:24.196 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0864000000000003e-05 2024-09-15T21:47:24.196 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.196 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.197 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.197 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421854+0000", 2024-09-15T21:47:24.197 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055902240000000004 2024-09-15T21:47:24.197 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.197 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.197 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.197 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422629+0000", 2024-09-15T21:47:24.197 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00077550700000000004 2024-09-15T21:47:24.197 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.197 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.197 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.197 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422678+0000", 2024-09-15T21:47:24.197 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8810999999999997e-05 2024-09-15T21:47:24.197 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.198 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.198 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.198 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422702+0000", 2024-09-15T21:47:24.198 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3983999999999999e-05 2024-09-15T21:47:24.198 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.198 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.198 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.198 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.198 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.198 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:24.198 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:24.198 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.74238141, 2024-09-15T21:47:24.198 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0094270580000000003, 2024-09-15T21:47:24.198 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.198 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.199 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.199 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.199 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.199 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.199 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:24.199 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.199 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.199 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.199 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.199 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:24.199 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.199 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.199 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.199 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.199 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:24.200 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7500000000000001e-07 2024-09-15T21:47:24.200 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.200 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.200 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.200 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413386+0000", 2024-09-15T21:47:24.200 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5619999999999997e-06 2024-09-15T21:47:24.200 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.200 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.200 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.200 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413918+0000", 2024-09-15T21:47:24.200 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00053162499999999998 2024-09-15T21:47:24.201 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.201 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.201 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.201 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413923+0000", 2024-09-15T21:47:24.201 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9280000000000001e-06 2024-09-15T21:47:24.201 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.201 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.201 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.201 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416492+0000", 2024-09-15T21:47:24.201 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0025687140000000002 2024-09-15T21:47:24.201 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.201 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.201 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.202 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416558+0000", 2024-09-15T21:47:24.202 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6699000000000006e-05 2024-09-15T21:47:24.202 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.202 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.202 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:24.202 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416691+0000", 2024-09-15T21:47:24.202 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013263199999999999 2024-09-15T21:47:24.202 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.202 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.202 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.202 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421879+0000", 2024-09-15T21:47:24.202 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051881089999999998 2024-09-15T21:47:24.202 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.203 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.203 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.203 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422742+0000", 2024-09-15T21:47:24.203 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000862845 2024-09-15T21:47:24.203 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.203 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.203 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.203 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422786+0000", 2024-09-15T21:47:24.203 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4332000000000002e-05 2024-09-15T21:47:24.203 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.203 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.203 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.203 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422809+0000", 2024-09-15T21:47:24.203 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2337e-05 2024-09-15T21:47:24.203 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.204 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.204 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.204 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.204 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.204 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:4 2.10 2:0f65c9b8:::60a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.204 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:24.204 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 49.700095241, 2024-09-15T21:47:24.204 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058710300000000002, 2024-09-15T21:47:24.204 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.204 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.204 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.204 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.204 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.205 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.205 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:24.205 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.205 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.205 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.205 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.205 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:24.205 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.205 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.205 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.205 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.205 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:24.205 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0800000000000005e-07 2024-09-15T21:47:24.205 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.206 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.206 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.206 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455672+0000", 2024-09-15T21:47:24.206 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3900000000000002e-06 2024-09-15T21:47:24.206 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.206 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.206 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.206 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456538+0000", 2024-09-15T21:47:24.206 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00086613100000000002 2024-09-15T21:47:24.206 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.206 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.206 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.206 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456545+0000", 2024-09-15T21:47:24.206 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4830000000000002e-06 2024-09-15T21:47:24.207 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.207 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.207 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.207 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456594+0000", 2024-09-15T21:47:24.207 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8429e-05 2024-09-15T21:47:24.207 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.207 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.207 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.207 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456675+0000", 2024-09-15T21:47:24.207 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1516999999999999e-05 2024-09-15T21:47:24.207 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.207 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.207 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:24.208 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456772+0000", 2024-09-15T21:47:24.208 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6523999999999999e-05 2024-09-15T21:47:24.208 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.208 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.208 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.208 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.460299+0000", 2024-09-15T21:47:24.208 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035274859999999998 2024-09-15T21:47:24.208 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.208 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.208 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.208 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461451+0000", 2024-09-15T21:47:24.208 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011518100000000001 2024-09-15T21:47:24.208 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.208 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.208 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.209 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461498+0000", 2024-09-15T21:47:24.209 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7406e-05 2024-09-15T21:47:24.209 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.209 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.209 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.209 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461539+0000", 2024-09-15T21:47:24.209 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0346000000000003e-05 2024-09-15T21:47:24.209 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.209 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.209 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.209 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.209 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.209 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:31 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.209 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:24.210 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 49.700089978999998, 2024-09-15T21:47:24.210 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0078158979999999999, 2024-09-15T21:47:24.210 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.210 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.210 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:24.210 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.210 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.210 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.210 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:24.210 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.210 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.210 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.210 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.210 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:24.210 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.211 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.211 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.211 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.211 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:24.211 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0499999999999999e-07 2024-09-15T21:47:24.211 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.211 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.211 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.211 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455675+0000", 2024-09-15T21:47:24.211 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.376e-06 2024-09-15T21:47:24.211 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.211 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.211 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.211 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456964+0000", 2024-09-15T21:47:24.211 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001289204 2024-09-15T21:47:24.212 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.212 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.212 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.212 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456970+0000", 2024-09-15T21:47:24.212 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2129999999999996e-06 2024-09-15T21:47:24.212 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.212 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.212 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.212 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457032+0000", 2024-09-15T21:47:24.212 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2209e-05 2024-09-15T21:47:24.212 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.212 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.212 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.212 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457134+0000", 2024-09-15T21:47:24.213 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010204000000000001 2024-09-15T21:47:24.213 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.213 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.213 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.213 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463489+0000", 2024-09-15T21:47:24.213 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0063545509999999999 2024-09-15T21:47:24.213 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.213 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.213 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.213 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.213 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.213 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.213 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:24.213 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.680330214999998, 2024-09-15T21:47:24.213 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057594079999999997, 2024-09-15T21:47:24.214 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.214 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.214 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.214 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.214 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.214 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.214 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:24.214 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.214 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.214 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.214 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.214 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:24.215 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.215 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.215 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.215 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.215 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:24.215 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1999999999999999e-07 2024-09-15T21:47:24.215 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.215 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.215 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.215 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475437+0000", 2024-09-15T21:47:24.215 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5429999999999998e-06 2024-09-15T21:47:24.215 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.215 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.215 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.215 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475686+0000", 2024-09-15T21:47:24.216 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000248908 2024-09-15T21:47:24.216 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.216 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.216 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.216 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475695+0000", 2024-09-15T21:47:24.216 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.9740000000000008e-06 2024-09-15T21:47:24.216 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.216 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.216 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.216 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476291+0000", 2024-09-15T21:47:24.216 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00059651500000000004 2024-09-15T21:47:24.216 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.216 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.216 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.217 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476366+0000", 2024-09-15T21:47:24.217 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4531999999999997e-05 2024-09-15T21:47:24.217 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.217 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.217 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:24.217 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476491+0000", 2024-09-15T21:47:24.217 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012527299999999999 2024-09-15T21:47:24.217 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.217 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.217 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.217 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480600+0000", 2024-09-15T21:47:24.217 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0041085870000000003 2024-09-15T21:47:24.217 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.217 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.218 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.218 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481089+0000", 2024-09-15T21:47:24.218 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000489037 2024-09-15T21:47:24.218 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.218 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.218 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.218 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481152+0000", 2024-09-15T21:47:24.218 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3136e-05 2024-09-15T21:47:24.218 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.218 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.218 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.218 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481192+0000", 2024-09-15T21:47:24.218 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0182999999999998e-05 2024-09-15T21:47:24.218 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.218 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.219 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.219 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.219 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.219 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:17 2.10 2:0e9da45e:::61b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.219 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:24.219 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.680324554000002, 2024-09-15T21:47:24.219 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058311830000000002, 2024-09-15T21:47:24.219 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.219 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.219 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.219 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.219 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.219 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.219 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:24.219 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.220 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.220 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.220 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.220 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:24.220 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.220 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.220 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.220 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.220 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475439+0000", 2024-09-15T21:47:24.220 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1300000000000001e-07 2024-09-15T21:47:24.220 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.220 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.220 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.220 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475441+0000", 2024-09-15T21:47:24.221 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.419e-06 2024-09-15T21:47:24.221 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.221 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.221 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.221 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:24.221 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000269187 2024-09-15T21:47:24.221 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.221 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.221 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.221 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475715+0000", 2024-09-15T21:47:24.221 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3660000000000002e-06 2024-09-15T21:47:24.221 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.221 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.221 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.222 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476671+0000", 2024-09-15T21:47:24.222 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000956507 2024-09-15T21:47:24.222 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.222 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.222 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.222 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477500+0000", 2024-09-15T21:47:24.222 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00082897400000000001 2024-09-15T21:47:24.222 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.222 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.222 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:24.222 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477625+0000", 2024-09-15T21:47:24.222 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012513100000000001 2024-09-15T21:47:24.222 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.222 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.222 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.223 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480520+0000", 2024-09-15T21:47:24.223 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002894715 2024-09-15T21:47:24.223 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.223 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.223 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.223 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481184+0000", 2024-09-15T21:47:24.223 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000663801 2024-09-15T21:47:24.223 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.223 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.223 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.223 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481232+0000", 2024-09-15T21:47:24.223 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8457e-05 2024-09-15T21:47:24.223 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.223 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.224 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.224 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481270+0000", 2024-09-15T21:47:24.224 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7313e-05 2024-09-15T21:47:24.224 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.224 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.224 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.224 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.224 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.224 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.224 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:24.224 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.680320678000001, 2024-09-15T21:47:24.224 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0075605530000000002, 2024-09-15T21:47:24.224 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.224 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.225 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.225 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.225 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.225 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.225 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:24.225 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.225 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.225 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.225 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.225 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:24.225 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.225 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.225 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.225 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.226 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:24.226 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.53e-07 2024-09-15T21:47:24.226 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.226 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.226 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.226 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475445+0000", 2024-09-15T21:47:24.226 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7360000000000001e-06 2024-09-15T21:47:24.226 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.226 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.226 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.226 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475735+0000", 2024-09-15T21:47:24.226 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028975799999999999 2024-09-15T21:47:24.226 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.226 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.226 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.227 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475740+0000", 2024-09-15T21:47:24.227 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8250000000000004e-06 2024-09-15T21:47:24.227 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.227 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.227 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.227 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478479+0000", 2024-09-15T21:47:24.227 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002739525 2024-09-15T21:47:24.227 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.227 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.227 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.227 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478556+0000", 2024-09-15T21:47:24.227 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6213999999999999e-05 2024-09-15T21:47:24.227 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.227 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.228 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:24.228 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478672+0000", 2024-09-15T21:47:24.228 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011605 2024-09-15T21:47:24.228 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.228 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.228 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.228 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481323+0000", 2024-09-15T21:47:24.228 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0026518819999999999 2024-09-15T21:47:24.228 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.228 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.228 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.228 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482918+0000", 2024-09-15T21:47:24.228 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001594105 2024-09-15T21:47:24.228 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.229 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.229 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.229 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482965+0000", 2024-09-15T21:47:24.229 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6950000000000003e-05 2024-09-15T21:47:24.229 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.229 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.229 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.229 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483003+0000", 2024-09-15T21:47:24.229 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8254999999999999e-05 2024-09-15T21:47:24.229 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.229 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.229 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.229 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.229 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.230 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.230 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:24.230 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.680316621999999, 2024-09-15T21:47:24.230 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076582890000000004, 2024-09-15T21:47:24.230 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.230 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.230 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.230 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.230 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.230 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.230 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:24.230 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.230 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.230 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.231 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.231 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:24.231 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.231 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.231 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.231 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.231 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475447+0000", 2024-09-15T21:47:24.231 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4699999999999998e-07 2024-09-15T21:47:24.231 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.231 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.231 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.231 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475449+0000", 2024-09-15T21:47:24.231 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.176e-06 2024-09-15T21:47:24.231 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475752+0000", 2024-09-15T21:47:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030374200000000002 2024-09-15T21:47:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475757+0000", 2024-09-15T21:47:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2209999999999999e-06 2024-09-15T21:47:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477441+0000", 2024-09-15T21:47:24.232 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016843489999999999 2024-09-15T21:47:24.233 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.233 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.233 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.233 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477567+0000", 2024-09-15T21:47:24.233 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012552099999999999 2024-09-15T21:47:24.233 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.233 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.233 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:24.233 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477712+0000", 2024-09-15T21:47:24.233 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000145427 2024-09-15T21:47:24.233 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.233 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.233 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.233 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482935+0000", 2024-09-15T21:47:24.233 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052231200000000004 2024-09-15T21:47:24.234 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.234 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.234 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.234 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483015+0000", 2024-09-15T21:47:24.234 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9634000000000004e-05 2024-09-15T21:47:24.234 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.234 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.234 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.234 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483065+0000", 2024-09-15T21:47:24.234 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9869e-05 2024-09-15T21:47:24.234 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.234 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.234 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.234 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483105+0000", 2024-09-15T21:47:24.235 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9983e-05 2024-09-15T21:47:24.235 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.235 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.235 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.235 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.235 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.235 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.235 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:24.235 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.680310081000002, 2024-09-15T21:47:24.235 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076459379999999997, 2024-09-15T21:47:24.235 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.235 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.235 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.235 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.235 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.236 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.236 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:24.236 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.236 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.236 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.236 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.236 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:24.236 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.236 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.236 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.237 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.237 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:24.237 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.58e-07 2024-09-15T21:47:24.237 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.237 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.237 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.237 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475455+0000", 2024-09-15T21:47:24.237 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.075e-06 2024-09-15T21:47:24.237 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.237 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.237 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.237 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475769+0000", 2024-09-15T21:47:24.237 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00031336999999999999 2024-09-15T21:47:24.237 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.237 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.238 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.238 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475773+0000", 2024-09-15T21:47:24.238 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8080000000000004e-06 2024-09-15T21:47:24.238 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.238 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.238 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.238 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478950+0000", 2024-09-15T21:47:24.238 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031763870000000001 2024-09-15T21:47:24.238 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.238 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.238 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.238 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479067+0000", 2024-09-15T21:47:24.238 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000117543 2024-09-15T21:47:24.238 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.238 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.239 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:24.239 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479196+0000", 2024-09-15T21:47:24.239 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000128987 2024-09-15T21:47:24.239 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.239 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.239 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.239 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481366+0000", 2024-09-15T21:47:24.239 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021699089999999998 2024-09-15T21:47:24.239 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.239 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.239 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.239 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483015+0000", 2024-09-15T21:47:24.239 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016486719999999999 2024-09-15T21:47:24.240 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.240 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.240 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.240 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483061+0000", 2024-09-15T21:47:24.240 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5729999999999998e-05 2024-09-15T21:47:24.240 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.240 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.240 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.240 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483099+0000", 2024-09-15T21:47:24.240 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8099000000000002e-05 2024-09-15T21:47:24.240 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.240 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.240 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.240 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.240 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.241 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.241 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:24.241 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.680262972000001, 2024-09-15T21:47:24.241 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077290859999999996, 2024-09-15T21:47:24.241 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.241 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.241 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.241 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.241 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.241 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.241 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:24.241 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.241 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.242 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.242 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.242 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:24.242 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.242 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.242 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.242 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.242 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475501+0000", 2024-09-15T21:47:24.242 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1900000000000002e-07 2024-09-15T21:47:24.242 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.242 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.242 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.242 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475504+0000", 2024-09-15T21:47:24.242 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.168e-06 2024-09-15T21:47:24.243 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.243 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.243 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.243 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477037+0000", 2024-09-15T21:47:24.243 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001533202 2024-09-15T21:47:24.243 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.243 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.243 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.243 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477050+0000", 2024-09-15T21:47:24.243 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3132999999999999e-05 2024-09-15T21:47:24.243 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.243 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.244 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.244 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479477+0000", 2024-09-15T21:47:24.244 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002426489 2024-09-15T21:47:24.244 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.244 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.244 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.244 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479553+0000", 2024-09-15T21:47:24.244 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6583999999999997e-05 2024-09-15T21:47:24.244 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.244 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.244 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:24.244 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479637+0000", 2024-09-15T21:47:24.244 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4172999999999995e-05 2024-09-15T21:47:24.244 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.245 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.245 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.245 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483108+0000", 2024-09-15T21:47:24.245 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003470356 2024-09-15T21:47:24.245 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.245 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.245 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.245 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483160+0000", 2024-09-15T21:47:24.245 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2507999999999999e-05 2024-09-15T21:47:24.245 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.245 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.245 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.245 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483202+0000", 2024-09-15T21:47:24.246 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1758999999999998e-05 2024-09-15T21:47:24.246 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.246 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.246 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.246 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483229+0000", 2024-09-15T21:47:24.246 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6995000000000002e-05 2024-09-15T21:47:24.246 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.246 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.246 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.246 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.246 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.246 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.246 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:24.246 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.680258015, 2024-09-15T21:47:24.246 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.55145717800000005, 2024-09-15T21:47:24.247 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.247 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.247 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:24.247 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.247 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.247 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.247 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:24.247 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.247 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.247 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.247 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.247 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:24.248 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.248 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.248 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.248 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.248 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:24.248 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.65e-07 2024-09-15T21:47:24.248 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.248 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.248 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.248 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475507+0000", 2024-09-15T21:47:24.248 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4699999999999999e-06 2024-09-15T21:47:24.248 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.248 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.248 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.249 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.025935+0000", 2024-09-15T21:47:24.249 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.55042854200000002 2024-09-15T21:47:24.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.249 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.249 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.249 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.025965+0000", 2024-09-15T21:47:24.249 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9995999999999999e-05 2024-09-15T21:47:24.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.249 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.249 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.249 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026016+0000", 2024-09-15T21:47:24.249 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1310000000000002e-05 2024-09-15T21:47:24.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.249 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.249 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.250 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026122+0000", 2024-09-15T21:47:24.250 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105605 2024-09-15T21:47:24.250 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.250 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.250 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.250 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026962+0000", 2024-09-15T21:47:24.250 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00083998999999999999 2024-09-15T21:47:24.250 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.250 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.250 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.250 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.250 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.250 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:5 2.e 2:75ee5fd8:::629.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.250 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:24.251 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 32.591051649999997, 2024-09-15T21:47:24.251 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0071955470000000001, 2024-09-15T21:47:24.251 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.251 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.251 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.251 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.251 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.251 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.251 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:24.251 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.251 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.251 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.251 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:24.252 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.252 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.252 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.252 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.252 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564713+0000", 2024-09-15T21:47:24.252 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.866e-06 2024-09-15T21:47:24.252 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.252 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.252 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.252 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564721+0000", 2024-09-15T21:47:24.252 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3299999999999999e-06 2024-09-15T21:47:24.252 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.252 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.252 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.252 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564762+0000", 2024-09-15T21:47:24.253 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0707999999999997e-05 2024-09-15T21:47:24.253 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.253 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.253 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.253 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564774+0000", 2024-09-15T21:47:24.253 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1883e-05 2024-09-15T21:47:24.253 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.253 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.253 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.253 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564877+0000", 2024-09-15T21:47:24.253 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010254700000000001 2024-09-15T21:47:24.253 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.253 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.254 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.254 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565066+0000", 2024-09-15T21:47:24.254 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018933700000000001 2024-09-15T21:47:24.254 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.254 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.254 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:24.254 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565164+0000", 2024-09-15T21:47:24.254 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7633999999999994e-05 2024-09-15T21:47:24.254 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.254 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.254 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.254 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569030+0000", 2024-09-15T21:47:24.254 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038669070000000002 2024-09-15T21:47:24.255 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.255 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.255 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.255 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571818+0000", 2024-09-15T21:47:24.255 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027878930000000001 2024-09-15T21:47:24.255 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.255 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.255 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.255 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571868+0000", 2024-09-15T21:47:24.255 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9880999999999999e-05 2024-09-15T21:47:24.255 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.255 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.256 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.256 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571907+0000", 2024-09-15T21:47:24.256 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8560999999999999e-05 2024-09-15T21:47:24.256 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.256 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.256 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.256 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.256 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.256 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:6 2.e 2:75ee5fd8:::629.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.256 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:24.256 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 32.591037522999997, 2024-09-15T21:47:24.256 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073433220000000002, 2024-09-15T21:47:24.256 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.257 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.257 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.257 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.257 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.257 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.257 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:24.257 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.257 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.257 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.257 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.257 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:24.257 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.257 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.258 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.258 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.258 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564726+0000", 2024-09-15T21:47:24.258 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7399999999999999e-07 2024-09-15T21:47:24.258 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.258 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.258 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.258 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564728+0000", 2024-09-15T21:47:24.258 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.447e-06 2024-09-15T21:47:24.258 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.258 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.258 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.258 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565446+0000", 2024-09-15T21:47:24.259 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00071774999999999996 2024-09-15T21:47:24.259 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.259 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.259 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.259 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565453+0000", 2024-09-15T21:47:24.259 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3239999999999997e-06 2024-09-15T21:47:24.259 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.259 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.259 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.259 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565515+0000", 2024-09-15T21:47:24.259 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1768999999999998e-05 2024-09-15T21:47:24.259 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.260 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.260 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.260 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565591+0000", 2024-09-15T21:47:24.260 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5873999999999999e-05 2024-09-15T21:47:24.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.260 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.260 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:24.260 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565709+0000", 2024-09-15T21:47:24.260 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011838899999999999 2024-09-15T21:47:24.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.261 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.261 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569039+0000", 2024-09-15T21:47:24.261 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0033292170000000002 2024-09-15T21:47:24.261 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.261 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.261 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571977+0000", 2024-09-15T21:47:24.261 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0029387829999999999 2024-09-15T21:47:24.261 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.261 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.261 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.572032+0000", 2024-09-15T21:47:24.262 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4540999999999998e-05 2024-09-15T21:47:24.262 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.262 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.262 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.262 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.572069+0000", 2024-09-15T21:47:24.262 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6854000000000003e-05 2024-09-15T21:47:24.262 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.262 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.262 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.262 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.262 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.262 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:2 2.1 2:82971f97:::404.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.262 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:24.262 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 32.591033555999999, 2024-09-15T21:47:24.263 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007245357, 2024-09-15T21:47:24.263 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.264 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.264 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.264 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.264 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.264 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.264 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:24.264 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.264 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.264 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.264 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.264 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:24.265 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.265 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.265 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564730+0000", 2024-09-15T21:47:24.265 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2999999999999999e-07 2024-09-15T21:47:24.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.265 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.265 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564731+0000", 2024-09-15T21:47:24.265 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.367e-06 2024-09-15T21:47:24.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.266 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.266 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565484+0000", 2024-09-15T21:47:24.266 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075311599999999996 2024-09-15T21:47:24.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.266 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.266 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.266 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565490+0000", 2024-09-15T21:47:24.266 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8100000000000003e-06 2024-09-15T21:47:24.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.266 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.266 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.266 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565828+0000", 2024-09-15T21:47:24.266 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000337836 2024-09-15T21:47:24.267 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.267 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.267 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.267 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565964+0000", 2024-09-15T21:47:24.267 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013641599999999999 2024-09-15T21:47:24.267 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.267 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.267 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:24.267 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566071+0000", 2024-09-15T21:47:24.267 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000107126 2024-09-15T21:47:24.267 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.267 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.267 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.267 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571361+0000", 2024-09-15T21:47:24.268 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052896210000000004 2024-09-15T21:47:24.268 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.268 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.268 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.268 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571874+0000", 2024-09-15T21:47:24.268 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051320599999999999 2024-09-15T21:47:24.268 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.268 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.268 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.268 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571936+0000", 2024-09-15T21:47:24.268 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1650999999999996e-05 2024-09-15T21:47:24.268 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.268 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.268 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.269 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571975+0000", 2024-09-15T21:47:24.269 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8977999999999998e-05 2024-09-15T21:47:24.269 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.269 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.269 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.269 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.269 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.269 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:24.270 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:47:24.271 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:24.269+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:24.371 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:24.368+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:24.380 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:24.380 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:24.380 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:24.380 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:24.389 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:47:24.472 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:24.471+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:24.539 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:24.539 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:24.539 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:24.540 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:24.540 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:24.549 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:47:24.573 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:24.572+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:24.657 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:24.657 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:24.657 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:24.657 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:24.657 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.658 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:24.658 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:47:24.658 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 56.240343905000003, 2024-09-15T21:47:24.658 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047486860000000002, 2024-09-15T21:47:24.658 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.658 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.658 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:24.658 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.658 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.658 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.658 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:47:24.658 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.658 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.658 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.659 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.659 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:47:24.659 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.659 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.659 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.659 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.659 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413816+0000", 2024-09-15T21:47:24.659 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4810000000000003e-06 2024-09-15T21:47:24.659 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.659 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.659 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.659 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413827+0000", 2024-09-15T21:47:24.659 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.187e-05 2024-09-15T21:47:24.660 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.660 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.660 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.660 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413862+0000", 2024-09-15T21:47:24.660 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4041000000000002e-05 2024-09-15T21:47:24.660 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.660 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.660 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.660 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413871+0000", 2024-09-15T21:47:24.660 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8919999999999995e-06 2024-09-15T21:47:24.660 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.660 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.660 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.661 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413997+0000", 2024-09-15T21:47:24.661 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012601299999999999 2024-09-15T21:47:24.661 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.661 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.661 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.661 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414100+0000", 2024-09-15T21:47:24.661 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102555 2024-09-15T21:47:24.661 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.661 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.661 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.661 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418558+0000", 2024-09-15T21:47:24.661 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044578339999999999 2024-09-15T21:47:24.661 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.662 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.662 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.662 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.662 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.662 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:24.662 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:47:24.662 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 56.236699090999998, 2024-09-15T21:47:24.662 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045423590000000002, 2024-09-15T21:47:24.662 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.662 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.663 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.663 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.663 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.663 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.663 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:47:24.663 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.663 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.663 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.663 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.663 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:47:24.663 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.663 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.663 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.663 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.664 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417455+0000", 2024-09-15T21:47:24.664 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9100000000000003e-07 2024-09-15T21:47:24.664 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.664 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.664 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.664 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417463+0000", 2024-09-15T21:47:24.664 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3750000000000003e-06 2024-09-15T21:47:24.664 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.664 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.664 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.664 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417475+0000", 2024-09-15T21:47:24.664 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2473000000000001e-05 2024-09-15T21:47:24.664 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.665 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.665 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.665 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417482+0000", 2024-09-15T21:47:24.665 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4010000000000002e-06 2024-09-15T21:47:24.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.665 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.665 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.665 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417524+0000", 2024-09-15T21:47:24.665 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1870999999999999e-05 2024-09-15T21:47:24.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.665 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.665 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.665 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417542+0000", 2024-09-15T21:47:24.666 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8224e-05 2024-09-15T21:47:24.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.666 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.666 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.666 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421955+0000", 2024-09-15T21:47:24.666 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044130740000000003 2024-09-15T21:47:24.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.666 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.666 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.666 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421996+0000", 2024-09-15T21:47:24.666 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.125e-05 2024-09-15T21:47:24.666 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.666 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.666 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.667 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.667 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.667 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 7 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:24.667 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:24.667 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 56.236438286000002, 2024-09-15T21:47:24.667 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043086519999999996, 2024-09-15T21:47:24.667 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.667 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.667 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:47:24.667 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.667 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.667 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.667 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:24.668 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.668 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.668 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.668 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.668 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:24.668 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.668 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.668 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.668 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.668 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:24.668 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.58e-07 2024-09-15T21:47:24.668 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.668 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.669 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.669 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417722+0000", 2024-09-15T21:47:24.669 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4729999999999997e-06 2024-09-15T21:47:24.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.669 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.669 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.669 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417737+0000", 2024-09-15T21:47:24.669 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5155e-05 2024-09-15T21:47:24.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.669 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.669 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.669 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417752+0000", 2024-09-15T21:47:24.669 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4863e-05 2024-09-15T21:47:24.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.670 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.670 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.670 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417819+0000", 2024-09-15T21:47:24.670 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7680000000000003e-05 2024-09-15T21:47:24.670 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.670 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.671 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.671 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422023+0000", 2024-09-15T21:47:24.671 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042040230000000003 2024-09-15T21:47:24.672 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.672 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.672 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.672 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.672 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.672 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:25 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.672 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:47:24.672 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 50.196949256000003, 2024-09-15T21:47:24.672 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070572830000000001, 2024-09-15T21:47:24.672 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.672 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.672 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.672 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.673 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.673 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.673 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:47:24.673 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.673 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.673 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.673 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.673 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:47:24.673 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.673 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.673 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.673 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.673 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457205+0000", 2024-09-15T21:47:24.674 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6899999999999999e-06 2024-09-15T21:47:24.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.674 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.674 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.674 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457211+0000", 2024-09-15T21:47:24.674 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.626e-06 2024-09-15T21:47:24.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.674 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.674 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.674 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457244+0000", 2024-09-15T21:47:24.674 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2691999999999998e-05 2024-09-15T21:47:24.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.674 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.675 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.675 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457251+0000", 2024-09-15T21:47:24.675 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3839999999999999e-06 2024-09-15T21:47:24.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.675 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.675 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.675 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457309+0000", 2024-09-15T21:47:24.675 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8023999999999999e-05 2024-09-15T21:47:24.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.675 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.675 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.675 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457413+0000", 2024-09-15T21:47:24.675 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010331999999999999 2024-09-15T21:47:24.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.676 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.676 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:24.676 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:24.676 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118439 2024-09-15T21:47:24.676 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.676 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.676 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.676 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463246+0000", 2024-09-15T21:47:24.676 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057148219999999996 2024-09-15T21:47:24.676 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.676 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.676 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.677 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463428+0000", 2024-09-15T21:47:24.677 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018218299999999999 2024-09-15T21:47:24.677 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.677 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.677 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.677 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464212+0000", 2024-09-15T21:47:24.677 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00078402799999999998 2024-09-15T21:47:24.677 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.677 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.677 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.677 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464261+0000", 2024-09-15T21:47:24.677 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9075e-05 2024-09-15T21:47:24.677 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.678 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.678 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.678 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.678 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.678 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:28 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.678 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:24.678 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 50.196939229000002, 2024-09-15T21:47:24.678 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0072141449999999999, 2024-09-15T21:47:24.678 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.678 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.678 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:24.678 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.679 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.679 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.679 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:24.679 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.679 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.679 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.679 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:24.679 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.679 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.679 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.679 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:24.680 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:47:24.680 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.680 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.680 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.680 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457217+0000", 2024-09-15T21:47:24.680 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7259999999999998e-06 2024-09-15T21:47:24.680 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.680 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.680 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.680 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457842+0000", 2024-09-15T21:47:24.680 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00062482599999999998 2024-09-15T21:47:24.680 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.681 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.681 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.681 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457848+0000", 2024-09-15T21:47:24.681 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7089999999999996e-06 2024-09-15T21:47:24.681 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.681 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.681 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.681 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457895+0000", 2024-09-15T21:47:24.681 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6317999999999999e-05 2024-09-15T21:47:24.681 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.681 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.681 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.681 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458013+0000", 2024-09-15T21:47:24.681 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011848100000000001 2024-09-15T21:47:24.682 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.682 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.682 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.682 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464428+0000", 2024-09-15T21:47:24.682 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0064146990000000003 2024-09-15T21:47:24.682 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.682 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.682 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.682 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.682 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.682 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:29 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.682 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:47:24.682 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 50.196934786, 2024-09-15T21:47:24.683 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073658980000000001, 2024-09-15T21:47:24.683 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.683 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.683 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:24.683 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.683 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.683 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.683 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:47:24.683 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.683 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.683 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.683 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.683 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:47:24.683 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.684 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.684 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.684 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.684 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457219+0000", 2024-09-15T21:47:24.684 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2000000000000001e-07 2024-09-15T21:47:24.684 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.684 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.684 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.684 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457226+0000", 2024-09-15T21:47:24.684 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.956e-06 2024-09-15T21:47:24.684 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.684 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.685 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.685 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458266+0000", 2024-09-15T21:47:24.685 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001040476 2024-09-15T21:47:24.685 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.686 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.686 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.686 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458273+0000", 2024-09-15T21:47:24.686 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0690000000000004e-06 2024-09-15T21:47:24.686 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.686 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.686 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.686 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458319+0000", 2024-09-15T21:47:24.686 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5668999999999999e-05 2024-09-15T21:47:24.686 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.686 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.686 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.687 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458415+0000", 2024-09-15T21:47:24.687 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6701999999999997e-05 2024-09-15T21:47:24.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.687 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.687 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.687 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464584+0000", 2024-09-15T21:47:24.687 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061687060000000004 2024-09-15T21:47:24.687 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.687 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.687 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.687 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.687 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.1 epoch 22/19 rep_tid 12 entries 22'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.10:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:24.688 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:47:24.688 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 50.19550478, 2024-09-15T21:47:24.688 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0039881550000000002, 2024-09-15T21:47:24.688 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.688 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.688 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:47:24.688 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.688 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.689 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.689 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:47:24.689 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.689 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.689 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.689 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.689 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:47:24.689 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.689 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.689 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.689 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.690 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458649+0000", 2024-09-15T21:47:24.690 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9599999999999999e-07 2024-09-15T21:47:24.690 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.690 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.690 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.690 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458656+0000", 2024-09-15T21:47:24.690 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2629999999999997e-06 2024-09-15T21:47:24.690 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.690 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.690 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.690 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458670+0000", 2024-09-15T21:47:24.690 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4166e-05 2024-09-15T21:47:24.690 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.691 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.691 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.691 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458682+0000", 2024-09-15T21:47:24.691 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1865000000000001e-05 2024-09-15T21:47:24.691 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.691 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.691 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.691 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458749+0000", 2024-09-15T21:47:24.691 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6507000000000005e-05 2024-09-15T21:47:24.691 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.691 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.691 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.691 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462636+0000", 2024-09-15T21:47:24.692 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038877579999999998 2024-09-15T21:47:24.692 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.692 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.692 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.692 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.692 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.692 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.1.10:12 2.12 e22/19 2:48cd7578:::60e.00000000:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:24.692 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:24.692 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 50.195190447999998, 2024-09-15T21:47:24.692 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042479500000000003, 2024-09-15T21:47:24.692 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.693 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.693 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.693 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.693 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.693 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.693 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:24.693 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.693 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.693 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.693 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.693 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:24.693 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.693 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.693 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.694 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.694 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:24.694 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2900000000000004e-07 2024-09-15T21:47:24.694 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.694 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.694 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.694 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458968+0000", 2024-09-15T21:47:24.694 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2039999999999996e-06 2024-09-15T21:47:24.694 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.694 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.694 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.694 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458977+0000", 2024-09-15T21:47:24.695 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.6179999999999995e-06 2024-09-15T21:47:24.695 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.695 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.695 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.695 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458983+0000", 2024-09-15T21:47:24.695 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9510000000000003e-06 2024-09-15T21:47:24.695 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.695 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.695 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.695 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459479+0000", 2024-09-15T21:47:24.695 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000496274 2024-09-15T21:47:24.695 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.695 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.695 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.696 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459496+0000", 2024-09-15T21:47:24.696 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6538e-05 2024-09-15T21:47:24.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.696 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.696 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.696 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463169+0000", 2024-09-15T21:47:24.696 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003672832 2024-09-15T21:47:24.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.696 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.696 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.696 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463211+0000", 2024-09-15T21:47:24.696 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2004e-05 2024-09-15T21:47:24.697 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.697 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.697 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.697 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.697 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.697 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.12 epoch 22/19 rep_tid 2 entries 22'3 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.10:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:24.697 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:47:24.697 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 50.194510893999997, 2024-09-15T21:47:24.697 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035859609999999999, 2024-09-15T21:47:24.697 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.697 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.697 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:47:24.697 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.697 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.698 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.698 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:47:24.698 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.698 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.698 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.698 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.698 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:47:24.698 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.698 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.698 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.698 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.698 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459643+0000", 2024-09-15T21:47:24.698 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2200000000000004e-07 2024-09-15T21:47:24.699 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.699 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.699 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.699 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459650+0000", 2024-09-15T21:47:24.699 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4479999999999997e-06 2024-09-15T21:47:24.699 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.699 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.699 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.699 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459665+0000", 2024-09-15T21:47:24.699 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4586999999999999e-05 2024-09-15T21:47:24.699 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.699 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.699 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.700 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459676+0000", 2024-09-15T21:47:24.700 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1673e-05 2024-09-15T21:47:24.700 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.700 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.700 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.700 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459732+0000", 2024-09-15T21:47:24.700 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5612999999999998e-05 2024-09-15T21:47:24.700 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.700 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.700 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.700 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463228+0000", 2024-09-15T21:47:24.700 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0034961179999999999 2024-09-15T21:47:24.700 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.700 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.701 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.701 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.701 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.701 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:34 2.1b 2:db457a8a:::201.00000001:head [write 0~588] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.701 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:47:24.701 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 50.189299257999998, 2024-09-15T21:47:24.701 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070946129999999996, 2024-09-15T21:47:24.701 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.701 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.701 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.701 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.701 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.701 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.702 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:47:24.702 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.702 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.702 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.702 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.702 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:47:24.702 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.702 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.702 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.702 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.702 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464855+0000", 2024-09-15T21:47:24.702 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5880000000000001e-06 2024-09-15T21:47:24.702 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.702 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.703 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.703 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464863+0000", 2024-09-15T21:47:24.703 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0879999999999995e-06 2024-09-15T21:47:24.703 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.703 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.703 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.703 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464889+0000", 2024-09-15T21:47:24.703 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5729e-05 2024-09-15T21:47:24.703 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.703 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.703 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.703 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464898+0000", 2024-09-15T21:47:24.703 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0389999999999993e-06 2024-09-15T21:47:24.703 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.704 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.704 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.704 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464962+0000", 2024-09-15T21:47:24.704 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3567999999999999e-05 2024-09-15T21:47:24.704 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.704 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.704 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.704 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.465041+0000", 2024-09-15T21:47:24.704 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8889999999999999e-05 2024-09-15T21:47:24.704 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.704 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.704 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:24.704 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.465173+0000", 2024-09-15T21:47:24.705 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013226899999999999 2024-09-15T21:47:24.705 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.705 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.705 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.705 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.466319+0000", 2024-09-15T21:47:24.705 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011463039999999999 2024-09-15T21:47:24.705 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.705 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.705 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.705 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471825+0000", 2024-09-15T21:47:24.705 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055062160000000004 2024-09-15T21:47:24.705 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.705 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.706 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.706 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471886+0000", 2024-09-15T21:47:24.706 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0503e-05 2024-09-15T21:47:24.706 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.706 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.706 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.706 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471948+0000", 2024-09-15T21:47:24.706 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2419e-05 2024-09-15T21:47:24.706 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.706 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.706 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.706 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.706 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.706 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.707 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:47:24.707 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 44.178641094, 2024-09-15T21:47:24.707 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047360500000000003, 2024-09-15T21:47:24.707 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.707 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.707 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.707 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.707 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.707 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.707 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:47:24.707 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.707 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.707 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.707 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.708 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:47:24.708 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.708 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.708 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.708 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.708 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475514+0000", 2024-09-15T21:47:24.708 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0990000000000001e-06 2024-09-15T21:47:24.708 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.708 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.708 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.708 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475524+0000", 2024-09-15T21:47:24.708 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5850000000000006e-06 2024-09-15T21:47:24.708 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.709 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.709 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.709 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475557+0000", 2024-09-15T21:47:24.709 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3090000000000003e-05 2024-09-15T21:47:24.709 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.709 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.709 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.709 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475568+0000", 2024-09-15T21:47:24.709 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1116e-05 2024-09-15T21:47:24.709 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.709 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.709 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.709 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475649+0000", 2024-09-15T21:47:24.710 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0815999999999993e-05 2024-09-15T21:47:24.710 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.710 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.710 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.710 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475805+0000", 2024-09-15T21:47:24.710 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015667200000000001 2024-09-15T21:47:24.710 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.710 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.710 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:24.710 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475944+0000", 2024-09-15T21:47:24.710 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000138651 2024-09-15T21:47:24.710 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.710 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.710 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.711 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476900+0000", 2024-09-15T21:47:24.711 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00095588699999999999 2024-09-15T21:47:24.711 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.711 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.711 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.711 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480153+0000", 2024-09-15T21:47:24.711 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0032535200000000002 2024-09-15T21:47:24.711 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.711 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.711 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.711 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480203+0000", 2024-09-15T21:47:24.712 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9932999999999998e-05 2024-09-15T21:47:24.712 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.712 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.712 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.712 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480248+0000", 2024-09-15T21:47:24.712 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4681000000000001e-05 2024-09-15T21:47:24.712 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.712 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.712 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.712 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.712 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.712 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.712 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:24.713 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 44.178625152999999, 2024-09-15T21:47:24.713 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049206709999999997, 2024-09-15T21:47:24.713 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.713 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.713 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.713 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.713 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.713 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.713 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:24.713 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.713 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.713 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.714 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.714 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:24.714 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.714 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.714 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.714 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.714 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:24.714 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1699999999999999e-07 2024-09-15T21:47:24.714 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.714 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.714 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.714 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475531+0000", 2024-09-15T21:47:24.714 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.05e-06 2024-09-15T21:47:24.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.715 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.715 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.715 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475743+0000", 2024-09-15T21:47:24.715 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021138099999999999 2024-09-15T21:47:24.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.715 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.715 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.715 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475751+0000", 2024-09-15T21:47:24.715 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7770000000000002e-06 2024-09-15T21:47:24.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.715 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.715 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.716 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476272+0000", 2024-09-15T21:47:24.716 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052158399999999996 2024-09-15T21:47:24.716 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.716 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.716 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.716 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476348+0000", 2024-09-15T21:47:24.716 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5963000000000004e-05 2024-09-15T21:47:24.716 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.716 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.716 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:24.716 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476474+0000", 2024-09-15T21:47:24.716 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012638800000000001 2024-09-15T21:47:24.716 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.716 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.717 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.717 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478194+0000", 2024-09-15T21:47:24.717 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001719768 2024-09-15T21:47:24.717 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.717 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.717 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.717 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480358+0000", 2024-09-15T21:47:24.717 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021637340000000001 2024-09-15T21:47:24.717 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.717 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.717 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.717 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480411+0000", 2024-09-15T21:47:24.717 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2720999999999999e-05 2024-09-15T21:47:24.718 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.718 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.718 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.718 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480449+0000", 2024-09-15T21:47:24.718 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7888e-05 2024-09-15T21:47:24.718 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.718 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.718 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.718 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.718 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.718 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.718 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:24.718 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 44.178620363999997, 2024-09-15T21:47:24.719 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005215478, 2024-09-15T21:47:24.719 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.719 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.719 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:24.719 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.719 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.719 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.719 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:24.719 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.719 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.719 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.719 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.719 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:24.720 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.720 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.720 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.720 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.720 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:24.720 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5199999999999998e-07 2024-09-15T21:47:24.720 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.720 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.720 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.720 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475535+0000", 2024-09-15T21:47:24.720 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9520000000000001e-06 2024-09-15T21:47:24.720 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.720 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.721 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.721 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475785+0000", 2024-09-15T21:47:24.721 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024966099999999998 2024-09-15T21:47:24.721 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.721 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.721 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.721 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475799+0000", 2024-09-15T21:47:24.721 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4649e-05 2024-09-15T21:47:24.721 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.721 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.721 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.721 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475856+0000", 2024-09-15T21:47:24.721 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6956000000000002e-05 2024-09-15T21:47:24.722 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.722 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.722 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.722 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476016+0000", 2024-09-15T21:47:24.722 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000159457 2024-09-15T21:47:24.722 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.722 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.722 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.722 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480748+0000", 2024-09-15T21:47:24.722 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047324510000000004 2024-09-15T21:47:24.722 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.722 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.722 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.722 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.723 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.723 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.2.12:1 2.1 e22/19 2:85bbe569:::202.00000000:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:24.723 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:47:24.723 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 44.175817893000001, 2024-09-15T21:47:24.723 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045423649999999996, 2024-09-15T21:47:24.723 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.723 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.723 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.723 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.723 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.723 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.723 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:47:24.723 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.723 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.723 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.724 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.724 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:47:24.724 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.724 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.724 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.724 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.724 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478336+0000", 2024-09-15T21:47:24.724 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4499999999999996e-07 2024-09-15T21:47:24.724 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.724 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.724 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.724 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478342+0000", 2024-09-15T21:47:24.724 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5579999999999997e-06 2024-09-15T21:47:24.725 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.725 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.725 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.725 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478356+0000", 2024-09-15T21:47:24.725 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3567000000000001e-05 2024-09-15T21:47:24.725 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.725 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.725 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.725 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478364+0000", 2024-09-15T21:47:24.725 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7889999999999995e-06 2024-09-15T21:47:24.725 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.725 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.725 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.726 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478416+0000", 2024-09-15T21:47:24.726 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2188000000000002e-05 2024-09-15T21:47:24.726 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.726 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.726 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.726 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478433+0000", 2024-09-15T21:47:24.726 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6674e-05 2024-09-15T21:47:24.726 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.726 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.726 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.726 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482832+0000", 2024-09-15T21:47:24.726 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043996110000000003 2024-09-15T21:47:24.726 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.727 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.727 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.727 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482878+0000", 2024-09-15T21:47:24.727 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5232999999999999e-05 2024-09-15T21:47:24.727 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.727 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.727 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.727 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.727 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.727 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:25 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:24.727 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:47:24.727 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 33.088957854, 2024-09-15T21:47:24.727 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058459619999999997, 2024-09-15T21:47:24.728 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.728 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.728 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.728 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.728 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.728 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.728 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:47:24.728 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.728 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.728 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.728 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.728 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:47:24.728 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.728 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.729 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.729 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.729 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565197+0000", 2024-09-15T21:47:24.729 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7889999999999999e-06 2024-09-15T21:47:24.729 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.729 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.729 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.729 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565208+0000", 2024-09-15T21:47:24.729 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1358999999999999e-05 2024-09-15T21:47:24.729 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.729 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.729 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.729 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565237+0000", 2024-09-15T21:47:24.730 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8626e-05 2024-09-15T21:47:24.730 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.730 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.730 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.730 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565249+0000", 2024-09-15T21:47:24.730 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1897e-05 2024-09-15T21:47:24.730 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.730 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.730 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.730 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565332+0000", 2024-09-15T21:47:24.730 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2943999999999997e-05 2024-09-15T21:47:24.730 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.730 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.730 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.731 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565452+0000", 2024-09-15T21:47:24.731 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119992 2024-09-15T21:47:24.731 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.731 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.731 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:24.731 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565610+0000", 2024-09-15T21:47:24.731 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015845099999999999 2024-09-15T21:47:24.731 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.731 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.731 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:24.731 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.568999+0000", 2024-09-15T21:47:24.731 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0033887190000000001 2024-09-15T21:47:24.731 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.732 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.732 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:24.732 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.570979+0000", 2024-09-15T21:47:24.732 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019804919999999999 2024-09-15T21:47:24.732 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.732 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.732 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.732 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571014+0000", 2024-09-15T21:47:24.733 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4331e-05 2024-09-15T21:47:24.733 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.733 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.733 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.733 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571041+0000", 2024-09-15T21:47:24.733 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7362e-05 2024-09-15T21:47:24.733 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.733 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.733 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.733 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.733 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.733 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:5 2.e e22/19 2:75ee5fd8:::629.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:47:24.733 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:47:24.734 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 33.088282208000003, 2024-09-15T21:47:24.734 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057947119999999996, 2024-09-15T21:47:24.734 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.734 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.734 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.734 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.734 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.734 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.734 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:47:24.734 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.734 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.734 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.734 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.735 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:47:24.735 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.735 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.735 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.735 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.735 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565872+0000", 2024-09-15T21:47:24.735 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9800000000000002e-07 2024-09-15T21:47:24.735 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.735 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.735 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.735 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565880+0000", 2024-09-15T21:47:24.735 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7619999999999995e-06 2024-09-15T21:47:24.735 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.735 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.736 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.736 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565890+0000", 2024-09-15T21:47:24.736 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0648999999999999e-05 2024-09-15T21:47:24.736 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.736 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.736 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.736 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565897+0000", 2024-09-15T21:47:24.736 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.153e-06 2024-09-15T21:47:24.736 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.736 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.736 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.736 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565966+0000", 2024-09-15T21:47:24.736 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9046000000000003e-05 2024-09-15T21:47:24.736 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.737 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.737 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.737 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565985+0000", 2024-09-15T21:47:24.737 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8695e-05 2024-09-15T21:47:24.737 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.737 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.737 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.737 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571625+0000", 2024-09-15T21:47:24.737 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056397139999999997 2024-09-15T21:47:24.737 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.737 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.737 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.737 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571666+0000", 2024-09-15T21:47:24.737 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0695000000000003e-05 2024-09-15T21:47:24.738 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.738 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.738 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.738 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.738 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.738 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:6 2.e e22/19 2:75ee5fd8:::629.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:47:24.738 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:47:24.738 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 33.088219960000004, 2024-09-15T21:47:24.738 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057607429999999996, 2024-09-15T21:47:24.739 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.739 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.739 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.739 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.739 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.739 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.739 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:47:24.739 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.739 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.739 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.739 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.739 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:47:24.739 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.740 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.740 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.740 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.740 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565934+0000", 2024-09-15T21:47:24.740 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6199999999999998e-07 2024-09-15T21:47:24.740 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.740 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.740 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.740 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565938+0000", 2024-09-15T21:47:24.740 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4970000000000003e-06 2024-09-15T21:47:24.740 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.740 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.740 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.740 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565945+0000", 2024-09-15T21:47:24.741 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5069999999999997e-06 2024-09-15T21:47:24.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.741 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.741 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.741 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565948+0000", 2024-09-15T21:47:24.741 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:47:24.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.741 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.741 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.741 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566200+0000", 2024-09-15T21:47:24.741 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025170200000000002 2024-09-15T21:47:24.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.741 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.741 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.742 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566218+0000", 2024-09-15T21:47:24.742 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7543000000000001e-05 2024-09-15T21:47:24.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.742 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.742 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.742 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571668+0000", 2024-09-15T21:47:24.742 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054502999999999999 2024-09-15T21:47:24.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.742 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.742 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.742 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571694+0000", 2024-09-15T21:47:24.742 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5979e-05 2024-09-15T21:47:24.742 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.742 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.743 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.743 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.743 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.743 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:2 2.1 e22/19 2:82971f97:::404.00000000:head v 22'5, mlcod=22'3)", 2024-09-15T21:47:24.743 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:47:24.743 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 33.088012024000001, 2024-09-15T21:47:24.743 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056016019999999998, 2024-09-15T21:47:24.743 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.743 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.743 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.743 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.743 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.743 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.743 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:47:24.744 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.744 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.744 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.744 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.744 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:47:24.744 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.744 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.744 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.744 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.744 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566142+0000", 2024-09-15T21:47:24.744 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:47:24.744 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.744 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.744 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.745 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566149+0000", 2024-09-15T21:47:24.745 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7510000000000008e-06 2024-09-15T21:47:24.745 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.745 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.745 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.745 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566161+0000", 2024-09-15T21:47:24.745 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1259e-05 2024-09-15T21:47:24.745 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.745 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.745 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.745 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566168+0000", 2024-09-15T21:47:24.745 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.255e-06 2024-09-15T21:47:24.745 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.746 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.746 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.746 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566227+0000", 2024-09-15T21:47:24.746 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8961e-05 2024-09-15T21:47:24.746 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.746 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.746 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.746 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566247+0000", 2024-09-15T21:47:24.746 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0536000000000001e-05 2024-09-15T21:47:24.746 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.746 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.746 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.746 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571694+0000", 2024-09-15T21:47:24.747 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005446762 2024-09-15T21:47:24.747 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.747 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.747 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.747 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571743+0000", 2024-09-15T21:47:24.747 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8396999999999998e-05 2024-09-15T21:47:24.747 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.747 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.747 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.747 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.747 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.747 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:17 2.14 e22/19 2:2eebc8f4:::62f.00000000:head v 22'9, mlcod=22'7)", 2024-09-15T21:47:24.747 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:47:24.747 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 33.087995927000001, 2024-09-15T21:47:24.748 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056532860000000004, 2024-09-15T21:47:24.748 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.748 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.748 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.748 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.748 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.748 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.748 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:47:24.748 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.748 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.748 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.748 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.749 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:47:24.749 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.749 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.749 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.749 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.749 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566158+0000", 2024-09-15T21:47:24.749 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.92e-07 2024-09-15T21:47:24.749 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.749 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.749 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.749 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566172+0000", 2024-09-15T21:47:24.750 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3976e-05 2024-09-15T21:47:24.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.750 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.750 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.750 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566407+0000", 2024-09-15T21:47:24.750 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023483900000000001 2024-09-15T21:47:24.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.750 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.750 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.750 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566416+0000", 2024-09-15T21:47:24.750 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0780000000000002e-06 2024-09-15T21:47:24.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.750 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.751 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.751 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566476+0000", 2024-09-15T21:47:24.751 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9908999999999998e-05 2024-09-15T21:47:24.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.751 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.751 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.751 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566493+0000", 2024-09-15T21:47:24.751 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7567e-05 2024-09-15T21:47:24.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.751 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.751 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.751 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571762+0000", 2024-09-15T21:47:24.751 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052682959999999996 2024-09-15T21:47:24.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.752 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.752 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.752 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571810+0000", 2024-09-15T21:47:24.752 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8829000000000003e-05 2024-09-15T21:47:24.752 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.752 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.752 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.752 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.752 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.752 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:18 2.14 e22/19 2:2eebc8f4:::62f.00000000:head v 22'10, mlcod=22'7)", 2024-09-15T21:47:24.752 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:47:24.752 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 33.087706851999997, 2024-09-15T21:47:24.752 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053992579999999997, 2024-09-15T21:47:24.752 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:24.752 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:24.753 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:24.753 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:24.753 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.753 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:24.753 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:47:24.753 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.753 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.753 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.753 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:24.753 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:47:24.753 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:24.753 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.753 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.753 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:24.754 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566447+0000", 2024-09-15T21:47:24.754 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2e-07 2024-09-15T21:47:24.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.754 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.754 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:24.754 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566452+0000", 2024-09-15T21:47:24.754 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0390000000000004e-06 2024-09-15T21:47:24.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.754 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.754 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:24.754 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566460+0000", 2024-09-15T21:47:24.754 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1389999999999995e-06 2024-09-15T21:47:24.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.754 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.755 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:24.755 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566473+0000", 2024-09-15T21:47:24.755 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3093e-05 2024-09-15T21:47:24.755 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.755 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.755 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:24.755 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566688+0000", 2024-09-15T21:47:24.755 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021537100000000001 2024-09-15T21:47:24.755 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.755 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.755 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:24.755 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566706+0000", 2024-09-15T21:47:24.755 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7802999999999999e-05 2024-09-15T21:47:24.756 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.756 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.756 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:24.756 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571813+0000", 2024-09-15T21:47:24.756 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051065140000000004 2024-09-15T21:47:24.756 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:24.756 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:24.756 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:24.756 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571845+0000", 2024-09-15T21:47:24.756 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2879000000000001e-05 2024-09-15T21:47:24.756 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.756 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.756 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.757 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:24.757 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:24.757 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:24.758 INFO:teuthology.orchestra.run.smithi122.stdout:get_pg_num_history pg_num_history pg_num_history(e29 pg_nums {1={15=1},2={19=32},3={20=64}} deleted_pools ) 2024-09-15T21:47:24.758 INFO:teuthology.orchestra.run.smithi122.stdout:Importing pgid 3.22 2024-09-15T21:47:24.758 INFO:teuthology.orchestra.run.smithi122.stdout:write_pg epoch 21 info 3.22( empty local-lis/les=20/21 n=0 ec=20/20 lis/c=20/20 les/c/f=21/21/0 sis=20) 2024-09-15T21:47:24.758 INFO:teuthology.orchestra.run.smithi122.stdout:Import successful 2024-09-15T21:47:24.759 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:24.756+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:24.759 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:47:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:24.741+0000 7f4e677fe640 1 -- 172.21.15.100:0/2789721966 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v34) ==== 101+0+0 (secure 0 0 0) 0x7f4e8015c3a0 con 0x7f4e800ccef0 2024-09-15T21:47:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is unset 2024-09-15T21:47:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:24.744+0000 7f4e8570a640 1 -- 172.21.15.100:0/2789721966 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4e6c06c2b0 msgr2=0x7f4e6c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:24.744+0000 7f4e8570a640 1 --2- 172.21.15.100:0/2789721966 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4e6c06c2b0 0x7f4e6c06e770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4e801eed00 tx=0x7f4e74008040 comp rx=0 tx=0).stop 2024-09-15T21:47:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:24.744+0000 7f4e8570a640 1 -- 172.21.15.100:0/2789721966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e800ccef0 msgr2=0x7f4e801e12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:24.744+0000 7f4e8570a640 1 --2- 172.21.15.100:0/2789721966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e800ccef0 0x7f4e801e12c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4e7002a220 tx=0x7f4e700046c0 comp rx=0 tx=0).stop 2024-09-15T21:47:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:24.744+0000 7f4e8570a640 1 -- 172.21.15.100:0/2789721966 shutdown_connections 2024-09-15T21:47:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:24.744+0000 7f4e8570a640 1 --2- 172.21.15.100:0/2789721966 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4e6c06c2b0 0x7f4e6c06e770 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:24.744+0000 7f4e8570a640 1 --2- 172.21.15.100:0/2789721966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e800d11d0 0x7f4e8014e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:24.744+0000 7f4e8570a640 1 --2- 172.21.15.100:0/2789721966 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e800d0df0 0x7f4e801f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:24.744+0000 7f4e8570a640 1 --2- 172.21.15.100:0/2789721966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e800ccef0 0x7f4e801e12c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:24.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:24.744+0000 7f4e8570a640 1 -- 172.21.15.100:0/2789721966 >> 172.21.15.100:0/2789721966 conn(0x7f4e80067980 msgr2=0x7f4e80154520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:24.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:24.744+0000 7f4e8570a640 1 -- 172.21.15.100:0/2789721966 shutdown_connections 2024-09-15T21:47:24.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:24.744+0000 7f4e8570a640 1 -- 172.21.15.100:0/2789721966 wait complete. 2024-09-15T21:47:24.859 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:24.857+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:24.864 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:24.865 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:24.865 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:24.865 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:24.873 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:47:24.960 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:24.958+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:25.017 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:25.018 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:25.018 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:25.026 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:47:25.060 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:25.059+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:25.161 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:25.160+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:25.171 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:47:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 56.755612755000001, 2024-09-15T21:47:25.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061533810000000003, 2024-09-15T21:47:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:47:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:47:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413085+0000", 2024-09-15T21:47:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8390000000000003e-06 2024-09-15T21:47:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413092+0000", 2024-09-15T21:47:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3479999999999998e-06 2024-09-15T21:47:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413109+0000", 2024-09-15T21:47:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7192000000000001e-05 2024-09-15T21:47:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413118+0000", 2024-09-15T21:47:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.935e-06 2024-09-15T21:47:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413219+0000", 2024-09-15T21:47:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010121 2024-09-15T21:47:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413381+0000", 2024-09-15T21:47:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016148399999999999 2024-09-15T21:47:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413525+0000", 2024-09-15T21:47:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014428100000000001 2024-09-15T21:47:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415866+0000", 2024-09-15T21:47:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023409300000000002 2024-09-15T21:47:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419156+0000", 2024-09-15T21:47:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032903580000000002 2024-09-15T21:47:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419199+0000", 2024-09-15T21:47:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2171e-05 2024-09-15T21:47:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419232+0000", 2024-09-15T21:47:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3633000000000001e-05 2024-09-15T21:47:25.176 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.4:28 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-15T21:47:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:47:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 56.754349245999997, 2024-09-15T21:47:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002614594, 2024-09-15T21:47:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:47:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:47:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414344+0000", 2024-09-15T21:47:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6589999999999999e-06 2024-09-15T21:47:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414355+0000", 2024-09-15T21:47:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0604999999999999e-05 2024-09-15T21:47:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414370+0000", 2024-09-15T21:47:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-15T21:47:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414379+0000", 2024-09-15T21:47:25.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2449999999999998e-06 2024-09-15T21:47:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414477+0000", 2024-09-15T21:47:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8968000000000005e-05 2024-09-15T21:47:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414504+0000", 2024-09-15T21:47:25.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6899000000000001e-05 2024-09-15T21:47:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416837+0000", 2024-09-15T21:47:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023327130000000001 2024-09-15T21:47:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416957+0000", 2024-09-15T21:47:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000119796 2024-09-15T21:47:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.181 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.1.10:8 2.b e22/19 2:d70a01c6:::60c.00000000:head v 22'4, mlcod=22'1)", 2024-09-15T21:47:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:47:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.710769794000001, 2024-09-15T21:47:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027633890000000002, 2024-09-15T21:47:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:47:25.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:47:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457923+0000", 2024-09-15T21:47:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.136e-06 2024-09-15T21:47:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457931+0000", 2024-09-15T21:47:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1529999999999997e-06 2024-09-15T21:47:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457946+0000", 2024-09-15T21:47:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4482e-05 2024-09-15T21:47:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457956+0000", 2024-09-15T21:47:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0774999999999999e-05 2024-09-15T21:47:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458016+0000", 2024-09-15T21:47:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9247999999999999e-05 2024-09-15T21:47:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458036+0000", 2024-09-15T21:47:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0920999999999998e-05 2024-09-15T21:47:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460636+0000", 2024-09-15T21:47:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025992089999999999 2024-09-15T21:47:25.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460685+0000", 2024-09-15T21:47:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9465000000000002e-05 2024-09-15T21:47:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.18 epoch 22/19 rep_tid 5 entries 22'1 (0'0) error 2:1d00e16c:::201.00000005:head by mds.1.10:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:47:25.186 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 50.709077663000002, 2024-09-15T21:47:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003509599, 2024-09-15T21:47:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:47:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:47:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459615+0000", 2024-09-15T21:47:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4200000000000005e-07 2024-09-15T21:47:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459626+0000", 2024-09-15T21:47:25.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1456e-05 2024-09-15T21:47:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459650+0000", 2024-09-15T21:47:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.429e-05 2024-09-15T21:47:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459664+0000", 2024-09-15T21:47:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3805000000000001e-05 2024-09-15T21:47:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459784+0000", 2024-09-15T21:47:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000119614 2024-09-15T21:47:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463123+0000", 2024-09-15T21:47:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033395920000000002 2024-09-15T21:47:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:7 2.13 e22/19 2:cdf3ac81:::616.00000000:head v 22'5, mlcod=22'3)", 2024-09-15T21:47:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:47:25.190 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.691296885, 2024-09-15T21:47:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002138615, 2024-09-15T21:47:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:47:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:47:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477396+0000", 2024-09-15T21:47:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4190000000000001e-06 2024-09-15T21:47:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477405+0000", 2024-09-15T21:47:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2320000000000007e-06 2024-09-15T21:47:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477419+0000", 2024-09-15T21:47:25.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3589e-05 2024-09-15T21:47:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477428+0000", 2024-09-15T21:47:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4779999999999997e-06 2024-09-15T21:47:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477493+0000", 2024-09-15T21:47:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4546000000000002e-05 2024-09-15T21:47:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477511+0000", 2024-09-15T21:47:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7793000000000001e-05 2024-09-15T21:47:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479481+0000", 2024-09-15T21:47:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019701660000000002 2024-09-15T21:47:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479533+0000", 2024-09-15T21:47:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2392000000000003e-05 2024-09-15T21:47:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.194 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:26 2.11 e22/19 2:893cd530:::mds2_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-15T21:47:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.691149959000001, 2024-09-15T21:47:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020450450000000001, 2024-09-15T21:47:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:25.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9699999999999995e-07 2024-09-15T21:47:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477555+0000", 2024-09-15T21:47:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3144e-05 2024-09-15T21:47:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477565+0000", 2024-09-15T21:47:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8579999999999995e-06 2024-09-15T21:47:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477573+0000", 2024-09-15T21:47:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4869999999999996e-06 2024-09-15T21:47:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477632+0000", 2024-09-15T21:47:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9178999999999997e-05 2024-09-15T21:47:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477654+0000", 2024-09-15T21:47:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2234000000000001e-05 2024-09-15T21:47:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479526+0000", 2024-09-15T21:47:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018717600000000001 2024-09-15T21:47:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479587+0000", 2024-09-15T21:47:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0686000000000001e-05 2024-09-15T21:47:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:22 2.1a e22/19 2:5fd68f60:::61d.00000000:head v 22'3, mlcod=0'0)", 2024-09-15T21:47:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.690789903999999, 2024-09-15T21:47:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023694380000000002, 2024-09-15T21:47:25.199 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4899999999999995e-07 2024-09-15T21:47:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477910+0000", 2024-09-15T21:47:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:47:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.201 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477920+0000", 2024-09-15T21:47:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9250000000000007e-06 2024-09-15T21:47:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477927+0000", 2024-09-15T21:47:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0940000000000002e-06 2024-09-15T21:47:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477985+0000", 2024-09-15T21:47:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8451e-05 2024-09-15T21:47:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478005+0000", 2024-09-15T21:47:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0064e-05 2024-09-15T21:47:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480223+0000", 2024-09-15T21:47:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022177910000000002 2024-09-15T21:47:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480271+0000", 2024-09-15T21:47:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8164999999999997e-05 2024-09-15T21:47:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:8 2.13 e22/19 2:cdf3ac81:::616.00000000:head v 22'6, mlcod=22'3)", 2024-09-15T21:47:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:47:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.690590944999997, 2024-09-15T21:47:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021760249999999998, 2024-09-15T21:47:25.204 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:47:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:47:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478102+0000", 2024-09-15T21:47:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3099999999999996e-07 2024-09-15T21:47:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478124+0000", 2024-09-15T21:47:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2790999999999999e-05 2024-09-15T21:47:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478136+0000", 2024-09-15T21:47:25.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1922999999999999e-05 2024-09-15T21:47:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478144+0000", 2024-09-15T21:47:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9300000000000003e-06 2024-09-15T21:47:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478192+0000", 2024-09-15T21:47:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8155999999999998e-05 2024-09-15T21:47:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.207 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478209+0000", 2024-09-15T21:47:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6816e-05 2024-09-15T21:47:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480243+0000", 2024-09-15T21:47:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020336199999999999 2024-09-15T21:47:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480277+0000", 2024-09-15T21:47:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3858e-05 2024-09-15T21:47:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.208 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.11 epoch 22/19 rep_tid 19 entries 22'2 (0'0) error 2:8975f766:::202.00000001:head by mds.2.12:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 44.690465007, 2024-09-15T21:47:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023882299999999999, 2024-09-15T21:47:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:25.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2399999999999998e-07 2024-09-15T21:47:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478236+0000", 2024-09-15T21:47:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5299999999999996e-06 2024-09-15T21:47:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478248+0000", 2024-09-15T21:47:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2806000000000001e-05 2024-09-15T21:47:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478264+0000", 2024-09-15T21:47:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5468000000000002e-05 2024-09-15T21:47:25.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478339+0000", 2024-09-15T21:47:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4851000000000006e-05 2024-09-15T21:47:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480615+0000", 2024-09-15T21:47:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022760509999999999 2024-09-15T21:47:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:3 2.1a e22/19 2:5fdc322d:::61e.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:47:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:47:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.622280404999998, 2024-09-15T21:47:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023961220000000001, 2024-09-15T21:47:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:47:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.213 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:47:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546413+0000", 2024-09-15T21:47:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.812e-06 2024-09-15T21:47:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546424+0000", 2024-09-15T21:47:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1313e-05 2024-09-15T21:47:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546444+0000", 2024-09-15T21:47:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9706000000000001e-05 2024-09-15T21:47:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546453+0000", 2024-09-15T21:47:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:47:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546519+0000", 2024-09-15T21:47:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5710999999999995e-05 2024-09-15T21:47:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546539+0000", 2024-09-15T21:47:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0275000000000002e-05 2024-09-15T21:47:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548763+0000", 2024-09-15T21:47:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002223226 2024-09-15T21:47:25.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548807+0000", 2024-09-15T21:47:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4598000000000002e-05 2024-09-15T21:47:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:19 2.11 e22/19 2:890f50f7:::626.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:47:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:47:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.621983245000003, 2024-09-15T21:47:25.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022671280000000002, 2024-09-15T21:47:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:47:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:47:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546709+0000", 2024-09-15T21:47:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4499999999999997e-07 2024-09-15T21:47:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546723+0000", 2024-09-15T21:47:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4623999999999999e-05 2024-09-15T21:47:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.219 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546734+0000", 2024-09-15T21:47:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0331e-05 2024-09-15T21:47:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546741+0000", 2024-09-15T21:47:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.075e-06 2024-09-15T21:47:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546954+0000", 2024-09-15T21:47:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021275600000000001 2024-09-15T21:47:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546971+0000", 2024-09-15T21:47:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7503000000000002e-05 2024-09-15T21:47:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548926+0000", 2024-09-15T21:47:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019550129999999998 2024-09-15T21:47:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548975+0000", 2024-09-15T21:47:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9280999999999998e-05 2024-09-15T21:47:25.221 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.d epoch 22/19 rep_tid 18 entries 22'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.14:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:47:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.621976895000003, 2024-09-15T21:47:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021738489999999998, 2024-09-15T21:47:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:47:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:47:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546716+0000", 2024-09-15T21:47:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.615e-06 2024-09-15T21:47:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546730+0000", 2024-09-15T21:47:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3519999999999999e-05 2024-09-15T21:47:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546753+0000", 2024-09-15T21:47:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3643999999999999e-05 2024-09-15T21:47:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546770+0000", 2024-09-15T21:47:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6518999999999999e-05 2024-09-15T21:47:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546900+0000", 2024-09-15T21:47:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013049099999999999 2024-09-15T21:47:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548888+0000", 2024-09-15T21:47:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019880599999999998 2024-09-15T21:47:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:4 2.1a e22/19 2:5fdc322d:::61e.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:47:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:47:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.621868247000002, 2024-09-15T21:47:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002968769, 2024-09-15T21:47:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:47:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:47:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546824+0000", 2024-09-15T21:47:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6499999999999999e-07 2024-09-15T21:47:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546831+0000", 2024-09-15T21:47:25.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9940000000000003e-06 2024-09-15T21:47:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546840+0000", 2024-09-15T21:47:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3689999999999996e-06 2024-09-15T21:47:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546847+0000", 2024-09-15T21:47:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0589999999999997e-06 2024-09-15T21:47:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547709+0000", 2024-09-15T21:47:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00086144699999999999 2024-09-15T21:47:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547730+0000", 2024-09-15T21:47:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1542000000000001e-05 2024-09-15T21:47:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549750+0000", 2024-09-15T21:47:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002020165 2024-09-15T21:47:25.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549792+0000", 2024-09-15T21:47:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1628000000000002e-05 2024-09-15T21:47:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:20 2.11 e22/19 2:890f50f7:::626.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:47:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.620843639, 2024-09-15T21:47:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017734579999999999, 2024-09-15T21:47:25.230 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:47:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547856+0000", 2024-09-15T21:47:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7999999999999999e-06 2024-09-15T21:47:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547867+0000", 2024-09-15T21:47:25.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0971999999999999e-05 2024-09-15T21:47:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547881+0000", 2024-09-15T21:47:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4168999999999999e-05 2024-09-15T21:47:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547935+0000", 2024-09-15T21:47:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3721999999999998e-05 2024-09-15T21:47:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547976+0000", 2024-09-15T21:47:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1248999999999998e-05 2024-09-15T21:47:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549559+0000", 2024-09-15T21:47:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001582815 2024-09-15T21:47:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549621+0000", 2024-09-15T21:47:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2156999999999995e-05 2024-09-15T21:47:25.234 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:13 2.1a e22/19 2:594e21d0:::623.00000000:head v 22'6, mlcod=22'2)", 2024-09-15T21:47:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:47:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.620778643000001, 2024-09-15T21:47:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019085300000000001, 2024-09-15T21:47:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:47:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:47:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547914+0000", 2024-09-15T21:47:25.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1500000000000004e-07 2024-09-15T21:47:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547920+0000", 2024-09-15T21:47:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5239999999999997e-06 2024-09-15T21:47:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547930+0000", 2024-09-15T21:47:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5109999999999992e-06 2024-09-15T21:47:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.237 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547937+0000", 2024-09-15T21:47:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5410000000000003e-06 2024-09-15T21:47:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548518+0000", 2024-09-15T21:47:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00058074299999999999 2024-09-15T21:47:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548538+0000", 2024-09-15T21:47:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9959e-05 2024-09-15T21:47:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549795+0000", 2024-09-15T21:47:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012570120000000001 2024-09-15T21:47:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549821+0000", 2024-09-15T21:47:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6625e-05 2024-09-15T21:47:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:23 2.18 e22/19 2:1c1db442:::103.00000000.inode:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:25.239 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:47:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.620717225, 2024-09-15T21:47:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002713198, 2024-09-15T21:47:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:47:25.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:47:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547975+0000", 2024-09-15T21:47:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3900000000000004e-07 2024-09-15T21:47:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547983+0000", 2024-09-15T21:47:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6690000000000006e-06 2024-09-15T21:47:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547994+0000", 2024-09-15T21:47:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1503e-05 2024-09-15T21:47:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548002+0000", 2024-09-15T21:47:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1340000000000004e-06 2024-09-15T21:47:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548098+0000", 2024-09-15T21:47:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5791000000000005e-05 2024-09-15T21:47:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548118+0000", 2024-09-15T21:47:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0129e-05 2024-09-15T21:47:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550643+0000", 2024-09-15T21:47:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002524728 2024-09-15T21:47:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550688+0000", 2024-09-15T21:47:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4404999999999999e-05 2024-09-15T21:47:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:14 2.1a e22/19 2:594e21d0:::623.00000000:head v 22'7, mlcod=22'2)", 2024-09-15T21:47:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:47:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.620702264999998, 2024-09-15T21:47:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024358579999999999, 2024-09-15T21:47:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.244 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:47:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:47:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547990+0000", 2024-09-15T21:47:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5799999999999999e-07 2024-09-15T21:47:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548011+0000", 2024-09-15T21:47:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0942999999999999e-05 2024-09-15T21:47:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548020+0000", 2024-09-15T21:47:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2380000000000003e-06 2024-09-15T21:47:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548027+0000", 2024-09-15T21:47:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6429999999999998e-06 2024-09-15T21:47:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548848+0000", 2024-09-15T21:47:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082167999999999998 2024-09-15T21:47:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548868+0000", 2024-09-15T21:47:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9791000000000001e-05 2024-09-15T21:47:25.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550387+0000", 2024-09-15T21:47:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001519322 2024-09-15T21:47:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550425+0000", 2024-09-15T21:47:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7682999999999998e-05 2024-09-15T21:47:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.248 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:35 2.f 2:f426ad4f:::mds2_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:47:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 39.430590580000001, 2024-09-15T21:47:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017328949999999999, 2024-09-15T21:47:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:47:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:47:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738103+0000", 2024-09-15T21:47:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0020000000000001e-06 2024-09-15T21:47:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738112+0000", 2024-09-15T21:47:25.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2220000000000008e-06 2024-09-15T21:47:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738127+0000", 2024-09-15T21:47:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5016e-05 2024-09-15T21:47:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738137+0000", 2024-09-15T21:47:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0301e-05 2024-09-15T21:47:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738216+0000", 2024-09-15T21:47:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8356000000000007e-05 2024-09-15T21:47:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738340+0000", 2024-09-15T21:47:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012372800000000001 2024-09-15T21:47:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738453+0000", 2024-09-15T21:47:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000113785 2024-09-15T21:47:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739115+0000", 2024-09-15T21:47:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00066178500000000002 2024-09-15T21:47:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739745+0000", 2024-09-15T21:47:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00063017900000000003 2024-09-15T21:47:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739783+0000", 2024-09-15T21:47:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7941000000000001e-05 2024-09-15T21:47:25.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739834+0000", 2024-09-15T21:47:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.058e-05 2024-09-15T21:47:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.4.16:13 2.13 e22/19 2:c81eaac8:::62d.00000000:head v 22'8, mlcod=22'6)", 2024-09-15T21:47:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:47:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.603185521, 2024-09-15T21:47:25.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002151778, 2024-09-15T21:47:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:47:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:47:25.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565507+0000", 2024-09-15T21:47:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3340000000000001e-06 2024-09-15T21:47:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565518+0000", 2024-09-15T21:47:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0339999999999999e-05 2024-09-15T21:47:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.256 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565536+0000", 2024-09-15T21:47:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8647e-05 2024-09-15T21:47:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565545+0000", 2024-09-15T21:47:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1430000000000004e-06 2024-09-15T21:47:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565611+0000", 2024-09-15T21:47:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5129000000000006e-05 2024-09-15T21:47:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565634+0000", 2024-09-15T21:47:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3173e-05 2024-09-15T21:47:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567561+0000", 2024-09-15T21:47:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019268180000000001 2024-09-15T21:47:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567658+0000", 2024-09-15T21:47:25.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7194000000000005e-05 2024-09-15T21:47:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.d epoch 22/19 rep_tid 24 entries 22'3 (0'0) error 2:b2057508:::204.00000002:head by mds.4.16:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 22'1)", 2024-09-15T21:47:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:47:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 33.603024658999999, 2024-09-15T21:47:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026915049999999999, 2024-09-15T21:47:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:47:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:47:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565668+0000", 2024-09-15T21:47:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.113e-06 2024-09-15T21:47:25.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565684+0000", 2024-09-15T21:47:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5943999999999998e-05 2024-09-15T21:47:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565705+0000", 2024-09-15T21:47:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0761999999999999e-05 2024-09-15T21:47:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565724+0000", 2024-09-15T21:47:25.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9208e-05 2024-09-15T21:47:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565817+0000", 2024-09-15T21:47:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3218000000000001e-05 2024-09-15T21:47:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568358+0000", 2024-09-15T21:47:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00254126 2024-09-15T21:47:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.262 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.263 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.263 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:25.264 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:47:25.265 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /var/log/ceph/exp.3.22.5 2024-09-15T21:47:25.265 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:25.262+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:25.370 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:25.369+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:25.410 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:25.410 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:25.411 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:25.411 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:25.419 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:47:25.465 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:25.464+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:25.562 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:25.562 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:25.562 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:25.562 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:25.562 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:25.566 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:25.565+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:25.571 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:47:25.667 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:25.666+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:25.716 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:25.716 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.30015092, 2024-09-15T21:47:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0050344960000000003, 2024-09-15T21:47:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0199999999999998e-07 2024-09-15T21:47:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413470+0000", 2024-09-15T21:47:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4329999999999998e-06 2024-09-15T21:47:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413579+0000", 2024-09-15T21:47:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108593 2024-09-15T21:47:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413587+0000", 2024-09-15T21:47:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3100000000000001e-06 2024-09-15T21:47:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413691+0000", 2024-09-15T21:47:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010357 2024-09-15T21:47:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413887+0000", 2024-09-15T21:47:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019666799999999999 2024-09-15T21:47:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414027+0000", 2024-09-15T21:47:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013996700000000001 2024-09-15T21:47:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416360+0000", 2024-09-15T21:47:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002332204 2024-09-15T21:47:25.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418437+0000", 2024-09-15T21:47:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002077279 2024-09-15T21:47:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418469+0000", 2024-09-15T21:47:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2359e-05 2024-09-15T21:47:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418502+0000", 2024-09-15T21:47:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2811000000000001e-05 2024-09-15T21:47:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.30014688, 2024-09-15T21:47:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0053538780000000003, 2024-09-15T21:47:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.722 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:25.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2000000000000001e-07 2024-09-15T21:47:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413474+0000", 2024-09-15T21:47:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-15T21:47:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413612+0000", 2024-09-15T21:47:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000137727 2024-09-15T21:47:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413618+0000", 2024-09-15T21:47:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.241e-06 2024-09-15T21:47:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414443+0000", 2024-09-15T21:47:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082447 2024-09-15T21:47:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414523+0000", 2024-09-15T21:47:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0059e-05 2024-09-15T21:47:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414653+0000", 2024-09-15T21:47:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013031699999999999 2024-09-15T21:47:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416366+0000", 2024-09-15T21:47:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001712917 2024-09-15T21:47:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418769+0000", 2024-09-15T21:47:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024031529999999999 2024-09-15T21:47:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418802+0000", 2024-09-15T21:47:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2925000000000001e-05 2024-09-15T21:47:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418825+0000", 2024-09-15T21:47:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3204e-05 2024-09-15T21:47:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.727 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.300138859, 2024-09-15T21:47:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059708640000000002, 2024-09-15T21:47:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7300000000000002e-07 2024-09-15T21:47:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413482+0000", 2024-09-15T21:47:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2359999999999999e-06 2024-09-15T21:47:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414323+0000", 2024-09-15T21:47:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00084095399999999999 2024-09-15T21:47:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414332+0000", 2024-09-15T21:47:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0880000000000001e-06 2024-09-15T21:47:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415439+0000", 2024-09-15T21:47:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011071869999999999 2024-09-15T21:47:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415554+0000", 2024-09-15T21:47:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114499 2024-09-15T21:47:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415682+0000", 2024-09-15T21:47:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012802500000000001 2024-09-15T21:47:25.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.417957+0000", 2024-09-15T21:47:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022746720000000002 2024-09-15T21:47:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419385+0000", 2024-09-15T21:47:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001428826 2024-09-15T21:47:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419420+0000", 2024-09-15T21:47:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4875000000000001e-05 2024-09-15T21:47:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419450+0000", 2024-09-15T21:47:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0128999999999999e-05 2024-09-15T21:47:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:25.733 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.300135040999997, 2024-09-15T21:47:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0087186220000000005, 2024-09-15T21:47:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:25.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413484+0000", 2024-09-15T21:47:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-15T21:47:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413487+0000", 2024-09-15T21:47:25.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.534e-06 2024-09-15T21:47:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414353+0000", 2024-09-15T21:47:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00086616999999999998 2024-09-15T21:47:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414359+0000", 2024-09-15T21:47:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3279999999999996e-06 2024-09-15T21:47:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416041+0000", 2024-09-15T21:47:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001682467 2024-09-15T21:47:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416158+0000", 2024-09-15T21:47:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117242 2024-09-15T21:47:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416284+0000", 2024-09-15T21:47:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012509000000000001 2024-09-15T21:47:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.737 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.417992+0000", 2024-09-15T21:47:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017087059999999999 2024-09-15T21:47:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422142+0000", 2024-09-15T21:47:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041501180000000004 2024-09-15T21:47:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422176+0000", 2024-09-15T21:47:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4109999999999997e-05 2024-09-15T21:47:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422202+0000", 2024-09-15T21:47:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5585e-05 2024-09-15T21:47:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:25.739 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.300129616, 2024-09-15T21:47:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0088000370000000001, 2024-09-15T21:47:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413491+0000", 2024-09-15T21:47:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.773e-06 2024-09-15T21:47:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414372+0000", 2024-09-15T21:47:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00088122000000000005 2024-09-15T21:47:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414378+0000", 2024-09-15T21:47:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4380000000000003e-06 2024-09-15T21:47:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416631+0000", 2024-09-15T21:47:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022530319999999999 2024-09-15T21:47:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416765+0000", 2024-09-15T21:47:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013423499999999999 2024-09-15T21:47:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422289+0000", 2024-09-15T21:47:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055240530000000001 2024-09-15T21:47:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.743 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.258561297, 2024-09-15T21:47:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056900609999999997, 2024-09-15T21:47:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455058+0000", 2024-09-15T21:47:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.51e-07 2024-09-15T21:47:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455061+0000", 2024-09-15T21:47:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.145e-06 2024-09-15T21:47:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455158+0000", 2024-09-15T21:47:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7101000000000004e-05 2024-09-15T21:47:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455165+0000", 2024-09-15T21:47:25.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1019999999999999e-06 2024-09-15T21:47:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456433+0000", 2024-09-15T21:47:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012678259999999999 2024-09-15T21:47:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456517+0000", 2024-09-15T21:47:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4373e-05 2024-09-15T21:47:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:25.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456677+0000", 2024-09-15T21:47:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015963300000000001 2024-09-15T21:47:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458636+0000", 2024-09-15T21:47:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019594030000000002 2024-09-15T21:47:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460665+0000", 2024-09-15T21:47:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020283549999999999 2024-09-15T21:47:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460707+0000", 2024-09-15T21:47:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2398000000000003e-05 2024-09-15T21:47:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460747+0000", 2024-09-15T21:47:25.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0274000000000001e-05 2024-09-15T21:47:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.258554103000002, 2024-09-15T21:47:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086420030000000005, 2024-09-15T21:47:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455065+0000", 2024-09-15T21:47:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3700000000000001e-07 2024-09-15T21:47:25.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455068+0000", 2024-09-15T21:47:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.915e-06 2024-09-15T21:47:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455366+0000", 2024-09-15T21:47:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00029825400000000002 2024-09-15T21:47:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455374+0000", 2024-09-15T21:47:25.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6240000000000008e-06 2024-09-15T21:47:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456199+0000", 2024-09-15T21:47:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082448500000000002 2024-09-15T21:47:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457034+0000", 2024-09-15T21:47:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00083553399999999997 2024-09-15T21:47:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457171+0000", 2024-09-15T21:47:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013653600000000001 2024-09-15T21:47:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462663+0000", 2024-09-15T21:47:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054924420000000002 2024-09-15T21:47:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463613+0000", 2024-09-15T21:47:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094971499999999996 2024-09-15T21:47:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463663+0000", 2024-09-15T21:47:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9636999999999999e-05 2024-09-15T21:47:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463706+0000", 2024-09-15T21:47:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3523999999999999e-05 2024-09-15T21:47:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.755 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.258549575000004, 2024-09-15T21:47:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090408420000000003, 2024-09-15T21:47:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:25.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.79e-07 2024-09-15T21:47:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455072+0000", 2024-09-15T21:47:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6189999999999998e-06 2024-09-15T21:47:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455403+0000", 2024-09-15T21:47:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033167299999999999 2024-09-15T21:47:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455420+0000", 2024-09-15T21:47:25.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6518999999999999e-05 2024-09-15T21:47:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457448+0000", 2024-09-15T21:47:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020275710000000001 2024-09-15T21:47:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3844999999999994e-05 2024-09-15T21:47:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457684+0000", 2024-09-15T21:47:25.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015239 2024-09-15T21:47:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462678+0000", 2024-09-15T21:47:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049939279999999999 2024-09-15T21:47:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464017+0000", 2024-09-15T21:47:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013397140000000001 2024-09-15T21:47:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464071+0000", 2024-09-15T21:47:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3735e-05 2024-09-15T21:47:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464110+0000", 2024-09-15T21:47:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8569000000000003e-05 2024-09-15T21:47:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:25.761 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:25.762 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.258537961999998, 2024-09-15T21:47:25.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091802889999999995, 2024-09-15T21:47:25.762 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.762 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.762 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.762 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:25.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:25.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455081+0000", 2024-09-15T21:47:25.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:25.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455084+0000", 2024-09-15T21:47:25.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7810000000000001e-06 2024-09-15T21:47:25.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455536+0000", 2024-09-15T21:47:25.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00045219100000000001 2024-09-15T21:47:25.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455583+0000", 2024-09-15T21:47:25.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7308999999999997e-05 2024-09-15T21:47:25.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457998+0000", 2024-09-15T21:47:25.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024154290000000002 2024-09-15T21:47:25.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458147+0000", 2024-09-15T21:47:25.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014834599999999999 2024-09-15T21:47:25.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:25.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458265+0000", 2024-09-15T21:47:25.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118458 2024-09-15T21:47:25.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464134+0000", 2024-09-15T21:47:25.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005869038 2024-09-15T21:47:25.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464187+0000", 2024-09-15T21:47:25.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2343999999999999e-05 2024-09-15T21:47:25.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464222+0000", 2024-09-15T21:47:25.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5447000000000001e-05 2024-09-15T21:47:25.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464261+0000", 2024-09-15T21:47:25.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8659999999999999e-05 2024-09-15T21:47:25.767 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.767 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.767 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.767 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:25.768 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:25.768 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 45.237915583000003, 2024-09-15T21:47:25.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048468290000000004, 2024-09-15T21:47:25.768 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.768 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.768 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.768 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:25.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:25.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:25.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.35e-07 2024-09-15T21:47:25.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475706+0000", 2024-09-15T21:47:25.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.994e-06 2024-09-15T21:47:25.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475770+0000", 2024-09-15T21:47:25.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3904000000000003e-05 2024-09-15T21:47:25.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475775+0000", 2024-09-15T21:47:25.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2490000000000001e-06 2024-09-15T21:47:25.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476502+0000", 2024-09-15T21:47:25.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072697300000000001 2024-09-15T21:47:25.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476611+0000", 2024-09-15T21:47:25.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0001091 2024-09-15T21:47:25.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:25.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476807+0000", 2024-09-15T21:47:25.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000195892 2024-09-15T21:47:25.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479678+0000", 2024-09-15T21:47:25.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028704709999999999 2024-09-15T21:47:25.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480444+0000", 2024-09-15T21:47:25.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00076602099999999998 2024-09-15T21:47:25.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.772 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480502+0000", 2024-09-15T21:47:25.772 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8313999999999998e-05 2024-09-15T21:47:25.772 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.773 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.773 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480550+0000", 2024-09-15T21:47:25.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7676000000000002e-05 2024-09-15T21:47:25.773 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.773 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.773 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.773 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:25.773 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:25.773 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 45.237908408000003, 2024-09-15T21:47:25.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051250929999999998, 2024-09-15T21:47:25.774 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.774 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.774 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:25.774 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:25.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.774 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:25.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:25.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1e-07 2024-09-15T21:47:25.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475723+0000", 2024-09-15T21:47:25.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2758000000000001e-05 2024-09-15T21:47:25.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475972+0000", 2024-09-15T21:47:25.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024886299999999999 2024-09-15T21:47:25.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475979+0000", 2024-09-15T21:47:25.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0230000000000004e-06 2024-09-15T21:47:25.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476560+0000", 2024-09-15T21:47:25.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00058097599999999998 2024-09-15T21:47:25.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476689+0000", 2024-09-15T21:47:25.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129486 2024-09-15T21:47:25.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480835+0000", 2024-09-15T21:47:25.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041457769999999998 2024-09-15T21:47:25.777 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.777 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.777 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.777 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:25.778 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:25.778 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.168993182999998, 2024-09-15T21:47:25.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054420570000000001, 2024-09-15T21:47:25.778 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.778 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.778 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.778 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:25.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:25.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544626+0000", 2024-09-15T21:47:25.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8100000000000003e-07 2024-09-15T21:47:25.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544629+0000", 2024-09-15T21:47:25.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9869999999999999e-06 2024-09-15T21:47:25.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544775+0000", 2024-09-15T21:47:25.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014610599999999999 2024-09-15T21:47:25.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544782+0000", 2024-09-15T21:47:25.780 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4109999999999999e-06 2024-09-15T21:47:25.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.780 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.780 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545617+0000", 2024-09-15T21:47:25.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00083468300000000002 2024-09-15T21:47:25.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545690+0000", 2024-09-15T21:47:25.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3545000000000001e-05 2024-09-15T21:47:25.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.781 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.781 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:25.781 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545803+0000", 2024-09-15T21:47:25.781 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000112083 2024-09-15T21:47:25.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548353+0000", 2024-09-15T21:47:25.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025507220000000001 2024-09-15T21:47:25.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549964+0000", 2024-09-15T21:47:25.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001610382 2024-09-15T21:47:25.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.782 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.782 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550021+0000", 2024-09-15T21:47:25.782 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6898999999999999e-05 2024-09-15T21:47:25.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.783 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.783 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550067+0000", 2024-09-15T21:47:25.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758000000000002e-05 2024-09-15T21:47:25.783 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.783 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.783 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.783 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.783 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:25.783 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:25.783 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.168987905000002, 2024-09-15T21:47:25.783 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056323399999999996, 2024-09-15T21:47:25.784 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.784 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.784 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.784 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:25.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.784 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.784 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:25.784 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:25.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:47:25.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.785 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544634+0000", 2024-09-15T21:47:25.785 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.58e-06 2024-09-15T21:47:25.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.785 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.785 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544800+0000", 2024-09-15T21:47:25.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016576899999999999 2024-09-15T21:47:25.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544808+0000", 2024-09-15T21:47:25.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5179999999999998e-06 2024-09-15T21:47:25.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.786 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.786 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546075+0000", 2024-09-15T21:47:25.786 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012676969999999999 2024-09-15T21:47:25.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546179+0000", 2024-09-15T21:47:25.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000103935 2024-09-15T21:47:25.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:25.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546286+0000", 2024-09-15T21:47:25.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106758 2024-09-15T21:47:25.787 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.787 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.787 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548528+0000", 2024-09-15T21:47:25.787 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022420510000000001 2024-09-15T21:47:25.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550119+0000", 2024-09-15T21:47:25.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015907079999999999 2024-09-15T21:47:25.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.788 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.788 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.788 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550202+0000", 2024-09-15T21:47:25.788 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3374000000000005e-05 2024-09-15T21:47:25.788 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.789 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.789 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550263+0000", 2024-09-15T21:47:25.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0643999999999997e-05 2024-09-15T21:47:25.789 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.789 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.789 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.789 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.789 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.789 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:25.789 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:25.789 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.168982933999999, 2024-09-15T21:47:25.789 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061905470000000002, 2024-09-15T21:47:25.790 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.790 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.790 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.790 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:25.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.790 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.790 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.790 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:25.790 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.790 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:25.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6899999999999999e-07 2024-09-15T21:47:25.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544638+0000", 2024-09-15T21:47:25.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3920000000000001e-06 2024-09-15T21:47:25.791 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.791 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.791 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.791 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544819+0000", 2024-09-15T21:47:25.791 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018065099999999999 2024-09-15T21:47:25.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544825+0000", 2024-09-15T21:47:25.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7239999999999997e-06 2024-09-15T21:47:25.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546583+0000", 2024-09-15T21:47:25.792 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017588219999999999 2024-09-15T21:47:25.792 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.792 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.792 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.792 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546665+0000", 2024-09-15T21:47:25.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1693000000000005e-05 2024-09-15T21:47:25.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:25.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546808+0000", 2024-09-15T21:47:25.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142993 2024-09-15T21:47:25.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.793 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.793 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548533+0000", 2024-09-15T21:47:25.793 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017246 2024-09-15T21:47:25.793 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.793 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550606+0000", 2024-09-15T21:47:25.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020733969999999998 2024-09-15T21:47:25.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550783+0000", 2024-09-15T21:47:25.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000176565 2024-09-15T21:47:25.794 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.794 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.794 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.794 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550826+0000", 2024-09-15T21:47:25.794 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3441e-05 2024-09-15T21:47:25.794 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.795 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.795 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.795 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.795 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.795 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:25.795 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:25.795 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.168975213000003, 2024-09-15T21:47:25.795 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052813010000000004, 2024-09-15T21:47:25.795 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.795 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.795 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.795 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:25.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:25.796 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.796 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.796 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.796 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.796 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:25.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3799999999999999e-07 2024-09-15T21:47:25.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544647+0000", 2024-09-15T21:47:25.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6569999999999999e-06 2024-09-15T21:47:25.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.797 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544858+0000", 2024-09-15T21:47:25.797 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021069900000000001 2024-09-15T21:47:25.797 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.797 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.797 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544862+0000", 2024-09-15T21:47:25.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4900000000000002e-06 2024-09-15T21:47:25.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545610+0000", 2024-09-15T21:47:25.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00074786999999999998 2024-09-15T21:47:25.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.798 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.798 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.798 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545706+0000", 2024-09-15T21:47:25.798 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6271e-05 2024-09-15T21:47:25.798 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:25.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545834+0000", 2024-09-15T21:47:25.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012759499999999999 2024-09-15T21:47:25.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548554+0000", 2024-09-15T21:47:25.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027199339999999998 2024-09-15T21:47:25.799 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.799 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.799 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.799 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549818+0000", 2024-09-15T21:47:25.799 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012642199999999999 2024-09-15T21:47:25.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549884+0000", 2024-09-15T21:47:25.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5843e-05 2024-09-15T21:47:25.800 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.800 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.800 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.800 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549925+0000", 2024-09-15T21:47:25.800 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0484000000000001e-05 2024-09-15T21:47:25.800 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.800 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.801 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.801 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.801 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:25.801 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:25.801 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 40.168966535999999, 2024-09-15T21:47:25.801 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051106110000000001, 2024-09-15T21:47:25.801 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.801 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.801 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.801 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.801 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.801 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.801 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:25.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:25.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.802 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:25.802 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2700000000000001e-07 2024-09-15T21:47:25.802 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.802 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.802 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544667+0000", 2024-09-15T21:47:25.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4746000000000001e-05 2024-09-15T21:47:25.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545021+0000", 2024-09-15T21:47:25.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035412900000000001 2024-09-15T21:47:25.803 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.803 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.803 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.803 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545052+0000", 2024-09-15T21:47:25.803 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1158000000000002e-05 2024-09-15T21:47:25.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546050+0000", 2024-09-15T21:47:25.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00099782800000000008 2024-09-15T21:47:25.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.804 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546291+0000", 2024-09-15T21:47:25.804 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024093500000000001 2024-09-15T21:47:25.804 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.804 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.804 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:25.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546434+0000", 2024-09-15T21:47:25.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014308600000000001 2024-09-15T21:47:25.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548583+0000", 2024-09-15T21:47:25.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021491029999999999 2024-09-15T21:47:25.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.805 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.805 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.805 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549695+0000", 2024-09-15T21:47:25.805 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011116419999999999 2024-09-15T21:47:25.805 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549726+0000", 2024-09-15T21:47:25.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1195000000000002e-05 2024-09-15T21:47:25.806 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.806 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.806 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.806 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549763+0000", 2024-09-15T21:47:25.806 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6562e-05 2024-09-15T21:47:25.806 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.806 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.806 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:17 2.14 2:2eebc8f4:::62f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.148536923999998, 2024-09-15T21:47:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071004800000000002, 2024-09-15T21:47:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:25.807 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565083+0000", 2024-09-15T21:47:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.689e-06 2024-09-15T21:47:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.808 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565090+0000", 2024-09-15T21:47:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5930000000000002e-06 2024-09-15T21:47:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565119+0000", 2024-09-15T21:47:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8785999999999998e-05 2024-09-15T21:47:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565126+0000", 2024-09-15T21:47:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0110000000000003e-06 2024-09-15T21:47:25.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565237+0000", 2024-09-15T21:47:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011161 2024-09-15T21:47:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565383+0000", 2024-09-15T21:47:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014626900000000001 2024-09-15T21:47:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:25.810 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565520+0000", 2024-09-15T21:47:25.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013676399999999999 2024-09-15T21:47:25.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569909+0000", 2024-09-15T21:47:25.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043890149999999996 2024-09-15T21:47:25.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.811 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572096+0000", 2024-09-15T21:47:25.811 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021866559999999999 2024-09-15T21:47:25.811 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.811 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.811 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572144+0000", 2024-09-15T21:47:25.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7725000000000001e-05 2024-09-15T21:47:25.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.812 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.812 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572182+0000", 2024-09-15T21:47:25.812 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8362000000000003e-05 2024-09-15T21:47:25.812 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.812 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.812 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.812 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.812 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.813 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:25.813 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:25.813 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.148525825999997, 2024-09-15T21:47:25.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072110280000000004, 2024-09-15T21:47:25.813 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.813 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.813 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.813 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.813 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.813 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:25.813 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.813 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.813 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:25.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:25.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4299999999999999e-07 2024-09-15T21:47:25.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.814 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.814 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.814 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565096+0000", 2024-09-15T21:47:25.814 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4319999999999999e-06 2024-09-15T21:47:25.814 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565152+0000", 2024-09-15T21:47:25.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5686999999999997e-05 2024-09-15T21:47:25.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565157+0000", 2024-09-15T21:47:25.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2700000000000004e-06 2024-09-15T21:47:25.815 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.815 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.815 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565847+0000", 2024-09-15T21:47:25.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068937299999999996 2024-09-15T21:47:25.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565940+0000", 2024-09-15T21:47:25.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3573999999999996e-05 2024-09-15T21:47:25.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:25.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566085+0000", 2024-09-15T21:47:25.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000144902 2024-09-15T21:47:25.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570595+0000", 2024-09-15T21:47:25.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045103469999999996 2024-09-15T21:47:25.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572224+0000", 2024-09-15T21:47:25.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016282930000000001 2024-09-15T21:47:25.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572260+0000", 2024-09-15T21:47:25.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6103999999999999e-05 2024-09-15T21:47:25.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572304+0000", 2024-09-15T21:47:25.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3803000000000001e-05 2024-09-15T21:47:25.818 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.818 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.818 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.818 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:25.818 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:25.818 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.148521361, 2024-09-15T21:47:25.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055995389999999997, 2024-09-15T21:47:25.819 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.819 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.819 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.819 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:25.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:25.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:25.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9999999999999999e-07 2024-09-15T21:47:25.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:25.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2399999999999999e-06 2024-09-15T21:47:25.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565280+0000", 2024-09-15T21:47:25.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017929400000000001 2024-09-15T21:47:25.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565292+0000", 2024-09-15T21:47:25.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2193e-05 2024-09-15T21:47:25.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565357+0000", 2024-09-15T21:47:25.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4684000000000006e-05 2024-09-15T21:47:25.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565499+0000", 2024-09-15T21:47:25.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142032 2024-09-15T21:47:25.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:25.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565634+0000", 2024-09-15T21:47:25.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000135613 2024-09-15T21:47:25.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569941+0000", 2024-09-15T21:47:25.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043066789999999999 2024-09-15T21:47:25.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570585+0000", 2024-09-15T21:47:25.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00064364700000000001 2024-09-15T21:47:25.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570647+0000", 2024-09-15T21:47:25.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2173000000000003e-05 2024-09-15T21:47:25.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570697+0000", 2024-09-15T21:47:25.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9784000000000003e-05 2024-09-15T21:47:25.824 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.824 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.824 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.824 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:25.824 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:25.824 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 34.148517130999998, 2024-09-15T21:47:25.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066548939999999997, 2024-09-15T21:47:25.825 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:25.825 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:25.825 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:25.825 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:25.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:25.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:25.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:25.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:25.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:25.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:25.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565102+0000", 2024-09-15T21:47:25.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-15T21:47:25.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:25.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565103+0000", 2024-09-15T21:47:25.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6700000000000001e-06 2024-09-15T21:47:25.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:25.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565608+0000", 2024-09-15T21:47:25.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050518299999999998 2024-09-15T21:47:25.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:25.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565616+0000", 2024-09-15T21:47:25.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8199999999999997e-06 2024-09-15T21:47:25.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:25.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566598+0000", 2024-09-15T21:47:25.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00098176300000000008 2024-09-15T21:47:25.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:25.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566871+0000", 2024-09-15T21:47:25.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027312799999999998 2024-09-15T21:47:25.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:25.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567015+0000", 2024-09-15T21:47:25.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014428499999999999 2024-09-15T21:47:25.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:25.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570729+0000", 2024-09-15T21:47:25.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037132459999999999 2024-09-15T21:47:25.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:25.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571667+0000", 2024-09-15T21:47:25.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00093862299999999997 2024-09-15T21:47:25.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:25.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571718+0000", 2024-09-15T21:47:25.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1146999999999997e-05 2024-09-15T21:47:25.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:25.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:25.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:25.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571756+0000", 2024-09-15T21:47:25.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7821000000000003e-05 2024-09-15T21:47:25.830 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.830 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.830 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.830 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:25.830 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:25.830 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:25.831 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:47:25.832 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:25.830+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:25.833 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:47:25.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.927+0000 7f4b9355f640 1 Processor -- start 2024-09-15T21:47:25.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.928+0000 7f4b9355f640 1 -- start start 2024-09-15T21:47:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.928+0000 7f4b9355f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f4b8c06c6e0 0x7f4b8c0c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.928+0000 7f4b9355f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b8c06cac0 0x7f4b8c158100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.928+0000 7f4b9355f640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b8c0c37c0 con 0x7f4b8c0c9ea0 2024-09-15T21:47:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.928+0000 7f4b9355f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f4b8c072920 con 0x7f4b8c06cac0 2024-09-15T21:47:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.928+0000 7f4b9355f640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f4b8c158640 con 0x7f4b8c06c6e0 2024-09-15T21:47:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.929+0000 7f4b91afc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b8c06cac0 0x7f4b8c158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.929+0000 7f4b91afc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4b8c06cac0 0x7f4b8c158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51734/0 (socket says 172.21.15.100:51734) 2024-09-15T21:47:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.929+0000 7f4b91afc640 1 -- 172.21.15.100:0/4230407510 learned_addr learned my addr 172.21.15.100:0/4230407510 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.929+0000 7f4b912fb640 1 --2- 172.21.15.100:0/4230407510 >> v2:172.21.15.122:3300/0 conn(0x7f4b8c06c6e0 0x7f4b8c0c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:25.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.929+0000 7f4b91afc640 1 -- 172.21.15.100:0/4230407510 >> v2:172.21.15.122:3300/0 conn(0x7f4b8c06c6e0 msgr2=0x7f4b8c0c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:25.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.929+0000 7f4b91afc640 1 --2- 172.21.15.100:0/4230407510 >> v2:172.21.15.122:3300/0 conn(0x7f4b8c06c6e0 0x7f4b8c0c31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:25.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.929+0000 7f4b91afc640 1 -- 172.21.15.100:0/4230407510 >> v1:172.21.15.122:6789/0 conn(0x7f4b8c0c9ea0 legacy=0x7f4b8c072210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:25.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.929+0000 7f4b91afc640 1 -- 172.21.15.100:0/4230407510 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8c071130 con 0x7f4b8c06cac0 2024-09-15T21:47:25.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.930+0000 7f4b91afc640 1 --2- 172.21.15.100:0/4230407510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b8c06cac0 0x7f4b8c158100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c071300 tx=0x7f4b8402b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a786a732547da37b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:25.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.930+0000 7f4b83fff640 1 -- 172.21.15.100:0/4230407510 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84008500 con 0x7f4b8c06cac0 2024-09-15T21:47:25.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.930+0000 7f4b83fff640 1 -- 172.21.15.100:0/4230407510 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b840086a0 con 0x7f4b8c06cac0 2024-09-15T21:47:25.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.930+0000 7f4b83fff640 1 -- 172.21.15.100:0/4230407510 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b84008a70 con 0x7f4b8c06cac0 2024-09-15T21:47:25.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.931+0000 7f4b9355f640 1 -- 172.21.15.100:0/4230407510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b8c06cac0 msgr2=0x7f4b8c158100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:25.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.931+0000 7f4b9355f640 1 --2- 172.21.15.100:0/4230407510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b8c06cac0 0x7f4b8c158100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c071300 tx=0x7f4b8402b8c0 comp rx=0 tx=0).stop 2024-09-15T21:47:25.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.931+0000 7f4b9355f640 1 -- 172.21.15.100:0/4230407510 shutdown_connections 2024-09-15T21:47:25.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.931+0000 7f4b9355f640 1 --2- 172.21.15.100:0/4230407510 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b8c06cac0 0x7f4b8c158100 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:25.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.931+0000 7f4b9355f640 1 --2- 172.21.15.100:0/4230407510 >> v2:172.21.15.122:3300/0 conn(0x7f4b8c06c6e0 0x7f4b8c0c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:25.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.931+0000 7f4b9355f640 1 -- 172.21.15.100:0/4230407510 >> 172.21.15.100:0/4230407510 conn(0x7f4b8c0c9380 msgr2=0x7f4b8c0c9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:25.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.931+0000 7f4b9355f640 1 -- 172.21.15.100:0/4230407510 shutdown_connections 2024-09-15T21:47:25.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.931+0000 7f4b9355f640 1 -- 172.21.15.100:0/4230407510 wait complete. 2024-09-15T21:47:25.935 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:25.931+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:25.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.933+0000 7f4b9355f640 1 Processor -- start 2024-09-15T21:47:25.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.933+0000 7f4b9355f640 1 -- start start 2024-09-15T21:47:25.936 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:25.936 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:25.936 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:25.936 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:25.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.934+0000 7f4b9355f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b8c06c6e0 0x7f4b8c0c2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:25.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.934+0000 7f4b9355f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b8c06cac0 0x7f4b8c1f06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:25.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.934+0000 7f4b9355f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4b8c0c9ea0 0x7f4b8c1f0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:25.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.934+0000 7f4b9355f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c071690 con 0x7f4b8c06c6e0 2024-09-15T21:47:25.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.934+0000 7f4b9355f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c0713d0 con 0x7f4b8c06cac0 2024-09-15T21:47:25.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.934+0000 7f4b9355f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4b8c071e90 con 0x7f4b8c0c9ea0 2024-09-15T21:47:25.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.934+0000 7f4b912fb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b8c06c6e0 0x7f4b8c0c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:25.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.934+0000 7f4b912fb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b8c06c6e0 0x7f4b8c0c2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:51736/0 (socket says 172.21.15.100:51736) 2024-09-15T21:47:25.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.934+0000 7f4b912fb640 1 -- 172.21.15.100:0/2835969048 learned_addr learned my addr 172.21.15.100:0/2835969048 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:25.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.934+0000 7f4b90afa640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b8c06cac0 0x7f4b8c1f06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:25.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.934+0000 7f4b91afc640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4b8c0c9ea0 0x7f4b8c1f0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:25.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.934+0000 7f4b912fb640 1 -- 172.21.15.100:0/2835969048 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4b8c0c9ea0 msgr2=0x7f4b8c1f0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:25.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.934+0000 7f4b912fb640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4b8c0c9ea0 0x7f4b8c1f0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:25.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.934+0000 7f4b912fb640 1 -- 172.21.15.100:0/2835969048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b8c06cac0 msgr2=0x7f4b8c1f06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:25.939 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.934+0000 7f4b912fb640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b8c06cac0 0x7f4b8c1f06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:25.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.934+0000 7f4b912fb640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8c1f2da0 con 0x7f4b8c06c6e0 2024-09-15T21:47:25.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.934+0000 7f4b91afc640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4b8c0c9ea0 0x7f4b8c1f0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:47:25.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.934+0000 7f4b912fb640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4b8c06c6e0 0x7f4b8c0c2500 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f4b8800e870 tx=0x7f4b8800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:25.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.935+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88017070 con 0x7f4b8c06c6e0 2024-09-15T21:47:25.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.935+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b8800c8e0 con 0x7f4b8c06c6e0 2024-09-15T21:47:25.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.935+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8800ccd0 con 0x7f4b8c06c6e0 2024-09-15T21:47:25.940 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.935+0000 7f4b9355f640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8c0ba1b0 con 0x7f4b8c06c6e0 2024-09-15T21:47:25.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.935+0000 7f4b9355f640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b8c1efeb0 con 0x7f4b8c06c6e0 2024-09-15T21:47:25.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.936+0000 7f4b9355f640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f4b8c069420 con 0x7f4b8c06c6e0 2024-09-15T21:47:25.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.936+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4b8801a070 con 0x7f4b8c06c6e0 2024-09-15T21:47:25.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.937+0000 7f4b81ffb640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4b7006c2b0 0x7f4b7006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:25.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.937+0000 7f4b90afa640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4b7006c2b0 0x7f4b7006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:25.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.937+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5854+0+0 (secure 0 0 0) 0x7f4b8808b270 con 0x7f4b8c06c6e0 2024-09-15T21:47:25.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.937+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v34) ==== 54+0+16 (secure 0 0 0) 0x7f4b8808d8f0 con 0x7f4b8c06c6e0 2024-09-15T21:47:25.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.938+0000 7f4b9355f640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f4b8c1f00f0 0x7f4b8c150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:25.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.938+0000 7f4b90afa640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4b7006c2b0 0x7f4b7006e770 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c157620 tx=0x7f4b7c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:25.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.938+0000 7f4b9355f640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4b8c0c4f90 con 0x7f4b8c1f00f0 2024-09-15T21:47:25.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.938+0000 7f4b91afc640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f4b8c1f00f0 0x7f4b8c150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:25.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.938+0000 7f4b91afc640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f4b8c1f00f0 0x7f4b8c150e40 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:25.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.939+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4b8c0c4f90 con 0x7f4b8c1f00f0 2024-09-15T21:47:25.944 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:47:25.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.953+0000 7f4b9355f640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4b8c0c4cb0 con 0x7f4b8c1f00f0 2024-09-15T21:47:25.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.956+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4b8c0c4cb0 con 0x7f4b8c1f00f0 2024-09-15T21:47:25.957 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:25.958 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:25.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.956+0000 7f4b9355f640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f4b8c1eee50 0x7f4b8c1ef250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:25.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.956+0000 7f4b9355f640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f4b8c1ef790 con 0x7f4b8c1eee50 2024-09-15T21:47:25.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.956+0000 7f4b912fb640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f4b8c1eee50 0x7f4b8c1ef250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:25.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.956+0000 7f4b912fb640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f4b8c1eee50 0x7f4b8c1ef250 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:25.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.957+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4b8c1ef790 con 0x7f4b8c1eee50 2024-09-15T21:47:25.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.972+0000 7f4b9355f640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4b8c0cc470 con 0x7f4b8c1eee50 2024-09-15T21:47:25.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.974+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4b8c0cc470 con 0x7f4b8c1eee50 2024-09-15T21:47:25.976 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:25.978 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:25.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.974+0000 7f4b9355f640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f4b8c0bafc0 0x7f4b8c062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:25.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.974+0000 7f4b9355f640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f4b8c0cc620 con 0x7f4b8c0bafc0 2024-09-15T21:47:25.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.974+0000 7f4b90afa640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f4b8c0bafc0 0x7f4b8c062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:25.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.975+0000 7f4b90afa640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f4b8c0bafc0 0x7f4b8c062630 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:25.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.975+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4b8c0cc620 con 0x7f4b8c0bafc0 2024-09-15T21:47:25.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.989+0000 7f4b9355f640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4b8c0bb5f0 con 0x7f4b8c0bafc0 2024-09-15T21:47:25.992 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.991+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4b8c0bb5f0 con 0x7f4b8c0bafc0 2024-09-15T21:47:25.993 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:25.993 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:25.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.991+0000 7f4b9355f640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f4b8c0c7420 0x7f4b8c0c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:25.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.991+0000 7f4b9355f640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f4b8c063440 con 0x7f4b8c0c7420 2024-09-15T21:47:25.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.991+0000 7f4b91afc640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f4b8c0c7420 0x7f4b8c0c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:25.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.991+0000 7f4b91afc640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f4b8c0c7420 0x7f4b8c0c7860 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:25.994 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:25.992+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4b8c063440 con 0x7f4b8c0c7420 2024-09-15T21:47:26.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.006+0000 7f4b9355f640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4b8c0c7ff0 con 0x7f4b8c0c7420 2024-09-15T21:47:26.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.009+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4b8c0c7ff0 con 0x7f4b8c0c7420 2024-09-15T21:47:26.010 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:26.011 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:26.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.009+0000 7f4b9355f640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f4b8c0c8160 0x7f4b8c02cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:26.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.009+0000 7f4b9355f640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f4b8c02d090 con 0x7f4b8c0c8160 2024-09-15T21:47:26.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.009+0000 7f4b912fb640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f4b8c0c8160 0x7f4b8c02cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:26.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.009+0000 7f4b912fb640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f4b8c0c8160 0x7f4b8c02cb50 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:26.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.010+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4b8c02d090 con 0x7f4b8c0c8160 2024-09-15T21:47:26.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.024+0000 7f4b9355f640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4b8c02d090 con 0x7f4b8c0c8160 2024-09-15T21:47:26.027 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.025+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4b8c02d090 con 0x7f4b8c0c8160 2024-09-15T21:47:26.027 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:26.028 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:26.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.026+0000 7f4b9355f640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=35}) -- 0x7f4b8c02d090 con 0x7f4b8c06c6e0 2024-09-15T21:47:26.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.026+0000 7f4b9355f640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4b8c02d3b0 con 0x7f4b8c06c6e0 2024-09-15T21:47:26.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.026+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4b88092070 con 0x7f4b8c06c6e0 2024-09-15T21:47:26.029 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:47:26.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.026+0000 7f4b9355f640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f4b8c203800 0x7f4b8c205bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:26.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.026+0000 7f4b9355f640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f4b8c02da40 con 0x7f4b8c203800 2024-09-15T21:47:26.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.026+0000 7f4b90afa640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f4b8c203800 0x7f4b8c205bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:26.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.027+0000 7f4b90afa640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f4b8c203800 0x7f4b8c205bf0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:26.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.027+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4b8c02da40 con 0x7f4b8c203800 2024-09-15T21:47:26.033 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:26.031+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:26.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.041+0000 7f4b9355f640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4b8c02da40 con 0x7f4b8c203800 2024-09-15T21:47:26.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.043+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4b8c02da40 con 0x7f4b8c203800 2024-09-15T21:47:26.045 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:26.046 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:26.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.043+0000 7f4b9355f640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f4b8c206640 0x7f4b8c208a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:26.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.043+0000 7f4b9355f640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f4b8c20b8d0 con 0x7f4b8c206640 2024-09-15T21:47:26.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.043+0000 7f4b91afc640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f4b8c206640 0x7f4b8c208a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:26.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.044+0000 7f4b91afc640 1 --2- 172.21.15.100:0/2835969048 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f4b8c206640 0x7f4b8c208a30 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:26.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.044+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4b8c20b8d0 con 0x7f4b8c206640 2024-09-15T21:47:26.060 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.059+0000 7f4b9355f640 1 -- 172.21.15.100:0/2835969048 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f4b8c02d6c0 con 0x7f4b8c206640 2024-09-15T21:47:26.062 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.060+0000 7f4b81ffb640 1 -- 172.21.15.100:0/2835969048 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4b8c02d6c0 con 0x7f4b8c206640 2024-09-15T21:47:26.062 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:26.062 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:26.066 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:47:26.066 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:47:26.089 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:26.089 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:26.097 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:47:26.134 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:26.132+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:26.235 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:26.234+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:26.241 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:26.241 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:26.250 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:47:26.337 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:26.335+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:26.359 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:26.359 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:26.368 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:47:26.452 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:26.451+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:26.516 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:26.516 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:26.525 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:47:26.538 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:26.537+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:26.638 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:26.637+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:26.671 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:26.671 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:26.680 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:47:26.739 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:26.738+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:26.763 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:47:26.790 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:26.790 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:26.800 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:47:26.840 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:26.839+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:26.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.867+0000 7f685f28c640 1 Processor -- start 2024-09-15T21:47:26.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.867+0000 7f685f28c640 1 -- start start 2024-09-15T21:47:26.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.868+0000 7f685f28c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6858155440 0x7f68580cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:26.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.868+0000 7f685f28c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f68580cb7d0 0x7f68580cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:26.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.868+0000 7f685f28c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f685806d0e0 0x7f68580cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:26.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.868+0000 7f685f28c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f685815b470 con 0x7f68580cb7d0 2024-09-15T21:47:26.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.868+0000 7f685f28c640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f6858155820 con 0x7f6858155440 2024-09-15T21:47:26.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.868+0000 7f685f28c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f685815e7a0 con 0x7f685806d0e0 2024-09-15T21:47:26.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.868+0000 7f685c827640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f68580cb7d0 0x7f68580cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:26.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.868+0000 7f685c827640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f68580cb7d0 0x7f68580cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36540/0 (socket says 172.21.15.100:36540) 2024-09-15T21:47:26.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.868+0000 7f685c827640 1 -- 172.21.15.100:0/1062525997 learned_addr learned my addr 172.21.15.100:0/1062525997 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:26.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.869+0000 7f685d028640 1 --2- 172.21.15.100:0/1062525997 >> v2:172.21.15.122:3300/0 conn(0x7f6858155440 0x7f68580cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:26.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.869+0000 7f685d829640 1 --2- 172.21.15.100:0/1062525997 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f685806d0e0 0x7f68580cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:26.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.869+0000 7f685c827640 1 -- 172.21.15.100:0/1062525997 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f685806d0e0 msgr2=0x7f68580cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:26.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.869+0000 7f685c827640 1 --2- 172.21.15.100:0/1062525997 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f685806d0e0 0x7f68580cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:26.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.869+0000 7f685c827640 1 -- 172.21.15.100:0/1062525997 >> v2:172.21.15.122:3300/0 conn(0x7f6858155440 msgr2=0x7f68580cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:26.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.869+0000 7f685c827640 1 --2- 172.21.15.100:0/1062525997 >> v2:172.21.15.122:3300/0 conn(0x7f6858155440 0x7f68580cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:26.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.869+0000 7f685c827640 1 -- 172.21.15.100:0/1062525997 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685815ea40 con 0x7f68580cb7d0 2024-09-15T21:47:26.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.869+0000 7f685d829640 1 --2- 172.21.15.100:0/1062525997 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f685806d0e0 0x7f68580cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:47:26.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.870+0000 7f685c827640 1 --2- 172.21.15.100:0/1062525997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68580cb7d0 0x7f68580cd960 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f68580ce040 tx=0x7f684802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b414d9aeb96e3f7f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:26.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.870+0000 7f684ffff640 1 -- 172.21.15.100:0/1062525997 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848037070 con 0x7f68580cb7d0 2024-09-15T21:47:26.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.870+0000 7f684ffff640 1 -- 172.21.15.100:0/1062525997 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6848032440 con 0x7f68580cb7d0 2024-09-15T21:47:26.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.870+0000 7f684ffff640 1 -- 172.21.15.100:0/1062525997 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848032810 con 0x7f68580cb7d0 2024-09-15T21:47:26.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.871+0000 7f685f28c640 1 -- 172.21.15.100:0/1062525997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68580cb7d0 msgr2=0x7f68580cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:26.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.871+0000 7f685f28c640 1 --2- 172.21.15.100:0/1062525997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68580cb7d0 0x7f68580cd960 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f68580ce040 tx=0x7f684802a210 comp rx=0 tx=0).stop 2024-09-15T21:47:26.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.871+0000 7f685f28c640 1 -- 172.21.15.100:0/1062525997 shutdown_connections 2024-09-15T21:47:26.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.871+0000 7f685f28c640 1 --2- 172.21.15.100:0/1062525997 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f685806d0e0 0x7f68580cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:26.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.871+0000 7f685f28c640 1 --2- 172.21.15.100:0/1062525997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68580cb7d0 0x7f68580cd960 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:26.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.871+0000 7f685f28c640 1 --2- 172.21.15.100:0/1062525997 >> v2:172.21.15.122:3300/0 conn(0x7f6858155440 0x7f68580cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:26.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.871+0000 7f685f28c640 1 -- 172.21.15.100:0/1062525997 >> 172.21.15.100:0/1062525997 conn(0x7f68580680d0 msgr2=0x7f685806d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:26.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.871+0000 7f685f28c640 1 -- 172.21.15.100:0/1062525997 shutdown_connections 2024-09-15T21:47:26.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.871+0000 7f685f28c640 1 -- 172.21.15.100:0/1062525997 wait complete. 2024-09-15T21:47:26.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.873+0000 7f685f28c640 1 Processor -- start 2024-09-15T21:47:26.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.873+0000 7f685f28c640 1 -- start start 2024-09-15T21:47:26.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.874+0000 7f685f28c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f685806d0e0 0x7f6858150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.874+0000 7f685f28c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68580cb7d0 0x7f6858153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.874+0000 7f685f28c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6858155440 0x7f6858153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.874+0000 7f685f28c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f685815f2e0 con 0x7f68580cb7d0 2024-09-15T21:47:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.874+0000 7f685f28c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f685815f000 con 0x7f6858155440 2024-09-15T21:47:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.874+0000 7f685f28c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f685815b070 con 0x7f685806d0e0 2024-09-15T21:47:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.875+0000 7f685c827640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68580cb7d0 0x7f6858153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.875+0000 7f685c827640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68580cb7d0 0x7f6858153320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36550/0 (socket says 172.21.15.100:36550) 2024-09-15T21:47:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.875+0000 7f685c827640 1 -- 172.21.15.100:0/4278056653 learned_addr learned my addr 172.21.15.100:0/4278056653 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.875+0000 7f685d829640 1 --2- 172.21.15.100:0/4278056653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6858155440 0x7f6858153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.875+0000 7f685d028640 1 --2- 172.21.15.100:0/4278056653 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f685806d0e0 0x7f6858150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.875+0000 7f685c827640 1 -- 172.21.15.100:0/4278056653 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f685806d0e0 msgr2=0x7f6858150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:26.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.875+0000 7f685c827640 1 --2- 172.21.15.100:0/4278056653 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f685806d0e0 0x7f6858150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:26.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.875+0000 7f685c827640 1 -- 172.21.15.100:0/4278056653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6858155440 msgr2=0x7f6858153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:26.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.875+0000 7f685c827640 1 --2- 172.21.15.100:0/4278056653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6858155440 0x7f6858153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:26.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.875+0000 7f685c827640 1 -- 172.21.15.100:0/4278056653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6858154890 con 0x7f68580cb7d0 2024-09-15T21:47:26.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.875+0000 7f685c827640 1 --2- 172.21.15.100:0/4278056653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68580cb7d0 0x7f6858153320 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6848000c00 tx=0x7f68480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:26.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.875+0000 7f684dffb640 1 -- 172.21.15.100:0/4278056653 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848037040 con 0x7f68580cb7d0 2024-09-15T21:47:26.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.875+0000 7f684dffb640 1 -- 172.21.15.100:0/4278056653 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6848042070 con 0x7f68580cb7d0 2024-09-15T21:47:26.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.875+0000 7f685f28c640 1 -- 172.21.15.100:0/4278056653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68581f20f0 con 0x7f68580cb7d0 2024-09-15T21:47:26.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.875+0000 7f684dffb640 1 -- 172.21.15.100:0/4278056653 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848030880 con 0x7f68580cb7d0 2024-09-15T21:47:26.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.876+0000 7f685f28c640 1 -- 172.21.15.100:0/4278056653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68581f23c0 con 0x7f68580cb7d0 2024-09-15T21:47:26.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.877+0000 7f685f28c640 1 -- 172.21.15.100:0/4278056653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f685806e780 con 0x7f68580cb7d0 2024-09-15T21:47:26.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.878+0000 7f684dffb640 1 -- 172.21.15.100:0/4278056653 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6848030a20 con 0x7f68580cb7d0 2024-09-15T21:47:26.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.878+0000 7f684dffb640 1 --2- 172.21.15.100:0/4278056653 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f683c06c2b0 0x7f683c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:26.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.878+0000 7f684dffb640 1 -- 172.21.15.100:0/4278056653 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5854+0+0 (secure 0 0 0) 0x7f68480abbe0 con 0x7f68580cb7d0 2024-09-15T21:47:26.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.878+0000 7f685d028640 1 --2- 172.21.15.100:0/4278056653 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f683c06c2b0 0x7f683c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:26.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.879+0000 7f685d028640 1 --2- 172.21.15.100:0/4278056653 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f683c06c2b0 0x7f683c06e770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6854009ac0 tx=0x7f6854009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:26.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:26.883+0000 7f684dffb640 1 -- 172.21.15.100:0/4278056653 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6848079c70 con 0x7f68580cb7d0 2024-09-15T21:47:26.941 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:26.940+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:26.950 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:26.950 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:26.959 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:47:27.044 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:27.042+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:27.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:27.073+0000 7f685f28c640 1 -- 172.21.15.100:0/4278056653 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f68580b3940 con 0x7f68580cb7d0 2024-09-15T21:47:27.115 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:27.115 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:27.124 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:47:27.143 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:27.142+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:27.243 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:27.242+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:27.273 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:27.273 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:27.283 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:47:27.366 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:27.365+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:27.434 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:27.434 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:27.444 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:47:27.446 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:27.444+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:27.548 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:27.547+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:27.595 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:27.596 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:27.605 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:47:27.649 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:27.647+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:27.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:27.747+0000 7f684dffb640 1 -- 172.21.15.100:0/4278056653 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v35) ==== 87+0+0 (secure 0 0 0) 0x7f684807df70 con 0x7f68580cb7d0 2024-09-15T21:47:27.749 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is set 2024-09-15T21:47:27.752 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:27.747+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:27.750+0000 7f685f28c640 1 -- 172.21.15.100:0/4278056653 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f683c06c2b0 msgr2=0x7f683c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:27.750+0000 7f685f28c640 1 --2- 172.21.15.100:0/4278056653 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f683c06c2b0 0x7f683c06e770 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6854009ac0 tx=0x7f6854009190 comp rx=0 tx=0).stop 2024-09-15T21:47:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:27.750+0000 7f685f28c640 1 -- 172.21.15.100:0/4278056653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68580cb7d0 msgr2=0x7f6858153320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:27.750+0000 7f685f28c640 1 --2- 172.21.15.100:0/4278056653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68580cb7d0 0x7f6858153320 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6848000c00 tx=0x7f68480046c0 comp rx=0 tx=0).stop 2024-09-15T21:47:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:27.750+0000 7f685f28c640 1 -- 172.21.15.100:0/4278056653 shutdown_connections 2024-09-15T21:47:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:27.750+0000 7f685f28c640 1 --2- 172.21.15.100:0/4278056653 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f683c06c2b0 0x7f683c06e770 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:27.750+0000 7f685f28c640 1 --2- 172.21.15.100:0/4278056653 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6858155440 0x7f6858153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:27.750+0000 7f685f28c640 1 --2- 172.21.15.100:0/4278056653 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f68580cb7d0 0x7f6858153320 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:27.750+0000 7f685f28c640 1 --2- 172.21.15.100:0/4278056653 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f685806d0e0 0x7f6858150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:27.750+0000 7f685f28c640 1 -- 172.21.15.100:0/4278056653 >> 172.21.15.100:0/4278056653 conn(0x7f68580680d0 msgr2=0x7f68580cd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:27.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:27.750+0000 7f685f28c640 1 -- 172.21.15.100:0/4278056653 shutdown_connections 2024-09-15T21:47:27.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:27.750+0000 7f685f28c640 1 -- 172.21.15.100:0/4278056653 wait complete. 2024-09-15T21:47:27.757 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:27.757 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:27.768 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:47:27.851 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:27.850+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:27.917 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:27.917 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:27.926 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:47:27.967 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:27.965+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:28.051 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:28.050+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:28.068 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:47:28.071 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:28.071 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:28.082 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:47:28.153 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:28.151+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:28.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.164+0000 7f57d0ae4640 1 Processor -- start 2024-09-15T21:47:28.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.164+0000 7f57d0ae4640 1 -- start start 2024-09-15T21:47:28.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.165+0000 7f57d0ae4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f57cc155290 0x7f57cc15e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:28.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.165+0000 7f57d0ae4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f57cc155670 0x7f57cc15bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:28.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.165+0000 7f57d0ae4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f57cc0c65b0 0x7f57cc0c69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:28.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.165+0000 7f57d0ae4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f57cc15c0a0 con 0x7f57cc155670 2024-09-15T21:47:28.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.165+0000 7f57d0ae4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f57cc15edb0 con 0x7f57cc0c65b0 2024-09-15T21:47:28.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.165+0000 7f57d0ae4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f57cc15f090 con 0x7f57cc155290 2024-09-15T21:47:28.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.165+0000 7f57c9d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f57cc155670 0x7f57cc15bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:28.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.165+0000 7f57c9d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f57cc155670 0x7f57cc15bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36562/0 (socket says 172.21.15.100:36562) 2024-09-15T21:47:28.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.165+0000 7f57c9d9b640 1 -- 172.21.15.100:0/1215301166 learned_addr learned my addr 172.21.15.100:0/1215301166 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:28.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.165+0000 7f57ca59c640 1 --2- 172.21.15.100:0/1215301166 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f57cc155290 0x7f57cc15e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:28.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.165+0000 7f57cad9d640 1 --2- 172.21.15.100:0/1215301166 >> v2:172.21.15.122:3300/0 conn(0x7f57cc0c65b0 0x7f57cc0c69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:28.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.165+0000 7f57c9d9b640 1 -- 172.21.15.100:0/1215301166 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f57cc155290 msgr2=0x7f57cc15e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:28.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.165+0000 7f57c9d9b640 1 --2- 172.21.15.100:0/1215301166 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f57cc155290 0x7f57cc15e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:28.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.165+0000 7f57c9d9b640 1 -- 172.21.15.100:0/1215301166 >> v2:172.21.15.122:3300/0 conn(0x7f57cc0c65b0 msgr2=0x7f57cc0c69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:28.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.165+0000 7f57c9d9b640 1 --2- 172.21.15.100:0/1215301166 >> v2:172.21.15.122:3300/0 conn(0x7f57cc0c65b0 0x7f57cc0c69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:28.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.165+0000 7f57c9d9b640 1 -- 172.21.15.100:0/1215301166 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc15f370 con 0x7f57cc155670 2024-09-15T21:47:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.166+0000 7f57c9d9b640 1 --2- 172.21.15.100:0/1215301166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57cc155670 0x7f57cc15bb60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f57cc06d5b0 tx=0x7f57c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335d5b819ed06feb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.167+0000 7f57c959a640 1 -- 172.21.15.100:0/1215301166 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0037070 con 0x7f57cc155670 2024-09-15T21:47:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.167+0000 7f57c959a640 1 -- 172.21.15.100:0/1215301166 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c0032440 con 0x7f57cc155670 2024-09-15T21:47:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.167+0000 7f57c959a640 1 -- 172.21.15.100:0/1215301166 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0032810 con 0x7f57cc155670 2024-09-15T21:47:28.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.167+0000 7f57d0ae4640 1 -- 172.21.15.100:0/1215301166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57cc155670 msgr2=0x7f57cc15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.167+0000 7f57d0ae4640 1 --2- 172.21.15.100:0/1215301166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57cc155670 0x7f57cc15bb60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f57cc06d5b0 tx=0x7f57c002a1e0 comp rx=0 tx=0).stop 2024-09-15T21:47:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.167+0000 7f57d0ae4640 1 -- 172.21.15.100:0/1215301166 shutdown_connections 2024-09-15T21:47:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.167+0000 7f57d0ae4640 1 --2- 172.21.15.100:0/1215301166 >> v2:172.21.15.122:3300/0 conn(0x7f57cc0c65b0 0x7f57cc0c69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.167+0000 7f57d0ae4640 1 --2- 172.21.15.100:0/1215301166 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57cc155670 0x7f57cc15bb60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:28.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.167+0000 7f57d0ae4640 1 --2- 172.21.15.100:0/1215301166 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f57cc155290 0x7f57cc15e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.167+0000 7f57d0ae4640 1 -- 172.21.15.100:0/1215301166 >> 172.21.15.100:0/1215301166 conn(0x7f57cc073310 msgr2=0x7f57cc073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.167+0000 7f57d0ae4640 1 -- 172.21.15.100:0/1215301166 shutdown_connections 2024-09-15T21:47:28.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.167+0000 7f57d0ae4640 1 -- 172.21.15.100:0/1215301166 wait complete. 2024-09-15T21:47:28.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.169+0000 7f57d0ae4640 1 Processor -- start 2024-09-15T21:47:28.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.169+0000 7f57d0ae4640 1 -- start start 2024-09-15T21:47:28.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.169+0000 7f57d0ae4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57cc0c65b0 0x7f57cc158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.169+0000 7f57d0ae4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f57cc155290 0x7f57cc153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.169+0000 7f57d0ae4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57cc155670 0x7f57cc144f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.169+0000 7f57d0ae4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc0b39e0 con 0x7f57cc155670 2024-09-15T21:47:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.169+0000 7f57d0ae4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc0c6090 con 0x7f57cc0c65b0 2024-09-15T21:47:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.169+0000 7f57d0ae4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f57cc15c590 con 0x7f57cc155290 2024-09-15T21:47:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.170+0000 7f57cad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57cc155670 0x7f57cc144f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.170+0000 7f57cad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57cc155670 0x7f57cc144f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36568/0 (socket says 172.21.15.100:36568) 2024-09-15T21:47:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.170+0000 7f57cad9d640 1 -- 172.21.15.100:0/3314375460 learned_addr learned my addr 172.21.15.100:0/3314375460 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.170+0000 7f57c9d9b640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f57cc155290 0x7f57cc153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.170+0000 7f57ca59c640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57cc0c65b0 0x7f57cc158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.170+0000 7f57cad9d640 1 -- 172.21.15.100:0/3314375460 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f57cc155290 msgr2=0x7f57cc153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.170+0000 7f57cad9d640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f57cc155290 0x7f57cc153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:28.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.170+0000 7f57cad9d640 1 -- 172.21.15.100:0/3314375460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57cc0c65b0 msgr2=0x7f57cc158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.170+0000 7f57cad9d640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57cc0c65b0 0x7f57cc158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.170+0000 7f57cad9d640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc1573d0 con 0x7f57cc155670 2024-09-15T21:47:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.170+0000 7f57cad9d640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f57cc155670 0x7f57cc144f10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f57c400ef00 tx=0x7f57c400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.170+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c4018020 con 0x7f57cc155670 2024-09-15T21:47:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.170+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c4014400 con 0x7f57cc155670 2024-09-15T21:47:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.170+0000 7f57d0ae4640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57cc157670 con 0x7f57cc155670 2024-09-15T21:47:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.170+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c40147f0 con 0x7f57cc155670 2024-09-15T21:47:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.170+0000 7f57d0ae4640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57cc157c30 con 0x7f57cc155670 2024-09-15T21:47:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.171+0000 7f57d0ae4640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f57cc0d19e0 con 0x7f57cc155670 2024-09-15T21:47:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.172+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f57c4005d10 con 0x7f57cc155670 2024-09-15T21:47:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.173+0000 7f57baffd640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f57ac06c2b0 0x7f57ac06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.173+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5854+0+0 (secure 0 0 0) 0x7f57c408c7a0 con 0x7f57cc155670 2024-09-15T21:47:28.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.173+0000 7f57ca59c640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f57ac06c2b0 0x7f57ac06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.173+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v35) ==== 54+0+16 (secure 0 0 0) 0x7f57cc0d19e0 con 0x7f57cc155670 2024-09-15T21:47:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.173+0000 7f57d0ae4640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f57cc159750 0x7f57cc145960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:28.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.173+0000 7f57d0ae4640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f57cc0c69e0 con 0x7f57cc159750 2024-09-15T21:47:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.173+0000 7f57ca59c640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f57ac06c2b0 0x7f57ac06e770 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009b40 tx=0x7f57bc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.173+0000 7f57c9d9b640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f57cc159750 0x7f57cc145960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.174+0000 7f57c9d9b640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f57cc159750 0x7f57cc145960 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:28.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.175+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57cc0c69e0 con 0x7f57cc159750 2024-09-15T21:47:28.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.190+0000 7f57d0ae4640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f57cc0b5b80 con 0x7f57cc159750 2024-09-15T21:47:28.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.192+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f57cc0b5b80 con 0x7f57cc159750 2024-09-15T21:47:28.194 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:28.195 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:28.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.192+0000 7f57d0ae4640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f57cc155a50 0x7f57cc1f4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:28.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.192+0000 7f57d0ae4640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f57cc0b58a0 con 0x7f57cc155a50 2024-09-15T21:47:28.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.193+0000 7f57ca59c640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f57cc155a50 0x7f57cc1f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:28.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.193+0000 7f57ca59c640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f57cc155a50 0x7f57cc1f4850 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:28.195 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.194+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57cc0b58a0 con 0x7f57cc155a50 2024-09-15T21:47:28.199 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:28.199 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:28.208 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:47:28.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.208+0000 7f57d0ae4640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f57cc0b55c0 con 0x7f57cc155a50 2024-09-15T21:47:28.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.209+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f57cc0b55c0 con 0x7f57cc155a50 2024-09-15T21:47:28.211 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:28.212 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.209+0000 7f57d0ae4640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f57cc15a900 0x7f57cc062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.209+0000 7f57d0ae4640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f57cc1f2280 con 0x7f57cc15a900 2024-09-15T21:47:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.210+0000 7f57c9d9b640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f57cc15a900 0x7f57cc062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.210+0000 7f57c9d9b640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f57cc15a900 0x7f57cc062630 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.210+0000 7f57c9d9b640 1 -- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f57cc15a900 msgr2=0x7f57cc062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 23 2024-09-15T21:47:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.210+0000 7f57c9d9b640 1 -- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f57cc15a900 msgr2=0x7f57cc062630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-15T21:47:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.210+0000 7f57c9d9b640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f57cc15a900 0x7f57cc062630 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-15T21:47:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.210+0000 7f57c9d9b640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f57cc15a900 0x7f57cc062630 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.210+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f57cc15a900 msgr2=0x7f57cc062630 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:47:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.210+0000 7f57baffd640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f57cc15a900 0x7f57cc062630 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.210+0000 7f57baffd640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f57ac076260 0x7f57ac078650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.210+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f57cc0b55c0 con 0x7f57ac076260 2024-09-15T21:47:28.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.210+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=36}) -- 0x7f57ac078b90 con 0x7f57cc155670 2024-09-15T21:47:28.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.210+0000 7f57c9d9b640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f57ac076260 0x7f57ac078650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:28.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.211+0000 7f57c9d9b640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f57ac076260 0x7f57ac078650 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:28.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.211+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57cc0b55c0 con 0x7f57ac076260 2024-09-15T21:47:28.227 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.225+0000 7f57d0ae4640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f57cc062880 con 0x7f57ac076260 2024-09-15T21:47:28.228 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.226+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f57cc062880 con 0x7f57ac076260 2024-09-15T21:47:28.228 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:28.229 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:28.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.227+0000 7f57d0ae4640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f57cc063360 0x7f57cc02c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:28.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.227+0000 7f57d0ae4640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f57cc1f2430 con 0x7f57cc063360 2024-09-15T21:47:28.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.227+0000 7f57cad9d640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f57cc063360 0x7f57cc02c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:28.229 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.227+0000 7f57cad9d640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f57cc063360 0x7f57cc02c640 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:28.230 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.228+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57cc1f2430 con 0x7f57cc063360 2024-09-15T21:47:28.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.243+0000 7f57d0ae4640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f57cc02cdd0 con 0x7f57cc063360 2024-09-15T21:47:28.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.245+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f57cc02cdd0 con 0x7f57cc063360 2024-09-15T21:47:28.248 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.245+0000 7f57d0ae4640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f57cc02d660 0x7f57cc0cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.245+0000 7f57d0ae4640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f57cc02daa0 con 0x7f57cc02d660 2024-09-15T21:47:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.246+0000 7f57ca59c640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f57cc02d660 0x7f57cc0cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.246+0000 7f57ca59c640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f57cc02d660 0x7f57cc0cb5b0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:28.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.247+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57cc02daa0 con 0x7f57cc02d660 2024-09-15T21:47:28.253 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:28.252+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:28.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.263+0000 7f57d0ae4640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f57cc02daa0 con 0x7f57cc02d660 2024-09-15T21:47:28.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.265+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f57cc02daa0 con 0x7f57cc02d660 2024-09-15T21:47:28.267 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:28.268 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:28.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.265+0000 7f57d0ae4640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f57cc02daa0 con 0x7f57cc155670 2024-09-15T21:47:28.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.265+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f57c405a7a0 con 0x7f57cc155670 2024-09-15T21:47:28.268 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:47:28.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.266+0000 7f57d0ae4640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f57cc0cc3c0 0x7f57cc0cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:28.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.266+0000 7f57d0ae4640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f57cc0ccd40 con 0x7f57cc0cc3c0 2024-09-15T21:47:28.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.266+0000 7f57c9d9b640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f57cc0cc3c0 0x7f57cc0cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:28.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.266+0000 7f57c9d9b640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f57cc0cc3c0 0x7f57cc0cc800 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:28.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.267+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57cc0ccd40 con 0x7f57cc0cc3c0 2024-09-15T21:47:28.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.283+0000 7f57d0ae4640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f57cc0ccd40 con 0x7f57cc0cc3c0 2024-09-15T21:47:28.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.285+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f57cc0ccd40 con 0x7f57cc0cc3c0 2024-09-15T21:47:28.287 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:28.287 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:28.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.285+0000 7f57d0ae4640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f57cc205e90 0x7f57cc208280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:28.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.285+0000 7f57d0ae4640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f57cc20b2a0 con 0x7f57cc205e90 2024-09-15T21:47:28.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.286+0000 7f57cad9d640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f57cc205e90 0x7f57cc208280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:28.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.286+0000 7f57cad9d640 1 --2- 172.21.15.100:0/3314375460 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f57cc205e90 0x7f57cc208280 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:28.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.287+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f57cc20b2a0 con 0x7f57cc205e90 2024-09-15T21:47:28.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.300+0000 7f57d0ae4640 1 -- 172.21.15.100:0/3314375460 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f57cc2088b0 con 0x7f57cc205e90 2024-09-15T21:47:28.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:28.303+0000 7f57baffd640 1 -- 172.21.15.100:0/3314375460 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f57cc2088b0 con 0x7f57cc205e90 2024-09-15T21:47:28.305 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:28.305 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:28.308 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:47:28.308 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:47:28.314 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:28.315 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:28.315 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:28.315 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:28.315 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:28.323 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:47:28.355 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:28.353+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:28.427 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:28.427 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:28.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:28.427 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:28.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.427 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:28.427 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:47:28.428 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.011574261, 2024-09-15T21:47:28.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061533810000000003, 2024-09-15T21:47:28.428 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.428 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.428 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:28.428 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:47:28.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:47:28.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413085+0000", 2024-09-15T21:47:28.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8390000000000003e-06 2024-09-15T21:47:28.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413092+0000", 2024-09-15T21:47:28.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3479999999999998e-06 2024-09-15T21:47:28.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413109+0000", 2024-09-15T21:47:28.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7192000000000001e-05 2024-09-15T21:47:28.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413118+0000", 2024-09-15T21:47:28.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.935e-06 2024-09-15T21:47:28.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413219+0000", 2024-09-15T21:47:28.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010121 2024-09-15T21:47:28.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:28.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413381+0000", 2024-09-15T21:47:28.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016148399999999999 2024-09-15T21:47:28.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:28.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413525+0000", 2024-09-15T21:47:28.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014428100000000001 2024-09-15T21:47:28.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:28.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415866+0000", 2024-09-15T21:47:28.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023409300000000002 2024-09-15T21:47:28.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:28.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419156+0000", 2024-09-15T21:47:28.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032903580000000002 2024-09-15T21:47:28.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:28.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419199+0000", 2024-09-15T21:47:28.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2171e-05 2024-09-15T21:47:28.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419232+0000", 2024-09-15T21:47:28.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3633000000000001e-05 2024-09-15T21:47:28.433 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.433 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.433 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.433 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.4:28 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-15T21:47:28.433 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:47:28.433 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.010310752000002, 2024-09-15T21:47:28.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002614594, 2024-09-15T21:47:28.433 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.434 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.434 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:28.434 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:47:28.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:47:28.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414344+0000", 2024-09-15T21:47:28.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6589999999999999e-06 2024-09-15T21:47:28.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414355+0000", 2024-09-15T21:47:28.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0604999999999999e-05 2024-09-15T21:47:28.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414370+0000", 2024-09-15T21:47:28.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-15T21:47:28.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414379+0000", 2024-09-15T21:47:28.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2449999999999998e-06 2024-09-15T21:47:28.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414477+0000", 2024-09-15T21:47:28.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8968000000000005e-05 2024-09-15T21:47:28.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:28.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414504+0000", 2024-09-15T21:47:28.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6899000000000001e-05 2024-09-15T21:47:28.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:28.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416837+0000", 2024-09-15T21:47:28.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023327130000000001 2024-09-15T21:47:28.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416957+0000", 2024-09-15T21:47:28.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000119796 2024-09-15T21:47:28.438 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.438 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.438 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.439 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.1.10:8 2.b e22/19 2:d70a01c6:::60c.00000000:head v 22'4, mlcod=22'1)", 2024-09-15T21:47:28.439 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:47:28.439 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.966731299999999, 2024-09-15T21:47:28.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027633890000000002, 2024-09-15T21:47:28.439 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.439 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.439 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:28.439 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.439 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:47:28.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:47:28.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457923+0000", 2024-09-15T21:47:28.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.136e-06 2024-09-15T21:47:28.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457931+0000", 2024-09-15T21:47:28.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1529999999999997e-06 2024-09-15T21:47:28.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457946+0000", 2024-09-15T21:47:28.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4482e-05 2024-09-15T21:47:28.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457956+0000", 2024-09-15T21:47:28.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0774999999999999e-05 2024-09-15T21:47:28.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458016+0000", 2024-09-15T21:47:28.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9247999999999999e-05 2024-09-15T21:47:28.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:28.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458036+0000", 2024-09-15T21:47:28.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0920999999999998e-05 2024-09-15T21:47:28.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:28.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460636+0000", 2024-09-15T21:47:28.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025992089999999999 2024-09-15T21:47:28.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460685+0000", 2024-09-15T21:47:28.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9465000000000002e-05 2024-09-15T21:47:28.443 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.443 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.443 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.443 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.18 epoch 22/19 rep_tid 5 entries 22'1 (0'0) error 2:1d00e16c:::201.00000005:head by mds.1.10:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:28.444 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:47:28.444 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.965039169000001, 2024-09-15T21:47:28.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003509599, 2024-09-15T21:47:28.444 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.444 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.445 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:28.445 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:47:28.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.445 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:47:28.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459615+0000", 2024-09-15T21:47:28.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4200000000000005e-07 2024-09-15T21:47:28.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459626+0000", 2024-09-15T21:47:28.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1456e-05 2024-09-15T21:47:28.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459650+0000", 2024-09-15T21:47:28.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.429e-05 2024-09-15T21:47:28.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459664+0000", 2024-09-15T21:47:28.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3805000000000001e-05 2024-09-15T21:47:28.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459784+0000", 2024-09-15T21:47:28.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000119614 2024-09-15T21:47:28.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463123+0000", 2024-09-15T21:47:28.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033395920000000002 2024-09-15T21:47:28.448 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.448 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.448 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.448 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:7 2.13 e22/19 2:cdf3ac81:::616.00000000:head v 22'5, mlcod=22'3)", 2024-09-15T21:47:28.448 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:47:28.448 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 47.947258390999998, 2024-09-15T21:47:28.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002138615, 2024-09-15T21:47:28.448 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.448 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.448 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:28.448 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:47:28.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:47:28.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477396+0000", 2024-09-15T21:47:28.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4190000000000001e-06 2024-09-15T21:47:28.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477405+0000", 2024-09-15T21:47:28.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2320000000000007e-06 2024-09-15T21:47:28.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477419+0000", 2024-09-15T21:47:28.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3589e-05 2024-09-15T21:47:28.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.451 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477428+0000", 2024-09-15T21:47:28.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4779999999999997e-06 2024-09-15T21:47:28.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.452 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477493+0000", 2024-09-15T21:47:28.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4546000000000002e-05 2024-09-15T21:47:28.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:28.452 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477511+0000", 2024-09-15T21:47:28.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7793000000000001e-05 2024-09-15T21:47:28.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:28.452 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479481+0000", 2024-09-15T21:47:28.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019701660000000002 2024-09-15T21:47:28.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479533+0000", 2024-09-15T21:47:28.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2392000000000003e-05 2024-09-15T21:47:28.453 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.453 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.453 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.454 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:26 2.11 e22/19 2:893cd530:::mds2_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-15T21:47:28.454 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:28.454 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 47.947111464999999, 2024-09-15T21:47:28.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020450450000000001, 2024-09-15T21:47:28.454 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.454 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.454 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:28.454 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:28.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:28.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:28.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9699999999999995e-07 2024-09-15T21:47:28.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477555+0000", 2024-09-15T21:47:28.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3144e-05 2024-09-15T21:47:28.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477565+0000", 2024-09-15T21:47:28.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8579999999999995e-06 2024-09-15T21:47:28.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.457 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477573+0000", 2024-09-15T21:47:28.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4869999999999996e-06 2024-09-15T21:47:28.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.457 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477632+0000", 2024-09-15T21:47:28.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9178999999999997e-05 2024-09-15T21:47:28.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.457 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:28.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477654+0000", 2024-09-15T21:47:28.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2234000000000001e-05 2024-09-15T21:47:28.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:28.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479526+0000", 2024-09-15T21:47:28.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018717600000000001 2024-09-15T21:47:28.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479587+0000", 2024-09-15T21:47:28.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0686000000000001e-05 2024-09-15T21:47:28.459 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.459 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.459 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.459 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:22 2.1a e22/19 2:5fd68f60:::61d.00000000:head v 22'3, mlcod=0'0)", 2024-09-15T21:47:28.459 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:28.460 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 47.946751409999997, 2024-09-15T21:47:28.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023694380000000002, 2024-09-15T21:47:28.460 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.460 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.460 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:28.460 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:28.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:28.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:28.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4899999999999995e-07 2024-09-15T21:47:28.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477910+0000", 2024-09-15T21:47:28.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:47:28.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477920+0000", 2024-09-15T21:47:28.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9250000000000007e-06 2024-09-15T21:47:28.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477927+0000", 2024-09-15T21:47:28.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0940000000000002e-06 2024-09-15T21:47:28.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477985+0000", 2024-09-15T21:47:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8451e-05 2024-09-15T21:47:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478005+0000", 2024-09-15T21:47:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0064e-05 2024-09-15T21:47:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480223+0000", 2024-09-15T21:47:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022177910000000002 2024-09-15T21:47:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480271+0000", 2024-09-15T21:47:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8164999999999997e-05 2024-09-15T21:47:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:8 2.13 e22/19 2:cdf3ac81:::616.00000000:head v 22'6, mlcod=22'3)", 2024-09-15T21:47:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:47:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 47.946552451000002, 2024-09-15T21:47:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021760249999999998, 2024-09-15T21:47:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:47:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:47:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478102+0000", 2024-09-15T21:47:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3099999999999996e-07 2024-09-15T21:47:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478124+0000", 2024-09-15T21:47:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2790999999999999e-05 2024-09-15T21:47:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478136+0000", 2024-09-15T21:47:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1922999999999999e-05 2024-09-15T21:47:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478144+0000", 2024-09-15T21:47:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9300000000000003e-06 2024-09-15T21:47:28.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478192+0000", 2024-09-15T21:47:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8155999999999998e-05 2024-09-15T21:47:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478209+0000", 2024-09-15T21:47:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6816e-05 2024-09-15T21:47:28.469 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480243+0000", 2024-09-15T21:47:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020336199999999999 2024-09-15T21:47:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480277+0000", 2024-09-15T21:47:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3858e-05 2024-09-15T21:47:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.470 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.11 epoch 22/19 rep_tid 19 entries 22'2 (0'0) error 2:8975f766:::202.00000001:head by mds.2.12:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 47.946426512999999, 2024-09-15T21:47:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023882299999999999, 2024-09-15T21:47:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2399999999999998e-07 2024-09-15T21:47:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478236+0000", 2024-09-15T21:47:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5299999999999996e-06 2024-09-15T21:47:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478248+0000", 2024-09-15T21:47:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2806000000000001e-05 2024-09-15T21:47:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478264+0000", 2024-09-15T21:47:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5468000000000002e-05 2024-09-15T21:47:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478339+0000", 2024-09-15T21:47:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4851000000000006e-05 2024-09-15T21:47:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480615+0000", 2024-09-15T21:47:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022760509999999999 2024-09-15T21:47:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:3 2.1a e22/19 2:5fdc322d:::61e.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:47:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:47:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 42.878241911000003, 2024-09-15T21:47:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023961220000000001, 2024-09-15T21:47:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:47:28.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:47:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546413+0000", 2024-09-15T21:47:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.812e-06 2024-09-15T21:47:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546424+0000", 2024-09-15T21:47:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1313e-05 2024-09-15T21:47:28.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546444+0000", 2024-09-15T21:47:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9706000000000001e-05 2024-09-15T21:47:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546453+0000", 2024-09-15T21:47:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:47:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546519+0000", 2024-09-15T21:47:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5710999999999995e-05 2024-09-15T21:47:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546539+0000", 2024-09-15T21:47:28.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0275000000000002e-05 2024-09-15T21:47:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548763+0000", 2024-09-15T21:47:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002223226 2024-09-15T21:47:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548807+0000", 2024-09-15T21:47:28.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4598000000000002e-05 2024-09-15T21:47:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:19 2.11 e22/19 2:890f50f7:::626.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:47:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:47:28.482 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 42.877944751000001, 2024-09-15T21:47:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022671280000000002, 2024-09-15T21:47:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:47:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:47:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546709+0000", 2024-09-15T21:47:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4499999999999997e-07 2024-09-15T21:47:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546723+0000", 2024-09-15T21:47:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4623999999999999e-05 2024-09-15T21:47:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546734+0000", 2024-09-15T21:47:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0331e-05 2024-09-15T21:47:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546741+0000", 2024-09-15T21:47:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.075e-06 2024-09-15T21:47:28.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546954+0000", 2024-09-15T21:47:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021275600000000001 2024-09-15T21:47:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546971+0000", 2024-09-15T21:47:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7503000000000002e-05 2024-09-15T21:47:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548926+0000", 2024-09-15T21:47:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019550129999999998 2024-09-15T21:47:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548975+0000", 2024-09-15T21:47:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9280999999999998e-05 2024-09-15T21:47:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.487 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.d epoch 22/19 rep_tid 18 entries 22'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.14:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:47:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 42.877938401000002, 2024-09-15T21:47:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021738489999999998, 2024-09-15T21:47:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:28.488 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:47:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:47:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546716+0000", 2024-09-15T21:47:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.615e-06 2024-09-15T21:47:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546730+0000", 2024-09-15T21:47:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3519999999999999e-05 2024-09-15T21:47:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546753+0000", 2024-09-15T21:47:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3643999999999999e-05 2024-09-15T21:47:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546770+0000", 2024-09-15T21:47:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6518999999999999e-05 2024-09-15T21:47:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546900+0000", 2024-09-15T21:47:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013049099999999999 2024-09-15T21:47:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548888+0000", 2024-09-15T21:47:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019880599999999998 2024-09-15T21:47:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:4 2.1a e22/19 2:5fdc322d:::61e.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:47:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:47:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 42.877829753, 2024-09-15T21:47:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002968769, 2024-09-15T21:47:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:47:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:47:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546824+0000", 2024-09-15T21:47:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6499999999999999e-07 2024-09-15T21:47:28.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546831+0000", 2024-09-15T21:47:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9940000000000003e-06 2024-09-15T21:47:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546840+0000", 2024-09-15T21:47:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3689999999999996e-06 2024-09-15T21:47:28.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546847+0000", 2024-09-15T21:47:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0589999999999997e-06 2024-09-15T21:47:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547709+0000", 2024-09-15T21:47:28.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00086144699999999999 2024-09-15T21:47:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547730+0000", 2024-09-15T21:47:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1542000000000001e-05 2024-09-15T21:47:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549750+0000", 2024-09-15T21:47:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002020165 2024-09-15T21:47:28.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549792+0000", 2024-09-15T21:47:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1628000000000002e-05 2024-09-15T21:47:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:20 2.11 e22/19 2:890f50f7:::626.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:47:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:28.498 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 42.876805144999999, 2024-09-15T21:47:28.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017734579999999999, 2024-09-15T21:47:28.499 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.499 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.499 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:28.499 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:28.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:28.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:28.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:47:28.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547856+0000", 2024-09-15T21:47:28.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7999999999999999e-06 2024-09-15T21:47:28.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.501 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.501 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547867+0000", 2024-09-15T21:47:28.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0971999999999999e-05 2024-09-15T21:47:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547881+0000", 2024-09-15T21:47:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4168999999999999e-05 2024-09-15T21:47:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547935+0000", 2024-09-15T21:47:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3721999999999998e-05 2024-09-15T21:47:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547976+0000", 2024-09-15T21:47:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1248999999999998e-05 2024-09-15T21:47:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549559+0000", 2024-09-15T21:47:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001582815 2024-09-15T21:47:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549621+0000", 2024-09-15T21:47:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2156999999999995e-05 2024-09-15T21:47:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:13 2.1a e22/19 2:594e21d0:::623.00000000:head v 22'6, mlcod=22'2)", 2024-09-15T21:47:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:47:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 42.876740149, 2024-09-15T21:47:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019085300000000001, 2024-09-15T21:47:28.504 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:47:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:47:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547914+0000", 2024-09-15T21:47:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1500000000000004e-07 2024-09-15T21:47:28.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547920+0000", 2024-09-15T21:47:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5239999999999997e-06 2024-09-15T21:47:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547930+0000", 2024-09-15T21:47:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5109999999999992e-06 2024-09-15T21:47:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547937+0000", 2024-09-15T21:47:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5410000000000003e-06 2024-09-15T21:47:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548518+0000", 2024-09-15T21:47:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00058074299999999999 2024-09-15T21:47:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548538+0000", 2024-09-15T21:47:28.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9959e-05 2024-09-15T21:47:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549795+0000", 2024-09-15T21:47:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012570120000000001 2024-09-15T21:47:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549821+0000", 2024-09-15T21:47:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6625e-05 2024-09-15T21:47:28.509 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:23 2.18 e22/19 2:1c1db442:::103.00000000.inode:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:47:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 42.876678730999998, 2024-09-15T21:47:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002713198, 2024-09-15T21:47:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:28.510 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:47:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:47:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547975+0000", 2024-09-15T21:47:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3900000000000004e-07 2024-09-15T21:47:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547983+0000", 2024-09-15T21:47:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6690000000000006e-06 2024-09-15T21:47:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547994+0000", 2024-09-15T21:47:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1503e-05 2024-09-15T21:47:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548002+0000", 2024-09-15T21:47:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1340000000000004e-06 2024-09-15T21:47:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548098+0000", 2024-09-15T21:47:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5791000000000005e-05 2024-09-15T21:47:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548118+0000", 2024-09-15T21:47:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0129e-05 2024-09-15T21:47:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550643+0000", 2024-09-15T21:47:28.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002524728 2024-09-15T21:47:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550688+0000", 2024-09-15T21:47:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4404999999999999e-05 2024-09-15T21:47:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:14 2.1a e22/19 2:594e21d0:::623.00000000:head v 22'7, mlcod=22'2)", 2024-09-15T21:47:28.515 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:47:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 42.876663770999997, 2024-09-15T21:47:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024358579999999999, 2024-09-15T21:47:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:47:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:47:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547990+0000", 2024-09-15T21:47:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5799999999999999e-07 2024-09-15T21:47:28.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548011+0000", 2024-09-15T21:47:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0942999999999999e-05 2024-09-15T21:47:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548020+0000", 2024-09-15T21:47:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2380000000000003e-06 2024-09-15T21:47:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548027+0000", 2024-09-15T21:47:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6429999999999998e-06 2024-09-15T21:47:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548848+0000", 2024-09-15T21:47:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082167999999999998 2024-09-15T21:47:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548868+0000", 2024-09-15T21:47:28.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9791000000000001e-05 2024-09-15T21:47:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550387+0000", 2024-09-15T21:47:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001519322 2024-09-15T21:47:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550425+0000", 2024-09-15T21:47:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7682999999999998e-05 2024-09-15T21:47:28.520 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:35 2.f 2:f426ad4f:::mds2_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:47:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 42.686552085999999, 2024-09-15T21:47:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017328949999999999, 2024-09-15T21:47:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.521 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:47:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:47:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738103+0000", 2024-09-15T21:47:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0020000000000001e-06 2024-09-15T21:47:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738112+0000", 2024-09-15T21:47:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2220000000000008e-06 2024-09-15T21:47:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738127+0000", 2024-09-15T21:47:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5016e-05 2024-09-15T21:47:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738137+0000", 2024-09-15T21:47:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0301e-05 2024-09-15T21:47:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738216+0000", 2024-09-15T21:47:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8356000000000007e-05 2024-09-15T21:47:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738340+0000", 2024-09-15T21:47:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012372800000000001 2024-09-15T21:47:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738453+0000", 2024-09-15T21:47:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000113785 2024-09-15T21:47:28.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739115+0000", 2024-09-15T21:47:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00066178500000000002 2024-09-15T21:47:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739745+0000", 2024-09-15T21:47:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00063017900000000003 2024-09-15T21:47:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739783+0000", 2024-09-15T21:47:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7941000000000001e-05 2024-09-15T21:47:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739834+0000", 2024-09-15T21:47:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.058e-05 2024-09-15T21:47:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.527 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.4.16:13 2.13 e22/19 2:c81eaac8:::62d.00000000:head v 22'8, mlcod=22'6)", 2024-09-15T21:47:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:47:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.859147026999999, 2024-09-15T21:47:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002151778, 2024-09-15T21:47:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:47:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:47:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565507+0000", 2024-09-15T21:47:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3340000000000001e-06 2024-09-15T21:47:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565518+0000", 2024-09-15T21:47:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0339999999999999e-05 2024-09-15T21:47:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565536+0000", 2024-09-15T21:47:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8647e-05 2024-09-15T21:47:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565545+0000", 2024-09-15T21:47:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1430000000000004e-06 2024-09-15T21:47:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565611+0000", 2024-09-15T21:47:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5129000000000006e-05 2024-09-15T21:47:28.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565634+0000", 2024-09-15T21:47:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3173e-05 2024-09-15T21:47:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567561+0000", 2024-09-15T21:47:28.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019268180000000001 2024-09-15T21:47:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567658+0000", 2024-09-15T21:47:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7194000000000005e-05 2024-09-15T21:47:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.d epoch 22/19 rep_tid 24 entries 22'3 (0'0) error 2:b2057508:::204.00000002:head by mds.4.16:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 22'1)", 2024-09-15T21:47:28.533 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:47:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 36.858986164999997, 2024-09-15T21:47:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026915049999999999, 2024-09-15T21:47:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:47:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:47:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565668+0000", 2024-09-15T21:47:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.113e-06 2024-09-15T21:47:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565684+0000", 2024-09-15T21:47:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5943999999999998e-05 2024-09-15T21:47:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565705+0000", 2024-09-15T21:47:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0761999999999999e-05 2024-09-15T21:47:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565724+0000", 2024-09-15T21:47:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9208e-05 2024-09-15T21:47:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565817+0000", 2024-09-15T21:47:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3218000000000001e-05 2024-09-15T21:47:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568358+0000", 2024-09-15T21:47:28.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00254126 2024-09-15T21:47:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.538 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.538 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:28.540 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:47:28.540 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:28.538+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:28.640 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:28.638+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:28.683 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:28.684 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:28.684 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:28.684 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:28.692 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:47:28.777 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:28.774+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:28.837 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:28.838 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:28.838 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:28.838 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:28.838 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:28.841 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:28.839+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:28.846 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:47:28.942 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:28.939+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:28.993 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:28.993 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.576750697000001, 2024-09-15T21:47:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0050344960000000003, 2024-09-15T21:47:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:28.994 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0199999999999998e-07 2024-09-15T21:47:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413470+0000", 2024-09-15T21:47:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4329999999999998e-06 2024-09-15T21:47:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.995 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413579+0000", 2024-09-15T21:47:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108593 2024-09-15T21:47:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413587+0000", 2024-09-15T21:47:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3100000000000001e-06 2024-09-15T21:47:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413691+0000", 2024-09-15T21:47:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010357 2024-09-15T21:47:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.996 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413887+0000", 2024-09-15T21:47:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019666799999999999 2024-09-15T21:47:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414027+0000", 2024-09-15T21:47:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013996700000000001 2024-09-15T21:47:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416360+0000", 2024-09-15T21:47:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002332204 2024-09-15T21:47:28.997 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418437+0000", 2024-09-15T21:47:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002077279 2024-09-15T21:47:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418469+0000", 2024-09-15T21:47:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2359e-05 2024-09-15T21:47:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418502+0000", 2024-09-15T21:47:28.998 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2811000000000001e-05 2024-09-15T21:47:28.999 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.999 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:28.999 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:28.999 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:28.999 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:28.999 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.576746657000001, 2024-09-15T21:47:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0053538780000000003, 2024-09-15T21:47:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.000 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2000000000000001e-07 2024-09-15T21:47:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413474+0000", 2024-09-15T21:47:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-15T21:47:29.001 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413612+0000", 2024-09-15T21:47:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000137727 2024-09-15T21:47:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413618+0000", 2024-09-15T21:47:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.241e-06 2024-09-15T21:47:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414443+0000", 2024-09-15T21:47:29.002 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082447 2024-09-15T21:47:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414523+0000", 2024-09-15T21:47:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0059e-05 2024-09-15T21:47:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414653+0000", 2024-09-15T21:47:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013031699999999999 2024-09-15T21:47:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.003 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416366+0000", 2024-09-15T21:47:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001712917 2024-09-15T21:47:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418769+0000", 2024-09-15T21:47:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024031529999999999 2024-09-15T21:47:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418802+0000", 2024-09-15T21:47:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2925000000000001e-05 2024-09-15T21:47:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.004 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418825+0000", 2024-09-15T21:47:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3204e-05 2024-09-15T21:47:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.576738636000002, 2024-09-15T21:47:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059708640000000002, 2024-09-15T21:47:29.005 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.006 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7300000000000002e-07 2024-09-15T21:47:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413482+0000", 2024-09-15T21:47:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2359999999999999e-06 2024-09-15T21:47:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414323+0000", 2024-09-15T21:47:29.007 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00084095399999999999 2024-09-15T21:47:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414332+0000", 2024-09-15T21:47:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0880000000000001e-06 2024-09-15T21:47:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415439+0000", 2024-09-15T21:47:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011071869999999999 2024-09-15T21:47:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.008 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415554+0000", 2024-09-15T21:47:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114499 2024-09-15T21:47:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415682+0000", 2024-09-15T21:47:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012802500000000001 2024-09-15T21:47:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.417957+0000", 2024-09-15T21:47:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022746720000000002 2024-09-15T21:47:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.009 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419385+0000", 2024-09-15T21:47:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001428826 2024-09-15T21:47:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419420+0000", 2024-09-15T21:47:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4875000000000001e-05 2024-09-15T21:47:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419450+0000", 2024-09-15T21:47:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0128999999999999e-05 2024-09-15T21:47:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.010 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.576734817999998, 2024-09-15T21:47:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0087186220000000005, 2024-09-15T21:47:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.011 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413484+0000", 2024-09-15T21:47:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-15T21:47:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.012 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413487+0000", 2024-09-15T21:47:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.534e-06 2024-09-15T21:47:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414353+0000", 2024-09-15T21:47:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00086616999999999998 2024-09-15T21:47:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414359+0000", 2024-09-15T21:47:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3279999999999996e-06 2024-09-15T21:47:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.013 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416041+0000", 2024-09-15T21:47:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001682467 2024-09-15T21:47:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416158+0000", 2024-09-15T21:47:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117242 2024-09-15T21:47:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416284+0000", 2024-09-15T21:47:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012509000000000001 2024-09-15T21:47:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.014 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.417992+0000", 2024-09-15T21:47:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017087059999999999 2024-09-15T21:47:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422142+0000", 2024-09-15T21:47:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041501180000000004 2024-09-15T21:47:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422176+0000", 2024-09-15T21:47:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4109999999999997e-05 2024-09-15T21:47:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.015 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422202+0000", 2024-09-15T21:47:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5585e-05 2024-09-15T21:47:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 60.576729393000001, 2024-09-15T21:47:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0088000370000000001, 2024-09-15T21:47:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.016 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.017 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413491+0000", 2024-09-15T21:47:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.773e-06 2024-09-15T21:47:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414372+0000", 2024-09-15T21:47:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00088122000000000005 2024-09-15T21:47:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414378+0000", 2024-09-15T21:47:29.018 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4380000000000003e-06 2024-09-15T21:47:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416631+0000", 2024-09-15T21:47:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022530319999999999 2024-09-15T21:47:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416765+0000", 2024-09-15T21:47:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013423499999999999 2024-09-15T21:47:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422289+0000", 2024-09-15T21:47:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055240530000000001 2024-09-15T21:47:29.019 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.020 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.020 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.020 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.020 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:29.020 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:29.020 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.535161074000001, 2024-09-15T21:47:29.020 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056900609999999997, 2024-09-15T21:47:29.020 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.020 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.020 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.020 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.020 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.020 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:29.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:29.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.021 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455058+0000", 2024-09-15T21:47:29.021 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.51e-07 2024-09-15T21:47:29.021 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.021 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.021 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455061+0000", 2024-09-15T21:47:29.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.145e-06 2024-09-15T21:47:29.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455158+0000", 2024-09-15T21:47:29.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7101000000000004e-05 2024-09-15T21:47:29.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.022 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455165+0000", 2024-09-15T21:47:29.022 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1019999999999999e-06 2024-09-15T21:47:29.022 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.022 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.022 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456433+0000", 2024-09-15T21:47:29.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012678259999999999 2024-09-15T21:47:29.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456517+0000", 2024-09-15T21:47:29.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4373e-05 2024-09-15T21:47:29.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:29.023 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456677+0000", 2024-09-15T21:47:29.023 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015963300000000001 2024-09-15T21:47:29.023 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.023 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.023 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458636+0000", 2024-09-15T21:47:29.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019594030000000002 2024-09-15T21:47:29.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460665+0000", 2024-09-15T21:47:29.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020283549999999999 2024-09-15T21:47:29.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.024 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.024 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460707+0000", 2024-09-15T21:47:29.024 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2398000000000003e-05 2024-09-15T21:47:29.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.025 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.025 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460747+0000", 2024-09-15T21:47:29.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0274000000000001e-05 2024-09-15T21:47:29.025 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.025 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.025 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.025 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:29.025 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:29.025 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.535153880000003, 2024-09-15T21:47:29.025 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086420030000000005, 2024-09-15T21:47:29.025 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.026 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.026 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.026 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:29.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:29.026 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.026 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.026 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.026 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455065+0000", 2024-09-15T21:47:29.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3700000000000001e-07 2024-09-15T21:47:29.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455068+0000", 2024-09-15T21:47:29.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.915e-06 2024-09-15T21:47:29.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455366+0000", 2024-09-15T21:47:29.027 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00029825400000000002 2024-09-15T21:47:29.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.027 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.027 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455374+0000", 2024-09-15T21:47:29.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6240000000000008e-06 2024-09-15T21:47:29.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456199+0000", 2024-09-15T21:47:29.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082448500000000002 2024-09-15T21:47:29.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.028 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.028 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457034+0000", 2024-09-15T21:47:29.028 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00083553399999999997 2024-09-15T21:47:29.028 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:29.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457171+0000", 2024-09-15T21:47:29.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013653600000000001 2024-09-15T21:47:29.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.029 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462663+0000", 2024-09-15T21:47:29.029 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054924420000000002 2024-09-15T21:47:29.029 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.029 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.029 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463613+0000", 2024-09-15T21:47:29.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094971499999999996 2024-09-15T21:47:29.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463663+0000", 2024-09-15T21:47:29.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9636999999999999e-05 2024-09-15T21:47:29.030 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.030 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.030 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463706+0000", 2024-09-15T21:47:29.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3523999999999999e-05 2024-09-15T21:47:29.030 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.030 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.030 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.031 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:29.031 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:29.031 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.535149351999998, 2024-09-15T21:47:29.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090408420000000003, 2024-09-15T21:47:29.031 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.031 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.031 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.031 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:29.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:29.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:29.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.79e-07 2024-09-15T21:47:29.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455072+0000", 2024-09-15T21:47:29.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6189999999999998e-06 2024-09-15T21:47:29.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455403+0000", 2024-09-15T21:47:29.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033167299999999999 2024-09-15T21:47:29.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455420+0000", 2024-09-15T21:47:29.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6518999999999999e-05 2024-09-15T21:47:29.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457448+0000", 2024-09-15T21:47:29.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020275710000000001 2024-09-15T21:47:29.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:29.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3844999999999994e-05 2024-09-15T21:47:29.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:29.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457684+0000", 2024-09-15T21:47:29.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015239 2024-09-15T21:47:29.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462678+0000", 2024-09-15T21:47:29.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049939279999999999 2024-09-15T21:47:29.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464017+0000", 2024-09-15T21:47:29.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013397140000000001 2024-09-15T21:47:29.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464071+0000", 2024-09-15T21:47:29.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3735e-05 2024-09-15T21:47:29.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464110+0000", 2024-09-15T21:47:29.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8569000000000003e-05 2024-09-15T21:47:29.036 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.036 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.036 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.036 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:29.036 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:29.036 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 54.535137739, 2024-09-15T21:47:29.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091802889999999995, 2024-09-15T21:47:29.037 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.037 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.037 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.037 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:29.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:29.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.038 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.038 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455081+0000", 2024-09-15T21:47:29.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:29.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.038 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.038 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455084+0000", 2024-09-15T21:47:29.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7810000000000001e-06 2024-09-15T21:47:29.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.038 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.038 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455536+0000", 2024-09-15T21:47:29.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00045219100000000001 2024-09-15T21:47:29.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455583+0000", 2024-09-15T21:47:29.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7308999999999997e-05 2024-09-15T21:47:29.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457998+0000", 2024-09-15T21:47:29.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024154290000000002 2024-09-15T21:47:29.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458147+0000", 2024-09-15T21:47:29.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014834599999999999 2024-09-15T21:47:29.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:29.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458265+0000", 2024-09-15T21:47:29.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118458 2024-09-15T21:47:29.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464134+0000", 2024-09-15T21:47:29.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005869038 2024-09-15T21:47:29.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464187+0000", 2024-09-15T21:47:29.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2343999999999999e-05 2024-09-15T21:47:29.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464222+0000", 2024-09-15T21:47:29.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5447000000000001e-05 2024-09-15T21:47:29.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464261+0000", 2024-09-15T21:47:29.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8659999999999999e-05 2024-09-15T21:47:29.041 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.042 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.042 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.042 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:29.042 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:29.042 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.514515359999997, 2024-09-15T21:47:29.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048468290000000004, 2024-09-15T21:47:29.042 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.042 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.042 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.042 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:29.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:29.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:29.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.35e-07 2024-09-15T21:47:29.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475706+0000", 2024-09-15T21:47:29.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.994e-06 2024-09-15T21:47:29.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475770+0000", 2024-09-15T21:47:29.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3904000000000003e-05 2024-09-15T21:47:29.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475775+0000", 2024-09-15T21:47:29.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2490000000000001e-06 2024-09-15T21:47:29.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476502+0000", 2024-09-15T21:47:29.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072697300000000001 2024-09-15T21:47:29.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476611+0000", 2024-09-15T21:47:29.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0001091 2024-09-15T21:47:29.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:29.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476807+0000", 2024-09-15T21:47:29.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000195892 2024-09-15T21:47:29.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479678+0000", 2024-09-15T21:47:29.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028704709999999999 2024-09-15T21:47:29.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480444+0000", 2024-09-15T21:47:29.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00076602099999999998 2024-09-15T21:47:29.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480502+0000", 2024-09-15T21:47:29.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8313999999999998e-05 2024-09-15T21:47:29.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480550+0000", 2024-09-15T21:47:29.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7676000000000002e-05 2024-09-15T21:47:29.047 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.047 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.047 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.047 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:29.047 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:29.047 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.514508184999997, 2024-09-15T21:47:29.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051250929999999998, 2024-09-15T21:47:29.048 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.048 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.048 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:29.048 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:29.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:29.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:29.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1e-07 2024-09-15T21:47:29.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475723+0000", 2024-09-15T21:47:29.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2758000000000001e-05 2024-09-15T21:47:29.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475972+0000", 2024-09-15T21:47:29.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024886299999999999 2024-09-15T21:47:29.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475979+0000", 2024-09-15T21:47:29.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0230000000000004e-06 2024-09-15T21:47:29.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476560+0000", 2024-09-15T21:47:29.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00058097599999999998 2024-09-15T21:47:29.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476689+0000", 2024-09-15T21:47:29.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129486 2024-09-15T21:47:29.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480835+0000", 2024-09-15T21:47:29.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041457769999999998 2024-09-15T21:47:29.051 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.051 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.051 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.051 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:29.051 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:29.052 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 43.445592959999999, 2024-09-15T21:47:29.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054420570000000001, 2024-09-15T21:47:29.052 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.052 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.052 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.052 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:29.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:29.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544626+0000", 2024-09-15T21:47:29.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8100000000000003e-07 2024-09-15T21:47:29.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544629+0000", 2024-09-15T21:47:29.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9869999999999999e-06 2024-09-15T21:47:29.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544775+0000", 2024-09-15T21:47:29.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014610599999999999 2024-09-15T21:47:29.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544782+0000", 2024-09-15T21:47:29.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4109999999999999e-06 2024-09-15T21:47:29.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545617+0000", 2024-09-15T21:47:29.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00083468300000000002 2024-09-15T21:47:29.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545690+0000", 2024-09-15T21:47:29.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3545000000000001e-05 2024-09-15T21:47:29.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:29.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545803+0000", 2024-09-15T21:47:29.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000112083 2024-09-15T21:47:29.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548353+0000", 2024-09-15T21:47:29.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025507220000000001 2024-09-15T21:47:29.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549964+0000", 2024-09-15T21:47:29.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001610382 2024-09-15T21:47:29.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550021+0000", 2024-09-15T21:47:29.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6898999999999999e-05 2024-09-15T21:47:29.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550067+0000", 2024-09-15T21:47:29.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758000000000002e-05 2024-09-15T21:47:29.057 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.057 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.057 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.057 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:29.057 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:29.057 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 43.445587682000003, 2024-09-15T21:47:29.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056323399999999996, 2024-09-15T21:47:29.058 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.058 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.058 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.058 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:29.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:29.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:29.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:47:29.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544634+0000", 2024-09-15T21:47:29.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.58e-06 2024-09-15T21:47:29.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544800+0000", 2024-09-15T21:47:29.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016576899999999999 2024-09-15T21:47:29.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544808+0000", 2024-09-15T21:47:29.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5179999999999998e-06 2024-09-15T21:47:29.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546075+0000", 2024-09-15T21:47:29.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012676969999999999 2024-09-15T21:47:29.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546179+0000", 2024-09-15T21:47:29.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000103935 2024-09-15T21:47:29.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:29.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546286+0000", 2024-09-15T21:47:29.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106758 2024-09-15T21:47:29.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548528+0000", 2024-09-15T21:47:29.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022420510000000001 2024-09-15T21:47:29.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550119+0000", 2024-09-15T21:47:29.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015907079999999999 2024-09-15T21:47:29.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550202+0000", 2024-09-15T21:47:29.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3374000000000005e-05 2024-09-15T21:47:29.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550263+0000", 2024-09-15T21:47:29.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0643999999999997e-05 2024-09-15T21:47:29.062 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.063 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.063 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.063 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:29.063 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:29.063 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 43.445582711, 2024-09-15T21:47:29.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061905470000000002, 2024-09-15T21:47:29.063 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.063 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.063 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.063 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:29.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:29.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:29.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6899999999999999e-07 2024-09-15T21:47:29.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544638+0000", 2024-09-15T21:47:29.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3920000000000001e-06 2024-09-15T21:47:29.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544819+0000", 2024-09-15T21:47:29.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018065099999999999 2024-09-15T21:47:29.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544825+0000", 2024-09-15T21:47:29.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7239999999999997e-06 2024-09-15T21:47:29.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546583+0000", 2024-09-15T21:47:29.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017588219999999999 2024-09-15T21:47:29.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546665+0000", 2024-09-15T21:47:29.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1693000000000005e-05 2024-09-15T21:47:29.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:29.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546808+0000", 2024-09-15T21:47:29.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142993 2024-09-15T21:47:29.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548533+0000", 2024-09-15T21:47:29.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017246 2024-09-15T21:47:29.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550606+0000", 2024-09-15T21:47:29.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020733969999999998 2024-09-15T21:47:29.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550783+0000", 2024-09-15T21:47:29.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000176565 2024-09-15T21:47:29.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550826+0000", 2024-09-15T21:47:29.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3441e-05 2024-09-15T21:47:29.068 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.068 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.068 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.068 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:29.068 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:29.069 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 43.445574989999997, 2024-09-15T21:47:29.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052813010000000004, 2024-09-15T21:47:29.069 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.069 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.069 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.069 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:29.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:29.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:29.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3799999999999999e-07 2024-09-15T21:47:29.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544647+0000", 2024-09-15T21:47:29.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6569999999999999e-06 2024-09-15T21:47:29.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544858+0000", 2024-09-15T21:47:29.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021069900000000001 2024-09-15T21:47:29.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544862+0000", 2024-09-15T21:47:29.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4900000000000002e-06 2024-09-15T21:47:29.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545610+0000", 2024-09-15T21:47:29.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00074786999999999998 2024-09-15T21:47:29.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545706+0000", 2024-09-15T21:47:29.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6271e-05 2024-09-15T21:47:29.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:29.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545834+0000", 2024-09-15T21:47:29.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012759499999999999 2024-09-15T21:47:29.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548554+0000", 2024-09-15T21:47:29.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027199339999999998 2024-09-15T21:47:29.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549818+0000", 2024-09-15T21:47:29.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012642199999999999 2024-09-15T21:47:29.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549884+0000", 2024-09-15T21:47:29.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5843e-05 2024-09-15T21:47:29.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549925+0000", 2024-09-15T21:47:29.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0484000000000001e-05 2024-09-15T21:47:29.075 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.075 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.075 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.075 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:29.075 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:29.076 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 43.445566313, 2024-09-15T21:47:29.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051106110000000001, 2024-09-15T21:47:29.076 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.076 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.076 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.076 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:29.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:29.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:29.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2700000000000001e-07 2024-09-15T21:47:29.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544667+0000", 2024-09-15T21:47:29.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4746000000000001e-05 2024-09-15T21:47:29.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545021+0000", 2024-09-15T21:47:29.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035412900000000001 2024-09-15T21:47:29.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545052+0000", 2024-09-15T21:47:29.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1158000000000002e-05 2024-09-15T21:47:29.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546050+0000", 2024-09-15T21:47:29.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00099782800000000008 2024-09-15T21:47:29.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546291+0000", 2024-09-15T21:47:29.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024093500000000001 2024-09-15T21:47:29.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:29.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546434+0000", 2024-09-15T21:47:29.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014308600000000001 2024-09-15T21:47:29.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548583+0000", 2024-09-15T21:47:29.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021491029999999999 2024-09-15T21:47:29.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549695+0000", 2024-09-15T21:47:29.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011116419999999999 2024-09-15T21:47:29.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549726+0000", 2024-09-15T21:47:29.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1195000000000002e-05 2024-09-15T21:47:29.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549763+0000", 2024-09-15T21:47:29.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6562e-05 2024-09-15T21:47:29.081 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.081 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.081 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.081 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:17 2.14 2:2eebc8f4:::62f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:29.081 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:29.081 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.425136701, 2024-09-15T21:47:29.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071004800000000002, 2024-09-15T21:47:29.081 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.081 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.081 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.081 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:29.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:29.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565083+0000", 2024-09-15T21:47:29.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.689e-06 2024-09-15T21:47:29.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565090+0000", 2024-09-15T21:47:29.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5930000000000002e-06 2024-09-15T21:47:29.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565119+0000", 2024-09-15T21:47:29.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8785999999999998e-05 2024-09-15T21:47:29.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565126+0000", 2024-09-15T21:47:29.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0110000000000003e-06 2024-09-15T21:47:29.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565237+0000", 2024-09-15T21:47:29.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011161 2024-09-15T21:47:29.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565383+0000", 2024-09-15T21:47:29.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014626900000000001 2024-09-15T21:47:29.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:29.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565520+0000", 2024-09-15T21:47:29.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013676399999999999 2024-09-15T21:47:29.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569909+0000", 2024-09-15T21:47:29.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043890149999999996 2024-09-15T21:47:29.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572096+0000", 2024-09-15T21:47:29.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021866559999999999 2024-09-15T21:47:29.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572144+0000", 2024-09-15T21:47:29.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7725000000000001e-05 2024-09-15T21:47:29.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572182+0000", 2024-09-15T21:47:29.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8362000000000003e-05 2024-09-15T21:47:29.086 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.086 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.086 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.086 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:29.087 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:29.087 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.425125602999998, 2024-09-15T21:47:29.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072110280000000004, 2024-09-15T21:47:29.087 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.087 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.087 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.087 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:29.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:29.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:29.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4299999999999999e-07 2024-09-15T21:47:29.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565096+0000", 2024-09-15T21:47:29.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4319999999999999e-06 2024-09-15T21:47:29.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565152+0000", 2024-09-15T21:47:29.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5686999999999997e-05 2024-09-15T21:47:29.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565157+0000", 2024-09-15T21:47:29.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2700000000000004e-06 2024-09-15T21:47:29.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565847+0000", 2024-09-15T21:47:29.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068937299999999996 2024-09-15T21:47:29.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565940+0000", 2024-09-15T21:47:29.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3573999999999996e-05 2024-09-15T21:47:29.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:29.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566085+0000", 2024-09-15T21:47:29.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000144902 2024-09-15T21:47:29.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570595+0000", 2024-09-15T21:47:29.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045103469999999996 2024-09-15T21:47:29.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572224+0000", 2024-09-15T21:47:29.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016282930000000001 2024-09-15T21:47:29.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572260+0000", 2024-09-15T21:47:29.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6103999999999999e-05 2024-09-15T21:47:29.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572304+0000", 2024-09-15T21:47:29.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3803000000000001e-05 2024-09-15T21:47:29.092 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.092 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.092 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.092 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:29.092 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:29.092 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.425121138000002, 2024-09-15T21:47:29.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055995389999999997, 2024-09-15T21:47:29.092 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.092 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.092 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.093 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:29.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:29.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:29.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9999999999999999e-07 2024-09-15T21:47:29.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:29.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2399999999999999e-06 2024-09-15T21:47:29.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565280+0000", 2024-09-15T21:47:29.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017929400000000001 2024-09-15T21:47:29.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565292+0000", 2024-09-15T21:47:29.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2193e-05 2024-09-15T21:47:29.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565357+0000", 2024-09-15T21:47:29.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4684000000000006e-05 2024-09-15T21:47:29.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565499+0000", 2024-09-15T21:47:29.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142032 2024-09-15T21:47:29.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:29.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565634+0000", 2024-09-15T21:47:29.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000135613 2024-09-15T21:47:29.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569941+0000", 2024-09-15T21:47:29.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043066789999999999 2024-09-15T21:47:29.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570585+0000", 2024-09-15T21:47:29.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00064364700000000001 2024-09-15T21:47:29.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570647+0000", 2024-09-15T21:47:29.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2173000000000003e-05 2024-09-15T21:47:29.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570697+0000", 2024-09-15T21:47:29.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9784000000000003e-05 2024-09-15T21:47:29.097 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.097 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.097 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.098 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:29.098 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:29.098 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.425116908, 2024-09-15T21:47:29.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066548939999999997, 2024-09-15T21:47:29.098 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.098 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.098 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.098 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:29.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:29.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565102+0000", 2024-09-15T21:47:29.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-15T21:47:29.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565103+0000", 2024-09-15T21:47:29.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6700000000000001e-06 2024-09-15T21:47:29.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565608+0000", 2024-09-15T21:47:29.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050518299999999998 2024-09-15T21:47:29.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565616+0000", 2024-09-15T21:47:29.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8199999999999997e-06 2024-09-15T21:47:29.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566598+0000", 2024-09-15T21:47:29.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00098176300000000008 2024-09-15T21:47:29.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566871+0000", 2024-09-15T21:47:29.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027312799999999998 2024-09-15T21:47:29.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:29.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567015+0000", 2024-09-15T21:47:29.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014428499999999999 2024-09-15T21:47:29.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570729+0000", 2024-09-15T21:47:29.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037132459999999999 2024-09-15T21:47:29.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571667+0000", 2024-09-15T21:47:29.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00093862299999999997 2024-09-15T21:47:29.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571718+0000", 2024-09-15T21:47:29.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1146999999999997e-05 2024-09-15T21:47:29.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571756+0000", 2024-09-15T21:47:29.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7821000000000003e-05 2024-09-15T21:47:29.103 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.103 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.103 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.103 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.103 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.103 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:29.105 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:47:29.105 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:29.103+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:29.205 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:29.204+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:29.249 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:29.249 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:29.249 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:29.249 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:29.257 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:47:29.306 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:29.304+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:29.360 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:29.361 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:29.361 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:29.361 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:29.361 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:29.369 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:47:29.414 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:29.411+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:29.475 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:47:29.475 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.237821756000002, 2024-09-15T21:47:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0944283340000001, 2024-09-15T21:47:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2024-09-15T21:47:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.476 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234360+0000", 2024-09-15T21:47:29.477 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3710000000000001e-06 2024-09-15T21:47:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234374+0000", 2024-09-15T21:47:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4003e-05 2024-09-15T21:47:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234392+0000", 2024-09-15T21:47:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7294000000000001e-05 2024-09-15T21:47:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.478 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234405+0000", 2024-09-15T21:47:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2961e-05 2024-09-15T21:47:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234483+0000", 2024-09-15T21:47:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8850000000000006e-05 2024-09-15T21:47:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-15T21:47:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234491+0000", 2024-09-15T21:47:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3529999999999997e-06 2024-09-15T21:47:29.479 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327253+0000", 2024-09-15T21:47:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.092761962 2024-09-15T21:47:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327456+0000", 2024-09-15T21:47:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000202893 2024-09-15T21:47:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.480 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327652+0000", 2024-09-15T21:47:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019613000000000001 2024-09-15T21:47:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328368+0000", 2024-09-15T21:47:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000716582 2024-09-15T21:47:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328682+0000", 2024-09-15T21:47:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000313825 2024-09-15T21:47:29.481 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328743+0000", 2024-09-15T21:47:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0571e-05 2024-09-15T21:47:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328787+0000", 2024-09-15T21:47:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4539000000000002e-05 2024-09-15T21:47:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.482 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 75.116656354, 2024-09-15T21:47:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0089551270000000002, 2024-09-15T21:47:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 35 2024-09-15T21:47:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.483 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355525+0000", 2024-09-15T21:47:29.484 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:47:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355606+0000", 2024-09-15T21:47:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1723000000000003e-05 2024-09-15T21:47:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355624+0000", 2024-09-15T21:47:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7645000000000001e-05 2024-09-15T21:47:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.485 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355629+0000", 2024-09-15T21:47:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7319999999999998e-06 2024-09-15T21:47:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358315+0000", 2024-09-15T21:47:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026858659999999999 2024-09-15T21:47:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358373+0000", 2024-09-15T21:47:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8593e-05 2024-09-15T21:47:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.486 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358521+0000", 2024-09-15T21:47:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014817500000000001 2024-09-15T21:47:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360353+0000", 2024-09-15T21:47:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018317100000000001 2024-09-15T21:47:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364424+0000", 2024-09-15T21:47:29.487 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040707970000000001 2024-09-15T21:47:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364452+0000", 2024-09-15T21:47:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7937999999999999e-05 2024-09-15T21:47:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364479+0000", 2024-09-15T21:47:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7744999999999999e-05 2024-09-15T21:47:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.488 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 75.116572391000005, 2024-09-15T21:47:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090267869999999997, 2024-09-15T21:47:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 36 2024-09-15T21:47:29.489 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.490 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355609+0000", 2024-09-15T21:47:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3099999999999998e-07 2024-09-15T21:47:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355612+0000", 2024-09-15T21:47:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1489999999999998e-06 2024-09-15T21:47:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355637+0000", 2024-09-15T21:47:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.508e-05 2024-09-15T21:47:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.491 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355640+0000", 2024-09-15T21:47:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7930000000000002e-06 2024-09-15T21:47:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359195+0000", 2024-09-15T21:47:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035555529999999999 2024-09-15T21:47:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359283+0000", 2024-09-15T21:47:29.492 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8042999999999997e-05 2024-09-15T21:47:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359419+0000", 2024-09-15T21:47:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013576699999999999 2024-09-15T21:47:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360361+0000", 2024-09-15T21:47:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094171000000000001 2024-09-15T21:47:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.493 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364536+0000", 2024-09-15T21:47:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00417484 2024-09-15T21:47:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364590+0000", 2024-09-15T21:47:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4355999999999999e-05 2024-09-15T21:47:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364635+0000", 2024-09-15T21:47:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5065000000000003e-05 2024-09-15T21:47:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.494 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 75.116567786000005, 2024-09-15T21:47:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091162750000000001, 2024-09-15T21:47:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 37 2024-09-15T21:47:29.495 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.496 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:47:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355615+0000", 2024-09-15T21:47:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-15T21:47:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355646+0000", 2024-09-15T21:47:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0391000000000001e-05 2024-09-15T21:47:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.497 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355648+0000", 2024-09-15T21:47:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6309999999999999e-06 2024-09-15T21:47:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359770+0000", 2024-09-15T21:47:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041222899999999998 2024-09-15T21:47:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359854+0000", 2024-09-15T21:47:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3399000000000006e-05 2024-09-15T21:47:29.498 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359988+0000", 2024-09-15T21:47:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013442200000000001 2024-09-15T21:47:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360524+0000", 2024-09-15T21:47:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00053582900000000004 2024-09-15T21:47:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364673+0000", 2024-09-15T21:47:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041493199999999997 2024-09-15T21:47:29.499 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364704+0000", 2024-09-15T21:47:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0380999999999999e-05 2024-09-15T21:47:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364729+0000", 2024-09-15T21:47:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5335999999999999e-05 2024-09-15T21:47:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.500 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 75.047850548, 2024-09-15T21:47:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0026270020000001, 2024-09-15T21:47:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 88 2024-09-15T21:47:29.501 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.502 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424331+0000", 2024-09-15T21:47:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4299999999999998e-07 2024-09-15T21:47:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424333+0000", 2024-09-15T21:47:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2359999999999999e-06 2024-09-15T21:47:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422895+0000", 2024-09-15T21:47:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99856184199999998 2024-09-15T21:47:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422902+0000", 2024-09-15T21:47:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4440000000000002e-06 2024-09-15T21:47:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423053+0000", 2024-09-15T21:47:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000150958 2024-09-15T21:47:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423160+0000", 2024-09-15T21:47:29.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106774 2024-09-15T21:47:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423344+0000", 2024-09-15T21:47:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018424199999999999 2024-09-15T21:47:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.424730+0000", 2024-09-15T21:47:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001386155 2024-09-15T21:47:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426756+0000", 2024-09-15T21:47:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020261150000000002 2024-09-15T21:47:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426906+0000", 2024-09-15T21:47:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000149176 2024-09-15T21:47:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426957+0000", 2024-09-15T21:47:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1616999999999997e-05 2024-09-15T21:47:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.506 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 73.917777349999994, 2024-09-15T21:47:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095037790000000004, 2024-09-15T21:47:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:29.507 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 114 2024-09-15T21:47:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554404+0000", 2024-09-15T21:47:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2399999999999999e-07 2024-09-15T21:47:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554407+0000", 2024-09-15T21:47:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-15T21:47:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554632+0000", 2024-09-15T21:47:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022568999999999999 2024-09-15T21:47:29.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554640+0000", 2024-09-15T21:47:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7970000000000001e-06 2024-09-15T21:47:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561325+0000", 2024-09-15T21:47:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066848020000000001 2024-09-15T21:47:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561402+0000", 2024-09-15T21:47:29.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6851e-05 2024-09-15T21:47:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561530+0000", 2024-09-15T21:47:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012863999999999999 2024-09-15T21:47:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563144+0000", 2024-09-15T21:47:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016132849999999999 2024-09-15T21:47:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563800+0000", 2024-09-15T21:47:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00065586099999999999 2024-09-15T21:47:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563857+0000", 2024-09-15T21:47:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.702e-05 2024-09-15T21:47:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563907+0000", 2024-09-15T21:47:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0479000000000002e-05 2024-09-15T21:47:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.512 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 73.917772353000004, 2024-09-15T21:47:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011813423999999999, 2024-09-15T21:47:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:29.513 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 115 2024-09-15T21:47:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554409+0000", 2024-09-15T21:47:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1100000000000002e-07 2024-09-15T21:47:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554498+0000", 2024-09-15T21:47:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9642999999999995e-05 2024-09-15T21:47:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554656+0000", 2024-09-15T21:47:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000158114 2024-09-15T21:47:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554661+0000", 2024-09-15T21:47:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0730000000000004e-06 2024-09-15T21:47:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561862+0000", 2024-09-15T21:47:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072002999999999998 2024-09-15T21:47:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561939+0000", 2024-09-15T21:47:29.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7265e-05 2024-09-15T21:47:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562073+0000", 2024-09-15T21:47:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013371600000000001 2024-09-15T21:47:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563196+0000", 2024-09-15T21:47:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011231679999999999 2024-09-15T21:47:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566124+0000", 2024-09-15T21:47:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002928403 2024-09-15T21:47:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566183+0000", 2024-09-15T21:47:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8328000000000001e-05 2024-09-15T21:47:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566222+0000", 2024-09-15T21:47:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9103000000000002e-05 2024-09-15T21:47:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.518 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 73.917678913000003, 2024-09-15T21:47:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01181426, 2024-09-15T21:47:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 116 2024-09-15T21:47:29.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554503+0000", 2024-09-15T21:47:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8000000000000005e-07 2024-09-15T21:47:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554506+0000", 2024-09-15T21:47:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5899999999999999e-06 2024-09-15T21:47:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554674+0000", 2024-09-15T21:47:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016763600000000001 2024-09-15T21:47:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554678+0000", 2024-09-15T21:47:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5310000000000003e-06 2024-09-15T21:47:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562714+0000", 2024-09-15T21:47:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.008035258 2024-09-15T21:47:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562765+0000", 2024-09-15T21:47:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1093e-05 2024-09-15T21:47:29.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562845+0000", 2024-09-15T21:47:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0427999999999996e-05 2024-09-15T21:47:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563943+0000", 2024-09-15T21:47:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097647 2024-09-15T21:47:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566232+0000", 2024-09-15T21:47:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022890850000000002 2024-09-15T21:47:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566279+0000", 2024-09-15T21:47:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6692e-05 2024-09-15T21:47:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566316+0000", 2024-09-15T21:47:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.752e-05 2024-09-15T21:47:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.524 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 73.917672813999999, 2024-09-15T21:47:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012232125, 2024-09-15T21:47:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:29.525 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 117 2024-09-15T21:47:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7700000000000001e-07 2024-09-15T21:47:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554511+0000", 2024-09-15T21:47:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8430000000000001e-06 2024-09-15T21:47:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554690+0000", 2024-09-15T21:47:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017856299999999999 2024-09-15T21:47:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554698+0000", 2024-09-15T21:47:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6200000000000005e-06 2024-09-15T21:47:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563224+0000", 2024-09-15T21:47:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0085262640000000004 2024-09-15T21:47:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563342+0000", 2024-09-15T21:47:29.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117687 2024-09-15T21:47:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563472+0000", 2024-09-15T21:47:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012957999999999999 2024-09-15T21:47:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566367+0000", 2024-09-15T21:47:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028952280000000001 2024-09-15T21:47:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566680+0000", 2024-09-15T21:47:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031345100000000002 2024-09-15T21:47:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566714+0000", 2024-09-15T21:47:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3701000000000001e-05 2024-09-15T21:47:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566740+0000", 2024-09-15T21:47:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5911e-05 2024-09-15T21:47:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.530 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 73.574780731999994, 2024-09-15T21:47:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069806590000000002, 2024-09-15T21:47:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:29.531 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 134 2024-09-15T21:47:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:29.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897401+0000", 2024-09-15T21:47:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0500000000000003e-07 2024-09-15T21:47:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897492+0000", 2024-09-15T21:47:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1770999999999999e-05 2024-09-15T21:47:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897750+0000", 2024-09-15T21:47:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025778299999999998 2024-09-15T21:47:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897756+0000", 2024-09-15T21:47:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2140000000000001e-06 2024-09-15T21:47:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899312+0000", 2024-09-15T21:47:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001556144 2024-09-15T21:47:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899372+0000", 2024-09-15T21:47:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0015000000000001e-05 2024-09-15T21:47:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899444+0000", 2024-09-15T21:47:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1923999999999998e-05 2024-09-15T21:47:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901354+0000", 2024-09-15T21:47:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019092530000000001 2024-09-15T21:47:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904327+0000", 2024-09-15T21:47:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029734520000000001 2024-09-15T21:47:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904360+0000", 2024-09-15T21:47:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3216000000000002e-05 2024-09-15T21:47:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904381+0000", 2024-09-15T21:47:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0282e-05 2024-09-15T21:47:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.537 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:29.538 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:29.538 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 73.574684875000003, 2024-09-15T21:47:29.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070121059999999997, 2024-09-15T21:47:29.538 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.538 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.538 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.538 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:29.538 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:29.538 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:29.538 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 135 2024-09-15T21:47:29.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.538 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:29.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:29.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897497+0000", 2024-09-15T21:47:29.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4799999999999997e-07 2024-09-15T21:47:29.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897500+0000", 2024-09-15T21:47:29.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2949999999999998e-06 2024-09-15T21:47:29.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897781+0000", 2024-09-15T21:47:29.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028064300000000002 2024-09-15T21:47:29.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897786+0000", 2024-09-15T21:47:29.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5169999999999999e-06 2024-09-15T21:47:29.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899857+0000", 2024-09-15T21:47:29.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020705340000000002 2024-09-15T21:47:29.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899932+0000", 2024-09-15T21:47:29.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5840999999999994e-05 2024-09-15T21:47:29.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900005+0000", 2024-09-15T21:47:29.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2716e-05 2024-09-15T21:47:29.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902296+0000", 2024-09-15T21:47:29.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022912010000000001 2024-09-15T21:47:29.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904424+0000", 2024-09-15T21:47:29.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021278600000000001 2024-09-15T21:47:29.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904472+0000", 2024-09-15T21:47:29.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7586000000000001e-05 2024-09-15T21:47:29.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904508+0000", 2024-09-15T21:47:29.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6164999999999997e-05 2024-09-15T21:47:29.543 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.543 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.543 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.544 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:29.544 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:29.544 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 73.574660383999998, 2024-09-15T21:47:29.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070611429999999998, 2024-09-15T21:47:29.544 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.544 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.544 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.544 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:29.544 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:29.544 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:29.544 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 136 2024-09-15T21:47:29.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.545 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:29.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:29.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897521+0000", 2024-09-15T21:47:29.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2900000000000004e-07 2024-09-15T21:47:29.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897527+0000", 2024-09-15T21:47:29.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8490000000000002e-06 2024-09-15T21:47:29.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897799+0000", 2024-09-15T21:47:29.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027219499999999998 2024-09-15T21:47:29.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897804+0000", 2024-09-15T21:47:29.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4219999999999999e-06 2024-09-15T21:47:29.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900248+0000", 2024-09-15T21:47:29.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002443457 2024-09-15T21:47:29.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900306+0000", 2024-09-15T21:47:29.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8263000000000001e-05 2024-09-15T21:47:29.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900385+0000", 2024-09-15T21:47:29.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9161999999999999e-05 2024-09-15T21:47:29.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902300+0000", 2024-09-15T21:47:29.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001914947 2024-09-15T21:47:29.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904536+0000", 2024-09-15T21:47:29.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022354419999999998 2024-09-15T21:47:29.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904562+0000", 2024-09-15T21:47:29.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6378999999999999e-05 2024-09-15T21:47:29.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904581+0000", 2024-09-15T21:47:29.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9497999999999999e-05 2024-09-15T21:47:29.550 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.550 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.550 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.550 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:29.550 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:29.550 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 73.574651817000003, 2024-09-15T21:47:29.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007180306, 2024-09-15T21:47:29.550 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.550 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.550 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.550 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:29.551 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:29.551 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:29.551 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 137 2024-09-15T21:47:29.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.551 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:29.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:29.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:29.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1399999999999998e-07 2024-09-15T21:47:29.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897612+0000", 2024-09-15T21:47:29.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2607000000000004e-05 2024-09-15T21:47:29.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897815+0000", 2024-09-15T21:47:29.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020352400000000001 2024-09-15T21:47:29.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897821+0000", 2024-09-15T21:47:29.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4670000000000003e-06 2024-09-15T21:47:29.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900665+0000", 2024-09-15T21:47:29.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028443689999999998 2024-09-15T21:47:29.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900714+0000", 2024-09-15T21:47:29.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8812e-05 2024-09-15T21:47:29.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900799+0000", 2024-09-15T21:47:29.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5129000000000004e-05 2024-09-15T21:47:29.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903413+0000", 2024-09-15T21:47:29.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026142890000000001 2024-09-15T21:47:29.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904625+0000", 2024-09-15T21:47:29.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012121149999999999 2024-09-15T21:47:29.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904673+0000", 2024-09-15T21:47:29.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7540999999999998e-05 2024-09-15T21:47:29.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904709+0000", 2024-09-15T21:47:29.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6139000000000001e-05 2024-09-15T21:47:29.556 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.556 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.556 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.557 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:29.557 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:29.557 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 73.574565862, 2024-09-15T21:47:29.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071727450000000003, 2024-09-15T21:47:29.557 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.557 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.557 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.557 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:29.557 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:29.557 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:29.557 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 138 2024-09-15T21:47:29.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.558 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:29.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:29.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:29.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:47:29.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897619+0000", 2024-09-15T21:47:29.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3869999999999999e-06 2024-09-15T21:47:29.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897832+0000", 2024-09-15T21:47:29.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021329500000000001 2024-09-15T21:47:29.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897837+0000", 2024-09-15T21:47:29.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3730000000000001e-06 2024-09-15T21:47:29.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901391+0000", 2024-09-15T21:47:29.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003553601 2024-09-15T21:47:29.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901466+0000", 2024-09-15T21:47:29.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5130999999999996e-05 2024-09-15T21:47:29.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901601+0000", 2024-09-15T21:47:29.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134797 2024-09-15T21:47:29.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903417+0000", 2024-09-15T21:47:29.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001816156 2024-09-15T21:47:29.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904741+0000", 2024-09-15T21:47:29.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013241049999999999 2024-09-15T21:47:29.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904767+0000", 2024-09-15T21:47:29.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5876000000000001e-05 2024-09-15T21:47:29.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904788+0000", 2024-09-15T21:47:29.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0449999999999999e-05 2024-09-15T21:47:29.562 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.562 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.563 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.563 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:29.563 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:29.563 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 73.574560208999998, 2024-09-15T21:47:29.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072970250000000004, 2024-09-15T21:47:29.563 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.563 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.563 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.563 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:29.563 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:29.563 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:29.564 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 139 2024-09-15T21:47:29.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.564 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:29.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:29.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897621+0000", 2024-09-15T21:47:29.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3099999999999999e-07 2024-09-15T21:47:29.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897625+0000", 2024-09-15T21:47:29.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.698e-06 2024-09-15T21:47:29.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897848+0000", 2024-09-15T21:47:29.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022375699999999999 2024-09-15T21:47:29.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897853+0000", 2024-09-15T21:47:29.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7709999999999997e-06 2024-09-15T21:47:29.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901929+0000", 2024-09-15T21:47:29.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040757780000000004 2024-09-15T21:47:29.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901976+0000", 2024-09-15T21:47:29.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6751e-05 2024-09-15T21:47:29.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902054+0000", 2024-09-15T21:47:29.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8856999999999994e-05 2024-09-15T21:47:29.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903421+0000", 2024-09-15T21:47:29.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00136619 2024-09-15T21:47:29.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904827+0000", 2024-09-15T21:47:29.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0014067369999999999 2024-09-15T21:47:29.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904881+0000", 2024-09-15T21:47:29.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3520999999999998e-05 2024-09-15T21:47:29.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904918+0000", 2024-09-15T21:47:29.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6634000000000002e-05 2024-09-15T21:47:29.568 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.568 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.569 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.569 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:29.569 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:29.569 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 73.574554379999995, 2024-09-15T21:47:29.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097221730000000006, 2024-09-15T21:47:29.569 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.569 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.569 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.569 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:29.569 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:29.569 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:29.570 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 140 2024-09-15T21:47:29.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.570 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:29.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:29.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897627+0000", 2024-09-15T21:47:29.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-15T21:47:29.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897698+0000", 2024-09-15T21:47:29.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1722000000000002e-05 2024-09-15T21:47:29.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897863+0000", 2024-09-15T21:47:29.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016479099999999999 2024-09-15T21:47:29.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897868+0000", 2024-09-15T21:47:29.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4950000000000002e-06 2024-09-15T21:47:29.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902335+0000", 2024-09-15T21:47:29.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004467632 2024-09-15T21:47:29.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902414+0000", 2024-09-15T21:47:29.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8688000000000003e-05 2024-09-15T21:47:29.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902535+0000", 2024-09-15T21:47:29.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012066900000000001 2024-09-15T21:47:29.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903714+0000", 2024-09-15T21:47:29.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011794520000000001 2024-09-15T21:47:29.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907115+0000", 2024-09-15T21:47:29.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034008879999999999 2024-09-15T21:47:29.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907250+0000", 2024-09-15T21:47:29.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134657 2024-09-15T21:47:29.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907348+0000", 2024-09-15T21:47:29.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8838000000000005e-05 2024-09-15T21:47:29.574 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.574 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.574 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.575 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:29.575 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:29.575 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 73.574479299000004, 2024-09-15T21:47:29.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097618050000000001, 2024-09-15T21:47:29.575 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.575 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.575 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.575 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:29.575 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:29.575 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:29.575 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 141 2024-09-15T21:47:29.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.575 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:29.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:29.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897702+0000", 2024-09-15T21:47:29.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6199999999999998e-07 2024-09-15T21:47:29.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897705+0000", 2024-09-15T21:47:29.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0180000000000002e-06 2024-09-15T21:47:29.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897889+0000", 2024-09-15T21:47:29.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000183678 2024-09-15T21:47:29.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897894+0000", 2024-09-15T21:47:29.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8930000000000004e-06 2024-09-15T21:47:29.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903099+0000", 2024-09-15T21:47:29.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052056280000000003 2024-09-15T21:47:29.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903151+0000", 2024-09-15T21:47:29.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2163000000000002e-05 2024-09-15T21:47:29.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903220+0000", 2024-09-15T21:47:29.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8483000000000001e-05 2024-09-15T21:47:29.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904794+0000", 2024-09-15T21:47:29.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001574521 2024-09-15T21:47:29.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907405+0000", 2024-09-15T21:47:29.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026110339999999999 2024-09-15T21:47:29.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907440+0000", 2024-09-15T21:47:29.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4564999999999999e-05 2024-09-15T21:47:29.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907463+0000", 2024-09-15T21:47:29.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3260000000000001e-05 2024-09-15T21:47:29.581 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.581 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.581 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.581 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:22 2.8 2:1470b2f8:::613.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:29.581 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:29.581 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.017038972000002, 2024-09-15T21:47:29.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007454937, 2024-09-15T21:47:29.581 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.581 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.581 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.581 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:29.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:29.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:29.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6800000000000002e-07 2024-09-15T21:47:29.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455145+0000", 2024-09-15T21:47:29.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4470000000000002e-06 2024-09-15T21:47:29.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456425+0000", 2024-09-15T21:47:29.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012798200000000001 2024-09-15T21:47:29.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456433+0000", 2024-09-15T21:47:29.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7389999999999999e-06 2024-09-15T21:47:29.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:29.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097865 2024-09-15T21:47:29.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457614+0000", 2024-09-15T21:47:29.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3560000000000006e-05 2024-09-15T21:47:29.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457998+0000", 2024-09-15T21:47:29.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00038372699999999999 2024-09-15T21:47:29.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460488+0000", 2024-09-15T21:47:29.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002489855 2024-09-15T21:47:29.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462512+0000", 2024-09-15T21:47:29.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020235269999999998 2024-09-15T21:47:29.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462554+0000", 2024-09-15T21:47:29.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2472000000000003e-05 2024-09-15T21:47:29.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462597+0000", 2024-09-15T21:47:29.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2657000000000002e-05 2024-09-15T21:47:29.586 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.586 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.586 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.586 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:2 2.5 2:a6e36b3f:::401.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:29.586 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:29.587 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 55.017034002999999, 2024-09-15T21:47:29.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074484590000000002, 2024-09-15T21:47:29.587 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.587 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.587 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.587 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:29.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:29.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:29.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:47:29.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455150+0000", 2024-09-15T21:47:29.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7319999999999998e-06 2024-09-15T21:47:29.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456449+0000", 2024-09-15T21:47:29.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012991490000000001 2024-09-15T21:47:29.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456454+0000", 2024-09-15T21:47:29.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0390000000000004e-06 2024-09-15T21:47:29.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456513+0000", 2024-09-15T21:47:29.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8768999999999999e-05 2024-09-15T21:47:29.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456628+0000", 2024-09-15T21:47:29.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115641 2024-09-15T21:47:29.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456741+0000", 2024-09-15T21:47:29.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000113094 2024-09-15T21:47:29.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458520+0000", 2024-09-15T21:47:29.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001778993 2024-09-15T21:47:29.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462494+0000", 2024-09-15T21:47:29.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039737009999999996 2024-09-15T21:47:29.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462549+0000", 2024-09-15T21:47:29.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4537000000000003e-05 2024-09-15T21:47:29.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462595+0000", 2024-09-15T21:47:29.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6511000000000002e-05 2024-09-15T21:47:29.592 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.592 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.592 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.592 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:27 2.5 2:a5fa685f:::504.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:29.592 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:29.592 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 37.907341619, 2024-09-15T21:47:29.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071804599999999996, 2024-09-15T21:47:29.592 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:29.592 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:29.592 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.593 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:29.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:29.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:29.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:29.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:29.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:29.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:29.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:29.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-15T21:47:29.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:29.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564841+0000", 2024-09-15T21:47:29.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-15T21:47:29.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:29.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565126+0000", 2024-09-15T21:47:29.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028469900000000002 2024-09-15T21:47:29.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:29.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565133+0000", 2024-09-15T21:47:29.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9480000000000002e-06 2024-09-15T21:47:29.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:29.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568006+0000", 2024-09-15T21:47:29.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028730330000000001 2024-09-15T21:47:29.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:29.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568119+0000", 2024-09-15T21:47:29.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011325899999999999 2024-09-15T21:47:29.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:29.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568235+0000", 2024-09-15T21:47:29.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115868 2024-09-15T21:47:29.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:29.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569668+0000", 2024-09-15T21:47:29.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001433099 2024-09-15T21:47:29.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:29.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571926+0000", 2024-09-15T21:47:29.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022572410000000001 2024-09-15T21:47:29.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:29.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571981+0000", 2024-09-15T21:47:29.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5383000000000002e-05 2024-09-15T21:47:29.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:29.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:29.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:29.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572020+0000", 2024-09-15T21:47:29.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8574e-05 2024-09-15T21:47:29.597 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.597 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.597 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.598 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:29.598 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:29.598 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:29.599 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:29.597+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:29.599 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:47:29.700 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:29.698+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:29.713 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:29.714 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:29.714 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:29.714 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:29.723 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:47:29.769 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:47:29.807 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:29.805+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:29.870 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:29.870 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:29.870 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:29.870 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:29.870 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:29.879 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:47:29.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.914+0000 7f37a487b640 1 Processor -- start 2024-09-15T21:47:29.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.914+0000 7f37a487b640 1 -- start start 2024-09-15T21:47:29.917 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:29.914+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:29.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.915+0000 7f37a487b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37a0155440 0x7f37a00cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:29.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.915+0000 7f37a487b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f37a00cc8f0 0x7f37a00cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:29.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.915+0000 7f37a487b640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37a015e7a0 con 0x7f37a00cb7d0 2024-09-15T21:47:29.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.915+0000 7f37a487b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f37a015b470 con 0x7f37a00cc8f0 2024-09-15T21:47:29.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.915+0000 7f37a487b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f37a0155820 con 0x7f37a0155440 2024-09-15T21:47:29.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.915+0000 7f379ed9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f37a00cc8f0 0x7f37a00cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:29.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.915+0000 7f379ed9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f37a00cc8f0 0x7f37a00cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36572/0 (socket says 172.21.15.100:36572) 2024-09-15T21:47:29.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.915+0000 7f379ed9d640 1 -- 172.21.15.100:0/2650855390 learned_addr learned my addr 172.21.15.100:0/2650855390 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:29.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.915+0000 7f379e59c640 1 --2- 172.21.15.100:0/2650855390 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37a0155440 0x7f37a00cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:29.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.915+0000 7f379ed9d640 1 -- 172.21.15.100:0/2650855390 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37a0155440 msgr2=0x7f37a00cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:29.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.915+0000 7f379ed9d640 1 --2- 172.21.15.100:0/2650855390 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37a0155440 0x7f37a00cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:29.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.916+0000 7f379ed9d640 1 -- 172.21.15.100:0/2650855390 >> v1:172.21.15.122:6789/0 conn(0x7f37a00cb7d0 legacy=0x7f37a00cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:29.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.916+0000 7f379ed9d640 1 -- 172.21.15.100:0/2650855390 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a015ea40 con 0x7f37a00cc8f0 2024-09-15T21:47:29.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.917+0000 7f379ed9d640 1 --2- 172.21.15.100:0/2650855390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37a00cc8f0 0x7f37a00cd960 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3798009e50 tx=0x7f379802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a76a3075bba60a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:29.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.917+0000 7f379d59a640 1 -- 172.21.15.100:0/2650855390 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798036070 con 0x7f37a00cc8f0 2024-09-15T21:47:29.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.917+0000 7f379d59a640 1 -- 172.21.15.100:0/2650855390 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3798031600 con 0x7f37a00cc8f0 2024-09-15T21:47:29.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.917+0000 7f379d59a640 1 -- 172.21.15.100:0/2650855390 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37980319d0 con 0x7f37a00cc8f0 2024-09-15T21:47:29.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.918+0000 7f37a487b640 1 -- 172.21.15.100:0/2650855390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37a00cc8f0 msgr2=0x7f37a00cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:29.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.918+0000 7f37a487b640 1 --2- 172.21.15.100:0/2650855390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37a00cc8f0 0x7f37a00cd960 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3798009e50 tx=0x7f379802a710 comp rx=0 tx=0).stop 2024-09-15T21:47:29.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.918+0000 7f37a487b640 1 -- 172.21.15.100:0/2650855390 shutdown_connections 2024-09-15T21:47:29.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.918+0000 7f37a487b640 1 --2- 172.21.15.100:0/2650855390 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37a00cc8f0 0x7f37a00cd960 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:29.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.918+0000 7f37a487b640 1 --2- 172.21.15.100:0/2650855390 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37a0155440 0x7f37a00cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:29.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.918+0000 7f37a487b640 1 -- 172.21.15.100:0/2650855390 >> 172.21.15.100:0/2650855390 conn(0x7f37a00680d0 msgr2=0x7f37a006d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:29.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.918+0000 7f37a487b640 1 -- 172.21.15.100:0/2650855390 shutdown_connections 2024-09-15T21:47:29.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.918+0000 7f37a487b640 1 -- 172.21.15.100:0/2650855390 wait complete. 2024-09-15T21:47:29.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.919+0000 7f37a487b640 1 Processor -- start 2024-09-15T21:47:29.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.919+0000 7f37a487b640 1 -- start start 2024-09-15T21:47:29.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.920+0000 7f37a487b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37a00cb7d0 0x7f37a0149500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:29.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.920+0000 7f37a487b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37a00cc8f0 0x7f37a00cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:29.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.920+0000 7f37a487b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37a0155440 0x7f37a01442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:29.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.920+0000 7f37a487b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f37a015f2e0 con 0x7f37a0155440 2024-09-15T21:47:29.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.920+0000 7f37a487b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f37a015f000 con 0x7f37a00cb7d0 2024-09-15T21:47:29.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.920+0000 7f37a487b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f37a015b070 con 0x7f37a00cc8f0 2024-09-15T21:47:29.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.920+0000 7f379ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37a0155440 0x7f37a01442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:29.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.920+0000 7f379dd9b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37a00cc8f0 0x7f37a00cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:29.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.920+0000 7f379e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37a00cb7d0 0x7f37a0149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:29.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.920+0000 7f379ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37a0155440 0x7f37a01442e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36580/0 (socket says 172.21.15.100:36580) 2024-09-15T21:47:29.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.920+0000 7f379dd9b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37a00cc8f0 0x7f37a00cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:52236/0 (socket says 172.21.15.100:52236) 2024-09-15T21:47:29.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.920+0000 7f379ed9d640 1 -- 172.21.15.100:0/1249376044 learned_addr learned my addr 172.21.15.100:0/1249376044 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:29.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.921+0000 7f379ed9d640 1 -- 172.21.15.100:0/1249376044 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37a00cc8f0 msgr2=0x7f37a00cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:29.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.921+0000 7f379ed9d640 1 --2- 172.21.15.100:0/1249376044 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37a00cc8f0 0x7f37a00cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:29.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.921+0000 7f379ed9d640 1 -- 172.21.15.100:0/1249376044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37a00cb7d0 msgr2=0x7f37a0149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:29.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.921+0000 7f379ed9d640 1 --2- 172.21.15.100:0/1249376044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37a00cb7d0 0x7f37a0149500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:29.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.921+0000 7f379ed9d640 1 -- 172.21.15.100:0/1249376044 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a00c83a0 con 0x7f37a0155440 2024-09-15T21:47:29.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.921+0000 7f379e59c640 1 --2- 172.21.15.100:0/1249376044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37a00cb7d0 0x7f37a0149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:29.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.921+0000 7f379dd9b640 1 --2- 172.21.15.100:0/1249376044 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37a00cc8f0 0x7f37a00cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:29.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.921+0000 7f379ed9d640 1 --2- 172.21.15.100:0/1249376044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37a0155440 0x7f37a01442e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3798009f70 tx=0x7f3798004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:29.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.921+0000 7f378effd640 1 -- 172.21.15.100:0/1249376044 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798041020 con 0x7f37a0155440 2024-09-15T21:47:29.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.921+0000 7f378effd640 1 -- 172.21.15.100:0/1249376044 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37980077a0 con 0x7f37a0155440 2024-09-15T21:47:29.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.921+0000 7f378effd640 1 -- 172.21.15.100:0/1249376044 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3798036040 con 0x7f37a0155440 2024-09-15T21:47:29.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.921+0000 7f37a487b640 1 -- 172.21.15.100:0/1249376044 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a00c7de0 con 0x7f37a0155440 2024-09-15T21:47:29.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.921+0000 7f37a487b640 1 -- 172.21.15.100:0/1249376044 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a01f4bd0 con 0x7f37a0155440 2024-09-15T21:47:29.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.923+0000 7f37a487b640 1 -- 172.21.15.100:0/1249376044 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37a00b3940 con 0x7f37a0155440 2024-09-15T21:47:29.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.923+0000 7f378effd640 1 -- 172.21.15.100:0/1249376044 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3798044020 con 0x7f37a0155440 2024-09-15T21:47:29.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.923+0000 7f378effd640 1 --2- 172.21.15.100:0/1249376044 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f378006c2b0 0x7f378006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:29.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.924+0000 7f378effd640 1 -- 172.21.15.100:0/1249376044 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5854+0+0 (secure 0 0 0) 0x7f37980ac280 con 0x7f37a0155440 2024-09-15T21:47:29.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.924+0000 7f379e59c640 1 --2- 172.21.15.100:0/1249376044 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f378006c2b0 0x7f378006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:29.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.924+0000 7f379e59c640 1 --2- 172.21.15.100:0/1249376044 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f378006c2b0 0x7f378006e770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3790009e90 tx=0x7f3790009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:29.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:29.932+0000 7f378effd640 1 -- 172.21.15.100:0/1249376044 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f379804b020 con 0x7f37a0155440 2024-09-15T21:47:29.995 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:29.996 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:29.996 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:29.996 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:29.996 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:29.996 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:32 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'18, mlcod=17'14)", 2024-09-15T21:47:29.996 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:29.997 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 75.634736719000003, 2024-09-15T21:47:29.997 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052297760000000002, 2024-09-15T21:47:29.997 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:29.997 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:29.997 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:29.998 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:29.998 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:29.998 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:29.998 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:29.998 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:29.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:29.999 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:29.999 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:29.999 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:29.999 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:29.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.000 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.000 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.001 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:30.001 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.06e-07 2024-09-15T21:47:30.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.001 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.001 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.002 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356780+0000", 2024-09-15T21:47:30.002 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4208999999999994e-05 2024-09-15T21:47:30.002 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.002 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.002 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.002 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356789+0000", 2024-09-15T21:47:30.003 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2800000000000003e-06 2024-09-15T21:47:30.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.003 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.003 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.003 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356793+0000", 2024-09-15T21:47:30.004 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:47:30.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.004 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.004 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.004 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356838+0000", 2024-09-15T21:47:30.005 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4851999999999997e-05 2024-09-15T21:47:30.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.005 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.005 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.005 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356854+0000", 2024-09-15T21:47:30.006 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6124999999999999e-05 2024-09-15T21:47:30.006 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.006 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.006 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.006 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.361892+0000", 2024-09-15T21:47:30.007 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050379140000000001 2024-09-15T21:47:30.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.007 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.007 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.007 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.361935+0000", 2024-09-15T21:47:30.007 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3217999999999999e-05 2024-09-15T21:47:30.008 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.008 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.008 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.008 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.008 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.009 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:35 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'21, mlcod=17'15)", 2024-09-15T21:47:30.009 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:30.009 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 75.632699588999998, 2024-09-15T21:47:30.009 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055652039999999998, 2024-09-15T21:47:30.009 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.010 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.010 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.010 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.011 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.011 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.011 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:30.011 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.011 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.012 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.012 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:30.012 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.012 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.012 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.013 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.013 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358744+0000", 2024-09-15T21:47:30.013 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4399999999999998e-07 2024-09-15T21:47:30.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.015 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.016 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.016 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358773+0000", 2024-09-15T21:47:30.016 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8847e-05 2024-09-15T21:47:30.016 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.017 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.017 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.017 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358783+0000", 2024-09-15T21:47:30.017 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0299000000000001e-05 2024-09-15T21:47:30.017 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.018 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.018 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.018 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358796+0000", 2024-09-15T21:47:30.018 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3499000000000001e-05 2024-09-15T21:47:30.018 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.019 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.019 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.019 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358850+0000", 2024-09-15T21:47:30.019 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3952000000000001e-05 2024-09-15T21:47:30.019 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.019 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.020 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.020 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358872+0000", 2024-09-15T21:47:30.020 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1424999999999999e-05 2024-09-15T21:47:30.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.020 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.021 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.021 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364273+0000", 2024-09-15T21:47:30.021 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054011299999999996 2024-09-15T21:47:30.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.021 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.022 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.022 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364308+0000", 2024-09-15T21:47:30.022 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5108000000000003e-05 2024-09-15T21:47:30.022 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.022 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.023 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.023 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.023 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:36 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'22, mlcod=17'15)", 2024-09-15T21:47:30.023 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:30.023 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 75.631905399000004, 2024-09-15T21:47:30.024 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047904530000000001, 2024-09-15T21:47:30.024 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.024 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.024 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.024 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.025 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.025 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.025 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:30.025 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.025 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.026 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.026 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:30.026 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.026 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.027 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.027 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:30.027 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6000000000000004e-07 2024-09-15T21:47:30.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.027 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.027 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.028 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359542+0000", 2024-09-15T21:47:30.028 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3510000000000004e-06 2024-09-15T21:47:30.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.028 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.028 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.029 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359552+0000", 2024-09-15T21:47:30.029 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0263e-05 2024-09-15T21:47:30.029 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.029 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.029 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.029 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359557+0000", 2024-09-15T21:47:30.030 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0479999999999998e-06 2024-09-15T21:47:30.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.030 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.030 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.030 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359593+0000", 2024-09-15T21:47:30.031 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6151e-05 2024-09-15T21:47:30.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.031 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.031 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.031 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359608+0000", 2024-09-15T21:47:30.031 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4609e-05 2024-09-15T21:47:30.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.032 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.032 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.032 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364310+0000", 2024-09-15T21:47:30.032 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047025510000000001 2024-09-15T21:47:30.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.033 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.033 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.033 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364327+0000", 2024-09-15T21:47:30.033 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6920000000000001e-05 2024-09-15T21:47:30.033 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.034 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.034 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.034 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=18'85)", 2024-09-15T21:47:30.035 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:30.035 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 74.093359464000002, 2024-09-15T21:47:30.035 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057443499999999996, 2024-09-15T21:47:30.035 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.035 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.036 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.036 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.036 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.036 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.036 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:30.036 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.037 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.037 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.037 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:30.037 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.038 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.038 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.038 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:30.038 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8899999999999999e-07 2024-09-15T21:47:30.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.039 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.039 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.039 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898167+0000", 2024-09-15T21:47:30.039 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3171999999999996e-05 2024-09-15T21:47:30.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.040 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.040 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.040 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898177+0000", 2024-09-15T21:47:30.040 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0866000000000001e-05 2024-09-15T21:47:30.040 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.041 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.041 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.041 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898185+0000", 2024-09-15T21:47:30.041 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2490000000000004e-06 2024-09-15T21:47:30.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.041 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.042 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.042 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898224+0000", 2024-09-15T21:47:30.042 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9237999999999999e-05 2024-09-15T21:47:30.042 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.042 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.043 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.043 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898238+0000", 2024-09-15T21:47:30.043 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.366e-05 2024-09-15T21:47:30.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.043 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.043 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.044 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.903803+0000", 2024-09-15T21:47:30.044 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005565291 2024-09-15T21:47:30.044 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.044 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.044 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.045 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.903827+0000", 2024-09-15T21:47:30.045 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4284999999999999e-05 2024-09-15T21:47:30.045 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.045 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.045 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.045 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.046 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.046 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:132 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'90, mlcod=18'85)", 2024-09-15T21:47:30.046 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:30.046 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 74.092867740000003, 2024-09-15T21:47:30.046 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055085469999999999, 2024-09-15T21:47:30.047 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.047 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.047 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.047 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.047 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.047 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.048 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:30.048 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.048 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.048 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.049 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:30.049 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.049 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.049 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.049 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.049 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:30.050 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9299999999999999e-07 2024-09-15T21:47:30.050 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.050 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.050 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.050 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898579+0000", 2024-09-15T21:47:30.050 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6650000000000001e-06 2024-09-15T21:47:30.050 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.050 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.050 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.050 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898586+0000", 2024-09-15T21:47:30.050 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8429999999999993e-06 2024-09-15T21:47:30.050 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.050 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.050 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.051 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898592+0000", 2024-09-15T21:47:30.051 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3990000000000003e-06 2024-09-15T21:47:30.051 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.051 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.051 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.051 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898621+0000", 2024-09-15T21:47:30.051 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9019000000000001e-05 2024-09-15T21:47:30.051 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.051 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.051 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.051 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898630+0000", 2024-09-15T21:47:30.051 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5540000000000003e-06 2024-09-15T21:47:30.051 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.052 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.052 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.052 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904047+0000", 2024-09-15T21:47:30.052 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054167169999999997 2024-09-15T21:47:30.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.052 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.052 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.052 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904083+0000", 2024-09-15T21:47:30.052 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5957000000000001e-05 2024-09-15T21:47:30.052 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.052 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.052 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.052 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.053 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-15T21:47:30.053 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:30.053 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 74.092278590999996, 2024-09-15T21:47:30.053 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049424780000000001, 2024-09-15T21:47:30.053 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.053 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.053 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.053 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.053 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.053 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.053 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:30.053 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.053 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.053 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.054 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.054 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:30.054 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.054 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.054 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.054 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.054 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899166+0000", 2024-09-15T21:47:30.054 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0779999999999998e-06 2024-09-15T21:47:30.054 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.054 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.054 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.054 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899174+0000", 2024-09-15T21:47:30.054 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.729e-06 2024-09-15T21:47:30.054 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.055 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.055 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.055 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899181+0000", 2024-09-15T21:47:30.055 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4080000000000001e-06 2024-09-15T21:47:30.055 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.055 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.055 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.055 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899185+0000", 2024-09-15T21:47:30.055 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0840000000000002e-06 2024-09-15T21:47:30.055 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.055 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.055 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.055 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899240+0000", 2024-09-15T21:47:30.056 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4874999999999999e-05 2024-09-15T21:47:30.056 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.056 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.056 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.056 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899257+0000", 2024-09-15T21:47:30.056 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7549000000000001e-05 2024-09-15T21:47:30.056 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.056 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.056 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.056 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904085+0000", 2024-09-15T21:47:30.056 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048272640000000004 2024-09-15T21:47:30.056 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.056 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.056 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.057 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904106+0000", 2024-09-15T21:47:30.057 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1491000000000001e-05 2024-09-15T21:47:30.057 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.057 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.057 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.057 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.057 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.057 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:30.057 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:30.057 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 61.578055315, 2024-09-15T21:47:30.057 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046608459999999997, 2024-09-15T21:47:30.057 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.057 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.058 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.058 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.058 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.058 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.058 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:30.058 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.058 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.058 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.058 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:30.058 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.058 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.059 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.059 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413391+0000", 2024-09-15T21:47:30.059 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9899999999999999e-06 2024-09-15T21:47:30.059 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.059 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.059 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.059 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413399+0000", 2024-09-15T21:47:30.059 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6310000000000003e-06 2024-09-15T21:47:30.059 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.059 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.059 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.059 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413444+0000", 2024-09-15T21:47:30.059 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5052000000000002e-05 2024-09-15T21:47:30.059 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.060 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.060 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.060 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413453+0000", 2024-09-15T21:47:30.060 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8869999999999998e-06 2024-09-15T21:47:30.060 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.060 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.060 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.060 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413576+0000", 2024-09-15T21:47:30.060 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012358700000000001 2024-09-15T21:47:30.060 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.060 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.060 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.060 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413679+0000", 2024-09-15T21:47:30.061 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000103318 2024-09-15T21:47:30.061 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.061 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.061 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:30.061 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413785+0000", 2024-09-15T21:47:30.061 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010561 2024-09-15T21:47:30.061 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.061 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.061 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.061 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416276+0000", 2024-09-15T21:47:30.061 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002490754 2024-09-15T21:47:30.061 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.061 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.061 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.062 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417949+0000", 2024-09-15T21:47:30.062 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016733519999999999 2024-09-15T21:47:30.062 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.062 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.062 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.062 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418005+0000", 2024-09-15T21:47:30.062 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5747e-05 2024-09-15T21:47:30.062 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.062 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.062 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.062 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418048+0000", 2024-09-15T21:47:30.062 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2917999999999998e-05 2024-09-15T21:47:30.062 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.063 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.063 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.063 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.063 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:30.063 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:30.063 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 61.578040844999997, 2024-09-15T21:47:30.063 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051419999999999999, 2024-09-15T21:47:30.063 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.063 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.063 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.063 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.063 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.063 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.064 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:30.064 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.064 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.064 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.064 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.064 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:30.064 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.064 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.064 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.064 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.064 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413402+0000", 2024-09-15T21:47:30.064 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6899999999999998e-07 2024-09-15T21:47:30.064 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.064 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.065 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.065 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413405+0000", 2024-09-15T21:47:30.065 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8109999999999999e-06 2024-09-15T21:47:30.065 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.065 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.065 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.065 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413517+0000", 2024-09-15T21:47:30.065 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000112592 2024-09-15T21:47:30.065 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.065 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.065 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.065 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413527+0000", 2024-09-15T21:47:30.065 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0144e-05 2024-09-15T21:47:30.065 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.066 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.066 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.066 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414155+0000", 2024-09-15T21:47:30.066 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00062767200000000002 2024-09-15T21:47:30.066 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.066 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.066 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.066 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414239+0000", 2024-09-15T21:47:30.066 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3607000000000001e-05 2024-09-15T21:47:30.066 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.066 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.066 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:30.066 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414368+0000", 2024-09-15T21:47:30.066 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000129049 2024-09-15T21:47:30.067 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.067 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.067 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.067 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416332+0000", 2024-09-15T21:47:30.067 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019645109999999999 2024-09-15T21:47:30.067 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.067 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.067 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.067 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418446+0000", 2024-09-15T21:47:30.067 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021137640000000002 2024-09-15T21:47:30.067 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.067 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.067 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.067 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418503+0000", 2024-09-15T21:47:30.068 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7166999999999997e-05 2024-09-15T21:47:30.068 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.068 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.068 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.068 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418543+0000", 2024-09-15T21:47:30.068 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0314000000000001e-05 2024-09-15T21:47:30.068 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.068 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.068 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.068 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.068 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.068 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:30.068 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:30.068 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 61.578027654000003, 2024-09-15T21:47:30.069 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057945979999999998, 2024-09-15T21:47:30.069 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.069 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.069 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.069 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.069 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.069 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.069 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:30.069 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.069 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.069 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.069 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.069 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:30.069 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.070 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.070 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.070 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.070 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:30.070 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9499999999999998e-07 2024-09-15T21:47:30.070 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.070 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.070 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.070 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413417+0000", 2024-09-15T21:47:30.070 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3910000000000001e-06 2024-09-15T21:47:30.070 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.070 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.070 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.070 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414570+0000", 2024-09-15T21:47:30.071 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011525890000000001 2024-09-15T21:47:30.071 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.071 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.071 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.071 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414577+0000", 2024-09-15T21:47:30.071 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1840000000000002e-06 2024-09-15T21:47:30.071 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.071 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.071 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.071 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414733+0000", 2024-09-15T21:47:30.071 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015563900000000001 2024-09-15T21:47:30.071 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.071 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.071 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.072 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414831+0000", 2024-09-15T21:47:30.072 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8138000000000001e-05 2024-09-15T21:47:30.072 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.072 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.072 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:30.072 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414956+0000", 2024-09-15T21:47:30.072 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000124604 2024-09-15T21:47:30.072 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.072 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.072 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.072 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418804+0000", 2024-09-15T21:47:30.072 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038484299999999999 2024-09-15T21:47:30.072 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.073 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.073 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.073 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419124+0000", 2024-09-15T21:47:30.073 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00032017699999999999 2024-09-15T21:47:30.073 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.073 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.073 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.073 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419171+0000", 2024-09-15T21:47:30.073 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6520000000000002e-05 2024-09-15T21:47:30.073 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.073 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.073 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.073 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419209+0000", 2024-09-15T21:47:30.074 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8531000000000001e-05 2024-09-15T21:47:30.074 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.074 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.074 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.074 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.074 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.074 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:30.074 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:30.074 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 61.578020557999999, 2024-09-15T21:47:30.074 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059183300000000003, 2024-09-15T21:47:30.074 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.074 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.074 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:30.074 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.075 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.075 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.075 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:30.075 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.075 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.075 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.075 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.075 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:30.075 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.075 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.075 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.075 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.075 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:30.075 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3000000000000002e-07 2024-09-15T21:47:30.076 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.076 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.076 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.076 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413426+0000", 2024-09-15T21:47:30.076 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7110000000000001e-06 2024-09-15T21:47:30.076 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.076 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.076 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.076 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414621+0000", 2024-09-15T21:47:30.076 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001195022 2024-09-15T21:47:30.076 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.076 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.076 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.077 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414625+0000", 2024-09-15T21:47:30.077 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6360000000000002e-06 2024-09-15T21:47:30.077 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.077 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.077 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.077 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415269+0000", 2024-09-15T21:47:30.077 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00064315399999999995 2024-09-15T21:47:30.077 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.077 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.077 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.077 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415356+0000", 2024-09-15T21:47:30.077 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7410999999999994e-05 2024-09-15T21:47:30.077 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.077 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.078 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.078 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419340+0000", 2024-09-15T21:47:30.078 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0039840659999999996 2024-09-15T21:47:30.078 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.078 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.078 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.078 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.078 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.078 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-15T21:47:30.078 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:30.078 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 61.577712966999997, 2024-09-15T21:47:30.078 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053379719999999999, 2024-09-15T21:47:30.078 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.078 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.079 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.079 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.079 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.079 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.079 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:30.079 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.079 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.079 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.079 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.079 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:30.079 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.079 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.079 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.079 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.080 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413730+0000", 2024-09-15T21:47:30.080 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1379999999999999e-06 2024-09-15T21:47:30.080 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.080 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.080 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.080 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413740+0000", 2024-09-15T21:47:30.080 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4830000000000005e-06 2024-09-15T21:47:30.080 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.080 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.080 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.080 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413751+0000", 2024-09-15T21:47:30.080 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1228e-05 2024-09-15T21:47:30.080 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.080 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.081 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.081 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413758+0000", 2024-09-15T21:47:30.081 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1289999999999998e-06 2024-09-15T21:47:30.081 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.081 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.081 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.081 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414038+0000", 2024-09-15T21:47:30.081 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00027935999999999998 2024-09-15T21:47:30.081 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.081 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.081 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.081 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414055+0000", 2024-09-15T21:47:30.081 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7325000000000001e-05 2024-09-15T21:47:30.081 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.082 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.082 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.082 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419028+0000", 2024-09-15T21:47:30.082 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049730720000000003 2024-09-15T21:47:30.082 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.082 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.082 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.082 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419067+0000", 2024-09-15T21:47:30.082 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9236999999999997e-05 2024-09-15T21:47:30.082 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.082 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.082 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.082 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.082 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.083 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.083 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:30.083 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 55.535789700000002, 2024-09-15T21:47:30.083 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.92695576499999999, 2024-09-15T21:47:30.083 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.083 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.083 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.083 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.083 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.083 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.083 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:30.083 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.083 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.083 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.084 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.084 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:30.084 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.084 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.084 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.084 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.084 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:30.084 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8500000000000002e-07 2024-09-15T21:47:30.084 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.084 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.084 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.084 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455655+0000", 2024-09-15T21:47:30.084 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.401e-06 2024-09-15T21:47:30.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.085 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.085 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.085 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.380965+0000", 2024-09-15T21:47:30.085 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.92530946000000003 2024-09-15T21:47:30.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.085 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.085 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.085 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.380981+0000", 2024-09-15T21:47:30.085 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6209e-05 2024-09-15T21:47:30.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.085 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.085 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.085 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381086+0000", 2024-09-15T21:47:30.086 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105011 2024-09-15T21:47:30.086 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.086 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.086 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.086 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381154+0000", 2024-09-15T21:47:30.086 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7596999999999997e-05 2024-09-15T21:47:30.086 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.086 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.086 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:30.086 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381249+0000", 2024-09-15T21:47:30.086 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5354999999999998e-05 2024-09-15T21:47:30.086 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.086 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.086 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.087 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381844+0000", 2024-09-15T21:47:30.087 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00059518600000000004 2024-09-15T21:47:30.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.087 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.087 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.087 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382531+0000", 2024-09-15T21:47:30.087 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00068671499999999996 2024-09-15T21:47:30.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.087 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.087 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.087 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382570+0000", 2024-09-15T21:47:30.087 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9373999999999999e-05 2024-09-15T21:47:30.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.088 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.088 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.088 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382608+0000", 2024-09-15T21:47:30.088 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8071999999999997e-05 2024-09-15T21:47:30.088 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.088 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.088 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.088 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.088 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.088 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:30.088 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 49.515880441, 2024-09-15T21:47:30.088 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047117319999999997, 2024-09-15T21:47:30.088 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.089 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.089 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.089 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.089 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.089 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.089 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:30.089 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.089 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.089 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.089 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.089 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:30.089 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.089 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.089 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.090 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.090 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:30.090 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4700000000000001e-07 2024-09-15T21:47:30.090 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.090 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.090 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.090 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475566+0000", 2024-09-15T21:47:30.090 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1669999999999999e-06 2024-09-15T21:47:30.090 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.090 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.090 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.090 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476316+0000", 2024-09-15T21:47:30.090 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075022999999999995 2024-09-15T21:47:30.090 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.091 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.091 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.091 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476324+0000", 2024-09-15T21:47:30.091 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.259e-06 2024-09-15T21:47:30.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.091 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.091 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.091 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476373+0000", 2024-09-15T21:47:30.091 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8807999999999998e-05 2024-09-15T21:47:30.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.091 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.091 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.091 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476446+0000", 2024-09-15T21:47:30.091 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3138000000000003e-05 2024-09-15T21:47:30.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.092 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.092 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:30.092 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476593+0000", 2024-09-15T21:47:30.092 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00014680199999999999 2024-09-15T21:47:30.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.092 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.092 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.092 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480189+0000", 2024-09-15T21:47:30.092 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035961370000000001 2024-09-15T21:47:30.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.092 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.092 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.093 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480198+0000", 2024-09-15T21:47:30.093 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.6649999999999992e-06 2024-09-15T21:47:30.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.093 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.093 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.093 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480235+0000", 2024-09-15T21:47:30.093 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6943999999999997e-05 2024-09-15T21:47:30.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.093 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.093 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.093 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480274+0000", 2024-09-15T21:47:30.093 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9035000000000001e-05 2024-09-15T21:47:30.093 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.093 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.094 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.094 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.094 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.094 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:30.094 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 49.515875438000002, 2024-09-15T21:47:30.094 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046830109999999999, 2024-09-15T21:47:30.094 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.094 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.094 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.094 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.094 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.094 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.094 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:30.095 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.095 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.095 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.095 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.095 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:30.095 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.095 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.095 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.095 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.095 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:30.095 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.53e-07 2024-09-15T21:47:30.095 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.095 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.095 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.096 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475570+0000", 2024-09-15T21:47:30.096 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7470000000000001e-06 2024-09-15T21:47:30.096 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.096 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.096 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.096 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476353+0000", 2024-09-15T21:47:30.096 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00078312300000000004 2024-09-15T21:47:30.096 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.096 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.096 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.096 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476359+0000", 2024-09-15T21:47:30.096 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.117e-06 2024-09-15T21:47:30.096 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.097 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.097 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.097 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476412+0000", 2024-09-15T21:47:30.097 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2398999999999998e-05 2024-09-15T21:47:30.097 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.097 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.097 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.097 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476524+0000", 2024-09-15T21:47:30.097 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011293 2024-09-15T21:47:30.097 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.097 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.097 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:30.097 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476909+0000", 2024-09-15T21:47:30.097 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00038490099999999999 2024-09-15T21:47:30.098 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.098 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.098 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.098 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479761+0000", 2024-09-15T21:47:30.098 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002851737 2024-09-15T21:47:30.098 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.098 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.098 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.098 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480154+0000", 2024-09-15T21:47:30.098 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00039307399999999998 2024-09-15T21:47:30.098 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.098 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.098 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.098 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480203+0000", 2024-09-15T21:47:30.098 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8418000000000003e-05 2024-09-15T21:47:30.099 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.099 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.099 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.099 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480250+0000", 2024-09-15T21:47:30.099 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7311999999999997e-05 2024-09-15T21:47:30.099 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.099 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.099 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.099 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.099 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.099 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.099 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:30.099 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 49.515871275999999, 2024-09-15T21:47:30.100 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005035398, 2024-09-15T21:47:30.100 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.100 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.100 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.100 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.100 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.100 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.100 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:30.100 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.100 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.100 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.100 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.100 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:30.100 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.101 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.101 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.101 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.101 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:30.101 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.22e-07 2024-09-15T21:47:30.101 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.101 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.101 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.101 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475574+0000", 2024-09-15T21:47:30.101 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.593e-06 2024-09-15T21:47:30.101 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.101 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.101 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.101 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476579+0000", 2024-09-15T21:47:30.102 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010053989999999999 2024-09-15T21:47:30.102 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.102 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.102 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.102 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476588+0000", 2024-09-15T21:47:30.102 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4530000000000002e-06 2024-09-15T21:47:30.102 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.102 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.102 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.102 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477723+0000", 2024-09-15T21:47:30.102 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001134962 2024-09-15T21:47:30.102 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.102 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.103 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.103 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477801+0000", 2024-09-15T21:47:30.103 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8228999999999999e-05 2024-09-15T21:47:30.103 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.103 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.103 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:30.103 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477946+0000", 2024-09-15T21:47:30.103 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00014553400000000001 2024-09-15T21:47:30.103 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.103 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.103 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.103 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480258+0000", 2024-09-15T21:47:30.103 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023112039999999999 2024-09-15T21:47:30.103 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.104 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.104 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.104 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480504+0000", 2024-09-15T21:47:30.104 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024667500000000002 2024-09-15T21:47:30.104 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.104 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.104 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.104 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480556+0000", 2024-09-15T21:47:30.104 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2114999999999998e-05 2024-09-15T21:47:30.104 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.104 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.104 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.104 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480606+0000", 2024-09-15T21:47:30.104 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0012000000000002e-05 2024-09-15T21:47:30.105 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.105 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.105 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.105 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.105 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.105 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.105 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:30.105 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 49.515867346, 2024-09-15T21:47:30.105 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050672490000000002, 2024-09-15T21:47:30.105 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.105 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.105 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.105 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.105 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.106 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.106 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:30.106 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.106 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.106 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.106 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:30.106 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.106 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.106 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.106 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:30.106 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:30.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.107 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.107 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.107 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475577+0000", 2024-09-15T21:47:30.107 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2290000000000002e-06 2024-09-15T21:47:30.107 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.107 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.107 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.107 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476605+0000", 2024-09-15T21:47:30.107 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010278780000000001 2024-09-15T21:47:30.107 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.107 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.107 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.107 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476611+0000", 2024-09-15T21:47:30.107 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5600000000000001e-06 2024-09-15T21:47:30.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.108 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.108 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.108 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477053+0000", 2024-09-15T21:47:30.108 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044162800000000001 2024-09-15T21:47:30.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.108 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.108 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.108 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477180+0000", 2024-09-15T21:47:30.108 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000127129 2024-09-15T21:47:30.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.108 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.108 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:30.108 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477288+0000", 2024-09-15T21:47:30.109 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010789999999999999 2024-09-15T21:47:30.109 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.109 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.109 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.109 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480288+0000", 2024-09-15T21:47:30.109 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003000052 2024-09-15T21:47:30.109 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.109 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.109 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.109 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480568+0000", 2024-09-15T21:47:30.109 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028026899999999998 2024-09-15T21:47:30.109 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.109 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.109 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.110 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480609+0000", 2024-09-15T21:47:30.110 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1437999999999999e-05 2024-09-15T21:47:30.110 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.110 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.110 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.110 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480642+0000", 2024-09-15T21:47:30.110 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2879999999999997e-05 2024-09-15T21:47:30.110 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.110 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.110 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.110 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.110 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.110 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:20 2.1f 2:fdd45aa0:::61c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.110 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:30.111 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 49.515863674999999, 2024-09-15T21:47:30.111 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053112589999999996, 2024-09-15T21:47:30.111 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.111 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.111 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.111 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.111 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.111 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.111 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:30.111 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.111 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.111 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.111 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.111 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:30.111 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.112 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.112 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.112 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.112 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:30.112 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:30.112 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.112 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.112 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.112 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475581+0000", 2024-09-15T21:47:30.112 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1009999999999999e-06 2024-09-15T21:47:30.112 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.112 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.112 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.113 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476628+0000", 2024-09-15T21:47:30.113 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010465380000000001 2024-09-15T21:47:30.113 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.113 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.113 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.113 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476633+0000", 2024-09-15T21:47:30.113 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9869999999999999e-06 2024-09-15T21:47:30.113 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.113 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.113 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.113 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477618+0000", 2024-09-15T21:47:30.113 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00098539299999999999 2024-09-15T21:47:30.113 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.113 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.114 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.114 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477696+0000", 2024-09-15T21:47:30.114 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8509000000000003e-05 2024-09-15T21:47:30.114 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.114 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.114 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:30.114 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477824+0000", 2024-09-15T21:47:30.114 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000127062 2024-09-15T21:47:30.114 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.114 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.114 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.114 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480294+0000", 2024-09-15T21:47:30.114 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024703939999999999 2024-09-15T21:47:30.114 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.115 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.115 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.115 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480788+0000", 2024-09-15T21:47:30.115 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00049443800000000004 2024-09-15T21:47:30.115 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.115 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.115 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.115 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480845+0000", 2024-09-15T21:47:30.115 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6762000000000003e-05 2024-09-15T21:47:30.115 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.115 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.115 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.115 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480890+0000", 2024-09-15T21:47:30.115 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4789000000000001e-05 2024-09-15T21:47:30.116 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.116 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.116 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.116 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.116 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.116 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.116 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:30.116 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 49.515860203999999, 2024-09-15T21:47:30.116 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007191546, 2024-09-15T21:47:30.116 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.116 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.116 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:30.116 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.116 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.117 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.117 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:30.117 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.117 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.117 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.117 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.117 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:30.117 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.117 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.117 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.117 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.117 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:30.117 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9999999999999999e-07 2024-09-15T21:47:30.117 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.118 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.118 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.118 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475585+0000", 2024-09-15T21:47:30.118 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.384e-06 2024-09-15T21:47:30.118 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.118 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.118 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.118 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476665+0000", 2024-09-15T21:47:30.118 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010797140000000001 2024-09-15T21:47:30.118 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.118 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.118 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.118 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476671+0000", 2024-09-15T21:47:30.118 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3940000000000001e-06 2024-09-15T21:47:30.119 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.119 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.119 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.119 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478048+0000", 2024-09-15T21:47:30.119 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013767530000000001 2024-09-15T21:47:30.119 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.119 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.119 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.119 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478177+0000", 2024-09-15T21:47:30.119 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012898099999999999 2024-09-15T21:47:30.119 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.119 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.119 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.119 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482774+0000", 2024-09-15T21:47:30.120 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045970200000000003 2024-09-15T21:47:30.120 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.120 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.120 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.120 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.120 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.120 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.120 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:30.120 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 44.446630538999997, 2024-09-15T21:47:30.120 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055444480000000004, 2024-09-15T21:47:30.120 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.120 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.120 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.121 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.121 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.121 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.121 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:30.121 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.121 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.121 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.121 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.121 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:30.121 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.121 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.121 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.121 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.121 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:30.121 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4200000000000002e-07 2024-09-15T21:47:30.122 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.122 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.122 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.122 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544814+0000", 2024-09-15T21:47:30.122 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9980000000000002e-06 2024-09-15T21:47:30.122 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.122 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.122 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.122 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544872+0000", 2024-09-15T21:47:30.122 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8405000000000001e-05 2024-09-15T21:47:30.122 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.122 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.122 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.122 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544877+0000", 2024-09-15T21:47:30.123 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.143e-06 2024-09-15T21:47:30.123 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.123 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.123 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.123 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545750+0000", 2024-09-15T21:47:30.123 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00087310000000000003 2024-09-15T21:47:30.123 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.123 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.123 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.123 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545850+0000", 2024-09-15T21:47:30.123 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9939999999999995e-05 2024-09-15T21:47:30.123 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.123 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.123 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:30.124 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546014+0000", 2024-09-15T21:47:30.124 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00016408500000000001 2024-09-15T21:47:30.124 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.124 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.124 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.124 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.548748+0000", 2024-09-15T21:47:30.124 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027339539999999998 2024-09-15T21:47:30.124 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.124 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.124 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.124 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550265+0000", 2024-09-15T21:47:30.124 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015170019999999999 2024-09-15T21:47:30.124 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.124 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.125 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.125 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550315+0000", 2024-09-15T21:47:30.125 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9867999999999998e-05 2024-09-15T21:47:30.125 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.125 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.125 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.125 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550356+0000", 2024-09-15T21:47:30.125 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1711000000000001e-05 2024-09-15T21:47:30.125 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.125 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.125 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.125 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.125 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.125 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.126 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:30.126 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 44.446627337999999, 2024-09-15T21:47:30.126 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058729619999999998, 2024-09-15T21:47:30.126 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.126 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.126 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.126 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.126 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.126 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.126 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:30.126 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.126 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.126 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.126 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.126 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:30.127 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.127 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.127 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.127 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.127 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:30.127 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0599999999999999e-07 2024-09-15T21:47:30.127 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.127 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.127 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.127 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544821+0000", 2024-09-15T21:47:30.127 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4079999999999997e-06 2024-09-15T21:47:30.127 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.127 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.128 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.128 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544939+0000", 2024-09-15T21:47:30.128 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118532 2024-09-15T21:47:30.128 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.128 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.128 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.128 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544946+0000", 2024-09-15T21:47:30.128 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9260000000000003e-06 2024-09-15T21:47:30.128 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.128 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.128 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.128 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545011+0000", 2024-09-15T21:47:30.128 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4572000000000005e-05 2024-09-15T21:47:30.128 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.129 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.129 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.129 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545134+0000", 2024-09-15T21:47:30.129 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012309899999999999 2024-09-15T21:47:30.129 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.129 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.129 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:30.129 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545271+0000", 2024-09-15T21:47:30.129 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013746599999999999 2024-09-15T21:47:30.129 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.129 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.129 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.129 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550243+0000", 2024-09-15T21:47:30.129 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049714529999999998 2024-09-15T21:47:30.130 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.130 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.130 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.130 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550605+0000", 2024-09-15T21:47:30.130 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00036260699999999998 2024-09-15T21:47:30.130 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.130 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.130 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.130 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550662+0000", 2024-09-15T21:47:30.130 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6573000000000002e-05 2024-09-15T21:47:30.130 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.130 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.130 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.130 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550688+0000", 2024-09-15T21:47:30.131 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6120000000000001e-05 2024-09-15T21:47:30.131 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.131 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.131 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.131 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.131 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.131 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:30.133 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:47:30.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.115+0000 7f37a487b640 1 -- 172.21.15.100:0/1249376044 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f37a014f4c0 con 0x7f37a0155440 2024-09-15T21:47:30.133 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:30.130+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:30.233 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:30.231+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:30.283 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:30.284 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:30.284 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:30.284 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:30.293 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:47:30.309 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:47:30.330 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-15T21:47:30.331 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:47:30.331 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-15T21:47:30.331 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-15T21:47:30.331 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-15T21:47:30.332 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-15T21:47:30.334 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:30.332+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:30.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.431+0000 7fe777aba640 1 Processor -- start 2024-09-15T21:47:30.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.431+0000 7fe777aba640 1 -- start start 2024-09-15T21:47:30.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.432+0000 7fe777aba640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe770155670 0x7fe77015e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:30.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.432+0000 7fe777aba640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe77015bb60 0x7fe77006d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:30.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.432+0000 7fe777aba640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe77015edb0 con 0x7fe770155290 2024-09-15T21:47:30.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.432+0000 7fe777aba640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fe77015f090 con 0x7fe770155670 2024-09-15T21:47:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.432+0000 7fe777aba640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fe77015f370 con 0x7fe77015bb60 2024-09-15T21:47:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.432+0000 7fe775055640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe770155670 0x7fe77015e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.432+0000 7fe775055640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe770155670 0x7fe77015e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36584/0 (socket says 172.21.15.100:36584) 2024-09-15T21:47:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.432+0000 7fe775055640 1 -- 172.21.15.100:0/684465547 learned_addr learned my addr 172.21.15.100:0/684465547 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.432+0000 7fe776057640 1 --2- 172.21.15.100:0/684465547 >> v2:172.21.15.122:3300/0 conn(0x7fe77015bb60 0x7fe77006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.433+0000 7fe775055640 1 -- 172.21.15.100:0/684465547 >> v2:172.21.15.122:3300/0 conn(0x7fe77015bb60 msgr2=0x7fe77006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.433+0000 7fe775055640 1 --2- 172.21.15.100:0/684465547 >> v2:172.21.15.122:3300/0 conn(0x7fe77015bb60 0x7fe77006d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.433+0000 7fe775055640 1 -- 172.21.15.100:0/684465547 >> v1:172.21.15.122:6789/0 conn(0x7fe770155290 legacy=0x7fe7700c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:30.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.433+0000 7fe775055640 1 -- 172.21.15.100:0/684465547 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7700c5db0 con 0x7fe770155670 2024-09-15T21:47:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.433+0000 7fe775055640 1 --2- 172.21.15.100:0/684465547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe770155670 0x7fe77015e690 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe7700cde70 tx=0x7fe76002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e443a9c59b72dfde server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.434+0000 7fe774854640 1 -- 172.21.15.100:0/684465547 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760037070 con 0x7fe770155670 2024-09-15T21:47:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.434+0000 7fe774854640 1 -- 172.21.15.100:0/684465547 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe760032440 con 0x7fe770155670 2024-09-15T21:47:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.434+0000 7fe774854640 1 -- 172.21.15.100:0/684465547 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760032810 con 0x7fe770155670 2024-09-15T21:47:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.434+0000 7fe777aba640 1 -- 172.21.15.100:0/684465547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe770155670 msgr2=0x7fe77015e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.434+0000 7fe777aba640 1 --2- 172.21.15.100:0/684465547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe770155670 0x7fe77015e690 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe7700cde70 tx=0x7fe76002a270 comp rx=0 tx=0).stop 2024-09-15T21:47:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.434+0000 7fe777aba640 1 -- 172.21.15.100:0/684465547 shutdown_connections 2024-09-15T21:47:30.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.434+0000 7fe777aba640 1 --2- 172.21.15.100:0/684465547 >> v2:172.21.15.122:3300/0 conn(0x7fe77015bb60 0x7fe77006d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:30.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.434+0000 7fe777aba640 1 --2- 172.21.15.100:0/684465547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe770155670 0x7fe77015e690 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:30.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.434+0000 7fe777aba640 1 -- 172.21.15.100:0/684465547 >> 172.21.15.100:0/684465547 conn(0x7fe770073310 msgr2=0x7fe770073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:30.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.434+0000 7fe777aba640 1 -- 172.21.15.100:0/684465547 shutdown_connections 2024-09-15T21:47:30.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.434+0000 7fe777aba640 1 -- 172.21.15.100:0/684465547 wait complete. 2024-09-15T21:47:30.439 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:30.432+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:30.440 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:30.440 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-15T21:47:30.440 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:30.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.435+0000 7fe777aba640 1 Processor -- start 2024-09-15T21:47:30.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.435+0000 7fe777aba640 1 -- start start 2024-09-15T21:47:30.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.436+0000 7fe777aba640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe770155290 0x7fe77014c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:30.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.436+0000 7fe777aba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe770155670 0x7fe770148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:30.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.436+0000 7fe777aba640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77015bb60 0x7fe7700cca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:30.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.436+0000 7fe777aba640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe77015c590 con 0x7fe77015bb60 2024-09-15T21:47:30.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.436+0000 7fe777aba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe7700b39e0 con 0x7fe770155670 2024-09-15T21:47:30.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.436+0000 7fe777aba640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe7700c6cc0 con 0x7fe770155290 2024-09-15T21:47:30.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.436+0000 7fe776057640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77015bb60 0x7fe7700cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:30.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.436+0000 7fe776057640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77015bb60 0x7fe7700cca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36586/0 (socket says 172.21.15.100:36586) 2024-09-15T21:47:30.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.436+0000 7fe776057640 1 -- 172.21.15.100:0/1323454313 learned_addr learned my addr 172.21.15.100:0/1323454313 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:30.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.436+0000 7fe775856640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe770155290 0x7fe77014c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:30.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.436+0000 7fe775055640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe770155670 0x7fe770148bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:30.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.437+0000 7fe776057640 1 -- 172.21.15.100:0/1323454313 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe770155290 msgr2=0x7fe77014c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:30.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.437+0000 7fe776057640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe770155290 0x7fe77014c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:30.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.437+0000 7fe776057640 1 -- 172.21.15.100:0/1323454313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe770155670 msgr2=0x7fe770148bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:30.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.437+0000 7fe776057640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe770155670 0x7fe770148bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:30.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.437+0000 7fe776057640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe770149770 con 0x7fe77015bb60 2024-09-15T21:47:30.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.437+0000 7fe776057640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe77015bb60 0x7fe7700cca90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe76800efc0 tx=0x7fe76800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:30.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.437+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768018020 con 0x7fe77015bb60 2024-09-15T21:47:30.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.437+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe768014480 con 0x7fe77015bb60 2024-09-15T21:47:30.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.437+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768014870 con 0x7fe77015bb60 2024-09-15T21:47:30.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.437+0000 7fe777aba640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe770149a50 con 0x7fe77015bb60 2024-09-15T21:47:30.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.437+0000 7fe777aba640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe77014b6b0 con 0x7fe77015bb60 2024-09-15T21:47:30.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.438+0000 7fe777aba640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fe77015bf70 con 0x7fe77015bb60 2024-09-15T21:47:30.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.439+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe768005d10 con 0x7fe77015bb60 2024-09-15T21:47:30.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.439+0000 7fe7667fc640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe75406c2b0 0x7fe75406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:30.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.439+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5854+0+0 (secure 0 0 0) 0x7fe76808be20 con 0x7fe77015bb60 2024-09-15T21:47:30.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.439+0000 7fe775856640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe75406c2b0 0x7fe75406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:30.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.439+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v35) ==== 54+0+16 (secure 0 0 0) 0x7fe768022070 con 0x7fe77015bb60 2024-09-15T21:47:30.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.440+0000 7fe777aba640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fe7700ccfd0 0x7fe770152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:30.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.440+0000 7fe777aba640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe770149d30 con 0x7fe7700ccfd0 2024-09-15T21:47:30.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.440+0000 7fe775856640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe75406c2b0 0x7fe75406e770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe76c000f60 tx=0x7fe76c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:30.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.440+0000 7fe775055640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fe7700ccfd0 0x7fe770152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:30.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.440+0000 7fe775055640 1 -- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fe7700ccfd0 msgr2=0x7fe770152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send send error: (32) Broken pipe 2024-09-15T21:47:30.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.440+0000 7fe775055640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fe7700ccfd0 0x7fe770152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).write auth request write failed r=-32 ((32) Broken pipe) 2024-09-15T21:47:30.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.440+0000 7fe775055640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fe7700ccfd0 0x7fe770152c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-15T21:47:30.447 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:30.447 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:30.447 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:30.447 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:30.447 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:30.453 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:47:30.536 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:30.533+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:30.559 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:30.559 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:30.559 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:30.559 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:30.560 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.560 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:30.560 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:30.560 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 62.143866223000003, 2024-09-15T21:47:30.560 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006101291, 2024-09-15T21:47:30.560 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.560 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.560 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.560 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.560 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.560 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.560 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:30.561 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.561 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.561 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.561 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.561 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:30.561 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.561 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.561 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.561 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.561 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413116+0000", 2024-09-15T21:47:30.561 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9700000000000003e-07 2024-09-15T21:47:30.561 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.561 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.561 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.562 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413118+0000", 2024-09-15T21:47:30.562 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2910000000000002e-06 2024-09-15T21:47:30.562 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.562 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.562 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.562 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413220+0000", 2024-09-15T21:47:30.562 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102056 2024-09-15T21:47:30.562 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.562 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.562 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.562 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413225+0000", 2024-09-15T21:47:30.562 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0490000000000003e-06 2024-09-15T21:47:30.562 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.563 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.563 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.563 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414808+0000", 2024-09-15T21:47:30.563 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015826900000000001 2024-09-15T21:47:30.563 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.563 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.563 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.563 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414907+0000", 2024-09-15T21:47:30.563 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9600999999999997e-05 2024-09-15T21:47:30.563 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.563 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.563 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:30.563 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415054+0000", 2024-09-15T21:47:30.563 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000146532 2024-09-15T21:47:30.564 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.564 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.564 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.564 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418056+0000", 2024-09-15T21:47:30.564 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030020519999999998 2024-09-15T21:47:30.564 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.564 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.564 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.564 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419128+0000", 2024-09-15T21:47:30.564 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001071671 2024-09-15T21:47:30.564 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.564 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.564 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.564 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419184+0000", 2024-09-15T21:47:30.565 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6125000000000002e-05 2024-09-15T21:47:30.565 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.565 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.565 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.565 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419217+0000", 2024-09-15T21:47:30.565 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2926999999999998e-05 2024-09-15T21:47:30.565 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.565 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.565 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.565 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.566 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.566 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:30.566 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:30.566 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 62.143862697000003, 2024-09-15T21:47:30.566 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059975130000000003, 2024-09-15T21:47:30.566 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.566 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.566 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.566 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.566 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.566 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.566 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:30.566 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.566 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.567 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.567 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.567 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:30.567 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.567 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.567 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.567 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.567 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:30.567 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6199999999999999e-07 2024-09-15T21:47:30.567 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.567 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.567 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.567 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413121+0000", 2024-09-15T21:47:30.567 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2000000000000001e-06 2024-09-15T21:47:30.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.568 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.568 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.568 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413292+0000", 2024-09-15T21:47:30.568 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017034400000000001 2024-09-15T21:47:30.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.568 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.568 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.568 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413300+0000", 2024-09-15T21:47:30.568 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8539999999999997e-06 2024-09-15T21:47:30.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.568 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.568 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.569 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414299+0000", 2024-09-15T21:47:30.569 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0009992829999999999 2024-09-15T21:47:30.569 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.569 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.569 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.569 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414390+0000", 2024-09-15T21:47:30.569 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1237999999999996e-05 2024-09-15T21:47:30.569 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.569 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.569 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:30.569 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414490+0000", 2024-09-15T21:47:30.569 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9438999999999994e-05 2024-09-15T21:47:30.569 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.569 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.570 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.570 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417393+0000", 2024-09-15T21:47:30.570 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002903306 2024-09-15T21:47:30.570 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.570 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.570 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.570 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419023+0000", 2024-09-15T21:47:30.570 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001630292 2024-09-15T21:47:30.570 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.570 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.570 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.570 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419067+0000", 2024-09-15T21:47:30.571 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4017999999999998e-05 2024-09-15T21:47:30.571 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.571 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.571 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.571 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419116+0000", 2024-09-15T21:47:30.571 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9277000000000003e-05 2024-09-15T21:47:30.571 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.571 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.571 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.571 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.572 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.572 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:30.572 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:30.572 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 62.141164500000002, 2024-09-15T21:47:30.572 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060879280000000003, 2024-09-15T21:47:30.572 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.572 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.572 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.572 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.572 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.572 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.572 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:30.572 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.573 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.573 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.573 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:30.573 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.573 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.573 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.573 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415818+0000", 2024-09-15T21:47:30.573 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:47:30.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.573 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.573 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.574 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415854+0000", 2024-09-15T21:47:30.574 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6706999999999999e-05 2024-09-15T21:47:30.574 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.574 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.574 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.574 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415860+0000", 2024-09-15T21:47:30.574 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0789999999999997e-06 2024-09-15T21:47:30.574 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.574 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.574 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.574 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415865+0000", 2024-09-15T21:47:30.574 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4580000000000003e-06 2024-09-15T21:47:30.575 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.575 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.575 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.575 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416092+0000", 2024-09-15T21:47:30.575 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00022739400000000001 2024-09-15T21:47:30.575 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.575 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.575 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.575 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416108+0000", 2024-09-15T21:47:30.575 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.613e-05 2024-09-15T21:47:30.575 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.575 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.575 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.575 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421866+0000", 2024-09-15T21:47:30.576 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005757585 2024-09-15T21:47:30.576 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.576 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.576 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.576 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421905+0000", 2024-09-15T21:47:30.576 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9214000000000001e-05 2024-09-15T21:47:30.576 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.576 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.576 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.576 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.576 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.576 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-15T21:47:30.576 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:30.577 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 62.141104339999998, 2024-09-15T21:47:30.577 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060637570000000003, 2024-09-15T21:47:30.577 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.577 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.577 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.577 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.577 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.577 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.577 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:30.577 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.577 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.577 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.577 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.578 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:30.578 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.578 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.578 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.578 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415878+0000", 2024-09-15T21:47:30.578 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.34e-07 2024-09-15T21:47:30.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.578 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.578 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.578 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415881+0000", 2024-09-15T21:47:30.578 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.354e-06 2024-09-15T21:47:30.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.578 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.579 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.579 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415887+0000", 2024-09-15T21:47:30.579 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:47:30.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.579 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.579 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.579 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415891+0000", 2024-09-15T21:47:30.579 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2690000000000001e-06 2024-09-15T21:47:30.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.579 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.579 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.579 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416267+0000", 2024-09-15T21:47:30.579 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00037614799999999998 2024-09-15T21:47:30.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.580 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.580 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.580 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416288+0000", 2024-09-15T21:47:30.580 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0604000000000001e-05 2024-09-15T21:47:30.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.580 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.580 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.580 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421908+0000", 2024-09-15T21:47:30.580 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056203390000000002 2024-09-15T21:47:30.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.580 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.580 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.580 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421941+0000", 2024-09-15T21:47:30.581 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3079e-05 2024-09-15T21:47:30.581 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.581 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.581 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.581 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.581 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.581 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:30.581 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:30.581 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 62.141085912999998, 2024-09-15T21:47:30.581 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006080284, 2024-09-15T21:47:30.581 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.581 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.581 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.581 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.582 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.582 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.582 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:30.582 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.582 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.582 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.582 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:30.582 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.582 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.582 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.582 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:30.582 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6899999999999999e-07 2024-09-15T21:47:30.583 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.583 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.583 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.583 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415898+0000", 2024-09-15T21:47:30.583 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.951e-06 2024-09-15T21:47:30.583 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.583 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.583 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.583 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415920+0000", 2024-09-15T21:47:30.583 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1651999999999999e-05 2024-09-15T21:47:30.583 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.583 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.583 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.583 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415925+0000", 2024-09-15T21:47:30.584 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.463e-06 2024-09-15T21:47:30.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.584 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.584 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.584 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416449+0000", 2024-09-15T21:47:30.584 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052384000000000003 2024-09-15T21:47:30.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.584 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.584 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.584 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416466+0000", 2024-09-15T21:47:30.584 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7108999999999998e-05 2024-09-15T21:47:30.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.584 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.585 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.585 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421943+0000", 2024-09-15T21:47:30.585 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054771569999999999 2024-09-15T21:47:30.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.585 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.585 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.585 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421976+0000", 2024-09-15T21:47:30.585 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2943e-05 2024-09-15T21:47:30.585 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.585 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.585 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.585 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.586 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-15T21:47:30.586 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:30.586 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 62.141046676999999, 2024-09-15T21:47:30.586 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060643340000000002, 2024-09-15T21:47:30.586 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.586 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.586 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.586 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.586 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.586 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.586 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:30.586 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.586 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.587 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.587 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:30.587 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.587 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.587 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.587 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:30.587 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4200000000000002e-07 2024-09-15T21:47:30.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.587 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.587 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.587 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415939+0000", 2024-09-15T21:47:30.587 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.439e-06 2024-09-15T21:47:30.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.587 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.588 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.588 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415945+0000", 2024-09-15T21:47:30.588 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7740000000000001e-06 2024-09-15T21:47:30.588 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.588 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.588 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.588 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415948+0000", 2024-09-15T21:47:30.588 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.737e-06 2024-09-15T21:47:30.588 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.588 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.588 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.588 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416605+0000", 2024-09-15T21:47:30.588 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00065721699999999998 2024-09-15T21:47:30.588 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.589 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.589 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.589 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416620+0000", 2024-09-15T21:47:30.589 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4564e-05 2024-09-15T21:47:30.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.589 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.589 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.589 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421977+0000", 2024-09-15T21:47:30.589 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053572369999999999 2024-09-15T21:47:30.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.589 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.589 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.589 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421999+0000", 2024-09-15T21:47:30.589 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2024000000000001e-05 2024-09-15T21:47:30.590 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.590 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.590 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.590 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.590 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.590 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-15T21:47:30.590 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:30.590 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 62.140846017999998, 2024-09-15T21:47:30.590 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058922669999999996, 2024-09-15T21:47:30.590 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.590 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.590 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.590 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.590 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.591 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.591 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:30.591 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.591 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.591 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.591 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:30.591 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.591 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.591 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.591 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:30.591 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6499999999999999e-07 2024-09-15T21:47:30.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.591 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.592 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.592 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416142+0000", 2024-09-15T21:47:30.592 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3929999999999999e-06 2024-09-15T21:47:30.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.592 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.592 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.592 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416151+0000", 2024-09-15T21:47:30.592 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5840000000000001e-06 2024-09-15T21:47:30.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.592 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.592 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.592 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416157+0000", 2024-09-15T21:47:30.592 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7590000000000003e-06 2024-09-15T21:47:30.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.593 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.593 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.593 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416767+0000", 2024-09-15T21:47:30.593 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00061034900000000005 2024-09-15T21:47:30.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.593 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.593 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.593 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416782+0000", 2024-09-15T21:47:30.593 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.483e-05 2024-09-15T21:47:30.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.593 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.593 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.593 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422002+0000", 2024-09-15T21:47:30.594 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052202639999999996 2024-09-15T21:47:30.594 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.594 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.594 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.594 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422028+0000", 2024-09-15T21:47:30.594 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5423e-05 2024-09-15T21:47:30.594 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.594 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.594 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.594 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.594 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.594 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-15T21:47:30.594 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:30.594 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 62.140439516999997, 2024-09-15T21:47:30.594 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055107769999999997, 2024-09-15T21:47:30.595 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.595 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.595 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.595 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.595 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.595 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.595 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:30.595 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.595 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.595 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.595 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:30.595 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.596 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.596 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.596 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416543+0000", 2024-09-15T21:47:30.596 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5400000000000001e-07 2024-09-15T21:47:30.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.596 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.596 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.596 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416548+0000", 2024-09-15T21:47:30.596 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8659999999999997e-06 2024-09-15T21:47:30.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.596 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.596 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.596 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416555+0000", 2024-09-15T21:47:30.596 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7190000000000001e-06 2024-09-15T21:47:30.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.597 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.597 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.597 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416560+0000", 2024-09-15T21:47:30.597 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3270000000000002e-06 2024-09-15T21:47:30.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.597 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.597 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.597 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416916+0000", 2024-09-15T21:47:30.597 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00035661200000000002 2024-09-15T21:47:30.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.597 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.597 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.597 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416932+0000", 2024-09-15T21:47:30.597 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5824e-05 2024-09-15T21:47:30.598 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.598 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.598 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.598 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422030+0000", 2024-09-15T21:47:30.598 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050975229999999996 2024-09-15T21:47:30.598 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.598 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.598 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.598 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422053+0000", 2024-09-15T21:47:30.598 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3252e-05 2024-09-15T21:47:30.598 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.598 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.598 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.599 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.599 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.599 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-15T21:47:30.599 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:30.599 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 62.140041017999998, 2024-09-15T21:47:30.599 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051422289999999999, 2024-09-15T21:47:30.599 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.599 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.599 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.599 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.599 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.599 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.599 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:30.599 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.599 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.600 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.600 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.600 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:30.600 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.600 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.600 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.600 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.600 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:30.600 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2699999999999999e-07 2024-09-15T21:47:30.600 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.600 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.600 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.600 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416946+0000", 2024-09-15T21:47:30.600 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1390000000000003e-06 2024-09-15T21:47:30.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.601 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.601 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.601 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416957+0000", 2024-09-15T21:47:30.601 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.044e-05 2024-09-15T21:47:30.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.601 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.601 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.601 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416963+0000", 2024-09-15T21:47:30.601 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5320000000000003e-06 2024-09-15T21:47:30.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.601 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.601 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.601 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417090+0000", 2024-09-15T21:47:30.602 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126549 2024-09-15T21:47:30.602 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.602 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.602 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.602 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417114+0000", 2024-09-15T21:47:30.602 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3736000000000001e-05 2024-09-15T21:47:30.602 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.602 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.602 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.602 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422055+0000", 2024-09-15T21:47:30.602 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049410410000000002 2024-09-15T21:47:30.602 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.602 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.602 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.602 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422083+0000", 2024-09-15T21:47:30.603 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8164999999999999e-05 2024-09-15T21:47:30.603 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.603 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.603 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.603 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.603 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.603 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.603 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:30.603 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 56.095549046000002, 2024-09-15T21:47:30.603 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007521186, 2024-09-15T21:47:30.604 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.604 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.604 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.604 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.604 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.604 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.604 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:30.604 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.604 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.604 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.604 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:30.604 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.605 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.605 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.605 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:30.605 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9999999999999998e-07 2024-09-15T21:47:30.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.605 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.605 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.605 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461443+0000", 2024-09-15T21:47:30.605 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0448000000000001e-05 2024-09-15T21:47:30.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.606 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.606 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.606 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462146+0000", 2024-09-15T21:47:30.606 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00070223499999999995 2024-09-15T21:47:30.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.606 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.606 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.606 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462154+0000", 2024-09-15T21:47:30.606 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9060000000000002e-06 2024-09-15T21:47:30.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.606 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.606 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.607 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462254+0000", 2024-09-15T21:47:30.607 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9897999999999998e-05 2024-09-15T21:47:30.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.607 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.607 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.607 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462330+0000", 2024-09-15T21:47:30.607 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6551000000000006e-05 2024-09-15T21:47:30.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.607 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.607 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:30.607 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462454+0000", 2024-09-15T21:47:30.607 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000124213 2024-09-15T21:47:30.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.608 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.608 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.608 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463681+0000", 2024-09-15T21:47:30.608 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012265729999999999 2024-09-15T21:47:30.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.608 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.608 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.608 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468817+0000", 2024-09-15T21:47:30.608 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051361430000000001 2024-09-15T21:47:30.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.608 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.608 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.608 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468892+0000", 2024-09-15T21:47:30.608 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5480000000000002e-05 2024-09-15T21:47:30.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.609 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.609 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.609 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468954+0000", 2024-09-15T21:47:30.609 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1339000000000003e-05 2024-09-15T21:47:30.609 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.609 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.609 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.609 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.609 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.609 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:30.609 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 56.095532431000002, 2024-09-15T21:47:30.610 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077296149999999996, 2024-09-15T21:47:30.610 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.610 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.610 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.610 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.610 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.610 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.610 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:30.610 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.610 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.610 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.610 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.610 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:30.610 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.611 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.611 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.611 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.611 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461450+0000", 2024-09-15T21:47:30.611 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0499999999999999e-07 2024-09-15T21:47:30.611 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.611 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.611 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.611 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461452+0000", 2024-09-15T21:47:30.611 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.007e-06 2024-09-15T21:47:30.611 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.611 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.611 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.611 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462209+0000", 2024-09-15T21:47:30.612 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075781000000000004 2024-09-15T21:47:30.612 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.612 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.612 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.612 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462214+0000", 2024-09-15T21:47:30.612 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1420000000000001e-06 2024-09-15T21:47:30.612 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.612 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.612 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.612 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462681+0000", 2024-09-15T21:47:30.612 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046634200000000002 2024-09-15T21:47:30.612 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.612 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.613 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.613 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462772+0000", 2024-09-15T21:47:30.613 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1414000000000003e-05 2024-09-15T21:47:30.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.613 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.613 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:30.613 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462871+0000", 2024-09-15T21:47:30.613 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8455000000000005e-05 2024-09-15T21:47:30.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.613 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.613 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.613 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464138+0000", 2024-09-15T21:47:30.613 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012673910000000001 2024-09-15T21:47:30.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.614 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.614 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.614 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469079+0000", 2024-09-15T21:47:30.614 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049407799999999997 2024-09-15T21:47:30.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.614 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.614 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.614 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469141+0000", 2024-09-15T21:47:30.614 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2169999999999996e-05 2024-09-15T21:47:30.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.614 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.614 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.615 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469179+0000", 2024-09-15T21:47:30.615 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7799000000000002e-05 2024-09-15T21:47:30.615 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.615 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.615 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.615 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.615 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.615 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.616 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:30.616 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 50.081416623000003, 2024-09-15T21:47:30.616 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004990497, 2024-09-15T21:47:30.616 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.616 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.616 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.616 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.616 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.616 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.616 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:30.616 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.616 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.616 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.617 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.617 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:30.617 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.617 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.617 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.617 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.617 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:30.617 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4729999999999999e-06 2024-09-15T21:47:30.617 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.617 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.617 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.617 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475576+0000", 2024-09-15T21:47:30.617 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1749999999999994e-06 2024-09-15T21:47:30.618 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.618 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.618 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.618 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475619+0000", 2024-09-15T21:47:30.618 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3557000000000003e-05 2024-09-15T21:47:30.618 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.618 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.618 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.618 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475631+0000", 2024-09-15T21:47:30.618 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1739999999999999e-05 2024-09-15T21:47:30.618 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.618 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.618 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.619 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475770+0000", 2024-09-15T21:47:30.619 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013950599999999999 2024-09-15T21:47:30.619 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.619 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.619 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.619 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475887+0000", 2024-09-15T21:47:30.619 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011616 2024-09-15T21:47:30.619 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.619 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.619 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:30.619 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476018+0000", 2024-09-15T21:47:30.619 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013119499999999999 2024-09-15T21:47:30.619 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.619 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.620 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.620 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478442+0000", 2024-09-15T21:47:30.620 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024240870000000001 2024-09-15T21:47:30.620 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.620 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.620 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.620 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480457+0000", 2024-09-15T21:47:30.620 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020151779999999998 2024-09-15T21:47:30.620 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.620 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.621 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.621 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480511+0000", 2024-09-15T21:47:30.621 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3582000000000003e-05 2024-09-15T21:47:30.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.621 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.621 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.621 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480556+0000", 2024-09-15T21:47:30.621 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4844e-05 2024-09-15T21:47:30.621 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.621 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.621 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.621 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.621 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.622 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:30.622 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 50.081402304999997, 2024-09-15T21:47:30.622 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051124810000000003, 2024-09-15T21:47:30.622 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.622 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.622 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.622 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.622 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.622 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.622 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:30.622 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.622 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.622 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.622 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.623 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:30.623 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.623 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.623 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.623 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.623 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475580+0000", 2024-09-15T21:47:30.623 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7800000000000002e-07 2024-09-15T21:47:30.623 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.623 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.623 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.623 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475595+0000", 2024-09-15T21:47:30.623 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4919000000000001e-05 2024-09-15T21:47:30.623 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.623 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.624 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.624 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475662+0000", 2024-09-15T21:47:30.624 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7769999999999997e-05 2024-09-15T21:47:30.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.624 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.624 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.624 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475669+0000", 2024-09-15T21:47:30.624 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.703e-06 2024-09-15T21:47:30.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.624 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.624 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.624 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476781+0000", 2024-09-15T21:47:30.624 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011118300000000001 2024-09-15T21:47:30.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.625 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.625 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.625 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476863+0000", 2024-09-15T21:47:30.625 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2208999999999998e-05 2024-09-15T21:47:30.625 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.625 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.625 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:30.625 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477040+0000", 2024-09-15T21:47:30.625 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000176424 2024-09-15T21:47:30.625 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.625 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.625 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.625 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480154+0000", 2024-09-15T21:47:30.625 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031147589999999999 2024-09-15T21:47:30.626 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.626 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.626 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.626 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480596+0000", 2024-09-15T21:47:30.626 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044172699999999999 2024-09-15T21:47:30.626 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.626 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.626 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.626 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480652+0000", 2024-09-15T21:47:30.626 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6350000000000001e-05 2024-09-15T21:47:30.626 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.626 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.626 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.626 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480692+0000", 2024-09-15T21:47:30.626 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9412000000000001e-05 2024-09-15T21:47:30.627 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.627 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.627 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.627 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.627 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:6 2.9 2:941d59bc:::61f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.627 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:30.627 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 45.012191772999998, 2024-09-15T21:47:30.627 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057427529999999997, 2024-09-15T21:47:30.627 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.627 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.627 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.627 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.627 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.628 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.628 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:30.628 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.628 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.628 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.628 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.628 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:30.628 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.628 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.628 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.628 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.628 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:30.628 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.41e-07 2024-09-15T21:47:30.628 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.629 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.629 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.629 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544792+0000", 2024-09-15T21:47:30.629 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9929999999999998e-06 2024-09-15T21:47:30.629 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.629 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.629 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.629 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544885+0000", 2024-09-15T21:47:30.629 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2825000000000007e-05 2024-09-15T21:47:30.629 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.629 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.632 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.632 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544891+0000", 2024-09-15T21:47:30.632 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.93e-06 2024-09-15T21:47:30.632 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.632 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.632 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.632 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546766+0000", 2024-09-15T21:47:30.633 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018747569999999999 2024-09-15T21:47:30.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.633 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.633 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.633 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546864+0000", 2024-09-15T21:47:30.633 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8603000000000004e-05 2024-09-15T21:47:30.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.633 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.633 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:30.633 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547028+0000", 2024-09-15T21:47:30.633 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000163461 2024-09-15T21:47:30.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.633 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.633 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.633 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549499+0000", 2024-09-15T21:47:30.634 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024711749999999999 2024-09-15T21:47:30.634 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.634 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.634 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.634 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550417+0000", 2024-09-15T21:47:30.634 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00091817199999999998 2024-09-15T21:47:30.634 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.634 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.634 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.634 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550476+0000", 2024-09-15T21:47:30.634 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8635999999999999e-05 2024-09-15T21:47:30.634 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.634 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.635 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.635 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550533+0000", 2024-09-15T21:47:30.635 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6959999999999997e-05 2024-09-15T21:47:30.635 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.635 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.635 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.635 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.635 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.635 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.635 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:30.635 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 45.012177307999998, 2024-09-15T21:47:30.635 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004917121, 2024-09-15T21:47:30.636 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.636 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.636 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.636 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.636 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.636 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.636 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:30.636 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.636 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.636 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.636 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.636 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:30.636 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.636 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.637 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.637 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.637 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544805+0000", 2024-09-15T21:47:30.637 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4600000000000001e-07 2024-09-15T21:47:30.637 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.637 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.637 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.637 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544817+0000", 2024-09-15T21:47:30.637 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2023e-05 2024-09-15T21:47:30.637 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.637 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.637 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.637 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545000+0000", 2024-09-15T21:47:30.638 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018384400000000001 2024-09-15T21:47:30.638 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.638 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.638 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.638 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545007+0000", 2024-09-15T21:47:30.638 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1249999999999998e-06 2024-09-15T21:47:30.638 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.638 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.638 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.638 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545670+0000", 2024-09-15T21:47:30.638 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00066324999999999999 2024-09-15T21:47:30.638 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.638 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.639 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.639 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545749+0000", 2024-09-15T21:47:30.639 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9239000000000005e-05 2024-09-15T21:47:30.639 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.639 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.639 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:30.639 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545885+0000", 2024-09-15T21:47:30.639 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013586199999999999 2024-09-15T21:47:30.639 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.639 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.639 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.639 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547944+0000", 2024-09-15T21:47:30.639 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020594720000000001 2024-09-15T21:47:30.639 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.640 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.640 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.640 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549630+0000", 2024-09-15T21:47:30.640 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001685831 2024-09-15T21:47:30.640 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.640 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.640 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.640 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549681+0000", 2024-09-15T21:47:30.640 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1187000000000003e-05 2024-09-15T21:47:30.640 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.641 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.641 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.641 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549721+0000", 2024-09-15T21:47:30.641 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0042e-05 2024-09-15T21:47:30.641 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.641 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.641 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.641 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.641 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.641 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.641 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:30.641 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 45.012163559999998, 2024-09-15T21:47:30.641 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050615549999999997, 2024-09-15T21:47:30.642 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.642 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.642 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.642 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.642 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.642 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.642 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:30.642 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.642 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.642 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.642 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.642 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:30.642 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.642 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.642 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.643 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.643 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:30.643 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:47:30.643 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.643 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.643 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.643 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544821+0000", 2024-09-15T21:47:30.643 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4399999999999999e-06 2024-09-15T21:47:30.643 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.643 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.643 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.643 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545025+0000", 2024-09-15T21:47:30.643 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020379400000000001 2024-09-15T21:47:30.644 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.644 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.644 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.644 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545030+0000", 2024-09-15T21:47:30.644 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3109999999999996e-06 2024-09-15T21:47:30.644 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.644 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.644 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.644 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546196+0000", 2024-09-15T21:47:30.644 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001166261 2024-09-15T21:47:30.644 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.644 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.644 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.644 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546307+0000", 2024-09-15T21:47:30.645 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000110641 2024-09-15T21:47:30.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.645 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.645 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:30.645 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546422+0000", 2024-09-15T21:47:30.645 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000115399 2024-09-15T21:47:30.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.645 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.645 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.645 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547973+0000", 2024-09-15T21:47:30.645 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001551228 2024-09-15T21:47:30.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.645 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.645 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.646 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549815+0000", 2024-09-15T21:47:30.646 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001841685 2024-09-15T21:47:30.646 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.646 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.646 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.646 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549856+0000", 2024-09-15T21:47:30.646 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.091e-05 2024-09-15T21:47:30.646 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.646 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.646 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.646 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549880+0000", 2024-09-15T21:47:30.646 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3580000000000001e-05 2024-09-15T21:47:30.646 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.646 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.647 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.647 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.647 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.647 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.647 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:30.647 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 45.012159552, 2024-09-15T21:47:30.647 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056931389999999998, 2024-09-15T21:47:30.647 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.647 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.647 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.647 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.647 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.647 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.647 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:30.648 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.648 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.648 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.648 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.648 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:30.648 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.648 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.648 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.648 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.648 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:30.648 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:47:30.648 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.648 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.648 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.649 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544828+0000", 2024-09-15T21:47:30.649 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4369999999999998e-06 2024-09-15T21:47:30.649 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.649 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.649 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.649 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545041+0000", 2024-09-15T21:47:30.649 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000212695 2024-09-15T21:47:30.649 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.649 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.649 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.649 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545058+0000", 2024-09-15T21:47:30.649 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7717999999999999e-05 2024-09-15T21:47:30.649 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.649 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.650 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.650 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546958+0000", 2024-09-15T21:47:30.650 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018995539999999999 2024-09-15T21:47:30.650 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.650 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.650 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.650 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547057+0000", 2024-09-15T21:47:30.650 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9370999999999994e-05 2024-09-15T21:47:30.650 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.650 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.650 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:30.650 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547579+0000", 2024-09-15T21:47:30.650 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052198200000000004 2024-09-15T21:47:30.650 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.650 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.650 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.651 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549929+0000", 2024-09-15T21:47:30.651 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023494649999999998 2024-09-15T21:47:30.651 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.651 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.651 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.651 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550401+0000", 2024-09-15T21:47:30.651 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00047277300000000001 2024-09-15T21:47:30.651 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.651 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.651 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.651 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550458+0000", 2024-09-15T21:47:30.651 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6756999999999999e-05 2024-09-15T21:47:30.651 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.651 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.652 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.652 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550515+0000", 2024-09-15T21:47:30.652 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7093999999999999e-05 2024-09-15T21:47:30.652 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.652 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.652 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.652 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.652 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.652 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:26 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.652 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:30.652 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 45.012152895, 2024-09-15T21:47:30.652 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050112029999999997, 2024-09-15T21:47:30.652 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.652 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.653 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.653 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.653 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.653 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.653 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:30.653 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.653 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.653 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.653 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.653 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:30.653 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.653 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.653 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.653 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.653 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:30.654 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.36e-07 2024-09-15T21:47:30.654 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.654 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.654 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.654 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545086+0000", 2024-09-15T21:47:30.654 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025704699999999999 2024-09-15T21:47:30.654 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.654 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.654 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.654 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545119+0000", 2024-09-15T21:47:30.654 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2734999999999997e-05 2024-09-15T21:47:30.654 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.654 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.655 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.655 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545126+0000", 2024-09-15T21:47:30.655 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5599999999999996e-06 2024-09-15T21:47:30.655 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.655 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.655 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545253+0000", 2024-09-15T21:47:30.655 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012644500000000001 2024-09-15T21:47:30.655 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.655 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.655 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545366+0000", 2024-09-15T21:47:30.655 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011349999999999999 2024-09-15T21:47:30.655 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.656 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:30.656 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545673+0000", 2024-09-15T21:47:30.656 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030643599999999999 2024-09-15T21:47:30.656 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.656 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.656 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.656 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547924+0000", 2024-09-15T21:47:30.656 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0022516189999999998 2024-09-15T21:47:30.656 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.656 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.656 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.656 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549746+0000", 2024-09-15T21:47:30.657 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018217629999999999 2024-09-15T21:47:30.657 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.657 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.657 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.657 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549800+0000", 2024-09-15T21:47:30.657 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3412999999999998e-05 2024-09-15T21:47:30.657 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.657 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.657 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.657 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549840+0000", 2024-09-15T21:47:30.657 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0448999999999998e-05 2024-09-15T21:47:30.657 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.657 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.657 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.658 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.658 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.658 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:30 2.4 2:218afd09:::203.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.658 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:30.658 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 45.011892066000001, 2024-09-15T21:47:30.658 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055481230000000003, 2024-09-15T21:47:30.658 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.658 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.658 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:30.658 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.658 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.658 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.658 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:30.659 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.659 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.659 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.659 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.659 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:30.659 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.659 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.659 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.659 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.659 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545091+0000", 2024-09-15T21:47:30.659 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7000000000000003e-07 2024-09-15T21:47:30.659 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.659 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.659 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.659 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545094+0000", 2024-09-15T21:47:30.660 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9569999999999998e-06 2024-09-15T21:47:30.660 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.660 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.660 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.660 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545156+0000", 2024-09-15T21:47:30.660 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2277e-05 2024-09-15T21:47:30.660 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.660 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.660 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.660 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545162+0000", 2024-09-15T21:47:30.660 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9120000000000003e-06 2024-09-15T21:47:30.660 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.660 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.661 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.661 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545262+0000", 2024-09-15T21:47:30.661 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010001900000000001 2024-09-15T21:47:30.661 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.661 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.661 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.661 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545388+0000", 2024-09-15T21:47:30.661 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126243 2024-09-15T21:47:30.661 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.661 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.661 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.661 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550638+0000", 2024-09-15T21:47:30.661 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052497450000000001 2024-09-15T21:47:30.661 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.662 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.662 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.662 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.662 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.662 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:10 2.9 2:954a9a2b:::62b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:30.662 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:30.662 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 38.992208605999998, 2024-09-15T21:47:30.662 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048438860000000004, 2024-09-15T21:47:30.662 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:30.662 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:30.662 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:30.662 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:30.662 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.662 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:30.663 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:30.663 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.663 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.663 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.663 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:30.663 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:30.663 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:30.663 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.663 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.663 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:30.663 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:30.663 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0400000000000002e-07 2024-09-15T21:47:30.663 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.664 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.664 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:30.664 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564786+0000", 2024-09-15T21:47:30.664 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2412000000000001e-05 2024-09-15T21:47:30.664 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.664 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.664 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:30.664 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564991+0000", 2024-09-15T21:47:30.664 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020483900000000001 2024-09-15T21:47:30.664 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.664 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.664 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:30.664 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564998+0000", 2024-09-15T21:47:30.664 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2459999999999997e-06 2024-09-15T21:47:30.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.665 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.665 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:30.665 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565422+0000", 2024-09-15T21:47:30.665 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00042420100000000003 2024-09-15T21:47:30.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.665 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.665 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:30.665 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565515+0000", 2024-09-15T21:47:30.665 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2449999999999997e-05 2024-09-15T21:47:30.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.665 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.665 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:30.666 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565642+0000", 2024-09-15T21:47:30.666 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012743699999999999 2024-09-15T21:47:30.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.666 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.666 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:30.666 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566656+0000", 2024-09-15T21:47:30.666 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010138580000000001 2024-09-15T21:47:30.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.666 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.666 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:30.666 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569552+0000", 2024-09-15T21:47:30.666 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028966819999999998 2024-09-15T21:47:30.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.666 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.667 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:30.667 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569594+0000", 2024-09-15T21:47:30.667 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1235000000000001e-05 2024-09-15T21:47:30.667 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:30.667 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:30.667 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:30.667 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569617+0000", 2024-09-15T21:47:30.667 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3221999999999999e-05 2024-09-15T21:47:30.667 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.667 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.668 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.668 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:30.668 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:30.668 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:30.669 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:30.666+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:30.670 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:47:30.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.641+0000 7fe775055640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fe7700ccfd0 0x7fe770152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:30.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.641+0000 7fe775055640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fe7700ccfd0 0x7fe770152c40 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:30.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.642+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe770149d30 con 0x7fe7700ccfd0 2024-09-15T21:47:30.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.657+0000 7fe777aba640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe7700cd3b0 con 0x7fe7700ccfd0 2024-09-15T21:47:30.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.658+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe7700cd3b0 con 0x7fe7700ccfd0 2024-09-15T21:47:30.671 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:30.672 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:30.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.658+0000 7fe777aba640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fe7701f43c0 0x7fe770158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:30.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.658+0000 7fe777aba640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fe7701f47a0 con 0x7fe7701f43c0 2024-09-15T21:47:30.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.659+0000 7fe775856640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fe7701f43c0 0x7fe770158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:30.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.659+0000 7fe775856640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fe7701f43c0 0x7fe770158d40 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:30.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.659+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe7701f47a0 con 0x7fe7701f43c0 2024-09-15T21:47:30.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.676+0000 7fe777aba640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe770149100 con 0x7fe7701f43c0 2024-09-15T21:47:30.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.679+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe770149100 con 0x7fe7701f43c0 2024-09-15T21:47:30.681 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:30.681 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:30.681 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.679+0000 7fe777aba640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fe770159790 0x7fe770062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:30.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.679+0000 7fe777aba640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fe770062b70 con 0x7fe770159790 2024-09-15T21:47:30.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.679+0000 7fe775055640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fe770159790 0x7fe770062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:30.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.680+0000 7fe775055640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fe770159790 0x7fe770062630 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:30.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.681+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe770062b70 con 0x7fe770159790 2024-09-15T21:47:30.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.708+0000 7fe777aba640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe770063020 con 0x7fe770159790 2024-09-15T21:47:30.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.710+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe770063020 con 0x7fe770159790 2024-09-15T21:47:30.712 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:30.713 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:30.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.710+0000 7fe777aba640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fe7700cb8f0 0x7fe7700cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:30.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.710+0000 7fe777aba640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fe7700635f0 con 0x7fe7700cb8f0 2024-09-15T21:47:30.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.711+0000 7fe776057640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fe7700cb8f0 0x7fe7700cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:30.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.711+0000 7fe776057640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fe7700cb8f0 0x7fe7700cbd50 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:30.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.712+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe768059eb0 con 0x7fe7700cb8f0 2024-09-15T21:47:30.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.726+0000 7fe777aba640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe77002c640 con 0x7fe7700cb8f0 2024-09-15T21:47:30.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.728+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe77002c640 con 0x7fe7700cb8f0 2024-09-15T21:47:30.730 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:30.731 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:30.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.729+0000 7fe777aba640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fe77002cd60 0x7fe77002d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:30.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.729+0000 7fe777aba640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fe77002c640 con 0x7fe77002cd60 2024-09-15T21:47:30.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.729+0000 7fe775856640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fe77002cd60 0x7fe77002d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:30.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.729+0000 7fe775856640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fe77002cd60 0x7fe77002d180 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:30.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.730+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe77002c640 con 0x7fe77002cd60 2024-09-15T21:47:30.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.744+0000 7fe777aba640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe77002c640 con 0x7fe77002cd60 2024-09-15T21:47:30.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.746+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe77002c640 con 0x7fe77002cd60 2024-09-15T21:47:30.748 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:30.749 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:30.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.747+0000 7fe777aba640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=36}) -- 0x7fe77002d6c0 con 0x7fe77015bb60 2024-09-15T21:47:30.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.747+0000 7fe777aba640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe77002c640 con 0x7fe77015bb60 2024-09-15T21:47:30.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.747+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fe768059810 con 0x7fe77015bb60 2024-09-15T21:47:30.750 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:47:30.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.747+0000 7fe777aba640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fe7702058b0 0x7fe770207ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:30.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.747+0000 7fe777aba640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fe77020aa70 con 0x7fe7702058b0 2024-09-15T21:47:30.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.747+0000 7fe775055640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fe7702058b0 0x7fe770207ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:30.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.748+0000 7fe775055640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fe7702058b0 0x7fe770207ca0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:30.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.748+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe77020aa70 con 0x7fe7702058b0 2024-09-15T21:47:30.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.762+0000 7fe777aba640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe7702081e0 con 0x7fe7702058b0 2024-09-15T21:47:30.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.764+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe7702081e0 con 0x7fe7702058b0 2024-09-15T21:47:30.766 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:30.767 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:30.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.764+0000 7fe777aba640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fe770208860 0x7fe77020ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:30.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.764+0000 7fe777aba640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fe77020daf0 con 0x7fe770208860 2024-09-15T21:47:30.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.765+0000 7fe776057640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fe770208860 0x7fe77020ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:30.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.765+0000 7fe776057640 1 --2- 172.21.15.100:0/1323454313 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fe770208860 0x7fe77020ac50 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:30.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.765+0000 7f378effd640 1 -- 172.21.15.100:0/1249376044 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v36) ==== 97+0+0 (secure 0 0 0) 0x7f379807a500 con 0x7f37a0155440 2024-09-15T21:47:30.768 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is set 2024-09-15T21:47:30.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.766+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe77020daf0 con 0x7fe770208860 2024-09-15T21:47:30.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.768+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== mon.0 v2:172.21.15.100:3300/0 8 ==== osd_map(36..36 src has 1..36) ==== 288+0+0 (secure 0 0 0) 0x7fe76808a3d0 con 0x7fe77015bb60 2024-09-15T21:47:30.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.768+0000 7f37a487b640 1 -- 172.21.15.100:0/1249376044 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f378006c2b0 msgr2=0x7f378006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:30.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.768+0000 7f37a487b640 1 --2- 172.21.15.100:0/1249376044 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f378006c2b0 0x7f378006e770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3790009e90 tx=0x7f3790009190 comp rx=0 tx=0).stop 2024-09-15T21:47:30.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.768+0000 7f37a487b640 1 -- 172.21.15.100:0/1249376044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37a0155440 msgr2=0x7f37a01442e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:30.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.768+0000 7f37a487b640 1 --2- 172.21.15.100:0/1249376044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37a0155440 0x7f37a01442e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3798009f70 tx=0x7f3798004a40 comp rx=0 tx=0).stop 2024-09-15T21:47:30.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.768+0000 7f37a487b640 1 -- 172.21.15.100:0/1249376044 shutdown_connections 2024-09-15T21:47:30.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.768+0000 7f37a487b640 1 --2- 172.21.15.100:0/1249376044 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f378006c2b0 0x7f378006e770 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:30.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.768+0000 7f37a487b640 1 --2- 172.21.15.100:0/1249376044 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f37a0155440 0x7f37a01442e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:30.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.768+0000 7f37a487b640 1 --2- 172.21.15.100:0/1249376044 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f37a00cc8f0 0x7f37a00cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:30.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.768+0000 7f37a487b640 1 --2- 172.21.15.100:0/1249376044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f37a00cb7d0 0x7f37a0149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:30.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.768+0000 7f37a487b640 1 -- 172.21.15.100:0/1249376044 >> 172.21.15.100:0/1249376044 conn(0x7f37a00680d0 msgr2=0x7f37a0159880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:30.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.768+0000 7f37a487b640 1 -- 172.21.15.100:0/1249376044 shutdown_connections 2024-09-15T21:47:30.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.768+0000 7f37a487b640 1 -- 172.21.15.100:0/1249376044 wait complete. 2024-09-15T21:47:30.772 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:30.767+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:30.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.782+0000 7fe777aba640 1 -- 172.21.15.100:0/1323454313 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe7700741b0 con 0x7fe770208860 2024-09-15T21:47:30.786 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:30.786 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:30.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:30.785+0000 7fe7667fc640 1 -- 172.21.15.100:0/1323454313 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe7700741b0 con 0x7fe770208860 2024-09-15T21:47:30.787 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:30.787 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:30.791 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:47:30.791 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:47:30.796 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:47:30.879 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:30.878+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:30.941 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:30.941 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:30.949 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:47:30.971 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:30.969+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:31.072 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:31.070+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:31.096 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:31.096 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:31.105 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:47:31.189 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:31.188+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:31.259 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:31.260 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:31.269 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:47:31.272 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:31.270+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:31.373 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:31.371+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:31.416 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:31.416 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:31.426 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:47:31.474 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:31.472+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:31.531 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:31.532 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:31.541 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:47:31.582 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:31.581+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:31.660 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:31.660 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:31.669 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:47:31.713 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:31.711+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:31.774 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:31.774 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:31.776 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:31.773+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:31.783 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:47:31.877 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:31.875+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:31.926 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:31.926 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:31.935 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:47:31.978 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:31.976+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:32.078 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:32.078 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:32.079 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:32.077+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:32.087 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:47:32.180 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:32.178+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:32.189 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:32.189 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:32.198 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:47:32.281 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:32.279+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:32.341 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:32.341 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:32.350 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:47:32.382 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:32.381+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:32.483 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:32.482+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:32.493 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:32.493 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:32.502 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:47:32.584 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:32.583+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:32.604 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:32.605 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:32.613 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:47:32.685 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:32.684+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:32.716 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:32.716 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:32.724 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:47:32.784 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:47:32.787 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:32.785+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:32.791 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:47:32.841 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:32.841 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:32.853 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:47:32.895 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:32.893+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.911+0000 7f580812a640 1 Processor -- start 2024-09-15T21:47:32.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.912+0000 7f580812a640 1 -- start start 2024-09-15T21:47:32.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.913+0000 7f580812a640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f580006c490 0x7f580006c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:32.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.913+0000 7f580812a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f58000c7bd0 0x7f5800163700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:32.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.913+0000 7f580812a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58000c4ed0 con 0x7f5800153060 2024-09-15T21:47:32.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.913+0000 7f580812a640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f58000c5170 con 0x7f580006c490 2024-09-15T21:47:32.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.913+0000 7f580812a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f58000c5450 con 0x7f58000c7bd0 2024-09-15T21:47:32.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.913+0000 7f58056c5640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f5800153060 0x7f5800158f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:44058/0 (socket says 172.21.15.100:44058) 2024-09-15T21:47:32.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.913+0000 7f58056c5640 1 -- 172.21.15.100:0/1722552481 learned_addr learned my addr 172.21.15.100:0/1722552481 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:32.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.913+0000 7f58066c7640 1 --2- 172.21.15.100:0/1722552481 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f58000c7bd0 0x7f5800163700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:32.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.913+0000 7f5805ec6640 1 --2- 172.21.15.100:0/1722552481 >> v2:172.21.15.122:3300/0 conn(0x7f580006c490 0x7f580006c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:32.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.913+0000 7f58066c7640 1 -- 172.21.15.100:0/1722552481 >> v2:172.21.15.122:3300/0 conn(0x7f580006c490 msgr2=0x7f580006c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:32.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.913+0000 7f58066c7640 1 --2- 172.21.15.100:0/1722552481 >> v2:172.21.15.122:3300/0 conn(0x7f580006c490 0x7f580006c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:32.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.913+0000 7f58066c7640 1 -- 172.21.15.100:0/1722552481 >> v1:172.21.15.100:6789/0 conn(0x7f5800153060 legacy=0x7f5800158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:32.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.913+0000 7f58066c7640 1 -- 172.21.15.100:0/1722552481 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58000c5730 con 0x7f58000c7bd0 2024-09-15T21:47:32.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.913+0000 7f5805ec6640 1 --2- 172.21.15.100:0/1722552481 >> v2:172.21.15.122:3300/0 conn(0x7f580006c490 0x7f580006c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:32.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.915+0000 7f58066c7640 1 --2- 172.21.15.100:0/1722552481 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f58000c7bd0 0x7f5800163700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009d90 tx=0x7f57f802a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40a97ca8ebc98b21 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:32.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.915+0000 7f5804ec4640 1 -- 172.21.15.100:0/1722552481 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8007740 con 0x7f58000c7bd0 2024-09-15T21:47:32.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.915+0000 7f5804ec4640 1 -- 172.21.15.100:0/1722552481 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f80078e0 con 0x7f58000c7bd0 2024-09-15T21:47:32.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.915+0000 7f5804ec4640 1 -- 172.21.15.100:0/1722552481 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8007cb0 con 0x7f58000c7bd0 2024-09-15T21:47:32.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.916+0000 7f580812a640 1 -- 172.21.15.100:0/1722552481 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f58000c7bd0 msgr2=0x7f5800163700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:32.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.916+0000 7f580812a640 1 --2- 172.21.15.100:0/1722552481 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f58000c7bd0 0x7f5800163700 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f57f8009d90 tx=0x7f57f802a6f0 comp rx=0 tx=0).stop 2024-09-15T21:47:32.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.916+0000 7f580812a640 1 -- 172.21.15.100:0/1722552481 shutdown_connections 2024-09-15T21:47:32.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.916+0000 7f580812a640 1 --2- 172.21.15.100:0/1722552481 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f58000c7bd0 0x7f5800163700 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:32.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.916+0000 7f580812a640 1 --2- 172.21.15.100:0/1722552481 >> v2:172.21.15.122:3300/0 conn(0x7f580006c490 0x7f580006c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:32.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.916+0000 7f580812a640 1 -- 172.21.15.100:0/1722552481 >> 172.21.15.100:0/1722552481 conn(0x7f58000ca430 msgr2=0x7f58000ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:32.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.916+0000 7f580812a640 1 -- 172.21.15.100:0/1722552481 shutdown_connections 2024-09-15T21:47:32.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.916+0000 7f580812a640 1 -- 172.21.15.100:0/1722552481 wait complete. 2024-09-15T21:47:32.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.918+0000 7f580812a640 1 Processor -- start 2024-09-15T21:47:32.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.918+0000 7f580812a640 1 -- start start 2024-09-15T21:47:32.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.919+0000 7f580812a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f580006c490 0x7f5800157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:32.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.919+0000 7f580812a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58000c7bd0 0x7f580014bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:32.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.919+0000 7f580812a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5800153060 0x7f580014c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:32.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.919+0000 7f580812a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f58000ca160 con 0x7f5800153060 2024-09-15T21:47:32.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.919+0000 7f580812a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f580006c1b0 con 0x7f58000c7bd0 2024-09-15T21:47:32.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.919+0000 7f580812a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f58000c4970 con 0x7f580006c490 2024-09-15T21:47:32.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.919+0000 7f58056c5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58000c7bd0 0x7f580014bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:32.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.919+0000 7f58056c5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58000c7bd0 0x7f580014bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:34298/0 (socket says 172.21.15.100:34298) 2024-09-15T21:47:32.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.919+0000 7f58056c5640 1 -- 172.21.15.100:0/1869336093 learned_addr learned my addr 172.21.15.100:0/1869336093 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:32.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.919+0000 7f5805ec6640 1 --2- 172.21.15.100:0/1869336093 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f580006c490 0x7f5800157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:32.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.919+0000 7f58066c7640 1 --2- 172.21.15.100:0/1869336093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5800153060 0x7f580014c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.919+0000 7f5805ec6640 1 -- 172.21.15.100:0/1869336093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58000c7bd0 msgr2=0x7f580014bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.920+0000 7f5805ec6640 1 --2- 172.21.15.100:0/1869336093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58000c7bd0 0x7f580014bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.920+0000 7f5805ec6640 1 -- 172.21.15.100:0/1869336093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5800153060 msgr2=0x7f580014c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.920+0000 7f5805ec6640 1 --2- 172.21.15.100:0/1869336093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5800153060 0x7f580014c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.920+0000 7f5805ec6640 1 -- 172.21.15.100:0/1869336093 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58001f0de0 con 0x7f580006c490 2024-09-15T21:47:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.920+0000 7f58066c7640 1 --2- 172.21.15.100:0/1869336093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5800153060 0x7f580014c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.920+0000 7f5805ec6640 1 --2- 172.21.15.100:0/1869336093 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f580006c490 0x7f5800157cf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f57f400e930 tx=0x7f57f400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.920+0000 7f57f27fc640 1 -- 172.21.15.100:0/1869336093 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f4017070 con 0x7f580006c490 2024-09-15T21:47:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.920+0000 7f57f27fc640 1 -- 172.21.15.100:0/1869336093 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57f400ca00 con 0x7f580006c490 2024-09-15T21:47:32.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.920+0000 7f57f27fc640 1 -- 172.21.15.100:0/1869336093 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f400cdd0 con 0x7f580006c490 2024-09-15T21:47:32.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.920+0000 7f580812a640 1 -- 172.21.15.100:0/1869336093 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58001f10c0 con 0x7f580006c490 2024-09-15T21:47:32.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.920+0000 7f580812a640 1 -- 172.21.15.100:0/1869336093 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f58001f1680 con 0x7f580006c490 2024-09-15T21:47:32.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.922+0000 7f580812a640 1 -- 172.21.15.100:0/1869336093 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f580006bbe0 con 0x7f580006c490 2024-09-15T21:47:32.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.922+0000 7f57f27fc640 1 -- 172.21.15.100:0/1869336093 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f57f401a070 con 0x7f580006c490 2024-09-15T21:47:32.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.923+0000 7f57f27fc640 1 --2- 172.21.15.100:0/1869336093 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f57e406c2e0 0x7f57e406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:32.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.923+0000 7f57f27fc640 1 -- 172.21.15.100:0/1869336093 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(36..36 src has 1..36) ==== 5854+0+0 (secure 0 0 0) 0x7f57f408b730 con 0x7f580006c490 2024-09-15T21:47:32.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.923+0000 7f58056c5640 1 --2- 172.21.15.100:0/1869336093 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f57e406c2e0 0x7f57e406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:32.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.923+0000 7f58056c5640 1 --2- 172.21.15.100:0/1869336093 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f57e406c2e0 0x7f57e406e7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f580015d1e0 tx=0x7f57fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:32.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.927+0000 7f57f27fc640 1 -- 172.21.15.100:0/1869336093 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f57f4059690 con 0x7f580006c490 2024-09-15T21:47:32.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.968+0000 7f11c4082640 1 Processor -- start 2024-09-15T21:47:32.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.968+0000 7f11c4082640 1 -- start start 2024-09-15T21:47:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.970+0000 7f11c4082640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f11bc151c70 0x7f11bc072f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:32.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.970+0000 7f11c4082640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f11bc152050 0x7f11bc0700f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.970+0000 7f11c4082640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11bc158510 con 0x7f11bc071f80 2024-09-15T21:47:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.970+0000 7f11c4082640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f11bc158d90 con 0x7f11bc151c70 2024-09-15T21:47:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.970+0000 7f11c4082640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f11bc1587d0 con 0x7f11bc152050 2024-09-15T21:47:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.970+0000 7f11c161d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f11bc152050 0x7f11bc0700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.970+0000 7f11c161d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f11bc152050 0x7f11bc0700f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:34302/0 (socket says 172.21.15.100:34302) 2024-09-15T21:47:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.970+0000 7f11c161d640 1 -- 172.21.15.100:0/1359369364 learned_addr learned my addr 172.21.15.100:0/1359369364 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.970+0000 7f11c1e1e640 1 --2- 172.21.15.100:0/1359369364 >> v2:172.21.15.100:3300/0 conn(0x7f11bc151c70 0x7f11bc072f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.970+0000 7f11c161d640 1 -- 172.21.15.100:0/1359369364 >> v2:172.21.15.100:3300/0 conn(0x7f11bc151c70 msgr2=0x7f11bc072f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.970+0000 7f11c161d640 1 --2- 172.21.15.100:0/1359369364 >> v2:172.21.15.100:3300/0 conn(0x7f11bc151c70 0x7f11bc072f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.970+0000 7f11c161d640 1 -- 172.21.15.100:0/1359369364 >> v1:172.21.15.100:6789/0 conn(0x7f11bc071f80 legacy=0x7f11bc0ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:32.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.971+0000 7f11c161d640 1 -- 172.21.15.100:0/1359369364 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc158ab0 con 0x7f11bc152050 2024-09-15T21:47:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.972+0000 7f11c161d640 1 --2- 172.21.15.100:0/1359369364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc152050 0x7f11bc0700f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f11bc06ec30 tx=0x7f11ac02e6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8c32574c4967d88 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.972+0000 7f11c0e1c640 1 -- 172.21.15.100:0/1359369364 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac03a070 con 0x7f11bc152050 2024-09-15T21:47:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.972+0000 7f11c0e1c640 1 -- 172.21.15.100:0/1359369364 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ac035600 con 0x7f11bc152050 2024-09-15T21:47:32.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.972+0000 7f11c0e1c640 1 -- 172.21.15.100:0/1359369364 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac0359d0 con 0x7f11bc152050 2024-09-15T21:47:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.973+0000 7f11c4082640 1 -- 172.21.15.100:0/1359369364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc152050 msgr2=0x7f11bc0700f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.973+0000 7f11c4082640 1 --2- 172.21.15.100:0/1359369364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc152050 0x7f11bc0700f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f11bc06ec30 tx=0x7f11ac02e6a0 comp rx=0 tx=0).stop 2024-09-15T21:47:32.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.973+0000 7f11c4082640 1 -- 172.21.15.100:0/1359369364 shutdown_connections 2024-09-15T21:47:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.973+0000 7f11c4082640 1 --2- 172.21.15.100:0/1359369364 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc152050 0x7f11bc0700f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.973+0000 7f11c4082640 1 --2- 172.21.15.100:0/1359369364 >> v2:172.21.15.100:3300/0 conn(0x7f11bc151c70 0x7f11bc072f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.973+0000 7f11c4082640 1 -- 172.21.15.100:0/1359369364 >> 172.21.15.100:0/1359369364 conn(0x7f11bc06ccf0 msgr2=0x7f11bc06d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.973+0000 7f11c4082640 1 -- 172.21.15.100:0/1359369364 shutdown_connections 2024-09-15T21:47:32.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.973+0000 7f11c4082640 1 -- 172.21.15.100:0/1359369364 wait complete. 2024-09-15T21:47:32.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.975+0000 7f11c4082640 1 Processor -- start 2024-09-15T21:47:32.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.975+0000 7f11c4082640 1 -- start start 2024-09-15T21:47:32.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.976+0000 7f11c4082640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11bc071f80 0x7f11bc1f0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:32.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.976+0000 7f11c4082640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc151c70 0x7f11bc1df840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:32.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.976+0000 7f11c4082640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f11bc152050 0x7f11bc1ece30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:32.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.976+0000 7f11c4082640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc06fbd0 con 0x7f11bc071f80 2024-09-15T21:47:32.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.976+0000 7f11c4082640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc06f8f0 con 0x7f11bc151c70 2024-09-15T21:47:32.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.976+0000 7f11c4082640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f11bc0c4790 con 0x7f11bc152050 2024-09-15T21:47:32.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.976+0000 7f11c161d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc151c70 0x7f11bc1df840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:32.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.976+0000 7f11c1e1e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11bc071f80 0x7f11bc1f0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:32.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.976+0000 7f11c261f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f11bc152050 0x7f11bc1ece30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:32.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.977+0000 7f11c1e1e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11bc071f80 0x7f11bc1f0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36620/0 (socket says 172.21.15.100:36620) 2024-09-15T21:47:32.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.977+0000 7f11c1e1e640 1 -- 172.21.15.100:0/758802845 learned_addr learned my addr 172.21.15.100:0/758802845 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:32.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.977+0000 7f11c161d640 1 -- 172.21.15.100:0/758802845 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f11bc152050 msgr2=0x7f11bc1ece30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:32.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.977+0000 7f11c161d640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f11bc152050 0x7f11bc1ece30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:32.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.977+0000 7f11c161d640 1 -- 172.21.15.100:0/758802845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11bc071f80 msgr2=0x7f11bc1f0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:32.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.977+0000 7f11c161d640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f11bc071f80 0x7f11bc1f0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:32.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.977+0000 7f11c161d640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc152e10 con 0x7f11bc151c70 2024-09-15T21:47:32.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.977+0000 7f11c161d640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f11bc151c70 0x7f11bc1df840 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f11ac034000 tx=0x7f11ac0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:32.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.977+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac045020 con 0x7f11bc151c70 2024-09-15T21:47:32.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.977+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11ac00bb20 con 0x7f11bc151c70 2024-09-15T21:47:32.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.977+0000 7f11c4082640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11bc153950 con 0x7f11bc151c70 2024-09-15T21:47:32.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.978+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac03a040 con 0x7f11bc151c70 2024-09-15T21:47:32.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.978+0000 7f11c4082640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11bc1533b0 con 0x7f11bc151c70 2024-09-15T21:47:32.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.979+0000 7f11c4082640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f11bc069160 con 0x7f11bc151c70 2024-09-15T21:47:32.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.979+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f11ac048020 con 0x7f11bc151c70 2024-09-15T21:47:32.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.980+0000 7f11b27fc640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f11a006c300 0x7f11a006e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:32.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.980+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5854+0+0 (secure 0 0 0) 0x7f11ac0b0130 con 0x7f11bc151c70 2024-09-15T21:47:32.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.980+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v36) ==== 54+0+16 (secure 0 0 0) 0x7f11ac04f030 con 0x7f11bc151c70 2024-09-15T21:47:32.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.981+0000 7f11c1e1e640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f11a006c300 0x7f11a006e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:32.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.981+0000 7f119ffff640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f1198001650 0x7f1198003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:32.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.981+0000 7f119ffff640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1198006bb0 con 0x7f1198001650 2024-09-15T21:47:32.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.981+0000 7f11c261f640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f1198001650 0x7f1198003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:32.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.982+0000 7f11c1e1e640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f11a006c300 0x7f11a006e7c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009b60 tx=0x7f11b8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:32.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.982+0000 7f11c261f640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f1198001650 0x7f1198003b10 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:32.984 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:32.982+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1198006bb0 con 0x7f1198001650 2024-09-15T21:47:32.987 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:32.986+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:33.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.009+0000 7f119ffff640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f1198005c60 con 0x7f1198001650 2024-09-15T21:47:33.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.012+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1198005c60 con 0x7f1198001650 2024-09-15T21:47:33.014 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:33.015 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:33.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.012+0000 7f119ffff640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f1198004190 0x7f1198004640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:33.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.012+0000 7f119ffff640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f1198004c00 con 0x7f1198004190 2024-09-15T21:47:33.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.013+0000 7f11c1e1e640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f1198004190 0x7f1198004640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:33.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.013+0000 7f11c1e1e640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f1198004190 0x7f1198004640 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:33.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.014+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1198004c00 con 0x7f1198004190 2024-09-15T21:47:33.044 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:33.044 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:33.045 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:33.045 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:33.045 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:33.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.044+0000 7f119ffff640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f11980051a0 con 0x7f1198004190 2024-09-15T21:47:33.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.046+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f11980051a0 con 0x7f1198004190 2024-09-15T21:47:33.048 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:33.049 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:33.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.047+0000 7f119ffff640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f1198009370 0x7f119800b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:33.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.047+0000 7f119ffff640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f119800e720 con 0x7f1198009370 2024-09-15T21:47:33.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.047+0000 7f11c161d640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f1198009370 0x7f119800b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:33.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.047+0000 7f11c161d640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f1198009370 0x7f119800b760 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:33.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.048+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f119800e720 con 0x7f1198009370 2024-09-15T21:47:33.057 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:47:33.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.074+0000 7f119ffff640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f119800d730 con 0x7f1198009370 2024-09-15T21:47:33.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.077+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f119800d730 con 0x7f1198009370 2024-09-15T21:47:33.079 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:33.080 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:33.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.077+0000 7f119ffff640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f119800bd20 0x7f119800c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:33.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.077+0000 7f119ffff640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f119800c6c0 con 0x7f119800bd20 2024-09-15T21:47:33.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.077+0000 7f11c261f640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f119800bd20 0x7f119800c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:33.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.078+0000 7f11c261f640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f119800bd20 0x7f119800c180 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:33.080 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.079+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f119800c6c0 con 0x7f119800bd20 2024-09-15T21:47:33.088 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:33.086+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:33.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.096+0000 7f119ffff640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f119800c6c0 con 0x7f119800bd20 2024-09-15T21:47:33.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.099+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f119800c6c0 con 0x7f119800bd20 2024-09-15T21:47:33.101 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:33.101 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:33.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.099+0000 7f119ffff640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f11980108e0 0x7f1198012cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:33.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.099+0000 7f119ffff640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f1198015aa0 con 0x7f11980108e0 2024-09-15T21:47:33.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.099+0000 7f11c1e1e640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f11980108e0 0x7f1198012cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:33.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.100+0000 7f11c1e1e640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f11980108e0 0x7f1198012cd0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:33.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.100+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1198015aa0 con 0x7f11980108e0 2024-09-15T21:47:33.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.127+0000 7f119ffff640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f1198013210 con 0x7f11980108e0 2024-09-15T21:47:33.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.129+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f1198013210 con 0x7f11980108e0 2024-09-15T21:47:33.131 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:33.133 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:33.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.129+0000 7f119ffff640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=37}) -- 0x7f1198015cb0 con 0x7f11bc151c70 2024-09-15T21:47:33.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.129+0000 7f119ffff640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1198015e80 con 0x7f11bc151c70 2024-09-15T21:47:33.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.129+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f11ac0b7070 con 0x7f11bc151c70 2024-09-15T21:47:33.133 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:47:33.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.130+0000 7f119ffff640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f1198013cf0 0x7f1198014140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:33.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.130+0000 7f119ffff640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f1198015ad0 con 0x7f1198013cf0 2024-09-15T21:47:33.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.130+0000 7f11c161d640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f1198013cf0 0x7f1198014140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:33.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.130+0000 7f11c161d640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f1198013cf0 0x7f1198014140 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:33.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.131+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1198015ad0 con 0x7f1198013cf0 2024-09-15T21:47:33.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.158+0000 7f119ffff640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f11980147f0 con 0x7f1198013cf0 2024-09-15T21:47:33.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.160+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f11980147f0 con 0x7f1198013cf0 2024-09-15T21:47:33.161 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:33.162 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:33.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.160+0000 7f119ffff640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f1198014f10 0x7f1198015370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:33.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.160+0000 7f119ffff640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f119801c970 con 0x7f1198014f10 2024-09-15T21:47:33.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.160+0000 7f11c261f640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f1198014f10 0x7f1198015370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:33.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.161+0000 7f11c261f640 1 --2- 172.21.15.100:0/758802845 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f1198014f10 0x7f1198015370 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:33.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.162+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f119801c970 con 0x7f1198014f10 2024-09-15T21:47:33.188 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:33.187+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:33.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.193+0000 7f119ffff640 1 -- 172.21.15.100:0/758802845 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f119801b7f0 con 0x7f1198014f10 2024-09-15T21:47:33.196 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.195+0000 7f11b27fc640 1 -- 172.21.15.100:0/758802845 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f119801b7f0 con 0x7f1198014f10 2024-09-15T21:47:33.197 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:33.197 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:33.201 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:47:33.201 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:47:33.224 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.223+0000 7f580812a640 1 -- 172.21.15.100:0/1869336093 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f580015d210 con 0x7f580006c490 2024-09-15T21:47:33.235 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:33.235 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:33.235 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:33.235 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:33.235 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.235 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:32 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'18, mlcod=17'14)", 2024-09-15T21:47:33.235 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:33.236 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 78.875106431000006, 2024-09-15T21:47:33.236 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052297760000000002, 2024-09-15T21:47:33.236 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.236 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.236 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.236 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.236 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.236 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.236 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:33.236 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.236 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.237 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.237 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.237 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:33.237 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.237 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.237 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.237 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.237 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:33.237 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.06e-07 2024-09-15T21:47:33.237 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.237 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.237 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.238 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356780+0000", 2024-09-15T21:47:33.238 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4208999999999994e-05 2024-09-15T21:47:33.238 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.238 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.238 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.238 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356789+0000", 2024-09-15T21:47:33.238 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2800000000000003e-06 2024-09-15T21:47:33.238 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.238 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.239 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.239 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356793+0000", 2024-09-15T21:47:33.239 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:47:33.239 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.239 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.239 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.239 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356838+0000", 2024-09-15T21:47:33.239 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4851999999999997e-05 2024-09-15T21:47:33.239 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.239 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.239 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.240 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356854+0000", 2024-09-15T21:47:33.240 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6124999999999999e-05 2024-09-15T21:47:33.240 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.240 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.240 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.240 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.361892+0000", 2024-09-15T21:47:33.240 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050379140000000001 2024-09-15T21:47:33.240 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.240 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.240 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.240 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.361935+0000", 2024-09-15T21:47:33.240 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3217999999999999e-05 2024-09-15T21:47:33.241 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.241 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.241 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.241 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.241 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.241 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:35 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'21, mlcod=17'15)", 2024-09-15T21:47:33.241 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:33.241 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 78.873069301000001, 2024-09-15T21:47:33.241 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055652039999999998, 2024-09-15T21:47:33.241 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.242 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.242 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.242 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.242 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.242 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.242 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:33.242 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.242 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.242 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.242 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.242 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:33.243 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.243 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.243 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.243 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.243 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358744+0000", 2024-09-15T21:47:33.243 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4399999999999998e-07 2024-09-15T21:47:33.243 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.243 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.243 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.243 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358773+0000", 2024-09-15T21:47:33.244 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8847e-05 2024-09-15T21:47:33.244 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.244 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.244 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.244 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358783+0000", 2024-09-15T21:47:33.244 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0299000000000001e-05 2024-09-15T21:47:33.244 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.244 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.244 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.244 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358796+0000", 2024-09-15T21:47:33.245 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3499000000000001e-05 2024-09-15T21:47:33.245 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.245 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.245 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.245 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358850+0000", 2024-09-15T21:47:33.245 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3952000000000001e-05 2024-09-15T21:47:33.245 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.245 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.245 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.245 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358872+0000", 2024-09-15T21:47:33.245 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1424999999999999e-05 2024-09-15T21:47:33.246 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.246 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.246 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.246 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364273+0000", 2024-09-15T21:47:33.246 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054011299999999996 2024-09-15T21:47:33.246 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.246 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.246 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.246 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364308+0000", 2024-09-15T21:47:33.246 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5108000000000003e-05 2024-09-15T21:47:33.246 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.247 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.247 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.247 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.247 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.247 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:36 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'22, mlcod=17'15)", 2024-09-15T21:47:33.247 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:33.247 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 78.872275110999993, 2024-09-15T21:47:33.247 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047904530000000001, 2024-09-15T21:47:33.247 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.247 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.247 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.248 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.248 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.248 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.248 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:33.248 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.248 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.248 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.248 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.248 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:33.248 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.248 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.248 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.249 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.249 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:33.249 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6000000000000004e-07 2024-09-15T21:47:33.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.249 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.249 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.249 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359542+0000", 2024-09-15T21:47:33.249 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3510000000000004e-06 2024-09-15T21:47:33.249 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.249 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.249 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.250 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359552+0000", 2024-09-15T21:47:33.250 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0263e-05 2024-09-15T21:47:33.250 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.250 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.250 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.250 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359557+0000", 2024-09-15T21:47:33.250 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0479999999999998e-06 2024-09-15T21:47:33.250 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.250 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.250 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.250 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359593+0000", 2024-09-15T21:47:33.250 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6151e-05 2024-09-15T21:47:33.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.251 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.251 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.251 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359608+0000", 2024-09-15T21:47:33.251 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4609e-05 2024-09-15T21:47:33.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.251 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.251 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.251 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364310+0000", 2024-09-15T21:47:33.251 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047025510000000001 2024-09-15T21:47:33.251 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.251 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.252 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.252 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364327+0000", 2024-09-15T21:47:33.252 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6920000000000001e-05 2024-09-15T21:47:33.252 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.252 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.252 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.252 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.252 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.252 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=18'85)", 2024-09-15T21:47:33.252 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:33.252 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 77.333729176000006, 2024-09-15T21:47:33.252 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057443499999999996, 2024-09-15T21:47:33.253 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.253 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.253 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.253 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.253 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.253 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.253 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:33.253 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.253 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.254 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.254 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.254 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:33.254 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.254 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.254 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.254 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.254 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:33.254 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8899999999999999e-07 2024-09-15T21:47:33.254 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.255 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.255 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.255 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898167+0000", 2024-09-15T21:47:33.255 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3171999999999996e-05 2024-09-15T21:47:33.255 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.255 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.255 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.255 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898177+0000", 2024-09-15T21:47:33.255 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0866000000000001e-05 2024-09-15T21:47:33.255 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.255 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.255 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.256 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898185+0000", 2024-09-15T21:47:33.256 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2490000000000004e-06 2024-09-15T21:47:33.257 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.257 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.257 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.257 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898224+0000", 2024-09-15T21:47:33.257 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9237999999999999e-05 2024-09-15T21:47:33.257 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.257 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.257 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.257 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898238+0000", 2024-09-15T21:47:33.257 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.366e-05 2024-09-15T21:47:33.257 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.257 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.258 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.258 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.903803+0000", 2024-09-15T21:47:33.258 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005565291 2024-09-15T21:47:33.258 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.258 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.258 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.258 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.903827+0000", 2024-09-15T21:47:33.258 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4284999999999999e-05 2024-09-15T21:47:33.258 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.258 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.258 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.258 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.258 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.258 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:132 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'90, mlcod=18'85)", 2024-09-15T21:47:33.258 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:33.259 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 77.333237452000006, 2024-09-15T21:47:33.259 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055085469999999999, 2024-09-15T21:47:33.259 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.259 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.259 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.259 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.259 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.259 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.259 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:33.259 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.259 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.259 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.259 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.259 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:33.260 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.260 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.260 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.260 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:33.260 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9299999999999999e-07 2024-09-15T21:47:33.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.260 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.260 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.260 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898579+0000", 2024-09-15T21:47:33.260 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6650000000000001e-06 2024-09-15T21:47:33.260 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.260 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.260 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.261 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898586+0000", 2024-09-15T21:47:33.261 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8429999999999993e-06 2024-09-15T21:47:33.261 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.261 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.261 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898592+0000", 2024-09-15T21:47:33.261 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3990000000000003e-06 2024-09-15T21:47:33.261 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.261 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.261 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898621+0000", 2024-09-15T21:47:33.261 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9019000000000001e-05 2024-09-15T21:47:33.261 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.261 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.262 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.262 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898630+0000", 2024-09-15T21:47:33.262 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5540000000000003e-06 2024-09-15T21:47:33.262 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.262 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.262 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.262 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904047+0000", 2024-09-15T21:47:33.262 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054167169999999997 2024-09-15T21:47:33.262 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.262 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.262 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.262 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904083+0000", 2024-09-15T21:47:33.262 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5957000000000001e-05 2024-09-15T21:47:33.262 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.263 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.263 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.263 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.263 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.263 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-15T21:47:33.263 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:33.263 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 77.332648302999999, 2024-09-15T21:47:33.263 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049424780000000001, 2024-09-15T21:47:33.263 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.263 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.263 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.263 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.263 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.263 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.263 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:33.264 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.264 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.264 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.264 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.264 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:33.264 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.264 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.264 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.264 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.264 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899166+0000", 2024-09-15T21:47:33.264 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0779999999999998e-06 2024-09-15T21:47:33.264 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.264 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.264 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.265 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899174+0000", 2024-09-15T21:47:33.265 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.729e-06 2024-09-15T21:47:33.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.265 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.265 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899181+0000", 2024-09-15T21:47:33.265 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4080000000000001e-06 2024-09-15T21:47:33.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.265 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.265 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899185+0000", 2024-09-15T21:47:33.265 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0840000000000002e-06 2024-09-15T21:47:33.265 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.265 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.266 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.266 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899240+0000", 2024-09-15T21:47:33.266 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4874999999999999e-05 2024-09-15T21:47:33.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.266 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.266 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.266 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899257+0000", 2024-09-15T21:47:33.266 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7549000000000001e-05 2024-09-15T21:47:33.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.266 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.266 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.266 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904085+0000", 2024-09-15T21:47:33.266 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048272640000000004 2024-09-15T21:47:33.266 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.266 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.267 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.267 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904106+0000", 2024-09-15T21:47:33.267 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1491000000000001e-05 2024-09-15T21:47:33.267 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.267 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.267 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.267 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.267 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.267 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:33.267 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:33.268 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 64.818425027000004, 2024-09-15T21:47:33.268 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046608459999999997, 2024-09-15T21:47:33.268 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.268 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.268 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.268 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.268 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.268 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.268 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:33.268 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.268 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.268 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.268 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.269 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:33.269 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.269 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.269 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.269 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413391+0000", 2024-09-15T21:47:33.269 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9899999999999999e-06 2024-09-15T21:47:33.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.269 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.269 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.269 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413399+0000", 2024-09-15T21:47:33.269 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6310000000000003e-06 2024-09-15T21:47:33.269 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.269 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.270 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.270 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413444+0000", 2024-09-15T21:47:33.270 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5052000000000002e-05 2024-09-15T21:47:33.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.270 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.270 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.270 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413453+0000", 2024-09-15T21:47:33.270 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8869999999999998e-06 2024-09-15T21:47:33.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.270 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.270 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.270 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413576+0000", 2024-09-15T21:47:33.270 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012358700000000001 2024-09-15T21:47:33.270 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.271 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.271 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.271 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413679+0000", 2024-09-15T21:47:33.271 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000103318 2024-09-15T21:47:33.271 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.271 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.271 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:33.271 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413785+0000", 2024-09-15T21:47:33.271 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010561 2024-09-15T21:47:33.271 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.271 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.271 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.271 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416276+0000", 2024-09-15T21:47:33.271 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002490754 2024-09-15T21:47:33.271 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.272 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.272 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.272 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417949+0000", 2024-09-15T21:47:33.272 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016733519999999999 2024-09-15T21:47:33.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.272 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.272 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.272 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418005+0000", 2024-09-15T21:47:33.272 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5747e-05 2024-09-15T21:47:33.272 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.272 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.272 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.272 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418048+0000", 2024-09-15T21:47:33.272 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2917999999999998e-05 2024-09-15T21:47:33.273 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.273 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.273 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.273 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.273 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.273 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:33.273 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:33.273 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 64.818410556999993, 2024-09-15T21:47:33.273 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051419999999999999, 2024-09-15T21:47:33.273 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.273 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.273 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.273 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.273 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.274 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.274 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:33.274 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.274 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.274 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.274 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:33.274 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.274 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.274 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.274 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413402+0000", 2024-09-15T21:47:33.274 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6899999999999998e-07 2024-09-15T21:47:33.274 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.275 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.275 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.275 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413405+0000", 2024-09-15T21:47:33.275 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8109999999999999e-06 2024-09-15T21:47:33.275 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.275 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.275 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.275 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413517+0000", 2024-09-15T21:47:33.275 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000112592 2024-09-15T21:47:33.275 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.275 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.275 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.276 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413527+0000", 2024-09-15T21:47:33.276 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0144e-05 2024-09-15T21:47:33.276 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.276 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.276 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.276 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414155+0000", 2024-09-15T21:47:33.276 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00062767200000000002 2024-09-15T21:47:33.276 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.276 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.276 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.276 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414239+0000", 2024-09-15T21:47:33.276 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3607000000000001e-05 2024-09-15T21:47:33.277 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.277 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.277 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:33.277 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414368+0000", 2024-09-15T21:47:33.277 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000129049 2024-09-15T21:47:33.277 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.277 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.277 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.277 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416332+0000", 2024-09-15T21:47:33.277 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019645109999999999 2024-09-15T21:47:33.277 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.277 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.278 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.278 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418446+0000", 2024-09-15T21:47:33.278 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021137640000000002 2024-09-15T21:47:33.278 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.278 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.278 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.278 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418503+0000", 2024-09-15T21:47:33.278 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7166999999999997e-05 2024-09-15T21:47:33.278 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.278 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.278 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.278 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418543+0000", 2024-09-15T21:47:33.278 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0314000000000001e-05 2024-09-15T21:47:33.278 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.279 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.279 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.279 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.279 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.279 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:33.279 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:33.279 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 64.818397365999999, 2024-09-15T21:47:33.279 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057945979999999998, 2024-09-15T21:47:33.279 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.279 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.279 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.279 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.279 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.279 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.280 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:33.280 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.280 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.280 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.280 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.280 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:33.280 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.280 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.280 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.280 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.280 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:33.280 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9499999999999998e-07 2024-09-15T21:47:33.280 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.280 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.281 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.281 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413417+0000", 2024-09-15T21:47:33.281 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3910000000000001e-06 2024-09-15T21:47:33.281 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.281 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.281 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.281 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414570+0000", 2024-09-15T21:47:33.281 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011525890000000001 2024-09-15T21:47:33.281 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.281 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.281 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.281 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414577+0000", 2024-09-15T21:47:33.281 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1840000000000002e-06 2024-09-15T21:47:33.281 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.282 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.282 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414733+0000", 2024-09-15T21:47:33.282 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015563900000000001 2024-09-15T21:47:33.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.282 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.282 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414831+0000", 2024-09-15T21:47:33.282 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8138000000000001e-05 2024-09-15T21:47:33.282 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.282 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.282 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:33.282 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414956+0000", 2024-09-15T21:47:33.283 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000124604 2024-09-15T21:47:33.283 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.283 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.283 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.283 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418804+0000", 2024-09-15T21:47:33.283 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038484299999999999 2024-09-15T21:47:33.283 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.283 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.283 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.283 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419124+0000", 2024-09-15T21:47:33.283 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00032017699999999999 2024-09-15T21:47:33.283 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.283 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.283 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.284 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419171+0000", 2024-09-15T21:47:33.284 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6520000000000002e-05 2024-09-15T21:47:33.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.284 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.284 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.284 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419209+0000", 2024-09-15T21:47:33.284 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8531000000000001e-05 2024-09-15T21:47:33.284 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.284 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.284 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.284 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.284 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.284 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:33.284 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:33.285 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 64.818390269999995, 2024-09-15T21:47:33.285 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059183300000000003, 2024-09-15T21:47:33.285 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.285 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.285 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:33.285 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.285 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.285 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.285 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:33.285 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.285 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.285 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.285 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.285 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:33.286 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.286 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.286 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:33.286 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3000000000000002e-07 2024-09-15T21:47:33.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.286 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.286 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413426+0000", 2024-09-15T21:47:33.286 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7110000000000001e-06 2024-09-15T21:47:33.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.287 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.287 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414621+0000", 2024-09-15T21:47:33.287 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001195022 2024-09-15T21:47:33.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.287 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.287 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.287 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414625+0000", 2024-09-15T21:47:33.287 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6360000000000002e-06 2024-09-15T21:47:33.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.287 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.287 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.287 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415269+0000", 2024-09-15T21:47:33.287 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00064315399999999995 2024-09-15T21:47:33.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.287 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.288 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.288 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415356+0000", 2024-09-15T21:47:33.288 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7410999999999994e-05 2024-09-15T21:47:33.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.288 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.288 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419340+0000", 2024-09-15T21:47:33.288 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0039840659999999996 2024-09-15T21:47:33.288 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.288 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.288 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.288 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-15T21:47:33.289 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:33.289 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 64.818082679, 2024-09-15T21:47:33.289 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053379719999999999, 2024-09-15T21:47:33.289 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.289 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.289 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.289 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.289 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.289 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.289 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:33.289 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.289 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.289 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.290 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:33.290 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.290 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.290 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.290 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413730+0000", 2024-09-15T21:47:33.290 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1379999999999999e-06 2024-09-15T21:47:33.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.290 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.290 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.290 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413740+0000", 2024-09-15T21:47:33.290 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4830000000000005e-06 2024-09-15T21:47:33.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.290 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.291 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.291 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413751+0000", 2024-09-15T21:47:33.291 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1228e-05 2024-09-15T21:47:33.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.291 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.291 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.291 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413758+0000", 2024-09-15T21:47:33.291 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1289999999999998e-06 2024-09-15T21:47:33.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.291 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.291 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.291 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414038+0000", 2024-09-15T21:47:33.291 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00027935999999999998 2024-09-15T21:47:33.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.292 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.292 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.292 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414055+0000", 2024-09-15T21:47:33.292 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7325000000000001e-05 2024-09-15T21:47:33.292 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.292 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.292 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.292 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419028+0000", 2024-09-15T21:47:33.292 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049730720000000003 2024-09-15T21:47:33.292 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.292 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.292 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.292 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419067+0000", 2024-09-15T21:47:33.292 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9236999999999997e-05 2024-09-15T21:47:33.293 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.293 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.293 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.293 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.293 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.293 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:33.293 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.776159411999998, 2024-09-15T21:47:33.293 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.92695576499999999, 2024-09-15T21:47:33.293 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.293 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.293 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.293 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.293 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.294 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.294 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:33.294 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.294 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.294 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.294 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:33.294 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.294 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.294 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.294 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:33.294 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8500000000000002e-07 2024-09-15T21:47:33.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.295 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.295 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.295 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455655+0000", 2024-09-15T21:47:33.295 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.401e-06 2024-09-15T21:47:33.295 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.295 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.295 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.295 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.380965+0000", 2024-09-15T21:47:33.295 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.92530946000000003 2024-09-15T21:47:33.295 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.295 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.295 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.295 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.380981+0000", 2024-09-15T21:47:33.295 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6209e-05 2024-09-15T21:47:33.296 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.296 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.296 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.296 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381086+0000", 2024-09-15T21:47:33.296 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105011 2024-09-15T21:47:33.296 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.296 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.296 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.296 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381154+0000", 2024-09-15T21:47:33.296 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7596999999999997e-05 2024-09-15T21:47:33.296 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.296 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.296 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:33.297 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381249+0000", 2024-09-15T21:47:33.297 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5354999999999998e-05 2024-09-15T21:47:33.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.297 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.297 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.297 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381844+0000", 2024-09-15T21:47:33.297 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00059518600000000004 2024-09-15T21:47:33.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.297 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.297 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.297 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382531+0000", 2024-09-15T21:47:33.297 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00068671499999999996 2024-09-15T21:47:33.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.297 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.298 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.298 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382570+0000", 2024-09-15T21:47:33.298 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9373999999999999e-05 2024-09-15T21:47:33.298 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.298 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.298 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.298 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382608+0000", 2024-09-15T21:47:33.298 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8071999999999997e-05 2024-09-15T21:47:33.298 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.298 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.298 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.298 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.298 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.299 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.299 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:33.299 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 52.756250153000003, 2024-09-15T21:47:33.299 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047117319999999997, 2024-09-15T21:47:33.299 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.299 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.299 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.299 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.299 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.299 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.299 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:33.299 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.299 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.299 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.300 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.300 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:33.300 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.300 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.300 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.300 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:33.300 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4700000000000001e-07 2024-09-15T21:47:33.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.300 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.300 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.300 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475566+0000", 2024-09-15T21:47:33.300 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1669999999999999e-06 2024-09-15T21:47:33.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.301 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.301 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476316+0000", 2024-09-15T21:47:33.301 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075022999999999995 2024-09-15T21:47:33.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.301 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.301 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476324+0000", 2024-09-15T21:47:33.301 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.259e-06 2024-09-15T21:47:33.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.301 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.302 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476373+0000", 2024-09-15T21:47:33.302 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8807999999999998e-05 2024-09-15T21:47:33.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.302 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.302 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.302 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476446+0000", 2024-09-15T21:47:33.302 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3138000000000003e-05 2024-09-15T21:47:33.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.302 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.302 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:33.302 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476593+0000", 2024-09-15T21:47:33.302 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00014680199999999999 2024-09-15T21:47:33.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.302 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480189+0000", 2024-09-15T21:47:33.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035961370000000001 2024-09-15T21:47:33.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480198+0000", 2024-09-15T21:47:33.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.6649999999999992e-06 2024-09-15T21:47:33.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480235+0000", 2024-09-15T21:47:33.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6943999999999997e-05 2024-09-15T21:47:33.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.304 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.304 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480274+0000", 2024-09-15T21:47:33.304 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9035000000000001e-05 2024-09-15T21:47:33.304 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.304 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.304 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.304 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.304 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.304 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:33.304 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 52.756245149999998, 2024-09-15T21:47:33.304 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046830109999999999, 2024-09-15T21:47:33.304 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.304 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.305 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.305 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.305 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.305 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:33.305 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.305 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.305 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:33.305 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.305 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.306 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:33.306 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.53e-07 2024-09-15T21:47:33.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.306 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.306 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475570+0000", 2024-09-15T21:47:33.306 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7470000000000001e-06 2024-09-15T21:47:33.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.306 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.306 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476353+0000", 2024-09-15T21:47:33.306 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00078312300000000004 2024-09-15T21:47:33.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.307 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476359+0000", 2024-09-15T21:47:33.307 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.117e-06 2024-09-15T21:47:33.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.307 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476412+0000", 2024-09-15T21:47:33.307 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2398999999999998e-05 2024-09-15T21:47:33.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.307 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476524+0000", 2024-09-15T21:47:33.307 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011293 2024-09-15T21:47:33.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.308 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:33.308 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476909+0000", 2024-09-15T21:47:33.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00038490099999999999 2024-09-15T21:47:33.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.308 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.308 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479761+0000", 2024-09-15T21:47:33.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002851737 2024-09-15T21:47:33.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.308 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.308 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480154+0000", 2024-09-15T21:47:33.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00039307399999999998 2024-09-15T21:47:33.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.309 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.309 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.309 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480203+0000", 2024-09-15T21:47:33.309 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8418000000000003e-05 2024-09-15T21:47:33.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.309 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.309 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.309 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480250+0000", 2024-09-15T21:47:33.309 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7311999999999997e-05 2024-09-15T21:47:33.309 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.309 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.309 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.309 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.310 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.310 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:33.310 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 52.756240988000002, 2024-09-15T21:47:33.310 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005035398, 2024-09-15T21:47:33.310 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.310 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.310 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.310 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.310 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.310 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.310 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:33.310 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.310 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.310 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.311 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.311 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:33.311 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.311 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.311 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.311 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:33.311 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.22e-07 2024-09-15T21:47:33.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.311 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.311 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.311 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475574+0000", 2024-09-15T21:47:33.311 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.593e-06 2024-09-15T21:47:33.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.312 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476579+0000", 2024-09-15T21:47:33.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010053989999999999 2024-09-15T21:47:33.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.312 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476588+0000", 2024-09-15T21:47:33.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4530000000000002e-06 2024-09-15T21:47:33.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.312 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477723+0000", 2024-09-15T21:47:33.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001134962 2024-09-15T21:47:33.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.313 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477801+0000", 2024-09-15T21:47:33.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8228999999999999e-05 2024-09-15T21:47:33.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.313 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:33.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477946+0000", 2024-09-15T21:47:33.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00014553400000000001 2024-09-15T21:47:33.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.313 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.314 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480258+0000", 2024-09-15T21:47:33.314 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023112039999999999 2024-09-15T21:47:33.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.314 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.314 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.314 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480504+0000", 2024-09-15T21:47:33.314 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024667500000000002 2024-09-15T21:47:33.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.314 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.314 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.315 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480556+0000", 2024-09-15T21:47:33.315 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2114999999999998e-05 2024-09-15T21:47:33.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.315 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.315 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480606+0000", 2024-09-15T21:47:33.315 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0012000000000002e-05 2024-09-15T21:47:33.315 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.315 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.315 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.315 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.315 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:33.316 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 52.756237057999996, 2024-09-15T21:47:33.316 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050672490000000002, 2024-09-15T21:47:33.316 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.316 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.316 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.316 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.316 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.316 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.316 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:33.316 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.316 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.316 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.316 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:33.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:33.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:33.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475577+0000", 2024-09-15T21:47:33.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2290000000000002e-06 2024-09-15T21:47:33.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.318 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.318 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476605+0000", 2024-09-15T21:47:33.318 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010278780000000001 2024-09-15T21:47:33.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.318 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.318 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476611+0000", 2024-09-15T21:47:33.318 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5600000000000001e-06 2024-09-15T21:47:33.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.318 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.318 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477053+0000", 2024-09-15T21:47:33.318 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044162800000000001 2024-09-15T21:47:33.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.319 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.319 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477180+0000", 2024-09-15T21:47:33.319 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000127129 2024-09-15T21:47:33.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.319 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:33.319 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477288+0000", 2024-09-15T21:47:33.319 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010789999999999999 2024-09-15T21:47:33.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.319 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.319 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480288+0000", 2024-09-15T21:47:33.320 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003000052 2024-09-15T21:47:33.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.320 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.320 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.320 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480568+0000", 2024-09-15T21:47:33.320 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028026899999999998 2024-09-15T21:47:33.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.320 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.320 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.320 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480609+0000", 2024-09-15T21:47:33.320 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1437999999999999e-05 2024-09-15T21:47:33.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.320 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.320 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.321 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480642+0000", 2024-09-15T21:47:33.321 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2879999999999997e-05 2024-09-15T21:47:33.321 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.321 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.321 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.321 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.321 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:20 2.1f 2:fdd45aa0:::61c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.321 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:33.321 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 52.756233387000002, 2024-09-15T21:47:33.321 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053112589999999996, 2024-09-15T21:47:33.321 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.321 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.322 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.322 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.322 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.322 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.322 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:33.322 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.322 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.322 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.322 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.322 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:33.322 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.322 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.322 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.322 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.322 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:33.323 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:33.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.323 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.323 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475581+0000", 2024-09-15T21:47:33.323 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1009999999999999e-06 2024-09-15T21:47:33.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.323 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.323 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476628+0000", 2024-09-15T21:47:33.323 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010465380000000001 2024-09-15T21:47:33.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.323 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.324 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476633+0000", 2024-09-15T21:47:33.324 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9869999999999999e-06 2024-09-15T21:47:33.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.324 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.324 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.324 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477618+0000", 2024-09-15T21:47:33.324 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00098539299999999999 2024-09-15T21:47:33.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.324 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.324 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.324 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477696+0000", 2024-09-15T21:47:33.324 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8509000000000003e-05 2024-09-15T21:47:33.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.325 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:33.325 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477824+0000", 2024-09-15T21:47:33.325 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000127062 2024-09-15T21:47:33.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.325 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.325 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480294+0000", 2024-09-15T21:47:33.325 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024703939999999999 2024-09-15T21:47:33.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.325 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.325 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480788+0000", 2024-09-15T21:47:33.326 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00049443800000000004 2024-09-15T21:47:33.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.326 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.326 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.326 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480845+0000", 2024-09-15T21:47:33.326 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6762000000000003e-05 2024-09-15T21:47:33.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.326 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.326 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.326 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480890+0000", 2024-09-15T21:47:33.326 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4789000000000001e-05 2024-09-15T21:47:33.326 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.326 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.326 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.327 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.327 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.327 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:33.327 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 52.756229916000002, 2024-09-15T21:47:33.327 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007191546, 2024-09-15T21:47:33.327 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.327 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.327 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:33.327 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.327 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.327 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.327 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:33.327 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.327 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.328 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.328 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.328 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:33.328 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.328 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.328 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.328 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.328 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:33.328 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9999999999999999e-07 2024-09-15T21:47:33.328 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.328 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.328 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.328 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475585+0000", 2024-09-15T21:47:33.328 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.384e-06 2024-09-15T21:47:33.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.329 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.329 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.329 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476665+0000", 2024-09-15T21:47:33.329 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010797140000000001 2024-09-15T21:47:33.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.329 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.329 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.329 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476671+0000", 2024-09-15T21:47:33.329 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3940000000000001e-06 2024-09-15T21:47:33.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.329 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.329 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.329 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478048+0000", 2024-09-15T21:47:33.330 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013767530000000001 2024-09-15T21:47:33.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.330 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.330 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478177+0000", 2024-09-15T21:47:33.330 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012898099999999999 2024-09-15T21:47:33.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.330 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.330 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482774+0000", 2024-09-15T21:47:33.330 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045970200000000003 2024-09-15T21:47:33.330 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.330 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.330 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.331 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.331 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.331 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:33.331 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 47.687000251000001, 2024-09-15T21:47:33.331 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055444480000000004, 2024-09-15T21:47:33.331 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.331 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.331 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.331 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.331 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.331 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.331 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:33.331 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.332 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.332 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.332 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:33.332 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.332 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.332 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.332 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:33.332 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4200000000000002e-07 2024-09-15T21:47:33.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.332 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.332 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.332 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544814+0000", 2024-09-15T21:47:33.332 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9980000000000002e-06 2024-09-15T21:47:33.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.333 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.333 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.333 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544872+0000", 2024-09-15T21:47:33.333 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8405000000000001e-05 2024-09-15T21:47:33.333 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.333 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.333 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.333 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544877+0000", 2024-09-15T21:47:33.333 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.143e-06 2024-09-15T21:47:33.333 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.333 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.333 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.333 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545750+0000", 2024-09-15T21:47:33.333 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00087310000000000003 2024-09-15T21:47:33.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.334 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.334 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.334 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545850+0000", 2024-09-15T21:47:33.334 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9939999999999995e-05 2024-09-15T21:47:33.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.334 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.334 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:33.334 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546014+0000", 2024-09-15T21:47:33.334 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00016408500000000001 2024-09-15T21:47:33.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.334 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.334 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.334 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.548748+0000", 2024-09-15T21:47:33.334 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027339539999999998 2024-09-15T21:47:33.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550265+0000", 2024-09-15T21:47:33.335 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015170019999999999 2024-09-15T21:47:33.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550315+0000", 2024-09-15T21:47:33.335 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9867999999999998e-05 2024-09-15T21:47:33.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550356+0000", 2024-09-15T21:47:33.335 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1711000000000001e-05 2024-09-15T21:47:33.336 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.336 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.336 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.336 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.336 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:33.336 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 47.686997050000002, 2024-09-15T21:47:33.336 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058729619999999998, 2024-09-15T21:47:33.336 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.336 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.336 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.336 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.337 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.337 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.337 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:33.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.337 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.337 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.337 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:33.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.337 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.337 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.337 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:33.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0599999999999999e-07 2024-09-15T21:47:33.338 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.338 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.338 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.338 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544821+0000", 2024-09-15T21:47:33.338 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4079999999999997e-06 2024-09-15T21:47:33.338 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.338 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.338 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.338 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544939+0000", 2024-09-15T21:47:33.338 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118532 2024-09-15T21:47:33.338 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.338 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.338 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.338 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544946+0000", 2024-09-15T21:47:33.339 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9260000000000003e-06 2024-09-15T21:47:33.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.339 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.339 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.339 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545011+0000", 2024-09-15T21:47:33.339 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4572000000000005e-05 2024-09-15T21:47:33.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.339 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.339 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.339 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545134+0000", 2024-09-15T21:47:33.339 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012309899999999999 2024-09-15T21:47:33.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.339 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.339 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:33.339 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545271+0000", 2024-09-15T21:47:33.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013746599999999999 2024-09-15T21:47:33.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.340 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.340 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550243+0000", 2024-09-15T21:47:33.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049714529999999998 2024-09-15T21:47:33.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.340 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.340 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550605+0000", 2024-09-15T21:47:33.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00036260699999999998 2024-09-15T21:47:33.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.341 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.341 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550662+0000", 2024-09-15T21:47:33.341 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6573000000000002e-05 2024-09-15T21:47:33.341 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.341 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.341 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.341 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550688+0000", 2024-09-15T21:47:33.341 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6120000000000001e-05 2024-09-15T21:47:33.341 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.341 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.341 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.341 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.341 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.341 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:33.343 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:47:33.343 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:33.341+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:33.444 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:33.443+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:33.511 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:33.511 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:33.511 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:33.511 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:33.520 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:47:33.545 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:33.543+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:33.646 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:33.644+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:33.651 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:33.651 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:33.651 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:33.651 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:33.651 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:33.663 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:47:33.746 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:33.745+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:33.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.777+0000 7f57f27fc640 1 -- 172.21.15.100:0/1869336093 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v37) ==== 91+0+0 (secure 0 0 0) 0x7f57f405d990 con 0x7f580006c490 2024-09-15T21:47:33.780 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is unset 2024-09-15T21:47:33.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.779+0000 7f580812a640 1 -- 172.21.15.100:0/1869336093 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f57e406c2e0 msgr2=0x7f57e406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:33.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.780+0000 7f580812a640 1 --2- 172.21.15.100:0/1869336093 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f57e406c2e0 0x7f57e406e7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f580015d1e0 tx=0x7f57fc009040 comp rx=0 tx=0).stop 2024-09-15T21:47:33.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.780+0000 7f580812a640 1 -- 172.21.15.100:0/1869336093 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f580006c490 msgr2=0x7f5800157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:33.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.780+0000 7f580812a640 1 --2- 172.21.15.100:0/1869336093 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f580006c490 0x7f5800157cf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f57f400e930 tx=0x7f57f400edf0 comp rx=0 tx=0).stop 2024-09-15T21:47:33.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.780+0000 7f580812a640 1 -- 172.21.15.100:0/1869336093 shutdown_connections 2024-09-15T21:47:33.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.780+0000 7f580812a640 1 --2- 172.21.15.100:0/1869336093 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f57e406c2e0 0x7f57e406e7a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:33.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.780+0000 7f580812a640 1 --2- 172.21.15.100:0/1869336093 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58000c7bd0 0x7f580014bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:33.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.780+0000 7f580812a640 1 --2- 172.21.15.100:0/1869336093 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5800153060 0x7f580014c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:33.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.780+0000 7f580812a640 1 --2- 172.21.15.100:0/1869336093 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f580006c490 0x7f5800157cf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:33.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.780+0000 7f580812a640 1 -- 172.21.15.100:0/1869336093 >> 172.21.15.100:0/1869336093 conn(0x7f58000ca430 msgr2=0x7f58001516c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:33.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.780+0000 7f580812a640 1 -- 172.21.15.100:0/1869336093 shutdown_connections 2024-09-15T21:47:33.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:33.780+0000 7f580812a640 1 -- 172.21.15.100:0/1869336093 wait complete. 2024-09-15T21:47:33.817 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:33.818 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:33.818 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:33.818 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:33.818 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.818 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:33.818 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:33.818 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 65.401490267, 2024-09-15T21:47:33.818 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006101291, 2024-09-15T21:47:33.818 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.818 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.818 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.819 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.819 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.819 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:33.819 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.819 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.819 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.819 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:33.819 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.819 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.819 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.819 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.820 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413116+0000", 2024-09-15T21:47:33.820 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9700000000000003e-07 2024-09-15T21:47:33.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.820 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.820 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.820 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413118+0000", 2024-09-15T21:47:33.820 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2910000000000002e-06 2024-09-15T21:47:33.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.820 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.820 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.820 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413220+0000", 2024-09-15T21:47:33.820 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102056 2024-09-15T21:47:33.820 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.821 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.821 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.821 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413225+0000", 2024-09-15T21:47:33.821 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0490000000000003e-06 2024-09-15T21:47:33.821 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.821 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.821 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.821 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414808+0000", 2024-09-15T21:47:33.821 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015826900000000001 2024-09-15T21:47:33.821 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.821 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.821 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.821 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414907+0000", 2024-09-15T21:47:33.821 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9600999999999997e-05 2024-09-15T21:47:33.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.822 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:33.822 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415054+0000", 2024-09-15T21:47:33.822 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000146532 2024-09-15T21:47:33.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.822 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.822 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418056+0000", 2024-09-15T21:47:33.822 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030020519999999998 2024-09-15T21:47:33.822 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.822 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.822 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.822 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419128+0000", 2024-09-15T21:47:33.823 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001071671 2024-09-15T21:47:33.823 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.823 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.823 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.823 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419184+0000", 2024-09-15T21:47:33.824 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6125000000000002e-05 2024-09-15T21:47:33.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.824 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.824 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.824 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419217+0000", 2024-09-15T21:47:33.824 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2926999999999998e-05 2024-09-15T21:47:33.824 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.824 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.824 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.824 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.824 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.825 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:33.825 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:33.825 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 65.401486740999999, 2024-09-15T21:47:33.825 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059975130000000003, 2024-09-15T21:47:33.825 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.825 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.825 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.825 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.825 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.825 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.825 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:33.825 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.825 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.826 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.826 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.826 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:33.826 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.826 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.826 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.826 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.826 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:33.826 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6199999999999999e-07 2024-09-15T21:47:33.826 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.826 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.826 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.826 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413121+0000", 2024-09-15T21:47:33.826 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2000000000000001e-06 2024-09-15T21:47:33.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.827 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.827 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413292+0000", 2024-09-15T21:47:33.827 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017034400000000001 2024-09-15T21:47:33.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.827 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.827 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413300+0000", 2024-09-15T21:47:33.827 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8539999999999997e-06 2024-09-15T21:47:33.827 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.827 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.827 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.828 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414299+0000", 2024-09-15T21:47:33.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0009992829999999999 2024-09-15T21:47:33.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.828 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.828 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.828 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414390+0000", 2024-09-15T21:47:33.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1237999999999996e-05 2024-09-15T21:47:33.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.828 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.828 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:33.828 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414490+0000", 2024-09-15T21:47:33.828 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9438999999999994e-05 2024-09-15T21:47:33.828 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.829 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.829 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.829 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417393+0000", 2024-09-15T21:47:33.829 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002903306 2024-09-15T21:47:33.829 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.829 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.829 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.829 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419023+0000", 2024-09-15T21:47:33.829 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001630292 2024-09-15T21:47:33.829 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.829 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.829 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.829 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419067+0000", 2024-09-15T21:47:33.829 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4017999999999998e-05 2024-09-15T21:47:33.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.830 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.830 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419116+0000", 2024-09-15T21:47:33.830 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9277000000000003e-05 2024-09-15T21:47:33.830 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.830 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.830 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.830 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.830 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.830 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:33.830 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:33.830 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 65.398788543999999, 2024-09-15T21:47:33.830 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060879280000000003, 2024-09-15T21:47:33.831 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.831 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.831 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.831 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.831 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.831 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.831 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:33.831 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.831 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.831 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.831 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.831 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:33.831 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415818+0000", 2024-09-15T21:47:33.832 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:47:33.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415854+0000", 2024-09-15T21:47:33.832 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6706999999999999e-05 2024-09-15T21:47:33.832 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.832 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.832 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.832 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415860+0000", 2024-09-15T21:47:33.833 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0789999999999997e-06 2024-09-15T21:47:33.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.833 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.833 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415865+0000", 2024-09-15T21:47:33.833 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4580000000000003e-06 2024-09-15T21:47:33.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.833 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.833 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.833 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416092+0000", 2024-09-15T21:47:33.833 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00022739400000000001 2024-09-15T21:47:33.833 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.834 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.834 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.834 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416108+0000", 2024-09-15T21:47:33.834 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.613e-05 2024-09-15T21:47:33.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.834 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.834 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.834 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421866+0000", 2024-09-15T21:47:33.834 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005757585 2024-09-15T21:47:33.834 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.834 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.834 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.834 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421905+0000", 2024-09-15T21:47:33.835 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9214000000000001e-05 2024-09-15T21:47:33.835 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.835 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.835 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.835 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.835 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.835 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-15T21:47:33.835 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:33.835 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 65.398728383999995, 2024-09-15T21:47:33.836 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060637570000000003, 2024-09-15T21:47:33.836 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.836 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.836 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.836 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.836 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.836 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.836 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:33.836 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.837 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.837 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:33.837 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.837 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.837 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415878+0000", 2024-09-15T21:47:33.837 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.34e-07 2024-09-15T21:47:33.837 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.837 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415881+0000", 2024-09-15T21:47:33.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.354e-06 2024-09-15T21:47:33.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415887+0000", 2024-09-15T21:47:33.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:47:33.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.838 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.838 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.838 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415891+0000", 2024-09-15T21:47:33.838 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2690000000000001e-06 2024-09-15T21:47:33.838 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.839 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.839 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416267+0000", 2024-09-15T21:47:33.839 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00037614799999999998 2024-09-15T21:47:33.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.839 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.839 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416288+0000", 2024-09-15T21:47:33.839 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0604000000000001e-05 2024-09-15T21:47:33.839 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.839 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.839 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.839 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421908+0000", 2024-09-15T21:47:33.839 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056203390000000002 2024-09-15T21:47:33.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.840 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.840 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.840 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421941+0000", 2024-09-15T21:47:33.840 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3079e-05 2024-09-15T21:47:33.840 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.840 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.840 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.840 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.840 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.840 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:33.840 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:33.840 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 65.398709956999994, 2024-09-15T21:47:33.841 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006080284, 2024-09-15T21:47:33.841 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.841 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.841 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.841 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.841 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.841 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.841 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:33.841 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.841 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.841 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.841 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.841 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:33.842 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.842 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.842 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.842 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:33.842 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6899999999999999e-07 2024-09-15T21:47:33.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.842 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.842 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.842 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415898+0000", 2024-09-15T21:47:33.842 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.951e-06 2024-09-15T21:47:33.842 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.842 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.842 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.843 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415920+0000", 2024-09-15T21:47:33.843 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1651999999999999e-05 2024-09-15T21:47:33.843 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.843 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.843 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.843 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415925+0000", 2024-09-15T21:47:33.843 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.463e-06 2024-09-15T21:47:33.843 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.843 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.843 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.843 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416449+0000", 2024-09-15T21:47:33.843 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052384000000000003 2024-09-15T21:47:33.843 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.844 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.844 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416466+0000", 2024-09-15T21:47:33.844 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7108999999999998e-05 2024-09-15T21:47:33.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.844 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.844 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421943+0000", 2024-09-15T21:47:33.844 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054771569999999999 2024-09-15T21:47:33.844 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.844 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.844 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.844 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421976+0000", 2024-09-15T21:47:33.844 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2943e-05 2024-09-15T21:47:33.845 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.845 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.845 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.845 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.845 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-15T21:47:33.845 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:33.845 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 65.398670721000002, 2024-09-15T21:47:33.845 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060643340000000002, 2024-09-15T21:47:33.845 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.845 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.845 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.845 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.845 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.846 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.846 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:33.846 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.846 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.846 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:33.846 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.846 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.846 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.846 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:33.846 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4200000000000002e-07 2024-09-15T21:47:33.846 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.847 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.847 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.847 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415939+0000", 2024-09-15T21:47:33.847 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.439e-06 2024-09-15T21:47:33.847 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.847 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.847 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.847 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415945+0000", 2024-09-15T21:47:33.847 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7740000000000001e-06 2024-09-15T21:47:33.847 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.847 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.847 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.847 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415948+0000", 2024-09-15T21:47:33.847 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.737e-06 2024-09-15T21:47:33.848 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.848 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.848 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.848 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416605+0000", 2024-09-15T21:47:33.848 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00065721699999999998 2024-09-15T21:47:33.848 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.848 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.848 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.848 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416620+0000", 2024-09-15T21:47:33.848 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4564e-05 2024-09-15T21:47:33.848 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.848 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.848 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.849 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421977+0000", 2024-09-15T21:47:33.849 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053572369999999999 2024-09-15T21:47:33.849 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.849 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.849 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.849 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421999+0000", 2024-09-15T21:47:33.849 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2024000000000001e-05 2024-09-15T21:47:33.849 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.849 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.849 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.849 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.849 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.849 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-15T21:47:33.849 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:33.850 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 65.398470062000001, 2024-09-15T21:47:33.850 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058922669999999996, 2024-09-15T21:47:33.850 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.850 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.850 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.850 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.850 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.850 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.850 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:33.850 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.850 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.850 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.850 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.850 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:33.851 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.851 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.851 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.851 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:33.851 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6499999999999999e-07 2024-09-15T21:47:33.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.851 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.851 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.851 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416142+0000", 2024-09-15T21:47:33.851 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3929999999999999e-06 2024-09-15T21:47:33.851 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.851 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.851 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.852 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416151+0000", 2024-09-15T21:47:33.852 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5840000000000001e-06 2024-09-15T21:47:33.852 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.852 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.852 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.852 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416157+0000", 2024-09-15T21:47:33.852 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7590000000000003e-06 2024-09-15T21:47:33.852 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.852 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.852 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.852 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416767+0000", 2024-09-15T21:47:33.852 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00061034900000000005 2024-09-15T21:47:33.852 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.853 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.853 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.853 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416782+0000", 2024-09-15T21:47:33.853 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.483e-05 2024-09-15T21:47:33.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.853 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.853 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.853 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422002+0000", 2024-09-15T21:47:33.853 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052202639999999996 2024-09-15T21:47:33.853 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.853 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.853 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.854 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422028+0000", 2024-09-15T21:47:33.854 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5423e-05 2024-09-15T21:47:33.854 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.854 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.854 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.854 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.854 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.854 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-15T21:47:33.854 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:33.854 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 65.398063561000001, 2024-09-15T21:47:33.854 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055107769999999997, 2024-09-15T21:47:33.854 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.854 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.854 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.855 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.855 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.855 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.855 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:33.855 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.855 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.855 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.855 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:33.855 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.855 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.855 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.855 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.855 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416543+0000", 2024-09-15T21:47:33.856 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5400000000000001e-07 2024-09-15T21:47:33.856 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.856 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.856 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.856 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416548+0000", 2024-09-15T21:47:33.856 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8659999999999997e-06 2024-09-15T21:47:33.856 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.856 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.856 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.856 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416555+0000", 2024-09-15T21:47:33.856 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7190000000000001e-06 2024-09-15T21:47:33.856 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.856 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.857 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.857 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416560+0000", 2024-09-15T21:47:33.857 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3270000000000002e-06 2024-09-15T21:47:33.857 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.857 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.857 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.857 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416916+0000", 2024-09-15T21:47:33.857 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00035661200000000002 2024-09-15T21:47:33.857 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.857 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.857 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.857 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416932+0000", 2024-09-15T21:47:33.857 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5824e-05 2024-09-15T21:47:33.857 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.858 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.858 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422030+0000", 2024-09-15T21:47:33.858 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050975229999999996 2024-09-15T21:47:33.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.858 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.858 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422053+0000", 2024-09-15T21:47:33.858 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3252e-05 2024-09-15T21:47:33.858 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.858 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.858 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.858 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.858 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.859 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-15T21:47:33.859 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:33.859 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 65.397665062000002, 2024-09-15T21:47:33.859 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051422289999999999, 2024-09-15T21:47:33.859 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.859 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.859 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.859 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.859 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.859 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.859 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:33.859 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.859 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.859 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.860 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.860 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:33.860 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.860 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.860 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.860 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.860 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:33.860 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2699999999999999e-07 2024-09-15T21:47:33.860 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.860 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.860 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.860 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416946+0000", 2024-09-15T21:47:33.860 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1390000000000003e-06 2024-09-15T21:47:33.861 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.861 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.861 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.861 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416957+0000", 2024-09-15T21:47:33.861 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.044e-05 2024-09-15T21:47:33.861 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.861 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.861 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.861 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416963+0000", 2024-09-15T21:47:33.861 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5320000000000003e-06 2024-09-15T21:47:33.861 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.861 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.861 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.862 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417090+0000", 2024-09-15T21:47:33.862 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126549 2024-09-15T21:47:33.862 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.862 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.862 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.862 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417114+0000", 2024-09-15T21:47:33.862 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3736000000000001e-05 2024-09-15T21:47:33.862 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.862 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.862 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.862 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422055+0000", 2024-09-15T21:47:33.862 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049410410000000002 2024-09-15T21:47:33.862 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.862 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.863 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.863 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422083+0000", 2024-09-15T21:47:33.863 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8164999999999999e-05 2024-09-15T21:47:33.863 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.863 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.863 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.863 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.863 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.863 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.863 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:33.863 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 59.353173089999999, 2024-09-15T21:47:33.863 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007521186, 2024-09-15T21:47:33.863 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.864 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.864 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.864 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.864 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.864 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.864 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:33.864 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.864 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.864 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.864 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:33.864 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.864 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.864 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.865 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.865 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:33.865 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9999999999999998e-07 2024-09-15T21:47:33.865 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.865 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.865 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.865 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461443+0000", 2024-09-15T21:47:33.865 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0448000000000001e-05 2024-09-15T21:47:33.865 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.865 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.865 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.865 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462146+0000", 2024-09-15T21:47:33.865 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00070223499999999995 2024-09-15T21:47:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462154+0000", 2024-09-15T21:47:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9060000000000002e-06 2024-09-15T21:47:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462254+0000", 2024-09-15T21:47:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9897999999999998e-05 2024-09-15T21:47:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.866 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462330+0000", 2024-09-15T21:47:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6551000000000006e-05 2024-09-15T21:47:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462454+0000", 2024-09-15T21:47:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000124213 2024-09-15T21:47:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463681+0000", 2024-09-15T21:47:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012265729999999999 2024-09-15T21:47:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.867 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.868 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468817+0000", 2024-09-15T21:47:33.868 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051361430000000001 2024-09-15T21:47:33.868 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.868 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.868 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.868 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468892+0000", 2024-09-15T21:47:33.868 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5480000000000002e-05 2024-09-15T21:47:33.868 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.868 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.868 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.868 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468954+0000", 2024-09-15T21:47:33.868 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1339000000000003e-05 2024-09-15T21:47:33.868 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 59.353156474999999, 2024-09-15T21:47:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077296149999999996, 2024-09-15T21:47:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.869 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.870 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:33.870 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.870 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.870 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.870 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.870 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:33.870 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.870 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.870 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.870 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.870 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461450+0000", 2024-09-15T21:47:33.870 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0499999999999999e-07 2024-09-15T21:47:33.870 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.871 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.871 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.871 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461452+0000", 2024-09-15T21:47:33.871 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.007e-06 2024-09-15T21:47:33.871 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.871 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.871 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.871 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462209+0000", 2024-09-15T21:47:33.871 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075781000000000004 2024-09-15T21:47:33.871 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.871 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.871 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.871 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462214+0000", 2024-09-15T21:47:33.871 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1420000000000001e-06 2024-09-15T21:47:33.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.872 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.872 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462681+0000", 2024-09-15T21:47:33.872 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046634200000000002 2024-09-15T21:47:33.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.872 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.872 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462772+0000", 2024-09-15T21:47:33.872 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1414000000000003e-05 2024-09-15T21:47:33.872 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.872 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.872 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:33.873 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462871+0000", 2024-09-15T21:47:33.873 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8455000000000005e-05 2024-09-15T21:47:33.873 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.873 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.873 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.873 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464138+0000", 2024-09-15T21:47:33.873 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012673910000000001 2024-09-15T21:47:33.873 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.873 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.873 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.873 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469079+0000", 2024-09-15T21:47:33.873 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049407799999999997 2024-09-15T21:47:33.873 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.874 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.874 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.874 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469141+0000", 2024-09-15T21:47:33.874 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2169999999999996e-05 2024-09-15T21:47:33.874 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.874 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.874 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.874 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469179+0000", 2024-09-15T21:47:33.874 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7799000000000002e-05 2024-09-15T21:47:33.874 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.874 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.874 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.874 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.874 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.875 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.875 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:33.875 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 53.339040666999999, 2024-09-15T21:47:33.875 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004990497, 2024-09-15T21:47:33.875 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.875 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.875 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.875 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.875 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.876 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.876 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:33.876 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.876 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.876 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.876 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.876 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:33.876 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.876 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.876 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.876 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.876 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:33.876 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4729999999999999e-06 2024-09-15T21:47:33.877 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.877 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.877 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.877 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475576+0000", 2024-09-15T21:47:33.877 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1749999999999994e-06 2024-09-15T21:47:33.877 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.877 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.877 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.877 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475619+0000", 2024-09-15T21:47:33.877 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3557000000000003e-05 2024-09-15T21:47:33.877 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.877 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.877 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.878 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475631+0000", 2024-09-15T21:47:33.878 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1739999999999999e-05 2024-09-15T21:47:33.878 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.878 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.878 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.878 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475770+0000", 2024-09-15T21:47:33.878 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013950599999999999 2024-09-15T21:47:33.878 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.878 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.878 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.878 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475887+0000", 2024-09-15T21:47:33.878 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011616 2024-09-15T21:47:33.878 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.879 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.879 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:33.879 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476018+0000", 2024-09-15T21:47:33.879 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013119499999999999 2024-09-15T21:47:33.879 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.879 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.879 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.879 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478442+0000", 2024-09-15T21:47:33.879 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024240870000000001 2024-09-15T21:47:33.880 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.880 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.880 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.880 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480457+0000", 2024-09-15T21:47:33.880 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020151779999999998 2024-09-15T21:47:33.880 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.880 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.880 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.880 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480511+0000", 2024-09-15T21:47:33.880 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3582000000000003e-05 2024-09-15T21:47:33.880 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.881 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.881 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.881 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480556+0000", 2024-09-15T21:47:33.881 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4844e-05 2024-09-15T21:47:33.881 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.881 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.881 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.881 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.881 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.881 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.881 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:33.881 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 53.339026349000001, 2024-09-15T21:47:33.881 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051124810000000003, 2024-09-15T21:47:33.881 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.882 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.882 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.882 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.882 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.882 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.882 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:33.882 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.882 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.882 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.882 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.882 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:33.882 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.882 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.883 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.883 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.883 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475580+0000", 2024-09-15T21:47:33.883 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7800000000000002e-07 2024-09-15T21:47:33.883 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.883 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.883 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.883 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475595+0000", 2024-09-15T21:47:33.883 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4919000000000001e-05 2024-09-15T21:47:33.883 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.883 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.883 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.883 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475662+0000", 2024-09-15T21:47:33.883 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7769999999999997e-05 2024-09-15T21:47:33.884 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.884 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.884 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.884 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475669+0000", 2024-09-15T21:47:33.884 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.703e-06 2024-09-15T21:47:33.884 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.884 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.884 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.884 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476781+0000", 2024-09-15T21:47:33.884 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011118300000000001 2024-09-15T21:47:33.884 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.884 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.884 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.884 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476863+0000", 2024-09-15T21:47:33.885 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2208999999999998e-05 2024-09-15T21:47:33.885 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.885 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.885 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:33.885 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477040+0000", 2024-09-15T21:47:33.885 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000176424 2024-09-15T21:47:33.885 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.885 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.885 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.885 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480154+0000", 2024-09-15T21:47:33.885 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031147589999999999 2024-09-15T21:47:33.885 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.885 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.886 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.886 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480596+0000", 2024-09-15T21:47:33.886 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044172699999999999 2024-09-15T21:47:33.886 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.886 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.886 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.886 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480652+0000", 2024-09-15T21:47:33.886 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6350000000000001e-05 2024-09-15T21:47:33.886 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.886 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.886 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.886 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480692+0000", 2024-09-15T21:47:33.886 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9412000000000001e-05 2024-09-15T21:47:33.886 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.887 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.887 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.887 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.887 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.887 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:6 2.9 2:941d59bc:::61f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.887 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:33.887 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 48.269815817000001, 2024-09-15T21:47:33.887 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057427529999999997, 2024-09-15T21:47:33.887 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.887 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.887 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.887 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.887 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.888 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.888 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:33.888 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.888 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.888 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.888 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:33.888 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.888 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.888 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.888 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:33.888 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.41e-07 2024-09-15T21:47:33.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.889 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.889 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.889 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544792+0000", 2024-09-15T21:47:33.889 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9929999999999998e-06 2024-09-15T21:47:33.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.889 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.890 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.890 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544885+0000", 2024-09-15T21:47:33.890 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2825000000000007e-05 2024-09-15T21:47:33.890 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.890 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.890 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.890 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544891+0000", 2024-09-15T21:47:33.890 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.93e-06 2024-09-15T21:47:33.890 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.890 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.890 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.890 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546766+0000", 2024-09-15T21:47:33.891 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018747569999999999 2024-09-15T21:47:33.891 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.891 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.891 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.891 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546864+0000", 2024-09-15T21:47:33.891 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8603000000000004e-05 2024-09-15T21:47:33.891 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.891 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.891 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:33.891 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547028+0000", 2024-09-15T21:47:33.891 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000163461 2024-09-15T21:47:33.891 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.892 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.892 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.892 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549499+0000", 2024-09-15T21:47:33.892 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024711749999999999 2024-09-15T21:47:33.892 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.892 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.892 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.892 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550417+0000", 2024-09-15T21:47:33.892 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00091817199999999998 2024-09-15T21:47:33.892 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.892 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.893 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.893 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550476+0000", 2024-09-15T21:47:33.893 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8635999999999999e-05 2024-09-15T21:47:33.893 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.893 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.893 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.893 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550533+0000", 2024-09-15T21:47:33.893 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6959999999999997e-05 2024-09-15T21:47:33.893 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.893 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.893 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.893 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.893 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.894 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.894 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:33.894 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 48.269801352000002, 2024-09-15T21:47:33.894 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004917121, 2024-09-15T21:47:33.894 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.894 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.894 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.894 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.894 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.894 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.894 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:33.894 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.894 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.895 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.895 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.895 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:33.895 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.895 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.895 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.895 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.895 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544805+0000", 2024-09-15T21:47:33.895 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4600000000000001e-07 2024-09-15T21:47:33.895 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.895 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.895 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.895 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544817+0000", 2024-09-15T21:47:33.895 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2023e-05 2024-09-15T21:47:33.896 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.896 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.896 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.896 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545000+0000", 2024-09-15T21:47:33.896 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018384400000000001 2024-09-15T21:47:33.896 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.896 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.896 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.896 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545007+0000", 2024-09-15T21:47:33.896 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1249999999999998e-06 2024-09-15T21:47:33.896 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.896 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.896 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.897 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545670+0000", 2024-09-15T21:47:33.897 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00066324999999999999 2024-09-15T21:47:33.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.897 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.897 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.897 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545749+0000", 2024-09-15T21:47:33.897 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9239000000000005e-05 2024-09-15T21:47:33.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.897 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.897 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:33.897 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545885+0000", 2024-09-15T21:47:33.897 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013586199999999999 2024-09-15T21:47:33.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.897 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.898 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.898 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547944+0000", 2024-09-15T21:47:33.898 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020594720000000001 2024-09-15T21:47:33.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.898 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.898 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.898 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549630+0000", 2024-09-15T21:47:33.898 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001685831 2024-09-15T21:47:33.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.898 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.898 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.898 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549681+0000", 2024-09-15T21:47:33.898 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1187000000000003e-05 2024-09-15T21:47:33.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.899 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.899 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.899 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549721+0000", 2024-09-15T21:47:33.899 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0042e-05 2024-09-15T21:47:33.899 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.899 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.899 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.899 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.899 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.899 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.899 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:33.899 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 48.269787604000001, 2024-09-15T21:47:33.900 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050615549999999997, 2024-09-15T21:47:33.900 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.900 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.900 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.900 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.900 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.900 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.900 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:33.900 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.900 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.900 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.900 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.900 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:33.901 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.901 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.901 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.901 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:33.901 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:47:33.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.901 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.901 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.901 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544821+0000", 2024-09-15T21:47:33.901 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4399999999999999e-06 2024-09-15T21:47:33.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.901 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.902 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.902 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545025+0000", 2024-09-15T21:47:33.902 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020379400000000001 2024-09-15T21:47:33.902 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.902 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.902 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.902 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545030+0000", 2024-09-15T21:47:33.902 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3109999999999996e-06 2024-09-15T21:47:33.902 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.902 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.902 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.902 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546196+0000", 2024-09-15T21:47:33.902 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001166261 2024-09-15T21:47:33.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.903 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.903 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.903 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546307+0000", 2024-09-15T21:47:33.903 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000110641 2024-09-15T21:47:33.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.903 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.903 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:33.903 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546422+0000", 2024-09-15T21:47:33.903 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000115399 2024-09-15T21:47:33.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.903 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.903 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.903 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547973+0000", 2024-09-15T21:47:33.904 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001551228 2024-09-15T21:47:33.904 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.904 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.904 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.904 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549815+0000", 2024-09-15T21:47:33.904 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001841685 2024-09-15T21:47:33.904 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.904 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.904 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.904 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549856+0000", 2024-09-15T21:47:33.904 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.091e-05 2024-09-15T21:47:33.904 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.904 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.905 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.905 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549880+0000", 2024-09-15T21:47:33.905 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3580000000000001e-05 2024-09-15T21:47:33.905 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.905 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.905 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.905 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.905 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.905 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.905 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:33.905 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 48.269783596000003, 2024-09-15T21:47:33.905 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056931389999999998, 2024-09-15T21:47:33.905 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.905 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.906 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.906 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.906 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.906 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.906 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:33.906 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.906 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.906 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.906 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:33.906 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.906 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.907 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.907 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:33.907 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:47:33.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.907 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.907 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.907 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544828+0000", 2024-09-15T21:47:33.907 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4369999999999998e-06 2024-09-15T21:47:33.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.907 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.907 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.907 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545041+0000", 2024-09-15T21:47:33.907 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000212695 2024-09-15T21:47:33.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.908 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.908 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545058+0000", 2024-09-15T21:47:33.908 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7717999999999999e-05 2024-09-15T21:47:33.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.908 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.908 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546958+0000", 2024-09-15T21:47:33.908 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018995539999999999 2024-09-15T21:47:33.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.908 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.909 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547057+0000", 2024-09-15T21:47:33.909 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9370999999999994e-05 2024-09-15T21:47:33.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.909 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.909 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:33.909 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547579+0000", 2024-09-15T21:47:33.909 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052198200000000004 2024-09-15T21:47:33.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.909 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.909 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.909 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549929+0000", 2024-09-15T21:47:33.909 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023494649999999998 2024-09-15T21:47:33.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.910 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.910 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.910 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550401+0000", 2024-09-15T21:47:33.910 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00047277300000000001 2024-09-15T21:47:33.910 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.910 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.910 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.910 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550458+0000", 2024-09-15T21:47:33.910 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6756999999999999e-05 2024-09-15T21:47:33.910 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.910 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.910 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.910 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550515+0000", 2024-09-15T21:47:33.910 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7093999999999999e-05 2024-09-15T21:47:33.911 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.911 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.911 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.911 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.911 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:26 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.911 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:33.911 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 48.269776939000003, 2024-09-15T21:47:33.911 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050112029999999997, 2024-09-15T21:47:33.911 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.911 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.911 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.911 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.912 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.912 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.912 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:33.912 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.912 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.912 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.912 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.912 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:33.912 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.912 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.912 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.912 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.912 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:33.913 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.36e-07 2024-09-15T21:47:33.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.913 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.913 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.913 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545086+0000", 2024-09-15T21:47:33.913 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025704699999999999 2024-09-15T21:47:33.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.913 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.913 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.913 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545119+0000", 2024-09-15T21:47:33.913 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2734999999999997e-05 2024-09-15T21:47:33.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.914 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.914 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.914 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545126+0000", 2024-09-15T21:47:33.914 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5599999999999996e-06 2024-09-15T21:47:33.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.914 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.914 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.914 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545253+0000", 2024-09-15T21:47:33.914 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012644500000000001 2024-09-15T21:47:33.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.914 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.914 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.914 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545366+0000", 2024-09-15T21:47:33.915 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011349999999999999 2024-09-15T21:47:33.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.915 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:33.915 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545673+0000", 2024-09-15T21:47:33.915 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030643599999999999 2024-09-15T21:47:33.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.915 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.915 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547924+0000", 2024-09-15T21:47:33.916 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0022516189999999998 2024-09-15T21:47:33.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.916 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.916 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.916 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549746+0000", 2024-09-15T21:47:33.916 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018217629999999999 2024-09-15T21:47:33.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.916 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.916 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.916 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549800+0000", 2024-09-15T21:47:33.916 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3412999999999998e-05 2024-09-15T21:47:33.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.917 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.917 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549840+0000", 2024-09-15T21:47:33.917 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0448999999999998e-05 2024-09-15T21:47:33.917 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.917 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.917 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.917 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:30 2.4 2:218afd09:::203.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.917 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:33.917 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 48.269516109999998, 2024-09-15T21:47:33.917 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055481230000000003, 2024-09-15T21:47:33.918 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.918 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.918 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:33.918 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.918 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.918 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.918 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:33.918 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.918 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.918 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.918 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.918 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:33.918 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.918 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.919 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.919 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545091+0000", 2024-09-15T21:47:33.919 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7000000000000003e-07 2024-09-15T21:47:33.919 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.919 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.919 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545094+0000", 2024-09-15T21:47:33.919 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9569999999999998e-06 2024-09-15T21:47:33.919 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.919 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.919 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545156+0000", 2024-09-15T21:47:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2277e-05 2024-09-15T21:47:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545162+0000", 2024-09-15T21:47:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9120000000000003e-06 2024-09-15T21:47:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545262+0000", 2024-09-15T21:47:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010001900000000001 2024-09-15T21:47:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.920 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545388+0000", 2024-09-15T21:47:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126243 2024-09-15T21:47:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550638+0000", 2024-09-15T21:47:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052497450000000001 2024-09-15T21:47:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.921 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:10 2.9 2:954a9a2b:::62b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.249832650000002, 2024-09-15T21:47:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048438860000000004, 2024-09-15T21:47:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:33.922 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0400000000000002e-07 2024-09-15T21:47:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.923 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564786+0000", 2024-09-15T21:47:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2412000000000001e-05 2024-09-15T21:47:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564991+0000", 2024-09-15T21:47:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020483900000000001 2024-09-15T21:47:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564998+0000", 2024-09-15T21:47:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2459999999999997e-06 2024-09-15T21:47:33.924 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565422+0000", 2024-09-15T21:47:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00042420100000000003 2024-09-15T21:47:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565515+0000", 2024-09-15T21:47:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2449999999999997e-05 2024-09-15T21:47:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565642+0000", 2024-09-15T21:47:33.925 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012743699999999999 2024-09-15T21:47:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566656+0000", 2024-09-15T21:47:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010138580000000001 2024-09-15T21:47:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569552+0000", 2024-09-15T21:47:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028966819999999998 2024-09-15T21:47:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.926 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569594+0000", 2024-09-15T21:47:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1235000000000001e-05 2024-09-15T21:47:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569617+0000", 2024-09-15T21:47:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3221999999999999e-05 2024-09-15T21:47:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:33.927 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:33.927 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:33.929 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:47:33.930 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:33.927+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:34.029 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:34.028+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:34.109 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:34.109 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:34.109 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:34.109 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:34.120 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:47:34.130 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:34.128+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:34.226 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:34.226 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:34.226 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:34.226 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:34.226 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:34.231 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:34.229+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:34.235 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:47:34.332 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:34.331+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:34.388 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:34.388 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:34.388 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:34.388 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:34.388 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.388 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:34.388 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:34.388 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 65.971929642000006, 2024-09-15T21:47:34.388 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00598016, 2024-09-15T21:47:34.389 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.389 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.389 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:34.389 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.389 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.389 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.389 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:34.389 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.389 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.390 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.390 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:34.390 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.390 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.390 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.390 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.390 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413339+0000", 2024-09-15T21:47:34.390 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6279999999999998e-06 2024-09-15T21:47:34.390 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.390 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.390 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.390 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413354+0000", 2024-09-15T21:47:34.391 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4877e-05 2024-09-15T21:47:34.391 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.391 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.391 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.391 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413418+0000", 2024-09-15T21:47:34.391 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4274000000000002e-05 2024-09-15T21:47:34.391 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.391 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.391 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.392 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413428+0000", 2024-09-15T21:47:34.392 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5899999999999997e-06 2024-09-15T21:47:34.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.392 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.392 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.392 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413563+0000", 2024-09-15T21:47:34.392 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013521900000000001 2024-09-15T21:47:34.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.393 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.393 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.393 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413678+0000", 2024-09-15T21:47:34.393 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000114904 2024-09-15T21:47:34.393 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.393 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.393 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:34.393 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413789+0000", 2024-09-15T21:47:34.393 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011069300000000001 2024-09-15T21:47:34.393 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.393 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.393 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:34.393 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417603+0000", 2024-09-15T21:47:34.393 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038138460000000001 2024-09-15T21:47:34.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.394 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:34.394 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419220+0000", 2024-09-15T21:47:34.394 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001617678 2024-09-15T21:47:34.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.394 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:34.394 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419275+0000", 2024-09-15T21:47:34.394 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4391999999999997e-05 2024-09-15T21:47:34.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.394 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.394 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419316+0000", 2024-09-15T21:47:34.395 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1059000000000001e-05 2024-09-15T21:47:34.395 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.395 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.395 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.395 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.395 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.395 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:34.395 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:34.395 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 65.971907513999994, 2024-09-15T21:47:34.395 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0087926470000000007, 2024-09-15T21:47:34.395 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.395 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.395 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:34.395 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.396 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.396 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.396 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:34.396 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.396 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.396 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.396 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.396 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:34.396 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.396 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.396 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.396 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.396 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:34.397 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4099999999999999e-07 2024-09-15T21:47:34.397 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.397 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.397 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.397 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413361+0000", 2024-09-15T21:47:34.397 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6189999999999998e-06 2024-09-15T21:47:34.397 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.397 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.398 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.398 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413461+0000", 2024-09-15T21:47:34.398 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000100053 2024-09-15T21:47:34.398 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.398 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.398 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.398 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413466+0000", 2024-09-15T21:47:34.398 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3850000000000001e-06 2024-09-15T21:47:34.398 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.398 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.398 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.398 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414073+0000", 2024-09-15T21:47:34.398 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000606724 2024-09-15T21:47:34.398 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.399 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.399 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.399 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414143+0000", 2024-09-15T21:47:34.399 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9727000000000005e-05 2024-09-15T21:47:34.399 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.399 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.399 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:34.399 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414259+0000", 2024-09-15T21:47:34.399 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000116178 2024-09-15T21:47:34.399 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.399 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.399 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:34.399 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417610+0000", 2024-09-15T21:47:34.399 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003351176 2024-09-15T21:47:34.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.400 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.400 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:34.400 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422058+0000", 2024-09-15T21:47:34.400 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004447599 2024-09-15T21:47:34.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.400 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.400 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:34.400 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422111+0000", 2024-09-15T21:47:34.400 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3606999999999997e-05 2024-09-15T21:47:34.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.400 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.401 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.401 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422150+0000", 2024-09-15T21:47:34.401 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9137999999999997e-05 2024-09-15T21:47:34.401 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.401 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.401 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.401 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.401 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:34.401 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:34.401 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 65.971903084000004, 2024-09-15T21:47:34.401 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0089009769999999992, 2024-09-15T21:47:34.402 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.402 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.402 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:34.402 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.402 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.402 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.402 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:34.402 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.402 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.402 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.402 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:34.402 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.403 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.403 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.403 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.403 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413363+0000", 2024-09-15T21:47:34.403 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3999999999999997e-07 2024-09-15T21:47:34.403 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.403 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.403 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.403 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413365+0000", 2024-09-15T21:47:34.403 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.216e-06 2024-09-15T21:47:34.403 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.403 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.403 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.403 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413481+0000", 2024-09-15T21:47:34.404 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000116311 2024-09-15T21:47:34.404 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.404 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.404 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.404 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413488+0000", 2024-09-15T21:47:34.404 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0110000000000003e-06 2024-09-15T21:47:34.404 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.404 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.404 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.405 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414452+0000", 2024-09-15T21:47:34.405 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00096385399999999999 2024-09-15T21:47:34.405 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.405 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.405 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414545+0000", 2024-09-15T21:47:34.405 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.3024999999999998e-05 2024-09-15T21:47:34.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.406 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:34.406 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414644+0000", 2024-09-15T21:47:34.406 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9098000000000006e-05 2024-09-15T21:47:34.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.406 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:34.406 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417667+0000", 2024-09-15T21:47:34.406 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030229409999999999 2024-09-15T21:47:34.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.406 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:34.407 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422200+0000", 2024-09-15T21:47:34.407 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045327880000000003 2024-09-15T21:47:34.407 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.407 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.407 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:34.407 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422228+0000", 2024-09-15T21:47:34.407 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8109000000000001e-05 2024-09-15T21:47:34.407 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.407 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.407 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.407 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422263+0000", 2024-09-15T21:47:34.407 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5284000000000003e-05 2024-09-15T21:47:34.408 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.408 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.408 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.408 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.408 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.408 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:34.408 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:34.408 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 65.971899497999999, 2024-09-15T21:47:34.408 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0089846820000000008, 2024-09-15T21:47:34.408 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.409 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.409 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:34.409 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.409 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.409 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.409 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:34.409 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.409 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.409 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.409 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.410 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:34.410 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.410 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.410 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.410 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:34.410 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7599999999999998e-07 2024-09-15T21:47:34.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.410 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.410 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.410 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413368+0000", 2024-09-15T21:47:34.410 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.058e-06 2024-09-15T21:47:34.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.410 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.411 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.411 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413832+0000", 2024-09-15T21:47:34.411 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046398700000000001 2024-09-15T21:47:34.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.411 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.411 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413841+0000", 2024-09-15T21:47:34.411 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1710000000000008e-06 2024-09-15T21:47:34.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.411 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.411 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414869+0000", 2024-09-15T21:47:34.411 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001027584 2024-09-15T21:47:34.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.412 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.412 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.412 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414946+0000", 2024-09-15T21:47:34.412 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7177999999999999e-05 2024-09-15T21:47:34.412 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.412 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.412 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:34.412 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415054+0000", 2024-09-15T21:47:34.412 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010807 2024-09-15T21:47:34.412 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.412 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.413 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:34.413 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417672+0000", 2024-09-15T21:47:34.413 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002618015 2024-09-15T21:47:34.413 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.413 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.413 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:34.413 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422295+0000", 2024-09-15T21:47:34.413 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046233300000000001 2024-09-15T21:47:34.413 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.413 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.413 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:34.413 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422330+0000", 2024-09-15T21:47:34.413 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4476e-05 2024-09-15T21:47:34.414 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.414 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.414 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.414 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422351+0000", 2024-09-15T21:47:34.414 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0537e-05 2024-09-15T21:47:34.414 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.414 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.414 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.414 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.414 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.414 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:34.414 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:34.414 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 65.971896138000005, 2024-09-15T21:47:34.415 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0090873700000000009, 2024-09-15T21:47:34.415 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.415 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.415 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:34.415 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.415 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.415 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.415 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:34.415 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.415 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.415 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.415 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.415 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:34.416 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.416 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.416 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.416 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:34.416 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6100000000000002e-07 2024-09-15T21:47:34.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.416 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.416 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.416 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413371+0000", 2024-09-15T21:47:34.416 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.764e-06 2024-09-15T21:47:34.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.416 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.417 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.417 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413859+0000", 2024-09-15T21:47:34.417 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00048751900000000001 2024-09-15T21:47:34.417 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.417 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.417 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.417 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413864+0000", 2024-09-15T21:47:34.417 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6200000000000004e-06 2024-09-15T21:47:34.417 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.417 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.417 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.417 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415253+0000", 2024-09-15T21:47:34.417 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013884050000000001 2024-09-15T21:47:34.418 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.418 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.418 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.418 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415364+0000", 2024-09-15T21:47:34.418 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000111119 2024-09-15T21:47:34.418 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.418 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.418 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:34.418 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415453+0000", 2024-09-15T21:47:34.418 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8641999999999996e-05 2024-09-15T21:47:34.418 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.418 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.418 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:34.418 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417676+0000", 2024-09-15T21:47:34.419 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00222368 2024-09-15T21:47:34.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.419 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.419 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:34.419 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422395+0000", 2024-09-15T21:47:34.419 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047191150000000003 2024-09-15T21:47:34.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.420 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.421 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:34.421 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422434+0000", 2024-09-15T21:47:34.421 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8525999999999997e-05 2024-09-15T21:47:34.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.421 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.421 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.421 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422457+0000", 2024-09-15T21:47:34.421 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2719000000000001e-05 2024-09-15T21:47:34.421 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.421 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.421 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.422 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.422 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.422 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:34.422 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:34.422 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 65.971893116000004, 2024-09-15T21:47:34.422 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0092139749999999992, 2024-09-15T21:47:34.422 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.422 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.422 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:34.422 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.422 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.422 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.423 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:34.423 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.423 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.423 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.423 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.423 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:34.423 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.423 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.423 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.423 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.423 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:34.423 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3799999999999999e-07 2024-09-15T21:47:34.423 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.424 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.424 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413375+0000", 2024-09-15T21:47:34.424 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1959999999999998e-06 2024-09-15T21:47:34.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.424 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.424 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413876+0000", 2024-09-15T21:47:34.424 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00050134500000000005 2024-09-15T21:47:34.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.424 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.424 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413881+0000", 2024-09-15T21:47:34.424 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3580000000000002e-06 2024-09-15T21:47:34.425 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.425 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.425 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.425 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415669+0000", 2024-09-15T21:47:34.425 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0017875090000000001 2024-09-15T21:47:34.425 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.425 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.425 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.425 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415737+0000", 2024-09-15T21:47:34.425 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8386000000000006e-05 2024-09-15T21:47:34.425 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.425 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:34.426 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415857+0000", 2024-09-15T21:47:34.426 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119906 2024-09-15T21:47:34.426 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:34.426 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421848+0000", 2024-09-15T21:47:34.426 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059903880000000001 2024-09-15T21:47:34.426 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:34.426 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422513+0000", 2024-09-15T21:47:34.426 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00066575599999999999 2024-09-15T21:47:34.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.427 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:34.427 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422549+0000", 2024-09-15T21:47:34.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5719999999999997e-05 2024-09-15T21:47:34.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.427 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.427 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422586+0000", 2024-09-15T21:47:34.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7172999999999998e-05 2024-09-15T21:47:34.427 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.427 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.427 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.428 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.428 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:34.428 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:34.428 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 65.971887293999998, 2024-09-15T21:47:34.428 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.009324087, 2024-09-15T21:47:34.428 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.428 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.428 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:34.428 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.428 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.428 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.428 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:34.428 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.429 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.429 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.429 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:34.429 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.429 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.429 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.429 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.429 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:34.429 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8900000000000001e-07 2024-09-15T21:47:34.429 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.429 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.429 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.429 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413381+0000", 2024-09-15T21:47:34.429 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1959999999999998e-06 2024-09-15T21:47:34.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.430 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413895+0000", 2024-09-15T21:47:34.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051466000000000001 2024-09-15T21:47:34.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.431 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.431 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.431 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413900+0000", 2024-09-15T21:47:34.431 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6090000000000003e-06 2024-09-15T21:47:34.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.431 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.431 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.431 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416081+0000", 2024-09-15T21:47:34.431 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021816090000000002 2024-09-15T21:47:34.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.431 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.431 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.432 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416173+0000", 2024-09-15T21:47:34.432 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1334000000000003e-05 2024-09-15T21:47:34.432 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.432 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.432 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:34.432 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416264+0000", 2024-09-15T21:47:34.432 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0864000000000003e-05 2024-09-15T21:47:34.432 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.432 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.432 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:34.432 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421854+0000", 2024-09-15T21:47:34.432 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055902240000000004 2024-09-15T21:47:34.432 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.432 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.433 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:34.433 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422629+0000", 2024-09-15T21:47:34.433 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00077550700000000004 2024-09-15T21:47:34.433 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.433 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.433 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:34.433 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422678+0000", 2024-09-15T21:47:34.433 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8810999999999997e-05 2024-09-15T21:47:34.433 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.433 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.433 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.433 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422702+0000", 2024-09-15T21:47:34.433 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3983999999999999e-05 2024-09-15T21:47:34.434 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.434 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.434 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.434 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.434 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.434 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:34.434 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:34.434 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 65.971883812000002, 2024-09-15T21:47:34.434 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0094270580000000003, 2024-09-15T21:47:34.434 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.434 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.434 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:34.435 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.435 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.435 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.435 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:34.435 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.435 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.435 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.435 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:34.435 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.435 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.435 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.436 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:34.436 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7500000000000001e-07 2024-09-15T21:47:34.436 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.436 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.436 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.436 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413386+0000", 2024-09-15T21:47:34.436 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5619999999999997e-06 2024-09-15T21:47:34.436 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.436 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.436 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.436 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413918+0000", 2024-09-15T21:47:34.436 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00053162499999999998 2024-09-15T21:47:34.436 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.437 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.437 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.437 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413923+0000", 2024-09-15T21:47:34.437 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9280000000000001e-06 2024-09-15T21:47:34.437 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.437 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.437 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.437 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416492+0000", 2024-09-15T21:47:34.437 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0025687140000000002 2024-09-15T21:47:34.437 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.437 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.437 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.437 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416558+0000", 2024-09-15T21:47:34.437 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6699000000000006e-05 2024-09-15T21:47:34.438 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.438 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.438 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:34.438 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416691+0000", 2024-09-15T21:47:34.438 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013263199999999999 2024-09-15T21:47:34.438 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.438 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.438 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:34.438 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421879+0000", 2024-09-15T21:47:34.438 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051881089999999998 2024-09-15T21:47:34.438 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.438 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.439 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:34.439 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422742+0000", 2024-09-15T21:47:34.439 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000862845 2024-09-15T21:47:34.439 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.439 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.439 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:34.439 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422786+0000", 2024-09-15T21:47:34.439 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4332000000000002e-05 2024-09-15T21:47:34.439 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.439 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.439 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.439 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422809+0000", 2024-09-15T21:47:34.439 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2337e-05 2024-09-15T21:47:34.440 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.440 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.440 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.440 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.440 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.440 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:4 2.10 2:0f65c9b8:::60a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:34.440 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:34.440 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 59.929597643000001, 2024-09-15T21:47:34.440 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058710300000000002, 2024-09-15T21:47:34.440 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.440 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.440 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:34.440 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.441 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.441 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.441 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:34.441 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.441 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.441 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.441 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.441 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:34.442 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.442 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.442 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.442 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.442 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:34.442 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0800000000000005e-07 2024-09-15T21:47:34.442 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.442 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.442 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.442 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455672+0000", 2024-09-15T21:47:34.443 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3900000000000002e-06 2024-09-15T21:47:34.443 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.443 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.443 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.443 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456538+0000", 2024-09-15T21:47:34.443 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00086613100000000002 2024-09-15T21:47:34.443 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.443 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.443 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.443 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456545+0000", 2024-09-15T21:47:34.443 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4830000000000002e-06 2024-09-15T21:47:34.443 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.444 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.444 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456594+0000", 2024-09-15T21:47:34.444 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8429e-05 2024-09-15T21:47:34.444 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.444 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.444 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456675+0000", 2024-09-15T21:47:34.444 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1516999999999999e-05 2024-09-15T21:47:34.444 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.444 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:34.445 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456772+0000", 2024-09-15T21:47:34.445 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6523999999999999e-05 2024-09-15T21:47:34.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.445 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.445 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:34.445 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.460299+0000", 2024-09-15T21:47:34.445 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035274859999999998 2024-09-15T21:47:34.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.445 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.445 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:34.445 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461451+0000", 2024-09-15T21:47:34.445 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011518100000000001 2024-09-15T21:47:34.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.446 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.446 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:34.446 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461498+0000", 2024-09-15T21:47:34.446 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7406e-05 2024-09-15T21:47:34.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.446 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.446 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.446 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461539+0000", 2024-09-15T21:47:34.446 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0346000000000003e-05 2024-09-15T21:47:34.446 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.446 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.447 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.447 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.447 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.447 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:31 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:34.447 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:34.447 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 59.929592380999999, 2024-09-15T21:47:34.447 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0078158979999999999, 2024-09-15T21:47:34.447 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.447 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.447 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:34.447 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.448 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.448 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.448 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:34.448 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.448 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.448 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.448 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.448 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:34.448 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.448 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.448 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.448 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.448 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:34.449 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0499999999999999e-07 2024-09-15T21:47:34.449 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.449 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.449 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.449 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455675+0000", 2024-09-15T21:47:34.449 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.376e-06 2024-09-15T21:47:34.449 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.449 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.449 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.450 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456964+0000", 2024-09-15T21:47:34.450 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001289204 2024-09-15T21:47:34.450 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.450 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.450 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.450 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456970+0000", 2024-09-15T21:47:34.450 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2129999999999996e-06 2024-09-15T21:47:34.450 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.450 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.450 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.450 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457032+0000", 2024-09-15T21:47:34.450 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2209e-05 2024-09-15T21:47:34.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.451 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.451 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457134+0000", 2024-09-15T21:47:34.451 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010204000000000001 2024-09-15T21:47:34.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.451 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.451 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463489+0000", 2024-09-15T21:47:34.451 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0063545509999999999 2024-09-15T21:47:34.451 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.451 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.451 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.452 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:34.452 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:34.452 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 53.909832616999999, 2024-09-15T21:47:34.452 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057594079999999997, 2024-09-15T21:47:34.452 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.452 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.452 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:34.452 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.452 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.452 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:34.453 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.453 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.453 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.453 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:34.453 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.453 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.453 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.453 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:34.453 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1999999999999999e-07 2024-09-15T21:47:34.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.453 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.453 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.454 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475437+0000", 2024-09-15T21:47:34.454 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5429999999999998e-06 2024-09-15T21:47:34.454 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.454 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.454 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.454 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475686+0000", 2024-09-15T21:47:34.454 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000248908 2024-09-15T21:47:34.454 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.454 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.454 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.454 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475695+0000", 2024-09-15T21:47:34.454 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.9740000000000008e-06 2024-09-15T21:47:34.454 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.454 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.455 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.455 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476291+0000", 2024-09-15T21:47:34.455 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00059651500000000004 2024-09-15T21:47:34.455 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.455 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.455 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.455 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476366+0000", 2024-09-15T21:47:34.455 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4531999999999997e-05 2024-09-15T21:47:34.455 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.455 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.455 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:34.455 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476491+0000", 2024-09-15T21:47:34.455 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012527299999999999 2024-09-15T21:47:34.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.456 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.456 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:34.456 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480600+0000", 2024-09-15T21:47:34.456 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0041085870000000003 2024-09-15T21:47:34.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.456 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.456 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:34.456 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481089+0000", 2024-09-15T21:47:34.456 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000489037 2024-09-15T21:47:34.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.456 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.456 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:34.457 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481152+0000", 2024-09-15T21:47:34.457 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3136e-05 2024-09-15T21:47:34.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.457 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.457 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481192+0000", 2024-09-15T21:47:34.457 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0182999999999998e-05 2024-09-15T21:47:34.457 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.457 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.457 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.457 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:17 2.10 2:0e9da45e:::61b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:34.458 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:34.458 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 53.909826956000003, 2024-09-15T21:47:34.458 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058311830000000002, 2024-09-15T21:47:34.458 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.458 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.458 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:34.458 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.458 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.458 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.458 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:34.458 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.458 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.458 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.458 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.459 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:34.459 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.459 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.459 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.459 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475439+0000", 2024-09-15T21:47:34.459 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1300000000000001e-07 2024-09-15T21:47:34.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.459 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.459 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.459 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475441+0000", 2024-09-15T21:47:34.459 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.419e-06 2024-09-15T21:47:34.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.459 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.460 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.460 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:34.460 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000269187 2024-09-15T21:47:34.460 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.460 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.460 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.460 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475715+0000", 2024-09-15T21:47:34.461 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3660000000000002e-06 2024-09-15T21:47:34.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.461 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.461 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.461 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476671+0000", 2024-09-15T21:47:34.461 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000956507 2024-09-15T21:47:34.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.462 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.462 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.462 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477500+0000", 2024-09-15T21:47:34.462 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00082897400000000001 2024-09-15T21:47:34.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.462 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.462 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:34.462 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477625+0000", 2024-09-15T21:47:34.462 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012513100000000001 2024-09-15T21:47:34.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.462 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.462 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:34.463 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480520+0000", 2024-09-15T21:47:34.463 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002894715 2024-09-15T21:47:34.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.463 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.463 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:34.463 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481184+0000", 2024-09-15T21:47:34.463 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000663801 2024-09-15T21:47:34.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.463 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.463 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:34.463 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481232+0000", 2024-09-15T21:47:34.463 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8457e-05 2024-09-15T21:47:34.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.463 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.464 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.464 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481270+0000", 2024-09-15T21:47:34.464 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7313e-05 2024-09-15T21:47:34.464 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.464 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.464 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.464 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.464 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:34.464 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:34.464 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 53.909823080000002, 2024-09-15T21:47:34.464 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0075605530000000002, 2024-09-15T21:47:34.464 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.465 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.465 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:34.465 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.465 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.465 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.465 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:34.465 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.465 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.465 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.465 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.465 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:34.465 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.465 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.465 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.466 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.466 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:34.466 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.53e-07 2024-09-15T21:47:34.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.466 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.466 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.466 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475445+0000", 2024-09-15T21:47:34.466 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7360000000000001e-06 2024-09-15T21:47:34.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.466 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.466 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.466 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475735+0000", 2024-09-15T21:47:34.466 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028975799999999999 2024-09-15T21:47:34.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.467 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.467 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475740+0000", 2024-09-15T21:47:34.467 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8250000000000004e-06 2024-09-15T21:47:34.467 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.467 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.467 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478479+0000", 2024-09-15T21:47:34.467 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002739525 2024-09-15T21:47:34.467 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.467 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.467 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478556+0000", 2024-09-15T21:47:34.468 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6213999999999999e-05 2024-09-15T21:47:34.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.468 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:34.468 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478672+0000", 2024-09-15T21:47:34.468 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011605 2024-09-15T21:47:34.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.468 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:34.468 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481323+0000", 2024-09-15T21:47:34.468 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0026518819999999999 2024-09-15T21:47:34.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:34.469 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482918+0000", 2024-09-15T21:47:34.469 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001594105 2024-09-15T21:47:34.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:34.469 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482965+0000", 2024-09-15T21:47:34.469 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6950000000000003e-05 2024-09-15T21:47:34.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.469 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483003+0000", 2024-09-15T21:47:34.469 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8254999999999999e-05 2024-09-15T21:47:34.469 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.470 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.470 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.470 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:34.470 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:34.470 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 53.909819024000001, 2024-09-15T21:47:34.470 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076582890000000004, 2024-09-15T21:47:34.470 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.470 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.470 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:34.470 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.471 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.471 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:34.471 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.471 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.471 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.471 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:34.471 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.471 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.471 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.471 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475447+0000", 2024-09-15T21:47:34.471 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4699999999999998e-07 2024-09-15T21:47:34.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.472 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.472 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.472 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475449+0000", 2024-09-15T21:47:34.472 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.176e-06 2024-09-15T21:47:34.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.472 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.472 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.472 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475752+0000", 2024-09-15T21:47:34.472 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030374200000000002 2024-09-15T21:47:34.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.472 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.472 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.472 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475757+0000", 2024-09-15T21:47:34.473 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2209999999999999e-06 2024-09-15T21:47:34.473 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.473 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.473 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.473 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477441+0000", 2024-09-15T21:47:34.473 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016843489999999999 2024-09-15T21:47:34.473 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.473 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.473 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.473 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477567+0000", 2024-09-15T21:47:34.473 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012552099999999999 2024-09-15T21:47:34.473 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.473 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.474 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:34.474 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477712+0000", 2024-09-15T21:47:34.474 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000145427 2024-09-15T21:47:34.474 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.474 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.474 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:34.474 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482935+0000", 2024-09-15T21:47:34.474 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052231200000000004 2024-09-15T21:47:34.474 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.474 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.474 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:34.474 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483015+0000", 2024-09-15T21:47:34.474 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9634000000000004e-05 2024-09-15T21:47:34.474 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.475 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.475 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:34.475 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483065+0000", 2024-09-15T21:47:34.475 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9869e-05 2024-09-15T21:47:34.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.475 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.475 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.475 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483105+0000", 2024-09-15T21:47:34.475 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9983e-05 2024-09-15T21:47:34.475 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.476 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.476 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.476 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.476 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:34.476 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:34.476 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 53.909812483000003, 2024-09-15T21:47:34.476 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076459379999999997, 2024-09-15T21:47:34.476 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.476 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.476 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:34.476 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.476 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.477 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.477 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:34.477 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.477 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.477 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.477 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:34.477 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.477 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.477 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.477 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:34.477 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.58e-07 2024-09-15T21:47:34.478 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.478 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.478 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.478 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475455+0000", 2024-09-15T21:47:34.478 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.075e-06 2024-09-15T21:47:34.478 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.478 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.478 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.478 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475769+0000", 2024-09-15T21:47:34.478 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00031336999999999999 2024-09-15T21:47:34.478 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.478 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.478 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.478 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475773+0000", 2024-09-15T21:47:34.479 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8080000000000004e-06 2024-09-15T21:47:34.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.479 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.479 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.479 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478950+0000", 2024-09-15T21:47:34.479 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031763870000000001 2024-09-15T21:47:34.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.479 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.479 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.479 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479067+0000", 2024-09-15T21:47:34.479 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000117543 2024-09-15T21:47:34.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.479 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.479 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:34.479 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479196+0000", 2024-09-15T21:47:34.480 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000128987 2024-09-15T21:47:34.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.480 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.480 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:34.480 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481366+0000", 2024-09-15T21:47:34.480 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021699089999999998 2024-09-15T21:47:34.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.480 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.480 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:34.480 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483015+0000", 2024-09-15T21:47:34.480 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016486719999999999 2024-09-15T21:47:34.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.481 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.481 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:34.481 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483061+0000", 2024-09-15T21:47:34.481 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5729999999999998e-05 2024-09-15T21:47:34.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.481 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.481 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.481 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483099+0000", 2024-09-15T21:47:34.481 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8099000000000002e-05 2024-09-15T21:47:34.481 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.481 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.481 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.482 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.482 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:34.482 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:34.482 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 53.909765374000003, 2024-09-15T21:47:34.482 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077290859999999996, 2024-09-15T21:47:34.482 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.482 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.482 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:34.482 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.482 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.482 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.482 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:34.482 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.483 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.483 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.483 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:34.483 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.483 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.483 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.483 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475501+0000", 2024-09-15T21:47:34.483 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1900000000000002e-07 2024-09-15T21:47:34.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.483 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.483 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.483 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475504+0000", 2024-09-15T21:47:34.484 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.168e-06 2024-09-15T21:47:34.484 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.484 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.484 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.484 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477037+0000", 2024-09-15T21:47:34.484 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001533202 2024-09-15T21:47:34.484 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.484 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.484 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.484 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477050+0000", 2024-09-15T21:47:34.484 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3132999999999999e-05 2024-09-15T21:47:34.484 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.485 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.485 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.485 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479477+0000", 2024-09-15T21:47:34.485 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002426489 2024-09-15T21:47:34.485 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.485 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.485 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.485 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479553+0000", 2024-09-15T21:47:34.485 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6583999999999997e-05 2024-09-15T21:47:34.485 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.485 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.485 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:34.485 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479637+0000", 2024-09-15T21:47:34.485 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4172999999999995e-05 2024-09-15T21:47:34.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.486 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.486 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:34.486 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483108+0000", 2024-09-15T21:47:34.486 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003470356 2024-09-15T21:47:34.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.486 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.486 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:34.486 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483160+0000", 2024-09-15T21:47:34.486 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2507999999999999e-05 2024-09-15T21:47:34.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.486 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.486 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:34.487 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483202+0000", 2024-09-15T21:47:34.487 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1758999999999998e-05 2024-09-15T21:47:34.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.487 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.487 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.487 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483229+0000", 2024-09-15T21:47:34.487 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6995000000000002e-05 2024-09-15T21:47:34.487 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.487 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.487 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.487 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.488 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:34.488 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:34.488 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 53.909760417000001, 2024-09-15T21:47:34.488 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.55145717800000005, 2024-09-15T21:47:34.488 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.488 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.488 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:34.488 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.488 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.488 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.488 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:34.488 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.488 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.489 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.489 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.489 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:34.489 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.489 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.489 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.489 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.489 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:34.489 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.65e-07 2024-09-15T21:47:34.489 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.489 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.489 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.490 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475507+0000", 2024-09-15T21:47:34.490 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4699999999999999e-06 2024-09-15T21:47:34.490 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.490 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.490 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.490 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.025935+0000", 2024-09-15T21:47:34.490 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.55042854200000002 2024-09-15T21:47:34.490 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.490 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.490 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.490 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.025965+0000", 2024-09-15T21:47:34.490 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9995999999999999e-05 2024-09-15T21:47:34.490 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.491 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.491 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.491 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026016+0000", 2024-09-15T21:47:34.491 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1310000000000002e-05 2024-09-15T21:47:34.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.491 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.491 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.491 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026122+0000", 2024-09-15T21:47:34.491 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105605 2024-09-15T21:47:34.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.491 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.491 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.491 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026962+0000", 2024-09-15T21:47:34.491 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00083998999999999999 2024-09-15T21:47:34.492 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.492 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.492 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.492 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.492 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:5 2.e 2:75ee5fd8:::629.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:34.492 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:34.492 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.820554051999999, 2024-09-15T21:47:34.492 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0071955470000000001, 2024-09-15T21:47:34.492 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.492 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.492 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:34.492 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.493 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.493 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.493 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:34.493 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.493 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.493 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.493 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:34.493 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.493 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.493 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.493 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564713+0000", 2024-09-15T21:47:34.494 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.866e-06 2024-09-15T21:47:34.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.494 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.494 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.494 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564721+0000", 2024-09-15T21:47:34.494 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3299999999999999e-06 2024-09-15T21:47:34.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.494 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.494 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.494 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564762+0000", 2024-09-15T21:47:34.494 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0707999999999997e-05 2024-09-15T21:47:34.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.494 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.494 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.495 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564774+0000", 2024-09-15T21:47:34.495 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1883e-05 2024-09-15T21:47:34.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.495 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.495 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.495 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564877+0000", 2024-09-15T21:47:34.495 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010254700000000001 2024-09-15T21:47:34.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.495 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.495 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.495 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565066+0000", 2024-09-15T21:47:34.495 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018933700000000001 2024-09-15T21:47:34.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.495 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.496 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:34.496 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565164+0000", 2024-09-15T21:47:34.496 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7633999999999994e-05 2024-09-15T21:47:34.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.496 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.496 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:34.496 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569030+0000", 2024-09-15T21:47:34.496 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038669070000000002 2024-09-15T21:47:34.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.496 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.496 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:34.496 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571818+0000", 2024-09-15T21:47:34.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027878930000000001 2024-09-15T21:47:34.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.497 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.497 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:34.497 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571868+0000", 2024-09-15T21:47:34.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9880999999999999e-05 2024-09-15T21:47:34.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.497 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.497 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.497 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571907+0000", 2024-09-15T21:47:34.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8560999999999999e-05 2024-09-15T21:47:34.497 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.497 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.498 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.498 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:6 2.e 2:75ee5fd8:::629.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:34.498 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:34.498 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.820539924999999, 2024-09-15T21:47:34.498 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073433220000000002, 2024-09-15T21:47:34.498 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.498 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.498 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:34.498 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.498 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.499 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:34.499 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.499 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.499 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:34.499 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.499 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.499 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564726+0000", 2024-09-15T21:47:34.499 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7399999999999999e-07 2024-09-15T21:47:34.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.500 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.500 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.500 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564728+0000", 2024-09-15T21:47:34.500 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.447e-06 2024-09-15T21:47:34.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.500 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.500 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.500 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565446+0000", 2024-09-15T21:47:34.500 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00071774999999999996 2024-09-15T21:47:34.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.500 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.500 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.501 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565453+0000", 2024-09-15T21:47:34.501 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3239999999999997e-06 2024-09-15T21:47:34.501 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.501 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.501 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.501 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565515+0000", 2024-09-15T21:47:34.501 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1768999999999998e-05 2024-09-15T21:47:34.501 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.501 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.501 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.501 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565591+0000", 2024-09-15T21:47:34.502 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5873999999999999e-05 2024-09-15T21:47:34.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.502 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.502 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:34.502 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565709+0000", 2024-09-15T21:47:34.502 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011838899999999999 2024-09-15T21:47:34.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.502 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.502 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:34.502 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569039+0000", 2024-09-15T21:47:34.502 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0033292170000000002 2024-09-15T21:47:34.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.502 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.503 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:34.503 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571977+0000", 2024-09-15T21:47:34.503 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0029387829999999999 2024-09-15T21:47:34.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.503 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.503 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:34.503 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.572032+0000", 2024-09-15T21:47:34.503 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4540999999999998e-05 2024-09-15T21:47:34.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.503 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.503 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.503 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.572069+0000", 2024-09-15T21:47:34.503 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6854000000000003e-05 2024-09-15T21:47:34.504 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.504 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.504 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.504 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.504 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:2 2.1 2:82971f97:::404.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:34.504 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:34.504 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 42.820535958000001, 2024-09-15T21:47:34.504 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007245357, 2024-09-15T21:47:34.504 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:34.504 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:34.504 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:34.504 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:34.505 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.505 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:34.505 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:34.505 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.505 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.506 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:34.506 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:34.506 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:34.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.506 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.506 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:34.506 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564730+0000", 2024-09-15T21:47:34.506 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2999999999999999e-07 2024-09-15T21:47:34.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.506 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.506 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:34.506 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564731+0000", 2024-09-15T21:47:34.506 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.367e-06 2024-09-15T21:47:34.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.507 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.507 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:34.507 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565484+0000", 2024-09-15T21:47:34.507 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075311599999999996 2024-09-15T21:47:34.507 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.507 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.507 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:34.507 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565490+0000", 2024-09-15T21:47:34.507 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8100000000000003e-06 2024-09-15T21:47:34.507 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.507 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.507 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:34.507 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565828+0000", 2024-09-15T21:47:34.508 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000337836 2024-09-15T21:47:34.508 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.508 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.508 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:34.508 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565964+0000", 2024-09-15T21:47:34.508 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013641599999999999 2024-09-15T21:47:34.508 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.508 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.508 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:34.508 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566071+0000", 2024-09-15T21:47:34.508 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000107126 2024-09-15T21:47:34.508 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.509 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.509 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:34.509 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571361+0000", 2024-09-15T21:47:34.509 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052896210000000004 2024-09-15T21:47:34.509 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.509 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.509 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:34.509 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571874+0000", 2024-09-15T21:47:34.509 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051320599999999999 2024-09-15T21:47:34.509 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.509 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.509 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:34.509 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571936+0000", 2024-09-15T21:47:34.509 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1650999999999996e-05 2024-09-15T21:47:34.510 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:34.510 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:34.510 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:34.510 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571975+0000", 2024-09-15T21:47:34.510 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8977999999999998e-05 2024-09-15T21:47:34.510 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.510 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.510 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.510 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:34.510 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:34.510 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:34.512 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:47:34.512 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:34.510+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:34.613 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:34.611+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:34.701 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:34.701 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:34.701 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:34.701 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:34.712 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:47:34.713 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:34.712+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:34.814 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:34.813+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:34.838 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:34.838 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:34.839 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:34.839 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:34.839 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:34.849 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:47:34.935 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:34.934+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:35.016 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:35.014+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:35.030 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:35.030 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:35.030 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:35.031 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:35.031 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.031 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:35.031 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:47:35.031 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 66.613698360000001, 2024-09-15T21:47:35.031 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047486860000000002, 2024-09-15T21:47:35.031 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.031 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.031 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:35.031 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.031 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.031 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.031 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:47:35.032 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.032 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.032 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.032 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:47:35.032 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.032 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.032 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.032 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413816+0000", 2024-09-15T21:47:35.032 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4810000000000003e-06 2024-09-15T21:47:35.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.032 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.033 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.033 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413827+0000", 2024-09-15T21:47:35.033 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.187e-05 2024-09-15T21:47:35.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.033 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.033 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.033 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413862+0000", 2024-09-15T21:47:35.033 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4041000000000002e-05 2024-09-15T21:47:35.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.033 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.033 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.033 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413871+0000", 2024-09-15T21:47:35.033 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8919999999999995e-06 2024-09-15T21:47:35.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.034 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.034 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413997+0000", 2024-09-15T21:47:35.034 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012601299999999999 2024-09-15T21:47:35.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.034 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:35.034 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414100+0000", 2024-09-15T21:47:35.034 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102555 2024-09-15T21:47:35.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.034 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.034 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418558+0000", 2024-09-15T21:47:35.035 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044578339999999999 2024-09-15T21:47:35.035 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.035 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.035 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.035 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.035 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:35.035 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:47:35.035 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 66.610053546000003, 2024-09-15T21:47:35.035 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045423590000000002, 2024-09-15T21:47:35.035 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.035 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.035 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:35.035 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.036 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.036 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.036 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:47:35.036 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.036 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.036 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.036 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.036 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:47:35.036 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.036 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.036 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.036 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.036 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417455+0000", 2024-09-15T21:47:35.036 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9100000000000003e-07 2024-09-15T21:47:35.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.037 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.037 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.037 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417463+0000", 2024-09-15T21:47:35.037 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3750000000000003e-06 2024-09-15T21:47:35.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.037 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.037 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.037 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417475+0000", 2024-09-15T21:47:35.037 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2473000000000001e-05 2024-09-15T21:47:35.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.037 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.037 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.037 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417482+0000", 2024-09-15T21:47:35.038 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4010000000000002e-06 2024-09-15T21:47:35.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.038 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.038 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.038 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417524+0000", 2024-09-15T21:47:35.038 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1870999999999999e-05 2024-09-15T21:47:35.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.038 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.038 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:35.038 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417542+0000", 2024-09-15T21:47:35.038 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8224e-05 2024-09-15T21:47:35.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.038 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.039 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:35.039 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421955+0000", 2024-09-15T21:47:35.039 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044130740000000003 2024-09-15T21:47:35.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.039 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.039 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.039 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421996+0000", 2024-09-15T21:47:35.039 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.125e-05 2024-09-15T21:47:35.039 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.039 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.039 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.039 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.040 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 7 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:35.040 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:35.040 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 66.609792741000007, 2024-09-15T21:47:35.040 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043086519999999996, 2024-09-15T21:47:35.040 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.040 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.040 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:47:35.040 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.040 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.040 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.040 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:35.040 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.040 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.040 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.58e-07 2024-09-15T21:47:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417722+0000", 2024-09-15T21:47:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4729999999999997e-06 2024-09-15T21:47:35.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.042 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.042 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.042 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417737+0000", 2024-09-15T21:47:35.042 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5155e-05 2024-09-15T21:47:35.042 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.042 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.042 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.042 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417752+0000", 2024-09-15T21:47:35.042 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4863e-05 2024-09-15T21:47:35.042 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.042 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.042 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.042 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417819+0000", 2024-09-15T21:47:35.042 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7680000000000003e-05 2024-09-15T21:47:35.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.043 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.043 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.043 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422023+0000", 2024-09-15T21:47:35.043 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042040230000000003 2024-09-15T21:47:35.043 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.043 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.043 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.043 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.043 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:25 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:35.043 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:47:35.043 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 60.570303711000001, 2024-09-15T21:47:35.043 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070572830000000001, 2024-09-15T21:47:35.044 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.044 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.044 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:35.044 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.044 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.044 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.044 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:47:35.044 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.044 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.044 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.044 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.044 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:47:35.044 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.044 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.045 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.045 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.045 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457205+0000", 2024-09-15T21:47:35.045 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6899999999999999e-06 2024-09-15T21:47:35.045 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.045 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.045 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.045 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457211+0000", 2024-09-15T21:47:35.045 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.626e-06 2024-09-15T21:47:35.045 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.045 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.045 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.045 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457244+0000", 2024-09-15T21:47:35.045 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2691999999999998e-05 2024-09-15T21:47:35.046 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.046 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.046 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.046 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457251+0000", 2024-09-15T21:47:35.046 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3839999999999999e-06 2024-09-15T21:47:35.046 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.046 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.046 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.046 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457309+0000", 2024-09-15T21:47:35.046 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8023999999999999e-05 2024-09-15T21:47:35.046 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.046 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.046 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:35.046 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457413+0000", 2024-09-15T21:47:35.047 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010331999999999999 2024-09-15T21:47:35.047 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.047 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.047 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:35.047 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:35.047 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118439 2024-09-15T21:47:35.047 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.047 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.047 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:35.047 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463246+0000", 2024-09-15T21:47:35.047 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057148219999999996 2024-09-15T21:47:35.047 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.047 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.047 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:35.048 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463428+0000", 2024-09-15T21:47:35.048 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018218299999999999 2024-09-15T21:47:35.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.048 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.048 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:35.048 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464212+0000", 2024-09-15T21:47:35.048 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00078402799999999998 2024-09-15T21:47:35.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.048 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.048 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.048 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464261+0000", 2024-09-15T21:47:35.048 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9075e-05 2024-09-15T21:47:35.048 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.049 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.049 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.049 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.049 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.049 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:28 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:35.049 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:35.049 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 60.570293683999999, 2024-09-15T21:47:35.049 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0072141449999999999, 2024-09-15T21:47:35.049 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.049 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.049 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:35.049 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.049 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.049 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.050 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:35.050 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.050 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.050 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.050 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.050 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:35.050 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.050 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.050 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.050 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.050 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:35.050 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:47:35.050 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.050 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.051 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.051 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457217+0000", 2024-09-15T21:47:35.051 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7259999999999998e-06 2024-09-15T21:47:35.051 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.051 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.051 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.051 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457842+0000", 2024-09-15T21:47:35.051 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00062482599999999998 2024-09-15T21:47:35.051 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.051 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.051 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.051 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457848+0000", 2024-09-15T21:47:35.051 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7089999999999996e-06 2024-09-15T21:47:35.051 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.052 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.052 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.052 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457895+0000", 2024-09-15T21:47:35.052 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6317999999999999e-05 2024-09-15T21:47:35.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.052 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.052 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:35.052 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458013+0000", 2024-09-15T21:47:35.052 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011848100000000001 2024-09-15T21:47:35.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.052 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.052 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.052 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464428+0000", 2024-09-15T21:47:35.052 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0064146990000000003 2024-09-15T21:47:35.053 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.053 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.053 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.053 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.053 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.053 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:29 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:35.053 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:47:35.053 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 60.570289240999998, 2024-09-15T21:47:35.053 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073658980000000001, 2024-09-15T21:47:35.053 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.053 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.053 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:35.053 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.053 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.054 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.054 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:47:35.054 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.054 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.054 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.054 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.054 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:47:35.054 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.054 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.054 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.054 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.054 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457219+0000", 2024-09-15T21:47:35.054 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2000000000000001e-07 2024-09-15T21:47:35.054 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.054 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.055 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.055 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457226+0000", 2024-09-15T21:47:35.055 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.956e-06 2024-09-15T21:47:35.055 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.055 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.055 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.055 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458266+0000", 2024-09-15T21:47:35.055 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001040476 2024-09-15T21:47:35.055 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.055 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.055 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.055 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458273+0000", 2024-09-15T21:47:35.055 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0690000000000004e-06 2024-09-15T21:47:35.055 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.056 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.056 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.056 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458319+0000", 2024-09-15T21:47:35.056 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5668999999999999e-05 2024-09-15T21:47:35.056 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.056 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.056 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:35.056 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458415+0000", 2024-09-15T21:47:35.056 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6701999999999997e-05 2024-09-15T21:47:35.056 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.056 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.056 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.056 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464584+0000", 2024-09-15T21:47:35.057 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061687060000000004 2024-09-15T21:47:35.057 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.057 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.057 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.057 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.057 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.057 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.1 epoch 22/19 rep_tid 12 entries 22'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.10:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:35.057 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:47:35.057 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 60.568859234999998, 2024-09-15T21:47:35.057 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0039881550000000002, 2024-09-15T21:47:35.057 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.057 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.057 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:47:35.057 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.058 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.058 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.058 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:47:35.058 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.058 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.058 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.058 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:47:35.058 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.058 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.058 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.058 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458649+0000", 2024-09-15T21:47:35.058 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9599999999999999e-07 2024-09-15T21:47:35.059 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.059 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.059 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.059 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458656+0000", 2024-09-15T21:47:35.059 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2629999999999997e-06 2024-09-15T21:47:35.059 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.059 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.059 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.059 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458670+0000", 2024-09-15T21:47:35.059 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4166e-05 2024-09-15T21:47:35.059 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.059 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.059 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.059 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458682+0000", 2024-09-15T21:47:35.060 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1865000000000001e-05 2024-09-15T21:47:35.060 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.060 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.060 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.060 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458749+0000", 2024-09-15T21:47:35.060 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6507000000000005e-05 2024-09-15T21:47:35.060 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.060 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.060 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.060 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462636+0000", 2024-09-15T21:47:35.060 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038877579999999998 2024-09-15T21:47:35.060 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.060 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.060 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.061 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.061 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.061 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.1.10:12 2.12 e22/19 2:48cd7578:::60e.00000000:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:35.061 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:35.061 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 60.568544903000003, 2024-09-15T21:47:35.061 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042479500000000003, 2024-09-15T21:47:35.061 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.061 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.061 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:35.061 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.061 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.061 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.061 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:35.061 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.062 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.062 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.062 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.062 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:35.062 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.062 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.062 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.062 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.062 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:35.062 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2900000000000004e-07 2024-09-15T21:47:35.062 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.062 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.062 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.062 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458968+0000", 2024-09-15T21:47:35.063 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2039999999999996e-06 2024-09-15T21:47:35.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.063 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.063 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.063 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458977+0000", 2024-09-15T21:47:35.063 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.6179999999999995e-06 2024-09-15T21:47:35.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.063 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.063 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.063 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458983+0000", 2024-09-15T21:47:35.063 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9510000000000003e-06 2024-09-15T21:47:35.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.063 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.063 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.064 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459479+0000", 2024-09-15T21:47:35.064 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000496274 2024-09-15T21:47:35.064 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.064 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.064 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:35.064 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459496+0000", 2024-09-15T21:47:35.064 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6538e-05 2024-09-15T21:47:35.064 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.064 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.064 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:35.064 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463169+0000", 2024-09-15T21:47:35.064 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003672832 2024-09-15T21:47:35.064 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.064 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.065 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.065 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463211+0000", 2024-09-15T21:47:35.065 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2004e-05 2024-09-15T21:47:35.065 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.065 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.065 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.065 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.065 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.065 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.12 epoch 22/19 rep_tid 2 entries 22'3 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.10:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:35.065 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:47:35.065 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 60.567865349000002, 2024-09-15T21:47:35.065 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035859609999999999, 2024-09-15T21:47:35.065 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.066 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.066 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:47:35.066 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.066 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.066 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.066 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:47:35.066 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.066 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.066 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.066 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.066 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:47:35.066 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.066 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.066 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.066 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.067 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459643+0000", 2024-09-15T21:47:35.067 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2200000000000004e-07 2024-09-15T21:47:35.067 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.067 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.067 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.067 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459650+0000", 2024-09-15T21:47:35.067 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4479999999999997e-06 2024-09-15T21:47:35.067 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.067 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.067 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.067 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459665+0000", 2024-09-15T21:47:35.067 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4586999999999999e-05 2024-09-15T21:47:35.067 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.067 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.068 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.068 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459676+0000", 2024-09-15T21:47:35.068 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1673e-05 2024-09-15T21:47:35.068 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.068 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.068 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.068 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459732+0000", 2024-09-15T21:47:35.068 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5612999999999998e-05 2024-09-15T21:47:35.068 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.068 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.068 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.068 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463228+0000", 2024-09-15T21:47:35.068 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0034961179999999999 2024-09-15T21:47:35.068 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.069 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.069 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.069 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.069 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.069 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:34 2.1b 2:db457a8a:::201.00000001:head [write 0~588] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:35.069 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:47:35.069 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 60.562653713000003, 2024-09-15T21:47:35.069 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070946129999999996, 2024-09-15T21:47:35.069 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.069 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.069 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:35.069 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.069 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.069 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.070 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:47:35.070 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.070 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.070 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.070 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.070 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:47:35.070 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.070 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.070 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.070 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.070 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464855+0000", 2024-09-15T21:47:35.070 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5880000000000001e-06 2024-09-15T21:47:35.070 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.070 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.071 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.071 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464863+0000", 2024-09-15T21:47:35.071 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0879999999999995e-06 2024-09-15T21:47:35.071 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.071 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.071 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.071 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464889+0000", 2024-09-15T21:47:35.071 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5729e-05 2024-09-15T21:47:35.071 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.071 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.071 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.071 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464898+0000", 2024-09-15T21:47:35.071 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0389999999999993e-06 2024-09-15T21:47:35.071 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.072 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.072 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.072 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464962+0000", 2024-09-15T21:47:35.072 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3567999999999999e-05 2024-09-15T21:47:35.072 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.072 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.072 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:35.072 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.465041+0000", 2024-09-15T21:47:35.072 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8889999999999999e-05 2024-09-15T21:47:35.072 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.072 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.072 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:35.072 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.465173+0000", 2024-09-15T21:47:35.073 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013226899999999999 2024-09-15T21:47:35.073 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.073 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.073 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:35.073 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.466319+0000", 2024-09-15T21:47:35.073 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011463039999999999 2024-09-15T21:47:35.073 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.073 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.073 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:35.073 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471825+0000", 2024-09-15T21:47:35.073 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055062160000000004 2024-09-15T21:47:35.073 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.073 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.073 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:35.074 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471886+0000", 2024-09-15T21:47:35.074 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0503e-05 2024-09-15T21:47:35.074 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.074 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.074 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.074 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471948+0000", 2024-09-15T21:47:35.074 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2419e-05 2024-09-15T21:47:35.074 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.074 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.074 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.074 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.074 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.075 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:35.075 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:47:35.075 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 54.551995548999997, 2024-09-15T21:47:35.075 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047360500000000003, 2024-09-15T21:47:35.075 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.075 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.075 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:35.075 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.075 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.075 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.075 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:47:35.075 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.075 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.075 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.076 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.076 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:47:35.076 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.076 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.076 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.076 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.076 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475514+0000", 2024-09-15T21:47:35.076 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0990000000000001e-06 2024-09-15T21:47:35.076 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.076 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.076 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.076 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475524+0000", 2024-09-15T21:47:35.076 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5850000000000006e-06 2024-09-15T21:47:35.077 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.077 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.077 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.077 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475557+0000", 2024-09-15T21:47:35.077 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3090000000000003e-05 2024-09-15T21:47:35.077 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.077 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.077 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.077 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475568+0000", 2024-09-15T21:47:35.077 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1116e-05 2024-09-15T21:47:35.077 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.077 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.077 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.077 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475649+0000", 2024-09-15T21:47:35.078 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0815999999999993e-05 2024-09-15T21:47:35.078 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.078 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.078 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:35.078 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475805+0000", 2024-09-15T21:47:35.078 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015667200000000001 2024-09-15T21:47:35.078 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.078 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.078 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:35.078 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475944+0000", 2024-09-15T21:47:35.078 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000138651 2024-09-15T21:47:35.078 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.078 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.078 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:35.079 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476900+0000", 2024-09-15T21:47:35.079 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00095588699999999999 2024-09-15T21:47:35.079 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.079 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.079 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:35.079 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480153+0000", 2024-09-15T21:47:35.079 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0032535200000000002 2024-09-15T21:47:35.079 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.079 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.079 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:35.079 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480203+0000", 2024-09-15T21:47:35.079 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9932999999999998e-05 2024-09-15T21:47:35.079 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.079 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.079 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.080 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480248+0000", 2024-09-15T21:47:35.080 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4681000000000001e-05 2024-09-15T21:47:35.080 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.080 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.080 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.080 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.080 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.080 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:35.080 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:35.080 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 54.551979608000003, 2024-09-15T21:47:35.080 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049206709999999997, 2024-09-15T21:47:35.080 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.080 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.081 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:35.081 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.081 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.081 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.081 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:35.081 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.081 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.081 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.081 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.081 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:35.081 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.082 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.082 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.082 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.082 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:35.082 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1699999999999999e-07 2024-09-15T21:47:35.082 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.082 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.082 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.082 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475531+0000", 2024-09-15T21:47:35.082 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.05e-06 2024-09-15T21:47:35.082 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.082 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.082 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.083 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475743+0000", 2024-09-15T21:47:35.083 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021138099999999999 2024-09-15T21:47:35.083 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.083 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.083 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.083 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475751+0000", 2024-09-15T21:47:35.083 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7770000000000002e-06 2024-09-15T21:47:35.083 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.083 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.083 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.083 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476272+0000", 2024-09-15T21:47:35.083 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052158399999999996 2024-09-15T21:47:35.083 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.083 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.083 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:35.084 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476348+0000", 2024-09-15T21:47:35.084 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5963000000000004e-05 2024-09-15T21:47:35.084 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.084 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.084 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:35.084 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476474+0000", 2024-09-15T21:47:35.084 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012638800000000001 2024-09-15T21:47:35.084 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.084 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.084 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:35.084 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478194+0000", 2024-09-15T21:47:35.085 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001719768 2024-09-15T21:47:35.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.085 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.085 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:35.085 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480358+0000", 2024-09-15T21:47:35.085 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021637340000000001 2024-09-15T21:47:35.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.085 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.085 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:35.085 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480411+0000", 2024-09-15T21:47:35.085 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2720999999999999e-05 2024-09-15T21:47:35.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.085 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.086 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.086 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480449+0000", 2024-09-15T21:47:35.086 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7888e-05 2024-09-15T21:47:35.086 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.086 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.086 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.086 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.086 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.086 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:35.086 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:35.086 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 54.551974819000002, 2024-09-15T21:47:35.086 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005215478, 2024-09-15T21:47:35.086 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.087 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.087 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:35.087 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.087 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.087 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.087 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:35.087 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.087 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.087 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.087 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:35.087 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.087 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.088 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.088 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:35.088 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5199999999999998e-07 2024-09-15T21:47:35.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.088 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.088 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.088 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475535+0000", 2024-09-15T21:47:35.088 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9520000000000001e-06 2024-09-15T21:47:35.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.088 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.088 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.088 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475785+0000", 2024-09-15T21:47:35.088 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024966099999999998 2024-09-15T21:47:35.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.089 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.089 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.089 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475799+0000", 2024-09-15T21:47:35.089 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4649e-05 2024-09-15T21:47:35.089 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.089 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.089 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.089 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475856+0000", 2024-09-15T21:47:35.089 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6956000000000002e-05 2024-09-15T21:47:35.089 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.089 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.089 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:35.089 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476016+0000", 2024-09-15T21:47:35.089 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000159457 2024-09-15T21:47:35.090 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.090 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.090 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.090 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480748+0000", 2024-09-15T21:47:35.090 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047324510000000004 2024-09-15T21:47:35.090 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.090 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.090 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.090 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.090 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.090 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.2.12:1 2.1 e22/19 2:85bbe569:::202.00000000:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:35.090 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:47:35.090 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 54.549172347999999, 2024-09-15T21:47:35.090 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045423649999999996, 2024-09-15T21:47:35.091 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.091 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.091 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:35.091 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.091 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.091 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.091 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:47:35.091 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.091 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.091 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.091 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:47:35.091 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.092 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.092 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.092 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478336+0000", 2024-09-15T21:47:35.092 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4499999999999996e-07 2024-09-15T21:47:35.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.092 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.092 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.092 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478342+0000", 2024-09-15T21:47:35.092 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5579999999999997e-06 2024-09-15T21:47:35.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.092 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.092 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.092 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478356+0000", 2024-09-15T21:47:35.093 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3567000000000001e-05 2024-09-15T21:47:35.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.093 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.093 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.093 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478364+0000", 2024-09-15T21:47:35.093 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7889999999999995e-06 2024-09-15T21:47:35.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.093 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.093 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.093 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478416+0000", 2024-09-15T21:47:35.093 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2188000000000002e-05 2024-09-15T21:47:35.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.093 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.093 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:35.094 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478433+0000", 2024-09-15T21:47:35.094 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6674e-05 2024-09-15T21:47:35.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.094 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.094 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:35.094 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482832+0000", 2024-09-15T21:47:35.094 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043996110000000003 2024-09-15T21:47:35.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.094 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.094 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.094 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482878+0000", 2024-09-15T21:47:35.094 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5232999999999999e-05 2024-09-15T21:47:35.094 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.094 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.095 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.095 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.095 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.095 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:25 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:35.095 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:47:35.095 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.462312308999998, 2024-09-15T21:47:35.095 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058459619999999997, 2024-09-15T21:47:35.095 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.095 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.095 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:35.095 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.095 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.095 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.095 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:47:35.096 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.096 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.096 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.096 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.096 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:47:35.096 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.096 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.096 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.096 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.096 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565197+0000", 2024-09-15T21:47:35.096 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7889999999999999e-06 2024-09-15T21:47:35.096 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.096 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.096 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.097 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565208+0000", 2024-09-15T21:47:35.097 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1358999999999999e-05 2024-09-15T21:47:35.097 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.097 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.097 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.097 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565237+0000", 2024-09-15T21:47:35.097 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8626e-05 2024-09-15T21:47:35.097 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.097 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.097 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.097 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565249+0000", 2024-09-15T21:47:35.097 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1897e-05 2024-09-15T21:47:35.097 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.097 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.098 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.098 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565332+0000", 2024-09-15T21:47:35.098 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2943999999999997e-05 2024-09-15T21:47:35.098 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.098 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.098 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:35.098 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565452+0000", 2024-09-15T21:47:35.098 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119992 2024-09-15T21:47:35.098 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.098 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.098 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:35.098 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565610+0000", 2024-09-15T21:47:35.098 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015845099999999999 2024-09-15T21:47:35.098 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.098 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.099 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:35.099 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.568999+0000", 2024-09-15T21:47:35.099 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0033887190000000001 2024-09-15T21:47:35.099 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.099 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.099 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:35.099 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.570979+0000", 2024-09-15T21:47:35.099 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019804919999999999 2024-09-15T21:47:35.099 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.099 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.099 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:35.099 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571014+0000", 2024-09-15T21:47:35.099 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4331e-05 2024-09-15T21:47:35.099 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.100 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.100 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.100 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571041+0000", 2024-09-15T21:47:35.100 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7362e-05 2024-09-15T21:47:35.100 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.100 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.100 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.100 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.100 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.100 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:5 2.e e22/19 2:75ee5fd8:::629.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:47:35.100 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:47:35.100 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.461636663, 2024-09-15T21:47:35.100 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057947119999999996, 2024-09-15T21:47:35.101 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.101 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.101 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:35.101 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.101 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.101 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.101 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:47:35.101 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.101 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.101 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.101 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.101 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:47:35.101 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.101 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.102 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.102 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.102 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565872+0000", 2024-09-15T21:47:35.102 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9800000000000002e-07 2024-09-15T21:47:35.102 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.102 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.102 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.102 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565880+0000", 2024-09-15T21:47:35.102 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7619999999999995e-06 2024-09-15T21:47:35.102 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.102 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.102 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.102 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565890+0000", 2024-09-15T21:47:35.102 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0648999999999999e-05 2024-09-15T21:47:35.103 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.103 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.103 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.103 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565897+0000", 2024-09-15T21:47:35.103 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.153e-06 2024-09-15T21:47:35.103 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.103 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.103 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.103 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565966+0000", 2024-09-15T21:47:35.103 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9046000000000003e-05 2024-09-15T21:47:35.103 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.103 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.103 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:35.103 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565985+0000", 2024-09-15T21:47:35.103 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8695e-05 2024-09-15T21:47:35.104 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.104 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.104 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:35.104 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571625+0000", 2024-09-15T21:47:35.104 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056397139999999997 2024-09-15T21:47:35.104 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.104 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.104 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.104 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571666+0000", 2024-09-15T21:47:35.104 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0695000000000003e-05 2024-09-15T21:47:35.104 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.104 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.104 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.104 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.105 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.105 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:6 2.e e22/19 2:75ee5fd8:::629.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:47:35.105 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:47:35.105 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.461574415000001, 2024-09-15T21:47:35.105 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057607429999999996, 2024-09-15T21:47:35.105 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.105 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.105 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:35.105 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.105 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.105 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.105 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:47:35.105 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.106 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.106 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.106 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:47:35.106 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.106 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.106 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.106 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565934+0000", 2024-09-15T21:47:35.106 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6199999999999998e-07 2024-09-15T21:47:35.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.106 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.106 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.106 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565938+0000", 2024-09-15T21:47:35.106 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4970000000000003e-06 2024-09-15T21:47:35.107 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.107 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.107 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.107 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565945+0000", 2024-09-15T21:47:35.107 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5069999999999997e-06 2024-09-15T21:47:35.107 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.107 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.107 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.107 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565948+0000", 2024-09-15T21:47:35.107 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:47:35.107 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.107 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.107 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.107 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566200+0000", 2024-09-15T21:47:35.108 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025170200000000002 2024-09-15T21:47:35.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.108 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.108 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:35.108 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566218+0000", 2024-09-15T21:47:35.108 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7543000000000001e-05 2024-09-15T21:47:35.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.108 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.108 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:35.108 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571668+0000", 2024-09-15T21:47:35.108 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054502999999999999 2024-09-15T21:47:35.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.108 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.108 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.109 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571694+0000", 2024-09-15T21:47:35.109 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5979e-05 2024-09-15T21:47:35.109 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.109 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.109 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.109 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.109 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.109 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:2 2.1 e22/19 2:82971f97:::404.00000000:head v 22'5, mlcod=22'3)", 2024-09-15T21:47:35.109 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:47:35.109 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.461366478999999, 2024-09-15T21:47:35.109 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056016019999999998, 2024-09-15T21:47:35.109 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.109 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.109 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:35.110 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.110 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.110 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.110 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:47:35.110 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.110 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.110 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.110 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.110 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:47:35.110 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.110 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.110 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.110 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.110 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566142+0000", 2024-09-15T21:47:35.111 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:47:35.111 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.111 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.111 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.111 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566149+0000", 2024-09-15T21:47:35.111 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7510000000000008e-06 2024-09-15T21:47:35.111 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.111 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.111 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.111 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566161+0000", 2024-09-15T21:47:35.111 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1259e-05 2024-09-15T21:47:35.111 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.111 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.111 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.112 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566168+0000", 2024-09-15T21:47:35.112 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.255e-06 2024-09-15T21:47:35.112 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.112 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.112 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.112 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566227+0000", 2024-09-15T21:47:35.112 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8961e-05 2024-09-15T21:47:35.112 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.112 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.112 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:35.112 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566247+0000", 2024-09-15T21:47:35.112 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0536000000000001e-05 2024-09-15T21:47:35.112 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.112 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.113 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:35.113 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571694+0000", 2024-09-15T21:47:35.113 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005446762 2024-09-15T21:47:35.113 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.113 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.113 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.113 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571743+0000", 2024-09-15T21:47:35.113 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8396999999999998e-05 2024-09-15T21:47:35.113 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.113 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.113 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.113 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.113 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.113 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:17 2.14 e22/19 2:2eebc8f4:::62f.00000000:head v 22'9, mlcod=22'7)", 2024-09-15T21:47:35.114 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:47:35.114 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.461350381999999, 2024-09-15T21:47:35.114 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056532860000000004, 2024-09-15T21:47:35.114 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.114 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.114 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:35.114 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.114 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.114 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.114 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:47:35.114 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.114 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.114 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.114 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.115 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:47:35.115 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.115 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.115 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.115 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.115 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566158+0000", 2024-09-15T21:47:35.115 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.92e-07 2024-09-15T21:47:35.115 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.115 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.115 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.115 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566172+0000", 2024-09-15T21:47:35.115 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3976e-05 2024-09-15T21:47:35.115 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.115 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.116 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.116 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566407+0000", 2024-09-15T21:47:35.116 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023483900000000001 2024-09-15T21:47:35.116 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.116 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.116 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.116 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566416+0000", 2024-09-15T21:47:35.116 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0780000000000002e-06 2024-09-15T21:47:35.116 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.116 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.116 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.116 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566476+0000", 2024-09-15T21:47:35.116 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9908999999999998e-05 2024-09-15T21:47:35.116 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.117 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.117 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:35.117 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566493+0000", 2024-09-15T21:47:35.117 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7567e-05 2024-09-15T21:47:35.117 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.117 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.117 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:35.117 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571762+0000", 2024-09-15T21:47:35.117 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052682959999999996 2024-09-15T21:47:35.117 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.117 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.117 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.117 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571810+0000", 2024-09-15T21:47:35.117 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8829000000000003e-05 2024-09-15T21:47:35.118 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.118 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.118 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.118 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.118 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.118 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:18 2.14 e22/19 2:2eebc8f4:::62f.00000000:head v 22'10, mlcod=22'7)", 2024-09-15T21:47:35.118 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:47:35.118 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 43.461061307000001, 2024-09-15T21:47:35.118 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053992579999999997, 2024-09-15T21:47:35.118 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:35.118 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:35.118 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:35.118 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:35.118 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.119 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:35.119 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:47:35.119 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.119 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.119 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.119 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:35.119 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:47:35.119 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:35.119 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.119 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.119 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:35.119 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566447+0000", 2024-09-15T21:47:35.119 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2e-07 2024-09-15T21:47:35.119 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.119 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.120 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:35.120 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566452+0000", 2024-09-15T21:47:35.120 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0390000000000004e-06 2024-09-15T21:47:35.120 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.120 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.120 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:35.120 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566460+0000", 2024-09-15T21:47:35.120 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1389999999999995e-06 2024-09-15T21:47:35.120 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.120 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.120 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:35.120 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566473+0000", 2024-09-15T21:47:35.120 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3093e-05 2024-09-15T21:47:35.120 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.121 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.121 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:35.121 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566688+0000", 2024-09-15T21:47:35.121 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021537100000000001 2024-09-15T21:47:35.121 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.121 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.121 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:35.121 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566706+0000", 2024-09-15T21:47:35.121 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7802999999999999e-05 2024-09-15T21:47:35.121 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.121 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.121 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:35.121 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571813+0000", 2024-09-15T21:47:35.121 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051065140000000004 2024-09-15T21:47:35.122 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:35.122 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:35.122 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:35.122 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571845+0000", 2024-09-15T21:47:35.122 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2879000000000001e-05 2024-09-15T21:47:35.122 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.122 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.122 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.122 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:35.122 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:35.122 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:35.124 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:47:35.124 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:35.122+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:35.203 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:47:35.224 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:35.222+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:35.247 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:35.247 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:35.247 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:35.247 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:35.258 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:47:35.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.300+0000 7fe52781e640 1 Processor -- start 2024-09-15T21:47:35.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.300+0000 7fe52781e640 1 -- start start 2024-09-15T21:47:35.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.301+0000 7fe52781e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe520155670 0x7fe52015e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:35.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.301+0000 7fe52781e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe52015edb0 con 0x7fe520155290 2024-09-15T21:47:35.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.301+0000 7fe52781e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe52015f090 con 0x7fe52015bb60 2024-09-15T21:47:35.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.301+0000 7fe52781e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fe52015f370 con 0x7fe520155670 2024-09-15T21:47:35.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.301+0000 7fe5255ba640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe520155290 0x7fe5200c65b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:44070/0 (socket says 172.21.15.100:44070) 2024-09-15T21:47:35.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.301+0000 7fe5255ba640 1 -- 172.21.15.100:0/3234743894 learned_addr learned my addr 172.21.15.100:0/3234743894 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:35.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.301+0000 7fe524db9640 1 --2- 172.21.15.100:0/3234743894 >> v2:172.21.15.100:3300/0 conn(0x7fe520155670 0x7fe52015e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:35.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.301+0000 7fe524db9640 1 -- 172.21.15.100:0/3234743894 >> v1:172.21.15.122:6789/0 conn(0x7fe52015bb60 legacy=0x7fe52006d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:35.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.301+0000 7fe524db9640 1 -- 172.21.15.100:0/3234743894 >> v1:172.21.15.100:6789/0 conn(0x7fe520155290 legacy=0x7fe5200c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:35.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.301+0000 7fe517fff640 1 -- 172.21.15.100:0/3234743894 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 791289789 0 0) 0x7fe52015edb0 con 0x7fe520155290 2024-09-15T21:47:35.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.301+0000 7fe524db9640 1 -- 172.21.15.100:0/3234743894 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5200c5db0 con 0x7fe520155670 2024-09-15T21:47:35.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.302+0000 7fe524db9640 1 --2- 172.21.15.100:0/3234743894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe520155670 0x7fe52015e690 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe5100097a0 tx=0x7fe51002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6288e56188f4e000 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:35.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.302+0000 7fe517fff640 1 -- 172.21.15.100:0/3234743894 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510037070 con 0x7fe520155670 2024-09-15T21:47:35.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.302+0000 7fe517fff640 1 -- 172.21.15.100:0/3234743894 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe510032440 con 0x7fe520155670 2024-09-15T21:47:35.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.302+0000 7fe517fff640 1 -- 172.21.15.100:0/3234743894 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510032810 con 0x7fe520155670 2024-09-15T21:47:35.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.303+0000 7fe52781e640 1 -- 172.21.15.100:0/3234743894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe520155670 msgr2=0x7fe52015e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:35.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.303+0000 7fe52781e640 1 --2- 172.21.15.100:0/3234743894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe520155670 0x7fe52015e690 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe5100097a0 tx=0x7fe51002a230 comp rx=0 tx=0).stop 2024-09-15T21:47:35.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.303+0000 7fe52781e640 1 -- 172.21.15.100:0/3234743894 shutdown_connections 2024-09-15T21:47:35.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.303+0000 7fe52781e640 1 --2- 172.21.15.100:0/3234743894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe520155670 0x7fe52015e690 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:35.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.303+0000 7fe52781e640 1 -- 172.21.15.100:0/3234743894 >> 172.21.15.100:0/3234743894 conn(0x7fe520073310 msgr2=0x7fe520073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:35.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.303+0000 7fe52781e640 1 -- 172.21.15.100:0/3234743894 shutdown_connections 2024-09-15T21:47:35.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.303+0000 7fe52781e640 1 -- 172.21.15.100:0/3234743894 wait complete. 2024-09-15T21:47:35.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.305+0000 7fe52781e640 1 Processor -- start 2024-09-15T21:47:35.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.305+0000 7fe52781e640 1 -- start start 2024-09-15T21:47:35.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.306+0000 7fe52781e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe520155290 0x7fe5200ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:35.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.306+0000 7fe52781e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe520155670 0x7fe520159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:35.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.306+0000 7fe52781e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe52015bb60 0x7fe52014def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:35.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.306+0000 7fe52781e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe52015c590 con 0x7fe52015bb60 2024-09-15T21:47:35.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.306+0000 7fe52781e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe5200b39e0 con 0x7fe520155670 2024-09-15T21:47:35.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.306+0000 7fe52781e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe5200c6cc0 con 0x7fe520155290 2024-09-15T21:47:35.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.306+0000 7fe525dbb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe52015bb60 0x7fe52014def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:35.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.306+0000 7fe525dbb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe52015bb60 0x7fe52014def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36640/0 (socket says 172.21.15.100:36640) 2024-09-15T21:47:35.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.306+0000 7fe525dbb640 1 -- 172.21.15.100:0/4150286742 learned_addr learned my addr 172.21.15.100:0/4150286742 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:35.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.306+0000 7fe5255ba640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe520155290 0x7fe5200ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:35.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.306+0000 7fe524db9640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe520155670 0x7fe520159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:35.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.307+0000 7fe525dbb640 1 -- 172.21.15.100:0/4150286742 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe520155290 msgr2=0x7fe5200ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:35.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.307+0000 7fe525dbb640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe520155290 0x7fe5200ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:35.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.307+0000 7fe525dbb640 1 -- 172.21.15.100:0/4150286742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe520155670 msgr2=0x7fe520159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:35.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.307+0000 7fe525dbb640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe520155670 0x7fe520159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:35.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.307+0000 7fe525dbb640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5201f0f20 con 0x7fe52015bb60 2024-09-15T21:47:35.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.307+0000 7fe5255ba640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe520155290 0x7fe5200ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:35.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.307+0000 7fe524db9640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe520155670 0x7fe520159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:35.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.307+0000 7fe525dbb640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe52015bb60 0x7fe52014def0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe51800c6e0 tx=0x7fe51800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:35.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.307+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518003b70 con 0x7fe52015bb60 2024-09-15T21:47:35.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.307+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe518003d30 con 0x7fe52015bb60 2024-09-15T21:47:35.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.307+0000 7fe52781e640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5200c9060 con 0x7fe52015bb60 2024-09-15T21:47:35.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.307+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe518005310 con 0x7fe52015bb60 2024-09-15T21:47:35.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.307+0000 7fe52781e640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe520159380 con 0x7fe52015bb60 2024-09-15T21:47:35.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.308+0000 7fe52781e640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fe52015bf70 con 0x7fe52015bb60 2024-09-15T21:47:35.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.309+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe5180054b0 con 0x7fe52015bb60 2024-09-15T21:47:35.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.309+0000 7fe515ffb640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe50406c2b0 0x7fe50406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:35.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.309+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5854+0+0 (secure 0 0 0) 0x7fe51808b240 con 0x7fe52015bb60 2024-09-15T21:47:35.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.309+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v37) ==== 54+0+16 (secure 0 0 0) 0x7fe5180592c0 con 0x7fe52015bb60 2024-09-15T21:47:35.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.310+0000 7fe5255ba640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe50406c2b0 0x7fe50406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:35.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.310+0000 7fe50b7fe640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fe4f8001650 0x7fe4f8003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:35.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.310+0000 7fe50b7fe640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe4f8006bb0 con 0x7fe4f8001650 2024-09-15T21:47:35.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.310+0000 7fe524db9640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fe4f8001650 0x7fe4f8003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:35.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.310+0000 7fe5255ba640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe50406c2b0 0x7fe50406e770 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe51c000f60 tx=0x7fe51c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:35.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.311+0000 7fe524db9640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fe4f8001650 0x7fe4f8003b10 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:35.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.311+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4f8006bb0 con 0x7fe4f8001650 2024-09-15T21:47:35.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.336+0000 7fe50b7fe640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4f8005c00 con 0x7fe4f8001650 2024-09-15T21:47:35.339 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.337+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4f8005c00 con 0x7fe4f8001650 2024-09-15T21:47:35.339 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:35.340 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:35.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.337+0000 7fe50b7fe640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fe4f8004190 0x7fe4f8004640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:35.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.337+0000 7fe50b7fe640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fe4f8004b80 con 0x7fe4f8004190 2024-09-15T21:47:35.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.338+0000 7fe5255ba640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fe4f8004190 0x7fe4f8004640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:35.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.338+0000 7fe5255ba640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fe4f8004190 0x7fe4f8004640 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:35.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.338+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4f8004b80 con 0x7fe4f8004190 2024-09-15T21:47:35.341 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:35.340+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:35.354 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.353+0000 7fe50b7fe640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4f80050f0 con 0x7fe4f8004190 2024-09-15T21:47:35.356 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.354+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4f80050f0 con 0x7fe4f8004190 2024-09-15T21:47:35.356 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:35.357 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:35.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.354+0000 7fe50b7fe640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fe4f8009340 0x7fe4f800b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:35.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.354+0000 7fe50b7fe640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fe4f800e660 con 0x7fe4f8009340 2024-09-15T21:47:35.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.355+0000 7fe524db9640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fe4f8009340 0x7fe4f800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:35.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.355+0000 7fe524db9640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fe4f8009340 0x7fe4f800b730 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:35.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.355+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4f800e660 con 0x7fe4f8009340 2024-09-15T21:47:35.452 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-15T21:47:35.452 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:47:35.452 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-15T21:47:35.453 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-15T21:47:35.453 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-15T21:47:35.453 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-15T21:47:35.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.381+0000 7fe50b7fe640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4f800bf40 con 0x7fe4f8009340 2024-09-15T21:47:35.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.383+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4f800bf40 con 0x7fe4f8009340 2024-09-15T21:47:35.454 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:35.455 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:35.455 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:35.455 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:35.455 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:35.455 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:35.456 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:47:35.457 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:35.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.383+0000 7fe50b7fe640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fe4f800c9c0 0x7fe4f800ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:35.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.383+0000 7fe50b7fe640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fe4f800d5b0 con 0x7fe4f800c9c0 2024-09-15T21:47:35.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.383+0000 7fe525dbb640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fe4f800c9c0 0x7fe4f800ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:35.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.383+0000 7fe525dbb640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fe4f800c9c0 0x7fe4f800ce70 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:35.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.384+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4f800d5b0 con 0x7fe4f800c9c0 2024-09-15T21:47:35.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.409+0000 7fe50b7fe640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4f800dac0 con 0x7fe4f800c9c0 2024-09-15T21:47:35.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.412+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4f800dac0 con 0x7fe4f800c9c0 2024-09-15T21:47:35.459 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:35.460 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:35.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.412+0000 7fe50b7fe640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fe4f800e090 0x7fe4f8012830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:35.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.412+0000 7fe50b7fe640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fe4f8015790 con 0x7fe4f800e090 2024-09-15T21:47:35.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.412+0000 7fe5255ba640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fe4f800e090 0x7fe4f8012830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:35.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.412+0000 7fe5255ba640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fe4f800e090 0x7fe4f8012830 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:35.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.413+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4f8015790 con 0x7fe4f800e090 2024-09-15T21:47:35.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.431+0000 7fe50b7fe640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4f800d860 con 0x7fe4f800e090 2024-09-15T21:47:35.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.432+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4f800d860 con 0x7fe4f800e090 2024-09-15T21:47:35.461 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:35.463 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:35.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.433+0000 7fe50b7fe640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=38}) -- 0x7fe4f8014560 con 0x7fe52015bb60 2024-09-15T21:47:35.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.433+0000 7fe50b7fe640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe4f800d860 con 0x7fe52015bb60 2024-09-15T21:47:35.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.433+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fe518092070 con 0x7fe52015bb60 2024-09-15T21:47:35.463 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:47:35.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.433+0000 7fe50b7fe640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fe4f8015240 0x7fe4f8012dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:35.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.433+0000 7fe50b7fe640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fe4f80135e0 con 0x7fe4f8015240 2024-09-15T21:47:35.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.433+0000 7fe524db9640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fe4f8015240 0x7fe4f8012dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:35.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.434+0000 7fe524db9640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fe4f8015240 0x7fe4f8012dd0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:35.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.434+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4f80135e0 con 0x7fe4f8015240 2024-09-15T21:47:35.464 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:35.451+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:35.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.454+0000 7fe50b7fe640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4f8013820 con 0x7fe4f8015240 2024-09-15T21:47:35.464 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.456+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4f8013820 con 0x7fe4f8015240 2024-09-15T21:47:35.465 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:35.466 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:35.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.456+0000 7fe50b7fe640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fe4f8013e40 0x7fe4f80199f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:35.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.456+0000 7fe50b7fe640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fe4f801c7c0 con 0x7fe4f8013e40 2024-09-15T21:47:35.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.456+0000 7fe525dbb640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fe4f8013e40 0x7fe4f80199f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:35.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.457+0000 7fe525dbb640 1 --2- 172.21.15.100:0/4150286742 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fe4f8013e40 0x7fe4f80199f0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:35.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.457+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4f801c7c0 con 0x7fe4f8013e40 2024-09-15T21:47:35.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.475+0000 7fe50b7fe640 1 -- 172.21.15.100:0/4150286742 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fe4f8019f30 con 0x7fe4f8013e40 2024-09-15T21:47:35.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.477+0000 7fe515ffb640 1 -- 172.21.15.100:0/4150286742 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fe4f8019f30 con 0x7fe4f8013e40 2024-09-15T21:47:35.479 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:35.480 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:35.484 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:47:35.484 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:47:35.554 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:35.552+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:35.600 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:35.601 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "" 2024-09-15T21:47:35.601 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:35.601 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:35.601 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:35.611 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:47:35.655 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:35.653+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:35.755 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:35.755 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:35.755 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:35.753+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:35.764 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:47:35.793 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:47:35.856 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:35.855+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:35.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.895+0000 7f80ec496640 1 Processor -- start 2024-09-15T21:47:35.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.895+0000 7f80ec496640 1 -- start start 2024-09-15T21:47:35.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.897+0000 7f80ec496640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f80e414e9c0 0x7f80e406df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:35.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.897+0000 7f80ec496640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f80e4072f20 0x7f80e40733d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:35.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.897+0000 7f80ec496640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80e4157e30 con 0x7f80e40c4600 2024-09-15T21:47:35.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.897+0000 7f80ec496640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f80e4073910 con 0x7f80e4072f20 2024-09-15T21:47:35.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.897+0000 7f80ec496640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f80e406e4c0 con 0x7f80e414e9c0 2024-09-15T21:47:35.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.897+0000 7f80eaa33640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f80e4072f20 0x7f80e40733d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:35.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.897+0000 7f80e9a31640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f80e40c4600 0x7f80e406ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.100:58266/0 (socket says 172.21.15.100:58266) 2024-09-15T21:47:35.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.897+0000 7f80eaa33640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f80e4072f20 0x7f80e40733d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36646/0 (socket says 172.21.15.100:36646) 2024-09-15T21:47:35.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.897+0000 7f80e9a31640 1 -- 172.21.15.100:0/777631643 learned_addr learned my addr 172.21.15.100:0/777631643 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:35.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.897+0000 7f80ea232640 1 --2- 172.21.15.100:0/777631643 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f80e414e9c0 0x7f80e406df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:35.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.897+0000 7f80eaa33640 1 -- 172.21.15.100:0/777631643 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f80e414e9c0 msgr2=0x7f80e406df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:35.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.897+0000 7f80eaa33640 1 --2- 172.21.15.100:0/777631643 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f80e414e9c0 0x7f80e406df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:35.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.897+0000 7f80eaa33640 1 -- 172.21.15.100:0/777631643 >> v1:172.21.15.122:6789/0 conn(0x7f80e40c4600 legacy=0x7f80e406ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:35.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.897+0000 7f80eaa33640 1 -- 172.21.15.100:0/777631643 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e41580d0 con 0x7f80e4072f20 2024-09-15T21:47:35.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.897+0000 7f80e9230640 1 -- 172.21.15.100:0/777631643 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 62643597 0 0) 0x7f80e4157e30 con 0x7f80e40c4600 2024-09-15T21:47:35.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.898+0000 7f80eaa33640 1 --2- 172.21.15.100:0/777631643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e4072f20 0x7f80e40733d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009d90 tx=0x7f80dc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510de8280bd2ed4d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:35.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.898+0000 7f80e9230640 1 -- 172.21.15.100:0/777631643 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc007740 con 0x7f80e4072f20 2024-09-15T21:47:35.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.898+0000 7f80e9230640 1 -- 172.21.15.100:0/777631643 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80dc0078e0 con 0x7f80e4072f20 2024-09-15T21:47:35.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.898+0000 7f80e9230640 1 -- 172.21.15.100:0/777631643 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80dc007cb0 con 0x7f80e4072f20 2024-09-15T21:47:35.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.899+0000 7f80ec496640 1 -- 172.21.15.100:0/777631643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e4072f20 msgr2=0x7f80e40733d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:35.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.899+0000 7f80ec496640 1 --2- 172.21.15.100:0/777631643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e4072f20 0x7f80e40733d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f80dc009d90 tx=0x7f80dc02a6f0 comp rx=0 tx=0).stop 2024-09-15T21:47:35.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.899+0000 7f80ec496640 1 -- 172.21.15.100:0/777631643 shutdown_connections 2024-09-15T21:47:35.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.899+0000 7f80ec496640 1 --2- 172.21.15.100:0/777631643 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e4072f20 0x7f80e40733d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:35.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.899+0000 7f80ec496640 1 --2- 172.21.15.100:0/777631643 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f80e414e9c0 0x7f80e406df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:35.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.899+0000 7f80ec496640 1 -- 172.21.15.100:0/777631643 >> 172.21.15.100:0/777631643 conn(0x7f80e40ca430 msgr2=0x7f80e40ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:35.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.899+0000 7f80ec496640 1 -- 172.21.15.100:0/777631643 shutdown_connections 2024-09-15T21:47:35.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.899+0000 7f80ec496640 1 -- 172.21.15.100:0/777631643 wait complete. 2024-09-15T21:47:35.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.901+0000 7f80ec496640 1 Processor -- start 2024-09-15T21:47:35.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.901+0000 7f80ec496640 1 -- start start 2024-09-15T21:47:35.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.902+0000 7f80ec496640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80e4072f20 0x7f80e40c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:35.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.902+0000 7f80ec496640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e40c4600 0x7f80e41550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:35.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.902+0000 7f80ec496640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f80e414e9c0 0x7f80e41f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:35.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.902+0000 7f80ec496640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f80e4158970 con 0x7f80e40c4600 2024-09-15T21:47:35.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.902+0000 7f80e9a31640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e40c4600 0x7f80e41550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:35.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.902+0000 7f80ec496640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f80e4158690 con 0x7f80e4072f20 2024-09-15T21:47:35.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.902+0000 7f80ec496640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f80e40caf20 con 0x7f80e414e9c0 2024-09-15T21:47:35.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.902+0000 7f80e9a31640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e40c4600 0x7f80e41550c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:36658/0 (socket says 172.21.15.100:36658) 2024-09-15T21:47:35.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.902+0000 7f80e9a31640 1 -- 172.21.15.100:0/3488450018 learned_addr learned my addr 172.21.15.100:0/3488450018 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:35.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.902+0000 7f80ea232640 1 --2- 172.21.15.100:0/3488450018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80e4072f20 0x7f80e40c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:35.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.902+0000 7f80eaa33640 1 --2- 172.21.15.100:0/3488450018 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f80e414e9c0 0x7f80e41f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:35.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.903+0000 7f80e9a31640 1 -- 172.21.15.100:0/3488450018 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f80e414e9c0 msgr2=0x7f80e41f20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:35.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.903+0000 7f80e9a31640 1 --2- 172.21.15.100:0/3488450018 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f80e414e9c0 0x7f80e41f20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:35.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.903+0000 7f80e9a31640 1 -- 172.21.15.100:0/3488450018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80e4072f20 msgr2=0x7f80e40c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:35.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.903+0000 7f80e9a31640 1 --2- 172.21.15.100:0/3488450018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80e4072f20 0x7f80e40c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:35.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.903+0000 7f80e9a31640 1 -- 172.21.15.100:0/3488450018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80e40c7a20 con 0x7f80e40c4600 2024-09-15T21:47:35.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.903+0000 7f80e9a31640 1 --2- 172.21.15.100:0/3488450018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e40c4600 0x7f80e41550c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f80d400c4e0 tx=0x7f80d400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:35.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.903+0000 7f80daffd640 1 -- 172.21.15.100:0/3488450018 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d4018070 con 0x7f80e40c4600 2024-09-15T21:47:35.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.903+0000 7f80daffd640 1 -- 172.21.15.100:0/3488450018 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80d4007500 con 0x7f80e40c4600 2024-09-15T21:47:35.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.903+0000 7f80ec496640 1 -- 172.21.15.100:0/3488450018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80e41f2bb0 con 0x7f80e40c4600 2024-09-15T21:47:35.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.903+0000 7f80ec496640 1 -- 172.21.15.100:0/3488450018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80e40c8240 con 0x7f80e40c4600 2024-09-15T21:47:35.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.903+0000 7f80daffd640 1 -- 172.21.15.100:0/3488450018 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80d4004a20 con 0x7f80e40c4600 2024-09-15T21:47:35.908 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:35.909 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:35.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.904+0000 7f80ec496640 1 -- 172.21.15.100:0/3488450018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80e40769a0 con 0x7f80e40c4600 2024-09-15T21:47:35.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.905+0000 7f80daffd640 1 -- 172.21.15.100:0/3488450018 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f80d401b070 con 0x7f80e40c4600 2024-09-15T21:47:35.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.906+0000 7f80daffd640 1 --2- 172.21.15.100:0/3488450018 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f80c806c2b0 0x7f80c806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:35.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.906+0000 7f80daffd640 1 -- 172.21.15.100:0/3488450018 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5854+0+0 (secure 0 0 0) 0x7f80d408f060 con 0x7f80e40c4600 2024-09-15T21:47:35.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.906+0000 7f80ea232640 1 --2- 172.21.15.100:0/3488450018 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f80c806c2b0 0x7f80c806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:35.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.906+0000 7f80ea232640 1 --2- 172.21.15.100:0/3488450018 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f80c806c2b0 0x7f80c806e770 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009aa0 tx=0x7f80e0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:35.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:35.914+0000 7f80daffd640 1 -- 172.21.15.100:0/3488450018 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f80e40769a0 con 0x7f80e40c4600 2024-09-15T21:47:35.917 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:47:35.958 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:35.956+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:36.059 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:36.057+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:36.060 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:36.060 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:36.069 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:47:36.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:36.140+0000 7f80ec496640 1 -- 172.21.15.100:0/3488450018 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f80e414eda0 con 0x7f80e40c4600 2024-09-15T21:47:36.160 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:36.158+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:36.213 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:36.213 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:36.222 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:47:36.260 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:36.259+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:36.361 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:36.360+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:36.364 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:36.365 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:36.374 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:47:36.462 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:36.461+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:36.476 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:36.477 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:36.485 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:47:36.563 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:36.562+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:36.592 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:36.592 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:36.601 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:47:36.665 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:36.663+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:36.727 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:36.727 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:36.738 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:47:36.782 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:36.781+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:36.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:36.785+0000 7f80daffd640 1 -- 172.21.15.100:0/3488450018 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v38) ==== 101+0+0 (secure 0 0 0) 0x7f80d4058720 con 0x7f80e40c4600 2024-09-15T21:47:36.787 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is unset 2024-09-15T21:47:36.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:36.788+0000 7f80ec496640 1 -- 172.21.15.100:0/3488450018 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f80c806c2b0 msgr2=0x7f80c806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:36.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:36.788+0000 7f80ec496640 1 --2- 172.21.15.100:0/3488450018 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f80c806c2b0 0x7f80c806e770 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009aa0 tx=0x7f80e0009190 comp rx=0 tx=0).stop 2024-09-15T21:47:36.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:36.788+0000 7f80ec496640 1 -- 172.21.15.100:0/3488450018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e40c4600 msgr2=0x7f80e41550c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:36.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:36.788+0000 7f80ec496640 1 --2- 172.21.15.100:0/3488450018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e40c4600 0x7f80e41550c0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f80d400c4e0 tx=0x7f80d400c9a0 comp rx=0 tx=0).stop 2024-09-15T21:47:36.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:36.788+0000 7f80ec496640 1 -- 172.21.15.100:0/3488450018 shutdown_connections 2024-09-15T21:47:36.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:36.788+0000 7f80ec496640 1 --2- 172.21.15.100:0/3488450018 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f80c806c2b0 0x7f80c806e770 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:36.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:36.788+0000 7f80ec496640 1 --2- 172.21.15.100:0/3488450018 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f80e414e9c0 0x7f80e41f20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:36.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:36.788+0000 7f80ec496640 1 --2- 172.21.15.100:0/3488450018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f80e40c4600 0x7f80e41550c0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:36.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:36.788+0000 7f80ec496640 1 --2- 172.21.15.100:0/3488450018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f80e4072f20 0x7f80e40c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:36.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:36.788+0000 7f80ec496640 1 -- 172.21.15.100:0/3488450018 >> 172.21.15.100:0/3488450018 conn(0x7f80e40ca430 msgr2=0x7f80e406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:36.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:36.789+0000 7f80ec496640 1 -- 172.21.15.100:0/3488450018 shutdown_connections 2024-09-15T21:47:36.791 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:36.789+0000 7f80ec496640 1 -- 172.21.15.100:0/3488450018 wait complete. 2024-09-15T21:47:36.866 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:36.865+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:36.871 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:36.871 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:36.880 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:47:36.967 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:36.966+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:37.060 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:37.060 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:37.068 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:37.067+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:37.072 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:47:37.169 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:37.167+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:37.253 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:37.253 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:37.265 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:47:37.307 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:37.305+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:37.371 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:37.370+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:37.433 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:37.433 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:37.445 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:47:37.472 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:37.471+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:37.485 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:47:37.558 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:37.559 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:37.570 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:47:37.573 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:37.571+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:37.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.613+0000 7f741dae4640 1 Processor -- start 2024-09-15T21:47:37.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.613+0000 7f741dae4640 1 -- start start 2024-09-15T21:47:37.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.614+0000 7f741dae4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f741806f380 0x7f7418072f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:37.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.614+0000 7f741dae4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f741806f760 0x7f74180c58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.614+0000 7f741dae4640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74180c79a0 con 0x7f74180c8110 2024-09-15T21:47:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.614+0000 7f741dae4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f74180c5440 con 0x7f741806f760 2024-09-15T21:47:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.614+0000 7f741dae4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f74180c8650 con 0x7f741806f380 2024-09-15T21:47:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.614+0000 7f7416ffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f741806f760 0x7f74180c58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.614+0000 7f74177fe640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f741806f380 0x7f7418072f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.614+0000 7f7416ffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f741806f760 0x7f74180c58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:59664/0 (socket says 172.21.15.100:59664) 2024-09-15T21:47:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.614+0000 7f7416ffd640 1 -- 172.21.15.100:0/2311592702 learned_addr learned my addr 172.21.15.100:0/2311592702 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.614+0000 7f7416ffd640 1 -- 172.21.15.100:0/2311592702 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f741806f380 msgr2=0x7f7418072f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.614+0000 7f7416ffd640 1 --2- 172.21.15.100:0/2311592702 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f741806f380 0x7f7418072f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.614+0000 7f7416ffd640 1 -- 172.21.15.100:0/2311592702 >> v1:172.21.15.122:6789/0 conn(0x7f74180c8110 legacy=0x7f741815b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.614+0000 7f7416ffd640 1 -- 172.21.15.100:0/2311592702 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74180c88f0 con 0x7f741806f760 2024-09-15T21:47:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.615+0000 7f7416ffd640 1 --2- 172.21.15.100:0/2311592702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f741806f760 0x7f74180c58d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f740c009870 tx=0x7f740c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2d558e65e2d715a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.616+0000 7f74167fc640 1 -- 172.21.15.100:0/2311592702 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c037070 con 0x7f741806f760 2024-09-15T21:47:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.616+0000 7f74167fc640 1 -- 172.21.15.100:0/2311592702 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f740c032440 con 0x7f741806f760 2024-09-15T21:47:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.616+0000 7f74167fc640 1 -- 172.21.15.100:0/2311592702 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740c032810 con 0x7f741806f760 2024-09-15T21:47:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.616+0000 7f741dae4640 1 -- 172.21.15.100:0/2311592702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f741806f760 msgr2=0x7f74180c58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.616+0000 7f741dae4640 1 --2- 172.21.15.100:0/2311592702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f741806f760 0x7f74180c58d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f740c009870 tx=0x7f740c02a210 comp rx=0 tx=0).stop 2024-09-15T21:47:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.616+0000 7f741dae4640 1 -- 172.21.15.100:0/2311592702 shutdown_connections 2024-09-15T21:47:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.616+0000 7f741dae4640 1 --2- 172.21.15.100:0/2311592702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f741806f760 0x7f74180c58d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.616+0000 7f741dae4640 1 --2- 172.21.15.100:0/2311592702 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f741806f380 0x7f7418072f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.616+0000 7f741dae4640 1 -- 172.21.15.100:0/2311592702 >> 172.21.15.100:0/2311592702 conn(0x7f741806bbe0 msgr2=0x7f741806bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.617+0000 7f741dae4640 1 -- 172.21.15.100:0/2311592702 shutdown_connections 2024-09-15T21:47:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.617+0000 7f741dae4640 1 -- 172.21.15.100:0/2311592702 wait complete. 2024-09-15T21:47:37.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.618+0000 7f741dae4640 1 Processor -- start 2024-09-15T21:47:37.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.618+0000 7f741dae4640 1 -- start start 2024-09-15T21:47:37.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.619+0000 7f741dae4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f741806f380 0x7f74180cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:37.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.619+0000 7f741dae4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f741806f760 0x7f7418150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:37.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.619+0000 7f741dae4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74180c8110 0x7f7418152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:37.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.619+0000 7f741dae4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f74180c4dd0 con 0x7f741806f380 2024-09-15T21:47:37.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.619+0000 7f741dae4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f74180c50d0 con 0x7f74180c8110 2024-09-15T21:47:37.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.619+0000 7f741dae4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7418073490 con 0x7f741806f760 2024-09-15T21:47:37.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.619+0000 7f7417fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74180c8110 0x7f7418152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:37.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.619+0000 7f7417fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74180c8110 0x7f7418152180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:59674/0 (socket says 172.21.15.100:59674) 2024-09-15T21:47:37.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.619+0000 7f7417fff640 1 -- 172.21.15.100:0/1765970739 learned_addr learned my addr 172.21.15.100:0/1765970739 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:37.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.619+0000 7f74177fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f741806f380 0x7f74180cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:37.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.619+0000 7f7416ffd640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f741806f760 0x7f7418150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:37.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.619+0000 7f7417fff640 1 -- 172.21.15.100:0/1765970739 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f741806f760 msgr2=0x7f7418150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:37.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.619+0000 7f7417fff640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f741806f760 0x7f7418150230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:37.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.619+0000 7f7417fff640 1 -- 172.21.15.100:0/1765970739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f741806f380 msgr2=0x7f74180cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:37.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.619+0000 7f7417fff640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f741806f380 0x7f74180cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:37.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.619+0000 7f7417fff640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74181ecf90 con 0x7f74180c8110 2024-09-15T21:47:37.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.620+0000 7f74177fe640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f741806f380 0x7f74180cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:37.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.620+0000 7f7417fff640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f74180c8110 0x7f7418152180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f741000c850 tx=0x7f741000cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:37.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.620+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7410003b90 con 0x7f74180c8110 2024-09-15T21:47:37.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.620+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7410003d30 con 0x7f74180c8110 2024-09-15T21:47:37.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.620+0000 7f741dae4640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74181ed230 con 0x7f74180c8110 2024-09-15T21:47:37.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.621+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741000aa70 con 0x7f74180c8110 2024-09-15T21:47:37.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.621+0000 7f741dae4640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74181ed7f0 con 0x7f74180c8110 2024-09-15T21:47:37.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.622+0000 7f741dae4640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f74180769a0 con 0x7f74180c8110 2024-09-15T21:47:37.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.622+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f741000ac90 con 0x7f74180c8110 2024-09-15T21:47:37.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.623+0000 7f73fffff640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f740406c300 0x7f740406e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:37.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.623+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5854+0+0 (secure 0 0 0) 0x7f741008aa80 con 0x7f74180c8110 2024-09-15T21:47:37.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.623+0000 7f74177fe640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f740406c300 0x7f740406e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:37.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.623+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v38) ==== 54+0+16 (secure 0 0 0) 0x7f74180769a0 con 0x7f74180c8110 2024-09-15T21:47:37.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.623+0000 7f741dae4640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f74181eca50 0x7f74181e3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:37.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.623+0000 7f741dae4640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7418072f50 con 0x7f74181eca50 2024-09-15T21:47:37.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.624+0000 7f7416ffd640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f74181eca50 0x7f74181e3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:37.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.624+0000 7f74177fe640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f740406c300 0x7f740406e7c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f740800b240 tx=0x7f7408014040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:37.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.624+0000 7f7416ffd640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f74181eca50 0x7f74181e3330 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:37.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.625+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7418072f50 con 0x7f74181eca50 2024-09-15T21:47:37.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.651+0000 7f741dae4640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7418067f00 con 0x7f74181eca50 2024-09-15T21:47:37.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.652+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7418067f00 con 0x7f74181eca50 2024-09-15T21:47:37.654 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:37.655 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:37.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.653+0000 7f741dae4640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f74180cc470 0x7f7418156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:37.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.653+0000 7f741dae4640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f7418072a60 con 0x7f74180cc470 2024-09-15T21:47:37.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.653+0000 7f74177fe640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f74180cc470 0x7f7418156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:37.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.653+0000 7f74177fe640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f74180cc470 0x7f7418156190 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:37.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.654+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7418072a60 con 0x7f74180cc470 2024-09-15T21:47:37.674 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:37.673+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:37.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.680+0000 7f741dae4640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7418156920 con 0x7f74180cc470 2024-09-15T21:47:37.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.682+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7418156920 con 0x7f74180cc470 2024-09-15T21:47:37.683 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:37.684 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:37.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.682+0000 7f741dae4640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f74181f0890 0x7f7418062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:37.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.682+0000 7f741dae4640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f74181e1260 con 0x7f74181f0890 2024-09-15T21:47:37.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.682+0000 7f7416ffd640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f74181f0890 0x7f7418062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:37.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.682+0000 7f7416ffd640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f74181f0890 0x7f7418062630 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:37.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.683+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f74181e1260 con 0x7f74181f0890 2024-09-15T21:47:37.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.707+0000 7f741dae4640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7418062b70 con 0x7f74181f0890 2024-09-15T21:47:37.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.710+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7418062b70 con 0x7f74181f0890 2024-09-15T21:47:37.711 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:37.712 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:37.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.710+0000 7f741dae4640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f74180c58d0 0x7f74180c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:37.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.710+0000 7f741dae4640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f7418062b70 con 0x7f74180c58d0 2024-09-15T21:47:37.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.710+0000 7f7417fff640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f74180c58d0 0x7f74180c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:37.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.710+0000 7f7417fff640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f74180c58d0 0x7f74180c5d30 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:37.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.711+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7418062b70 con 0x7f74180c58d0 2024-09-15T21:47:37.720 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:37.720 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:37.720 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:37.720 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:37.720 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:37.730 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:47:37.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.734+0000 7f741dae4640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7418062b70 con 0x7f74180c58d0 2024-09-15T21:47:37.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.736+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7418062b70 con 0x7f74180c58d0 2024-09-15T21:47:37.738 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:37.738 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:37.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.736+0000 7f741dae4640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f74180c65a0 0x7f741802cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:37.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.736+0000 7f741dae4640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f741802d090 con 0x7f74180c65a0 2024-09-15T21:47:37.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.736+0000 7f74177fe640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f74180c65a0 0x7f741802cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:37.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.737+0000 7f74177fe640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f74180c65a0 0x7f741802cb50 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:37.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.738+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f741802d090 con 0x7f74180c65a0 2024-09-15T21:47:37.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.760+0000 7f741dae4640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f741802d090 con 0x7f74180c65a0 2024-09-15T21:47:37.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.762+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f741802d090 con 0x7f74180c65a0 2024-09-15T21:47:37.764 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:37.765 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:37.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.763+0000 7f741dae4640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=39}) -- 0x7f741802d290 con 0x7f74180c8110 2024-09-15T21:47:37.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.763+0000 7f741dae4640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f741802d090 con 0x7f74180c8110 2024-09-15T21:47:37.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.763+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f7410058b90 con 0x7f74180c8110 2024-09-15T21:47:37.766 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:47:37.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.763+0000 7f741dae4640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f7418201870 0x7f7418203c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:37.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.763+0000 7f741dae4640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f741802da10 con 0x7f7418201870 2024-09-15T21:47:37.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.764+0000 7f7416ffd640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f7418201870 0x7f7418203c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:37.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.764+0000 7f7416ffd640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f7418201870 0x7f7418203c60 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:37.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.765+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f741802da10 con 0x7f7418201870 2024-09-15T21:47:37.775 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:37.774+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:37.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.781+0000 7f741dae4640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f741802da10 con 0x7f7418201870 2024-09-15T21:47:37.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.783+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f741802da10 con 0x7f7418201870 2024-09-15T21:47:37.785 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:37.786 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:37.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.784+0000 7f741dae4640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f74182046b0 0x7f7418206aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:37.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.784+0000 7f741dae4640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f7418209870 con 0x7f74182046b0 2024-09-15T21:47:37.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.784+0000 7f7417fff640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f74182046b0 0x7f7418206aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:37.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.784+0000 7f7417fff640 1 --2- 172.21.15.100:0/1765970739 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f74182046b0 0x7f7418206aa0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:37.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.785+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7418209870 con 0x7f74182046b0 2024-09-15T21:47:37.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.811+0000 7f741dae4640 1 -- 172.21.15.100:0/1765970739 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f7418206fe0 con 0x7f74182046b0 2024-09-15T21:47:37.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:37.813+0000 7f73fffff640 1 -- 172.21.15.100:0/1765970739 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7418206fe0 con 0x7f74182046b0 2024-09-15T21:47:37.815 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:37.815 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:37.819 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:47:37.820 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:47:37.876 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:37.875+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:37.886 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:37.886 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:37.886 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:37.887 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:37.887 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.887 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:37.887 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:37.887 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 69.470613708000002, 2024-09-15T21:47:37.887 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00598016, 2024-09-15T21:47:37.887 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.887 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.887 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:37.887 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.887 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.887 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.887 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:37.888 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.888 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.888 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.888 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:37.888 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.888 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.888 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.888 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413339+0000", 2024-09-15T21:47:37.888 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6279999999999998e-06 2024-09-15T21:47:37.888 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.888 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.889 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.889 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413354+0000", 2024-09-15T21:47:37.889 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4877e-05 2024-09-15T21:47:37.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.889 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.889 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.889 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413418+0000", 2024-09-15T21:47:37.889 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4274000000000002e-05 2024-09-15T21:47:37.889 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.889 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.890 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.890 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413428+0000", 2024-09-15T21:47:37.890 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5899999999999997e-06 2024-09-15T21:47:37.890 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.890 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.890 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.890 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413563+0000", 2024-09-15T21:47:37.890 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013521900000000001 2024-09-15T21:47:37.890 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.891 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.891 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:37.891 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413678+0000", 2024-09-15T21:47:37.891 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000114904 2024-09-15T21:47:37.891 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.891 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.891 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:37.891 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413789+0000", 2024-09-15T21:47:37.891 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011069300000000001 2024-09-15T21:47:37.891 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.891 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.891 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:37.891 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417603+0000", 2024-09-15T21:47:37.891 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038138460000000001 2024-09-15T21:47:37.892 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.892 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.892 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:37.892 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419220+0000", 2024-09-15T21:47:37.892 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001617678 2024-09-15T21:47:37.892 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.892 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.892 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:37.892 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419275+0000", 2024-09-15T21:47:37.892 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4391999999999997e-05 2024-09-15T21:47:37.892 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.892 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.892 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:37.893 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419316+0000", 2024-09-15T21:47:37.893 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1059000000000001e-05 2024-09-15T21:47:37.893 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.893 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:37.893 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.893 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.893 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.893 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:37.893 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:37.893 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 69.470591580000004, 2024-09-15T21:47:37.893 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0087926470000000007, 2024-09-15T21:47:37.893 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.894 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.894 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:37.894 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.894 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.894 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.894 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:37.894 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.894 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.894 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.894 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.894 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:37.894 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.894 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.894 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.895 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.895 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:37.895 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4099999999999999e-07 2024-09-15T21:47:37.895 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.895 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.895 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.895 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413361+0000", 2024-09-15T21:47:37.895 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6189999999999998e-06 2024-09-15T21:47:37.895 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.895 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.895 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.895 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413461+0000", 2024-09-15T21:47:37.896 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000100053 2024-09-15T21:47:37.896 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.896 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.896 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.896 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413466+0000", 2024-09-15T21:47:37.896 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3850000000000001e-06 2024-09-15T21:47:37.896 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.896 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.896 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.896 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414073+0000", 2024-09-15T21:47:37.896 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000606724 2024-09-15T21:47:37.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.897 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.897 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:37.897 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414143+0000", 2024-09-15T21:47:37.897 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9727000000000005e-05 2024-09-15T21:47:37.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.897 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.897 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:37.897 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414259+0000", 2024-09-15T21:47:37.897 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000116178 2024-09-15T21:47:37.897 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.898 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.898 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:37.898 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417610+0000", 2024-09-15T21:47:37.898 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003351176 2024-09-15T21:47:37.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.898 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.898 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:37.898 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422058+0000", 2024-09-15T21:47:37.898 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004447599 2024-09-15T21:47:37.898 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.898 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.898 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:37.899 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422111+0000", 2024-09-15T21:47:37.899 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3606999999999997e-05 2024-09-15T21:47:37.899 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.899 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.899 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:37.899 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422150+0000", 2024-09-15T21:47:37.899 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9137999999999997e-05 2024-09-15T21:47:37.899 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.899 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:37.899 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.899 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.899 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.899 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:37.899 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:37.899 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 69.47058715, 2024-09-15T21:47:37.900 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0089009769999999992, 2024-09-15T21:47:37.900 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.900 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.900 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:37.900 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.900 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.900 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.900 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:37.900 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.900 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.900 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.900 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.900 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:37.901 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.901 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.901 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.901 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413363+0000", 2024-09-15T21:47:37.901 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3999999999999997e-07 2024-09-15T21:47:37.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.901 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.901 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.901 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413365+0000", 2024-09-15T21:47:37.901 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.216e-06 2024-09-15T21:47:37.901 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.902 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.902 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.902 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413481+0000", 2024-09-15T21:47:37.902 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000116311 2024-09-15T21:47:37.902 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.902 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.902 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.902 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413488+0000", 2024-09-15T21:47:37.902 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0110000000000003e-06 2024-09-15T21:47:37.902 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.902 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.902 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.903 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414452+0000", 2024-09-15T21:47:37.903 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00096385399999999999 2024-09-15T21:47:37.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.903 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.903 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:37.903 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414545+0000", 2024-09-15T21:47:37.903 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.3024999999999998e-05 2024-09-15T21:47:37.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.903 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.903 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:37.903 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414644+0000", 2024-09-15T21:47:37.903 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9098000000000006e-05 2024-09-15T21:47:37.903 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.903 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.904 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:37.904 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417667+0000", 2024-09-15T21:47:37.904 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030229409999999999 2024-09-15T21:47:37.904 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.904 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.904 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:37.904 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422200+0000", 2024-09-15T21:47:37.904 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045327880000000003 2024-09-15T21:47:37.904 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.904 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.904 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:37.904 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422228+0000", 2024-09-15T21:47:37.904 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8109000000000001e-05 2024-09-15T21:47:37.904 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.905 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.905 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:37.905 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422263+0000", 2024-09-15T21:47:37.905 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5284000000000003e-05 2024-09-15T21:47:37.905 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.905 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:37.905 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.905 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.905 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.905 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:37.905 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:37.905 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 69.470583563999995, 2024-09-15T21:47:37.905 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0089846820000000008, 2024-09-15T21:47:37.905 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.906 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.906 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:37.906 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.906 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.906 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.906 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:37.906 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.906 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.906 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.906 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:37.906 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.906 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.906 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.907 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.907 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:37.907 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7599999999999998e-07 2024-09-15T21:47:37.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.907 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.907 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.907 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413368+0000", 2024-09-15T21:47:37.907 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.058e-06 2024-09-15T21:47:37.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.907 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.907 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.907 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413832+0000", 2024-09-15T21:47:37.907 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046398700000000001 2024-09-15T21:47:37.907 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.908 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.908 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413841+0000", 2024-09-15T21:47:37.908 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1710000000000008e-06 2024-09-15T21:47:37.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.908 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.908 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414869+0000", 2024-09-15T21:47:37.908 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001027584 2024-09-15T21:47:37.908 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.908 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.908 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:37.908 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414946+0000", 2024-09-15T21:47:37.909 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7177999999999999e-05 2024-09-15T21:47:37.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.909 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.909 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:37.909 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415054+0000", 2024-09-15T21:47:37.909 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010807 2024-09-15T21:47:37.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.909 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.909 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:37.909 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417672+0000", 2024-09-15T21:47:37.909 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002618015 2024-09-15T21:47:37.909 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.910 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.910 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:37.910 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422295+0000", 2024-09-15T21:47:37.910 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046233300000000001 2024-09-15T21:47:37.910 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.910 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.910 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:37.910 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422330+0000", 2024-09-15T21:47:37.910 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4476e-05 2024-09-15T21:47:37.910 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.910 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.910 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:37.911 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422351+0000", 2024-09-15T21:47:37.911 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0537e-05 2024-09-15T21:47:37.911 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.911 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:37.911 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.911 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.911 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.911 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:37.911 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:37.911 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 69.470580204000001, 2024-09-15T21:47:37.911 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0090873700000000009, 2024-09-15T21:47:37.911 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.911 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.912 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:37.912 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.913 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.913 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.913 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:37.913 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.913 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.913 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.913 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:37.913 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.913 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.914 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.914 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.914 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:37.914 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6100000000000002e-07 2024-09-15T21:47:37.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.914 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.914 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.914 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413371+0000", 2024-09-15T21:47:37.914 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.764e-06 2024-09-15T21:47:37.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.914 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.914 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.914 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413859+0000", 2024-09-15T21:47:37.914 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00048751900000000001 2024-09-15T21:47:37.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.915 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.915 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413864+0000", 2024-09-15T21:47:37.915 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6200000000000004e-06 2024-09-15T21:47:37.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.915 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.915 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415253+0000", 2024-09-15T21:47:37.915 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013884050000000001 2024-09-15T21:47:37.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.915 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:37.915 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415364+0000", 2024-09-15T21:47:37.916 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000111119 2024-09-15T21:47:37.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.916 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.916 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:37.916 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415453+0000", 2024-09-15T21:47:37.916 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8641999999999996e-05 2024-09-15T21:47:37.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.916 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.916 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:37.916 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417676+0000", 2024-09-15T21:47:37.917 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00222368 2024-09-15T21:47:37.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.917 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:37.917 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422395+0000", 2024-09-15T21:47:37.917 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047191150000000003 2024-09-15T21:47:37.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.917 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:37.917 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422434+0000", 2024-09-15T21:47:37.917 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8525999999999997e-05 2024-09-15T21:47:37.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.917 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:37.918 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422457+0000", 2024-09-15T21:47:37.918 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2719000000000001e-05 2024-09-15T21:47:37.918 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.918 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:37.918 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.918 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.918 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.918 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:37.918 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:37.918 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 69.470577182, 2024-09-15T21:47:37.918 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0092139749999999992, 2024-09-15T21:47:37.918 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.918 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.919 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:37.919 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.919 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.919 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:37.919 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.919 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.919 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.919 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:37.919 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.919 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.919 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.920 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:37.920 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3799999999999999e-07 2024-09-15T21:47:37.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.920 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.920 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.920 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413375+0000", 2024-09-15T21:47:37.920 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1959999999999998e-06 2024-09-15T21:47:37.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.920 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.920 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.920 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413876+0000", 2024-09-15T21:47:37.920 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00050134500000000005 2024-09-15T21:47:37.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.921 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.921 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.921 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413881+0000", 2024-09-15T21:47:37.921 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3580000000000002e-06 2024-09-15T21:47:37.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.921 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.921 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.921 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415669+0000", 2024-09-15T21:47:37.921 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0017875090000000001 2024-09-15T21:47:37.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.921 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.921 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:37.922 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415737+0000", 2024-09-15T21:47:37.922 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8386000000000006e-05 2024-09-15T21:47:37.922 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.922 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.922 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:37.922 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415857+0000", 2024-09-15T21:47:37.922 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119906 2024-09-15T21:47:37.922 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.922 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.922 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:37.922 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421848+0000", 2024-09-15T21:47:37.922 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059903880000000001 2024-09-15T21:47:37.922 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.922 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.923 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:37.923 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422513+0000", 2024-09-15T21:47:37.923 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00066575599999999999 2024-09-15T21:47:37.923 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.923 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.923 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:37.923 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422549+0000", 2024-09-15T21:47:37.923 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5719999999999997e-05 2024-09-15T21:47:37.923 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.923 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.923 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:37.923 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422586+0000", 2024-09-15T21:47:37.923 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7172999999999998e-05 2024-09-15T21:47:37.923 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.924 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:37.924 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.924 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.924 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.924 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:37.924 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:37.924 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 69.470571359999994, 2024-09-15T21:47:37.924 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.009324087, 2024-09-15T21:47:37.924 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.924 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.924 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:37.924 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.924 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.924 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.925 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:37.925 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.925 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.925 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.925 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:37.925 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.925 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.925 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.925 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:37.925 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8900000000000001e-07 2024-09-15T21:47:37.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.925 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.926 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.926 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413381+0000", 2024-09-15T21:47:37.926 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1959999999999998e-06 2024-09-15T21:47:37.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.926 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.926 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.926 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413895+0000", 2024-09-15T21:47:37.926 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051466000000000001 2024-09-15T21:47:37.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.926 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.926 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.926 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413900+0000", 2024-09-15T21:47:37.926 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6090000000000003e-06 2024-09-15T21:47:37.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.927 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.927 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.927 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416081+0000", 2024-09-15T21:47:37.927 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021816090000000002 2024-09-15T21:47:37.927 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.927 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.927 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:37.927 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416173+0000", 2024-09-15T21:47:37.927 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1334000000000003e-05 2024-09-15T21:47:37.927 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.927 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.927 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:37.927 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416264+0000", 2024-09-15T21:47:37.927 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0864000000000003e-05 2024-09-15T21:47:37.928 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.928 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.928 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:37.928 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421854+0000", 2024-09-15T21:47:37.928 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055902240000000004 2024-09-15T21:47:37.928 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.928 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.928 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:37.928 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422629+0000", 2024-09-15T21:47:37.928 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00077550700000000004 2024-09-15T21:47:37.928 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.928 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.928 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:37.928 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422678+0000", 2024-09-15T21:47:37.929 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8810999999999997e-05 2024-09-15T21:47:37.929 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.929 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.929 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:37.929 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422702+0000", 2024-09-15T21:47:37.929 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3983999999999999e-05 2024-09-15T21:47:37.929 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.929 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:37.929 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.929 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.929 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.929 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:37.929 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:37.930 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 69.470567877999997, 2024-09-15T21:47:37.930 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0094270580000000003, 2024-09-15T21:47:37.930 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.930 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.930 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:37.930 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.930 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.930 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.930 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:37.930 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.930 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.930 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.930 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.930 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:37.931 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.931 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.931 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.931 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.931 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:37.931 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7500000000000001e-07 2024-09-15T21:47:37.931 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.931 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.931 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.931 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413386+0000", 2024-09-15T21:47:37.931 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5619999999999997e-06 2024-09-15T21:47:37.931 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.931 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.931 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.932 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413918+0000", 2024-09-15T21:47:37.932 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00053162499999999998 2024-09-15T21:47:37.932 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.932 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.932 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.932 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413923+0000", 2024-09-15T21:47:37.932 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9280000000000001e-06 2024-09-15T21:47:37.932 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.932 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.932 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.932 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416492+0000", 2024-09-15T21:47:37.932 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0025687140000000002 2024-09-15T21:47:37.932 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.932 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.933 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:37.933 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416558+0000", 2024-09-15T21:47:37.933 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6699000000000006e-05 2024-09-15T21:47:37.933 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.933 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.933 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:37.933 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416691+0000", 2024-09-15T21:47:37.934 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013263199999999999 2024-09-15T21:47:37.934 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.934 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.934 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:37.934 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421879+0000", 2024-09-15T21:47:37.934 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051881089999999998 2024-09-15T21:47:37.934 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.934 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.934 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:37.934 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422742+0000", 2024-09-15T21:47:37.934 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000862845 2024-09-15T21:47:37.934 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.934 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.934 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:37.935 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422786+0000", 2024-09-15T21:47:37.935 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4332000000000002e-05 2024-09-15T21:47:37.935 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.935 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.935 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:37.935 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422809+0000", 2024-09-15T21:47:37.935 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2337e-05 2024-09-15T21:47:37.935 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.935 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:37.935 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.935 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.935 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.936 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:4 2.10 2:0f65c9b8:::60a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:37.936 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:37.936 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 63.428281708999997, 2024-09-15T21:47:37.936 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058710300000000002, 2024-09-15T21:47:37.936 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.936 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.936 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:37.936 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.936 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.936 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.937 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:37.937 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.937 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.937 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.937 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.937 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:37.937 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.937 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.937 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.937 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.938 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:37.938 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0800000000000005e-07 2024-09-15T21:47:37.938 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.938 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.938 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.938 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455672+0000", 2024-09-15T21:47:37.938 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3900000000000002e-06 2024-09-15T21:47:37.938 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.938 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.938 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.938 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456538+0000", 2024-09-15T21:47:37.938 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00086613100000000002 2024-09-15T21:47:37.938 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.938 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.939 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.939 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456545+0000", 2024-09-15T21:47:37.939 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4830000000000002e-06 2024-09-15T21:47:37.939 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.939 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.939 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.939 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456594+0000", 2024-09-15T21:47:37.939 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8429e-05 2024-09-15T21:47:37.939 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.939 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.939 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:37.939 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456675+0000", 2024-09-15T21:47:37.939 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1516999999999999e-05 2024-09-15T21:47:37.939 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.940 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.940 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:37.940 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456772+0000", 2024-09-15T21:47:37.940 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6523999999999999e-05 2024-09-15T21:47:37.940 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.940 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.940 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:37.940 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.460299+0000", 2024-09-15T21:47:37.940 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035274859999999998 2024-09-15T21:47:37.940 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.940 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.940 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:37.940 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461451+0000", 2024-09-15T21:47:37.940 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011518100000000001 2024-09-15T21:47:37.941 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.941 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.941 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:37.941 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461498+0000", 2024-09-15T21:47:37.941 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7406e-05 2024-09-15T21:47:37.941 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.941 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.941 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:37.941 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461539+0000", 2024-09-15T21:47:37.941 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0346000000000003e-05 2024-09-15T21:47:37.941 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.941 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:37.941 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.941 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.942 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.942 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:31 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:37.942 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:37.942 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 63.428276447000002, 2024-09-15T21:47:37.942 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0078158979999999999, 2024-09-15T21:47:37.942 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.942 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.942 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:37.942 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.942 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.942 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.942 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:37.942 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.942 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.943 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.943 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.943 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:37.943 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.943 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.943 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.943 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.943 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:37.943 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0499999999999999e-07 2024-09-15T21:47:37.943 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.943 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.943 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.943 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455675+0000", 2024-09-15T21:47:37.943 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.376e-06 2024-09-15T21:47:37.944 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.944 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.944 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.944 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456964+0000", 2024-09-15T21:47:37.944 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001289204 2024-09-15T21:47:37.944 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.944 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.944 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.944 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456970+0000", 2024-09-15T21:47:37.944 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2129999999999996e-06 2024-09-15T21:47:37.944 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.944 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.944 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.944 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457032+0000", 2024-09-15T21:47:37.945 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2209e-05 2024-09-15T21:47:37.945 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.945 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.945 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:37.945 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457134+0000", 2024-09-15T21:47:37.945 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010204000000000001 2024-09-15T21:47:37.945 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.945 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.945 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:37.945 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463489+0000", 2024-09-15T21:47:37.945 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0063545509999999999 2024-09-15T21:47:37.946 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.946 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:37.946 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.946 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.946 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.946 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:37.946 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:37.946 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 57.408516683000002, 2024-09-15T21:47:37.946 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057594079999999997, 2024-09-15T21:47:37.946 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.946 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.946 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:37.946 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.947 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.947 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.947 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:37.947 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.947 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.947 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.947 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.947 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:37.947 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.948 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.948 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.948 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.948 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:37.948 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1999999999999999e-07 2024-09-15T21:47:37.948 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.948 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.948 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.948 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475437+0000", 2024-09-15T21:47:37.948 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5429999999999998e-06 2024-09-15T21:47:37.948 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.948 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.948 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.948 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475686+0000", 2024-09-15T21:47:37.949 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000248908 2024-09-15T21:47:37.949 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.949 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.949 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.949 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475695+0000", 2024-09-15T21:47:37.949 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.9740000000000008e-06 2024-09-15T21:47:37.949 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.949 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.949 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.949 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476291+0000", 2024-09-15T21:47:37.949 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00059651500000000004 2024-09-15T21:47:37.949 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.949 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.950 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:37.950 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476366+0000", 2024-09-15T21:47:37.950 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4531999999999997e-05 2024-09-15T21:47:37.950 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.950 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.950 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:37.950 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476491+0000", 2024-09-15T21:47:37.950 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012527299999999999 2024-09-15T21:47:37.950 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.950 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.950 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:37.950 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480600+0000", 2024-09-15T21:47:37.950 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0041085870000000003 2024-09-15T21:47:37.950 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.950 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.951 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:37.951 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481089+0000", 2024-09-15T21:47:37.951 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000489037 2024-09-15T21:47:37.951 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.951 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.951 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:37.951 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481152+0000", 2024-09-15T21:47:37.951 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3136e-05 2024-09-15T21:47:37.951 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.951 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.951 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:37.951 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481192+0000", 2024-09-15T21:47:37.951 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0182999999999998e-05 2024-09-15T21:47:37.951 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.952 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:37.952 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.952 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.952 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.952 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:17 2.10 2:0e9da45e:::61b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:37.952 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:37.952 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 57.408511021999999, 2024-09-15T21:47:37.952 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058311830000000002, 2024-09-15T21:47:37.952 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.952 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.952 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:37.952 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.952 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.952 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.953 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:37.953 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.953 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.953 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.953 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.953 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:37.953 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.953 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.953 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.953 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.953 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475439+0000", 2024-09-15T21:47:37.953 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1300000000000001e-07 2024-09-15T21:47:37.953 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.953 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.954 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.954 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475441+0000", 2024-09-15T21:47:37.954 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.419e-06 2024-09-15T21:47:37.954 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.954 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.954 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.954 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:37.954 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000269187 2024-09-15T21:47:37.954 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.954 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.954 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.954 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475715+0000", 2024-09-15T21:47:37.954 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3660000000000002e-06 2024-09-15T21:47:37.954 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.955 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.955 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.955 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476671+0000", 2024-09-15T21:47:37.955 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000956507 2024-09-15T21:47:37.955 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.955 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.955 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:37.955 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477500+0000", 2024-09-15T21:47:37.955 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00082897400000000001 2024-09-15T21:47:37.955 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.955 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.955 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:37.955 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477625+0000", 2024-09-15T21:47:37.956 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012513100000000001 2024-09-15T21:47:37.956 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.956 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.956 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:37.956 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480520+0000", 2024-09-15T21:47:37.956 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002894715 2024-09-15T21:47:37.956 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.956 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.956 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:37.956 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481184+0000", 2024-09-15T21:47:37.956 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000663801 2024-09-15T21:47:37.956 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.956 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.956 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:37.957 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481232+0000", 2024-09-15T21:47:37.957 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8457e-05 2024-09-15T21:47:37.957 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.957 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.957 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:37.957 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481270+0000", 2024-09-15T21:47:37.957 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7313e-05 2024-09-15T21:47:37.957 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.957 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:37.957 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.957 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.957 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.957 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:37.957 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:37.958 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 57.408507145999998, 2024-09-15T21:47:37.958 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0075605530000000002, 2024-09-15T21:47:37.958 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.958 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.958 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:37.958 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.958 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.958 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.958 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:37.958 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.958 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.958 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.958 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.958 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:37.959 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.959 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.959 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.959 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.959 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:37.959 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.53e-07 2024-09-15T21:47:37.959 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.959 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.959 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.959 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475445+0000", 2024-09-15T21:47:37.960 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7360000000000001e-06 2024-09-15T21:47:37.960 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.960 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.960 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.960 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475735+0000", 2024-09-15T21:47:37.960 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028975799999999999 2024-09-15T21:47:37.960 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.960 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.960 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.960 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475740+0000", 2024-09-15T21:47:37.960 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8250000000000004e-06 2024-09-15T21:47:37.960 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.960 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.960 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.961 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478479+0000", 2024-09-15T21:47:37.961 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002739525 2024-09-15T21:47:37.961 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.961 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.961 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:37.961 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478556+0000", 2024-09-15T21:47:37.961 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6213999999999999e-05 2024-09-15T21:47:37.961 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.961 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.961 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:37.961 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478672+0000", 2024-09-15T21:47:37.961 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011605 2024-09-15T21:47:37.961 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.962 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.962 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:37.962 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481323+0000", 2024-09-15T21:47:37.962 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0026518819999999999 2024-09-15T21:47:37.962 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.962 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.962 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:37.962 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482918+0000", 2024-09-15T21:47:37.962 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001594105 2024-09-15T21:47:37.962 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.962 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.962 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:37.962 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482965+0000", 2024-09-15T21:47:37.962 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6950000000000003e-05 2024-09-15T21:47:37.963 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.963 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.963 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:37.963 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483003+0000", 2024-09-15T21:47:37.963 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8254999999999999e-05 2024-09-15T21:47:37.963 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.963 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:37.963 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.963 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.963 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.963 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:37.963 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:37.963 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 57.408503090000004, 2024-09-15T21:47:37.963 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076582890000000004, 2024-09-15T21:47:37.964 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.964 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.964 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:37.964 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.964 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.964 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.964 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:37.964 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.964 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.964 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.964 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.964 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:37.964 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.965 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.965 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.965 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.965 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475447+0000", 2024-09-15T21:47:37.965 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4699999999999998e-07 2024-09-15T21:47:37.965 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.965 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.965 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.965 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475449+0000", 2024-09-15T21:47:37.965 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.176e-06 2024-09-15T21:47:37.965 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.965 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.965 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.965 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475752+0000", 2024-09-15T21:47:37.965 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030374200000000002 2024-09-15T21:47:37.966 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.966 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.966 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.966 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475757+0000", 2024-09-15T21:47:37.966 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2209999999999999e-06 2024-09-15T21:47:37.966 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.966 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.966 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.966 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477441+0000", 2024-09-15T21:47:37.966 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016843489999999999 2024-09-15T21:47:37.966 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.966 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.966 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:37.966 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477567+0000", 2024-09-15T21:47:37.967 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012552099999999999 2024-09-15T21:47:37.967 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.967 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.967 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:37.967 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477712+0000", 2024-09-15T21:47:37.967 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000145427 2024-09-15T21:47:37.967 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.967 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.967 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:37.967 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482935+0000", 2024-09-15T21:47:37.967 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052231200000000004 2024-09-15T21:47:37.967 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.967 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.967 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:37.968 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483015+0000", 2024-09-15T21:47:37.968 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9634000000000004e-05 2024-09-15T21:47:37.968 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.968 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.968 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:37.968 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483065+0000", 2024-09-15T21:47:37.968 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9869e-05 2024-09-15T21:47:37.968 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.968 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.968 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:37.968 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483105+0000", 2024-09-15T21:47:37.968 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9983e-05 2024-09-15T21:47:37.968 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.969 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:37.969 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.969 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.969 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.969 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:37.969 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:37.969 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 57.408496548999999, 2024-09-15T21:47:37.969 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076459379999999997, 2024-09-15T21:47:37.969 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.969 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.969 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:37.969 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.969 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.969 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.970 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:37.970 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.970 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.970 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.970 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.970 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:37.970 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.970 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.970 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.970 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.970 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:37.970 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.58e-07 2024-09-15T21:47:37.970 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.970 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.971 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.971 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475455+0000", 2024-09-15T21:47:37.971 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.075e-06 2024-09-15T21:47:37.971 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.971 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.971 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.971 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475769+0000", 2024-09-15T21:47:37.971 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00031336999999999999 2024-09-15T21:47:37.971 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.971 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.971 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.971 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475773+0000", 2024-09-15T21:47:37.971 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8080000000000004e-06 2024-09-15T21:47:37.971 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.972 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.972 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.972 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478950+0000", 2024-09-15T21:47:37.972 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031763870000000001 2024-09-15T21:47:37.972 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.972 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.972 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:37.972 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479067+0000", 2024-09-15T21:47:37.972 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000117543 2024-09-15T21:47:37.972 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.972 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.972 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:37.972 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479196+0000", 2024-09-15T21:47:37.972 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000128987 2024-09-15T21:47:37.973 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.973 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.973 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:37.973 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481366+0000", 2024-09-15T21:47:37.973 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021699089999999998 2024-09-15T21:47:37.973 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.973 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.973 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:37.973 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483015+0000", 2024-09-15T21:47:37.973 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016486719999999999 2024-09-15T21:47:37.973 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.973 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.973 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:37.973 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483061+0000", 2024-09-15T21:47:37.974 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5729999999999998e-05 2024-09-15T21:47:37.974 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.974 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.974 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:37.974 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483099+0000", 2024-09-15T21:47:37.974 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8099000000000002e-05 2024-09-15T21:47:37.974 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.974 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:37.974 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.974 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.974 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.974 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:37.974 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:37.974 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 57.408449439999998, 2024-09-15T21:47:37.975 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077290859999999996, 2024-09-15T21:47:37.975 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.975 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.975 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:37.975 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.975 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.975 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.975 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:37.975 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.975 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.975 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.975 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.975 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:37.976 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.976 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.976 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.976 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.976 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475501+0000", 2024-09-15T21:47:37.976 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1900000000000002e-07 2024-09-15T21:47:37.976 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.976 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.976 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.976 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475504+0000", 2024-09-15T21:47:37.976 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.168e-06 2024-09-15T21:47:37.976 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.976 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.976 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.977 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477037+0000", 2024-09-15T21:47:37.977 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001533202 2024-09-15T21:47:37.977 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.977 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.977 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.977 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477050+0000", 2024-09-15T21:47:37.977 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3132999999999999e-05 2024-09-15T21:47:37.977 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.977 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.977 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.977 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479477+0000", 2024-09-15T21:47:37.977 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002426489 2024-09-15T21:47:37.977 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.978 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.978 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:37.978 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479553+0000", 2024-09-15T21:47:37.978 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6583999999999997e-05 2024-09-15T21:47:37.978 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.978 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.978 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:37.978 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479637+0000", 2024-09-15T21:47:37.978 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4172999999999995e-05 2024-09-15T21:47:37.978 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.978 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.978 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:37.978 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483108+0000", 2024-09-15T21:47:37.978 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003470356 2024-09-15T21:47:37.979 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.979 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.979 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:37.979 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483160+0000", 2024-09-15T21:47:37.979 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2507999999999999e-05 2024-09-15T21:47:37.979 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.979 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.979 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:37.979 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483202+0000", 2024-09-15T21:47:37.979 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1758999999999998e-05 2024-09-15T21:47:37.979 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.979 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.979 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:37.980 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483229+0000", 2024-09-15T21:47:37.980 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6995000000000002e-05 2024-09-15T21:47:37.980 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.980 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:37.980 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.980 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.980 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.980 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:37.980 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:37.980 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 57.408444482999997, 2024-09-15T21:47:37.980 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.55145717800000005, 2024-09-15T21:47:37.980 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.981 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.981 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:37.981 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.981 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.981 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.981 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:37.981 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.981 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.981 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.981 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.981 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:37.981 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.982 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.982 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.982 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:37.982 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.65e-07 2024-09-15T21:47:37.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.982 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.982 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.982 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475507+0000", 2024-09-15T21:47:37.982 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4699999999999999e-06 2024-09-15T21:47:37.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.982 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.982 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.982 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.025935+0000", 2024-09-15T21:47:37.983 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.55042854200000002 2024-09-15T21:47:37.983 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.983 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.983 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.983 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.025965+0000", 2024-09-15T21:47:37.983 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9995999999999999e-05 2024-09-15T21:47:37.983 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.983 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.983 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.983 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026016+0000", 2024-09-15T21:47:37.984 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1310000000000002e-05 2024-09-15T21:47:37.984 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.984 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.984 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:37.984 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026122+0000", 2024-09-15T21:47:37.984 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105605 2024-09-15T21:47:37.984 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.984 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.984 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:37.984 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026962+0000", 2024-09-15T21:47:37.984 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00083998999999999999 2024-09-15T21:47:37.984 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.984 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:37.984 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.985 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.985 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.985 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:5 2.e 2:75ee5fd8:::629.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:37.985 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:37.985 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.319238118000001, 2024-09-15T21:47:37.985 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0071955470000000001, 2024-09-15T21:47:37.985 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.985 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.985 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:37.985 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.985 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.985 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.986 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:37.986 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.986 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.986 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.986 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:37.986 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.987 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.987 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.987 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564713+0000", 2024-09-15T21:47:37.987 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.866e-06 2024-09-15T21:47:37.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.987 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.987 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.987 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564721+0000", 2024-09-15T21:47:37.987 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3299999999999999e-06 2024-09-15T21:47:37.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.987 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.987 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.987 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564762+0000", 2024-09-15T21:47:37.987 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0707999999999997e-05 2024-09-15T21:47:37.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.988 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.988 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.988 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564774+0000", 2024-09-15T21:47:37.988 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1883e-05 2024-09-15T21:47:37.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.988 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.988 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.988 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564877+0000", 2024-09-15T21:47:37.988 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010254700000000001 2024-09-15T21:47:37.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.988 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.988 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:37.988 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565066+0000", 2024-09-15T21:47:37.989 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018933700000000001 2024-09-15T21:47:37.989 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.989 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.989 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:37.989 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565164+0000", 2024-09-15T21:47:37.989 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7633999999999994e-05 2024-09-15T21:47:37.989 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.989 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.989 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:37.989 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569030+0000", 2024-09-15T21:47:37.989 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038669070000000002 2024-09-15T21:47:37.989 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.989 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.989 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:37.990 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571818+0000", 2024-09-15T21:47:37.990 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027878930000000001 2024-09-15T21:47:37.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.990 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.990 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:37.990 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571868+0000", 2024-09-15T21:47:37.990 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9880999999999999e-05 2024-09-15T21:47:37.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.990 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.990 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:37.990 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571907+0000", 2024-09-15T21:47:37.990 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8560999999999999e-05 2024-09-15T21:47:37.990 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.991 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:37.991 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.991 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.991 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.991 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:6 2.e 2:75ee5fd8:::629.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:37.991 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:37.991 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.319223991000001, 2024-09-15T21:47:37.991 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073433220000000002, 2024-09-15T21:47:37.991 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.991 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.991 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:37.991 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.991 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.991 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.992 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:37.992 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.992 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.992 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.992 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.992 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:37.992 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.992 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.992 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.992 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.992 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564726+0000", 2024-09-15T21:47:37.992 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7399999999999999e-07 2024-09-15T21:47:37.992 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.992 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.993 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.993 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564728+0000", 2024-09-15T21:47:37.993 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.447e-06 2024-09-15T21:47:37.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.993 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.993 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.993 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565446+0000", 2024-09-15T21:47:37.993 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00071774999999999996 2024-09-15T21:47:37.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.993 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.993 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.993 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565453+0000", 2024-09-15T21:47:37.993 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3239999999999997e-06 2024-09-15T21:47:37.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.994 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.994 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.994 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565515+0000", 2024-09-15T21:47:37.994 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1768999999999998e-05 2024-09-15T21:47:37.994 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.994 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.994 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:37.994 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565591+0000", 2024-09-15T21:47:37.994 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5873999999999999e-05 2024-09-15T21:47:37.994 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.994 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.994 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:37.994 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565709+0000", 2024-09-15T21:47:37.994 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011838899999999999 2024-09-15T21:47:37.995 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.995 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.995 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:37.995 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569039+0000", 2024-09-15T21:47:37.995 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0033292170000000002 2024-09-15T21:47:37.995 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.995 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.995 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:37.995 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571977+0000", 2024-09-15T21:47:37.995 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0029387829999999999 2024-09-15T21:47:37.995 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.995 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.995 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:37.996 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.572032+0000", 2024-09-15T21:47:37.996 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4540999999999998e-05 2024-09-15T21:47:37.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.996 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.996 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:37.996 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.572069+0000", 2024-09-15T21:47:37.996 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6854000000000003e-05 2024-09-15T21:47:37.996 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.996 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:37.996 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:37.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.996 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.996 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:2 2.1 2:82971f97:::404.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:37.997 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:37.997 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.319220024000003, 2024-09-15T21:47:37.997 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007245357, 2024-09-15T21:47:37.997 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:37.997 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:37.997 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:37.997 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:37.997 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.997 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:37.997 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:37.997 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.997 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.997 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.997 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:37.998 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:37.998 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:37.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.998 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.998 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:37.998 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564730+0000", 2024-09-15T21:47:37.998 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2999999999999999e-07 2024-09-15T21:47:37.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.998 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.998 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:37.998 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564731+0000", 2024-09-15T21:47:37.998 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.367e-06 2024-09-15T21:47:37.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.998 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.999 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:37.999 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565484+0000", 2024-09-15T21:47:37.999 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075311599999999996 2024-09-15T21:47:37.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.999 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.999 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:37.999 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565490+0000", 2024-09-15T21:47:37.999 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8100000000000003e-06 2024-09-15T21:47:37.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:37.999 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:37.999 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:37.999 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565828+0000", 2024-09-15T21:47:37.999 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000337836 2024-09-15T21:47:37.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.000 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.000 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:38.000 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565964+0000", 2024-09-15T21:47:38.000 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013641599999999999 2024-09-15T21:47:38.000 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.000 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.000 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:38.000 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566071+0000", 2024-09-15T21:47:38.000 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000107126 2024-09-15T21:47:38.000 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.000 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.000 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:38.000 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571361+0000", 2024-09-15T21:47:38.000 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052896210000000004 2024-09-15T21:47:38.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.001 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.001 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:38.001 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571874+0000", 2024-09-15T21:47:38.001 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051320599999999999 2024-09-15T21:47:38.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.001 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.001 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:38.001 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571936+0000", 2024-09-15T21:47:38.001 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1650999999999996e-05 2024-09-15T21:47:38.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.001 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.001 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.001 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571975+0000", 2024-09-15T21:47:38.002 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8977999999999998e-05 2024-09-15T21:47:38.002 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.002 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.002 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.002 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.002 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.002 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:38.003 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:47:38.004 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:38.002+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:38.104 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:38.103+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:38.177 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:38.177 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:38.177 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:38.177 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:38.189 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:47:38.205 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:38.204+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:38.306 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:38.305+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:38.335 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:38.335 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:38.335 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:38.335 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:38.335 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:38.345 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:47:38.429 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:38.428+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:38.494 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:38.495 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:38.495 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:38.495 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:38.495 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.495 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:38.495 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:47:38.495 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 70.077443337999995, 2024-09-15T21:47:38.495 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047486860000000002, 2024-09-15T21:47:38.495 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.495 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.496 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:38.496 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.496 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.496 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.496 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:47:38.496 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.496 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.496 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.496 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:47:38.496 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.497 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.497 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.497 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413816+0000", 2024-09-15T21:47:38.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4810000000000003e-06 2024-09-15T21:47:38.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.497 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.497 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.497 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413827+0000", 2024-09-15T21:47:38.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.187e-05 2024-09-15T21:47:38.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.497 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.498 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.498 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413862+0000", 2024-09-15T21:47:38.498 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4041000000000002e-05 2024-09-15T21:47:38.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.498 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.498 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413871+0000", 2024-09-15T21:47:38.498 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8919999999999995e-06 2024-09-15T21:47:38.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.498 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.499 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413997+0000", 2024-09-15T21:47:38.499 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012601299999999999 2024-09-15T21:47:38.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.499 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:38.499 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414100+0000", 2024-09-15T21:47:38.499 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102555 2024-09-15T21:47:38.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.499 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.499 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418558+0000", 2024-09-15T21:47:38.499 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044578339999999999 2024-09-15T21:47:38.500 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.500 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.500 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.500 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.500 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:38.500 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:47:38.500 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 70.073798523999997, 2024-09-15T21:47:38.500 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045423590000000002, 2024-09-15T21:47:38.500 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.500 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.501 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:38.501 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.501 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.501 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.501 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:47:38.501 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.501 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.501 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.501 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.501 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:47:38.501 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.501 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.502 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.502 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.502 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417455+0000", 2024-09-15T21:47:38.502 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9100000000000003e-07 2024-09-15T21:47:38.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.502 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.502 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.502 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417463+0000", 2024-09-15T21:47:38.502 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3750000000000003e-06 2024-09-15T21:47:38.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.502 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.503 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.503 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417475+0000", 2024-09-15T21:47:38.503 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2473000000000001e-05 2024-09-15T21:47:38.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.503 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.503 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.503 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417482+0000", 2024-09-15T21:47:38.503 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4010000000000002e-06 2024-09-15T21:47:38.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.503 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.503 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.504 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417524+0000", 2024-09-15T21:47:38.504 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1870999999999999e-05 2024-09-15T21:47:38.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.504 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.504 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:38.504 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417542+0000", 2024-09-15T21:47:38.504 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8224e-05 2024-09-15T21:47:38.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.504 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.504 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:38.504 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421955+0000", 2024-09-15T21:47:38.504 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044130740000000003 2024-09-15T21:47:38.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.505 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.505 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.505 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421996+0000", 2024-09-15T21:47:38.505 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.125e-05 2024-09-15T21:47:38.505 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.505 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.505 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.505 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.505 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 7 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:38.506 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:38.506 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 70.073537719000001, 2024-09-15T21:47:38.506 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043086519999999996, 2024-09-15T21:47:38.506 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.506 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.506 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:47:38.506 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.506 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.506 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.506 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:38.507 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.507 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.507 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.507 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.507 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:38.507 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.507 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.507 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.507 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.507 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:38.507 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.58e-07 2024-09-15T21:47:38.507 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.508 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.508 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.508 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417722+0000", 2024-09-15T21:47:38.508 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4729999999999997e-06 2024-09-15T21:47:38.508 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.508 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.508 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.508 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417737+0000", 2024-09-15T21:47:38.508 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5155e-05 2024-09-15T21:47:38.508 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.508 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.509 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.509 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417752+0000", 2024-09-15T21:47:38.509 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4863e-05 2024-09-15T21:47:38.509 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.509 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.509 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.509 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417819+0000", 2024-09-15T21:47:38.509 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7680000000000003e-05 2024-09-15T21:47:38.509 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.509 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.509 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.510 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422023+0000", 2024-09-15T21:47:38.510 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042040230000000003 2024-09-15T21:47:38.510 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.510 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.510 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.510 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.510 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.510 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:25 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:38.510 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:47:38.511 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 64.034048689000002, 2024-09-15T21:47:38.511 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070572830000000001, 2024-09-15T21:47:38.511 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.511 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.511 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:38.511 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.511 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.511 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.511 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:47:38.511 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.511 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.511 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.512 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.512 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:47:38.512 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.512 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.512 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.512 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.512 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457205+0000", 2024-09-15T21:47:38.512 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6899999999999999e-06 2024-09-15T21:47:38.513 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.513 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.513 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.513 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457211+0000", 2024-09-15T21:47:38.513 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.626e-06 2024-09-15T21:47:38.513 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.513 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.513 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.513 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457244+0000", 2024-09-15T21:47:38.513 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2691999999999998e-05 2024-09-15T21:47:38.513 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.513 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.514 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.514 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457251+0000", 2024-09-15T21:47:38.514 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3839999999999999e-06 2024-09-15T21:47:38.514 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.514 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.514 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.514 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457309+0000", 2024-09-15T21:47:38.514 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8023999999999999e-05 2024-09-15T21:47:38.514 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.514 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.515 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:38.515 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457413+0000", 2024-09-15T21:47:38.515 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010331999999999999 2024-09-15T21:47:38.515 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.515 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.515 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:38.515 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:38.515 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118439 2024-09-15T21:47:38.515 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.515 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.515 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:38.516 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463246+0000", 2024-09-15T21:47:38.516 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057148219999999996 2024-09-15T21:47:38.516 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.516 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.516 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:38.516 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463428+0000", 2024-09-15T21:47:38.516 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018218299999999999 2024-09-15T21:47:38.516 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.516 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.516 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:38.516 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464212+0000", 2024-09-15T21:47:38.516 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00078402799999999998 2024-09-15T21:47:38.517 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.517 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.517 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.517 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464261+0000", 2024-09-15T21:47:38.517 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9075e-05 2024-09-15T21:47:38.517 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.517 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.517 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.517 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.517 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.517 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:28 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:38.518 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:38.518 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 64.034038662, 2024-09-15T21:47:38.518 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0072141449999999999, 2024-09-15T21:47:38.518 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.518 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.518 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:38.518 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.518 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.518 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.518 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:38.518 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.519 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.519 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.519 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.519 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:38.519 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.519 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.519 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.519 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.519 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:38.519 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:47:38.519 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.519 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.520 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.520 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457217+0000", 2024-09-15T21:47:38.520 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7259999999999998e-06 2024-09-15T21:47:38.520 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.520 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.520 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.520 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457842+0000", 2024-09-15T21:47:38.520 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00062482599999999998 2024-09-15T21:47:38.520 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.520 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.521 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.521 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457848+0000", 2024-09-15T21:47:38.521 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7089999999999996e-06 2024-09-15T21:47:38.521 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.521 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.521 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.521 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457895+0000", 2024-09-15T21:47:38.521 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6317999999999999e-05 2024-09-15T21:47:38.521 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.521 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.521 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:38.521 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458013+0000", 2024-09-15T21:47:38.522 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011848100000000001 2024-09-15T21:47:38.522 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.522 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.522 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.522 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464428+0000", 2024-09-15T21:47:38.522 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0064146990000000003 2024-09-15T21:47:38.522 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.522 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.522 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.522 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.522 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.522 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:29 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:38.523 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:47:38.523 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 64.034034219000006, 2024-09-15T21:47:38.523 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073658980000000001, 2024-09-15T21:47:38.523 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.523 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.523 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:38.523 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.523 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.523 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.523 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:47:38.523 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.524 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.524 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.524 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.524 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:47:38.524 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.524 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.524 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.524 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.524 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457219+0000", 2024-09-15T21:47:38.524 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2000000000000001e-07 2024-09-15T21:47:38.524 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.525 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.525 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.525 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457226+0000", 2024-09-15T21:47:38.525 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.956e-06 2024-09-15T21:47:38.525 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.525 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.525 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.525 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458266+0000", 2024-09-15T21:47:38.525 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001040476 2024-09-15T21:47:38.525 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.525 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.526 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.526 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458273+0000", 2024-09-15T21:47:38.526 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0690000000000004e-06 2024-09-15T21:47:38.526 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.526 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.526 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.526 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458319+0000", 2024-09-15T21:47:38.526 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5668999999999999e-05 2024-09-15T21:47:38.526 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.526 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.526 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:38.526 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458415+0000", 2024-09-15T21:47:38.527 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6701999999999997e-05 2024-09-15T21:47:38.527 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.527 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.527 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.527 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464584+0000", 2024-09-15T21:47:38.527 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061687060000000004 2024-09-15T21:47:38.527 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.527 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.527 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.527 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.527 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.528 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.1 epoch 22/19 rep_tid 12 entries 22'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.10:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:38.528 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:47:38.528 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 64.032604212999999, 2024-09-15T21:47:38.528 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0039881550000000002, 2024-09-15T21:47:38.528 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.528 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.528 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:47:38.528 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.528 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.528 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.528 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:47:38.529 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.529 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.529 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.529 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.529 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:47:38.529 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.529 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.529 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.529 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.529 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458649+0000", 2024-09-15T21:47:38.530 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9599999999999999e-07 2024-09-15T21:47:38.530 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.530 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.530 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.530 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458656+0000", 2024-09-15T21:47:38.530 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2629999999999997e-06 2024-09-15T21:47:38.530 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.530 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.530 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.530 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458670+0000", 2024-09-15T21:47:38.530 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4166e-05 2024-09-15T21:47:38.531 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.531 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.531 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.531 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458682+0000", 2024-09-15T21:47:38.531 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1865000000000001e-05 2024-09-15T21:47:38.531 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.531 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.531 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.531 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458749+0000", 2024-09-15T21:47:38.531 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6507000000000005e-05 2024-09-15T21:47:38.531 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.531 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.532 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.532 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462636+0000", 2024-09-15T21:47:38.532 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038877579999999998 2024-09-15T21:47:38.532 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.532 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.532 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.532 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.533 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.533 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.1.10:12 2.12 e22/19 2:48cd7578:::60e.00000000:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:38.533 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:38.533 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 64.032289880999997, 2024-09-15T21:47:38.533 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042479500000000003, 2024-09-15T21:47:38.533 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.533 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.533 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:38.533 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.533 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.533 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.534 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:38.534 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.534 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.534 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.534 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.534 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:38.534 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.534 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.534 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.534 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.534 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:38.535 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2900000000000004e-07 2024-09-15T21:47:38.535 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.535 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.535 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.535 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458968+0000", 2024-09-15T21:47:38.535 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2039999999999996e-06 2024-09-15T21:47:38.535 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.535 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.536 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.536 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458977+0000", 2024-09-15T21:47:38.536 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.6179999999999995e-06 2024-09-15T21:47:38.536 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.536 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.536 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.536 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458983+0000", 2024-09-15T21:47:38.536 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9510000000000003e-06 2024-09-15T21:47:38.537 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.537 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.537 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.537 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459479+0000", 2024-09-15T21:47:38.537 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000496274 2024-09-15T21:47:38.537 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.537 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.537 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:38.537 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459496+0000", 2024-09-15T21:47:38.537 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6538e-05 2024-09-15T21:47:38.537 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.537 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.538 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:38.538 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463169+0000", 2024-09-15T21:47:38.538 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003672832 2024-09-15T21:47:38.538 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.538 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.538 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.538 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463211+0000", 2024-09-15T21:47:38.538 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2004e-05 2024-09-15T21:47:38.538 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.538 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.538 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.538 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.539 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.539 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.12 epoch 22/19 rep_tid 2 entries 22'3 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.10:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:38.539 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:47:38.539 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 64.031610326999996, 2024-09-15T21:47:38.539 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035859609999999999, 2024-09-15T21:47:38.539 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.539 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.539 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:47:38.539 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.539 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.540 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.540 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:47:38.540 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.540 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.540 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.540 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.540 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:47:38.540 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.540 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.540 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.541 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.541 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459643+0000", 2024-09-15T21:47:38.541 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2200000000000004e-07 2024-09-15T21:47:38.541 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.541 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.541 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.541 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459650+0000", 2024-09-15T21:47:38.541 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4479999999999997e-06 2024-09-15T21:47:38.541 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.541 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.541 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.542 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459665+0000", 2024-09-15T21:47:38.542 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4586999999999999e-05 2024-09-15T21:47:38.542 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.542 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.542 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.542 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459676+0000", 2024-09-15T21:47:38.542 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1673e-05 2024-09-15T21:47:38.542 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.542 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.542 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.542 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459732+0000", 2024-09-15T21:47:38.542 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5612999999999998e-05 2024-09-15T21:47:38.543 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.543 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.543 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.543 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463228+0000", 2024-09-15T21:47:38.543 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0034961179999999999 2024-09-15T21:47:38.543 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.543 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.543 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.543 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.543 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.544 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:34 2.1b 2:db457a8a:::201.00000001:head [write 0~588] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:38.544 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:47:38.544 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 64.026398690999997, 2024-09-15T21:47:38.544 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070946129999999996, 2024-09-15T21:47:38.544 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.544 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.544 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:38.545 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.545 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.545 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.545 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:47:38.545 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.545 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.545 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.545 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.545 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:47:38.545 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.545 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.546 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.546 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.546 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464855+0000", 2024-09-15T21:47:38.546 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5880000000000001e-06 2024-09-15T21:47:38.546 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.546 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.546 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.546 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464863+0000", 2024-09-15T21:47:38.546 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0879999999999995e-06 2024-09-15T21:47:38.546 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.546 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.546 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.547 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464889+0000", 2024-09-15T21:47:38.547 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5729e-05 2024-09-15T21:47:38.547 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.547 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.547 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.547 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464898+0000", 2024-09-15T21:47:38.547 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0389999999999993e-06 2024-09-15T21:47:38.547 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.547 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.547 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.548 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464962+0000", 2024-09-15T21:47:38.548 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3567999999999999e-05 2024-09-15T21:47:38.548 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.548 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.548 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:38.548 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.465041+0000", 2024-09-15T21:47:38.548 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8889999999999999e-05 2024-09-15T21:47:38.548 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.548 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.548 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:38.548 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.465173+0000", 2024-09-15T21:47:38.549 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013226899999999999 2024-09-15T21:47:38.549 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.549 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.549 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:38.549 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.466319+0000", 2024-09-15T21:47:38.549 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011463039999999999 2024-09-15T21:47:38.549 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.549 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.549 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:38.549 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471825+0000", 2024-09-15T21:47:38.549 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055062160000000004 2024-09-15T21:47:38.550 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.550 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.550 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:38.550 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471886+0000", 2024-09-15T21:47:38.550 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0503e-05 2024-09-15T21:47:38.550 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.550 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.550 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.550 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471948+0000", 2024-09-15T21:47:38.551 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2419e-05 2024-09-15T21:47:38.551 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.551 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.551 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.551 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.551 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.551 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:38.551 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:47:38.551 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.015740526999998, 2024-09-15T21:47:38.551 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047360500000000003, 2024-09-15T21:47:38.551 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.552 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.552 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:38.552 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.552 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.552 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.552 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:47:38.552 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.552 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.553 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.553 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.553 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:47:38.553 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.553 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.553 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.553 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.553 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475514+0000", 2024-09-15T21:47:38.553 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0990000000000001e-06 2024-09-15T21:47:38.553 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.553 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.553 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.554 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475524+0000", 2024-09-15T21:47:38.554 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5850000000000006e-06 2024-09-15T21:47:38.554 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.554 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.554 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.554 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475557+0000", 2024-09-15T21:47:38.554 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3090000000000003e-05 2024-09-15T21:47:38.554 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.554 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.554 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.554 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475568+0000", 2024-09-15T21:47:38.555 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1116e-05 2024-09-15T21:47:38.555 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.555 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.555 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.555 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475649+0000", 2024-09-15T21:47:38.555 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0815999999999993e-05 2024-09-15T21:47:38.555 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.555 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.555 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:38.555 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475805+0000", 2024-09-15T21:47:38.555 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015667200000000001 2024-09-15T21:47:38.555 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.556 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.556 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:38.556 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475944+0000", 2024-09-15T21:47:38.556 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000138651 2024-09-15T21:47:38.556 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.556 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.556 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:38.556 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476900+0000", 2024-09-15T21:47:38.556 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00095588699999999999 2024-09-15T21:47:38.556 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.556 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.557 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:38.557 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480153+0000", 2024-09-15T21:47:38.557 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0032535200000000002 2024-09-15T21:47:38.557 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.557 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.557 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:38.557 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480203+0000", 2024-09-15T21:47:38.557 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9932999999999998e-05 2024-09-15T21:47:38.557 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.557 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.557 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.557 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480248+0000", 2024-09-15T21:47:38.558 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4681000000000001e-05 2024-09-15T21:47:38.558 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.558 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.558 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.558 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.558 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.558 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:38.558 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:38.558 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.015724585999997, 2024-09-15T21:47:38.558 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049206709999999997, 2024-09-15T21:47:38.558 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.559 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.559 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:38.559 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.559 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.559 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.559 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:38.559 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.559 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.559 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.559 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.559 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:38.560 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.560 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.560 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.560 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.560 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:38.560 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1699999999999999e-07 2024-09-15T21:47:38.560 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.560 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.560 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.560 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475531+0000", 2024-09-15T21:47:38.560 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.05e-06 2024-09-15T21:47:38.560 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.561 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.561 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.561 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475743+0000", 2024-09-15T21:47:38.561 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021138099999999999 2024-09-15T21:47:38.561 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.561 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.561 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.561 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475751+0000", 2024-09-15T21:47:38.561 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7770000000000002e-06 2024-09-15T21:47:38.561 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.562 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.562 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.562 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476272+0000", 2024-09-15T21:47:38.562 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052158399999999996 2024-09-15T21:47:38.562 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.562 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.562 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:38.562 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476348+0000", 2024-09-15T21:47:38.562 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5963000000000004e-05 2024-09-15T21:47:38.563 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.563 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.563 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:38.563 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476474+0000", 2024-09-15T21:47:38.563 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012638800000000001 2024-09-15T21:47:38.563 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.563 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.563 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:38.563 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478194+0000", 2024-09-15T21:47:38.563 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001719768 2024-09-15T21:47:38.563 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.564 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.564 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:38.564 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480358+0000", 2024-09-15T21:47:38.564 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021637340000000001 2024-09-15T21:47:38.564 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.564 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.564 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:38.564 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480411+0000", 2024-09-15T21:47:38.564 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2720999999999999e-05 2024-09-15T21:47:38.564 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.564 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.564 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.565 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480449+0000", 2024-09-15T21:47:38.565 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7888e-05 2024-09-15T21:47:38.565 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.565 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.565 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.565 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.565 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.565 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:38.565 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:38.565 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.015719797000003, 2024-09-15T21:47:38.566 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005215478, 2024-09-15T21:47:38.566 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.566 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.566 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:38.566 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.566 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.566 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.566 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:38.566 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.566 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.566 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.567 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.567 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:38.567 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.567 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.567 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.567 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.567 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:38.567 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5199999999999998e-07 2024-09-15T21:47:38.567 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.567 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.567 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.567 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475535+0000", 2024-09-15T21:47:38.568 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9520000000000001e-06 2024-09-15T21:47:38.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.568 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.568 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.568 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475785+0000", 2024-09-15T21:47:38.568 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024966099999999998 2024-09-15T21:47:38.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.568 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.568 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.568 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475799+0000", 2024-09-15T21:47:38.568 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4649e-05 2024-09-15T21:47:38.569 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.569 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.569 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.569 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475856+0000", 2024-09-15T21:47:38.569 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6956000000000002e-05 2024-09-15T21:47:38.569 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.569 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.569 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:38.569 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476016+0000", 2024-09-15T21:47:38.569 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000159457 2024-09-15T21:47:38.569 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.570 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.570 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.570 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480748+0000", 2024-09-15T21:47:38.570 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047324510000000004 2024-09-15T21:47:38.570 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.570 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.570 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.570 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.570 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.570 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.2.12:1 2.1 e22/19 2:85bbe569:::202.00000000:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:38.570 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:47:38.570 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.012917326, 2024-09-15T21:47:38.570 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045423649999999996, 2024-09-15T21:47:38.571 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.571 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.571 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:38.571 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.571 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.571 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.571 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:47:38.571 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.571 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.571 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.571 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.572 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:47:38.572 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.572 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.572 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.572 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.572 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478336+0000", 2024-09-15T21:47:38.572 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4499999999999996e-07 2024-09-15T21:47:38.572 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.572 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.573 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.573 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478342+0000", 2024-09-15T21:47:38.573 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5579999999999997e-06 2024-09-15T21:47:38.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.573 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.573 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.573 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478356+0000", 2024-09-15T21:47:38.573 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3567000000000001e-05 2024-09-15T21:47:38.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.573 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.573 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.574 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478364+0000", 2024-09-15T21:47:38.574 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7889999999999995e-06 2024-09-15T21:47:38.574 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.574 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.574 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.574 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478416+0000", 2024-09-15T21:47:38.574 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2188000000000002e-05 2024-09-15T21:47:38.574 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.574 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.574 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:38.574 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478433+0000", 2024-09-15T21:47:38.575 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6674e-05 2024-09-15T21:47:38.575 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.575 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.575 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:38.575 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482832+0000", 2024-09-15T21:47:38.575 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043996110000000003 2024-09-15T21:47:38.575 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.575 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.575 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.575 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482878+0000", 2024-09-15T21:47:38.575 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5232999999999999e-05 2024-09-15T21:47:38.575 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.575 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.576 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.576 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.576 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.576 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:25 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:38.576 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:47:38.576 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.926057286999999, 2024-09-15T21:47:38.576 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058459619999999997, 2024-09-15T21:47:38.576 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.576 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.576 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:38.576 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.577 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.577 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.577 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:47:38.577 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.577 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.577 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.577 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.577 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:47:38.577 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.577 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.577 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.577 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.578 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565197+0000", 2024-09-15T21:47:38.578 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7889999999999999e-06 2024-09-15T21:47:38.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.578 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.578 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.578 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565208+0000", 2024-09-15T21:47:38.578 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1358999999999999e-05 2024-09-15T21:47:38.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.578 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.578 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.578 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565237+0000", 2024-09-15T21:47:38.578 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8626e-05 2024-09-15T21:47:38.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.579 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.579 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.579 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565249+0000", 2024-09-15T21:47:38.579 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1897e-05 2024-09-15T21:47:38.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.579 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.579 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.579 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565332+0000", 2024-09-15T21:47:38.579 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2943999999999997e-05 2024-09-15T21:47:38.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.580 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.580 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:38.580 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565452+0000", 2024-09-15T21:47:38.580 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119992 2024-09-15T21:47:38.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.580 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.580 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:38.580 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565610+0000", 2024-09-15T21:47:38.580 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015845099999999999 2024-09-15T21:47:38.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.581 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.581 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:38.581 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.568999+0000", 2024-09-15T21:47:38.581 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0033887190000000001 2024-09-15T21:47:38.581 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.581 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.581 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:38.581 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.570979+0000", 2024-09-15T21:47:38.581 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019804919999999999 2024-09-15T21:47:38.581 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.581 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.582 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:38.582 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571014+0000", 2024-09-15T21:47:38.582 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4331e-05 2024-09-15T21:47:38.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.582 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.582 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.582 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571041+0000", 2024-09-15T21:47:38.582 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7362e-05 2024-09-15T21:47:38.582 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.582 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.582 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.583 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.583 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:5 2.e e22/19 2:75ee5fd8:::629.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:47:38.583 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:47:38.583 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.925381641000001, 2024-09-15T21:47:38.583 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057947119999999996, 2024-09-15T21:47:38.583 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.583 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.583 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:38.583 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.583 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.583 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.584 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:47:38.584 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.584 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.584 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.584 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:47:38.584 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.584 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.584 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.584 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565872+0000", 2024-09-15T21:47:38.585 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9800000000000002e-07 2024-09-15T21:47:38.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.585 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.585 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.585 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565880+0000", 2024-09-15T21:47:38.585 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7619999999999995e-06 2024-09-15T21:47:38.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.585 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.585 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.585 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565890+0000", 2024-09-15T21:47:38.585 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0648999999999999e-05 2024-09-15T21:47:38.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.586 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.586 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.586 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565897+0000", 2024-09-15T21:47:38.586 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.153e-06 2024-09-15T21:47:38.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.586 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.586 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.586 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565966+0000", 2024-09-15T21:47:38.586 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9046000000000003e-05 2024-09-15T21:47:38.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.586 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.586 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:38.587 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565985+0000", 2024-09-15T21:47:38.587 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8695e-05 2024-09-15T21:47:38.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.587 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.587 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:38.587 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571625+0000", 2024-09-15T21:47:38.587 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056397139999999997 2024-09-15T21:47:38.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.588 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.588 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.588 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571666+0000", 2024-09-15T21:47:38.588 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0695000000000003e-05 2024-09-15T21:47:38.588 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.588 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.588 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.588 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.588 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.588 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:6 2.e e22/19 2:75ee5fd8:::629.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:47:38.588 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:47:38.589 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.925319393000002, 2024-09-15T21:47:38.589 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057607429999999996, 2024-09-15T21:47:38.589 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.589 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.589 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:38.589 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.589 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.589 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.589 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:47:38.589 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.590 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.590 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.590 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:47:38.590 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.590 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.590 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.590 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.590 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565934+0000", 2024-09-15T21:47:38.590 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6199999999999998e-07 2024-09-15T21:47:38.590 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.590 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.590 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.591 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565938+0000", 2024-09-15T21:47:38.591 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4970000000000003e-06 2024-09-15T21:47:38.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.591 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.591 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.591 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565945+0000", 2024-09-15T21:47:38.591 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5069999999999997e-06 2024-09-15T21:47:38.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.591 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.591 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.591 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565948+0000", 2024-09-15T21:47:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:47:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566200+0000", 2024-09-15T21:47:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025170200000000002 2024-09-15T21:47:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566218+0000", 2024-09-15T21:47:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7543000000000001e-05 2024-09-15T21:47:38.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.593 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.593 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:38.593 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571668+0000", 2024-09-15T21:47:38.593 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054502999999999999 2024-09-15T21:47:38.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.593 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.593 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.593 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571694+0000", 2024-09-15T21:47:38.593 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5979e-05 2024-09-15T21:47:38.593 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.593 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.594 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.594 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.594 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.594 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:2 2.1 e22/19 2:82971f97:::404.00000000:head v 22'5, mlcod=22'3)", 2024-09-15T21:47:38.594 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:47:38.594 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.925111457, 2024-09-15T21:47:38.594 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056016019999999998, 2024-09-15T21:47:38.594 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.594 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.594 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:38.594 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.595 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.595 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.595 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:47:38.595 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.595 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.595 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.595 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:47:38.595 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.595 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.595 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.596 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566142+0000", 2024-09-15T21:47:38.596 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:47:38.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.596 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.596 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.596 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566149+0000", 2024-09-15T21:47:38.596 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7510000000000008e-06 2024-09-15T21:47:38.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.596 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.596 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.596 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566161+0000", 2024-09-15T21:47:38.596 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1259e-05 2024-09-15T21:47:38.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.597 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.597 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.597 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566168+0000", 2024-09-15T21:47:38.597 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.255e-06 2024-09-15T21:47:38.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.597 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.597 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.597 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566227+0000", 2024-09-15T21:47:38.597 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8961e-05 2024-09-15T21:47:38.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.598 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.598 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:38.598 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566247+0000", 2024-09-15T21:47:38.598 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0536000000000001e-05 2024-09-15T21:47:38.598 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.598 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.598 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:38.598 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571694+0000", 2024-09-15T21:47:38.598 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005446762 2024-09-15T21:47:38.598 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.598 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.598 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.599 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571743+0000", 2024-09-15T21:47:38.599 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8396999999999998e-05 2024-09-15T21:47:38.599 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.599 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.599 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.599 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.599 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.599 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:17 2.14 e22/19 2:2eebc8f4:::62f.00000000:head v 22'9, mlcod=22'7)", 2024-09-15T21:47:38.599 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:47:38.599 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.92509536, 2024-09-15T21:47:38.600 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056532860000000004, 2024-09-15T21:47:38.600 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.600 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.600 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:38.600 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.600 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.600 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.600 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:47:38.600 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.600 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.600 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.601 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.601 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:47:38.601 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.601 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.601 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.601 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566158+0000", 2024-09-15T21:47:38.601 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.92e-07 2024-09-15T21:47:38.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.601 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.601 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.601 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566172+0000", 2024-09-15T21:47:38.602 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3976e-05 2024-09-15T21:47:38.602 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.602 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.602 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.602 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566407+0000", 2024-09-15T21:47:38.602 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023483900000000001 2024-09-15T21:47:38.602 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.602 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.602 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.602 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566416+0000", 2024-09-15T21:47:38.603 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0780000000000002e-06 2024-09-15T21:47:38.603 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.603 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.603 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.603 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566476+0000", 2024-09-15T21:47:38.603 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9908999999999998e-05 2024-09-15T21:47:38.603 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.603 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.603 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:38.603 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566493+0000", 2024-09-15T21:47:38.603 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7567e-05 2024-09-15T21:47:38.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.604 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.604 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:38.604 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571762+0000", 2024-09-15T21:47:38.604 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052682959999999996 2024-09-15T21:47:38.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.604 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.604 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.604 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571810+0000", 2024-09-15T21:47:38.604 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8829000000000003e-05 2024-09-15T21:47:38.604 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.605 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.605 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.605 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.605 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:18 2.14 e22/19 2:2eebc8f4:::62f.00000000:head v 22'10, mlcod=22'7)", 2024-09-15T21:47:38.605 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:47:38.605 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 46.924806285000003, 2024-09-15T21:47:38.605 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053992579999999997, 2024-09-15T21:47:38.605 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:38.605 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:38.605 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:38.605 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:38.606 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.606 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:38.606 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:47:38.607 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.607 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.608 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:38.608 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:47:38.608 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:38.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.608 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.608 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:38.608 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566447+0000", 2024-09-15T21:47:38.608 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2e-07 2024-09-15T21:47:38.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.609 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.609 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:38.609 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566452+0000", 2024-09-15T21:47:38.609 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0390000000000004e-06 2024-09-15T21:47:38.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.609 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.609 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:38.609 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566460+0000", 2024-09-15T21:47:38.609 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1389999999999995e-06 2024-09-15T21:47:38.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.609 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.609 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:38.610 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566473+0000", 2024-09-15T21:47:38.610 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3093e-05 2024-09-15T21:47:38.610 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.610 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.610 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:38.610 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566688+0000", 2024-09-15T21:47:38.610 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021537100000000001 2024-09-15T21:47:38.610 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.610 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.610 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:38.610 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566706+0000", 2024-09-15T21:47:38.610 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7802999999999999e-05 2024-09-15T21:47:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571813+0000", 2024-09-15T21:47:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051065140000000004 2024-09-15T21:47:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571845+0000", 2024-09-15T21:47:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2879000000000001e-05 2024-09-15T21:47:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.611 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.612 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.612 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:38.612 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:38.612 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:38.613 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:47:38.614 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:38.611+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:38.714 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:38.713+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:38.728 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:38.728 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:38.728 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:38.728 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:38.737 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:47:38.804 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:47:38.821 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:38.820+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:38.881 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:38.882 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:38.882 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:38.882 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:38.882 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:38.890 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:47:38.932 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:38.930+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:38.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.948+0000 7f13cb531640 1 Processor -- start 2024-09-15T21:47:38.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.948+0000 7f13cb531640 1 -- start start 2024-09-15T21:47:38.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.949+0000 7f13cb531640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f13c40cc8f0 0x7f13c41669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:38.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.949+0000 7f13cb531640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c415e7a0 con 0x7f13c4155440 2024-09-15T21:47:38.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.949+0000 7f13cb531640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c415ea40 con 0x7f13c40cb7d0 2024-09-15T21:47:38.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.949+0000 7f13cb531640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f13c415b470 con 0x7f13c40cc8f0 2024-09-15T21:47:38.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.949+0000 7f13c92cd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f13c4155440 0x7f13c40cd420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47908/0 (socket says 172.21.15.100:47908) 2024-09-15T21:47:38.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.949+0000 7f13c9ace640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f13c40cc8f0 0x7f13c41669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:38.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.949+0000 7f13c92cd640 1 -- 172.21.15.100:0/3204425633 learned_addr learned my addr 172.21.15.100:0/3204425633 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:38.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.950+0000 7f13bbfff640 1 -- 172.21.15.100:0/3204425633 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1052307512 0 0) 0x7f13c415ea40 con 0x7f13c40cb7d0 2024-09-15T21:47:38.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.950+0000 7f13bbfff640 1 -- 172.21.15.100:0/3204425633 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13b0003520 con 0x7f13c40cb7d0 2024-09-15T21:47:38.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.950+0000 7f13bbfff640 1 -- 172.21.15.100:0/3204425633 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 867428723 0 0) 0x7f13c415e7a0 con 0x7f13c4155440 2024-09-15T21:47:38.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.950+0000 7f13c9ace640 1 -- 172.21.15.100:0/3204425633 >> v1:172.21.15.122:6789/0 conn(0x7f13c40cb7d0 legacy=0x7f13c40cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:38.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.950+0000 7f13c9ace640 1 -- 172.21.15.100:0/3204425633 >> v1:172.21.15.100:6789/0 conn(0x7f13c4155440 legacy=0x7f13c40cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:38.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.950+0000 7f13c9ace640 1 -- 172.21.15.100:0/3204425633 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c415ed20 con 0x7f13c40cc8f0 2024-09-15T21:47:38.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.951+0000 7f13c9ace640 1 --2- 172.21.15.100:0/3204425633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13c40cc8f0 0x7f13c41669a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009d90 tx=0x7f13bc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978db6715356c24 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:38.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.951+0000 7f13bbfff640 1 -- 172.21.15.100:0/3204425633 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc007740 con 0x7f13c40cc8f0 2024-09-15T21:47:38.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.951+0000 7f13bbfff640 1 -- 172.21.15.100:0/3204425633 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13bc0078e0 con 0x7f13c40cc8f0 2024-09-15T21:47:38.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.951+0000 7f13bbfff640 1 -- 172.21.15.100:0/3204425633 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13bc007cb0 con 0x7f13c40cc8f0 2024-09-15T21:47:38.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.952+0000 7f13cb531640 1 -- 172.21.15.100:0/3204425633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13c40cc8f0 msgr2=0x7f13c41669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:38.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.952+0000 7f13cb531640 1 --2- 172.21.15.100:0/3204425633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13c40cc8f0 0x7f13c41669a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009d90 tx=0x7f13bc02a6f0 comp rx=0 tx=0).stop 2024-09-15T21:47:38.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.952+0000 7f13cb531640 1 -- 172.21.15.100:0/3204425633 shutdown_connections 2024-09-15T21:47:38.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.952+0000 7f13cb531640 1 --2- 172.21.15.100:0/3204425633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13c40cc8f0 0x7f13c41669a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:38.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.952+0000 7f13cb531640 1 -- 172.21.15.100:0/3204425633 >> 172.21.15.100:0/3204425633 conn(0x7f13c40680d0 msgr2=0x7f13c406d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:38.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.952+0000 7f13cb531640 1 -- 172.21.15.100:0/3204425633 shutdown_connections 2024-09-15T21:47:38.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.952+0000 7f13cb531640 1 -- 172.21.15.100:0/3204425633 wait complete. 2024-09-15T21:47:38.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.954+0000 7f13cb531640 1 Processor -- start 2024-09-15T21:47:38.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.954+0000 7f13cb531640 1 -- start start 2024-09-15T21:47:38.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.955+0000 7f13cb531640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13c40cb7d0 0x7f13c4154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:38.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.955+0000 7f13cb531640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13c40cc8f0 0x7f13c41f1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:38.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.955+0000 7f13cb531640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13c4155440 0x7f13c415c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:38.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.955+0000 7f13cb531640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f13c415f2e0 con 0x7f13c4155440 2024-09-15T21:47:38.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.955+0000 7f13cb531640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f13c4155820 con 0x7f13c40cb7d0 2024-09-15T21:47:38.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.955+0000 7f13cb531640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f13c415b070 con 0x7f13c40cc8f0 2024-09-15T21:47:38.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.955+0000 7f13c92cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13c40cb7d0 0x7f13c4154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:38.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.955+0000 7f13c9ace640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13c4155440 0x7f13c415c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:38.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.955+0000 7f13c92cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13c40cb7d0 0x7f13c4154720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:59686/0 (socket says 172.21.15.100:59686) 2024-09-15T21:47:38.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.955+0000 7f13c92cd640 1 -- 172.21.15.100:0/1301745173 learned_addr learned my addr 172.21.15.100:0/1301745173 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:38.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.955+0000 7f13c9ace640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13c4155440 0x7f13c415c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33700/0 (socket says 172.21.15.100:33700) 2024-09-15T21:47:38.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.955+0000 7f13c8acc640 1 --2- 172.21.15.100:0/1301745173 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13c40cc8f0 0x7f13c41f1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:38.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.956+0000 7f13c8acc640 1 -- 172.21.15.100:0/1301745173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13c40cb7d0 msgr2=0x7f13c4154720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:38.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.956+0000 7f13c8acc640 1 --2- 172.21.15.100:0/1301745173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13c40cb7d0 0x7f13c4154720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:38.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.956+0000 7f13c8acc640 1 -- 172.21.15.100:0/1301745173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13c4155440 msgr2=0x7f13c415c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:38.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.956+0000 7f13c8acc640 1 --2- 172.21.15.100:0/1301745173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13c4155440 0x7f13c415c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:38.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.956+0000 7f13c8acc640 1 -- 172.21.15.100:0/1301745173 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c40b4550 con 0x7f13c40cc8f0 2024-09-15T21:47:38.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.956+0000 7f13c9ace640 1 --2- 172.21.15.100:0/1301745173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13c4155440 0x7f13c415c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:38.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.956+0000 7f13c8acc640 1 --2- 172.21.15.100:0/1301745173 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13c40cc8f0 0x7f13c41f1510 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f13b400c3c0 tx=0x7f13b4001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:38.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.956+0000 7f13b9ffb640 1 -- 172.21.15.100:0/1301745173 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4017070 con 0x7f13c40cc8f0 2024-09-15T21:47:38.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.956+0000 7f13b9ffb640 1 -- 172.21.15.100:0/1301745173 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b40052d0 con 0x7f13c40cc8f0 2024-09-15T21:47:38.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.956+0000 7f13cb531640 1 -- 172.21.15.100:0/1301745173 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c40b3f90 con 0x7f13c40cc8f0 2024-09-15T21:47:38.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.956+0000 7f13b9ffb640 1 -- 172.21.15.100:0/1301745173 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b4003830 con 0x7f13c40cc8f0 2024-09-15T21:47:38.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.956+0000 7f13cb531640 1 -- 172.21.15.100:0/1301745173 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c41f0d10 con 0x7f13c40cc8f0 2024-09-15T21:47:38.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.957+0000 7f13cb531640 1 -- 172.21.15.100:0/1301745173 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c40b3940 con 0x7f13c40cc8f0 2024-09-15T21:47:38.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.959+0000 7f13b9ffb640 1 -- 172.21.15.100:0/1301745173 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f13b401a070 con 0x7f13c40cc8f0 2024-09-15T21:47:38.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.959+0000 7f13b9ffb640 1 --2- 172.21.15.100:0/1301745173 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f13a806c2b0 0x7f13a806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:38.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.959+0000 7f13b9ffb640 1 -- 172.21.15.100:0/1301745173 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(38..38 src has 1..38) ==== 5854+0+0 (secure 0 0 0) 0x7f13b408a6a0 con 0x7f13c40cc8f0 2024-09-15T21:47:38.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.959+0000 7f13c92cd640 1 --2- 172.21.15.100:0/1301745173 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f13a806c2b0 0x7f13a806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:38.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.960+0000 7f13c92cd640 1 --2- 172.21.15.100:0/1301745173 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f13a806c2b0 0x7f13a806e770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f13c0001110 tx=0x7f13c0001080 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:38.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:38.966+0000 7f13b9ffb640 1 -- 172.21.15.100:0/1301745173 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f13b40588a0 con 0x7f13c40cc8f0 2024-09-15T21:47:39.017 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:39.015+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:39.039 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:39.040 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:39.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:39.040 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:39.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.040 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:39.040 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:47:39.040 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 70.623186335, 2024-09-15T21:47:39.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061533810000000003, 2024-09-15T21:47:39.040 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.040 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.041 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.041 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:47:39.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:47:39.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413085+0000", 2024-09-15T21:47:39.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8390000000000003e-06 2024-09-15T21:47:39.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413092+0000", 2024-09-15T21:47:39.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3479999999999998e-06 2024-09-15T21:47:39.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413109+0000", 2024-09-15T21:47:39.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7192000000000001e-05 2024-09-15T21:47:39.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413118+0000", 2024-09-15T21:47:39.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.935e-06 2024-09-15T21:47:39.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413219+0000", 2024-09-15T21:47:39.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010121 2024-09-15T21:47:39.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413381+0000", 2024-09-15T21:47:39.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016148399999999999 2024-09-15T21:47:39.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:39.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413525+0000", 2024-09-15T21:47:39.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014428100000000001 2024-09-15T21:47:39.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.044 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415866+0000", 2024-09-15T21:47:39.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023409300000000002 2024-09-15T21:47:39.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.044 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419156+0000", 2024-09-15T21:47:39.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032903580000000002 2024-09-15T21:47:39.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419199+0000", 2024-09-15T21:47:39.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2171e-05 2024-09-15T21:47:39.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419232+0000", 2024-09-15T21:47:39.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3633000000000001e-05 2024-09-15T21:47:39.046 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.046 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.046 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.046 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.4:28 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-15T21:47:39.046 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:47:39.046 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 70.621922826000002, 2024-09-15T21:47:39.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002614594, 2024-09-15T21:47:39.046 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.047 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.047 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.047 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:47:39.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:47:39.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414344+0000", 2024-09-15T21:47:39.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6589999999999999e-06 2024-09-15T21:47:39.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414355+0000", 2024-09-15T21:47:39.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0604999999999999e-05 2024-09-15T21:47:39.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414370+0000", 2024-09-15T21:47:39.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-15T21:47:39.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414379+0000", 2024-09-15T21:47:39.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2449999999999998e-06 2024-09-15T21:47:39.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414477+0000", 2024-09-15T21:47:39.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8968000000000005e-05 2024-09-15T21:47:39.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414504+0000", 2024-09-15T21:47:39.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6899000000000001e-05 2024-09-15T21:47:39.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416837+0000", 2024-09-15T21:47:39.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023327130000000001 2024-09-15T21:47:39.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.050 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.050 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416957+0000", 2024-09-15T21:47:39.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000119796 2024-09-15T21:47:39.050 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.050 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.050 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.051 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.1.10:8 2.b e22/19 2:d70a01c6:::60c.00000000:head v 22'4, mlcod=22'1)", 2024-09-15T21:47:39.051 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:47:39.051 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.578343373999999, 2024-09-15T21:47:39.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027633890000000002, 2024-09-15T21:47:39.051 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.051 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.051 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.051 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:47:39.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:47:39.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457923+0000", 2024-09-15T21:47:39.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.136e-06 2024-09-15T21:47:39.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457931+0000", 2024-09-15T21:47:39.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1529999999999997e-06 2024-09-15T21:47:39.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457946+0000", 2024-09-15T21:47:39.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4482e-05 2024-09-15T21:47:39.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457956+0000", 2024-09-15T21:47:39.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0774999999999999e-05 2024-09-15T21:47:39.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458016+0000", 2024-09-15T21:47:39.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9247999999999999e-05 2024-09-15T21:47:39.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458036+0000", 2024-09-15T21:47:39.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0920999999999998e-05 2024-09-15T21:47:39.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460636+0000", 2024-09-15T21:47:39.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025992089999999999 2024-09-15T21:47:39.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460685+0000", 2024-09-15T21:47:39.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9465000000000002e-05 2024-09-15T21:47:39.055 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.055 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.055 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.055 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.18 epoch 22/19 rep_tid 5 entries 22'1 (0'0) error 2:1d00e16c:::201.00000005:head by mds.1.10:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:39.055 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:47:39.055 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 64.576651243000001, 2024-09-15T21:47:39.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003509599, 2024-09-15T21:47:39.056 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.056 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.056 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:39.056 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:47:39.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:47:39.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459615+0000", 2024-09-15T21:47:39.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4200000000000005e-07 2024-09-15T21:47:39.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459626+0000", 2024-09-15T21:47:39.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1456e-05 2024-09-15T21:47:39.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459650+0000", 2024-09-15T21:47:39.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.429e-05 2024-09-15T21:47:39.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459664+0000", 2024-09-15T21:47:39.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3805000000000001e-05 2024-09-15T21:47:39.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459784+0000", 2024-09-15T21:47:39.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000119614 2024-09-15T21:47:39.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463123+0000", 2024-09-15T21:47:39.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033395920000000002 2024-09-15T21:47:39.059 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.059 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.059 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.059 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:7 2.13 e22/19 2:cdf3ac81:::616.00000000:head v 22'5, mlcod=22'3)", 2024-09-15T21:47:39.059 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:47:39.059 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 58.558870464999998, 2024-09-15T21:47:39.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002138615, 2024-09-15T21:47:39.059 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.059 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.060 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.060 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:47:39.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:47:39.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477396+0000", 2024-09-15T21:47:39.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4190000000000001e-06 2024-09-15T21:47:39.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477405+0000", 2024-09-15T21:47:39.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2320000000000007e-06 2024-09-15T21:47:39.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477419+0000", 2024-09-15T21:47:39.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3589e-05 2024-09-15T21:47:39.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477428+0000", 2024-09-15T21:47:39.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4779999999999997e-06 2024-09-15T21:47:39.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477493+0000", 2024-09-15T21:47:39.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4546000000000002e-05 2024-09-15T21:47:39.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477511+0000", 2024-09-15T21:47:39.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7793000000000001e-05 2024-09-15T21:47:39.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479481+0000", 2024-09-15T21:47:39.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019701660000000002 2024-09-15T21:47:39.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479533+0000", 2024-09-15T21:47:39.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2392000000000003e-05 2024-09-15T21:47:39.063 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.064 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.064 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.064 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:26 2.11 e22/19 2:893cd530:::mds2_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-15T21:47:39.064 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:39.064 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 58.558723538999999, 2024-09-15T21:47:39.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020450450000000001, 2024-09-15T21:47:39.064 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.064 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.064 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.064 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:39.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:39.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:39.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9699999999999995e-07 2024-09-15T21:47:39.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477555+0000", 2024-09-15T21:47:39.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3144e-05 2024-09-15T21:47:39.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477565+0000", 2024-09-15T21:47:39.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8579999999999995e-06 2024-09-15T21:47:39.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477573+0000", 2024-09-15T21:47:39.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4869999999999996e-06 2024-09-15T21:47:39.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477632+0000", 2024-09-15T21:47:39.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9178999999999997e-05 2024-09-15T21:47:39.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477654+0000", 2024-09-15T21:47:39.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2234000000000001e-05 2024-09-15T21:47:39.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479526+0000", 2024-09-15T21:47:39.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018717600000000001 2024-09-15T21:47:39.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479587+0000", 2024-09-15T21:47:39.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0686000000000001e-05 2024-09-15T21:47:39.068 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.068 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.068 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.069 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:22 2.1a e22/19 2:5fd68f60:::61d.00000000:head v 22'3, mlcod=0'0)", 2024-09-15T21:47:39.069 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:39.069 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 58.558363483999997, 2024-09-15T21:47:39.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023694380000000002, 2024-09-15T21:47:39.069 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.069 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.069 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.069 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:39.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:39.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:39.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4899999999999995e-07 2024-09-15T21:47:39.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477910+0000", 2024-09-15T21:47:39.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:47:39.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477920+0000", 2024-09-15T21:47:39.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9250000000000007e-06 2024-09-15T21:47:39.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477927+0000", 2024-09-15T21:47:39.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0940000000000002e-06 2024-09-15T21:47:39.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477985+0000", 2024-09-15T21:47:39.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8451e-05 2024-09-15T21:47:39.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478005+0000", 2024-09-15T21:47:39.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0064e-05 2024-09-15T21:47:39.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480223+0000", 2024-09-15T21:47:39.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022177910000000002 2024-09-15T21:47:39.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480271+0000", 2024-09-15T21:47:39.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8164999999999997e-05 2024-09-15T21:47:39.073 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.073 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.073 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.073 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:8 2.13 e22/19 2:cdf3ac81:::616.00000000:head v 22'6, mlcod=22'3)", 2024-09-15T21:47:39.073 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:47:39.073 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 58.558164525000002, 2024-09-15T21:47:39.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021760249999999998, 2024-09-15T21:47:39.073 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.073 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.073 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.074 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:47:39.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:47:39.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478102+0000", 2024-09-15T21:47:39.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3099999999999996e-07 2024-09-15T21:47:39.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478124+0000", 2024-09-15T21:47:39.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2790999999999999e-05 2024-09-15T21:47:39.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478136+0000", 2024-09-15T21:47:39.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1922999999999999e-05 2024-09-15T21:47:39.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478144+0000", 2024-09-15T21:47:39.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9300000000000003e-06 2024-09-15T21:47:39.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478192+0000", 2024-09-15T21:47:39.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8155999999999998e-05 2024-09-15T21:47:39.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478209+0000", 2024-09-15T21:47:39.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6816e-05 2024-09-15T21:47:39.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480243+0000", 2024-09-15T21:47:39.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020336199999999999 2024-09-15T21:47:39.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480277+0000", 2024-09-15T21:47:39.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3858e-05 2024-09-15T21:47:39.077 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.077 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.077 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.077 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.11 epoch 22/19 rep_tid 19 entries 22'2 (0'0) error 2:8975f766:::202.00000001:head by mds.2.12:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:39.078 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:39.078 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 58.558038586999999, 2024-09-15T21:47:39.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023882299999999999, 2024-09-15T21:47:39.078 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.078 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.078 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:39.078 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:39.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:39.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:39.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2399999999999998e-07 2024-09-15T21:47:39.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478236+0000", 2024-09-15T21:47:39.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5299999999999996e-06 2024-09-15T21:47:39.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478248+0000", 2024-09-15T21:47:39.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2806000000000001e-05 2024-09-15T21:47:39.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478264+0000", 2024-09-15T21:47:39.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5468000000000002e-05 2024-09-15T21:47:39.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478339+0000", 2024-09-15T21:47:39.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4851000000000006e-05 2024-09-15T21:47:39.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480615+0000", 2024-09-15T21:47:39.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022760509999999999 2024-09-15T21:47:39.081 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.081 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.081 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.081 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:3 2.1a e22/19 2:5fdc322d:::61e.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:47:39.081 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:47:39.081 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.489853985000003, 2024-09-15T21:47:39.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023961220000000001, 2024-09-15T21:47:39.081 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.081 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.082 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.082 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:47:39.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:47:39.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546413+0000", 2024-09-15T21:47:39.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.812e-06 2024-09-15T21:47:39.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546424+0000", 2024-09-15T21:47:39.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1313e-05 2024-09-15T21:47:39.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546444+0000", 2024-09-15T21:47:39.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9706000000000001e-05 2024-09-15T21:47:39.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546453+0000", 2024-09-15T21:47:39.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:47:39.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546519+0000", 2024-09-15T21:47:39.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5710999999999995e-05 2024-09-15T21:47:39.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546539+0000", 2024-09-15T21:47:39.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0275000000000002e-05 2024-09-15T21:47:39.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548763+0000", 2024-09-15T21:47:39.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002223226 2024-09-15T21:47:39.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548807+0000", 2024-09-15T21:47:39.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4598000000000002e-05 2024-09-15T21:47:39.085 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.085 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.085 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.086 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:19 2.11 e22/19 2:890f50f7:::626.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:47:39.086 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:47:39.086 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.489556825000001, 2024-09-15T21:47:39.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022671280000000002, 2024-09-15T21:47:39.086 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.086 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.086 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.086 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:47:39.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:47:39.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546709+0000", 2024-09-15T21:47:39.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4499999999999997e-07 2024-09-15T21:47:39.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546723+0000", 2024-09-15T21:47:39.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4623999999999999e-05 2024-09-15T21:47:39.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546734+0000", 2024-09-15T21:47:39.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0331e-05 2024-09-15T21:47:39.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546741+0000", 2024-09-15T21:47:39.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.075e-06 2024-09-15T21:47:39.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546954+0000", 2024-09-15T21:47:39.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021275600000000001 2024-09-15T21:47:39.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546971+0000", 2024-09-15T21:47:39.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7503000000000002e-05 2024-09-15T21:47:39.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548926+0000", 2024-09-15T21:47:39.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019550129999999998 2024-09-15T21:47:39.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548975+0000", 2024-09-15T21:47:39.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9280999999999998e-05 2024-09-15T21:47:39.090 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.090 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.090 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.090 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.d epoch 22/19 rep_tid 18 entries 22'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.14:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:39.090 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:47:39.091 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.489550475000001, 2024-09-15T21:47:39.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021738489999999998, 2024-09-15T21:47:39.091 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.091 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.091 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:39.091 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:47:39.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:47:39.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546716+0000", 2024-09-15T21:47:39.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.615e-06 2024-09-15T21:47:39.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546730+0000", 2024-09-15T21:47:39.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3519999999999999e-05 2024-09-15T21:47:39.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546753+0000", 2024-09-15T21:47:39.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3643999999999999e-05 2024-09-15T21:47:39.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546770+0000", 2024-09-15T21:47:39.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6518999999999999e-05 2024-09-15T21:47:39.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546900+0000", 2024-09-15T21:47:39.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013049099999999999 2024-09-15T21:47:39.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548888+0000", 2024-09-15T21:47:39.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019880599999999998 2024-09-15T21:47:39.094 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.094 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.094 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.094 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:4 2.1a e22/19 2:5fdc322d:::61e.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:47:39.094 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:47:39.094 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.489441827, 2024-09-15T21:47:39.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002968769, 2024-09-15T21:47:39.094 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.094 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.094 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.095 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:47:39.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:47:39.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546824+0000", 2024-09-15T21:47:39.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6499999999999999e-07 2024-09-15T21:47:39.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546831+0000", 2024-09-15T21:47:39.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9940000000000003e-06 2024-09-15T21:47:39.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546840+0000", 2024-09-15T21:47:39.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3689999999999996e-06 2024-09-15T21:47:39.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546847+0000", 2024-09-15T21:47:39.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0589999999999997e-06 2024-09-15T21:47:39.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547709+0000", 2024-09-15T21:47:39.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00086144699999999999 2024-09-15T21:47:39.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547730+0000", 2024-09-15T21:47:39.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1542000000000001e-05 2024-09-15T21:47:39.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549750+0000", 2024-09-15T21:47:39.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002020165 2024-09-15T21:47:39.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549792+0000", 2024-09-15T21:47:39.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1628000000000002e-05 2024-09-15T21:47:39.100 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.100 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.100 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.100 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:20 2.11 e22/19 2:890f50f7:::626.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:47:39.100 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:39.100 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.488417218999999, 2024-09-15T21:47:39.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017734579999999999, 2024-09-15T21:47:39.100 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.100 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.100 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.100 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:39.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:39.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:39.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:47:39.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547856+0000", 2024-09-15T21:47:39.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7999999999999999e-06 2024-09-15T21:47:39.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547867+0000", 2024-09-15T21:47:39.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0971999999999999e-05 2024-09-15T21:47:39.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547881+0000", 2024-09-15T21:47:39.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4168999999999999e-05 2024-09-15T21:47:39.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547935+0000", 2024-09-15T21:47:39.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3721999999999998e-05 2024-09-15T21:47:39.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547976+0000", 2024-09-15T21:47:39.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1248999999999998e-05 2024-09-15T21:47:39.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549559+0000", 2024-09-15T21:47:39.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001582815 2024-09-15T21:47:39.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549621+0000", 2024-09-15T21:47:39.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2156999999999995e-05 2024-09-15T21:47:39.104 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.104 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.104 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.104 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:13 2.1a e22/19 2:594e21d0:::623.00000000:head v 22'6, mlcod=22'2)", 2024-09-15T21:47:39.104 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:47:39.104 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.488352223, 2024-09-15T21:47:39.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019085300000000001, 2024-09-15T21:47:39.105 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.105 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.105 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.105 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:47:39.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:47:39.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547914+0000", 2024-09-15T21:47:39.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1500000000000004e-07 2024-09-15T21:47:39.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547920+0000", 2024-09-15T21:47:39.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5239999999999997e-06 2024-09-15T21:47:39.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547930+0000", 2024-09-15T21:47:39.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5109999999999992e-06 2024-09-15T21:47:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547937+0000", 2024-09-15T21:47:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5410000000000003e-06 2024-09-15T21:47:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548518+0000", 2024-09-15T21:47:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00058074299999999999 2024-09-15T21:47:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548538+0000", 2024-09-15T21:47:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9959e-05 2024-09-15T21:47:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549795+0000", 2024-09-15T21:47:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012570120000000001 2024-09-15T21:47:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549821+0000", 2024-09-15T21:47:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6625e-05 2024-09-15T21:47:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.108 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:23 2.18 e22/19 2:1c1db442:::103.00000000.inode:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:47:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.488290804999998, 2024-09-15T21:47:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002713198, 2024-09-15T21:47:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:47:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:47:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547975+0000", 2024-09-15T21:47:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3900000000000004e-07 2024-09-15T21:47:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547983+0000", 2024-09-15T21:47:39.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6690000000000006e-06 2024-09-15T21:47:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547994+0000", 2024-09-15T21:47:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1503e-05 2024-09-15T21:47:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548002+0000", 2024-09-15T21:47:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1340000000000004e-06 2024-09-15T21:47:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548098+0000", 2024-09-15T21:47:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5791000000000005e-05 2024-09-15T21:47:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548118+0000", 2024-09-15T21:47:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0129e-05 2024-09-15T21:47:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550643+0000", 2024-09-15T21:47:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002524728 2024-09-15T21:47:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550688+0000", 2024-09-15T21:47:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4404999999999999e-05 2024-09-15T21:47:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:14 2.1a e22/19 2:594e21d0:::623.00000000:head v 22'7, mlcod=22'2)", 2024-09-15T21:47:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:47:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.488275844999997, 2024-09-15T21:47:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024358579999999999, 2024-09-15T21:47:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.113 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:47:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:47:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547990+0000", 2024-09-15T21:47:39.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5799999999999999e-07 2024-09-15T21:47:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548011+0000", 2024-09-15T21:47:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0942999999999999e-05 2024-09-15T21:47:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548020+0000", 2024-09-15T21:47:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2380000000000003e-06 2024-09-15T21:47:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548027+0000", 2024-09-15T21:47:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6429999999999998e-06 2024-09-15T21:47:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548848+0000", 2024-09-15T21:47:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082167999999999998 2024-09-15T21:47:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548868+0000", 2024-09-15T21:47:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9791000000000001e-05 2024-09-15T21:47:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550387+0000", 2024-09-15T21:47:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001519322 2024-09-15T21:47:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550425+0000", 2024-09-15T21:47:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7682999999999998e-05 2024-09-15T21:47:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:35 2.f 2:f426ad4f:::mds2_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:39.117 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:47:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.298164159999999, 2024-09-15T21:47:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017328949999999999, 2024-09-15T21:47:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:47:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:47:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738103+0000", 2024-09-15T21:47:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0020000000000001e-06 2024-09-15T21:47:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738112+0000", 2024-09-15T21:47:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2220000000000008e-06 2024-09-15T21:47:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.119 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738127+0000", 2024-09-15T21:47:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5016e-05 2024-09-15T21:47:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738137+0000", 2024-09-15T21:47:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0301e-05 2024-09-15T21:47:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738216+0000", 2024-09-15T21:47:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8356000000000007e-05 2024-09-15T21:47:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738340+0000", 2024-09-15T21:47:39.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012372800000000001 2024-09-15T21:47:39.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:39.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738453+0000", 2024-09-15T21:47:39.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000113785 2024-09-15T21:47:39.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739115+0000", 2024-09-15T21:47:39.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00066178500000000002 2024-09-15T21:47:39.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739745+0000", 2024-09-15T21:47:39.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00063017900000000003 2024-09-15T21:47:39.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739783+0000", 2024-09-15T21:47:39.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7941000000000001e-05 2024-09-15T21:47:39.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739834+0000", 2024-09-15T21:47:39.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.058e-05 2024-09-15T21:47:39.122 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.123 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.123 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.123 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.4.16:13 2.13 e22/19 2:c81eaac8:::62d.00000000:head v 22'8, mlcod=22'6)", 2024-09-15T21:47:39.123 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:47:39.123 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 47.470759100999999, 2024-09-15T21:47:39.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002151778, 2024-09-15T21:47:39.123 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.123 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.123 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.123 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:47:39.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:47:39.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565507+0000", 2024-09-15T21:47:39.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3340000000000001e-06 2024-09-15T21:47:39.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565518+0000", 2024-09-15T21:47:39.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0339999999999999e-05 2024-09-15T21:47:39.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565536+0000", 2024-09-15T21:47:39.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8647e-05 2024-09-15T21:47:39.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565545+0000", 2024-09-15T21:47:39.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1430000000000004e-06 2024-09-15T21:47:39.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565611+0000", 2024-09-15T21:47:39.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5129000000000006e-05 2024-09-15T21:47:39.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565634+0000", 2024-09-15T21:47:39.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3173e-05 2024-09-15T21:47:39.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567561+0000", 2024-09-15T21:47:39.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019268180000000001 2024-09-15T21:47:39.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567658+0000", 2024-09-15T21:47:39.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7194000000000005e-05 2024-09-15T21:47:39.127 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.127 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.127 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.127 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.d epoch 22/19 rep_tid 24 entries 22'3 (0'0) error 2:b2057508:::204.00000002:head by mds.4.16:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 22'1)", 2024-09-15T21:47:39.127 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:47:39.127 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 47.470598238999997, 2024-09-15T21:47:39.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026915049999999999, 2024-09-15T21:47:39.128 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.128 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.128 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:39.128 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:47:39.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:47:39.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565668+0000", 2024-09-15T21:47:39.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.113e-06 2024-09-15T21:47:39.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565684+0000", 2024-09-15T21:47:39.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5943999999999998e-05 2024-09-15T21:47:39.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565705+0000", 2024-09-15T21:47:39.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0761999999999999e-05 2024-09-15T21:47:39.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565724+0000", 2024-09-15T21:47:39.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9208e-05 2024-09-15T21:47:39.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565817+0000", 2024-09-15T21:47:39.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3218000000000001e-05 2024-09-15T21:47:39.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568358+0000", 2024-09-15T21:47:39.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00254126 2024-09-15T21:47:39.131 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.131 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.131 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.131 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.131 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.131 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:39.131 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:47:39.132 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:39.131+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:39.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:39.166+0000 7f13cb531640 1 -- 172.21.15.100:0/1301745173 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f13c4154c60 con 0x7f13c40cc8f0 2024-09-15T21:47:39.234 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:39.232+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:39.235 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:39.235 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:39.235 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:39.235 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:39.245 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:47:39.334 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:39.333+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:39.348 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:39.348 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:39.348 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:39.348 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:39.348 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:39.357 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:47:39.440 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:39.439+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:39.502 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:39.502 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:39.502 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:39.502 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:39.502 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.502 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:39.503 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:39.503 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.086026532000005, 2024-09-15T21:47:39.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0050344960000000003, 2024-09-15T21:47:39.503 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.503 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.503 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.503 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.503 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.503 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:39.503 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.503 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.503 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:39.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:39.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0199999999999998e-07 2024-09-15T21:47:39.504 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.504 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.504 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.504 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413470+0000", 2024-09-15T21:47:39.504 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4329999999999998e-06 2024-09-15T21:47:39.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413579+0000", 2024-09-15T21:47:39.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108593 2024-09-15T21:47:39.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413587+0000", 2024-09-15T21:47:39.505 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3100000000000001e-06 2024-09-15T21:47:39.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.505 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.505 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.505 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413691+0000", 2024-09-15T21:47:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010357 2024-09-15T21:47:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413887+0000", 2024-09-15T21:47:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019666799999999999 2024-09-15T21:47:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414027+0000", 2024-09-15T21:47:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013996700000000001 2024-09-15T21:47:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416360+0000", 2024-09-15T21:47:39.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002332204 2024-09-15T21:47:39.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418437+0000", 2024-09-15T21:47:39.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002077279 2024-09-15T21:47:39.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.507 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.507 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418469+0000", 2024-09-15T21:47:39.507 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2359e-05 2024-09-15T21:47:39.507 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.507 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418502+0000", 2024-09-15T21:47:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2811000000000001e-05 2024-09-15T21:47:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.086022491999998, 2024-09-15T21:47:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0053538780000000003, 2024-09-15T21:47:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2000000000000001e-07 2024-09-15T21:47:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413474+0000", 2024-09-15T21:47:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-15T21:47:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413612+0000", 2024-09-15T21:47:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000137727 2024-09-15T21:47:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413618+0000", 2024-09-15T21:47:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.241e-06 2024-09-15T21:47:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414443+0000", 2024-09-15T21:47:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082447 2024-09-15T21:47:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414523+0000", 2024-09-15T21:47:39.511 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0059e-05 2024-09-15T21:47:39.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:39.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414653+0000", 2024-09-15T21:47:39.512 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013031699999999999 2024-09-15T21:47:39.512 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.512 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.512 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.512 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416366+0000", 2024-09-15T21:47:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001712917 2024-09-15T21:47:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418769+0000", 2024-09-15T21:47:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024031529999999999 2024-09-15T21:47:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418802+0000", 2024-09-15T21:47:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2925000000000001e-05 2024-09-15T21:47:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.513 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418825+0000", 2024-09-15T21:47:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3204e-05 2024-09-15T21:47:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.086014470999999, 2024-09-15T21:47:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059708640000000002, 2024-09-15T21:47:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.514 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.515 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:39.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7300000000000002e-07 2024-09-15T21:47:39.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413482+0000", 2024-09-15T21:47:39.516 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2359999999999999e-06 2024-09-15T21:47:39.516 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.516 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.516 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.516 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414323+0000", 2024-09-15T21:47:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00084095399999999999 2024-09-15T21:47:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414332+0000", 2024-09-15T21:47:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0880000000000001e-06 2024-09-15T21:47:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415439+0000", 2024-09-15T21:47:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011071869999999999 2024-09-15T21:47:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415554+0000", 2024-09-15T21:47:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114499 2024-09-15T21:47:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415682+0000", 2024-09-15T21:47:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012802500000000001 2024-09-15T21:47:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.417957+0000", 2024-09-15T21:47:39.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022746720000000002 2024-09-15T21:47:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419385+0000", 2024-09-15T21:47:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001428826 2024-09-15T21:47:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419420+0000", 2024-09-15T21:47:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4875000000000001e-05 2024-09-15T21:47:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419450+0000", 2024-09-15T21:47:39.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0128999999999999e-05 2024-09-15T21:47:39.520 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.520 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.520 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.520 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:39.520 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:39.520 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.086010653000002, 2024-09-15T21:47:39.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0087186220000000005, 2024-09-15T21:47:39.520 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.520 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413484+0000", 2024-09-15T21:47:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-15T21:47:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413487+0000", 2024-09-15T21:47:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.534e-06 2024-09-15T21:47:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414353+0000", 2024-09-15T21:47:39.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00086616999999999998 2024-09-15T21:47:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414359+0000", 2024-09-15T21:47:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3279999999999996e-06 2024-09-15T21:47:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416041+0000", 2024-09-15T21:47:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001682467 2024-09-15T21:47:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416158+0000", 2024-09-15T21:47:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117242 2024-09-15T21:47:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416284+0000", 2024-09-15T21:47:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012509000000000001 2024-09-15T21:47:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.417992+0000", 2024-09-15T21:47:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017087059999999999 2024-09-15T21:47:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422142+0000", 2024-09-15T21:47:39.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041501180000000004 2024-09-15T21:47:39.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422176+0000", 2024-09-15T21:47:39.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4109999999999997e-05 2024-09-15T21:47:39.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422202+0000", 2024-09-15T21:47:39.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5585e-05 2024-09-15T21:47:39.526 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.526 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.526 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.526 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:39.526 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:39.526 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.086005228000005, 2024-09-15T21:47:39.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0088000370000000001, 2024-09-15T21:47:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:39.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413491+0000", 2024-09-15T21:47:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.773e-06 2024-09-15T21:47:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414372+0000", 2024-09-15T21:47:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00088122000000000005 2024-09-15T21:47:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414378+0000", 2024-09-15T21:47:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4380000000000003e-06 2024-09-15T21:47:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416631+0000", 2024-09-15T21:47:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022530319999999999 2024-09-15T21:47:39.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416765+0000", 2024-09-15T21:47:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013423499999999999 2024-09-15T21:47:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422289+0000", 2024-09-15T21:47:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055240530000000001 2024-09-15T21:47:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 65.044436908999998, 2024-09-15T21:47:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056900609999999997, 2024-09-15T21:47:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455058+0000", 2024-09-15T21:47:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.51e-07 2024-09-15T21:47:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455061+0000", 2024-09-15T21:47:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.145e-06 2024-09-15T21:47:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455158+0000", 2024-09-15T21:47:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7101000000000004e-05 2024-09-15T21:47:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455165+0000", 2024-09-15T21:47:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1019999999999999e-06 2024-09-15T21:47:39.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456433+0000", 2024-09-15T21:47:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012678259999999999 2024-09-15T21:47:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456517+0000", 2024-09-15T21:47:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4373e-05 2024-09-15T21:47:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:39.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456677+0000", 2024-09-15T21:47:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015963300000000001 2024-09-15T21:47:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458636+0000", 2024-09-15T21:47:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019594030000000002 2024-09-15T21:47:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460665+0000", 2024-09-15T21:47:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020283549999999999 2024-09-15T21:47:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.535 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460707+0000", 2024-09-15T21:47:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2398000000000003e-05 2024-09-15T21:47:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460747+0000", 2024-09-15T21:47:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0274000000000001e-05 2024-09-15T21:47:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:39.536 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 65.044429715000007, 2024-09-15T21:47:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086420030000000005, 2024-09-15T21:47:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:39.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455065+0000", 2024-09-15T21:47:39.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3700000000000001e-07 2024-09-15T21:47:39.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455068+0000", 2024-09-15T21:47:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.915e-06 2024-09-15T21:47:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455366+0000", 2024-09-15T21:47:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00029825400000000002 2024-09-15T21:47:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455374+0000", 2024-09-15T21:47:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6240000000000008e-06 2024-09-15T21:47:39.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456199+0000", 2024-09-15T21:47:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082448500000000002 2024-09-15T21:47:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457034+0000", 2024-09-15T21:47:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00083553399999999997 2024-09-15T21:47:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457171+0000", 2024-09-15T21:47:39.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013653600000000001 2024-09-15T21:47:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462663+0000", 2024-09-15T21:47:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054924420000000002 2024-09-15T21:47:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463613+0000", 2024-09-15T21:47:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094971499999999996 2024-09-15T21:47:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463663+0000", 2024-09-15T21:47:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9636999999999999e-05 2024-09-15T21:47:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463706+0000", 2024-09-15T21:47:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3523999999999999e-05 2024-09-15T21:47:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:39.542 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 65.044425187000002, 2024-09-15T21:47:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090408420000000003, 2024-09-15T21:47:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:39.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.79e-07 2024-09-15T21:47:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455072+0000", 2024-09-15T21:47:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6189999999999998e-06 2024-09-15T21:47:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.544 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455403+0000", 2024-09-15T21:47:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033167299999999999 2024-09-15T21:47:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455420+0000", 2024-09-15T21:47:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6518999999999999e-05 2024-09-15T21:47:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457448+0000", 2024-09-15T21:47:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020275710000000001 2024-09-15T21:47:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.545 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3844999999999994e-05 2024-09-15T21:47:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457684+0000", 2024-09-15T21:47:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015239 2024-09-15T21:47:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462678+0000", 2024-09-15T21:47:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049939279999999999 2024-09-15T21:47:39.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464017+0000", 2024-09-15T21:47:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013397140000000001 2024-09-15T21:47:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464071+0000", 2024-09-15T21:47:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3735e-05 2024-09-15T21:47:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464110+0000", 2024-09-15T21:47:39.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8569000000000003e-05 2024-09-15T21:47:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 65.044413574000004, 2024-09-15T21:47:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091802889999999995, 2024-09-15T21:47:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.548 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455081+0000", 2024-09-15T21:47:39.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455084+0000", 2024-09-15T21:47:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7810000000000001e-06 2024-09-15T21:47:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455536+0000", 2024-09-15T21:47:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00045219100000000001 2024-09-15T21:47:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455583+0000", 2024-09-15T21:47:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7308999999999997e-05 2024-09-15T21:47:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457998+0000", 2024-09-15T21:47:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024154290000000002 2024-09-15T21:47:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458147+0000", 2024-09-15T21:47:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014834599999999999 2024-09-15T21:47:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458265+0000", 2024-09-15T21:47:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118458 2024-09-15T21:47:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464134+0000", 2024-09-15T21:47:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005869038 2024-09-15T21:47:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464187+0000", 2024-09-15T21:47:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2343999999999999e-05 2024-09-15T21:47:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464222+0000", 2024-09-15T21:47:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5447000000000001e-05 2024-09-15T21:47:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464261+0000", 2024-09-15T21:47:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8659999999999999e-05 2024-09-15T21:47:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.553 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 59.023791195000001, 2024-09-15T21:47:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048468290000000004, 2024-09-15T21:47:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.35e-07 2024-09-15T21:47:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475706+0000", 2024-09-15T21:47:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.994e-06 2024-09-15T21:47:39.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475770+0000", 2024-09-15T21:47:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3904000000000003e-05 2024-09-15T21:47:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475775+0000", 2024-09-15T21:47:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2490000000000001e-06 2024-09-15T21:47:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476502+0000", 2024-09-15T21:47:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072697300000000001 2024-09-15T21:47:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476611+0000", 2024-09-15T21:47:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0001091 2024-09-15T21:47:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476807+0000", 2024-09-15T21:47:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000195892 2024-09-15T21:47:39.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479678+0000", 2024-09-15T21:47:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028704709999999999 2024-09-15T21:47:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480444+0000", 2024-09-15T21:47:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00076602099999999998 2024-09-15T21:47:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480502+0000", 2024-09-15T21:47:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8313999999999998e-05 2024-09-15T21:47:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480550+0000", 2024-09-15T21:47:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7676000000000002e-05 2024-09-15T21:47:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:39.559 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:39.560 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 59.023784020000001, 2024-09-15T21:47:39.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051250929999999998, 2024-09-15T21:47:39.560 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.560 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.560 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:39.560 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:39.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1e-07 2024-09-15T21:47:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475723+0000", 2024-09-15T21:47:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2758000000000001e-05 2024-09-15T21:47:39.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475972+0000", 2024-09-15T21:47:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024886299999999999 2024-09-15T21:47:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475979+0000", 2024-09-15T21:47:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0230000000000004e-06 2024-09-15T21:47:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476560+0000", 2024-09-15T21:47:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00058097599999999998 2024-09-15T21:47:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476689+0000", 2024-09-15T21:47:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129486 2024-09-15T21:47:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480835+0000", 2024-09-15T21:47:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041457769999999998 2024-09-15T21:47:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.563 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.954868795000003, 2024-09-15T21:47:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054420570000000001, 2024-09-15T21:47:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544626+0000", 2024-09-15T21:47:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8100000000000003e-07 2024-09-15T21:47:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544629+0000", 2024-09-15T21:47:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9869999999999999e-06 2024-09-15T21:47:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544775+0000", 2024-09-15T21:47:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014610599999999999 2024-09-15T21:47:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544782+0000", 2024-09-15T21:47:39.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4109999999999999e-06 2024-09-15T21:47:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545617+0000", 2024-09-15T21:47:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00083468300000000002 2024-09-15T21:47:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545690+0000", 2024-09-15T21:47:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3545000000000001e-05 2024-09-15T21:47:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545803+0000", 2024-09-15T21:47:39.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000112083 2024-09-15T21:47:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548353+0000", 2024-09-15T21:47:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025507220000000001 2024-09-15T21:47:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549964+0000", 2024-09-15T21:47:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001610382 2024-09-15T21:47:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550021+0000", 2024-09-15T21:47:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6898999999999999e-05 2024-09-15T21:47:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550067+0000", 2024-09-15T21:47:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758000000000002e-05 2024-09-15T21:47:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:39.569 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.954863517, 2024-09-15T21:47:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056323399999999996, 2024-09-15T21:47:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:39.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:47:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544634+0000", 2024-09-15T21:47:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.58e-06 2024-09-15T21:47:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544800+0000", 2024-09-15T21:47:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016576899999999999 2024-09-15T21:47:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544808+0000", 2024-09-15T21:47:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5179999999999998e-06 2024-09-15T21:47:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546075+0000", 2024-09-15T21:47:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012676969999999999 2024-09-15T21:47:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546179+0000", 2024-09-15T21:47:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000103935 2024-09-15T21:47:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546286+0000", 2024-09-15T21:47:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106758 2024-09-15T21:47:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548528+0000", 2024-09-15T21:47:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022420510000000001 2024-09-15T21:47:39.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550119+0000", 2024-09-15T21:47:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015907079999999999 2024-09-15T21:47:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550202+0000", 2024-09-15T21:47:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3374000000000005e-05 2024-09-15T21:47:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550263+0000", 2024-09-15T21:47:39.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0643999999999997e-05 2024-09-15T21:47:39.575 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.575 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.575 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.575 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:39.575 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:39.575 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.954858545999997, 2024-09-15T21:47:39.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061905470000000002, 2024-09-15T21:47:39.575 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.575 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6899999999999999e-07 2024-09-15T21:47:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544638+0000", 2024-09-15T21:47:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3920000000000001e-06 2024-09-15T21:47:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544819+0000", 2024-09-15T21:47:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018065099999999999 2024-09-15T21:47:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544825+0000", 2024-09-15T21:47:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7239999999999997e-06 2024-09-15T21:47:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546583+0000", 2024-09-15T21:47:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017588219999999999 2024-09-15T21:47:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546665+0000", 2024-09-15T21:47:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1693000000000005e-05 2024-09-15T21:47:39.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546808+0000", 2024-09-15T21:47:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142993 2024-09-15T21:47:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548533+0000", 2024-09-15T21:47:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017246 2024-09-15T21:47:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550606+0000", 2024-09-15T21:47:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020733969999999998 2024-09-15T21:47:39.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550783+0000", 2024-09-15T21:47:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000176565 2024-09-15T21:47:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550826+0000", 2024-09-15T21:47:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3441e-05 2024-09-15T21:47:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.954850825000001, 2024-09-15T21:47:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052813010000000004, 2024-09-15T21:47:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3799999999999999e-07 2024-09-15T21:47:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544647+0000", 2024-09-15T21:47:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6569999999999999e-06 2024-09-15T21:47:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544858+0000", 2024-09-15T21:47:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021069900000000001 2024-09-15T21:47:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544862+0000", 2024-09-15T21:47:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4900000000000002e-06 2024-09-15T21:47:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545610+0000", 2024-09-15T21:47:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00074786999999999998 2024-09-15T21:47:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545706+0000", 2024-09-15T21:47:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6271e-05 2024-09-15T21:47:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545834+0000", 2024-09-15T21:47:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012759499999999999 2024-09-15T21:47:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548554+0000", 2024-09-15T21:47:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027199339999999998 2024-09-15T21:47:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549818+0000", 2024-09-15T21:47:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012642199999999999 2024-09-15T21:47:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549884+0000", 2024-09-15T21:47:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5843e-05 2024-09-15T21:47:39.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549925+0000", 2024-09-15T21:47:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0484000000000001e-05 2024-09-15T21:47:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 53.954842147999997, 2024-09-15T21:47:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051106110000000001, 2024-09-15T21:47:39.586 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2700000000000001e-07 2024-09-15T21:47:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544667+0000", 2024-09-15T21:47:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4746000000000001e-05 2024-09-15T21:47:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545021+0000", 2024-09-15T21:47:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035412900000000001 2024-09-15T21:47:39.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545052+0000", 2024-09-15T21:47:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1158000000000002e-05 2024-09-15T21:47:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546050+0000", 2024-09-15T21:47:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00099782800000000008 2024-09-15T21:47:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546291+0000", 2024-09-15T21:47:39.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024093500000000001 2024-09-15T21:47:39.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:39.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546434+0000", 2024-09-15T21:47:39.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014308600000000001 2024-09-15T21:47:39.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548583+0000", 2024-09-15T21:47:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021491029999999999 2024-09-15T21:47:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549695+0000", 2024-09-15T21:47:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011116419999999999 2024-09-15T21:47:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549726+0000", 2024-09-15T21:47:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1195000000000002e-05 2024-09-15T21:47:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549763+0000", 2024-09-15T21:47:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6562e-05 2024-09-15T21:47:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:17 2.14 2:2eebc8f4:::62f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 47.934412535999996, 2024-09-15T21:47:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071004800000000002, 2024-09-15T21:47:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.592 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565083+0000", 2024-09-15T21:47:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.689e-06 2024-09-15T21:47:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565090+0000", 2024-09-15T21:47:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5930000000000002e-06 2024-09-15T21:47:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565119+0000", 2024-09-15T21:47:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8785999999999998e-05 2024-09-15T21:47:39.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565126+0000", 2024-09-15T21:47:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0110000000000003e-06 2024-09-15T21:47:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565237+0000", 2024-09-15T21:47:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011161 2024-09-15T21:47:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565383+0000", 2024-09-15T21:47:39.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014626900000000001 2024-09-15T21:47:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565520+0000", 2024-09-15T21:47:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013676399999999999 2024-09-15T21:47:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569909+0000", 2024-09-15T21:47:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043890149999999996 2024-09-15T21:47:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572096+0000", 2024-09-15T21:47:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021866559999999999 2024-09-15T21:47:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572144+0000", 2024-09-15T21:47:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7725000000000001e-05 2024-09-15T21:47:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572182+0000", 2024-09-15T21:47:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8362000000000003e-05 2024-09-15T21:47:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.597 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 47.934401438000002, 2024-09-15T21:47:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072110280000000004, 2024-09-15T21:47:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:39.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4299999999999999e-07 2024-09-15T21:47:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565096+0000", 2024-09-15T21:47:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4319999999999999e-06 2024-09-15T21:47:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565152+0000", 2024-09-15T21:47:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5686999999999997e-05 2024-09-15T21:47:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565157+0000", 2024-09-15T21:47:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2700000000000004e-06 2024-09-15T21:47:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565847+0000", 2024-09-15T21:47:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068937299999999996 2024-09-15T21:47:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565940+0000", 2024-09-15T21:47:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3573999999999996e-05 2024-09-15T21:47:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566085+0000", 2024-09-15T21:47:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000144902 2024-09-15T21:47:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570595+0000", 2024-09-15T21:47:39.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045103469999999996 2024-09-15T21:47:39.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572224+0000", 2024-09-15T21:47:39.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016282930000000001 2024-09-15T21:47:39.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572260+0000", 2024-09-15T21:47:39.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6103999999999999e-05 2024-09-15T21:47:39.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572304+0000", 2024-09-15T21:47:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3803000000000001e-05 2024-09-15T21:47:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 47.934396972999998, 2024-09-15T21:47:39.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055995389999999997, 2024-09-15T21:47:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9999999999999999e-07 2024-09-15T21:47:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2399999999999999e-06 2024-09-15T21:47:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565280+0000", 2024-09-15T21:47:39.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017929400000000001 2024-09-15T21:47:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565292+0000", 2024-09-15T21:47:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2193e-05 2024-09-15T21:47:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565357+0000", 2024-09-15T21:47:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4684000000000006e-05 2024-09-15T21:47:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565499+0000", 2024-09-15T21:47:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142032 2024-09-15T21:47:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565634+0000", 2024-09-15T21:47:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000135613 2024-09-15T21:47:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569941+0000", 2024-09-15T21:47:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043066789999999999 2024-09-15T21:47:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570585+0000", 2024-09-15T21:47:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00064364700000000001 2024-09-15T21:47:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570647+0000", 2024-09-15T21:47:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2173000000000003e-05 2024-09-15T21:47:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570697+0000", 2024-09-15T21:47:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9784000000000003e-05 2024-09-15T21:47:39.608 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 47.934392742999997, 2024-09-15T21:47:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066548939999999997, 2024-09-15T21:47:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:39.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565102+0000", 2024-09-15T21:47:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-15T21:47:39.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565103+0000", 2024-09-15T21:47:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6700000000000001e-06 2024-09-15T21:47:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565608+0000", 2024-09-15T21:47:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050518299999999998 2024-09-15T21:47:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565616+0000", 2024-09-15T21:47:39.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8199999999999997e-06 2024-09-15T21:47:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566598+0000", 2024-09-15T21:47:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00098176300000000008 2024-09-15T21:47:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566871+0000", 2024-09-15T21:47:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027312799999999998 2024-09-15T21:47:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567015+0000", 2024-09-15T21:47:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014428499999999999 2024-09-15T21:47:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570729+0000", 2024-09-15T21:47:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037132459999999999 2024-09-15T21:47:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571667+0000", 2024-09-15T21:47:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00093862299999999997 2024-09-15T21:47:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571718+0000", 2024-09-15T21:47:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1146999999999997e-05 2024-09-15T21:47:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571756+0000", 2024-09-15T21:47:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7821000000000003e-05 2024-09-15T21:47:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:39.614 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:39.614 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:39.616 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:47:39.617 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:39.614+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:39.716 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:39.715+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:39.761 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:39.761 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:39.761 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:39.761 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:39.769 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:47:39.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:39.796+0000 7f13b9ffb640 1 -- 172.21.15.100:0/1301745173 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v39) ==== 87+0+0 (secure 0 0 0) 0x7f13b405cba0 con 0x7f13c40cc8f0 2024-09-15T21:47:39.798 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is set 2024-09-15T21:47:39.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:39.799+0000 7f13cb531640 1 -- 172.21.15.100:0/1301745173 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f13a806c2b0 msgr2=0x7f13a806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:39.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:39.799+0000 7f13cb531640 1 --2- 172.21.15.100:0/1301745173 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f13a806c2b0 0x7f13a806e770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f13c0001110 tx=0x7f13c0001080 comp rx=0 tx=0).stop 2024-09-15T21:47:39.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:39.799+0000 7f13cb531640 1 -- 172.21.15.100:0/1301745173 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13c40cc8f0 msgr2=0x7f13c41f1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:39.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:39.799+0000 7f13cb531640 1 --2- 172.21.15.100:0/1301745173 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13c40cc8f0 0x7f13c41f1510 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f13b400c3c0 tx=0x7f13b4001e70 comp rx=0 tx=0).stop 2024-09-15T21:47:39.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:39.799+0000 7f13cb531640 1 -- 172.21.15.100:0/1301745173 shutdown_connections 2024-09-15T21:47:39.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:39.799+0000 7f13cb531640 1 --2- 172.21.15.100:0/1301745173 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f13a806c2b0 0x7f13a806e770 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:39.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:39.799+0000 7f13cb531640 1 --2- 172.21.15.100:0/1301745173 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13c4155440 0x7f13c415c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:39.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:39.799+0000 7f13cb531640 1 --2- 172.21.15.100:0/1301745173 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f13c40cc8f0 0x7f13c41f1510 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:39.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:39.799+0000 7f13cb531640 1 --2- 172.21.15.100:0/1301745173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13c40cb7d0 0x7f13c4154720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:39.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:39.799+0000 7f13cb531640 1 -- 172.21.15.100:0/1301745173 >> 172.21.15.100:0/1301745173 conn(0x7f13c40680d0 msgr2=0x7f13c40c5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:39.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:39.799+0000 7f13cb531640 1 -- 172.21.15.100:0/1301745173 shutdown_connections 2024-09-15T21:47:39.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:39.799+0000 7f13cb531640 1 -- 172.21.15.100:0/1301745173 wait complete. 2024-09-15T21:47:39.817 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:39.815+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:39.822 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:47:39.874 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:39.874 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:39.874 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:39.874 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:39.874 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:39.883 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:47:39.919 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:39.917+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:40.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.001+0000 7fc41fe5e640 1 Processor -- start 2024-09-15T21:47:40.003 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.002+0000 7fc41fe5e640 1 -- start start 2024-09-15T21:47:40.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.003+0000 7fc41fe5e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc418155290 0x7fc41815e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:40.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.003+0000 7fc41fe5e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc41815bb60 0x7fc41806d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:40.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.003+0000 7fc41fe5e640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc41815edb0 con 0x7fc418155670 2024-09-15T21:47:40.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.003+0000 7fc41fe5e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc41815f090 con 0x7fc41815bb60 2024-09-15T21:47:40.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.003+0000 7fc41fe5e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc41815f370 con 0x7fc418155290 2024-09-15T21:47:40.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.003+0000 7fc41d3f9640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fc418155670 0x7fc4180c65b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47920/0 (socket says 172.21.15.100:47920) 2024-09-15T21:47:40.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.003+0000 7fc41d3f9640 1 -- 172.21.15.100:0/3669786756 learned_addr learned my addr 172.21.15.100:0/3669786756 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:40.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.003+0000 7fc41dbfa640 1 --2- 172.21.15.100:0/3669786756 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc418155290 0x7fc41815e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:40.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.003+0000 7fc41e3fb640 1 --2- 172.21.15.100:0/3669786756 >> v2:172.21.15.100:3300/0 conn(0x7fc41815bb60 0x7fc41806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:40.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.003+0000 7fc41cbf8640 1 -- 172.21.15.100:0/3669786756 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3905232959 0 0) 0x7fc41815edb0 con 0x7fc418155670 2024-09-15T21:47:40.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.003+0000 7fc41cbf8640 1 -- 172.21.15.100:0/3669786756 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc404003520 con 0x7fc418155670 2024-09-15T21:47:40.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.004+0000 7fc41dbfa640 1 -- 172.21.15.100:0/3669786756 >> v2:172.21.15.100:3300/0 conn(0x7fc41815bb60 msgr2=0x7fc41806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:40.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.004+0000 7fc41dbfa640 1 --2- 172.21.15.100:0/3669786756 >> v2:172.21.15.100:3300/0 conn(0x7fc41815bb60 0x7fc41806d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:40.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.004+0000 7fc41dbfa640 1 -- 172.21.15.100:0/3669786756 >> v1:172.21.15.100:6789/0 conn(0x7fc418155670 legacy=0x7fc4180c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:40.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.004+0000 7fc41cbf8640 1 -- 172.21.15.100:0/3669786756 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1355936987 0 0) 0x7fc404003520 con 0x7fc418155670 2024-09-15T21:47:40.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.004+0000 7fc41dbfa640 1 -- 172.21.15.100:0/3669786756 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4180c5db0 con 0x7fc418155290 2024-09-15T21:47:40.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.004+0000 7fc41e3fb640 1 --2- 172.21.15.100:0/3669786756 >> v2:172.21.15.100:3300/0 conn(0x7fc41815bb60 0x7fc41806d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:40.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.005+0000 7fc41dbfa640 1 --2- 172.21.15.100:0/3669786756 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc418155290 0x7fc41815e690 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc4140097a0 tx=0x7fc41402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d34690e5ae8ebc0b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:40.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.005+0000 7fc41cbf8640 1 -- 172.21.15.100:0/3669786756 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414037070 con 0x7fc418155290 2024-09-15T21:47:40.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.005+0000 7fc41cbf8640 1 -- 172.21.15.100:0/3669786756 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc41402aba0 con 0x7fc418155290 2024-09-15T21:47:40.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.005+0000 7fc41cbf8640 1 -- 172.21.15.100:0/3669786756 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408001670 con 0x7fc418155290 2024-09-15T21:47:40.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.006+0000 7fc41fe5e640 1 -- 172.21.15.100:0/3669786756 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc418155290 msgr2=0x7fc41815e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:40.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.006+0000 7fc41fe5e640 1 --2- 172.21.15.100:0/3669786756 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc418155290 0x7fc41815e690 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc4140097a0 tx=0x7fc41402a110 comp rx=0 tx=0).stop 2024-09-15T21:47:40.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.006+0000 7fc41fe5e640 1 -- 172.21.15.100:0/3669786756 shutdown_connections 2024-09-15T21:47:40.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.006+0000 7fc41fe5e640 1 --2- 172.21.15.100:0/3669786756 >> v2:172.21.15.100:3300/0 conn(0x7fc41815bb60 0x7fc41806d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:40.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.006+0000 7fc41fe5e640 1 --2- 172.21.15.100:0/3669786756 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc418155290 0x7fc41815e690 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:40.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.006+0000 7fc41fe5e640 1 -- 172.21.15.100:0/3669786756 >> 172.21.15.100:0/3669786756 conn(0x7fc418073310 msgr2=0x7fc418073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:40.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.006+0000 7fc41fe5e640 1 -- 172.21.15.100:0/3669786756 shutdown_connections 2024-09-15T21:47:40.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.006+0000 7fc41fe5e640 1 -- 172.21.15.100:0/3669786756 wait complete. 2024-09-15T21:47:40.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.008+0000 7fc41fe5e640 1 Processor -- start 2024-09-15T21:47:40.010 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.008+0000 7fc41fe5e640 1 -- start start 2024-09-15T21:47:40.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.009+0000 7fc41fe5e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc418155290 0x7fc41814a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:40.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.009+0000 7fc41fe5e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc418155670 0x7fc4181f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:40.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.009+0000 7fc41fe5e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc41815bb60 0x7fc418148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:40.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.009+0000 7fc41dbfa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc418155290 0x7fc41814a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:40.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.009+0000 7fc41dbfa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc418155290 0x7fc41814a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33712/0 (socket says 172.21.15.100:33712) 2024-09-15T21:47:40.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.009+0000 7fc41dbfa640 1 -- 172.21.15.100:0/2334705401 learned_addr learned my addr 172.21.15.100:0/2334705401 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:40.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.009+0000 7fc41d3f9640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc418155670 0x7fc4181f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:40.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.009+0000 7fc41e3fb640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc41815bb60 0x7fc418148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:40.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.010+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc41815bf90 con 0x7fc418155290 2024-09-15T21:47:40.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.010+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4180b39e0 con 0x7fc41815bb60 2024-09-15T21:47:40.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.010+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc41815c590 con 0x7fc418155670 2024-09-15T21:47:40.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.010+0000 7fc41dbfa640 1 -- 172.21.15.100:0/2334705401 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc418155670 msgr2=0x7fc4181f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:40.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.010+0000 7fc41dbfa640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc418155670 0x7fc4181f43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:40.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.010+0000 7fc41dbfa640 1 -- 172.21.15.100:0/2334705401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc41815bb60 msgr2=0x7fc418148400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:40.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.010+0000 7fc41dbfa640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc41815bb60 0x7fc418148400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:40.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.010+0000 7fc41dbfa640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4180cf840 con 0x7fc418155290 2024-09-15T21:47:40.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.010+0000 7fc41dbfa640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc418155290 0x7fc41814a900 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc414035040 tx=0x7fc4140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:40.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.011+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414037050 con 0x7fc418155290 2024-09-15T21:47:40.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.011+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc414043070 con 0x7fc418155290 2024-09-15T21:47:40.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.011+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414006e10 con 0x7fc418155290 2024-09-15T21:47:40.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.011+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc418152ab0 con 0x7fc418155290 2024-09-15T21:47:40.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.011+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4180cea80 con 0x7fc418155290 2024-09-15T21:47:40.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.012+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fc41815e690 con 0x7fc418155290 2024-09-15T21:47:40.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.013+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc41403ca50 con 0x7fc418155290 2024-09-15T21:47:40.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.013+0000 7fc40e7fc640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc3fc06c2b0 0x7fc3fc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:40.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.014+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5854+0+0 (secure 0 0 0) 0x7fc4140ac100 con 0x7fc418155290 2024-09-15T21:47:40.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.014+0000 7fc41d3f9640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc3fc06c2b0 0x7fc3fc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:40.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.014+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v39) ==== 54+0+16 (secure 0 0 0) 0x7fc4140ae780 con 0x7fc418155290 2024-09-15T21:47:40.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.014+0000 7fc41fe5e640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fc4181f4900 0x7fc418148940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:40.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.014+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc418152850 con 0x7fc4181f4900 2024-09-15T21:47:40.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.014+0000 7fc41e3fb640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fc4181f4900 0x7fc418148940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:40.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.014+0000 7fc41d3f9640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc3fc06c2b0 0x7fc3fc06e770 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc418154ca0 tx=0x7fc408001690 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:40.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.014+0000 7fc41e3fb640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fc4181f4900 0x7fc418148940 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:40.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.015+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc418152850 con 0x7fc4181f4900 2024-09-15T21:47:40.020 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:40.018+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:40.030 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:40.030 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:40.030 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:40.030 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:40.030 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.030 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:47:40.030 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:40.031 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 86.793078828000006, 2024-09-15T21:47:40.031 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0944283340000001, 2024-09-15T21:47:40.031 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.031 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.031 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.031 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:40.031 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:40.031 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:40.031 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2024-09-15T21:47:40.031 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.031 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.031 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.031 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.031 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:40.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:40.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.032 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.032 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234360+0000", 2024-09-15T21:47:40.032 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3710000000000001e-06 2024-09-15T21:47:40.032 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.032 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234374+0000", 2024-09-15T21:47:40.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4003e-05 2024-09-15T21:47:40.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234392+0000", 2024-09-15T21:47:40.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7294000000000001e-05 2024-09-15T21:47:40.033 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.033 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.033 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.033 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234405+0000", 2024-09-15T21:47:40.033 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2961e-05 2024-09-15T21:47:40.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234483+0000", 2024-09-15T21:47:40.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8850000000000006e-05 2024-09-15T21:47:40.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-15T21:47:40.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234491+0000", 2024-09-15T21:47:40.034 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3529999999999997e-06 2024-09-15T21:47:40.034 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.034 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.034 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.034 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327253+0000", 2024-09-15T21:47:40.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.092761962 2024-09-15T21:47:40.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327456+0000", 2024-09-15T21:47:40.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000202893 2024-09-15T21:47:40.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.035 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327652+0000", 2024-09-15T21:47:40.035 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019613000000000001 2024-09-15T21:47:40.035 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.035 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.035 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328368+0000", 2024-09-15T21:47:40.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000716582 2024-09-15T21:47:40.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328682+0000", 2024-09-15T21:47:40.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000313825 2024-09-15T21:47:40.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.036 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.036 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.036 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328743+0000", 2024-09-15T21:47:40.036 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0571e-05 2024-09-15T21:47:40.036 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.037 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.037 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328787+0000", 2024-09-15T21:47:40.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4539000000000002e-05 2024-09-15T21:47:40.037 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.037 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.037 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.037 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.037 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.037 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:40.037 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:40.037 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.671913426000003, 2024-09-15T21:47:40.037 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0089551270000000002, 2024-09-15T21:47:40.037 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.038 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.038 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:40.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:40.038 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:40.038 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 35 2024-09-15T21:47:40.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.038 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.038 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.038 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.038 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:40.038 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.038 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:40.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355525+0000", 2024-09-15T21:47:40.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:47:40.039 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.039 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.039 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.039 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355606+0000", 2024-09-15T21:47:40.039 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1723000000000003e-05 2024-09-15T21:47:40.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355624+0000", 2024-09-15T21:47:40.040 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7645000000000001e-05 2024-09-15T21:47:40.040 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.040 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.040 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.040 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355629+0000", 2024-09-15T21:47:40.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7319999999999998e-06 2024-09-15T21:47:40.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358315+0000", 2024-09-15T21:47:40.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026858659999999999 2024-09-15T21:47:40.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.041 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358373+0000", 2024-09-15T21:47:40.041 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8593e-05 2024-09-15T21:47:40.041 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.041 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.041 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358521+0000", 2024-09-15T21:47:40.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014817500000000001 2024-09-15T21:47:40.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360353+0000", 2024-09-15T21:47:40.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018317100000000001 2024-09-15T21:47:40.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.042 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.042 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364424+0000", 2024-09-15T21:47:40.042 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040707970000000001 2024-09-15T21:47:40.042 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.042 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364452+0000", 2024-09-15T21:47:40.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7937999999999999e-05 2024-09-15T21:47:40.043 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.043 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.043 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.043 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364479+0000", 2024-09-15T21:47:40.043 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7744999999999999e-05 2024-09-15T21:47:40.043 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.043 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.044 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.044 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.044 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.044 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:40.044 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:40.044 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.671829462999995, 2024-09-15T21:47:40.044 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090267869999999997, 2024-09-15T21:47:40.044 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.044 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.044 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.044 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:40.044 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:40.044 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:40.044 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 36 2024-09-15T21:47:40.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.045 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:40.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.045 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:40.045 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.045 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.045 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.045 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355609+0000", 2024-09-15T21:47:40.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3099999999999998e-07 2024-09-15T21:47:40.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355612+0000", 2024-09-15T21:47:40.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1489999999999998e-06 2024-09-15T21:47:40.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.046 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.046 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355637+0000", 2024-09-15T21:47:40.046 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.508e-05 2024-09-15T21:47:40.046 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.046 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355640+0000", 2024-09-15T21:47:40.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7930000000000002e-06 2024-09-15T21:47:40.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359195+0000", 2024-09-15T21:47:40.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035555529999999999 2024-09-15T21:47:40.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.047 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.047 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.047 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359283+0000", 2024-09-15T21:47:40.047 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8042999999999997e-05 2024-09-15T21:47:40.047 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359419+0000", 2024-09-15T21:47:40.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013576699999999999 2024-09-15T21:47:40.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360361+0000", 2024-09-15T21:47:40.048 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094171000000000001 2024-09-15T21:47:40.048 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.048 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.048 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.048 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364536+0000", 2024-09-15T21:47:40.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00417484 2024-09-15T21:47:40.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364590+0000", 2024-09-15T21:47:40.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4355999999999999e-05 2024-09-15T21:47:40.049 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.049 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.049 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.049 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364635+0000", 2024-09-15T21:47:40.049 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5065000000000003e-05 2024-09-15T21:47:40.049 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.049 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.050 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.050 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.050 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.050 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:40.050 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:40.050 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.671824857999994, 2024-09-15T21:47:40.050 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091162750000000001, 2024-09-15T21:47:40.050 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.050 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.050 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.050 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:40.050 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:40.050 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:40.050 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 37 2024-09-15T21:47:40.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.051 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:40.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.051 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:40.051 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.051 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.051 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.051 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:40.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:47:40.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355615+0000", 2024-09-15T21:47:40.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-15T21:47:40.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.052 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.052 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355646+0000", 2024-09-15T21:47:40.052 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0391000000000001e-05 2024-09-15T21:47:40.052 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.052 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355648+0000", 2024-09-15T21:47:40.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6309999999999999e-06 2024-09-15T21:47:40.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359770+0000", 2024-09-15T21:47:40.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041222899999999998 2024-09-15T21:47:40.053 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.053 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.053 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359854+0000", 2024-09-15T21:47:40.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3399000000000006e-05 2024-09-15T21:47:40.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359988+0000", 2024-09-15T21:47:40.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013442200000000001 2024-09-15T21:47:40.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360524+0000", 2024-09-15T21:47:40.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00053582900000000004 2024-09-15T21:47:40.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364673+0000", 2024-09-15T21:47:40.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041493199999999997 2024-09-15T21:47:40.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364704+0000", 2024-09-15T21:47:40.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0380999999999999e-05 2024-09-15T21:47:40.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364729+0000", 2024-09-15T21:47:40.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5335999999999999e-05 2024-09-15T21:47:40.055 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.055 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.055 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.056 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:40.056 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:40.056 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.603107620000003, 2024-09-15T21:47:40.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0026270020000001, 2024-09-15T21:47:40.056 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.056 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.056 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.056 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:40.056 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:40.056 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:40.056 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 88 2024-09-15T21:47:40.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.056 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:40.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:40.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424331+0000", 2024-09-15T21:47:40.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4299999999999998e-07 2024-09-15T21:47:40.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424333+0000", 2024-09-15T21:47:40.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2359999999999999e-06 2024-09-15T21:47:40.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422895+0000", 2024-09-15T21:47:40.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99856184199999998 2024-09-15T21:47:40.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422902+0000", 2024-09-15T21:47:40.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4440000000000002e-06 2024-09-15T21:47:40.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423053+0000", 2024-09-15T21:47:40.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000150958 2024-09-15T21:47:40.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.059 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.059 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423160+0000", 2024-09-15T21:47:40.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106774 2024-09-15T21:47:40.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423344+0000", 2024-09-15T21:47:40.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018424199999999999 2024-09-15T21:47:40.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.424730+0000", 2024-09-15T21:47:40.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001386155 2024-09-15T21:47:40.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426756+0000", 2024-09-15T21:47:40.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020261150000000002 2024-09-15T21:47:40.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426906+0000", 2024-09-15T21:47:40.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000149176 2024-09-15T21:47:40.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426957+0000", 2024-09-15T21:47:40.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1616999999999997e-05 2024-09-15T21:47:40.061 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.061 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.061 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.062 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:40.062 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:40.062 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.473034421999998, 2024-09-15T21:47:40.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095037790000000004, 2024-09-15T21:47:40.062 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.062 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.062 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.062 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:40.062 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:40.064 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:40.064 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 114 2024-09-15T21:47:40.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.064 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:40.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:40.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554404+0000", 2024-09-15T21:47:40.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2399999999999999e-07 2024-09-15T21:47:40.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554407+0000", 2024-09-15T21:47:40.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-15T21:47:40.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554632+0000", 2024-09-15T21:47:40.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022568999999999999 2024-09-15T21:47:40.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554640+0000", 2024-09-15T21:47:40.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7970000000000001e-06 2024-09-15T21:47:40.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561325+0000", 2024-09-15T21:47:40.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066848020000000001 2024-09-15T21:47:40.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561402+0000", 2024-09-15T21:47:40.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6851e-05 2024-09-15T21:47:40.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561530+0000", 2024-09-15T21:47:40.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012863999999999999 2024-09-15T21:47:40.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563144+0000", 2024-09-15T21:47:40.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016132849999999999 2024-09-15T21:47:40.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563800+0000", 2024-09-15T21:47:40.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00065586099999999999 2024-09-15T21:47:40.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563857+0000", 2024-09-15T21:47:40.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.702e-05 2024-09-15T21:47:40.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563907+0000", 2024-09-15T21:47:40.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0479000000000002e-05 2024-09-15T21:47:40.069 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.069 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.069 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.069 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:40.069 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:40.069 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.473029424999993, 2024-09-15T21:47:40.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011813423999999999, 2024-09-15T21:47:40.070 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.070 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.070 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.070 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:40.070 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:40.070 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:40.070 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 115 2024-09-15T21:47:40.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.070 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:40.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:40.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554409+0000", 2024-09-15T21:47:40.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1100000000000002e-07 2024-09-15T21:47:40.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554498+0000", 2024-09-15T21:47:40.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9642999999999995e-05 2024-09-15T21:47:40.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554656+0000", 2024-09-15T21:47:40.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000158114 2024-09-15T21:47:40.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554661+0000", 2024-09-15T21:47:40.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0730000000000004e-06 2024-09-15T21:47:40.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561862+0000", 2024-09-15T21:47:40.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072002999999999998 2024-09-15T21:47:40.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561939+0000", 2024-09-15T21:47:40.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7265e-05 2024-09-15T21:47:40.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562073+0000", 2024-09-15T21:47:40.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013371600000000001 2024-09-15T21:47:40.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563196+0000", 2024-09-15T21:47:40.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011231679999999999 2024-09-15T21:47:40.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566124+0000", 2024-09-15T21:47:40.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002928403 2024-09-15T21:47:40.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566183+0000", 2024-09-15T21:47:40.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8328000000000001e-05 2024-09-15T21:47:40.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566222+0000", 2024-09-15T21:47:40.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9103000000000002e-05 2024-09-15T21:47:40.075 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.075 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.075 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.075 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:40.075 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:40.075 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.472935985000007, 2024-09-15T21:47:40.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01181426, 2024-09-15T21:47:40.075 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 116 2024-09-15T21:47:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554503+0000", 2024-09-15T21:47:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8000000000000005e-07 2024-09-15T21:47:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554506+0000", 2024-09-15T21:47:40.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5899999999999999e-06 2024-09-15T21:47:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554674+0000", 2024-09-15T21:47:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016763600000000001 2024-09-15T21:47:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554678+0000", 2024-09-15T21:47:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5310000000000003e-06 2024-09-15T21:47:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562714+0000", 2024-09-15T21:47:40.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.008035258 2024-09-15T21:47:40.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562765+0000", 2024-09-15T21:47:40.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1093e-05 2024-09-15T21:47:40.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562845+0000", 2024-09-15T21:47:40.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0427999999999996e-05 2024-09-15T21:47:40.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563943+0000", 2024-09-15T21:47:40.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097647 2024-09-15T21:47:40.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566232+0000", 2024-09-15T21:47:40.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022890850000000002 2024-09-15T21:47:40.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566279+0000", 2024-09-15T21:47:40.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6692e-05 2024-09-15T21:47:40.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566316+0000", 2024-09-15T21:47:40.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.752e-05 2024-09-15T21:47:40.081 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.081 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.081 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.081 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:40.081 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:40.081 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.472929886000003, 2024-09-15T21:47:40.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012232125, 2024-09-15T21:47:40.081 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.082 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.082 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.082 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:40.082 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:40.082 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:40.082 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 117 2024-09-15T21:47:40.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.082 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:40.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:40.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:40.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7700000000000001e-07 2024-09-15T21:47:40.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554511+0000", 2024-09-15T21:47:40.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8430000000000001e-06 2024-09-15T21:47:40.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554690+0000", 2024-09-15T21:47:40.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017856299999999999 2024-09-15T21:47:40.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554698+0000", 2024-09-15T21:47:40.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6200000000000005e-06 2024-09-15T21:47:40.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563224+0000", 2024-09-15T21:47:40.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0085262640000000004 2024-09-15T21:47:40.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563342+0000", 2024-09-15T21:47:40.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117687 2024-09-15T21:47:40.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563472+0000", 2024-09-15T21:47:40.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012957999999999999 2024-09-15T21:47:40.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566367+0000", 2024-09-15T21:47:40.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028952280000000001 2024-09-15T21:47:40.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566680+0000", 2024-09-15T21:47:40.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031345100000000002 2024-09-15T21:47:40.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566714+0000", 2024-09-15T21:47:40.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3701000000000001e-05 2024-09-15T21:47:40.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566740+0000", 2024-09-15T21:47:40.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5911e-05 2024-09-15T21:47:40.087 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.087 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.087 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.087 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:40.087 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:40.087 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.130037803999997, 2024-09-15T21:47:40.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069806590000000002, 2024-09-15T21:47:40.087 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.087 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.088 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.088 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:40.088 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:40.088 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:40.088 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 134 2024-09-15T21:47:40.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.088 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:40.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:40.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897401+0000", 2024-09-15T21:47:40.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0500000000000003e-07 2024-09-15T21:47:40.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897492+0000", 2024-09-15T21:47:40.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1770999999999999e-05 2024-09-15T21:47:40.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897750+0000", 2024-09-15T21:47:40.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025778299999999998 2024-09-15T21:47:40.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897756+0000", 2024-09-15T21:47:40.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2140000000000001e-06 2024-09-15T21:47:40.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899312+0000", 2024-09-15T21:47:40.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001556144 2024-09-15T21:47:40.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899372+0000", 2024-09-15T21:47:40.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0015000000000001e-05 2024-09-15T21:47:40.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899444+0000", 2024-09-15T21:47:40.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1923999999999998e-05 2024-09-15T21:47:40.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901354+0000", 2024-09-15T21:47:40.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019092530000000001 2024-09-15T21:47:40.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904327+0000", 2024-09-15T21:47:40.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029734520000000001 2024-09-15T21:47:40.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904360+0000", 2024-09-15T21:47:40.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3216000000000002e-05 2024-09-15T21:47:40.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904381+0000", 2024-09-15T21:47:40.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0282e-05 2024-09-15T21:47:40.093 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.093 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.093 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.093 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:40.093 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:40.093 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.129941947000006, 2024-09-15T21:47:40.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070121059999999997, 2024-09-15T21:47:40.093 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.093 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.093 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.094 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:40.094 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:40.094 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:40.094 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 135 2024-09-15T21:47:40.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.094 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:40.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:40.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897497+0000", 2024-09-15T21:47:40.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4799999999999997e-07 2024-09-15T21:47:40.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897500+0000", 2024-09-15T21:47:40.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2949999999999998e-06 2024-09-15T21:47:40.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897781+0000", 2024-09-15T21:47:40.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028064300000000002 2024-09-15T21:47:40.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897786+0000", 2024-09-15T21:47:40.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5169999999999999e-06 2024-09-15T21:47:40.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899857+0000", 2024-09-15T21:47:40.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020705340000000002 2024-09-15T21:47:40.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899932+0000", 2024-09-15T21:47:40.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5840999999999994e-05 2024-09-15T21:47:40.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900005+0000", 2024-09-15T21:47:40.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2716e-05 2024-09-15T21:47:40.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902296+0000", 2024-09-15T21:47:40.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022912010000000001 2024-09-15T21:47:40.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904424+0000", 2024-09-15T21:47:40.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021278600000000001 2024-09-15T21:47:40.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904472+0000", 2024-09-15T21:47:40.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7586000000000001e-05 2024-09-15T21:47:40.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904508+0000", 2024-09-15T21:47:40.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6164999999999997e-05 2024-09-15T21:47:40.099 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.099 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.099 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.099 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:40.100 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:40.100 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.129917456000001, 2024-09-15T21:47:40.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070611429999999998, 2024-09-15T21:47:40.100 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.100 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.100 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.100 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:40.100 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:40.100 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:40.101 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 136 2024-09-15T21:47:40.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.101 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:40.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:40.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897521+0000", 2024-09-15T21:47:40.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2900000000000004e-07 2024-09-15T21:47:40.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897527+0000", 2024-09-15T21:47:40.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8490000000000002e-06 2024-09-15T21:47:40.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897799+0000", 2024-09-15T21:47:40.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027219499999999998 2024-09-15T21:47:40.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897804+0000", 2024-09-15T21:47:40.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4219999999999999e-06 2024-09-15T21:47:40.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900248+0000", 2024-09-15T21:47:40.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002443457 2024-09-15T21:47:40.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900306+0000", 2024-09-15T21:47:40.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8263000000000001e-05 2024-09-15T21:47:40.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900385+0000", 2024-09-15T21:47:40.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9161999999999999e-05 2024-09-15T21:47:40.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902300+0000", 2024-09-15T21:47:40.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001914947 2024-09-15T21:47:40.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904536+0000", 2024-09-15T21:47:40.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022354419999999998 2024-09-15T21:47:40.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904562+0000", 2024-09-15T21:47:40.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6378999999999999e-05 2024-09-15T21:47:40.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904581+0000", 2024-09-15T21:47:40.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9497999999999999e-05 2024-09-15T21:47:40.106 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.107 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.107 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.107 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:40.107 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:40.107 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.129908889000006, 2024-09-15T21:47:40.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007180306, 2024-09-15T21:47:40.107 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.107 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.107 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.107 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:40.107 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:40.107 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:40.108 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 137 2024-09-15T21:47:40.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.108 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:40.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:40.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:40.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1399999999999998e-07 2024-09-15T21:47:40.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897612+0000", 2024-09-15T21:47:40.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2607000000000004e-05 2024-09-15T21:47:40.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897815+0000", 2024-09-15T21:47:40.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020352400000000001 2024-09-15T21:47:40.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897821+0000", 2024-09-15T21:47:40.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4670000000000003e-06 2024-09-15T21:47:40.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900665+0000", 2024-09-15T21:47:40.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028443689999999998 2024-09-15T21:47:40.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900714+0000", 2024-09-15T21:47:40.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8812e-05 2024-09-15T21:47:40.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900799+0000", 2024-09-15T21:47:40.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5129000000000004e-05 2024-09-15T21:47:40.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903413+0000", 2024-09-15T21:47:40.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026142890000000001 2024-09-15T21:47:40.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904625+0000", 2024-09-15T21:47:40.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012121149999999999 2024-09-15T21:47:40.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904673+0000", 2024-09-15T21:47:40.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7540999999999998e-05 2024-09-15T21:47:40.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904709+0000", 2024-09-15T21:47:40.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6139000000000001e-05 2024-09-15T21:47:40.112 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.112 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.112 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.113 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:40.113 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:40.113 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.129822934000003, 2024-09-15T21:47:40.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071727450000000003, 2024-09-15T21:47:40.113 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.113 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.113 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.113 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:40.113 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:40.113 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:40.113 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 138 2024-09-15T21:47:40.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.114 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:40.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:40.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:40.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:47:40.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897619+0000", 2024-09-15T21:47:40.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3869999999999999e-06 2024-09-15T21:47:40.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897832+0000", 2024-09-15T21:47:40.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021329500000000001 2024-09-15T21:47:40.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897837+0000", 2024-09-15T21:47:40.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3730000000000001e-06 2024-09-15T21:47:40.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901391+0000", 2024-09-15T21:47:40.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003553601 2024-09-15T21:47:40.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901466+0000", 2024-09-15T21:47:40.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5130999999999996e-05 2024-09-15T21:47:40.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901601+0000", 2024-09-15T21:47:40.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134797 2024-09-15T21:47:40.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903417+0000", 2024-09-15T21:47:40.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001816156 2024-09-15T21:47:40.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904741+0000", 2024-09-15T21:47:40.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013241049999999999 2024-09-15T21:47:40.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904767+0000", 2024-09-15T21:47:40.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5876000000000001e-05 2024-09-15T21:47:40.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904788+0000", 2024-09-15T21:47:40.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0449999999999999e-05 2024-09-15T21:47:40.118 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.129817281000001, 2024-09-15T21:47:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072970250000000004, 2024-09-15T21:47:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:40.119 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:40.120 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 139 2024-09-15T21:47:40.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.120 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:40.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:40.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897621+0000", 2024-09-15T21:47:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3099999999999999e-07 2024-09-15T21:47:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897625+0000", 2024-09-15T21:47:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.698e-06 2024-09-15T21:47:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897848+0000", 2024-09-15T21:47:40.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022375699999999999 2024-09-15T21:47:40.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897853+0000", 2024-09-15T21:47:40.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7709999999999997e-06 2024-09-15T21:47:40.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901929+0000", 2024-09-15T21:47:40.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040757780000000004 2024-09-15T21:47:40.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901976+0000", 2024-09-15T21:47:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6751e-05 2024-09-15T21:47:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902054+0000", 2024-09-15T21:47:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8856999999999994e-05 2024-09-15T21:47:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903421+0000", 2024-09-15T21:47:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00136619 2024-09-15T21:47:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904827+0000", 2024-09-15T21:47:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0014067369999999999 2024-09-15T21:47:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904881+0000", 2024-09-15T21:47:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3520999999999998e-05 2024-09-15T21:47:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904918+0000", 2024-09-15T21:47:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6634000000000002e-05 2024-09-15T21:47:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.129811451999998, 2024-09-15T21:47:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097221730000000006, 2024-09-15T21:47:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.125 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 140 2024-09-15T21:47:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897627+0000", 2024-09-15T21:47:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-15T21:47:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897698+0000", 2024-09-15T21:47:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1722000000000002e-05 2024-09-15T21:47:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897863+0000", 2024-09-15T21:47:40.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016479099999999999 2024-09-15T21:47:40.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897868+0000", 2024-09-15T21:47:40.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4950000000000002e-06 2024-09-15T21:47:40.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902335+0000", 2024-09-15T21:47:40.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004467632 2024-09-15T21:47:40.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902414+0000", 2024-09-15T21:47:40.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8688000000000003e-05 2024-09-15T21:47:40.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902535+0000", 2024-09-15T21:47:40.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012066900000000001 2024-09-15T21:47:40.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903714+0000", 2024-09-15T21:47:40.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011794520000000001 2024-09-15T21:47:40.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907115+0000", 2024-09-15T21:47:40.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034008879999999999 2024-09-15T21:47:40.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907250+0000", 2024-09-15T21:47:40.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134657 2024-09-15T21:47:40.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907348+0000", 2024-09-15T21:47:40.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8838000000000005e-05 2024-09-15T21:47:40.131 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.131 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.131 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.131 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:40.131 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:40.131 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.129736371000007, 2024-09-15T21:47:40.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097618050000000001, 2024-09-15T21:47:40.132 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.132 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.132 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.132 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:40.132 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:40.132 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:40.132 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 141 2024-09-15T21:47:40.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.132 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:40.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:40.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897702+0000", 2024-09-15T21:47:40.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6199999999999998e-07 2024-09-15T21:47:40.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897705+0000", 2024-09-15T21:47:40.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0180000000000002e-06 2024-09-15T21:47:40.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897889+0000", 2024-09-15T21:47:40.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000183678 2024-09-15T21:47:40.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897894+0000", 2024-09-15T21:47:40.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8930000000000004e-06 2024-09-15T21:47:40.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903099+0000", 2024-09-15T21:47:40.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052056280000000003 2024-09-15T21:47:40.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903151+0000", 2024-09-15T21:47:40.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2163000000000002e-05 2024-09-15T21:47:40.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903220+0000", 2024-09-15T21:47:40.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8483000000000001e-05 2024-09-15T21:47:40.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904794+0000", 2024-09-15T21:47:40.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001574521 2024-09-15T21:47:40.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907405+0000", 2024-09-15T21:47:40.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026110339999999999 2024-09-15T21:47:40.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907440+0000", 2024-09-15T21:47:40.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4564999999999999e-05 2024-09-15T21:47:40.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907463+0000", 2024-09-15T21:47:40.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3260000000000001e-05 2024-09-15T21:47:40.137 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.137 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.137 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.137 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:22 2.8 2:1470b2f8:::613.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:40.137 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:40.137 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 65.572296043999998, 2024-09-15T21:47:40.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007454937, 2024-09-15T21:47:40.137 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.138 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.138 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.138 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:40.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:40.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:40.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6800000000000002e-07 2024-09-15T21:47:40.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455145+0000", 2024-09-15T21:47:40.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4470000000000002e-06 2024-09-15T21:47:40.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456425+0000", 2024-09-15T21:47:40.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012798200000000001 2024-09-15T21:47:40.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456433+0000", 2024-09-15T21:47:40.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7389999999999999e-06 2024-09-15T21:47:40.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:40.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097865 2024-09-15T21:47:40.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457614+0000", 2024-09-15T21:47:40.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3560000000000006e-05 2024-09-15T21:47:40.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457998+0000", 2024-09-15T21:47:40.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00038372699999999999 2024-09-15T21:47:40.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460488+0000", 2024-09-15T21:47:40.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002489855 2024-09-15T21:47:40.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462512+0000", 2024-09-15T21:47:40.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020235269999999998 2024-09-15T21:47:40.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462554+0000", 2024-09-15T21:47:40.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2472000000000003e-05 2024-09-15T21:47:40.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462597+0000", 2024-09-15T21:47:40.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2657000000000002e-05 2024-09-15T21:47:40.143 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.143 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.143 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.143 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:2 2.5 2:a6e36b3f:::401.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:40.143 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:40.143 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 65.572291074999995, 2024-09-15T21:47:40.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074484590000000002, 2024-09-15T21:47:40.143 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.143 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.143 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.144 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:40.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:40.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:40.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:47:40.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455150+0000", 2024-09-15T21:47:40.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7319999999999998e-06 2024-09-15T21:47:40.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456449+0000", 2024-09-15T21:47:40.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012991490000000001 2024-09-15T21:47:40.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456454+0000", 2024-09-15T21:47:40.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0390000000000004e-06 2024-09-15T21:47:40.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456513+0000", 2024-09-15T21:47:40.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8768999999999999e-05 2024-09-15T21:47:40.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456628+0000", 2024-09-15T21:47:40.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115641 2024-09-15T21:47:40.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456741+0000", 2024-09-15T21:47:40.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000113094 2024-09-15T21:47:40.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458520+0000", 2024-09-15T21:47:40.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001778993 2024-09-15T21:47:40.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462494+0000", 2024-09-15T21:47:40.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039737009999999996 2024-09-15T21:47:40.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462549+0000", 2024-09-15T21:47:40.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4537000000000003e-05 2024-09-15T21:47:40.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462595+0000", 2024-09-15T21:47:40.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6511000000000002e-05 2024-09-15T21:47:40.148 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.148 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.148 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.149 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:27 2.5 2:a5fa685f:::504.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:40.149 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:40.149 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 48.462598690999997, 2024-09-15T21:47:40.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071804599999999996, 2024-09-15T21:47:40.149 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:40.149 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:40.149 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:40.149 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:40.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:40.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:40.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:40.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:40.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:40.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:40.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:40.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-15T21:47:40.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:40.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564841+0000", 2024-09-15T21:47:40.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-15T21:47:40.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:40.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565126+0000", 2024-09-15T21:47:40.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028469900000000002 2024-09-15T21:47:40.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:40.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565133+0000", 2024-09-15T21:47:40.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9480000000000002e-06 2024-09-15T21:47:40.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:40.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568006+0000", 2024-09-15T21:47:40.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028730330000000001 2024-09-15T21:47:40.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:40.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568119+0000", 2024-09-15T21:47:40.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011325899999999999 2024-09-15T21:47:40.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:40.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568235+0000", 2024-09-15T21:47:40.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115868 2024-09-15T21:47:40.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:40.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569668+0000", 2024-09-15T21:47:40.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001433099 2024-09-15T21:47:40.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:40.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571926+0000", 2024-09-15T21:47:40.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022572410000000001 2024-09-15T21:47:40.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:40.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571981+0000", 2024-09-15T21:47:40.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5383000000000002e-05 2024-09-15T21:47:40.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:40.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:40.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:40.154 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572020+0000", 2024-09-15T21:47:40.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8574e-05 2024-09-15T21:47:40.154 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.154 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.154 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.154 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:40.154 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:40.154 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:40.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.043+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc41815b7a0 con 0x7fc4181f4900 2024-09-15T21:47:40.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.046+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc41815b7a0 con 0x7fc4181f4900 2024-09-15T21:47:40.156 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:40.157 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:47:40.158 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:40.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.046+0000 7fc41fe5e640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fc418148e80 0x7fc4181f0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:40.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.046+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fc41815daf0 con 0x7fc418148e80 2024-09-15T21:47:40.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.046+0000 7fc41dbfa640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fc418148e80 0x7fc4181f0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:40.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.047+0000 7fc41dbfa640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fc418148e80 0x7fc4181f0e10 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:40.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.048+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc41815daf0 con 0x7fc418148e80 2024-09-15T21:47:40.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.076+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc418159190 con 0x7fc418148e80 2024-09-15T21:47:40.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.078+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc418159190 con 0x7fc418148e80 2024-09-15T21:47:40.158 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:40.159 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:40.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.079+0000 7fc41fe5e640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fc4180b5c20 0x7fc4181f1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:40.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.079+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fc4180cb130 con 0x7fc4180b5c20 2024-09-15T21:47:40.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.079+0000 7fc41d3f9640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fc4180b5c20 0x7fc4181f1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:40.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.079+0000 7fc41d3f9640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fc4180b5c20 0x7fc4181f1350 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:40.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.080+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4180cb130 con 0x7fc4180b5c20 2024-09-15T21:47:40.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.107+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc4181f1890 con 0x7fc4180b5c20 2024-09-15T21:47:40.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.109+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4181f1890 con 0x7fc4180b5c20 2024-09-15T21:47:40.160 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.109+0000 7fc41fe5e640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fc418062b40 0x7fc418062fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.109+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fc4181f1890 con 0x7fc418062b40 2024-09-15T21:47:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.110+0000 7fc41e3fb640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fc418062b40 0x7fc418062fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:40.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.110+0000 7fc41e3fb640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fc418062b40 0x7fc418062fa0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:40.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.111+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4181f1890 con 0x7fc418062b40 2024-09-15T21:47:40.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.138+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc4181f1890 con 0x7fc418062b40 2024-09-15T21:47:40.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.140+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc4181f1890 con 0x7fc418062b40 2024-09-15T21:47:40.162 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:40.163 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:40.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.141+0000 7fc41fe5e640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fc41814d0d0 0x7fc41814d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:40.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.141+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fc41814daf0 con 0x7fc41814d0d0 2024-09-15T21:47:40.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.141+0000 7fc41dbfa640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fc41814d0d0 0x7fc41814d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:40.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.141+0000 7fc41dbfa640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fc41814d0d0 0x7fc41814d530 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:40.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.142+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc41814daf0 con 0x7fc41814d0d0 2024-09-15T21:47:40.163 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:40.154+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:40.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.169+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc41802c640 con 0x7fc41814d0d0 2024-09-15T21:47:40.172 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.171+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc41802c640 con 0x7fc41814d0d0 2024-09-15T21:47:40.172 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:40.174 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:40.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.171+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=40}) -- 0x7fc418206eb0 con 0x7fc418155290 2024-09-15T21:47:40.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.171+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc41802c640 con 0x7fc418155290 2024-09-15T21:47:40.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.171+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc4140b3070 con 0x7fc418155290 2024-09-15T21:47:40.174 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:47:40.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.172+0000 7fc41fe5e640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fc41802d370 0x7fc4182057a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:40.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.172+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fc41802d9f0 con 0x7fc41802d370 2024-09-15T21:47:40.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.172+0000 7fc41d3f9640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fc41802d370 0x7fc4182057a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:40.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.172+0000 7fc41d3f9640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fc41802d370 0x7fc4182057a0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:40.175 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.173+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc41802d9f0 con 0x7fc41802d370 2024-09-15T21:47:40.201 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.200+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc41802d970 con 0x7fc41802d370 2024-09-15T21:47:40.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.202+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc41802d970 con 0x7fc41802d370 2024-09-15T21:47:40.203 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:40.204 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:40.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.202+0000 7fc41fe5e640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fc4182061f0 0x7fc418206630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:40.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.202+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fc41820d930 con 0x7fc4182061f0 2024-09-15T21:47:40.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.202+0000 7fc41e3fb640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fc4182061f0 0x7fc418206630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:40.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.203+0000 7fc41e3fb640 1 --2- 172.21.15.100:0/2334705401 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fc4182061f0 0x7fc418206630 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:40.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.204+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc41820d930 con 0x7fc4182061f0 2024-09-15T21:47:40.232 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.230+0000 7fc41fe5e640 1 -- 172.21.15.100:0/2334705401 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fc41820b0a0 con 0x7fc4182061f0 2024-09-15T21:47:40.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.232+0000 7fc40e7fc640 1 -- 172.21.15.100:0/2334705401 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fc41820b0a0 con 0x7fc4182061f0 2024-09-15T21:47:40.234 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:40.234 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:40.239 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:47:40.239 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:47:40.281 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:40.279+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:40.328 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:40.328 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:40.340 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:47:40.356 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:40.355+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:40.457 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:40.456+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:40.497 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:40.497 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:40.507 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:47:40.558 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:40.557+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:40.611 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-15T21:47:40.611 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:47:40.611 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:47:40.614 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:40.614 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:40.624 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:47:40.659 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:40.658+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:40.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.717+0000 7fe8fbd8d640 1 Processor -- start 2024-09-15T21:47:40.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.717+0000 7fe8fbd8d640 1 -- start start 2024-09-15T21:47:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.718+0000 7fe8fbd8d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe8f415c0b0 0x7fe8f40c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.718+0000 7fe8fbd8d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8f40cd9e0 con 0x7fe8f4072c30 2024-09-15T21:47:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.718+0000 7fe8fbd8d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8f40cdc80 con 0x7fe8f4073010 2024-09-15T21:47:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.718+0000 7fe8f9b29640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe8f4072c30 0x7fe8f41588c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47936/0 (socket says 172.21.15.100:47936) 2024-09-15T21:47:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.718+0000 7fe8f9b29640 1 -- 172.21.15.100:0/1993720713 learned_addr learned my addr 172.21.15.100:0/1993720713 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.719+0000 7fe8fbd8d640 1 -- 172.21.15.100:0/1993720713 --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fe8f415bc20 con 0x7fe8f415c0b0 2024-09-15T21:47:40.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.719+0000 7fe8fa32a640 1 --2- 172.21.15.100:0/1993720713 >> v2:172.21.15.122:3300/0 conn(0x7fe8f415c0b0 0x7fe8f40c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:40.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.719+0000 7fe8f8b27640 1 -- 172.21.15.100:0/1993720713 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 487076934 0 0) 0x7fe8f40cd9e0 con 0x7fe8f4072c30 2024-09-15T21:47:40.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.719+0000 7fe8f8b27640 1 -- 172.21.15.100:0/1993720713 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8e0005730 con 0x7fe8f4072c30 2024-09-15T21:47:40.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.719+0000 7fe8fa32a640 1 -- 172.21.15.100:0/1993720713 >> v1:172.21.15.122:6789/0 conn(0x7fe8f4073010 legacy=0x7fe8f41514a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:40.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.719+0000 7fe8fa32a640 1 -- 172.21.15.100:0/1993720713 >> v1:172.21.15.100:6789/0 conn(0x7fe8f4072c30 legacy=0x7fe8f41588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:40.722 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.719+0000 7fe8fa32a640 1 -- 172.21.15.100:0/1993720713 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f415b5d0 con 0x7fe8f415c0b0 2024-09-15T21:47:40.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.720+0000 7fe8fa32a640 1 --2- 172.21.15.100:0/1993720713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8f415c0b0 0x7fe8f40c7c10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009d90 tx=0x7fe8ec02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f453da6889046876 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:40.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.721+0000 7fe8f8b27640 1 -- 172.21.15.100:0/1993720713 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec007740 con 0x7fe8f415c0b0 2024-09-15T21:47:40.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.721+0000 7fe8f8b27640 1 -- 172.21.15.100:0/1993720713 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8ec0078e0 con 0x7fe8f415c0b0 2024-09-15T21:47:40.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.721+0000 7fe8f8b27640 1 -- 172.21.15.100:0/1993720713 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec007cb0 con 0x7fe8f415c0b0 2024-09-15T21:47:40.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.721+0000 7fe8fbd8d640 1 -- 172.21.15.100:0/1993720713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8f415c0b0 msgr2=0x7fe8f40c7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:40.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.721+0000 7fe8fbd8d640 1 --2- 172.21.15.100:0/1993720713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8f415c0b0 0x7fe8f40c7c10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009d90 tx=0x7fe8ec02a6f0 comp rx=0 tx=0).stop 2024-09-15T21:47:40.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.722+0000 7fe8fbd8d640 1 -- 172.21.15.100:0/1993720713 shutdown_connections 2024-09-15T21:47:40.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.722+0000 7fe8fbd8d640 1 --2- 172.21.15.100:0/1993720713 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8f415c0b0 0x7fe8f40c7c10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:40.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.722+0000 7fe8fbd8d640 1 -- 172.21.15.100:0/1993720713 >> 172.21.15.100:0/1993720713 conn(0x7fe8f4073820 msgr2=0x7fe8f4073c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:40.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.722+0000 7fe8fbd8d640 1 -- 172.21.15.100:0/1993720713 shutdown_connections 2024-09-15T21:47:40.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.722+0000 7fe8fbd8d640 1 -- 172.21.15.100:0/1993720713 wait complete. 2024-09-15T21:47:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.723+0000 7fe8fbd8d640 1 Processor -- start 2024-09-15T21:47:40.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.724+0000 7fe8fbd8d640 1 -- start start 2024-09-15T21:47:40.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.725+0000 7fe8fbd8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8f4072c30 0x7fe8f41f04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:40.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.725+0000 7fe8fbd8d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe8f4073010 0x7fe8f415a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:40.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.725+0000 7fe8f9b29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8f4072c30 0x7fe8f41f04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:40.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.725+0000 7fe8f9b29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8f4072c30 0x7fe8f41f04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33728/0 (socket says 172.21.15.100:33728) 2024-09-15T21:47:40.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.725+0000 7fe8f9b29640 1 -- 172.21.15.100:0/2533267153 learned_addr learned my addr 172.21.15.100:0/2533267153 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:40.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.725+0000 7fe8fbd8d640 1 --2- 172.21.15.100:0/2533267153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8f415c0b0 0x7fe8f40cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:40.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.725+0000 7fe8fbd8d640 1 -- 172.21.15.100:0/2533267153 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f40cd670 con 0x7fe8f4072c30 2024-09-15T21:47:40.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.725+0000 7fe8fbd8d640 1 -- 172.21.15.100:0/2533267153 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f40c9a90 con 0x7fe8f415c0b0 2024-09-15T21:47:40.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.725+0000 7fe8f9328640 1 --2- 172.21.15.100:0/2533267153 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe8f4073010 0x7fe8f415a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:40.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.725+0000 7fe8fa32a640 1 --2- 172.21.15.100:0/2533267153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8f415c0b0 0x7fe8f40cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:40.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.725+0000 7fe8fbd8d640 1 -- 172.21.15.100:0/2533267153 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe8f40c9c10 con 0x7fe8f4073010 2024-09-15T21:47:40.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.726+0000 7fe8f9328640 1 -- 172.21.15.100:0/2533267153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8f415c0b0 msgr2=0x7fe8f40cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:40.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.726+0000 7fe8f9328640 1 --2- 172.21.15.100:0/2533267153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8f415c0b0 0x7fe8f40cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:40.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.726+0000 7fe8f9328640 1 -- 172.21.15.100:0/2533267153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8f4072c30 msgr2=0x7fe8f41f04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:40.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.726+0000 7fe8f9328640 1 --2- 172.21.15.100:0/2533267153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8f4072c30 0x7fe8f41f04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:40.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.726+0000 7fe8f9328640 1 -- 172.21.15.100:0/2533267153 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f41f0a20 con 0x7fe8f4073010 2024-09-15T21:47:40.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.726+0000 7fe8f9328640 1 --2- 172.21.15.100:0/2533267153 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe8f4073010 0x7fe8f415a810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40cdbb0 tx=0x7fe8e400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:40.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.726+0000 7fe8ea7fc640 1 -- 172.21.15.100:0/2533267153 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4017070 con 0x7fe8f4073010 2024-09-15T21:47:40.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.726+0000 7fe8ea7fc640 1 -- 172.21.15.100:0/2533267153 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e40052d0 con 0x7fe8f4073010 2024-09-15T21:47:40.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.726+0000 7fe8fbd8d640 1 -- 172.21.15.100:0/2533267153 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f40cc7d0 con 0x7fe8f4073010 2024-09-15T21:47:40.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.727+0000 7fe8ea7fc640 1 -- 172.21.15.100:0/2533267153 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4003850 con 0x7fe8f4073010 2024-09-15T21:47:40.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.727+0000 7fe8fbd8d640 1 -- 172.21.15.100:0/2533267153 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f40ccd90 con 0x7fe8f4073010 2024-09-15T21:47:40.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.728+0000 7fe8ea7fc640 1 -- 172.21.15.100:0/2533267153 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe8e401a070 con 0x7fe8f4073010 2024-09-15T21:47:40.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.729+0000 7fe8ea7fc640 1 --2- 172.21.15.100:0/2533267153 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe8d806c2e0 0x7fe8d806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:40.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.729+0000 7fe8f9b29640 1 --2- 172.21.15.100:0/2533267153 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe8d806c2e0 0x7fe8d806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:40.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.730+0000 7fe8fbd8d640 1 -- 172.21.15.100:0/2533267153 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f40ccf90 con 0x7fe8f4073010 2024-09-15T21:47:40.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.730+0000 7fe8ea7fc640 1 -- 172.21.15.100:0/2533267153 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(39..39 src has 1..39) ==== 5854+0+0 (secure 0 0 0) 0x7fe8e408a710 con 0x7fe8f4073010 2024-09-15T21:47:40.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.730+0000 7fe8f9b29640 1 --2- 172.21.15.100:0/2533267153 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe8d806c2e0 0x7fe8d806e7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00011e0 tx=0x7fe8f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:40.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:40.735+0000 7fe8ea7fc640 1 -- 172.21.15.100:0/2533267153 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe8e4086210 con 0x7fe8f4073010 2024-09-15T21:47:40.761 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:40.758+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:40.774 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:40.774 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:40.783 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:47:40.867 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:40.866+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:40.929 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:40.929 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:40.938 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:47:40.981 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:40.979+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:41.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.049+0000 7fe8fbd8d640 1 -- 172.21.15.100:0/2533267153 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe8f4067d20 con 0x7fe8d806c2e0 2024-09-15T21:47:41.063 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.061+0000 7fe8ea7fc640 1 -- 172.21.15.100:0/2533267153 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252431 (secure 0 0 0) 0x7fe8f4067d20 con 0x7fe8d806c2e0 2024-09-15T21:47:41.064 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:41.062+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:41.068 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:47:41.071 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-15T21:47:41.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.066+0000 7fe8fbd8d640 1 -- 172.21.15.100:0/2533267153 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe8d806c2e0 msgr2=0x7fe8d806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:41.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.066+0000 7fe8fbd8d640 1 --2- 172.21.15.100:0/2533267153 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe8d806c2e0 0x7fe8d806e7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe8f00011e0 tx=0x7fe8f0009040 comp rx=0 tx=0).stop 2024-09-15T21:47:41.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.066+0000 7fe8fbd8d640 1 -- 172.21.15.100:0/2533267153 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe8f4073010 msgr2=0x7fe8f415a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:41.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.066+0000 7fe8fbd8d640 1 --2- 172.21.15.100:0/2533267153 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe8f4073010 0x7fe8f415a810 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40cdbb0 tx=0x7fe8e400c800 comp rx=0 tx=0).stop 2024-09-15T21:47:41.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.066+0000 7fe8fbd8d640 1 -- 172.21.15.100:0/2533267153 shutdown_connections 2024-09-15T21:47:41.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.066+0000 7fe8fbd8d640 1 --2- 172.21.15.100:0/2533267153 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe8d806c2e0 0x7fe8d806e7a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:41.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.066+0000 7fe8fbd8d640 1 --2- 172.21.15.100:0/2533267153 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8f415c0b0 0x7fe8f40cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:41.071 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.066+0000 7fe8fbd8d640 1 --2- 172.21.15.100:0/2533267153 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe8f4073010 0x7fe8f415a810 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:41.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.066+0000 7fe8fbd8d640 1 --2- 172.21.15.100:0/2533267153 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe8f4072c30 0x7fe8f41f04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:41.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.066+0000 7fe8fbd8d640 1 -- 172.21.15.100:0/2533267153 >> 172.21.15.100:0/2533267153 conn(0x7fe8f4073820 msgr2=0x7fe8f40717b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:41.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.066+0000 7fe8fbd8d640 1 -- 172.21.15.100:0/2533267153 shutdown_connections 2024-09-15T21:47:41.072 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.067+0000 7fe8fbd8d640 1 -- 172.21.15.100:0/2533267153 wait complete. 2024-09-15T21:47:41.083 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-09-15T21:47:39.820113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601806,"num_objects":100,"num_object_clones":0,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":373,"num_write_kb":2487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":303,"ondisk_log_size":303,"up":165,"acting":165,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229068,"kb_used_data":4524,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745652,"statfs":{"total":773094113280,"available":772859547648,"internally_reserved":0,"allocated":4632576,"data_stored":2520739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.043616"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.104287+0000","last_change":"2024-09-15T21:46:23.428662+0000","last_active":"2024-09-15T21:47:37.104287+0000","last_peered":"2024-09-15T21:47:37.104287+0000","last_clean":"2024-09-15T21:47:37.104287+0000","last_became_active":"2024-09-15T21:46:23.427735+0000","last_became_peered":"2024-09-15T21:46:23.427735+0000","last_unstale":"2024-09-15T21:47:37.104287+0000","last_undegraded":"2024-09-15T21:47:37.104287+0000","last_fullsized":"2024-09-15T21:47:37.104287+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:32:29.702757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.045169+0000","last_change":"2024-09-15T21:46:23.418716+0000","last_active":"2024-09-15T21:47:34.045169+0000","last_peered":"2024-09-15T21:47:34.045169+0000","last_clean":"2024-09-15T21:47:34.045169+0000","last_became_active":"2024-09-15T21:46:23.418471+0000","last_became_peered":"2024-09-15T21:46:23.418471+0000","last_unstale":"2024-09-15T21:47:34.045169+0000","last_undegraded":"2024-09-15T21:47:34.045169+0000","last_fullsized":"2024-09-15T21:47:34.045169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:49.599801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.105533+0000","last_change":"2024-09-15T21:46:23.426404+0000","last_active":"2024-09-15T21:47:37.105533+0000","last_peered":"2024-09-15T21:47:37.105533+0000","last_clean":"2024-09-15T21:47:37.105533+0000","last_became_active":"2024-09-15T21:46:23.426146+0000","last_became_peered":"2024-09-15T21:46:23.426146+0000","last_unstale":"2024-09-15T21:47:37.105533+0000","last_undegraded":"2024-09-15T21:47:37.105533+0000","last_fullsized":"2024-09-15T21:47:37.105533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:23.626109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":37,"state":"active+undersized","last_fresh":"2024-09-15T21:47:34.045604+0000","last_change":"2024-09-15T21:47:17.678668+0000","last_active":"2024-09-15T21:47:34.045604+0000","last_peered":"2024-09-15T21:47:34.045604+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"2024-09-15T21:47:17.678668+0000","last_became_peered":"2024-09-15T21:47:17.678668+0000","last_unstale":"2024-09-15T21:47:34.045604+0000","last_undegraded":"2024-09-15T21:47:34.045604+0000","last_fullsized":"2024-09-15T21:47:17.676835+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:19:18.656474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.104273+0000","last_change":"2024-09-15T21:46:23.427120+0000","last_active":"2024-09-15T21:47:37.104273+0000","last_peered":"2024-09-15T21:47:37.104273+0000","last_clean":"2024-09-15T21:47:37.104273+0000","last_became_active":"2024-09-15T21:46:23.426706+0000","last_became_peered":"2024-09-15T21:46:23.426706+0000","last_unstale":"2024-09-15T21:47:37.104273+0000","last_undegraded":"2024-09-15T21:47:37.104273+0000","last_fullsized":"2024-09-15T21:47:37.104273+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:29.513446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":23,"reported_epoch":38,"state":"active+undersized","last_fresh":"2024-09-15T21:47:37.104848+0000","last_change":"2024-09-15T21:47:17.677576+0000","last_active":"2024-09-15T21:47:37.104848+0000","last_peered":"2024-09-15T21:47:37.104848+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"2024-09-15T21:47:17.677576+0000","last_became_peered":"2024-09-15T21:47:17.677576+0000","last_unstale":"2024-09-15T21:47:37.104848+0000","last_undegraded":"2024-09-15T21:47:37.104848+0000","last_fullsized":"2024-09-15T21:47:17.673574+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:37.798541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.104000+0000","last_change":"2024-09-15T21:46:23.429541+0000","last_active":"2024-09-15T21:47:37.104000+0000","last_peered":"2024-09-15T21:47:37.104000+0000","last_clean":"2024-09-15T21:47:37.104000+0000","last_became_active":"2024-09-15T21:46:23.429263+0000","last_became_peered":"2024-09-15T21:46:23.429263+0000","last_unstale":"2024-09-15T21:47:37.104000+0000","last_undegraded":"2024-09-15T21:47:37.104000+0000","last_fullsized":"2024-09-15T21:47:37.104000+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:34:51.530981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":38,"reported_epoch":38,"state":"active+undersized","last_fresh":"2024-09-15T21:47:36.788243+0000","last_change":"2024-09-15T21:47:17.678866+0000","last_active":"2024-09-15T21:47:36.788243+0000","last_peered":"2024-09-15T21:47:36.788243+0000","last_clean":"2024-09-15T21:47:15.661655+0000","last_became_active":"2024-09-15T21:47:17.678866+0000","last_became_peered":"2024-09-15T21:47:17.678866+0000","last_unstale":"2024-09-15T21:47:36.788243+0000","last_undegraded":"2024-09-15T21:47:36.788243+0000","last_fullsized":"2024-09-15T21:47:17.676321+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:08:57.271339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+undersized","last_fresh":"2024-09-15T21:47:37.104374+0000","last_change":"2024-09-15T21:47:17.682282+0000","last_active":"2024-09-15T21:47:37.104374+0000","last_peered":"2024-09-15T21:47:37.104374+0000","last_clean":"2024-09-15T21:47:16.541888+0000","last_became_active":"2024-09-15T21:47:17.682282+0000","last_became_peered":"2024-09-15T21:47:17.682282+0000","last_unstale":"2024-09-15T21:47:37.104374+0000","last_undegraded":"2024-09-15T21:47:37.104374+0000","last_fullsized":"2024-09-15T21:47:17.679869+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:36:12.847210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+undersized","last_fresh":"2024-09-15T21:47:36.794230+0000","last_change":"2024-09-15T21:47:17.682411+0000","last_active":"2024-09-15T21:47:36.794230+0000","last_peered":"2024-09-15T21:47:36.794230+0000","last_clean":"2024-09-15T21:47:16.541732+0000","last_became_active":"2024-09-15T21:47:17.682411+0000","last_became_peered":"2024-09-15T21:47:17.682411+0000","last_unstale":"2024-09-15T21:47:36.794230+0000","last_undegraded":"2024-09-15T21:47:36.794230+0000","last_fullsized":"2024-09-15T21:47:17.679600+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:18:54.071687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":48,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.794064+0000","last_change":"2024-09-15T21:47:05.735261+0000","last_active":"2024-09-15T21:47:36.794064+0000","last_peered":"2024-09-15T21:47:36.794064+0000","last_clean":"2024-09-15T21:47:36.794064+0000","last_became_active":"2024-09-15T21:47:05.734921+0000","last_became_peered":"2024-09-15T21:47:05.734921+0000","last_unstale":"2024-09-15T21:47:36.794064+0000","last_undegraded":"2024-09-15T21:47:36.794064+0000","last_fullsized":"2024-09-15T21:47:36.794064+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:55.787926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":44,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.794214+0000","last_change":"2024-09-15T21:46:23.432534+0000","last_active":"2024-09-15T21:47:36.794214+0000","last_peered":"2024-09-15T21:47:36.794214+0000","last_clean":"2024-09-15T21:47:36.794214+0000","last_became_active":"2024-09-15T21:46:23.430170+0000","last_became_peered":"2024-09-15T21:46:23.430170+0000","last_unstale":"2024-09-15T21:47:36.794214+0000","last_undegraded":"2024-09-15T21:47:36.794214+0000","last_fullsized":"2024-09-15T21:47:36.794214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:11.463937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":42,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.045554+0000","last_change":"2024-09-15T21:46:23.427841+0000","last_active":"2024-09-15T21:47:34.045554+0000","last_peered":"2024-09-15T21:47:34.045554+0000","last_clean":"2024-09-15T21:47:34.045554+0000","last_became_active":"2024-09-15T21:46:23.427621+0000","last_became_peered":"2024-09-15T21:46:23.427621+0000","last_unstale":"2024-09-15T21:47:34.045554+0000","last_undegraded":"2024-09-15T21:47:34.045554+0000","last_fullsized":"2024-09-15T21:47:34.045554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:18.331943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.104917+0000","last_change":"2024-09-15T21:46:23.424707+0000","last_active":"2024-09-15T21:47:37.104917+0000","last_peered":"2024-09-15T21:47:37.104917+0000","last_clean":"2024-09-15T21:47:37.104917+0000","last_became_active":"2024-09-15T21:46:23.424447+0000","last_became_peered":"2024-09-15T21:46:23.424447+0000","last_unstale":"2024-09-15T21:47:37.104917+0000","last_undegraded":"2024-09-15T21:47:37.104917+0000","last_fullsized":"2024-09-15T21:47:37.104917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:23.617214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.044163+0000","last_change":"2024-09-15T21:46:23.430297+0000","last_active":"2024-09-15T21:47:34.044163+0000","last_peered":"2024-09-15T21:47:34.044163+0000","last_clean":"2024-09-15T21:47:34.044163+0000","last_became_active":"2024-09-15T21:46:23.430070+0000","last_became_peered":"2024-09-15T21:46:23.430070+0000","last_unstale":"2024-09-15T21:47:34.044163+0000","last_undegraded":"2024-09-15T21:47:34.044163+0000","last_fullsized":"2024-09-15T21:47:34.044163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:03:19.615022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":23,"reported_epoch":38,"state":"active+undersized","last_fresh":"2024-09-15T21:47:37.104397+0000","last_change":"2024-09-15T21:47:17.675883+0000","last_active":"2024-09-15T21:47:37.104397+0000","last_peered":"2024-09-15T21:47:37.104397+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"2024-09-15T21:47:17.675883+0000","last_became_peered":"2024-09-15T21:47:17.675883+0000","last_unstale":"2024-09-15T21:47:37.104397+0000","last_undegraded":"2024-09-15T21:47:37.104397+0000","last_fullsized":"2024-09-15T21:47:17.673461+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:10.352399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":23,"reported_epoch":38,"state":"active+undersized","last_fresh":"2024-09-15T21:47:37.104520+0000","last_change":"2024-09-15T21:47:17.682512+0000","last_active":"2024-09-15T21:47:37.104520+0000","last_peered":"2024-09-15T21:47:37.104520+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"2024-09-15T21:47:17.682512+0000","last_became_peered":"2024-09-15T21:47:17.682512+0000","last_unstale":"2024-09-15T21:47:37.104520+0000","last_undegraded":"2024-09-15T21:47:37.104520+0000","last_fullsized":"2024-09-15T21:47:17.680179+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:38.301504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.789263+0000","last_change":"2024-09-15T21:46:23.427570+0000","last_active":"2024-09-15T21:47:36.789263+0000","last_peered":"2024-09-15T21:47:36.789263+0000","last_clean":"2024-09-15T21:47:36.789263+0000","last_became_active":"2024-09-15T21:46:23.427304+0000","last_became_peered":"2024-09-15T21:46:23.427304+0000","last_unstale":"2024-09-15T21:47:36.789263+0000","last_undegraded":"2024-09-15T21:47:36.789263+0000","last_fullsized":"2024-09-15T21:47:36.789263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:37.345106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.104225+0000","last_change":"2024-09-15T21:46:23.431434+0000","last_active":"2024-09-15T21:47:37.104225+0000","last_peered":"2024-09-15T21:47:37.104225+0000","last_clean":"2024-09-15T21:47:37.104225+0000","last_became_active":"2024-09-15T21:46:23.430987+0000","last_became_peered":"2024-09-15T21:46:23.430987+0000","last_unstale":"2024-09-15T21:47:37.104225+0000","last_undegraded":"2024-09-15T21:47:37.104225+0000","last_fullsized":"2024-09-15T21:47:37.104225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:48:30.532378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.793885+0000","last_change":"2024-09-15T21:46:23.434558+0000","last_active":"2024-09-15T21:47:36.793885+0000","last_peered":"2024-09-15T21:47:36.793885+0000","last_clean":"2024-09-15T21:47:36.793885+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:47:36.793885+0000","last_undegraded":"2024-09-15T21:47:36.793885+0000","last_fullsized":"2024-09-15T21:47:36.793885+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:03:39.562977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.105128+0000","last_change":"2024-09-15T21:46:23.428323+0000","last_active":"2024-09-15T21:47:37.105128+0000","last_peered":"2024-09-15T21:47:37.105128+0000","last_clean":"2024-09-15T21:47:37.105128+0000","last_became_active":"2024-09-15T21:46:23.428016+0000","last_became_peered":"2024-09-15T21:46:23.428016+0000","last_unstale":"2024-09-15T21:47:37.105128+0000","last_undegraded":"2024-09-15T21:47:37.105128+0000","last_fullsized":"2024-09-15T21:47:37.105128+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:28:48.678550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":23,"reported_epoch":38,"state":"active+undersized","last_fresh":"2024-09-15T21:47:37.104015+0000","last_change":"2024-09-15T21:47:17.681935+0000","last_active":"2024-09-15T21:47:37.104015+0000","last_peered":"2024-09-15T21:47:37.104015+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"2024-09-15T21:47:17.681935+0000","last_became_peered":"2024-09-15T21:47:17.681935+0000","last_unstale":"2024-09-15T21:47:37.104015+0000","last_undegraded":"2024-09-15T21:47:37.104015+0000","last_fullsized":"2024-09-15T21:47:17.679256+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:13:26.451008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":42,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.043863+0000","last_change":"2024-09-15T21:46:23.431527+0000","last_active":"2024-09-15T21:47:34.043863+0000","last_peered":"2024-09-15T21:47:34.043863+0000","last_clean":"2024-09-15T21:47:34.043863+0000","last_became_active":"2024-09-15T21:46:23.430719+0000","last_became_peered":"2024-09-15T21:46:23.430719+0000","last_unstale":"2024-09-15T21:47:34.043863+0000","last_undegraded":"2024-09-15T21:47:34.043863+0000","last_fullsized":"2024-09-15T21:47:34.043863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:16.326529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":21,"reported_epoch":37,"state":"active+undersized","last_fresh":"2024-09-15T21:47:34.043489+0000","last_change":"2024-09-15T21:47:17.677476+0000","last_active":"2024-09-15T21:47:34.043489+0000","last_peered":"2024-09-15T21:47:34.043489+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"2024-09-15T21:47:17.677476+0000","last_became_peered":"2024-09-15T21:47:17.677476+0000","last_unstale":"2024-09-15T21:47:34.043489+0000","last_undegraded":"2024-09-15T21:47:34.043489+0000","last_fullsized":"2024-09-15T21:47:17.676272+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:53.609830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":42,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.045423+0000","last_change":"2024-09-15T21:46:23.432646+0000","last_active":"2024-09-15T21:47:34.045423+0000","last_peered":"2024-09-15T21:47:34.045423+0000","last_clean":"2024-09-15T21:47:34.045423+0000","last_became_active":"2024-09-15T21:46:23.432317+0000","last_became_peered":"2024-09-15T21:46:23.432317+0000","last_unstale":"2024-09-15T21:47:34.045423+0000","last_undegraded":"2024-09-15T21:47:34.045423+0000","last_fullsized":"2024-09-15T21:47:34.045423+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:49:07.706450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.794268+0000","last_change":"2024-09-15T21:46:23.428183+0000","last_active":"2024-09-15T21:47:36.794268+0000","last_peered":"2024-09-15T21:47:36.794268+0000","last_clean":"2024-09-15T21:47:36.794268+0000","last_became_active":"2024-09-15T21:46:23.427886+0000","last_became_peered":"2024-09-15T21:46:23.427886+0000","last_unstale":"2024-09-15T21:47:36.794268+0000","last_undegraded":"2024-09-15T21:47:36.794268+0000","last_fullsized":"2024-09-15T21:47:36.794268+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:15.584377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.044152+0000","last_change":"2024-09-15T21:46:23.421749+0000","last_active":"2024-09-15T21:47:34.044152+0000","last_peered":"2024-09-15T21:47:34.044152+0000","last_clean":"2024-09-15T21:47:34.044152+0000","last_became_active":"2024-09-15T21:46:23.421501+0000","last_became_peered":"2024-09-15T21:46:23.421501+0000","last_unstale":"2024-09-15T21:47:34.044152+0000","last_undegraded":"2024-09-15T21:47:34.044152+0000","last_fullsized":"2024-09-15T21:47:34.044152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:55.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":42,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.045960+0000","last_change":"2024-09-15T21:46:23.432591+0000","last_active":"2024-09-15T21:47:34.045960+0000","last_peered":"2024-09-15T21:47:34.045960+0000","last_clean":"2024-09-15T21:47:34.045960+0000","last_became_active":"2024-09-15T21:46:23.432326+0000","last_became_peered":"2024-09-15T21:46:23.432326+0000","last_unstale":"2024-09-15T21:47:34.045960+0000","last_undegraded":"2024-09-15T21:47:34.045960+0000","last_fullsized":"2024-09-15T21:47:34.045960+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:48.565579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.105209+0000","last_change":"2024-09-15T21:46:23.423428+0000","last_active":"2024-09-15T21:47:37.105209+0000","last_peered":"2024-09-15T21:47:37.105209+0000","last_clean":"2024-09-15T21:47:37.105209+0000","last_became_active":"2024-09-15T21:46:23.422734+0000","last_became_peered":"2024-09-15T21:46:23.422734+0000","last_unstale":"2024-09-15T21:47:37.105209+0000","last_undegraded":"2024-09-15T21:47:37.105209+0000","last_fullsized":"2024-09-15T21:47:37.105209+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:06.592891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":37,"state":"active+undersized","last_fresh":"2024-09-15T21:47:34.044295+0000","last_change":"2024-09-15T21:47:17.678751+0000","last_active":"2024-09-15T21:47:34.044295+0000","last_peered":"2024-09-15T21:47:34.044295+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"2024-09-15T21:47:17.678751+0000","last_became_peered":"2024-09-15T21:47:17.678751+0000","last_unstale":"2024-09-15T21:47:34.044295+0000","last_undegraded":"2024-09-15T21:47:34.044295+0000","last_fullsized":"2024-09-15T21:47:17.676378+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:09.165010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+undersized","last_fresh":"2024-09-15T21:47:36.789419+0000","last_change":"2024-09-15T21:47:17.678484+0000","last_active":"2024-09-15T21:47:36.789419+0000","last_peered":"2024-09-15T21:47:36.789419+0000","last_clean":"2024-09-15T21:47:15.661737+0000","last_became_active":"2024-09-15T21:47:17.678484+0000","last_became_peered":"2024-09-15T21:47:17.678484+0000","last_unstale":"2024-09-15T21:47:36.789419+0000","last_undegraded":"2024-09-15T21:47:36.789419+0000","last_fullsized":"2024-09-15T21:47:17.676826+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:25.987857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.793622+0000","last_change":"2024-09-15T21:46:23.429648+0000","last_active":"2024-09-15T21:47:36.793622+0000","last_peered":"2024-09-15T21:47:36.793622+0000","last_clean":"2024-09-15T21:47:36.793622+0000","last_became_active":"2024-09-15T21:46:23.429376+0000","last_became_peered":"2024-09-15T21:46:23.429376+0000","last_unstale":"2024-09-15T21:47:36.793622+0000","last_undegraded":"2024-09-15T21:47:36.793622+0000","last_fullsized":"2024-09-15T21:47:36.793622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:31.278283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":44,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.793888+0000","last_change":"2024-09-15T21:46:23.429847+0000","last_active":"2024-09-15T21:47:36.793888+0000","last_peered":"2024-09-15T21:47:36.793888+0000","last_clean":"2024-09-15T21:47:36.793888+0000","last_became_active":"2024-09-15T21:46:23.429442+0000","last_became_peered":"2024-09-15T21:46:23.429442+0000","last_unstale":"2024-09-15T21:47:36.793888+0000","last_undegraded":"2024-09-15T21:47:36.793888+0000","last_fullsized":"2024-09-15T21:47:36.793888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:31.232512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'6","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.045731+0000","last_change":"2024-09-15T21:46:22.429407+0000","last_active":"2024-09-15T21:47:34.045731+0000","last_peered":"2024-09-15T21:47:34.045731+0000","last_clean":"2024-09-15T21:47:34.045731+0000","last_became_active":"2024-09-15T21:46:22.429107+0000","last_became_peered":"2024-09-15T21:46:22.429107+0000","last_unstale":"2024-09-15T21:47:34.045731+0000","last_undegraded":"2024-09-15T21:47:34.045731+0000","last_fullsized":"2024-09-15T21:47:34.045731+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:14:54.680349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.105271+0000","last_change":"2024-09-15T21:46:23.434198+0000","last_active":"2024-09-15T21:47:37.105271+0000","last_peered":"2024-09-15T21:47:37.105271+0000","last_clean":"2024-09-15T21:47:37.105271+0000","last_became_active":"2024-09-15T21:46:23.433944+0000","last_became_peered":"2024-09-15T21:46:23.433944+0000","last_unstale":"2024-09-15T21:47:37.105271+0000","last_undegraded":"2024-09-15T21:47:37.105271+0000","last_fullsized":"2024-09-15T21:47:37.105271+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:25.412167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'7","reported_seq":52,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.793850+0000","last_change":"2024-09-15T21:46:22.425571+0000","last_active":"2024-09-15T21:47:36.793850+0000","last_peered":"2024-09-15T21:47:36.793850+0000","last_clean":"2024-09-15T21:47:36.793850+0000","last_became_active":"2024-09-15T21:46:22.425315+0000","last_became_peered":"2024-09-15T21:46:22.425315+0000","last_unstale":"2024-09-15T21:47:36.793850+0000","last_undegraded":"2024-09-15T21:47:36.793850+0000","last_fullsized":"2024-09-15T21:47:36.793850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:31.983301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'3","reported_seq":47,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.046093+0000","last_change":"2024-09-15T21:46:22.425492+0000","last_active":"2024-09-15T21:47:34.046093+0000","last_peered":"2024-09-15T21:47:34.046093+0000","last_clean":"2024-09-15T21:47:34.046093+0000","last_became_active":"2024-09-15T21:46:22.425226+0000","last_became_peered":"2024-09-15T21:46:22.425226+0000","last_unstale":"2024-09-15T21:47:34.046093+0000","last_undegraded":"2024-09-15T21:47:34.046093+0000","last_fullsized":"2024-09-15T21:47:34.046093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:24.322384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":42,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.043643+0000","last_change":"2024-09-15T21:46:23.429140+0000","last_active":"2024-09-15T21:47:34.043643+0000","last_peered":"2024-09-15T21:47:34.043643+0000","last_clean":"2024-09-15T21:47:34.043643+0000","last_became_active":"2024-09-15T21:46:23.428902+0000","last_became_peered":"2024-09-15T21:46:23.428902+0000","last_unstale":"2024-09-15T21:47:34.043643+0000","last_undegraded":"2024-09-15T21:47:34.043643+0000","last_fullsized":"2024-09-15T21:47:34.043643+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:11.256924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'3","reported_seq":36,"reported_epoch":38,"state":"active+undersized+degraded","last_fresh":"2024-09-15T21:47:36.793676+0000","last_change":"2024-09-15T21:47:17.673824+0000","last_active":"2024-09-15T21:47:36.793676+0000","last_peered":"2024-09-15T21:47:36.793676+0000","last_clean":"2024-09-15T21:46:34.463342+0000","last_became_active":"2024-09-15T21:47:17.673824+0000","last_became_peered":"2024-09-15T21:47:17.673824+0000","last_unstale":"2024-09-15T21:47:36.793676+0000","last_undegraded":"2024-09-15T21:47:17.673256+0000","last_fullsized":"2024-09-15T21:47:17.673148+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:25:13.209081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.044794+0000","last_change":"2024-09-15T21:46:23.427442+0000","last_active":"2024-09-15T21:47:34.044794+0000","last_peered":"2024-09-15T21:47:34.044794+0000","last_clean":"2024-09-15T21:47:34.044794+0000","last_became_active":"2024-09-15T21:46:23.427200+0000","last_became_peered":"2024-09-15T21:46:23.427200+0000","last_unstale":"2024-09-15T21:47:34.044794+0000","last_undegraded":"2024-09-15T21:47:34.044794+0000","last_fullsized":"2024-09-15T21:47:34.044794+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:27.248545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"22'5","reported_seq":52,"reported_epoch":38,"state":"active+undersized+degraded","last_fresh":"2024-09-15T21:47:36.793980+0000","last_change":"2024-09-15T21:47:17.681849+0000","last_active":"2024-09-15T21:47:36.793980+0000","last_peered":"2024-09-15T21:47:36.793980+0000","last_clean":"2024-09-15T21:47:16.541283+0000","last_became_active":"2024-09-15T21:47:17.681849+0000","last_became_peered":"2024-09-15T21:47:17.681849+0000","last_unstale":"2024-09-15T21:47:36.793980+0000","last_undegraded":"2024-09-15T21:47:17.679563+0000","last_fullsized":"2024-09-15T21:47:17.679483+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:49.371756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":3}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":23,"reported_epoch":38,"state":"active+undersized","last_fresh":"2024-09-15T21:47:37.104259+0000","last_change":"2024-09-15T21:47:17.681943+0000","last_active":"2024-09-15T21:47:37.104259+0000","last_peered":"2024-09-15T21:47:37.104259+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"2024-09-15T21:47:17.681943+0000","last_became_peered":"2024-09-15T21:47:17.681943+0000","last_unstale":"2024-09-15T21:47:37.104259+0000","last_undegraded":"2024-09-15T21:47:37.104259+0000","last_fullsized":"2024-09-15T21:47:17.679346+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:04.015745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.16","version":"22'4","reported_seq":37,"reported_epoch":38,"state":"active+undersized+degraded","last_fresh":"2024-09-15T21:47:36.794402+0000","last_change":"2024-09-15T21:47:17.682454+0000","last_active":"2024-09-15T21:47:36.794402+0000","last_peered":"2024-09-15T21:47:36.794402+0000","last_clean":"2024-09-15T21:46:34.464595+0000","last_became_active":"2024-09-15T21:47:17.682454+0000","last_became_peered":"2024-09-15T21:47:17.682454+0000","last_unstale":"2024-09-15T21:47:36.794402+0000","last_undegraded":"2024-09-15T21:47:17.680066+0000","last_fullsized":"2024-09-15T21:47:17.679994+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:26.955885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[{"shards":"6","objects":2}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+undersized","last_fresh":"2024-09-15T21:47:37.106022+0000","last_change":"2024-09-15T21:47:17.677780+0000","last_active":"2024-09-15T21:47:37.106022+0000","last_peered":"2024-09-15T21:47:37.106022+0000","last_clean":"2024-09-15T21:47:16.541160+0000","last_became_active":"2024-09-15T21:47:17.677780+0000","last_became_peered":"2024-09-15T21:47:17.677780+0000","last_unstale":"2024-09-15T21:47:37.106022+0000","last_undegraded":"2024-09-15T21:47:37.106022+0000","last_fullsized":"2024-09-15T21:47:17.673948+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:18:20.378307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'5","reported_seq":50,"reported_epoch":37,"state":"active+undersized+degraded","last_fresh":"2024-09-15T21:47:34.044246+0000","last_change":"2024-09-15T21:47:17.676005+0000","last_active":"2024-09-15T21:47:34.044246+0000","last_peered":"2024-09-15T21:47:34.044246+0000","last_clean":"2024-09-15T21:47:16.415936+0000","last_became_active":"2024-09-15T21:47:17.676005+0000","last_became_peered":"2024-09-15T21:47:17.676005+0000","last_unstale":"2024-09-15T21:47:34.044246+0000","last_undegraded":"2024-09-15T21:47:17.674934+0000","last_fullsized":"2024-09-15T21:47:17.674841+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:38:39.762119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":4}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":44,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.794304+0000","last_change":"2024-09-15T21:46:23.427416+0000","last_active":"2024-09-15T21:47:36.794304+0000","last_peered":"2024-09-15T21:47:36.794304+0000","last_clean":"2024-09-15T21:47:36.794304+0000","last_became_active":"2024-09-15T21:46:23.427057+0000","last_became_peered":"2024-09-15T21:46:23.427057+0000","last_unstale":"2024-09-15T21:47:36.794304+0000","last_undegraded":"2024-09-15T21:47:36.794304+0000","last_fullsized":"2024-09-15T21:47:36.794304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:42.671501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'13","reported_seq":58,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.793937+0000","last_change":"2024-09-15T21:46:22.426151+0000","last_active":"2024-09-15T21:47:36.793937+0000","last_peered":"2024-09-15T21:47:36.793937+0000","last_clean":"2024-09-15T21:47:36.793937+0000","last_became_active":"2024-09-15T21:46:22.425866+0000","last_became_peered":"2024-09-15T21:46:22.425866+0000","last_unstale":"2024-09-15T21:47:36.793937+0000","last_undegraded":"2024-09-15T21:47:36.793937+0000","last_fullsized":"2024-09-15T21:47:36.793937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:10.684265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4760,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.104133+0000","last_change":"2024-09-15T21:46:23.432292+0000","last_active":"2024-09-15T21:47:37.104133+0000","last_peered":"2024-09-15T21:47:37.104133+0000","last_clean":"2024-09-15T21:47:37.104133+0000","last_became_active":"2024-09-15T21:46:23.431877+0000","last_became_peered":"2024-09-15T21:46:23.431877+0000","last_unstale":"2024-09-15T21:47:37.104133+0000","last_undegraded":"2024-09-15T21:47:37.104133+0000","last_fullsized":"2024-09-15T21:47:37.104133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:16.857279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'9","reported_seq":54,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.105319+0000","last_change":"2024-09-15T21:46:22.427415+0000","last_active":"2024-09-15T21:47:37.105319+0000","last_peered":"2024-09-15T21:47:37.105319+0000","last_clean":"2024-09-15T21:47:37.105319+0000","last_became_active":"2024-09-15T21:46:22.427087+0000","last_became_peered":"2024-09-15T21:46:22.427087+0000","last_unstale":"2024-09-15T21:47:37.105319+0000","last_undegraded":"2024-09-15T21:47:37.105319+0000","last_fullsized":"2024-09-15T21:47:37.105319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:09.365202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.105041+0000","last_change":"2024-09-15T21:46:23.427238+0000","last_active":"2024-09-15T21:47:37.105041+0000","last_peered":"2024-09-15T21:47:37.105041+0000","last_clean":"2024-09-15T21:47:37.105041+0000","last_became_active":"2024-09-15T21:46:23.426993+0000","last_became_peered":"2024-09-15T21:46:23.426993+0000","last_unstale":"2024-09-15T21:47:37.105041+0000","last_undegraded":"2024-09-15T21:47:37.105041+0000","last_fullsized":"2024-09-15T21:47:37.105041+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:52:24.175554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"22'5","reported_seq":37,"reported_epoch":37,"state":"active+undersized+degraded","last_fresh":"2024-09-15T21:47:34.044455+0000","last_change":"2024-09-15T21:47:17.679190+0000","last_active":"2024-09-15T21:47:34.044455+0000","last_peered":"2024-09-15T21:47:34.044455+0000","last_clean":"2024-09-15T21:46:45.550646+0000","last_became_active":"2024-09-15T21:47:17.679190+0000","last_became_peered":"2024-09-15T21:47:17.679190+0000","last_unstale":"2024-09-15T21:47:34.044455+0000","last_undegraded":"2024-09-15T21:47:17.676807+0000","last_fullsized":"2024-09-15T21:47:17.676732+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:12:11.466080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":3}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.104348+0000","last_change":"2024-09-15T21:46:23.433924+0000","last_active":"2024-09-15T21:47:37.104348+0000","last_peered":"2024-09-15T21:47:37.104348+0000","last_clean":"2024-09-15T21:47:37.104348+0000","last_became_active":"2024-09-15T21:46:23.433606+0000","last_became_peered":"2024-09-15T21:46:23.433606+0000","last_unstale":"2024-09-15T21:47:37.104348+0000","last_undegraded":"2024-09-15T21:47:37.104348+0000","last_fullsized":"2024-09-15T21:47:37.104348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:09:35.744149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'5","reported_seq":50,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.793716+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:47:36.793716+0000","last_peered":"2024-09-15T21:47:36.793716+0000","last_clean":"2024-09-15T21:47:36.793716+0000","last_became_active":"2024-09-15T21:46:22.425237+0000","last_became_peered":"2024-09-15T21:46:22.425237+0000","last_unstale":"2024-09-15T21:47:36.793716+0000","last_undegraded":"2024-09-15T21:47:36.793716+0000","last_fullsized":"2024-09-15T21:47:36.793716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:59.460420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.793627+0000","last_change":"2024-09-15T21:46:23.427515+0000","last_active":"2024-09-15T21:47:36.793627+0000","last_peered":"2024-09-15T21:47:36.793627+0000","last_clean":"2024-09-15T21:47:36.793627+0000","last_became_active":"2024-09-15T21:46:23.427116+0000","last_became_peered":"2024-09-15T21:46:23.427116+0000","last_unstale":"2024-09-15T21:47:36.793627+0000","last_undegraded":"2024-09-15T21:47:36.793627+0000","last_fullsized":"2024-09-15T21:47:36.793627+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:35.499970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'17","reported_seq":62,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.788498+0000","last_change":"2024-09-15T21:46:22.419905+0000","last_active":"2024-09-15T21:47:36.788498+0000","last_peered":"2024-09-15T21:47:36.788498+0000","last_clean":"2024-09-15T21:47:36.788498+0000","last_became_active":"2024-09-15T21:46:22.419620+0000","last_became_peered":"2024-09-15T21:46:22.419620+0000","last_unstale":"2024-09-15T21:47:36.788498+0000","last_undegraded":"2024-09-15T21:47:36.788498+0000","last_fullsized":"2024-09-15T21:47:36.788498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:26:51.215156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.104015+0000","last_change":"2024-09-15T21:46:23.432975+0000","last_active":"2024-09-15T21:47:37.104015+0000","last_peered":"2024-09-15T21:47:37.104015+0000","last_clean":"2024-09-15T21:47:37.104015+0000","last_became_active":"2024-09-15T21:46:23.432718+0000","last_became_peered":"2024-09-15T21:46:23.432718+0000","last_unstale":"2024-09-15T21:47:37.104015+0000","last_undegraded":"2024-09-15T21:47:37.104015+0000","last_fullsized":"2024-09-15T21:47:37.104015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:04:59.789691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'2","reported_seq":48,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.794098+0000","last_change":"2024-09-15T21:46:22.429661+0000","last_active":"2024-09-15T21:47:36.794098+0000","last_peered":"2024-09-15T21:47:36.794098+0000","last_clean":"2024-09-15T21:47:36.794098+0000","last_became_active":"2024-09-15T21:46:22.429250+0000","last_became_peered":"2024-09-15T21:46:22.429250+0000","last_unstale":"2024-09-15T21:47:36.794098+0000","last_undegraded":"2024-09-15T21:47:36.794098+0000","last_fullsized":"2024-09-15T21:47:36.794098+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:29.341739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.104345+0000","last_change":"2024-09-15T21:46:23.435003+0000","last_active":"2024-09-15T21:47:37.104345+0000","last_peered":"2024-09-15T21:47:37.104345+0000","last_clean":"2024-09-15T21:47:37.104345+0000","last_became_active":"2024-09-15T21:46:23.434738+0000","last_became_peered":"2024-09-15T21:46:23.434738+0000","last_unstale":"2024-09-15T21:47:37.104345+0000","last_undegraded":"2024-09-15T21:47:37.104345+0000","last_fullsized":"2024-09-15T21:47:37.104345+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:53:15.814759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'5","reported_seq":50,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.789379+0000","last_change":"2024-09-15T21:46:22.425870+0000","last_active":"2024-09-15T21:47:36.789379+0000","last_peered":"2024-09-15T21:47:36.789379+0000","last_clean":"2024-09-15T21:47:36.789379+0000","last_became_active":"2024-09-15T21:46:22.425588+0000","last_became_peered":"2024-09-15T21:46:22.425588+0000","last_unstale":"2024-09-15T21:47:36.789379+0000","last_undegraded":"2024-09-15T21:47:36.789379+0000","last_fullsized":"2024-09-15T21:47:36.789379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:40:07.586608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.104387+0000","last_change":"2024-09-15T21:46:23.434657+0000","last_active":"2024-09-15T21:47:37.104387+0000","last_peered":"2024-09-15T21:47:37.104387+0000","last_clean":"2024-09-15T21:47:37.104387+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:47:37.104387+0000","last_undegraded":"2024-09-15T21:47:37.104387+0000","last_fullsized":"2024-09-15T21:47:37.104387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:02.545810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'3","reported_seq":46,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.045275+0000","last_change":"2024-09-15T21:46:22.425072+0000","last_active":"2024-09-15T21:47:34.045275+0000","last_peered":"2024-09-15T21:47:34.045275+0000","last_clean":"2024-09-15T21:47:34.045275+0000","last_became_active":"2024-09-15T21:46:22.424771+0000","last_became_peered":"2024-09-15T21:46:22.424771+0000","last_unstale":"2024-09-15T21:47:34.045275+0000","last_undegraded":"2024-09-15T21:47:34.045275+0000","last_fullsized":"2024-09-15T21:47:34.045275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:26.256785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":21,"reported_epoch":37,"state":"active+undersized","last_fresh":"2024-09-15T21:47:34.045818+0000","last_change":"2024-09-15T21:47:17.678705+0000","last_active":"2024-09-15T21:47:34.045818+0000","last_peered":"2024-09-15T21:47:34.045818+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"2024-09-15T21:47:17.678705+0000","last_became_peered":"2024-09-15T21:47:17.678705+0000","last_unstale":"2024-09-15T21:47:34.045818+0000","last_undegraded":"2024-09-15T21:47:34.045818+0000","last_fullsized":"2024-09-15T21:47:17.677159+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:02:21.969349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"22'5","reported_seq":52,"reported_epoch":38,"state":"active+undersized+degraded","last_fresh":"2024-09-15T21:47:36.788690+0000","last_change":"2024-09-15T21:47:17.679354+0000","last_active":"2024-09-15T21:47:36.788690+0000","last_peered":"2024-09-15T21:47:36.788690+0000","last_clean":"2024-09-15T21:47:15.661933+0000","last_became_active":"2024-09-15T21:47:17.679354+0000","last_became_peered":"2024-09-15T21:47:17.679354+0000","last_unstale":"2024-09-15T21:47:36.788690+0000","last_undegraded":"2024-09-15T21:47:17.677846+0000","last_fullsized":"2024-09-15T21:47:17.677773+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:46:21.127596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":3}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":38,"state":"active+undersized","last_fresh":"2024-09-15T21:47:36.788894+0000","last_change":"2024-09-15T21:47:17.679318+0000","last_active":"2024-09-15T21:47:36.788894+0000","last_peered":"2024-09-15T21:47:36.788894+0000","last_clean":"2024-09-15T21:47:15.661877+0000","last_became_active":"2024-09-15T21:47:17.679318+0000","last_became_peered":"2024-09-15T21:47:17.679318+0000","last_unstale":"2024-09-15T21:47:36.788894+0000","last_undegraded":"2024-09-15T21:47:36.788894+0000","last_fullsized":"2024-09-15T21:47:17.678041+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:24.362117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":202,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.103992+0000","last_change":"2024-09-15T21:46:14.328094+0000","last_active":"2024-09-15T21:47:37.103992+0000","last_peered":"2024-09-15T21:47:37.103992+0000","last_clean":"2024-09-15T21:47:37.103992+0000","last_became_active":"2024-09-15T21:46:14.326991+0000","last_became_peered":"2024-09-15T21:46:14.326991+0000","last_unstale":"2024-09-15T21:47:37.103992+0000","last_undegraded":"2024-09-15T21:47:37.103992+0000","last_fullsized":"2024-09-15T21:47:37.103992+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_clean_scrub_stamp":"2024-09-15T21:46:12.223400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:54:27.704130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'6","reported_seq":40,"reported_epoch":38,"state":"active+undersized+degraded","last_fresh":"2024-09-15T21:47:36.788423+0000","last_change":"2024-09-15T21:47:17.678951+0000","last_active":"2024-09-15T21:47:36.788423+0000","last_peered":"2024-09-15T21:47:36.788423+0000","last_clean":"2024-09-15T21:46:45.550469+0000","last_became_active":"2024-09-15T21:47:17.678951+0000","last_became_peered":"2024-09-15T21:47:17.678951+0000","last_unstale":"2024-09-15T21:47:36.788423+0000","last_undegraded":"2024-09-15T21:47:17.677497+0000","last_fullsized":"2024-09-15T21:47:17.677422+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:02:32.923057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":5}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":44,"reported_epoch":37,"state":"active+undersized","last_fresh":"2024-09-15T21:47:34.044625+0000","last_change":"2024-09-15T21:47:17.679056+0000","last_active":"2024-09-15T21:47:34.044625+0000","last_peered":"2024-09-15T21:47:34.044625+0000","last_clean":"2024-09-15T21:47:16.378582+0000","last_became_active":"2024-09-15T21:47:17.679056+0000","last_became_peered":"2024-09-15T21:47:17.679056+0000","last_unstale":"2024-09-15T21:47:34.044625+0000","last_undegraded":"2024-09-15T21:47:34.044625+0000","last_fullsized":"2024-09-15T21:47:17.676897+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:17.148235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'4","reported_seq":49,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.103987+0000","last_change":"2024-09-15T21:46:22.431113+0000","last_active":"2024-09-15T21:47:37.103987+0000","last_peered":"2024-09-15T21:47:37.103987+0000","last_clean":"2024-09-15T21:47:37.103987+0000","last_became_active":"2024-09-15T21:46:22.430832+0000","last_became_peered":"2024-09-15T21:46:22.430832+0000","last_unstale":"2024-09-15T21:47:37.103987+0000","last_undegraded":"2024-09-15T21:47:37.103987+0000","last_fullsized":"2024-09-15T21:47:37.103987+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:28.849259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.105089+0000","last_change":"2024-09-15T21:46:23.430396+0000","last_active":"2024-09-15T21:47:37.105089+0000","last_peered":"2024-09-15T21:47:37.105089+0000","last_clean":"2024-09-15T21:47:37.105089+0000","last_became_active":"2024-09-15T21:46:23.430138+0000","last_became_peered":"2024-09-15T21:46:23.430138+0000","last_unstale":"2024-09-15T21:47:37.105089+0000","last_undegraded":"2024-09-15T21:47:37.105089+0000","last_fullsized":"2024-09-15T21:47:37.105089+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:50.622949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"22'5","reported_seq":50,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.789627+0000","last_change":"2024-09-15T21:46:22.425808+0000","last_active":"2024-09-15T21:47:36.789627+0000","last_peered":"2024-09-15T21:47:36.789627+0000","last_clean":"2024-09-15T21:47:36.789627+0000","last_became_active":"2024-09-15T21:46:22.425510+0000","last_became_peered":"2024-09-15T21:46:22.425510+0000","last_unstale":"2024-09-15T21:47:36.789627+0000","last_undegraded":"2024-09-15T21:47:36.789627+0000","last_fullsized":"2024-09-15T21:47:36.789627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:11:00.338076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.044665+0000","last_change":"2024-09-15T21:46:23.427051+0000","last_active":"2024-09-15T21:47:34.044665+0000","last_peered":"2024-09-15T21:47:34.044665+0000","last_clean":"2024-09-15T21:47:34.044665+0000","last_became_active":"2024-09-15T21:46:23.426185+0000","last_became_peered":"2024-09-15T21:46:23.426185+0000","last_unstale":"2024-09-15T21:47:34.044665+0000","last_undegraded":"2024-09-15T21:47:34.044665+0000","last_fullsized":"2024-09-15T21:47:34.044665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:48.318405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'5","reported_seq":37,"reported_epoch":37,"state":"active+undersized+degraded","last_fresh":"2024-09-15T21:47:34.044298+0000","last_change":"2024-09-15T21:47:17.675398+0000","last_active":"2024-09-15T21:47:34.044298+0000","last_peered":"2024-09-15T21:47:34.044298+0000","last_clean":"2024-09-15T21:46:51.572233+0000","last_became_active":"2024-09-15T21:47:17.675398+0000","last_became_peered":"2024-09-15T21:47:17.675398+0000","last_unstale":"2024-09-15T21:47:34.044298+0000","last_undegraded":"2024-09-15T21:47:17.674545+0000","last_fullsized":"2024-09-15T21:47:17.674470+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:11:48.088954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":44,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.794175+0000","last_change":"2024-09-15T21:46:23.425699+0000","last_active":"2024-09-15T21:47:36.794175+0000","last_peered":"2024-09-15T21:47:36.794175+0000","last_clean":"2024-09-15T21:47:36.794175+0000","last_became_active":"2024-09-15T21:46:23.425416+0000","last_became_peered":"2024-09-15T21:46:23.425416+0000","last_unstale":"2024-09-15T21:47:36.794175+0000","last_undegraded":"2024-09-15T21:47:36.794175+0000","last_fullsized":"2024-09-15T21:47:36.794175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:00.325179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"22'1","reported_seq":44,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.044699+0000","last_change":"2024-09-15T21:46:22.426492+0000","last_active":"2024-09-15T21:47:34.044699+0000","last_peered":"2024-09-15T21:47:34.044699+0000","last_clean":"2024-09-15T21:47:34.044699+0000","last_became_active":"2024-09-15T21:46:22.426210+0000","last_became_peered":"2024-09-15T21:46:22.426210+0000","last_unstale":"2024-09-15T21:47:34.044699+0000","last_undegraded":"2024-09-15T21:47:34.044699+0000","last_fullsized":"2024-09-15T21:47:34.044699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:05.248091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":37,"state":"active+undersized","last_fresh":"2024-09-15T21:47:34.044960+0000","last_change":"2024-09-15T21:47:17.678499+0000","last_active":"2024-09-15T21:47:34.044960+0000","last_peered":"2024-09-15T21:47:34.044960+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"2024-09-15T21:47:17.678499+0000","last_became_peered":"2024-09-15T21:47:17.678499+0000","last_unstale":"2024-09-15T21:47:34.044960+0000","last_undegraded":"2024-09-15T21:47:34.044960+0000","last_fullsized":"2024-09-15T21:47:17.677213+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:44.377378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"22'4","reported_seq":49,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.104317+0000","last_change":"2024-09-15T21:46:22.430710+0000","last_active":"2024-09-15T21:47:37.104317+0000","last_peered":"2024-09-15T21:47:37.104317+0000","last_clean":"2024-09-15T21:47:37.104317+0000","last_became_active":"2024-09-15T21:46:22.430375+0000","last_became_peered":"2024-09-15T21:46:22.430375+0000","last_unstale":"2024-09-15T21:47:37.104317+0000","last_undegraded":"2024-09-15T21:47:37.104317+0000","last_fullsized":"2024-09-15T21:47:37.104317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:04.081179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":41,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.044607+0000","last_change":"2024-09-15T21:46:23.426729+0000","last_active":"2024-09-15T21:47:34.044607+0000","last_peered":"2024-09-15T21:47:34.044607+0000","last_clean":"2024-09-15T21:47:34.044607+0000","last_became_active":"2024-09-15T21:46:23.426495+0000","last_became_peered":"2024-09-15T21:46:23.426495+0000","last_unstale":"2024-09-15T21:47:34.044607+0000","last_undegraded":"2024-09-15T21:47:34.044607+0000","last_fullsized":"2024-09-15T21:47:34.044607+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:03.277997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'8","reported_seq":51,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.045233+0000","last_change":"2024-09-15T21:46:22.427137+0000","last_active":"2024-09-15T21:47:34.045233+0000","last_peered":"2024-09-15T21:47:34.045233+0000","last_clean":"2024-09-15T21:47:34.045233+0000","last_became_active":"2024-09-15T21:46:22.426855+0000","last_became_peered":"2024-09-15T21:46:22.426855+0000","last_unstale":"2024-09-15T21:47:34.045233+0000","last_undegraded":"2024-09-15T21:47:34.045233+0000","last_fullsized":"2024-09-15T21:47:34.045233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:40:14.277940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":42,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.045504+0000","last_change":"2024-09-15T21:46:23.427327+0000","last_active":"2024-09-15T21:47:34.045504+0000","last_peered":"2024-09-15T21:47:34.045504+0000","last_clean":"2024-09-15T21:47:34.045504+0000","last_became_active":"2024-09-15T21:46:23.427096+0000","last_became_peered":"2024-09-15T21:46:23.427096+0000","last_unstale":"2024-09-15T21:47:34.045504+0000","last_undegraded":"2024-09-15T21:47:34.045504+0000","last_fullsized":"2024-09-15T21:47:34.045504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:04:32.370390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'7","reported_seq":52,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.794096+0000","last_change":"2024-09-15T21:46:22.427798+0000","last_active":"2024-09-15T21:47:36.794096+0000","last_peered":"2024-09-15T21:47:36.794096+0000","last_clean":"2024-09-15T21:47:36.794096+0000","last_became_active":"2024-09-15T21:46:22.427537+0000","last_became_peered":"2024-09-15T21:46:22.427537+0000","last_unstale":"2024-09-15T21:47:36.794096+0000","last_undegraded":"2024-09-15T21:47:36.794096+0000","last_fullsized":"2024-09-15T21:47:36.794096+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:18:54.963834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.105725+0000","last_change":"2024-09-15T21:46:23.424266+0000","last_active":"2024-09-15T21:47:37.105725+0000","last_peered":"2024-09-15T21:47:37.105725+0000","last_clean":"2024-09-15T21:47:37.105725+0000","last_became_active":"2024-09-15T21:46:23.424007+0000","last_became_peered":"2024-09-15T21:46:23.424007+0000","last_unstale":"2024-09-15T21:47:37.105725+0000","last_undegraded":"2024-09-15T21:47:37.105725+0000","last_fullsized":"2024-09-15T21:47:37.105725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:23.492253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'2","reported_seq":56,"reported_epoch":38,"state":"active+undersized+degraded","last_fresh":"2024-09-15T21:47:37.104558+0000","last_change":"2024-09-15T21:47:17.677970+0000","last_active":"2024-09-15T21:47:37.104558+0000","last_peered":"2024-09-15T21:47:37.104558+0000","last_clean":"2024-09-15T21:47:16.541058+0000","last_became_active":"2024-09-15T21:47:17.677970+0000","last_became_peered":"2024-09-15T21:47:17.677970+0000","last_unstale":"2024-09-15T21:47:37.104558+0000","last_undegraded":"2024-09-15T21:47:17.673589+0000","last_fullsized":"2024-09-15T21:47:17.673508+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:41:01.651662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":44,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.793967+0000","last_change":"2024-09-15T21:46:23.430836+0000","last_active":"2024-09-15T21:47:36.793967+0000","last_peered":"2024-09-15T21:47:36.793967+0000","last_clean":"2024-09-15T21:47:36.793967+0000","last_became_active":"2024-09-15T21:46:23.429283+0000","last_became_peered":"2024-09-15T21:46:23.429283+0000","last_unstale":"2024-09-15T21:47:36.793967+0000","last_undegraded":"2024-09-15T21:47:36.793967+0000","last_fullsized":"2024-09-15T21:47:36.793967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:30.571376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'7","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.045215+0000","last_change":"2024-09-15T21:46:22.423795+0000","last_active":"2024-09-15T21:47:34.045215+0000","last_peered":"2024-09-15T21:47:34.045215+0000","last_clean":"2024-09-15T21:47:34.045215+0000","last_became_active":"2024-09-15T21:46:22.422327+0000","last_became_peered":"2024-09-15T21:46:22.422327+0000","last_unstale":"2024-09-15T21:47:34.045215+0000","last_undegraded":"2024-09-15T21:47:34.045215+0000","last_fullsized":"2024-09-15T21:47:34.045215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:16.670599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":42,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.043734+0000","last_change":"2024-09-15T21:46:23.428634+0000","last_active":"2024-09-15T21:47:34.043734+0000","last_peered":"2024-09-15T21:47:34.043734+0000","last_clean":"2024-09-15T21:47:34.043734+0000","last_became_active":"2024-09-15T21:46:23.428301+0000","last_became_peered":"2024-09-15T21:46:23.428301+0000","last_unstale":"2024-09-15T21:47:34.043734+0000","last_undegraded":"2024-09-15T21:47:34.043734+0000","last_fullsized":"2024-09-15T21:47:34.043734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:59.492942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'4","reported_seq":47,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.044480+0000","last_change":"2024-09-15T21:46:22.425810+0000","last_active":"2024-09-15T21:47:34.044480+0000","last_peered":"2024-09-15T21:47:34.044480+0000","last_clean":"2024-09-15T21:47:34.044480+0000","last_became_active":"2024-09-15T21:46:22.425478+0000","last_became_peered":"2024-09-15T21:46:22.425478+0000","last_unstale":"2024-09-15T21:47:34.044480+0000","last_undegraded":"2024-09-15T21:47:34.044480+0000","last_fullsized":"2024-09-15T21:47:34.044480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:53.296599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":42,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.045642+0000","last_change":"2024-09-15T21:46:23.426054+0000","last_active":"2024-09-15T21:47:34.045642+0000","last_peered":"2024-09-15T21:47:34.045642+0000","last_clean":"2024-09-15T21:47:34.045642+0000","last_became_active":"2024-09-15T21:46:23.425547+0000","last_became_peered":"2024-09-15T21:46:23.425547+0000","last_unstale":"2024-09-15T21:47:34.045642+0000","last_undegraded":"2024-09-15T21:47:34.045642+0000","last_fullsized":"2024-09-15T21:47:34.045642+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:56.838503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'4","reported_seq":49,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.104387+0000","last_change":"2024-09-15T21:46:22.428524+0000","last_active":"2024-09-15T21:47:37.104387+0000","last_peered":"2024-09-15T21:47:37.104387+0000","last_clean":"2024-09-15T21:47:37.104387+0000","last_became_active":"2024-09-15T21:46:22.428043+0000","last_became_peered":"2024-09-15T21:46:22.428043+0000","last_unstale":"2024-09-15T21:47:37.104387+0000","last_undegraded":"2024-09-15T21:47:37.104387+0000","last_fullsized":"2024-09-15T21:47:37.104387+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:07:08.393381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":43,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.793658+0000","last_change":"2024-09-15T21:46:23.434719+0000","last_active":"2024-09-15T21:47:36.793658+0000","last_peered":"2024-09-15T21:47:36.793658+0000","last_clean":"2024-09-15T21:47:36.793658+0000","last_became_active":"2024-09-15T21:46:23.434448+0000","last_became_peered":"2024-09-15T21:46:23.434448+0000","last_unstale":"2024-09-15T21:47:36.793658+0000","last_undegraded":"2024-09-15T21:47:36.793658+0000","last_fullsized":"2024-09-15T21:47:36.793658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:14.901671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":36,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:36.789048+0000","last_change":"2024-09-15T21:47:05.619504+0000","last_active":"2024-09-15T21:47:36.789048+0000","last_peered":"2024-09-15T21:47:36.789048+0000","last_clean":"2024-09-15T21:47:36.789048+0000","last_became_active":"2024-09-15T21:47:05.619253+0000","last_became_peered":"2024-09-15T21:47:05.619253+0000","last_unstale":"2024-09-15T21:47:36.789048+0000","last_undegraded":"2024-09-15T21:47:36.789048+0000","last_fullsized":"2024-09-15T21:47:36.789048+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:07:28.529814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'5","reported_seq":52,"reported_epoch":38,"state":"active+undersized+degraded","last_fresh":"2024-09-15T21:47:37.104206+0000","last_change":"2024-09-15T21:47:17.682266+0000","last_active":"2024-09-15T21:47:37.104206+0000","last_peered":"2024-09-15T21:47:37.104206+0000","last_clean":"2024-09-15T21:47:16.541475+0000","last_became_active":"2024-09-15T21:47:17.682266+0000","last_became_peered":"2024-09-15T21:47:17.682266+0000","last_unstale":"2024-09-15T21:47:37.104206+0000","last_undegraded":"2024-09-15T21:47:17.679583+0000","last_fullsized":"2024-09-15T21:47:17.679311+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:23.686208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":3}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":23,"reported_epoch":38,"state":"active+undersized","last_fresh":"2024-09-15T21:47:36.794042+0000","last_change":"2024-09-15T21:47:17.676515+0000","last_active":"2024-09-15T21:47:36.794042+0000","last_peered":"2024-09-15T21:47:36.794042+0000","last_clean":"2024-09-15T21:46:22.402110+0000","last_became_active":"2024-09-15T21:47:17.676515+0000","last_became_peered":"2024-09-15T21:47:17.676515+0000","last_unstale":"2024-09-15T21:47:36.794042+0000","last_undegraded":"2024-09-15T21:47:36.794042+0000","last_fullsized":"2024-09-15T21:47:17.673786+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:36:20.036399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"22'12","reported_seq":57,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.104180+0000","last_change":"2024-09-15T21:46:22.429870+0000","last_active":"2024-09-15T21:47:37.104180+0000","last_peered":"2024-09-15T21:47:37.104180+0000","last_clean":"2024-09-15T21:47:37.104180+0000","last_became_active":"2024-09-15T21:46:22.429578+0000","last_became_peered":"2024-09-15T21:46:22.429578+0000","last_unstale":"2024-09-15T21:47:37.104180+0000","last_undegraded":"2024-09-15T21:47:37.104180+0000","last_fullsized":"2024-09-15T21:47:37.104180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:41.678078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":45,"reported_epoch":38,"state":"active+undersized","last_fresh":"2024-09-15T21:47:37.105869+0000","last_change":"2024-09-15T21:47:17.677558+0000","last_active":"2024-09-15T21:47:37.105869+0000","last_peered":"2024-09-15T21:47:37.105869+0000","last_clean":"2024-09-15T21:47:16.541128+0000","last_became_active":"2024-09-15T21:47:17.677558+0000","last_became_peered":"2024-09-15T21:47:17.677558+0000","last_unstale":"2024-09-15T21:47:37.105869+0000","last_undegraded":"2024-09-15T21:47:37.105869+0000","last_fullsized":"2024-09-15T21:47:17.673699+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:55:04.587412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'2","reported_seq":46,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.045327+0000","last_change":"2024-09-15T21:46:22.428529+0000","last_active":"2024-09-15T21:47:34.045327+0000","last_peered":"2024-09-15T21:47:34.045327+0000","last_clean":"2024-09-15T21:47:34.045327+0000","last_became_active":"2024-09-15T21:46:22.428263+0000","last_became_peered":"2024-09-15T21:46:22.428263+0000","last_unstale":"2024-09-15T21:47:34.045327+0000","last_undegraded":"2024-09-15T21:47:34.045327+0000","last_fullsized":"2024-09-15T21:47:34.045327+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:22:40.160062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":42,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-15T21:47:34.045244+0000","last_change":"2024-09-15T21:46:23.427968+0000","last_active":"2024-09-15T21:47:34.045244+0000","last_peered":"2024-09-15T21:47:34.045244+0000","last_clean":"2024-09-15T21:47:34.045244+0000","last_became_active":"2024-09-15T21:46:23.427546+0000","last_became_peered":"2024-09-15T21:46:23.427546+0000","last_unstale":"2024-09-15T21:47:34.045244+0000","last_undegraded":"2024-09-15T21:47:34.045244+0000","last_fullsized":"2024-09-15T21:47:34.045244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:47.226097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'13","reported_seq":58,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-09-15T21:47:37.106167+0000","last_change":"2024-09-15T21:46:22.428258+0000","last_active":"2024-09-15T21:47:37.106167+0000","last_peered":"2024-09-15T21:47:37.106167+0000","last_clean":"2024-09-15T21:47:37.106167+0000","last_became_active":"2024-09-15T21:46:22.427954+0000","last_became_peered":"2024-09-15T21:46:22.427954+0000","last_unstale":"2024-09-15T21:47:37.106167+0000","last_undegraded":"2024-09-15T21:47:37.106167+0000","last_fullsized":"2024-09-15T21:47:37.106167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:22.440456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":109,"acting":109,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":11438,"num_objects":98,"num_object_clones":0,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":214,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":22876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":186,"ondisk_log_size":186,"up":54,"acting":54,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476756,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1040384,"data_stored":765811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509460,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28052,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343788,"statfs":{"total":96636764160,"available":96608038912,"internally_reserved":0,"allocated":1048576,"data_stored":767293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509462,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27464,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344376,"statfs":{"total":96636764160,"available":96608641024,"internally_reserved":0,"allocated":446464,"data_stored":180119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509454,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":324,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":331776,"data_stored":111184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509461,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":175899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509460,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":450560,"data_stored":176031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542164,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":434176,"data_stored":170047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31616,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340224,"statfs":{"total":96636764160,"available":96604389376,"internally_reserved":0,"allocated":438272,"data_stored":174355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":5550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:47:41.090 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg force-recovery 2.16 2.15 2.12 2.c 2.8 2.1c 2024-09-15T21:47:41.109 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:41.109 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:41.121 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:47:41.164 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:41.163+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:41.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.247+0000 7f9362186640 1 Processor -- start 2024-09-15T21:47:41.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.247+0000 7f9362186640 1 -- start start 2024-09-15T21:47:41.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.249+0000 7f9362186640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f935c15b370 0x7f935c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:41.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.249+0000 7f9362186640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f935c06db40 0x7f935c15b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:41.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.249+0000 7f9362186640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f935c0c9cf0 con 0x7f935c0c46e0 2024-09-15T21:47:41.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.249+0000 7f9362186640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f935c0c9200 con 0x7f935c06db40 2024-09-15T21:47:41.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.249+0000 7f935affd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f935c06db40 0x7f935c15b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:41.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.249+0000 7f9362186640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f935c0c60f0 con 0x7f935c15b370 2024-09-15T21:47:41.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.249+0000 7f935affd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f935c06db40 0x7f935c15b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33744/0 (socket says 172.21.15.100:33744) 2024-09-15T21:47:41.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.249+0000 7f935affd640 1 -- 172.21.15.100:0/35936441 learned_addr learned my addr 172.21.15.100:0/35936441 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:41.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.249+0000 7f935b7fe640 1 --2- 172.21.15.100:0/35936441 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f935c15b370 0x7f935c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:41.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.249+0000 7f935affd640 1 -- 172.21.15.100:0/35936441 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f935c15b370 msgr2=0x7f935c06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:41.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.249+0000 7f935affd640 1 --2- 172.21.15.100:0/35936441 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f935c15b370 0x7f935c06d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:41.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.249+0000 7f935affd640 1 -- 172.21.15.100:0/35936441 >> v1:172.21.15.122:6789/0 conn(0x7f935c0c46e0 legacy=0x7f935c15bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:41.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.249+0000 7f935a7fc640 1 -- 172.21.15.100:0/35936441 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 659767570 0 0) 0x7f935c0c9cf0 con 0x7f935c0c46e0 2024-09-15T21:47:41.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.249+0000 7f935affd640 1 -- 172.21.15.100:0/35936441 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935c0c9730 con 0x7f935c06db40 2024-09-15T21:47:41.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.251+0000 7f935affd640 1 --2- 172.21.15.100:0/35936441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935c06db40 0x7f935c15b750 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9350009870 tx=0x7f935002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9fc5878643a084c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:41.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.251+0000 7f935a7fc640 1 -- 172.21.15.100:0/35936441 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350037070 con 0x7f935c06db40 2024-09-15T21:47:41.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.251+0000 7f935a7fc640 1 -- 172.21.15.100:0/35936441 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9350032440 con 0x7f935c06db40 2024-09-15T21:47:41.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.251+0000 7f935a7fc640 1 -- 172.21.15.100:0/35936441 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9350032810 con 0x7f935c06db40 2024-09-15T21:47:41.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.252+0000 7f9362186640 1 -- 172.21.15.100:0/35936441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935c06db40 msgr2=0x7f935c15b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:41.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.252+0000 7f9362186640 1 --2- 172.21.15.100:0/35936441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935c06db40 0x7f935c15b750 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9350009870 tx=0x7f935002a140 comp rx=0 tx=0).stop 2024-09-15T21:47:41.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.252+0000 7f9362186640 1 -- 172.21.15.100:0/35936441 shutdown_connections 2024-09-15T21:47:41.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.252+0000 7f9362186640 1 --2- 172.21.15.100:0/35936441 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935c06db40 0x7f935c15b750 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:41.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.252+0000 7f9362186640 1 --2- 172.21.15.100:0/35936441 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f935c15b370 0x7f935c06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:41.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.252+0000 7f9362186640 1 -- 172.21.15.100:0/35936441 >> 172.21.15.100:0/35936441 conn(0x7f935c0ca390 msgr2=0x7f935c0ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:41.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.252+0000 7f9362186640 1 -- 172.21.15.100:0/35936441 shutdown_connections 2024-09-15T21:47:41.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.252+0000 7f9362186640 1 -- 172.21.15.100:0/35936441 wait complete. 2024-09-15T21:47:41.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.254+0000 7f9362186640 1 Processor -- start 2024-09-15T21:47:41.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.254+0000 7f9362186640 1 -- start start 2024-09-15T21:47:41.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.256+0000 7f9362186640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f935c06db40 0x7f935c06c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:41.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.256+0000 7f9362186640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935c0c46e0 0x7f935c1ec8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:41.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.256+0000 7f935b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f935c06db40 0x7f935c06c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:41.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.256+0000 7f935b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f935c06db40 0x7f935c06c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:59728/0 (socket says 172.21.15.100:59728) 2024-09-15T21:47:41.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.256+0000 7f935b7fe640 1 -- 172.21.15.100:0/697379178 learned_addr learned my addr 172.21.15.100:0/697379178 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:41.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.256+0000 7f935affd640 1 --2- 172.21.15.100:0/697379178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935c0c46e0 0x7f935c1ec8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:41.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.256+0000 7f9362186640 1 --2- 172.21.15.100:0/697379178 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f935c15b370 0x7f935c1ecde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:41.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.256+0000 7f9362186640 1 -- 172.21.15.100:0/697379178 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f935c06d0e0 con 0x7f935c0c46e0 2024-09-15T21:47:41.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.256+0000 7f9362186640 1 -- 172.21.15.100:0/697379178 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f935c06ce00 con 0x7f935c06db40 2024-09-15T21:47:41.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.256+0000 7f9362186640 1 -- 172.21.15.100:0/697379178 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f935c1523d0 con 0x7f935c15b370 2024-09-15T21:47:41.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.256+0000 7f935bfff640 1 --2- 172.21.15.100:0/697379178 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f935c15b370 0x7f935c1ecde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:41.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.257+0000 7f935b7fe640 1 -- 172.21.15.100:0/697379178 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f935c15b370 msgr2=0x7f935c1ecde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:41.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.257+0000 7f935b7fe640 1 --2- 172.21.15.100:0/697379178 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f935c15b370 0x7f935c1ecde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:41.259 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.257+0000 7f935b7fe640 1 -- 172.21.15.100:0/697379178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935c0c46e0 msgr2=0x7f935c1ec8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:41.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.257+0000 7f935b7fe640 1 --2- 172.21.15.100:0/697379178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935c0c46e0 0x7f935c1ec8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:41.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.257+0000 7f935b7fe640 1 -- 172.21.15.100:0/697379178 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935c1552e0 con 0x7f935c06db40 2024-09-15T21:47:41.260 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.257+0000 7f935b7fe640 1 --2- 172.21.15.100:0/697379178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f935c06db40 0x7f935c06c430 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f934c00eb40 tx=0x7f934c00d350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:41.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.258+0000 7f9343fff640 1 -- 172.21.15.100:0/697379178 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c010440 con 0x7f935c06db40 2024-09-15T21:47:41.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.258+0000 7f9343fff640 1 -- 172.21.15.100:0/697379178 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f934c0105e0 con 0x7f935c06db40 2024-09-15T21:47:41.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.258+0000 7f9343fff640 1 -- 172.21.15.100:0/697379178 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c0109d0 con 0x7f935c06db40 2024-09-15T21:47:41.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.258+0000 7f9362186640 1 -- 172.21.15.100:0/697379178 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935c1efeb0 con 0x7f935c06db40 2024-09-15T21:47:41.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.258+0000 7f9362186640 1 -- 172.21.15.100:0/697379178 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f935c1f0470 con 0x7f935c06db40 2024-09-15T21:47:41.261 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.259+0000 7f9362186640 1 -- 172.21.15.100:0/697379178 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f935c069080 con 0x7f935c06db40 2024-09-15T21:47:41.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.260+0000 7f9343fff640 1 -- 172.21.15.100:0/697379178 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f934c010b70 con 0x7f935c06db40 2024-09-15T21:47:41.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.260+0000 7f9343fff640 1 --2- 172.21.15.100:0/697379178 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f934806c380 0x7f934806e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:41.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.260+0000 7f9343fff640 1 -- 172.21.15.100:0/697379178 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5854+0+0 (secure 0 0 0) 0x7f934c08bac0 con 0x7f935c06db40 2024-09-15T21:47:41.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.261+0000 7f935affd640 1 --2- 172.21.15.100:0/697379178 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f934806c380 0x7f934806e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:41.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.261+0000 7f935affd640 1 --2- 172.21.15.100:0/697379178 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f934806c380 0x7f934806e840 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9350034000 tx=0x7f935002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:41.263 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:41.262+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:41.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.265+0000 7f9343fff640 1 -- 172.21.15.100:0/697379178 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f934c059aa0 con 0x7f935c06db40 2024-09-15T21:47:41.281 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:41.281 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:41.290 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:47:41.365 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:41.363+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:41.438 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:41.439 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:41.450 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:47:41.466 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:41.464+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:41.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.529+0000 7f9362186640 1 -- 172.21.15.100:0/697379178 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg force-recovery", "pgid": ["2.16", "2.15", "2.12", "2.c", "2.8", "2.1c"], "target": ["mon-mgr", ""]}) -- 0x7f935c0c42b0 con 0x7f934806c380 2024-09-15T21:47:41.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.530+0000 7f9343fff640 1 -- 172.21.15.100:0/697379178 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 instructing pg(s) [2.8] on osd.0 to force-recovery; instructing pg(s) [2.15] on osd.1 to force-recovery; instructing pg(s) [2.c] on osd.2 to force-recovery; instructing pg(s) [2.12] on osd.3 to force-recovery; instructing pg(s) [2.16] on osd.6 to force-recovery; instructing pg(s) [2.1c] on osd.7 to force-recovery; 2024-09-15T21:47:41.531 INFO:teuthology.orchestra.run.smithi100.stderr:) ==== 325+0+0 (secure 0 0 0) 0x7f93500326f0 con 0x7f934806c380 2024-09-15T21:47:41.532 INFO:teuthology.orchestra.run.smithi100.stderr:instructing pg(s) [2.8] on osd.0 to force-recovery; instructing pg(s) [2.15] on osd.1 to force-recovery; instructing pg(s) [2.c] on osd.2 to force-recovery; instructing pg(s) [2.12] on osd.3 to force-recovery; instructing pg(s) [2.16] on osd.6 to force-recovery; instructing pg(s) [2.1c] on osd.7 to force-recovery; 2024-09-15T21:47:41.532 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-15T21:47:41.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.531+0000 7f9362186640 1 -- 172.21.15.100:0/697379178 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f934806c380 msgr2=0x7f934806e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:41.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.531+0000 7f9362186640 1 --2- 172.21.15.100:0/697379178 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f934806c380 0x7f934806e840 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9350034000 tx=0x7f935002e000 comp rx=0 tx=0).stop 2024-09-15T21:47:41.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.532+0000 7f9362186640 1 -- 172.21.15.100:0/697379178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f935c06db40 msgr2=0x7f935c06c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:41.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.532+0000 7f9362186640 1 --2- 172.21.15.100:0/697379178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f935c06db40 0x7f935c06c430 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f934c00eb40 tx=0x7f934c00d350 comp rx=0 tx=0).stop 2024-09-15T21:47:41.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.532+0000 7f9362186640 1 -- 172.21.15.100:0/697379178 shutdown_connections 2024-09-15T21:47:41.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.532+0000 7f9362186640 1 --2- 172.21.15.100:0/697379178 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f934806c380 0x7f934806e840 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:41.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.532+0000 7f9362186640 1 --2- 172.21.15.100:0/697379178 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f935c15b370 0x7f935c1ecde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:41.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.532+0000 7f9362186640 1 --2- 172.21.15.100:0/697379178 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f935c0c46e0 0x7f935c1ec8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:41.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.532+0000 7f9362186640 1 --2- 172.21.15.100:0/697379178 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f935c06db40 0x7f935c06c430 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:41.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.532+0000 7f9362186640 1 -- 172.21.15.100:0/697379178 >> 172.21.15.100:0/697379178 conn(0x7f935c0ca390 msgr2=0x7f935c072290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:41.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.532+0000 7f9362186640 1 -- 172.21.15.100:0/697379178 shutdown_connections 2024-09-15T21:47:41.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.532+0000 7f9362186640 1 -- 172.21.15.100:0/697379178 wait complete. 2024-09-15T21:47:41.558 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:41.558 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:41.568 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:47:41.569 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:41.566+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:41.669 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:41.667+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:41.714 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:41.714 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:41.723 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:47:41.770 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:41.768+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:41.817 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:47:41.846 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:41.847 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:41.858 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:47:41.870 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:41.869+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:41.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.967+0000 7fd036713640 1 Processor -- start 2024-09-15T21:47:41.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.967+0000 7fd036713640 1 -- start start 2024-09-15T21:47:41.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.969+0000 7fd036713640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd03014e9c0 0x7fd03006df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:41.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.969+0000 7fd036713640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd030157e30 con 0x7fd0300c4600 2024-09-15T21:47:41.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.969+0000 7fd036713640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd03006e4c0 con 0x7fd030072f20 2024-09-15T21:47:41.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.969+0000 7fd036713640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fd0301580d0 con 0x7fd03014e9c0 2024-09-15T21:47:41.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.969+0000 7fd02f7fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd0300c4600 0x7fd03006ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47948/0 (socket says 172.21.15.100:47948) 2024-09-15T21:47:41.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.969+0000 7fd02f7fe640 1 -- 172.21.15.100:0/943369947 learned_addr learned my addr 172.21.15.100:0/943369947 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:41.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.969+0000 7fd02ffff640 1 --2- 172.21.15.100:0/943369947 >> v2:172.21.15.100:3300/0 conn(0x7fd03014e9c0 0x7fd03006df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:41.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.969+0000 7fd02effd640 1 -- 172.21.15.100:0/943369947 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1360678599 0 0) 0x7fd030157e30 con 0x7fd0300c4600 2024-09-15T21:47:41.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.969+0000 7fd02effd640 1 -- 172.21.15.100:0/943369947 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd018003520 con 0x7fd0300c4600 2024-09-15T21:47:41.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.969+0000 7fd02effd640 1 -- 172.21.15.100:0/943369947 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2438963965 0 0) 0x7fd03006e4c0 con 0x7fd030072f20 2024-09-15T21:47:41.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.969+0000 7fd02effd640 1 -- 172.21.15.100:0/943369947 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd030157e30 con 0x7fd030072f20 2024-09-15T21:47:41.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd02effd640 1 -- 172.21.15.100:0/943369947 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 711602533 0 0) 0x7fd018003520 con 0x7fd0300c4600 2024-09-15T21:47:41.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd02effd640 1 -- 172.21.15.100:0/943369947 >> v2:172.21.15.100:3300/0 conn(0x7fd03014e9c0 msgr2=0x7fd03006df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:41.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd02effd640 1 --2- 172.21.15.100:0/943369947 >> v2:172.21.15.100:3300/0 conn(0x7fd03014e9c0 0x7fd03006df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:41.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd02effd640 1 -- 172.21.15.100:0/943369947 >> v1:172.21.15.122:6789/0 conn(0x7fd030072f20 legacy=0x7fd030073390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:41.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd02effd640 1 -- 172.21.15.100:0/943369947 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0301583b0 con 0x7fd0300c4600 2024-09-15T21:47:41.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd02effd640 1 -- 172.21.15.100:0/943369947 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7fd024001670 con 0x7fd0300c4600 2024-09-15T21:47:41.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd02effd640 1 -- 172.21.15.100:0/943369947 >> v1:172.21.15.100:6789/0 conn(0x7fd0300c4600 legacy=0x7fd03006ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:41.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd02effd640 1 --2- 172.21.15.100:0/943369947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd018003b90 0x7fd018003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:41.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd02effd640 1 --2- 172.21.15.100:0/943369947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd018004800 0x7fd018004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:41.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd02effd640 1 --2- 172.21.15.100:0/943369947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0180099b0 0x7fd018005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:41.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd02effd640 1 -- 172.21.15.100:0/943369947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd0301580d0 con 0x7fd0180099b0 2024-09-15T21:47:41.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd02effd640 1 -- 172.21.15.100:0/943369947 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd018014820 con 0x7fd018003b90 2024-09-15T21:47:41.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd02effd640 1 -- 172.21.15.100:0/943369947 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd018014a20 con 0x7fd018004800 2024-09-15T21:47:41.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd034cb0640 1 --2- 172.21.15.100:0/943369947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0180099b0 0x7fd018005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:41.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd02ffff640 1 --2- 172.21.15.100:0/943369947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd018003b90 0x7fd018003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:41.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd02f7fe640 1 --2- 172.21.15.100:0/943369947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd018004800 0x7fd018004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:41.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd034cb0640 1 -- 172.21.15.100:0/943369947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd018004800 msgr2=0x7fd018004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:41.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd034cb0640 1 --2- 172.21.15.100:0/943369947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd018004800 0x7fd018004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:41.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd034cb0640 1 -- 172.21.15.100:0/943369947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd018003b90 msgr2=0x7fd018003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:41.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd034cb0640 1 --2- 172.21.15.100:0/943369947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd018003b90 0x7fd018003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:41.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.970+0000 7fd034cb0640 1 -- 172.21.15.100:0/943369947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd018003520 con 0x7fd0180099b0 2024-09-15T21:47:41.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.971+0000 7fd034cb0640 1 --2- 172.21.15.100:0/943369947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0180099b0 0x7fd018005220 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd028001fd0 tx=0x7fd018005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a669ae58aec666f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:41.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.971+0000 7fd034cb0640 1 -- 172.21.15.100:0/943369947 reap_dead start 2024-09-15T21:47:41.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.972+0000 7fd02effd640 1 -- 172.21.15.100:0/943369947 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028001bf0 con 0x7fd0180099b0 2024-09-15T21:47:41.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.972+0000 7fd02effd640 1 -- 172.21.15.100:0/943369947 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd028003b70 con 0x7fd0180099b0 2024-09-15T21:47:41.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.972+0000 7fd02effd640 1 -- 172.21.15.100:0/943369947 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0280050d0 con 0x7fd0180099b0 2024-09-15T21:47:41.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.973+0000 7fd036713640 1 -- 172.21.15.100:0/943369947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0180099b0 msgr2=0x7fd018005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:41.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.973+0000 7fd036713640 1 --2- 172.21.15.100:0/943369947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0180099b0 0x7fd018005220 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd028001fd0 tx=0x7fd018005830 comp rx=0 tx=0).stop 2024-09-15T21:47:41.978 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:41.969+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:41.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.973+0000 7fd036713640 1 -- 172.21.15.100:0/943369947 shutdown_connections 2024-09-15T21:47:41.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.973+0000 7fd036713640 1 --2- 172.21.15.100:0/943369947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd0180099b0 0x7fd018005220 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:41.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.973+0000 7fd036713640 1 -- 172.21.15.100:0/943369947 >> 172.21.15.100:0/943369947 conn(0x7fd0300ca430 msgr2=0x7fd0300ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:41.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.973+0000 7fd036713640 1 -- 172.21.15.100:0/943369947 shutdown_connections 2024-09-15T21:47:41.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.973+0000 7fd036713640 1 -- 172.21.15.100:0/943369947 wait complete. 2024-09-15T21:47:41.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.975+0000 7fd036713640 1 Processor -- start 2024-09-15T21:47:41.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.975+0000 7fd036713640 1 -- start start 2024-09-15T21:47:41.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.976+0000 7fd036713640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd0300ccf90 0x7fd03014de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:41.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.976+0000 7fd036713640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0300cd370 0x7fd030143c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:41.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.976+0000 7fd036713640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd03014c670 0x7fd03014cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:41.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.977+0000 7fd036713640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd0300caf20 con 0x7fd03014c670 2024-09-15T21:47:41.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.977+0000 7fd036713640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd030158970 con 0x7fd0300cd370 2024-09-15T21:47:41.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.977+0000 7fd036713640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd03014eda0 con 0x7fd0300ccf90 2024-09-15T21:47:41.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.977+0000 7fd02ffff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd0300ccf90 0x7fd03014de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:41.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.977+0000 7fd034cb0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd03014c670 0x7fd03014cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:41.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.977+0000 7fd02ffff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd0300ccf90 0x7fd03014de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:40066/0 (socket says 172.21.15.100:40066) 2024-09-15T21:47:41.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.977+0000 7fd02ffff640 1 -- 172.21.15.100:0/3125275960 learned_addr learned my addr 172.21.15.100:0/3125275960 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:41.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.977+0000 7fd02f7fe640 1 --2- 172.21.15.100:0/3125275960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0300cd370 0x7fd030143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:41.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.977+0000 7fd02ffff640 1 -- 172.21.15.100:0/3125275960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0300cd370 msgr2=0x7fd030143c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:41.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.977+0000 7fd02ffff640 1 --2- 172.21.15.100:0/3125275960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0300cd370 0x7fd030143c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:41.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.977+0000 7fd02ffff640 1 -- 172.21.15.100:0/3125275960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd03014c670 msgr2=0x7fd03014cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:41.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.977+0000 7fd02ffff640 1 --2- 172.21.15.100:0/3125275960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd03014c670 0x7fd03014cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:41.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.977+0000 7fd02ffff640 1 -- 172.21.15.100:0/3125275960 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03014d020 con 0x7fd0300ccf90 2024-09-15T21:47:41.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.977+0000 7fd034cb0640 1 --2- 172.21.15.100:0/3125275960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd03014c670 0x7fd03014cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:41.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.977+0000 7fd02f7fe640 1 --2- 172.21.15.100:0/3125275960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0300cd370 0x7fd030143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:41.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.977+0000 7fd02ffff640 1 --2- 172.21.15.100:0/3125275960 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd0300ccf90 0x7fd03014de90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd020011960 tx=0x7fd020011e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:41.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.978+0000 7fd02cff9640 1 -- 172.21.15.100:0/3125275960 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02000faf0 con 0x7fd0300ccf90 2024-09-15T21:47:41.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.978+0000 7fd02cff9640 1 -- 172.21.15.100:0/3125275960 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd02000fc90 con 0x7fd0300ccf90 2024-09-15T21:47:41.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.978+0000 7fd02cff9640 1 -- 172.21.15.100:0/3125275960 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd020014440 con 0x7fd0300ccf90 2024-09-15T21:47:41.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.978+0000 7fd036713640 1 -- 172.21.15.100:0/3125275960 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0301ec850 con 0x7fd0300ccf90 2024-09-15T21:47:41.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.978+0000 7fd036713640 1 -- 172.21.15.100:0/3125275960 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0301ecd90 con 0x7fd0300ccf90 2024-09-15T21:47:41.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.979+0000 7fd036713640 1 -- 172.21.15.100:0/3125275960 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd030073390 con 0x7fd0300ccf90 2024-09-15T21:47:41.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.980+0000 7fd02cff9640 1 -- 172.21.15.100:0/3125275960 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd020012040 con 0x7fd0300ccf90 2024-09-15T21:47:41.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.980+0000 7fd02cff9640 1 --2- 172.21.15.100:0/3125275960 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd01c06c2e0 0x7fd01c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:41.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.981+0000 7fd02cff9640 1 -- 172.21.15.100:0/3125275960 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(39..39 src has 1..39) ==== 5854+0+0 (secure 0 0 0) 0x7fd02008bab0 con 0x7fd0300ccf90 2024-09-15T21:47:41.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.981+0000 7fd02f7fe640 1 --2- 172.21.15.100:0/3125275960 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd01c06c2e0 0x7fd01c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:41.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.981+0000 7fd02f7fe640 1 --2- 172.21.15.100:0/3125275960 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd01c06c2e0 0x7fd01c06e7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd03014d4a0 tx=0x7fd024007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:41.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:41.985+0000 7fd02cff9640 1 -- 172.21.15.100:0/3125275960 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd020019030 con 0x7fd0300ccf90 2024-09-15T21:47:42.041 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:42.041 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:42.054 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:47:42.072 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:42.071+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:42.173 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:42.172+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:42.174 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:42.174 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:42.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.173+0000 7fd036713640 1 -- 172.21.15.100:0/3125275960 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7fd030146c70 con 0x7fd0300ccf90 2024-09-15T21:47:42.183 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:47:42.240 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:47:42.274 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:42.272+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:42.330 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:42.331 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:42.339 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:47:42.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.338+0000 7fb1bf477640 1 Processor -- start 2024-09-15T21:47:42.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.339+0000 7fb1bf477640 1 -- start start 2024-09-15T21:47:42.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.339+0000 7fb1bf477640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb1b8155290 0x7fb1b815e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:42.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.339+0000 7fb1bf477640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1b815edb0 con 0x7fb1b815bb60 2024-09-15T21:47:42.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.339+0000 7fb1bf477640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1b815f090 con 0x7fb1b8155670 2024-09-15T21:47:42.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.339+0000 7fb1bf477640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fb1b815f370 con 0x7fb1b8155290 2024-09-15T21:47:42.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.340+0000 7fb1bda14640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fb1b815bb60 0x7fb1b806d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47954/0 (socket says 172.21.15.100:47954) 2024-09-15T21:47:42.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.340+0000 7fb1bda14640 1 -- 172.21.15.100:0/2675886390 learned_addr learned my addr 172.21.15.100:0/2675886390 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:42.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.340+0000 7fb1bd213640 1 --2- 172.21.15.100:0/2675886390 >> v2:172.21.15.122:3300/0 conn(0x7fb1b8155290 0x7fb1b815e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:42.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.340+0000 7fb1affff640 1 -- 172.21.15.100:0/2675886390 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3822905952 0 0) 0x7fb1b815edb0 con 0x7fb1b815bb60 2024-09-15T21:47:42.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.340+0000 7fb1affff640 1 -- 172.21.15.100:0/2675886390 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1a4003520 con 0x7fb1b815bb60 2024-09-15T21:47:42.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.340+0000 7fb1affff640 1 -- 172.21.15.100:0/2675886390 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2182761508 0 0) 0x7fb1b815f090 con 0x7fb1b8155670 2024-09-15T21:47:42.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.340+0000 7fb1affff640 1 -- 172.21.15.100:0/2675886390 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1b815edb0 con 0x7fb1b8155670 2024-09-15T21:47:42.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.340+0000 7fb1bd213640 1 -- 172.21.15.100:0/2675886390 >> v1:172.21.15.122:6789/0 conn(0x7fb1b8155670 legacy=0x7fb1b80c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:42.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.340+0000 7fb1bd213640 1 -- 172.21.15.100:0/2675886390 >> v1:172.21.15.100:6789/0 conn(0x7fb1b815bb60 legacy=0x7fb1b806d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:42.343 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.340+0000 7fb1bd213640 1 -- 172.21.15.100:0/2675886390 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b80c5db0 con 0x7fb1b8155290 2024-09-15T21:47:42.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.341+0000 7fb1bd213640 1 --2- 172.21.15.100:0/2675886390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8155290 0x7fb1b815e690 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40097a0 tx=0x7fb1b402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca2d9a2f4717fe75 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:42.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.341+0000 7fb1affff640 1 -- 172.21.15.100:0/2675886390 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4037070 con 0x7fb1b8155290 2024-09-15T21:47:42.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.342+0000 7fb1affff640 1 -- 172.21.15.100:0/2675886390 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1b4032440 con 0x7fb1b8155290 2024-09-15T21:47:42.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.342+0000 7fb1affff640 1 -- 172.21.15.100:0/2675886390 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b40327f0 con 0x7fb1b8155290 2024-09-15T21:47:42.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.342+0000 7fb1bf477640 1 -- 172.21.15.100:0/2675886390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8155290 msgr2=0x7fb1b815e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:42.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.342+0000 7fb1bf477640 1 --2- 172.21.15.100:0/2675886390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8155290 0x7fb1b815e690 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40097a0 tx=0x7fb1b402a230 comp rx=0 tx=0).stop 2024-09-15T21:47:42.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.342+0000 7fb1bf477640 1 -- 172.21.15.100:0/2675886390 shutdown_connections 2024-09-15T21:47:42.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.342+0000 7fb1bf477640 1 --2- 172.21.15.100:0/2675886390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8155290 0x7fb1b815e690 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:42.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.342+0000 7fb1bf477640 1 -- 172.21.15.100:0/2675886390 >> 172.21.15.100:0/2675886390 conn(0x7fb1b8073310 msgr2=0x7fb1b8073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:42.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.342+0000 7fb1bf477640 1 -- 172.21.15.100:0/2675886390 shutdown_connections 2024-09-15T21:47:42.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.342+0000 7fb1bf477640 1 -- 172.21.15.100:0/2675886390 wait complete. 2024-09-15T21:47:42.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.344+0000 7fb1bf477640 1 Processor -- start 2024-09-15T21:47:42.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.344+0000 7fb1bf477640 1 -- start start 2024-09-15T21:47:42.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.345+0000 7fb1bf477640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1b8155290 0x7fb1b80b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:42.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.345+0000 7fb1bf477640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8155670 0x7fb1b8156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:42.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.345+0000 7fb1bf477640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb1b815bb60 0x7fb1b8150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:42.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.345+0000 7fb1bf477640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b815c590 con 0x7fb1b8155290 2024-09-15T21:47:42.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.345+0000 7fb1bf477640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b80b39e0 con 0x7fb1b8155670 2024-09-15T21:47:42.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.345+0000 7fb1bf477640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b80c6cc0 con 0x7fb1b815bb60 2024-09-15T21:47:42.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.345+0000 7fb1bd213640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1b8155290 0x7fb1b80b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:42.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.345+0000 7fb1bd213640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1b8155290 0x7fb1b80b5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33794/0 (socket says 172.21.15.100:33794) 2024-09-15T21:47:42.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.345+0000 7fb1bd213640 1 -- 172.21.15.100:0/2876817216 learned_addr learned my addr 172.21.15.100:0/2876817216 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:42.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.345+0000 7fb1bca12640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8155670 0x7fb1b8156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:42.348 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.345+0000 7fb1bda14640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb1b815bb60 0x7fb1b8150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:42.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.345+0000 7fb1bca12640 1 -- 172.21.15.100:0/2876817216 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb1b815bb60 msgr2=0x7fb1b8150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:42.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.345+0000 7fb1bca12640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb1b815bb60 0x7fb1b8150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:42.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.345+0000 7fb1bca12640 1 -- 172.21.15.100:0/2876817216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1b8155290 msgr2=0x7fb1b80b5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:42.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.345+0000 7fb1bca12640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1b8155290 0x7fb1b80b5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:42.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.345+0000 7fb1bca12640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b80cf170 con 0x7fb1b8155670 2024-09-15T21:47:42.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.345+0000 7fb1bd213640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb1b8155290 0x7fb1b80b5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:42.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.346+0000 7fb1bca12640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb1b8155670 0x7fb1b8156390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800c3c0 tx=0x7fb1a800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:42.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.346+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8017070 con 0x7fb1b8155670 2024-09-15T21:47:42.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.346+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a80052f0 con 0x7fb1b8155670 2024-09-15T21:47:42.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.346+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8003830 con 0x7fb1b8155670 2024-09-15T21:47:42.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.346+0000 7fb1bf477640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b81f13f0 con 0x7fb1b8155670 2024-09-15T21:47:42.349 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.346+0000 7fb1bf477640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b80cee90 con 0x7fb1b8155670 2024-09-15T21:47:42.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.347+0000 7fb1bf477640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fb1b815bf70 con 0x7fb1b8155670 2024-09-15T21:47:42.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.348+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb1a801a070 con 0x7fb1b8155670 2024-09-15T21:47:42.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.348+0000 7fb1adffb640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb19c06c2b0 0x7fb19c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:42.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.348+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5854+0+0 (secure 0 0 0) 0x7fb1a808a700 con 0x7fb1b8155670 2024-09-15T21:47:42.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.348+0000 7fb1bd213640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb19c06c2b0 0x7fb19c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:42.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.348+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v39) ==== 54+0+16 (secure 0 0 0) 0x7fb1a8090170 con 0x7fb1b8155670 2024-09-15T21:47:42.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.349+0000 7fb1bf477640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fb1b81508c0 0x7fb1b806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:42.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.349+0000 7fb1bf477640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb1b8152370 con 0x7fb1b81508c0 2024-09-15T21:47:42.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.349+0000 7fb1bda14640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fb1b81508c0 0x7fb1b806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:42.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.349+0000 7fb1bd213640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb19c06c2b0 0x7fb19c06e770 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40097a0 tx=0x7fb1b402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:42.351 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.349+0000 7fb1bda14640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fb1b81508c0 0x7fb1b806d720 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:42.352 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.350+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1b8152370 con 0x7fb1b81508c0 2024-09-15T21:47:42.375 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:42.373+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:42.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.377+0000 7fb1bf477640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb1b80cc7e0 con 0x7fb1b81508c0 2024-09-15T21:47:42.381 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.380+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb1b80cc7e0 con 0x7fb1b81508c0 2024-09-15T21:47:42.382 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:42.383 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:42.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.380+0000 7fb1bf477640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fb1b81568d0 0x7fb1b814eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:42.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.380+0000 7fb1bf477640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fb1b8155a50 con 0x7fb1b81568d0 2024-09-15T21:47:42.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.380+0000 7fb1bd213640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fb1b81568d0 0x7fb1b814eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:42.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.381+0000 7fb1bd213640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fb1b81568d0 0x7fb1b814eb10 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:42.383 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.381+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1b8155a50 con 0x7fb1b81568d0 2024-09-15T21:47:42.410 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.408+0000 7fb1bf477640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb1b81f3ba0 con 0x7fb1b81568d0 2024-09-15T21:47:42.411 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.409+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb1b81f3ba0 con 0x7fb1b81568d0 2024-09-15T21:47:42.411 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:42.412 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:42.412 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.410+0000 7fb1bf477640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fb1b81f4510 0x7fb1b8062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:42.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.410+0000 7fb1bf477640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fb1b81f4930 con 0x7fb1b81f4510 2024-09-15T21:47:42.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.410+0000 7fb1bca12640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fb1b81f4510 0x7fb1b8062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:42.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.410+0000 7fb1bca12640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fb1b81f4510 0x7fb1b8062630 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:42.413 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.411+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1b81f4930 con 0x7fb1b81f4510 2024-09-15T21:47:42.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.438+0000 7fb1bf477640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb1b8062dc0 con 0x7fb1b81f4510 2024-09-15T21:47:42.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.439+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb1b8062dc0 con 0x7fb1b81f4510 2024-09-15T21:47:42.441 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:42.442 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:42.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.440+0000 7fb1bf477640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fb1b80b3f90 0x7fb1b80b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:42.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.440+0000 7fb1bf477640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fb1b8062dc0 con 0x7fb1b80b3f90 2024-09-15T21:47:42.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.440+0000 7fb1bda14640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fb1b80b3f90 0x7fb1b80b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:42.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.440+0000 7fb1bda14640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fb1b80b3f90 0x7fb1b80b4390 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:42.443 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.441+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1b8062dc0 con 0x7fb1b80b3f90 2024-09-15T21:47:42.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.467+0000 7fb1bf477640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb1b8062dc0 con 0x7fb1b80b3f90 2024-09-15T21:47:42.470 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.469+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb1b8062dc0 con 0x7fb1b80b3f90 2024-09-15T21:47:42.471 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:42.471 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:42.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.469+0000 7fb1bf477640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fb1b802c640 0x7fb1b802caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:42.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.469+0000 7fb1bf477640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fb1b802cfe0 con 0x7fb1b802c640 2024-09-15T21:47:42.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.469+0000 7fb1bd213640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fb1b802c640 0x7fb1b802caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:42.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.470+0000 7fb1bd213640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fb1b802c640 0x7fb1b802caa0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:42.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.470+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1b802cfe0 con 0x7fb1b802c640 2024-09-15T21:47:42.476 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:42.475+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:42.484 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:42.484 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:42.493 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:47:42.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.497+0000 7fb1bf477640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb1b802cfe0 con 0x7fb1b802c640 2024-09-15T21:47:42.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.499+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb1b802cfe0 con 0x7fb1b802c640 2024-09-15T21:47:42.501 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:42.503 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:42.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.499+0000 7fb1bf477640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=40}) -- 0x7fb1b802cfe0 con 0x7fb1b8155670 2024-09-15T21:47:42.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.499+0000 7fb1bf477640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb1b802d370 con 0x7fb1b8155670 2024-09-15T21:47:42.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.499+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb1a8091070 con 0x7fb1b8155670 2024-09-15T21:47:42.504 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:47:42.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.499+0000 7fb1bf477640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fb1b802d970 0x7fb1b8207440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:42.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.499+0000 7fb1bf477640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fb1b820a290 con 0x7fb1b802d970 2024-09-15T21:47:42.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.500+0000 7fb1bca12640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fb1b802d970 0x7fb1b8207440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:42.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.500+0000 7fb1bca12640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fb1b802d970 0x7fb1b8207440 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:42.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.501+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1b820a290 con 0x7fb1b802d970 2024-09-15T21:47:42.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.527+0000 7fb1bf477640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb1b8207c50 con 0x7fb1b802d970 2024-09-15T21:47:42.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.530+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb1b8207c50 con 0x7fb1b802d970 2024-09-15T21:47:42.535 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:42.536 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:42.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.530+0000 7fb1bf477640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fb1b8208370 0x7fb1b82087b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:42.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.530+0000 7fb1bf477640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fb1b820d660 con 0x7fb1b8208370 2024-09-15T21:47:42.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.530+0000 7fb1bda14640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fb1b8208370 0x7fb1b82087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:42.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.531+0000 7fb1bda14640 1 --2- 172.21.15.100:0/2876817216 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fb1b8208370 0x7fb1b82087b0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:42.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.531+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb1b820d660 con 0x7fb1b8208370 2024-09-15T21:47:42.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.557+0000 7fb1bf477640 1 -- 172.21.15.100:0/2876817216 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fb1b820c4e0 con 0x7fb1b8208370 2024-09-15T21:47:42.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.559+0000 7fb1adffb640 1 -- 172.21.15.100:0/2876817216 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fb1b820c4e0 con 0x7fb1b8208370 2024-09-15T21:47:42.561 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:42.561 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:42.567 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:47:42.567 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:47:42.598 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:42.598 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:42.598 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:42.598 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:42.608 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:47:42.609 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:42.607+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:42.679 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:42.677+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:42.712 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:42.712 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:42.712 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:42.712 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:42.712 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:42.720 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:47:42.780 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:42.778+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:42.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.805+0000 7fd02cff9640 1 -- 172.21.15.100:0/3125275960 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v40) ==== 97+0+0 (secure 0 0 0) 0x7fd020059aa0 con 0x7fd0300ccf90 2024-09-15T21:47:42.807 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is set 2024-09-15T21:47:42.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.807+0000 7fd036713640 1 -- 172.21.15.100:0/3125275960 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd01c06c2e0 msgr2=0x7fd01c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:42.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.807+0000 7fd036713640 1 --2- 172.21.15.100:0/3125275960 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd01c06c2e0 0x7fd01c06e7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd03014d4a0 tx=0x7fd024007040 comp rx=0 tx=0).stop 2024-09-15T21:47:42.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.807+0000 7fd036713640 1 -- 172.21.15.100:0/3125275960 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd0300ccf90 msgr2=0x7fd03014de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:42.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.807+0000 7fd036713640 1 --2- 172.21.15.100:0/3125275960 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd0300ccf90 0x7fd03014de90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd020011960 tx=0x7fd020011e20 comp rx=0 tx=0).stop 2024-09-15T21:47:42.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.808+0000 7fd036713640 1 -- 172.21.15.100:0/3125275960 shutdown_connections 2024-09-15T21:47:42.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.808+0000 7fd036713640 1 --2- 172.21.15.100:0/3125275960 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd01c06c2e0 0x7fd01c06e7a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:42.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.808+0000 7fd036713640 1 --2- 172.21.15.100:0/3125275960 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0300cd370 0x7fd030143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:42.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.808+0000 7fd036713640 1 --2- 172.21.15.100:0/3125275960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd03014c670 0x7fd03014cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:42.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.808+0000 7fd036713640 1 --2- 172.21.15.100:0/3125275960 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd0300ccf90 0x7fd03014de90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:42.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.808+0000 7fd036713640 1 -- 172.21.15.100:0/3125275960 >> 172.21.15.100:0/3125275960 conn(0x7fd0300ca430 msgr2=0x7fd03006bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:42.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.808+0000 7fd036713640 1 -- 172.21.15.100:0/3125275960 shutdown_connections 2024-09-15T21:47:42.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:42.808+0000 7fd036713640 1 -- 172.21.15.100:0/3125275960 wait complete. 2024-09-15T21:47:42.828 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:42.828 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:42.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:42.828 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:42.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.829 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:42.829 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:42.829 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 74.411588527000006, 2024-09-15T21:47:42.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0050344960000000003, 2024-09-15T21:47:42.829 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.829 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.829 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:42.830 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:42.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0199999999999998e-07 2024-09-15T21:47:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413470+0000", 2024-09-15T21:47:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4329999999999998e-06 2024-09-15T21:47:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.833 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413579+0000", 2024-09-15T21:47:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108593 2024-09-15T21:47:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413587+0000", 2024-09-15T21:47:42.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3100000000000001e-06 2024-09-15T21:47:42.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413691+0000", 2024-09-15T21:47:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010357 2024-09-15T21:47:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413887+0000", 2024-09-15T21:47:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019666799999999999 2024-09-15T21:47:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414027+0000", 2024-09-15T21:47:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013996700000000001 2024-09-15T21:47:42.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416360+0000", 2024-09-15T21:47:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002332204 2024-09-15T21:47:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418437+0000", 2024-09-15T21:47:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002077279 2024-09-15T21:47:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418469+0000", 2024-09-15T21:47:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2359e-05 2024-09-15T21:47:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418502+0000", 2024-09-15T21:47:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2811000000000001e-05 2024-09-15T21:47:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.839 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 74.411584486999999, 2024-09-15T21:47:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0053538780000000003, 2024-09-15T21:47:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.840 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2000000000000001e-07 2024-09-15T21:47:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413474+0000", 2024-09-15T21:47:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-15T21:47:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413612+0000", 2024-09-15T21:47:42.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000137727 2024-09-15T21:47:42.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413618+0000", 2024-09-15T21:47:42.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.241e-06 2024-09-15T21:47:42.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414443+0000", 2024-09-15T21:47:42.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082447 2024-09-15T21:47:42.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414523+0000", 2024-09-15T21:47:42.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0059e-05 2024-09-15T21:47:42.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:42.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414653+0000", 2024-09-15T21:47:42.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013031699999999999 2024-09-15T21:47:42.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:42.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416366+0000", 2024-09-15T21:47:42.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001712917 2024-09-15T21:47:42.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:42.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418769+0000", 2024-09-15T21:47:42.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024031529999999999 2024-09-15T21:47:42.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:42.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418802+0000", 2024-09-15T21:47:42.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2925000000000001e-05 2024-09-15T21:47:42.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418825+0000", 2024-09-15T21:47:42.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3204e-05 2024-09-15T21:47:42.846 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.846 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.846 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.847 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:42.847 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:42.847 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 74.411576466, 2024-09-15T21:47:42.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059708640000000002, 2024-09-15T21:47:42.847 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.847 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.847 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:42.847 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:42.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:42.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:42.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7300000000000002e-07 2024-09-15T21:47:42.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413482+0000", 2024-09-15T21:47:42.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2359999999999999e-06 2024-09-15T21:47:42.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.849 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.849 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414323+0000", 2024-09-15T21:47:42.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00084095399999999999 2024-09-15T21:47:42.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414332+0000", 2024-09-15T21:47:42.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0880000000000001e-06 2024-09-15T21:47:42.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415439+0000", 2024-09-15T21:47:42.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011071869999999999 2024-09-15T21:47:42.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415554+0000", 2024-09-15T21:47:42.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114499 2024-09-15T21:47:42.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:42.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415682+0000", 2024-09-15T21:47:42.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012802500000000001 2024-09-15T21:47:42.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:42.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.417957+0000", 2024-09-15T21:47:42.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022746720000000002 2024-09-15T21:47:42.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419385+0000", 2024-09-15T21:47:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001428826 2024-09-15T21:47:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419420+0000", 2024-09-15T21:47:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4875000000000001e-05 2024-09-15T21:47:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419450+0000", 2024-09-15T21:47:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0128999999999999e-05 2024-09-15T21:47:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 74.411572648000003, 2024-09-15T21:47:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0087186220000000005, 2024-09-15T21:47:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413484+0000", 2024-09-15T21:47:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-15T21:47:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413487+0000", 2024-09-15T21:47:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.534e-06 2024-09-15T21:47:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414353+0000", 2024-09-15T21:47:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00086616999999999998 2024-09-15T21:47:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414359+0000", 2024-09-15T21:47:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3279999999999996e-06 2024-09-15T21:47:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416041+0000", 2024-09-15T21:47:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001682467 2024-09-15T21:47:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416158+0000", 2024-09-15T21:47:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117242 2024-09-15T21:47:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416284+0000", 2024-09-15T21:47:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012509000000000001 2024-09-15T21:47:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.417992+0000", 2024-09-15T21:47:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017087059999999999 2024-09-15T21:47:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422142+0000", 2024-09-15T21:47:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041501180000000004 2024-09-15T21:47:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422176+0000", 2024-09-15T21:47:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4109999999999997e-05 2024-09-15T21:47:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422202+0000", 2024-09-15T21:47:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5585e-05 2024-09-15T21:47:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 74.411567223000006, 2024-09-15T21:47:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0088000370000000001, 2024-09-15T21:47:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:42.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:42.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:42.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413491+0000", 2024-09-15T21:47:42.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.773e-06 2024-09-15T21:47:42.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414372+0000", 2024-09-15T21:47:42.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00088122000000000005 2024-09-15T21:47:42.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.863 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.863 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414378+0000", 2024-09-15T21:47:42.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4380000000000003e-06 2024-09-15T21:47:42.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416631+0000", 2024-09-15T21:47:42.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022530319999999999 2024-09-15T21:47:42.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416765+0000", 2024-09-15T21:47:42.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013423499999999999 2024-09-15T21:47:42.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422289+0000", 2024-09-15T21:47:42.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055240530000000001 2024-09-15T21:47:42.865 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.865 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.865 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.866 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:42.866 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:42.866 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.369998903999999, 2024-09-15T21:47:42.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056900609999999997, 2024-09-15T21:47:42.866 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.866 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.866 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:42.866 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:42.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:42.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455058+0000", 2024-09-15T21:47:42.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.51e-07 2024-09-15T21:47:42.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455061+0000", 2024-09-15T21:47:42.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.145e-06 2024-09-15T21:47:42.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455158+0000", 2024-09-15T21:47:42.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7101000000000004e-05 2024-09-15T21:47:42.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455165+0000", 2024-09-15T21:47:42.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1019999999999999e-06 2024-09-15T21:47:42.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456433+0000", 2024-09-15T21:47:42.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012678259999999999 2024-09-15T21:47:42.869 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456517+0000", 2024-09-15T21:47:42.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4373e-05 2024-09-15T21:47:42.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:42.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456677+0000", 2024-09-15T21:47:42.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015963300000000001 2024-09-15T21:47:42.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:42.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458636+0000", 2024-09-15T21:47:42.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019594030000000002 2024-09-15T21:47:42.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:42.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460665+0000", 2024-09-15T21:47:42.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020283549999999999 2024-09-15T21:47:42.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:42.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460707+0000", 2024-09-15T21:47:42.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2398000000000003e-05 2024-09-15T21:47:42.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460747+0000", 2024-09-15T21:47:42.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0274000000000001e-05 2024-09-15T21:47:42.872 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.872 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.872 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.872 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:42.872 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:42.873 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.369991709999994, 2024-09-15T21:47:42.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086420030000000005, 2024-09-15T21:47:42.873 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.873 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.873 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:42.873 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:42.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455065+0000", 2024-09-15T21:47:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3700000000000001e-07 2024-09-15T21:47:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455068+0000", 2024-09-15T21:47:42.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.915e-06 2024-09-15T21:47:42.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455366+0000", 2024-09-15T21:47:42.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00029825400000000002 2024-09-15T21:47:42.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455374+0000", 2024-09-15T21:47:42.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6240000000000008e-06 2024-09-15T21:47:42.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456199+0000", 2024-09-15T21:47:42.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082448500000000002 2024-09-15T21:47:42.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457034+0000", 2024-09-15T21:47:42.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00083553399999999997 2024-09-15T21:47:42.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:42.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457171+0000", 2024-09-15T21:47:42.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013653600000000001 2024-09-15T21:47:42.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:42.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462663+0000", 2024-09-15T21:47:42.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054924420000000002 2024-09-15T21:47:42.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:42.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463613+0000", 2024-09-15T21:47:42.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094971499999999996 2024-09-15T21:47:42.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:42.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463663+0000", 2024-09-15T21:47:42.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9636999999999999e-05 2024-09-15T21:47:42.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463706+0000", 2024-09-15T21:47:42.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3523999999999999e-05 2024-09-15T21:47:42.879 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.879 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.879 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.879 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:42.880 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:42.880 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.369987182000003, 2024-09-15T21:47:42.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090408420000000003, 2024-09-15T21:47:42.880 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.880 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.880 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:42.880 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:42.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:42.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:42.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.79e-07 2024-09-15T21:47:42.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455072+0000", 2024-09-15T21:47:42.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6189999999999998e-06 2024-09-15T21:47:42.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455403+0000", 2024-09-15T21:47:42.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033167299999999999 2024-09-15T21:47:42.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455420+0000", 2024-09-15T21:47:42.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6518999999999999e-05 2024-09-15T21:47:42.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457448+0000", 2024-09-15T21:47:42.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020275710000000001 2024-09-15T21:47:42.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:42.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3844999999999994e-05 2024-09-15T21:47:42.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:42.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457684+0000", 2024-09-15T21:47:42.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015239 2024-09-15T21:47:42.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:42.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462678+0000", 2024-09-15T21:47:42.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049939279999999999 2024-09-15T21:47:42.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:42.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464017+0000", 2024-09-15T21:47:42.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013397140000000001 2024-09-15T21:47:42.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:42.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464071+0000", 2024-09-15T21:47:42.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3735e-05 2024-09-15T21:47:42.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.886 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464110+0000", 2024-09-15T21:47:42.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8569000000000003e-05 2024-09-15T21:47:42.886 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.886 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.886 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.886 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:42.886 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:42.886 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.369975569000005, 2024-09-15T21:47:42.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091802889999999995, 2024-09-15T21:47:42.887 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.887 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.887 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:42.887 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:42.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:42.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455081+0000", 2024-09-15T21:47:42.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:42.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455084+0000", 2024-09-15T21:47:42.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7810000000000001e-06 2024-09-15T21:47:42.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455536+0000", 2024-09-15T21:47:42.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00045219100000000001 2024-09-15T21:47:42.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455583+0000", 2024-09-15T21:47:42.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7308999999999997e-05 2024-09-15T21:47:42.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457998+0000", 2024-09-15T21:47:42.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024154290000000002 2024-09-15T21:47:42.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458147+0000", 2024-09-15T21:47:42.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014834599999999999 2024-09-15T21:47:42.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:42.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458265+0000", 2024-09-15T21:47:42.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118458 2024-09-15T21:47:42.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:42.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464134+0000", 2024-09-15T21:47:42.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005869038 2024-09-15T21:47:42.891 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.891 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.891 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:42.891 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464187+0000", 2024-09-15T21:47:42.891 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2343999999999999e-05 2024-09-15T21:47:42.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:42.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464222+0000", 2024-09-15T21:47:42.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5447000000000001e-05 2024-09-15T21:47:42.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.892 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.892 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464261+0000", 2024-09-15T21:47:42.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8659999999999999e-05 2024-09-15T21:47:42.892 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.893 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.893 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.893 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:42.893 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:42.893 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.349353190000002, 2024-09-15T21:47:42.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048468290000000004, 2024-09-15T21:47:42.893 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.893 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.893 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:42.893 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:42.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:42.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:42.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.35e-07 2024-09-15T21:47:42.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475706+0000", 2024-09-15T21:47:42.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.994e-06 2024-09-15T21:47:42.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475770+0000", 2024-09-15T21:47:42.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3904000000000003e-05 2024-09-15T21:47:42.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475775+0000", 2024-09-15T21:47:42.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2490000000000001e-06 2024-09-15T21:47:42.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476502+0000", 2024-09-15T21:47:42.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072697300000000001 2024-09-15T21:47:42.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476611+0000", 2024-09-15T21:47:42.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0001091 2024-09-15T21:47:42.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:42.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476807+0000", 2024-09-15T21:47:42.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000195892 2024-09-15T21:47:42.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:42.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479678+0000", 2024-09-15T21:47:42.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028704709999999999 2024-09-15T21:47:42.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.898 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.898 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:42.898 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480444+0000", 2024-09-15T21:47:42.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00076602099999999998 2024-09-15T21:47:42.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.898 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.898 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:42.898 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480502+0000", 2024-09-15T21:47:42.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8313999999999998e-05 2024-09-15T21:47:42.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480550+0000", 2024-09-15T21:47:42.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7676000000000002e-05 2024-09-15T21:47:42.899 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.899 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.899 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.899 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:42.899 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:42.899 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.349346015000002, 2024-09-15T21:47:42.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051250929999999998, 2024-09-15T21:47:42.900 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.900 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.900 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:42.900 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:42.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:42.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:42.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1e-07 2024-09-15T21:47:42.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475723+0000", 2024-09-15T21:47:42.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2758000000000001e-05 2024-09-15T21:47:42.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475972+0000", 2024-09-15T21:47:42.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024886299999999999 2024-09-15T21:47:42.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475979+0000", 2024-09-15T21:47:42.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0230000000000004e-06 2024-09-15T21:47:42.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.903 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476560+0000", 2024-09-15T21:47:42.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00058097599999999998 2024-09-15T21:47:42.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.903 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476689+0000", 2024-09-15T21:47:42.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129486 2024-09-15T21:47:42.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.904 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.904 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480835+0000", 2024-09-15T21:47:42.904 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041457769999999998 2024-09-15T21:47:42.904 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.904 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.904 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.904 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:42.904 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:42.904 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.280430789999997, 2024-09-15T21:47:42.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054420570000000001, 2024-09-15T21:47:42.905 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.905 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.905 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:42.905 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:42.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:42.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544626+0000", 2024-09-15T21:47:42.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8100000000000003e-07 2024-09-15T21:47:42.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544629+0000", 2024-09-15T21:47:42.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9869999999999999e-06 2024-09-15T21:47:42.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544775+0000", 2024-09-15T21:47:42.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014610599999999999 2024-09-15T21:47:42.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544782+0000", 2024-09-15T21:47:42.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4109999999999999e-06 2024-09-15T21:47:42.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.908 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545617+0000", 2024-09-15T21:47:42.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00083468300000000002 2024-09-15T21:47:42.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.908 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545690+0000", 2024-09-15T21:47:42.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3545000000000001e-05 2024-09-15T21:47:42.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.909 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:42.909 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545803+0000", 2024-09-15T21:47:42.909 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000112083 2024-09-15T21:47:42.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.909 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:42.909 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548353+0000", 2024-09-15T21:47:42.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025507220000000001 2024-09-15T21:47:42.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.910 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:42.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549964+0000", 2024-09-15T21:47:42.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001610382 2024-09-15T21:47:42.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:42.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550021+0000", 2024-09-15T21:47:42.911 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6898999999999999e-05 2024-09-15T21:47:42.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550067+0000", 2024-09-15T21:47:42.911 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758000000000002e-05 2024-09-15T21:47:42.911 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.911 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.911 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.912 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.912 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:42.912 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:42.912 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.280425512000001, 2024-09-15T21:47:42.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056323399999999996, 2024-09-15T21:47:42.912 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.913 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.913 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:42.913 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.913 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.913 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:42.913 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.913 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.913 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:42.913 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.914 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:42.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:47:42.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.914 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544634+0000", 2024-09-15T21:47:42.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.58e-06 2024-09-15T21:47:42.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.915 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.915 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544800+0000", 2024-09-15T21:47:42.915 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016576899999999999 2024-09-15T21:47:42.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.915 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.915 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.915 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544808+0000", 2024-09-15T21:47:42.915 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5179999999999998e-06 2024-09-15T21:47:42.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.915 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.915 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.915 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546075+0000", 2024-09-15T21:47:42.916 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012676969999999999 2024-09-15T21:47:42.916 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.916 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.916 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546179+0000", 2024-09-15T21:47:42.916 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000103935 2024-09-15T21:47:42.916 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.916 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:42.916 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546286+0000", 2024-09-15T21:47:42.916 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106758 2024-09-15T21:47:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548528+0000", 2024-09-15T21:47:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022420510000000001 2024-09-15T21:47:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550119+0000", 2024-09-15T21:47:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015907079999999999 2024-09-15T21:47:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550202+0000", 2024-09-15T21:47:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3374000000000005e-05 2024-09-15T21:47:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550263+0000", 2024-09-15T21:47:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0643999999999997e-05 2024-09-15T21:47:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.918 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.280420540999998, 2024-09-15T21:47:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061905470000000002, 2024-09-15T21:47:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6899999999999999e-07 2024-09-15T21:47:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544638+0000", 2024-09-15T21:47:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3920000000000001e-06 2024-09-15T21:47:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.921 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544819+0000", 2024-09-15T21:47:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018065099999999999 2024-09-15T21:47:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544825+0000", 2024-09-15T21:47:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7239999999999997e-06 2024-09-15T21:47:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546583+0000", 2024-09-15T21:47:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017588219999999999 2024-09-15T21:47:42.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546665+0000", 2024-09-15T21:47:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1693000000000005e-05 2024-09-15T21:47:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546808+0000", 2024-09-15T21:47:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142993 2024-09-15T21:47:42.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548533+0000", 2024-09-15T21:47:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017246 2024-09-15T21:47:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550606+0000", 2024-09-15T21:47:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020733969999999998 2024-09-15T21:47:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550783+0000", 2024-09-15T21:47:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000176565 2024-09-15T21:47:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550826+0000", 2024-09-15T21:47:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3441e-05 2024-09-15T21:47:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.280412820000002, 2024-09-15T21:47:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052813010000000004, 2024-09-15T21:47:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.926 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3799999999999999e-07 2024-09-15T21:47:42.927 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544647+0000", 2024-09-15T21:47:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6569999999999999e-06 2024-09-15T21:47:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544858+0000", 2024-09-15T21:47:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021069900000000001 2024-09-15T21:47:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.928 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544862+0000", 2024-09-15T21:47:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4900000000000002e-06 2024-09-15T21:47:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545610+0000", 2024-09-15T21:47:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00074786999999999998 2024-09-15T21:47:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545706+0000", 2024-09-15T21:47:42.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6271e-05 2024-09-15T21:47:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545834+0000", 2024-09-15T21:47:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012759499999999999 2024-09-15T21:47:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548554+0000", 2024-09-15T21:47:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027199339999999998 2024-09-15T21:47:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549818+0000", 2024-09-15T21:47:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012642199999999999 2024-09-15T21:47:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549884+0000", 2024-09-15T21:47:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5843e-05 2024-09-15T21:47:42.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549925+0000", 2024-09-15T21:47:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0484000000000001e-05 2024-09-15T21:47:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:42.932 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 57.280404142999998, 2024-09-15T21:47:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051106110000000001, 2024-09-15T21:47:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.933 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2700000000000001e-07 2024-09-15T21:47:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.934 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544667+0000", 2024-09-15T21:47:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4746000000000001e-05 2024-09-15T21:47:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545021+0000", 2024-09-15T21:47:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035412900000000001 2024-09-15T21:47:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545052+0000", 2024-09-15T21:47:42.935 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1158000000000002e-05 2024-09-15T21:47:42.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.936 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.936 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.936 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546050+0000", 2024-09-15T21:47:42.936 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00099782800000000008 2024-09-15T21:47:42.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.936 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.936 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.936 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546291+0000", 2024-09-15T21:47:42.936 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024093500000000001 2024-09-15T21:47:42.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.936 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.937 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:42.937 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546434+0000", 2024-09-15T21:47:42.937 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014308600000000001 2024-09-15T21:47:42.937 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.937 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.937 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:42.937 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548583+0000", 2024-09-15T21:47:42.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021491029999999999 2024-09-15T21:47:42.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.938 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:42.938 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549695+0000", 2024-09-15T21:47:42.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011116419999999999 2024-09-15T21:47:42.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.938 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.938 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:42.938 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549726+0000", 2024-09-15T21:47:42.938 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1195000000000002e-05 2024-09-15T21:47:42.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.939 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.939 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549763+0000", 2024-09-15T21:47:42.939 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6562e-05 2024-09-15T21:47:42.939 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.939 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.939 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.939 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.939 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.939 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:17 2.14 2:2eebc8f4:::62f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:42.939 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:42.939 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.259974530999997, 2024-09-15T21:47:42.940 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071004800000000002, 2024-09-15T21:47:42.940 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.940 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.940 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:42.940 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.940 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.940 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.940 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:42.940 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.941 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.941 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.941 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:42.941 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.941 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.941 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.941 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.941 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565083+0000", 2024-09-15T21:47:42.941 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.689e-06 2024-09-15T21:47:42.941 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.941 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.942 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.942 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565090+0000", 2024-09-15T21:47:42.942 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5930000000000002e-06 2024-09-15T21:47:42.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.942 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.942 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.942 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565119+0000", 2024-09-15T21:47:42.942 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8785999999999998e-05 2024-09-15T21:47:42.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.942 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.942 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.942 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565126+0000", 2024-09-15T21:47:42.943 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0110000000000003e-06 2024-09-15T21:47:42.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.943 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.943 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.943 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565237+0000", 2024-09-15T21:47:42.943 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011161 2024-09-15T21:47:42.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.943 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.943 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.943 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565383+0000", 2024-09-15T21:47:42.943 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014626900000000001 2024-09-15T21:47:42.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.944 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.944 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:42.944 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565520+0000", 2024-09-15T21:47:42.944 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013676399999999999 2024-09-15T21:47:42.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.944 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.944 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:42.944 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569909+0000", 2024-09-15T21:47:42.944 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043890149999999996 2024-09-15T21:47:42.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.944 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.944 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:42.945 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572096+0000", 2024-09-15T21:47:42.945 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021866559999999999 2024-09-15T21:47:42.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.945 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.945 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:42.945 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572144+0000", 2024-09-15T21:47:42.945 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7725000000000001e-05 2024-09-15T21:47:42.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.945 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.945 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.945 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572182+0000", 2024-09-15T21:47:42.945 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8362000000000003e-05 2024-09-15T21:47:42.946 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.946 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.946 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.946 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.946 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.946 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:42.946 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:42.946 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.259963433000003, 2024-09-15T21:47:42.946 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072110280000000004, 2024-09-15T21:47:42.946 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.946 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.946 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:42.947 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.947 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.947 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:42.947 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.947 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.947 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:42.947 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.947 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.948 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.948 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:42.948 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4299999999999999e-07 2024-09-15T21:47:42.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.948 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.948 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.948 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565096+0000", 2024-09-15T21:47:42.948 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4319999999999999e-06 2024-09-15T21:47:42.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.948 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.948 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.948 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565152+0000", 2024-09-15T21:47:42.949 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5686999999999997e-05 2024-09-15T21:47:42.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.949 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.949 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.949 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565157+0000", 2024-09-15T21:47:42.949 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2700000000000004e-06 2024-09-15T21:47:42.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.949 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.949 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.949 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565847+0000", 2024-09-15T21:47:42.949 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068937299999999996 2024-09-15T21:47:42.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.950 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.950 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.950 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565940+0000", 2024-09-15T21:47:42.950 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3573999999999996e-05 2024-09-15T21:47:42.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.950 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.950 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:42.950 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566085+0000", 2024-09-15T21:47:42.950 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000144902 2024-09-15T21:47:42.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.951 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:42.951 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570595+0000", 2024-09-15T21:47:42.951 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045103469999999996 2024-09-15T21:47:42.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.951 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:42.951 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572224+0000", 2024-09-15T21:47:42.951 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016282930000000001 2024-09-15T21:47:42.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.951 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.951 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:42.952 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572260+0000", 2024-09-15T21:47:42.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6103999999999999e-05 2024-09-15T21:47:42.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.952 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.952 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.952 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572304+0000", 2024-09-15T21:47:42.952 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3803000000000001e-05 2024-09-15T21:47:42.952 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.952 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.952 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.953 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.953 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:42.953 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:42.953 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.259958967999999, 2024-09-15T21:47:42.953 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055995389999999997, 2024-09-15T21:47:42.953 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.953 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.953 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:42.953 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.953 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.953 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.954 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:42.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.954 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.954 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.954 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:42.954 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.954 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.955 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.955 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:42.955 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9999999999999999e-07 2024-09-15T21:47:42.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.955 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.955 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:42.955 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2399999999999999e-06 2024-09-15T21:47:42.955 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.955 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.955 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.956 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565280+0000", 2024-09-15T21:47:42.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017929400000000001 2024-09-15T21:47:42.956 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.956 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.956 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.956 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565292+0000", 2024-09-15T21:47:42.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2193e-05 2024-09-15T21:47:42.956 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.956 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.956 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.956 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565357+0000", 2024-09-15T21:47:42.956 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4684000000000006e-05 2024-09-15T21:47:42.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.957 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565499+0000", 2024-09-15T21:47:42.957 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142032 2024-09-15T21:47:42.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.957 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:42.957 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565634+0000", 2024-09-15T21:47:42.957 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000135613 2024-09-15T21:47:42.957 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.957 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:42.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569941+0000", 2024-09-15T21:47:42.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043066789999999999 2024-09-15T21:47:42.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:42.958 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570585+0000", 2024-09-15T21:47:42.958 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00064364700000000001 2024-09-15T21:47:42.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.958 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.958 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:42.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570647+0000", 2024-09-15T21:47:42.959 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2173000000000003e-05 2024-09-15T21:47:42.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.959 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.959 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570697+0000", 2024-09-15T21:47:42.959 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9784000000000003e-05 2024-09-15T21:47:42.959 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.959 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.959 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.959 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.960 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:42.960 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:42.960 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.259954737999998, 2024-09-15T21:47:42.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066548939999999997, 2024-09-15T21:47:42.960 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:42.960 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:42.960 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:42.960 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:42.960 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.960 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:42.960 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:42.960 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.961 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:42.961 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:42.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:42.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.961 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:42.961 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565102+0000", 2024-09-15T21:47:42.961 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-15T21:47:42.961 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.961 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:42.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565103+0000", 2024-09-15T21:47:42.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6700000000000001e-06 2024-09-15T21:47:42.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:42.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565608+0000", 2024-09-15T21:47:42.962 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050518299999999998 2024-09-15T21:47:42.962 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.962 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.962 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:42.962 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565616+0000", 2024-09-15T21:47:42.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8199999999999997e-06 2024-09-15T21:47:42.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:42.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566598+0000", 2024-09-15T21:47:42.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00098176300000000008 2024-09-15T21:47:42.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.963 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.963 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:42.963 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566871+0000", 2024-09-15T21:47:42.963 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027312799999999998 2024-09-15T21:47:42.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:42.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567015+0000", 2024-09-15T21:47:42.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014428499999999999 2024-09-15T21:47:42.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.964 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:42.964 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570729+0000", 2024-09-15T21:47:42.964 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037132459999999999 2024-09-15T21:47:42.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.964 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.965 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:42.965 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571667+0000", 2024-09-15T21:47:42.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00093862299999999997 2024-09-15T21:47:42.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.965 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:42.965 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571718+0000", 2024-09-15T21:47:42.965 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1146999999999997e-05 2024-09-15T21:47:42.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:42.965 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:42.965 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:42.966 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571756+0000", 2024-09-15T21:47:42.966 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7821000000000003e-05 2024-09-15T21:47:42.966 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.966 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.966 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.966 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:42.966 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:42.966 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:42.968 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:47:42.968 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:42.966+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:43.069 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:43.067+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:43.070 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:43.071 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:43.071 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:43.071 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:43.079 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:47:43.169 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:43.168+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:43.182 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:43.183 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:43.183 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:43.183 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:43.183 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:43.191 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:47:43.270 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:43.268+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:43.297 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:43.297 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:43.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:43.297 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:43.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.297 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:47:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 90.060048315000003, 2024-09-15T21:47:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0944283340000001, 2024-09-15T21:47:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2024-09-15T21:47:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234360+0000", 2024-09-15T21:47:43.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3710000000000001e-06 2024-09-15T21:47:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234374+0000", 2024-09-15T21:47:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4003e-05 2024-09-15T21:47:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234392+0000", 2024-09-15T21:47:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7294000000000001e-05 2024-09-15T21:47:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.300 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234405+0000", 2024-09-15T21:47:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2961e-05 2024-09-15T21:47:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234483+0000", 2024-09-15T21:47:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8850000000000006e-05 2024-09-15T21:47:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-15T21:47:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234491+0000", 2024-09-15T21:47:43.301 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3529999999999997e-06 2024-09-15T21:47:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327253+0000", 2024-09-15T21:47:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.092761962 2024-09-15T21:47:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327456+0000", 2024-09-15T21:47:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000202893 2024-09-15T21:47:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.302 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327652+0000", 2024-09-15T21:47:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019613000000000001 2024-09-15T21:47:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328368+0000", 2024-09-15T21:47:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000716582 2024-09-15T21:47:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328682+0000", 2024-09-15T21:47:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000313825 2024-09-15T21:47:43.303 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.304 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328743+0000", 2024-09-15T21:47:43.304 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0571e-05 2024-09-15T21:47:43.304 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.304 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.304 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328787+0000", 2024-09-15T21:47:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4539000000000002e-05 2024-09-15T21:47:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.305 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 88.938882913, 2024-09-15T21:47:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0089551270000000002, 2024-09-15T21:47:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 35 2024-09-15T21:47:43.306 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.307 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.307 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.308 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355525+0000", 2024-09-15T21:47:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:47:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355606+0000", 2024-09-15T21:47:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1723000000000003e-05 2024-09-15T21:47:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355624+0000", 2024-09-15T21:47:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7645000000000001e-05 2024-09-15T21:47:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.309 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.310 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355629+0000", 2024-09-15T21:47:43.310 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7319999999999998e-06 2024-09-15T21:47:43.310 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358315+0000", 2024-09-15T21:47:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026858659999999999 2024-09-15T21:47:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.311 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358373+0000", 2024-09-15T21:47:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8593e-05 2024-09-15T21:47:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358521+0000", 2024-09-15T21:47:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014817500000000001 2024-09-15T21:47:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360353+0000", 2024-09-15T21:47:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018317100000000001 2024-09-15T21:47:43.312 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364424+0000", 2024-09-15T21:47:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040707970000000001 2024-09-15T21:47:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364452+0000", 2024-09-15T21:47:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7937999999999999e-05 2024-09-15T21:47:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.313 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364479+0000", 2024-09-15T21:47:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7744999999999999e-05 2024-09-15T21:47:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 88.938798950000006, 2024-09-15T21:47:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090267869999999997, 2024-09-15T21:47:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 36 2024-09-15T21:47:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.315 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355609+0000", 2024-09-15T21:47:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3099999999999998e-07 2024-09-15T21:47:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.316 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355612+0000", 2024-09-15T21:47:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1489999999999998e-06 2024-09-15T21:47:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355637+0000", 2024-09-15T21:47:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.508e-05 2024-09-15T21:47:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355640+0000", 2024-09-15T21:47:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7930000000000002e-06 2024-09-15T21:47:43.317 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359195+0000", 2024-09-15T21:47:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035555529999999999 2024-09-15T21:47:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359283+0000", 2024-09-15T21:47:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8042999999999997e-05 2024-09-15T21:47:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.318 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359419+0000", 2024-09-15T21:47:43.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013576699999999999 2024-09-15T21:47:43.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360361+0000", 2024-09-15T21:47:43.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094171000000000001 2024-09-15T21:47:43.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.319 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.319 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364536+0000", 2024-09-15T21:47:43.319 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00417484 2024-09-15T21:47:43.319 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.319 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364590+0000", 2024-09-15T21:47:43.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4355999999999999e-05 2024-09-15T21:47:43.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.320 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.320 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.320 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364635+0000", 2024-09-15T21:47:43.320 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5065000000000003e-05 2024-09-15T21:47:43.320 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.320 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.320 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.320 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 88.938794345000005, 2024-09-15T21:47:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091162750000000001, 2024-09-15T21:47:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 37 2024-09-15T21:47:43.321 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.322 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:47:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355615+0000", 2024-09-15T21:47:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-15T21:47:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355646+0000", 2024-09-15T21:47:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0391000000000001e-05 2024-09-15T21:47:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.323 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355648+0000", 2024-09-15T21:47:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6309999999999999e-06 2024-09-15T21:47:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359770+0000", 2024-09-15T21:47:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041222899999999998 2024-09-15T21:47:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359854+0000", 2024-09-15T21:47:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3399000000000006e-05 2024-09-15T21:47:43.324 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359988+0000", 2024-09-15T21:47:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013442200000000001 2024-09-15T21:47:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360524+0000", 2024-09-15T21:47:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00053582900000000004 2024-09-15T21:47:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364673+0000", 2024-09-15T21:47:43.325 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041493199999999997 2024-09-15T21:47:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364704+0000", 2024-09-15T21:47:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0380999999999999e-05 2024-09-15T21:47:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364729+0000", 2024-09-15T21:47:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5335999999999999e-05 2024-09-15T21:47:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.326 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 88.870077107, 2024-09-15T21:47:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0026270020000001, 2024-09-15T21:47:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 88 2024-09-15T21:47:43.327 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.328 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424331+0000", 2024-09-15T21:47:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4299999999999998e-07 2024-09-15T21:47:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424333+0000", 2024-09-15T21:47:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2359999999999999e-06 2024-09-15T21:47:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422895+0000", 2024-09-15T21:47:43.329 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99856184199999998 2024-09-15T21:47:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422902+0000", 2024-09-15T21:47:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4440000000000002e-06 2024-09-15T21:47:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423053+0000", 2024-09-15T21:47:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000150958 2024-09-15T21:47:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.330 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423160+0000", 2024-09-15T21:47:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106774 2024-09-15T21:47:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423344+0000", 2024-09-15T21:47:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018424199999999999 2024-09-15T21:47:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.424730+0000", 2024-09-15T21:47:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001386155 2024-09-15T21:47:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.331 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426756+0000", 2024-09-15T21:47:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020261150000000002 2024-09-15T21:47:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426906+0000", 2024-09-15T21:47:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000149176 2024-09-15T21:47:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.332 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426957+0000", 2024-09-15T21:47:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1616999999999997e-05 2024-09-15T21:47:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 87.740003908999995, 2024-09-15T21:47:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095037790000000004, 2024-09-15T21:47:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.333 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 114 2024-09-15T21:47:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.334 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554404+0000", 2024-09-15T21:47:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2399999999999999e-07 2024-09-15T21:47:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554407+0000", 2024-09-15T21:47:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-15T21:47:43.335 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554632+0000", 2024-09-15T21:47:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022568999999999999 2024-09-15T21:47:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554640+0000", 2024-09-15T21:47:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7970000000000001e-06 2024-09-15T21:47:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.336 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561325+0000", 2024-09-15T21:47:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066848020000000001 2024-09-15T21:47:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561402+0000", 2024-09-15T21:47:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6851e-05 2024-09-15T21:47:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561530+0000", 2024-09-15T21:47:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012863999999999999 2024-09-15T21:47:43.337 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563144+0000", 2024-09-15T21:47:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016132849999999999 2024-09-15T21:47:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563800+0000", 2024-09-15T21:47:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00065586099999999999 2024-09-15T21:47:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563857+0000", 2024-09-15T21:47:43.338 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.702e-05 2024-09-15T21:47:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563907+0000", 2024-09-15T21:47:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0479000000000002e-05 2024-09-15T21:47:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:43.339 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 87.739998912000004, 2024-09-15T21:47:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011813423999999999, 2024-09-15T21:47:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 115 2024-09-15T21:47:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:43.340 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554409+0000", 2024-09-15T21:47:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1100000000000002e-07 2024-09-15T21:47:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.341 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554498+0000", 2024-09-15T21:47:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9642999999999995e-05 2024-09-15T21:47:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554656+0000", 2024-09-15T21:47:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000158114 2024-09-15T21:47:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554661+0000", 2024-09-15T21:47:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0730000000000004e-06 2024-09-15T21:47:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.342 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561862+0000", 2024-09-15T21:47:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072002999999999998 2024-09-15T21:47:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561939+0000", 2024-09-15T21:47:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7265e-05 2024-09-15T21:47:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562073+0000", 2024-09-15T21:47:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013371600000000001 2024-09-15T21:47:43.343 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563196+0000", 2024-09-15T21:47:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011231679999999999 2024-09-15T21:47:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566124+0000", 2024-09-15T21:47:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002928403 2024-09-15T21:47:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.344 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566183+0000", 2024-09-15T21:47:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8328000000000001e-05 2024-09-15T21:47:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566222+0000", 2024-09-15T21:47:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9103000000000002e-05 2024-09-15T21:47:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:43.345 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 87.739905472000004, 2024-09-15T21:47:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01181426, 2024-09-15T21:47:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 116 2024-09-15T21:47:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.346 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554503+0000", 2024-09-15T21:47:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8000000000000005e-07 2024-09-15T21:47:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.347 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554506+0000", 2024-09-15T21:47:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5899999999999999e-06 2024-09-15T21:47:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554674+0000", 2024-09-15T21:47:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016763600000000001 2024-09-15T21:47:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554678+0000", 2024-09-15T21:47:43.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5310000000000003e-06 2024-09-15T21:47:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562714+0000", 2024-09-15T21:47:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.008035258 2024-09-15T21:47:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562765+0000", 2024-09-15T21:47:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1093e-05 2024-09-15T21:47:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.349 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562845+0000", 2024-09-15T21:47:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0427999999999996e-05 2024-09-15T21:47:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563943+0000", 2024-09-15T21:47:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097647 2024-09-15T21:47:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566232+0000", 2024-09-15T21:47:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022890850000000002 2024-09-15T21:47:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566279+0000", 2024-09-15T21:47:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6692e-05 2024-09-15T21:47:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566316+0000", 2024-09-15T21:47:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.752e-05 2024-09-15T21:47:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:43.351 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 87.739899373, 2024-09-15T21:47:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012232125, 2024-09-15T21:47:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 117 2024-09-15T21:47:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7700000000000001e-07 2024-09-15T21:47:43.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554511+0000", 2024-09-15T21:47:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8430000000000001e-06 2024-09-15T21:47:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554690+0000", 2024-09-15T21:47:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017856299999999999 2024-09-15T21:47:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554698+0000", 2024-09-15T21:47:43.354 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6200000000000005e-06 2024-09-15T21:47:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563224+0000", 2024-09-15T21:47:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0085262640000000004 2024-09-15T21:47:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563342+0000", 2024-09-15T21:47:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117687 2024-09-15T21:47:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563472+0000", 2024-09-15T21:47:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012957999999999999 2024-09-15T21:47:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566367+0000", 2024-09-15T21:47:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028952280000000001 2024-09-15T21:47:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566680+0000", 2024-09-15T21:47:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031345100000000002 2024-09-15T21:47:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566714+0000", 2024-09-15T21:47:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3701000000000001e-05 2024-09-15T21:47:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566740+0000", 2024-09-15T21:47:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5911e-05 2024-09-15T21:47:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 87.397007290999994, 2024-09-15T21:47:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069806590000000002, 2024-09-15T21:47:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 134 2024-09-15T21:47:43.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897401+0000", 2024-09-15T21:47:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0500000000000003e-07 2024-09-15T21:47:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897492+0000", 2024-09-15T21:47:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1770999999999999e-05 2024-09-15T21:47:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897750+0000", 2024-09-15T21:47:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025778299999999998 2024-09-15T21:47:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897756+0000", 2024-09-15T21:47:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2140000000000001e-06 2024-09-15T21:47:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899312+0000", 2024-09-15T21:47:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001556144 2024-09-15T21:47:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899372+0000", 2024-09-15T21:47:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0015000000000001e-05 2024-09-15T21:47:43.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899444+0000", 2024-09-15T21:47:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1923999999999998e-05 2024-09-15T21:47:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901354+0000", 2024-09-15T21:47:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019092530000000001 2024-09-15T21:47:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904327+0000", 2024-09-15T21:47:43.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029734520000000001 2024-09-15T21:47:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904360+0000", 2024-09-15T21:47:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3216000000000002e-05 2024-09-15T21:47:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904381+0000", 2024-09-15T21:47:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0282e-05 2024-09-15T21:47:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.363 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 87.396911434000003, 2024-09-15T21:47:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070121059999999997, 2024-09-15T21:47:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 135 2024-09-15T21:47:43.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897497+0000", 2024-09-15T21:47:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4799999999999997e-07 2024-09-15T21:47:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897500+0000", 2024-09-15T21:47:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2949999999999998e-06 2024-09-15T21:47:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897781+0000", 2024-09-15T21:47:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028064300000000002 2024-09-15T21:47:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.366 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897786+0000", 2024-09-15T21:47:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5169999999999999e-06 2024-09-15T21:47:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899857+0000", 2024-09-15T21:47:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020705340000000002 2024-09-15T21:47:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899932+0000", 2024-09-15T21:47:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5840999999999994e-05 2024-09-15T21:47:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900005+0000", 2024-09-15T21:47:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2716e-05 2024-09-15T21:47:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902296+0000", 2024-09-15T21:47:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022912010000000001 2024-09-15T21:47:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904424+0000", 2024-09-15T21:47:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021278600000000001 2024-09-15T21:47:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904472+0000", 2024-09-15T21:47:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7586000000000001e-05 2024-09-15T21:47:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904508+0000", 2024-09-15T21:47:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6164999999999997e-05 2024-09-15T21:47:43.369 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 87.396886942999998, 2024-09-15T21:47:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070611429999999998, 2024-09-15T21:47:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:43.370 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 136 2024-09-15T21:47:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897521+0000", 2024-09-15T21:47:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2900000000000004e-07 2024-09-15T21:47:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897527+0000", 2024-09-15T21:47:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8490000000000002e-06 2024-09-15T21:47:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.372 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897799+0000", 2024-09-15T21:47:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027219499999999998 2024-09-15T21:47:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897804+0000", 2024-09-15T21:47:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4219999999999999e-06 2024-09-15T21:47:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900248+0000", 2024-09-15T21:47:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002443457 2024-09-15T21:47:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900306+0000", 2024-09-15T21:47:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8263000000000001e-05 2024-09-15T21:47:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900385+0000", 2024-09-15T21:47:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9161999999999999e-05 2024-09-15T21:47:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902300+0000", 2024-09-15T21:47:43.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001914947 2024-09-15T21:47:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904536+0000", 2024-09-15T21:47:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022354419999999998 2024-09-15T21:47:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904562+0000", 2024-09-15T21:47:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6378999999999999e-05 2024-09-15T21:47:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904581+0000", 2024-09-15T21:47:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9497999999999999e-05 2024-09-15T21:47:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 87.396878376000004, 2024-09-15T21:47:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007180306, 2024-09-15T21:47:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.376 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 137 2024-09-15T21:47:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1399999999999998e-07 2024-09-15T21:47:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897612+0000", 2024-09-15T21:47:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2607000000000004e-05 2024-09-15T21:47:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897815+0000", 2024-09-15T21:47:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020352400000000001 2024-09-15T21:47:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897821+0000", 2024-09-15T21:47:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4670000000000003e-06 2024-09-15T21:47:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900665+0000", 2024-09-15T21:47:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028443689999999998 2024-09-15T21:47:43.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900714+0000", 2024-09-15T21:47:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8812e-05 2024-09-15T21:47:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900799+0000", 2024-09-15T21:47:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5129000000000004e-05 2024-09-15T21:47:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903413+0000", 2024-09-15T21:47:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026142890000000001 2024-09-15T21:47:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904625+0000", 2024-09-15T21:47:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012121149999999999 2024-09-15T21:47:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904673+0000", 2024-09-15T21:47:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7540999999999998e-05 2024-09-15T21:47:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904709+0000", 2024-09-15T21:47:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6139000000000001e-05 2024-09-15T21:47:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 87.396792421000001, 2024-09-15T21:47:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071727450000000003, 2024-09-15T21:47:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.382 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 138 2024-09-15T21:47:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:47:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897619+0000", 2024-09-15T21:47:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3869999999999999e-06 2024-09-15T21:47:43.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897832+0000", 2024-09-15T21:47:43.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021329500000000001 2024-09-15T21:47:43.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897837+0000", 2024-09-15T21:47:43.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3730000000000001e-06 2024-09-15T21:47:43.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901391+0000", 2024-09-15T21:47:43.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003553601 2024-09-15T21:47:43.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901466+0000", 2024-09-15T21:47:43.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5130999999999996e-05 2024-09-15T21:47:43.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901601+0000", 2024-09-15T21:47:43.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134797 2024-09-15T21:47:43.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903417+0000", 2024-09-15T21:47:43.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001816156 2024-09-15T21:47:43.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904741+0000", 2024-09-15T21:47:43.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013241049999999999 2024-09-15T21:47:43.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904767+0000", 2024-09-15T21:47:43.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5876000000000001e-05 2024-09-15T21:47:43.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904788+0000", 2024-09-15T21:47:43.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0449999999999999e-05 2024-09-15T21:47:43.388 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.388 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.388 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.388 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:43.388 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:43.388 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 87.396786767999998, 2024-09-15T21:47:43.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072970250000000004, 2024-09-15T21:47:43.389 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.389 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.389 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.389 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:43.389 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:43.389 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:43.389 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 139 2024-09-15T21:47:43.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.389 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:43.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:43.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897621+0000", 2024-09-15T21:47:43.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3099999999999999e-07 2024-09-15T21:47:43.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897625+0000", 2024-09-15T21:47:43.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.698e-06 2024-09-15T21:47:43.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897848+0000", 2024-09-15T21:47:43.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022375699999999999 2024-09-15T21:47:43.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.391 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897853+0000", 2024-09-15T21:47:43.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7709999999999997e-06 2024-09-15T21:47:43.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.391 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901929+0000", 2024-09-15T21:47:43.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040757780000000004 2024-09-15T21:47:43.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901976+0000", 2024-09-15T21:47:43.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6751e-05 2024-09-15T21:47:43.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902054+0000", 2024-09-15T21:47:43.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8856999999999994e-05 2024-09-15T21:47:43.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903421+0000", 2024-09-15T21:47:43.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00136619 2024-09-15T21:47:43.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904827+0000", 2024-09-15T21:47:43.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0014067369999999999 2024-09-15T21:47:43.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904881+0000", 2024-09-15T21:47:43.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3520999999999998e-05 2024-09-15T21:47:43.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904918+0000", 2024-09-15T21:47:43.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6634000000000002e-05 2024-09-15T21:47:43.394 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.394 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.394 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.394 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:43.394 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:43.394 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 87.396780938999996, 2024-09-15T21:47:43.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097221730000000006, 2024-09-15T21:47:43.395 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.395 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.395 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.395 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:43.395 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:43.395 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:43.395 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 140 2024-09-15T21:47:43.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.395 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:43.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:43.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897627+0000", 2024-09-15T21:47:43.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-15T21:47:43.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897698+0000", 2024-09-15T21:47:43.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1722000000000002e-05 2024-09-15T21:47:43.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897863+0000", 2024-09-15T21:47:43.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016479099999999999 2024-09-15T21:47:43.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.397 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.397 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897868+0000", 2024-09-15T21:47:43.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4950000000000002e-06 2024-09-15T21:47:43.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902335+0000", 2024-09-15T21:47:43.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004467632 2024-09-15T21:47:43.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902414+0000", 2024-09-15T21:47:43.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8688000000000003e-05 2024-09-15T21:47:43.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902535+0000", 2024-09-15T21:47:43.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012066900000000001 2024-09-15T21:47:43.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903714+0000", 2024-09-15T21:47:43.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011794520000000001 2024-09-15T21:47:43.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907115+0000", 2024-09-15T21:47:43.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034008879999999999 2024-09-15T21:47:43.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907250+0000", 2024-09-15T21:47:43.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134657 2024-09-15T21:47:43.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907348+0000", 2024-09-15T21:47:43.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8838000000000005e-05 2024-09-15T21:47:43.400 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.400 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.400 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.400 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:43.400 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:43.401 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 87.396705858000004, 2024-09-15T21:47:43.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097618050000000001, 2024-09-15T21:47:43.401 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.401 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.401 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.401 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:43.401 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:43.401 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:43.401 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 141 2024-09-15T21:47:43.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.401 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:43.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:43.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897702+0000", 2024-09-15T21:47:43.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6199999999999998e-07 2024-09-15T21:47:43.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897705+0000", 2024-09-15T21:47:43.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0180000000000002e-06 2024-09-15T21:47:43.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897889+0000", 2024-09-15T21:47:43.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000183678 2024-09-15T21:47:43.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.403 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.403 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897894+0000", 2024-09-15T21:47:43.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8930000000000004e-06 2024-09-15T21:47:43.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903099+0000", 2024-09-15T21:47:43.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052056280000000003 2024-09-15T21:47:43.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903151+0000", 2024-09-15T21:47:43.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2163000000000002e-05 2024-09-15T21:47:43.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903220+0000", 2024-09-15T21:47:43.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8483000000000001e-05 2024-09-15T21:47:43.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904794+0000", 2024-09-15T21:47:43.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001574521 2024-09-15T21:47:43.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907405+0000", 2024-09-15T21:47:43.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026110339999999999 2024-09-15T21:47:43.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907440+0000", 2024-09-15T21:47:43.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4564999999999999e-05 2024-09-15T21:47:43.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907463+0000", 2024-09-15T21:47:43.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3260000000000001e-05 2024-09-15T21:47:43.406 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.407 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.407 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.407 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:22 2.8 2:1470b2f8:::613.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:43.407 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:43.407 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.839265530999995, 2024-09-15T21:47:43.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007454937, 2024-09-15T21:47:43.407 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.408 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.408 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.408 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:43.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:43.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:43.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6800000000000002e-07 2024-09-15T21:47:43.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.409 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455145+0000", 2024-09-15T21:47:43.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4470000000000002e-06 2024-09-15T21:47:43.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456425+0000", 2024-09-15T21:47:43.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012798200000000001 2024-09-15T21:47:43.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456433+0000", 2024-09-15T21:47:43.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7389999999999999e-06 2024-09-15T21:47:43.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097865 2024-09-15T21:47:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457614+0000", 2024-09-15T21:47:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3560000000000006e-05 2024-09-15T21:47:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457998+0000", 2024-09-15T21:47:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00038372699999999999 2024-09-15T21:47:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460488+0000", 2024-09-15T21:47:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002489855 2024-09-15T21:47:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462512+0000", 2024-09-15T21:47:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020235269999999998 2024-09-15T21:47:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462554+0000", 2024-09-15T21:47:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2472000000000003e-05 2024-09-15T21:47:43.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462597+0000", 2024-09-15T21:47:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2657000000000002e-05 2024-09-15T21:47:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:2 2.5 2:a6e36b3f:::401.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:43.413 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.839260562000007, 2024-09-15T21:47:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074484590000000002, 2024-09-15T21:47:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:47:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455150+0000", 2024-09-15T21:47:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7319999999999998e-06 2024-09-15T21:47:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.415 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456449+0000", 2024-09-15T21:47:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012991490000000001 2024-09-15T21:47:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456454+0000", 2024-09-15T21:47:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0390000000000004e-06 2024-09-15T21:47:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456513+0000", 2024-09-15T21:47:43.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8768999999999999e-05 2024-09-15T21:47:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456628+0000", 2024-09-15T21:47:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115641 2024-09-15T21:47:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456741+0000", 2024-09-15T21:47:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000113094 2024-09-15T21:47:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458520+0000", 2024-09-15T21:47:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001778993 2024-09-15T21:47:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462494+0000", 2024-09-15T21:47:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039737009999999996 2024-09-15T21:47:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462549+0000", 2024-09-15T21:47:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4537000000000003e-05 2024-09-15T21:47:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462595+0000", 2024-09-15T21:47:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6511000000000002e-05 2024-09-15T21:47:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:27 2.5 2:a5fa685f:::504.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 51.729568178000001, 2024-09-15T21:47:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071804599999999996, 2024-09-15T21:47:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:43.419 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-15T21:47:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564841+0000", 2024-09-15T21:47:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-15T21:47:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565126+0000", 2024-09-15T21:47:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028469900000000002 2024-09-15T21:47:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565133+0000", 2024-09-15T21:47:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9480000000000002e-06 2024-09-15T21:47:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568006+0000", 2024-09-15T21:47:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028730330000000001 2024-09-15T21:47:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568119+0000", 2024-09-15T21:47:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011325899999999999 2024-09-15T21:47:43.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:43.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568235+0000", 2024-09-15T21:47:43.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115868 2024-09-15T21:47:43.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:43.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569668+0000", 2024-09-15T21:47:43.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001433099 2024-09-15T21:47:43.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571926+0000", 2024-09-15T21:47:43.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022572410000000001 2024-09-15T21:47:43.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:43.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571981+0000", 2024-09-15T21:47:43.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5383000000000002e-05 2024-09-15T21:47:43.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:43.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:43.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:43.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572020+0000", 2024-09-15T21:47:43.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8574e-05 2024-09-15T21:47:43.424 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.424 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.424 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.425 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:43.425 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:43.425 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:43.426 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:47:43.427 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:43.424+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:43.527 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:43.526+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:43.536 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:43.536 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:43.536 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:43.536 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:43.545 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:47:43.629 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:43.628+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:43.729 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:43.727+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:43.748 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:43.749 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:43.749 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:43.749 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:43.749 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:43.761 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:47:43.829 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:43.828+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:43.912 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:43.912 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:43.912 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:43.912 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:43.913 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.913 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:32 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'18, mlcod=17'14)", 2024-09-15T21:47:43.913 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:43.913 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 89.552894373000001, 2024-09-15T21:47:43.913 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052297760000000002, 2024-09-15T21:47:43.913 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:43.913 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:43.913 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.913 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:43.913 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.913 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:43.913 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:43.914 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.914 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.914 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:43.914 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:43.914 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.914 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.914 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:43.914 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:43.914 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.06e-07 2024-09-15T21:47:43.914 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.914 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.914 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:43.915 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356780+0000", 2024-09-15T21:47:43.915 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4208999999999994e-05 2024-09-15T21:47:43.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.915 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:43.915 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356789+0000", 2024-09-15T21:47:43.915 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2800000000000003e-06 2024-09-15T21:47:43.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.915 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.915 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356793+0000", 2024-09-15T21:47:43.915 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:47:43.915 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.915 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.916 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:43.916 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356838+0000", 2024-09-15T21:47:43.916 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4851999999999997e-05 2024-09-15T21:47:43.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.916 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.916 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:43.916 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356854+0000", 2024-09-15T21:47:43.916 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6124999999999999e-05 2024-09-15T21:47:43.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.916 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.916 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:43.916 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.361892+0000", 2024-09-15T21:47:43.916 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050379140000000001 2024-09-15T21:47:43.916 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.917 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:43.917 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.361935+0000", 2024-09-15T21:47:43.917 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3217999999999999e-05 2024-09-15T21:47:43.917 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.917 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:43.917 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.917 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.917 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.917 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:35 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'21, mlcod=17'15)", 2024-09-15T21:47:43.917 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:43.917 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 89.550857242999996, 2024-09-15T21:47:43.917 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055652039999999998, 2024-09-15T21:47:43.918 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:43.918 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:43.918 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.918 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:43.918 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.918 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:43.918 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:43.918 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.918 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.918 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.918 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:43.918 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:43.918 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.918 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.919 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:43.919 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358744+0000", 2024-09-15T21:47:43.919 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4399999999999998e-07 2024-09-15T21:47:43.919 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.919 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:43.919 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358773+0000", 2024-09-15T21:47:43.919 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8847e-05 2024-09-15T21:47:43.919 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.919 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.920 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:43.920 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358783+0000", 2024-09-15T21:47:43.920 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0299000000000001e-05 2024-09-15T21:47:43.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.920 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.920 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.920 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358796+0000", 2024-09-15T21:47:43.920 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3499000000000001e-05 2024-09-15T21:47:43.920 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.920 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.920 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:43.920 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358850+0000", 2024-09-15T21:47:43.921 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3952000000000001e-05 2024-09-15T21:47:43.921 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.921 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.921 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:43.921 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358872+0000", 2024-09-15T21:47:43.921 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1424999999999999e-05 2024-09-15T21:47:43.922 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.922 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.922 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:43.922 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364273+0000", 2024-09-15T21:47:43.922 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054011299999999996 2024-09-15T21:47:43.922 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.922 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.922 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:43.922 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364308+0000", 2024-09-15T21:47:43.922 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5108000000000003e-05 2024-09-15T21:47:43.922 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.922 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:43.922 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.922 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.923 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.923 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:36 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'22, mlcod=17'15)", 2024-09-15T21:47:43.923 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:43.923 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 89.550063053000002, 2024-09-15T21:47:43.923 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047904530000000001, 2024-09-15T21:47:43.923 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:43.923 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:43.923 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.923 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:43.923 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.924 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:43.924 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:43.924 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.924 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.924 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.924 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:43.924 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:43.924 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.924 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.924 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.924 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:43.924 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:43.924 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6000000000000004e-07 2024-09-15T21:47:43.924 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.925 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.925 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:43.925 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359542+0000", 2024-09-15T21:47:43.925 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3510000000000004e-06 2024-09-15T21:47:43.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.925 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.925 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:43.925 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359552+0000", 2024-09-15T21:47:43.925 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0263e-05 2024-09-15T21:47:43.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.925 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.925 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.925 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359557+0000", 2024-09-15T21:47:43.925 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0479999999999998e-06 2024-09-15T21:47:43.925 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.926 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.926 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:43.926 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359593+0000", 2024-09-15T21:47:43.926 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6151e-05 2024-09-15T21:47:43.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.926 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.926 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:43.926 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359608+0000", 2024-09-15T21:47:43.926 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4609e-05 2024-09-15T21:47:43.926 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.926 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.926 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:43.926 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364310+0000", 2024-09-15T21:47:43.926 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047025510000000001 2024-09-15T21:47:43.927 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.927 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.927 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:43.927 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364327+0000", 2024-09-15T21:47:43.927 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6920000000000001e-05 2024-09-15T21:47:43.927 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.927 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:43.927 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.927 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.927 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.927 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=18'85)", 2024-09-15T21:47:43.928 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:43.928 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 88.011517118, 2024-09-15T21:47:43.928 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057443499999999996, 2024-09-15T21:47:43.928 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:43.928 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:43.928 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.928 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:43.928 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.928 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:43.928 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:43.929 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.929 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.929 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.929 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:43.929 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:43.929 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.929 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.929 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.929 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:43.929 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:43.929 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8899999999999999e-07 2024-09-15T21:47:43.929 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.930 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.930 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:43.930 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898167+0000", 2024-09-15T21:47:43.930 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3171999999999996e-05 2024-09-15T21:47:43.930 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.930 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.930 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:43.930 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898177+0000", 2024-09-15T21:47:43.930 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0866000000000001e-05 2024-09-15T21:47:43.930 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.930 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.930 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.930 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898185+0000", 2024-09-15T21:47:43.930 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2490000000000004e-06 2024-09-15T21:47:43.931 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.931 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.931 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:43.931 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898224+0000", 2024-09-15T21:47:43.931 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9237999999999999e-05 2024-09-15T21:47:43.931 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.931 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.931 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:43.931 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898238+0000", 2024-09-15T21:47:43.931 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.366e-05 2024-09-15T21:47:43.931 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.932 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.932 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:43.932 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.903803+0000", 2024-09-15T21:47:43.932 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005565291 2024-09-15T21:47:43.932 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.932 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.932 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:43.932 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.903827+0000", 2024-09-15T21:47:43.932 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4284999999999999e-05 2024-09-15T21:47:43.932 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.932 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:43.933 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.933 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.933 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.933 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:132 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'90, mlcod=18'85)", 2024-09-15T21:47:43.933 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:43.933 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 88.011025394000001, 2024-09-15T21:47:43.933 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055085469999999999, 2024-09-15T21:47:43.933 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:43.933 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:43.933 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.933 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:43.933 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.934 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:43.934 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:43.934 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.934 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.934 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.934 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:43.934 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:43.934 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.934 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.934 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.934 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:43.934 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:43.934 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9299999999999999e-07 2024-09-15T21:47:43.934 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.935 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.935 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:43.935 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898579+0000", 2024-09-15T21:47:43.935 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6650000000000001e-06 2024-09-15T21:47:43.935 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.935 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.935 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:43.935 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898586+0000", 2024-09-15T21:47:43.935 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8429999999999993e-06 2024-09-15T21:47:43.935 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.935 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.935 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.935 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898592+0000", 2024-09-15T21:47:43.935 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3990000000000003e-06 2024-09-15T21:47:43.936 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.936 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.936 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:43.936 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898621+0000", 2024-09-15T21:47:43.936 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9019000000000001e-05 2024-09-15T21:47:43.936 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.936 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.936 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:43.936 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898630+0000", 2024-09-15T21:47:43.936 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5540000000000003e-06 2024-09-15T21:47:43.936 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.936 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.936 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:43.936 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904047+0000", 2024-09-15T21:47:43.937 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054167169999999997 2024-09-15T21:47:43.937 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.937 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.937 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:43.937 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904083+0000", 2024-09-15T21:47:43.937 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5957000000000001e-05 2024-09-15T21:47:43.937 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.937 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:43.937 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.937 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.937 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.937 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-15T21:47:43.937 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:43.937 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 88.010436244999994, 2024-09-15T21:47:43.938 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049424780000000001, 2024-09-15T21:47:43.938 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:43.938 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:43.938 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.938 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:43.938 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.938 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:43.938 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:43.938 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.938 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.938 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.938 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:43.938 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:43.938 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.939 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.939 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.939 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:43.939 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899166+0000", 2024-09-15T21:47:43.939 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0779999999999998e-06 2024-09-15T21:47:43.939 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.939 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.939 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:43.939 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899174+0000", 2024-09-15T21:47:43.939 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.729e-06 2024-09-15T21:47:43.939 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.939 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.939 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:43.939 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899181+0000", 2024-09-15T21:47:43.940 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4080000000000001e-06 2024-09-15T21:47:43.940 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.940 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.940 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.940 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899185+0000", 2024-09-15T21:47:43.940 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0840000000000002e-06 2024-09-15T21:47:43.940 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.940 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.940 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:43.940 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899240+0000", 2024-09-15T21:47:43.940 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4874999999999999e-05 2024-09-15T21:47:43.940 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.940 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.941 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:43.941 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899257+0000", 2024-09-15T21:47:43.941 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7549000000000001e-05 2024-09-15T21:47:43.941 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.941 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.941 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:43.941 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904085+0000", 2024-09-15T21:47:43.941 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048272640000000004 2024-09-15T21:47:43.941 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.941 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.941 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:43.941 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904106+0000", 2024-09-15T21:47:43.941 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1491000000000001e-05 2024-09-15T21:47:43.941 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.942 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:43.942 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.942 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.942 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.942 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:43.942 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:43.942 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 75.496212968999998, 2024-09-15T21:47:43.942 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046608459999999997, 2024-09-15T21:47:43.942 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:43.942 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:43.942 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.942 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:43.942 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.942 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:43.943 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:43.943 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.943 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.943 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.943 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:43.943 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:43.943 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.943 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.943 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.943 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:43.943 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413391+0000", 2024-09-15T21:47:43.943 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9899999999999999e-06 2024-09-15T21:47:43.943 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.944 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.944 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:43.944 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413399+0000", 2024-09-15T21:47:43.944 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6310000000000003e-06 2024-09-15T21:47:43.944 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.944 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.944 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:43.944 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413444+0000", 2024-09-15T21:47:43.944 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5052000000000002e-05 2024-09-15T21:47:43.944 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.944 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.944 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.944 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413453+0000", 2024-09-15T21:47:43.944 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8869999999999998e-06 2024-09-15T21:47:43.945 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.945 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.945 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:43.945 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413576+0000", 2024-09-15T21:47:43.945 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012358700000000001 2024-09-15T21:47:43.945 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.945 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.945 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:43.945 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413679+0000", 2024-09-15T21:47:43.945 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000103318 2024-09-15T21:47:43.945 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.945 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.945 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:43.945 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413785+0000", 2024-09-15T21:47:43.946 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010561 2024-09-15T21:47:43.946 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.946 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.946 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:43.946 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416276+0000", 2024-09-15T21:47:43.946 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002490754 2024-09-15T21:47:43.946 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.946 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.946 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.946 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417949+0000", 2024-09-15T21:47:43.946 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016733519999999999 2024-09-15T21:47:43.946 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.946 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.947 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:43.947 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418005+0000", 2024-09-15T21:47:43.947 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5747e-05 2024-09-15T21:47:43.947 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.947 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.947 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:43.947 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418048+0000", 2024-09-15T21:47:43.947 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2917999999999998e-05 2024-09-15T21:47:43.947 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.947 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:43.947 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.947 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.947 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.947 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:43.947 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:43.948 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 75.496198499000002, 2024-09-15T21:47:43.948 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051419999999999999, 2024-09-15T21:47:43.948 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:43.948 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:43.948 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.948 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:43.948 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.948 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:43.948 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:43.948 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.948 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.948 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.948 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:43.949 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:43.949 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.949 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.949 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.949 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:43.949 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413402+0000", 2024-09-15T21:47:43.949 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6899999999999998e-07 2024-09-15T21:47:43.949 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.949 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.949 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:43.949 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413405+0000", 2024-09-15T21:47:43.949 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8109999999999999e-06 2024-09-15T21:47:43.949 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.949 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.949 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:43.950 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413517+0000", 2024-09-15T21:47:43.950 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000112592 2024-09-15T21:47:43.950 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.950 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.950 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.950 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413527+0000", 2024-09-15T21:47:43.950 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0144e-05 2024-09-15T21:47:43.950 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.950 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.950 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:43.950 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414155+0000", 2024-09-15T21:47:43.950 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00062767200000000002 2024-09-15T21:47:43.950 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.950 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.951 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:43.951 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414239+0000", 2024-09-15T21:47:43.951 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3607000000000001e-05 2024-09-15T21:47:43.951 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.951 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.951 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:43.951 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414368+0000", 2024-09-15T21:47:43.951 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000129049 2024-09-15T21:47:43.951 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.951 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.951 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:43.951 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416332+0000", 2024-09-15T21:47:43.951 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019645109999999999 2024-09-15T21:47:43.951 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.951 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.952 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.952 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418446+0000", 2024-09-15T21:47:43.952 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021137640000000002 2024-09-15T21:47:43.952 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.952 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.952 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:43.952 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418503+0000", 2024-09-15T21:47:43.952 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7166999999999997e-05 2024-09-15T21:47:43.952 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.952 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.952 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:43.952 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418543+0000", 2024-09-15T21:47:43.952 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0314000000000001e-05 2024-09-15T21:47:43.953 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.953 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:43.953 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.953 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.953 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.953 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:43.953 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:43.953 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 75.496185307999994, 2024-09-15T21:47:43.953 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057945979999999998, 2024-09-15T21:47:43.953 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:43.953 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:43.953 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.953 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:43.953 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.953 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:43.954 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:43.954 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.954 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.954 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.954 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:43.954 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:43.954 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.954 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.954 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.954 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:43.954 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:43.954 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9499999999999998e-07 2024-09-15T21:47:43.954 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.954 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.955 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:43.955 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413417+0000", 2024-09-15T21:47:43.955 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3910000000000001e-06 2024-09-15T21:47:43.955 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.955 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.955 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:43.955 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414570+0000", 2024-09-15T21:47:43.955 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011525890000000001 2024-09-15T21:47:43.955 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.955 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.956 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.956 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414577+0000", 2024-09-15T21:47:43.956 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1840000000000002e-06 2024-09-15T21:47:43.956 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.956 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.956 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:43.956 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414733+0000", 2024-09-15T21:47:43.956 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015563900000000001 2024-09-15T21:47:43.956 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.956 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.956 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:43.956 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414831+0000", 2024-09-15T21:47:43.956 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8138000000000001e-05 2024-09-15T21:47:43.957 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.957 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.957 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:43.957 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414956+0000", 2024-09-15T21:47:43.957 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000124604 2024-09-15T21:47:43.957 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.957 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.957 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.957 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418804+0000", 2024-09-15T21:47:43.957 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038484299999999999 2024-09-15T21:47:43.957 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.957 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.957 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:43.958 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419124+0000", 2024-09-15T21:47:43.958 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00032017699999999999 2024-09-15T21:47:43.958 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.958 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.958 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:43.958 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419171+0000", 2024-09-15T21:47:43.958 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6520000000000002e-05 2024-09-15T21:47:43.958 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.958 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.958 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:43.958 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419209+0000", 2024-09-15T21:47:43.958 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8531000000000001e-05 2024-09-15T21:47:43.958 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.958 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:43.958 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.959 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.959 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.959 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:43.959 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:43.959 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 75.496178212000004, 2024-09-15T21:47:43.959 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059183300000000003, 2024-09-15T21:47:43.959 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:43.959 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:43.959 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:43.959 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:43.959 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.959 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:43.959 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:43.959 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.960 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.960 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.960 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:43.960 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:43.960 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.960 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.960 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.960 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:43.960 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:43.960 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3000000000000002e-07 2024-09-15T21:47:43.960 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.960 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.960 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:43.961 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413426+0000", 2024-09-15T21:47:43.961 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7110000000000001e-06 2024-09-15T21:47:43.961 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.961 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.961 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:43.961 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414621+0000", 2024-09-15T21:47:43.961 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001195022 2024-09-15T21:47:43.961 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.961 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.961 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.961 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414625+0000", 2024-09-15T21:47:43.961 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6360000000000002e-06 2024-09-15T21:47:43.961 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.961 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.962 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:43.962 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415269+0000", 2024-09-15T21:47:43.962 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00064315399999999995 2024-09-15T21:47:43.962 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.962 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.962 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:43.962 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415356+0000", 2024-09-15T21:47:43.962 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7410999999999994e-05 2024-09-15T21:47:43.962 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.962 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.962 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:43.962 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419340+0000", 2024-09-15T21:47:43.962 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0039840659999999996 2024-09-15T21:47:43.962 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.962 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:43.963 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.963 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.963 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.963 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-15T21:47:43.963 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:43.963 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 75.495870620999995, 2024-09-15T21:47:43.963 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053379719999999999, 2024-09-15T21:47:43.963 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:43.963 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:43.963 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.963 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:43.963 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.963 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:43.963 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:43.964 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.964 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.964 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.964 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:43.964 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:43.964 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.964 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.964 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.964 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:43.964 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413730+0000", 2024-09-15T21:47:43.964 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1379999999999999e-06 2024-09-15T21:47:43.964 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.964 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.964 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:43.965 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413740+0000", 2024-09-15T21:47:43.965 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4830000000000005e-06 2024-09-15T21:47:43.965 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.965 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.965 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:43.965 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413751+0000", 2024-09-15T21:47:43.965 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1228e-05 2024-09-15T21:47:43.965 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.965 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.965 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.965 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413758+0000", 2024-09-15T21:47:43.965 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1289999999999998e-06 2024-09-15T21:47:43.965 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.965 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.966 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:43.966 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414038+0000", 2024-09-15T21:47:43.966 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00027935999999999998 2024-09-15T21:47:43.966 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.966 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.966 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:43.966 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414055+0000", 2024-09-15T21:47:43.966 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7325000000000001e-05 2024-09-15T21:47:43.966 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.966 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.966 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:43.966 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419028+0000", 2024-09-15T21:47:43.966 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049730720000000003 2024-09-15T21:47:43.966 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.966 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.967 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:43.967 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419067+0000", 2024-09-15T21:47:43.967 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9236999999999997e-05 2024-09-15T21:47:43.967 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.967 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:43.967 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.967 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.967 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.967 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:43.967 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:43.967 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 69.453947353999993, 2024-09-15T21:47:43.967 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.92695576499999999, 2024-09-15T21:47:43.967 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:43.967 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:43.968 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.968 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:43.968 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.968 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:43.968 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:43.968 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.968 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.968 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.968 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:43.968 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:43.968 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.968 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.968 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.969 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:43.969 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:43.969 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8500000000000002e-07 2024-09-15T21:47:43.969 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.969 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.969 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:43.969 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455655+0000", 2024-09-15T21:47:43.969 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.401e-06 2024-09-15T21:47:43.969 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.969 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.969 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:43.969 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.380965+0000", 2024-09-15T21:47:43.969 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.92530946000000003 2024-09-15T21:47:43.969 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.970 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.970 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.970 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.380981+0000", 2024-09-15T21:47:43.970 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6209e-05 2024-09-15T21:47:43.970 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.970 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.970 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:43.970 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381086+0000", 2024-09-15T21:47:43.970 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105011 2024-09-15T21:47:43.970 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.970 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.970 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:43.970 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381154+0000", 2024-09-15T21:47:43.970 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7596999999999997e-05 2024-09-15T21:47:43.971 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.971 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.971 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:43.971 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381249+0000", 2024-09-15T21:47:43.971 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5354999999999998e-05 2024-09-15T21:47:43.971 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.971 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.971 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:43.971 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381844+0000", 2024-09-15T21:47:43.971 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00059518600000000004 2024-09-15T21:47:43.971 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.971 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.971 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.971 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382531+0000", 2024-09-15T21:47:43.972 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00068671499999999996 2024-09-15T21:47:43.972 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.972 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.972 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:43.972 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382570+0000", 2024-09-15T21:47:43.972 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9373999999999999e-05 2024-09-15T21:47:43.972 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.972 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.972 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:43.972 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382608+0000", 2024-09-15T21:47:43.973 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8071999999999997e-05 2024-09-15T21:47:43.973 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.973 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:43.973 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.973 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.973 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.973 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:43.973 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:43.973 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 63.434038094999998, 2024-09-15T21:47:43.973 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047117319999999997, 2024-09-15T21:47:43.973 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:43.973 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:43.973 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.973 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:43.974 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.974 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:43.974 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:43.974 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.974 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.974 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.974 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:43.974 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:43.974 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.974 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.974 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.974 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:43.974 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:43.974 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4700000000000001e-07 2024-09-15T21:47:43.974 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.975 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.975 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:43.975 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475566+0000", 2024-09-15T21:47:43.975 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1669999999999999e-06 2024-09-15T21:47:43.975 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.975 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.975 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:43.975 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476316+0000", 2024-09-15T21:47:43.975 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075022999999999995 2024-09-15T21:47:43.975 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.975 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.975 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.975 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476324+0000", 2024-09-15T21:47:43.975 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.259e-06 2024-09-15T21:47:43.976 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.976 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.976 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:43.976 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476373+0000", 2024-09-15T21:47:43.976 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8807999999999998e-05 2024-09-15T21:47:43.976 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.976 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.976 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:43.976 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476446+0000", 2024-09-15T21:47:43.976 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3138000000000003e-05 2024-09-15T21:47:43.976 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.976 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.976 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:43.976 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476593+0000", 2024-09-15T21:47:43.977 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00014680199999999999 2024-09-15T21:47:43.977 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.977 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.977 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.977 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480189+0000", 2024-09-15T21:47:43.977 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035961370000000001 2024-09-15T21:47:43.977 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.977 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.977 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:43.977 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480198+0000", 2024-09-15T21:47:43.977 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.6649999999999992e-06 2024-09-15T21:47:43.977 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.977 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.977 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:43.978 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480235+0000", 2024-09-15T21:47:43.978 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6943999999999997e-05 2024-09-15T21:47:43.978 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.978 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.978 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:43.978 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480274+0000", 2024-09-15T21:47:43.978 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9035000000000001e-05 2024-09-15T21:47:43.978 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.978 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:43.978 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.978 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.978 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.978 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:43.978 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:43.978 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 63.434033092, 2024-09-15T21:47:43.979 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046830109999999999, 2024-09-15T21:47:43.979 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:43.979 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:43.979 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.979 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:43.979 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.979 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:43.979 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:43.979 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.979 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.979 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.979 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:43.979 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:43.979 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.980 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.980 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.980 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:43.980 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:43.980 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.53e-07 2024-09-15T21:47:43.980 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.980 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.980 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:43.980 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475570+0000", 2024-09-15T21:47:43.980 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7470000000000001e-06 2024-09-15T21:47:43.980 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.980 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.980 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:43.980 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476353+0000", 2024-09-15T21:47:43.980 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00078312300000000004 2024-09-15T21:47:43.981 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.981 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.981 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.981 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476359+0000", 2024-09-15T21:47:43.981 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.117e-06 2024-09-15T21:47:43.981 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.981 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.981 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:43.981 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476412+0000", 2024-09-15T21:47:43.981 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2398999999999998e-05 2024-09-15T21:47:43.981 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.981 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.981 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:43.981 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476524+0000", 2024-09-15T21:47:43.982 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011293 2024-09-15T21:47:43.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.982 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.982 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:43.982 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476909+0000", 2024-09-15T21:47:43.982 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00038490099999999999 2024-09-15T21:47:43.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.982 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.982 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.982 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479761+0000", 2024-09-15T21:47:43.982 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002851737 2024-09-15T21:47:43.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.982 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.982 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:43.983 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480154+0000", 2024-09-15T21:47:43.983 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00039307399999999998 2024-09-15T21:47:43.983 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.983 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.983 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:43.983 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480203+0000", 2024-09-15T21:47:43.983 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8418000000000003e-05 2024-09-15T21:47:43.983 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.983 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.983 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:43.983 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480250+0000", 2024-09-15T21:47:43.983 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7311999999999997e-05 2024-09-15T21:47:43.983 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.983 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:43.983 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.984 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.984 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.984 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:43.984 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:43.984 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 63.434028929999997, 2024-09-15T21:47:43.984 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005035398, 2024-09-15T21:47:43.984 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:43.984 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:43.984 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.984 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:43.984 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.984 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:43.984 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:43.985 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.985 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.985 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.985 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:43.985 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:43.985 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.985 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.985 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.985 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:43.985 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:43.985 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.22e-07 2024-09-15T21:47:43.985 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.985 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.985 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:43.985 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475574+0000", 2024-09-15T21:47:43.986 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.593e-06 2024-09-15T21:47:43.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.986 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.986 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:43.986 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476579+0000", 2024-09-15T21:47:43.986 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010053989999999999 2024-09-15T21:47:43.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.986 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.986 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.986 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476588+0000", 2024-09-15T21:47:43.986 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4530000000000002e-06 2024-09-15T21:47:43.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.986 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.986 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:43.987 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477723+0000", 2024-09-15T21:47:43.987 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001134962 2024-09-15T21:47:43.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.987 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.987 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:43.987 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477801+0000", 2024-09-15T21:47:43.987 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8228999999999999e-05 2024-09-15T21:47:43.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.987 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.987 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:43.987 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477946+0000", 2024-09-15T21:47:43.987 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00014553400000000001 2024-09-15T21:47:43.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.987 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.987 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:43.988 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480258+0000", 2024-09-15T21:47:43.988 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023112039999999999 2024-09-15T21:47:43.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.988 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.988 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.988 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480504+0000", 2024-09-15T21:47:43.988 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024667500000000002 2024-09-15T21:47:43.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.988 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.988 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:43.988 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480556+0000", 2024-09-15T21:47:43.988 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2114999999999998e-05 2024-09-15T21:47:43.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.988 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.989 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:43.989 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480606+0000", 2024-09-15T21:47:43.989 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0012000000000002e-05 2024-09-15T21:47:43.989 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.989 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:43.989 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.989 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.989 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.989 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:43.989 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:43.989 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 63.434024999999998, 2024-09-15T21:47:43.989 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050672490000000002, 2024-09-15T21:47:43.989 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:43.989 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:43.990 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475577+0000", 2024-09-15T21:47:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2290000000000002e-06 2024-09-15T21:47:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476605+0000", 2024-09-15T21:47:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010278780000000001 2024-09-15T21:47:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.991 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476611+0000", 2024-09-15T21:47:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5600000000000001e-06 2024-09-15T21:47:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477053+0000", 2024-09-15T21:47:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044162800000000001 2024-09-15T21:47:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477180+0000", 2024-09-15T21:47:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000127129 2024-09-15T21:47:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.992 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.993 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:43.993 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477288+0000", 2024-09-15T21:47:43.993 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010789999999999999 2024-09-15T21:47:43.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.993 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.993 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:43.993 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480288+0000", 2024-09-15T21:47:43.993 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003000052 2024-09-15T21:47:43.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.993 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.993 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.993 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480568+0000", 2024-09-15T21:47:43.993 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028026899999999998 2024-09-15T21:47:43.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.993 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.994 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:43.994 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480609+0000", 2024-09-15T21:47:43.994 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1437999999999999e-05 2024-09-15T21:47:43.994 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.994 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.994 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:43.994 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480642+0000", 2024-09-15T21:47:43.994 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2879999999999997e-05 2024-09-15T21:47:43.994 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.994 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:43.994 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:43.994 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.994 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.994 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:20 2.1f 2:fdd45aa0:::61c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:43.995 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:43.995 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 63.434021328999997, 2024-09-15T21:47:43.995 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053112589999999996, 2024-09-15T21:47:43.995 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:43.995 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:43.995 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:43.995 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:43.995 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.995 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:43.995 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:43.995 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.995 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.995 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.995 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:43.995 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:43.996 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:43.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.996 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.996 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:43.996 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:43.996 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:43.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.996 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.996 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:43.996 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475581+0000", 2024-09-15T21:47:43.996 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1009999999999999e-06 2024-09-15T21:47:43.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.996 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.996 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:43.997 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476628+0000", 2024-09-15T21:47:43.997 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010465380000000001 2024-09-15T21:47:43.997 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.997 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.997 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:43.997 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476633+0000", 2024-09-15T21:47:43.997 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9869999999999999e-06 2024-09-15T21:47:43.997 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.997 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.997 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:43.997 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477618+0000", 2024-09-15T21:47:43.997 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00098539299999999999 2024-09-15T21:47:43.997 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.998 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.998 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:43.998 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477696+0000", 2024-09-15T21:47:43.998 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8509000000000003e-05 2024-09-15T21:47:43.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.998 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.998 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:43.998 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477824+0000", 2024-09-15T21:47:43.998 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000127062 2024-09-15T21:47:43.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.998 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.998 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:43.998 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480294+0000", 2024-09-15T21:47:43.998 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024703939999999999 2024-09-15T21:47:43.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.999 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.999 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:43.999 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480788+0000", 2024-09-15T21:47:43.999 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00049443800000000004 2024-09-15T21:47:43.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.999 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.999 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:43.999 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480845+0000", 2024-09-15T21:47:43.999 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6762000000000003e-05 2024-09-15T21:47:43.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:43.999 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:43.999 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:43.999 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480890+0000", 2024-09-15T21:47:43.999 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4789000000000001e-05 2024-09-15T21:47:44.000 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.000 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.000 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.000 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.000 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.000 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:44.000 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:44.000 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 63.434017857999997, 2024-09-15T21:47:44.000 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007191546, 2024-09-15T21:47:44.000 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.000 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.000 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:44.000 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.000 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.001 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.001 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:44.001 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.001 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.001 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.001 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:44.001 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.001 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.001 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.001 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:44.001 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9999999999999999e-07 2024-09-15T21:47:44.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.002 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.002 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.002 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475585+0000", 2024-09-15T21:47:44.002 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.384e-06 2024-09-15T21:47:44.002 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.002 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.002 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.002 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476665+0000", 2024-09-15T21:47:44.002 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010797140000000001 2024-09-15T21:47:44.002 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.002 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.002 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.002 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476671+0000", 2024-09-15T21:47:44.002 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3940000000000001e-06 2024-09-15T21:47:44.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.003 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.003 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.003 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478048+0000", 2024-09-15T21:47:44.003 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013767530000000001 2024-09-15T21:47:44.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.003 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.003 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.003 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478177+0000", 2024-09-15T21:47:44.003 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012898099999999999 2024-09-15T21:47:44.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.003 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.003 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.003 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482774+0000", 2024-09-15T21:47:44.003 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045970200000000003 2024-09-15T21:47:44.004 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.004 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.004 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.004 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.004 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:44.004 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:44.004 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.364788193000003, 2024-09-15T21:47:44.004 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055444480000000004, 2024-09-15T21:47:44.004 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.004 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.004 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.004 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.004 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.005 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.005 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:44.005 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.005 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.005 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.005 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:44.005 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.005 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.005 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.005 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:44.005 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4200000000000002e-07 2024-09-15T21:47:44.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.006 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.006 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.006 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544814+0000", 2024-09-15T21:47:44.006 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9980000000000002e-06 2024-09-15T21:47:44.006 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.006 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.006 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.006 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544872+0000", 2024-09-15T21:47:44.006 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8405000000000001e-05 2024-09-15T21:47:44.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.007 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.007 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.007 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544877+0000", 2024-09-15T21:47:44.007 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.143e-06 2024-09-15T21:47:44.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.007 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.007 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.007 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545750+0000", 2024-09-15T21:47:44.007 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00087310000000000003 2024-09-15T21:47:44.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.007 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.008 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.008 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545850+0000", 2024-09-15T21:47:44.008 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9939999999999995e-05 2024-09-15T21:47:44.008 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.008 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.008 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:44.008 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546014+0000", 2024-09-15T21:47:44.008 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00016408500000000001 2024-09-15T21:47:44.008 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.008 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.008 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:44.008 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.548748+0000", 2024-09-15T21:47:44.009 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027339539999999998 2024-09-15T21:47:44.009 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.009 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.009 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:44.009 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550265+0000", 2024-09-15T21:47:44.009 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015170019999999999 2024-09-15T21:47:44.009 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.009 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.009 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.009 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550315+0000", 2024-09-15T21:47:44.009 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9867999999999998e-05 2024-09-15T21:47:44.009 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.009 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.009 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.009 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550356+0000", 2024-09-15T21:47:44.010 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1711000000000001e-05 2024-09-15T21:47:44.010 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.010 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.010 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.010 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.010 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.010 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:44.010 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:44.010 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.364784991999997, 2024-09-15T21:47:44.010 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058729619999999998, 2024-09-15T21:47:44.010 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.010 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.010 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.010 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.011 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.011 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.011 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:44.011 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.011 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.011 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.011 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:44.011 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.011 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.011 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.011 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:44.011 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0599999999999999e-07 2024-09-15T21:47:44.012 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.012 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.012 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.012 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544821+0000", 2024-09-15T21:47:44.012 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4079999999999997e-06 2024-09-15T21:47:44.012 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.012 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.012 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.012 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544939+0000", 2024-09-15T21:47:44.012 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118532 2024-09-15T21:47:44.012 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.012 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.012 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.012 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544946+0000", 2024-09-15T21:47:44.012 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9260000000000003e-06 2024-09-15T21:47:44.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.013 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.013 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.013 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545011+0000", 2024-09-15T21:47:44.013 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4572000000000005e-05 2024-09-15T21:47:44.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.013 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.013 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.013 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545134+0000", 2024-09-15T21:47:44.013 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012309899999999999 2024-09-15T21:47:44.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.013 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.013 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:44.013 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545271+0000", 2024-09-15T21:47:44.014 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013746599999999999 2024-09-15T21:47:44.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.014 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.014 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:44.014 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550243+0000", 2024-09-15T21:47:44.014 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049714529999999998 2024-09-15T21:47:44.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.014 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.014 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:44.014 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550605+0000", 2024-09-15T21:47:44.014 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00036260699999999998 2024-09-15T21:47:44.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.014 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.014 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.015 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550662+0000", 2024-09-15T21:47:44.015 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6573000000000002e-05 2024-09-15T21:47:44.015 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.015 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.015 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.015 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550688+0000", 2024-09-15T21:47:44.015 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6120000000000001e-05 2024-09-15T21:47:44.015 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.015 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.015 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.015 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.015 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.015 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:44.017 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:47:44.017 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:44.015+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:44.118 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:44.116+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:44.168 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:44.168 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:44.168 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:44.168 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:44.176 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:47:44.218 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:44.217+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:44.282 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:44.283 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:44.283 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:44.283 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:44.283 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:44.292 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:47:44.320 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:44.318+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:44.399 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:44.399 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:44.399 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:44.399 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:44.400 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.400 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:44.400 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:44.400 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 75.983609533000006, 2024-09-15T21:47:44.400 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006101291, 2024-09-15T21:47:44.400 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.400 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.400 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.400 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.400 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.400 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.400 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:44.401 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.401 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.401 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.401 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:44.401 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.401 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.401 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.401 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413116+0000", 2024-09-15T21:47:44.401 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9700000000000003e-07 2024-09-15T21:47:44.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.401 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.402 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.402 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413118+0000", 2024-09-15T21:47:44.402 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2910000000000002e-06 2024-09-15T21:47:44.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.402 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.402 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.402 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413220+0000", 2024-09-15T21:47:44.402 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102056 2024-09-15T21:47:44.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.402 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.402 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.402 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413225+0000", 2024-09-15T21:47:44.402 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0490000000000003e-06 2024-09-15T21:47:44.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.403 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.403 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.403 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414808+0000", 2024-09-15T21:47:44.403 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015826900000000001 2024-09-15T21:47:44.403 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.403 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.403 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.403 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414907+0000", 2024-09-15T21:47:44.403 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9600999999999997e-05 2024-09-15T21:47:44.404 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.404 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.404 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:44.404 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415054+0000", 2024-09-15T21:47:44.404 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000146532 2024-09-15T21:47:44.404 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.404 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.404 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:44.404 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418056+0000", 2024-09-15T21:47:44.405 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030020519999999998 2024-09-15T21:47:44.405 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.405 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:44.405 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419128+0000", 2024-09-15T21:47:44.405 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001071671 2024-09-15T21:47:44.405 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.405 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.405 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419184+0000", 2024-09-15T21:47:44.405 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6125000000000002e-05 2024-09-15T21:47:44.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.406 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.406 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419217+0000", 2024-09-15T21:47:44.406 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2926999999999998e-05 2024-09-15T21:47:44.406 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.406 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.406 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.406 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:44.407 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:44.407 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 75.983606007000006, 2024-09-15T21:47:44.407 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059975130000000003, 2024-09-15T21:47:44.407 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.407 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.407 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.407 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.407 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.407 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.407 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:44.407 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.408 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.408 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.408 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.408 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:44.408 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.408 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.408 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.408 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.408 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:44.408 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6199999999999999e-07 2024-09-15T21:47:44.408 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.409 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.409 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.409 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413121+0000", 2024-09-15T21:47:44.409 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2000000000000001e-06 2024-09-15T21:47:44.409 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.409 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.409 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.410 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413292+0000", 2024-09-15T21:47:44.410 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017034400000000001 2024-09-15T21:47:44.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.410 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.410 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.410 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413300+0000", 2024-09-15T21:47:44.410 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8539999999999997e-06 2024-09-15T21:47:44.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.410 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.410 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.410 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414299+0000", 2024-09-15T21:47:44.411 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0009992829999999999 2024-09-15T21:47:44.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.411 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.411 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414390+0000", 2024-09-15T21:47:44.411 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1237999999999996e-05 2024-09-15T21:47:44.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.411 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:44.411 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414490+0000", 2024-09-15T21:47:44.411 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9438999999999994e-05 2024-09-15T21:47:44.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.412 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:44.412 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417393+0000", 2024-09-15T21:47:44.412 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002903306 2024-09-15T21:47:44.412 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.412 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.412 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:44.412 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419023+0000", 2024-09-15T21:47:44.412 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001630292 2024-09-15T21:47:44.412 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.412 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.412 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.412 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419067+0000", 2024-09-15T21:47:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4017999999999998e-05 2024-09-15T21:47:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419116+0000", 2024-09-15T21:47:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9277000000000003e-05 2024-09-15T21:47:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:44.413 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:44.414 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 75.980907810000005, 2024-09-15T21:47:44.414 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060879280000000003, 2024-09-15T21:47:44.414 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.414 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.414 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.414 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.414 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.414 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.414 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:44.414 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.414 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.414 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.414 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.415 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:44.415 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.415 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.415 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.415 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.415 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415818+0000", 2024-09-15T21:47:44.415 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:47:44.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.416 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.416 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.416 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415854+0000", 2024-09-15T21:47:44.416 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6706999999999999e-05 2024-09-15T21:47:44.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.416 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.416 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.416 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415860+0000", 2024-09-15T21:47:44.416 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0789999999999997e-06 2024-09-15T21:47:44.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.416 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.417 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.417 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415865+0000", 2024-09-15T21:47:44.417 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4580000000000003e-06 2024-09-15T21:47:44.417 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.417 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.417 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.417 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416092+0000", 2024-09-15T21:47:44.417 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00022739400000000001 2024-09-15T21:47:44.417 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.417 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.417 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.417 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416108+0000", 2024-09-15T21:47:44.417 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.613e-05 2024-09-15T21:47:44.418 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.418 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.418 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.418 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421866+0000", 2024-09-15T21:47:44.418 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005757585 2024-09-15T21:47:44.418 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.418 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.418 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.418 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421905+0000", 2024-09-15T21:47:44.418 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9214000000000001e-05 2024-09-15T21:47:44.418 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.418 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.418 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.419 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.419 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-15T21:47:44.419 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:44.419 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 75.980847650000001, 2024-09-15T21:47:44.419 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060637570000000003, 2024-09-15T21:47:44.419 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.419 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.419 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.419 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.419 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.420 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.420 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:44.420 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.420 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.420 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.420 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.420 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:44.420 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.420 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.420 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.420 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.420 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415878+0000", 2024-09-15T21:47:44.421 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.34e-07 2024-09-15T21:47:44.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.421 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.421 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.421 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415881+0000", 2024-09-15T21:47:44.421 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.354e-06 2024-09-15T21:47:44.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.421 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.421 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.421 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415887+0000", 2024-09-15T21:47:44.421 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:47:44.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.421 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.422 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.422 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415891+0000", 2024-09-15T21:47:44.422 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2690000000000001e-06 2024-09-15T21:47:44.422 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.422 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.422 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.422 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416267+0000", 2024-09-15T21:47:44.422 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00037614799999999998 2024-09-15T21:47:44.422 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.422 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.422 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.422 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416288+0000", 2024-09-15T21:47:44.422 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0604000000000001e-05 2024-09-15T21:47:44.422 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.423 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.423 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.423 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421908+0000", 2024-09-15T21:47:44.423 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056203390000000002 2024-09-15T21:47:44.423 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.423 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.423 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.423 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421941+0000", 2024-09-15T21:47:44.423 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3079e-05 2024-09-15T21:47:44.423 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.423 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.423 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.423 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.423 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.424 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:44.424 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:44.424 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 75.980829223000001, 2024-09-15T21:47:44.424 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006080284, 2024-09-15T21:47:44.424 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.424 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.424 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.424 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.424 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.424 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:44.424 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.425 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.425 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.425 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:44.425 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.425 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.425 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.425 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.425 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:44.425 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6899999999999999e-07 2024-09-15T21:47:44.425 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.425 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.425 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.425 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415898+0000", 2024-09-15T21:47:44.426 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.951e-06 2024-09-15T21:47:44.426 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.426 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415920+0000", 2024-09-15T21:47:44.426 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1651999999999999e-05 2024-09-15T21:47:44.426 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.426 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415925+0000", 2024-09-15T21:47:44.426 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.463e-06 2024-09-15T21:47:44.426 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.427 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416449+0000", 2024-09-15T21:47:44.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052384000000000003 2024-09-15T21:47:44.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.427 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.427 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416466+0000", 2024-09-15T21:47:44.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7108999999999998e-05 2024-09-15T21:47:44.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.427 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.427 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421943+0000", 2024-09-15T21:47:44.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054771569999999999 2024-09-15T21:47:44.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.428 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.428 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.428 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421976+0000", 2024-09-15T21:47:44.428 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2943e-05 2024-09-15T21:47:44.428 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.428 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.428 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.428 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.428 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-15T21:47:44.428 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:44.428 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 75.980789986999994, 2024-09-15T21:47:44.429 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060643340000000002, 2024-09-15T21:47:44.429 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.429 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.429 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.429 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.429 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.429 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.429 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:44.429 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.429 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.429 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.429 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.429 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:44.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.430 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:44.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4200000000000002e-07 2024-09-15T21:47:44.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.430 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415939+0000", 2024-09-15T21:47:44.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.439e-06 2024-09-15T21:47:44.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.431 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.431 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.431 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415945+0000", 2024-09-15T21:47:44.431 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7740000000000001e-06 2024-09-15T21:47:44.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.431 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.431 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.431 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415948+0000", 2024-09-15T21:47:44.431 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.737e-06 2024-09-15T21:47:44.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.432 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.432 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.432 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416605+0000", 2024-09-15T21:47:44.432 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00065721699999999998 2024-09-15T21:47:44.432 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.432 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.432 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.432 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416620+0000", 2024-09-15T21:47:44.432 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4564e-05 2024-09-15T21:47:44.432 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.432 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.432 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.433 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421977+0000", 2024-09-15T21:47:44.433 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053572369999999999 2024-09-15T21:47:44.433 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.433 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.433 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.433 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421999+0000", 2024-09-15T21:47:44.433 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2024000000000001e-05 2024-09-15T21:47:44.433 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.433 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.433 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.433 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.433 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.434 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-15T21:47:44.434 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:44.434 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 75.980589327999994, 2024-09-15T21:47:44.434 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058922669999999996, 2024-09-15T21:47:44.434 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.434 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.434 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.434 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.434 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.434 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.434 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:44.434 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.434 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.435 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.435 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.435 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:44.435 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.435 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.435 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.435 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:44.435 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6499999999999999e-07 2024-09-15T21:47:44.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.435 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.435 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.435 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416142+0000", 2024-09-15T21:47:44.436 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3929999999999999e-06 2024-09-15T21:47:44.436 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.436 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.436 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.436 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416151+0000", 2024-09-15T21:47:44.436 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5840000000000001e-06 2024-09-15T21:47:44.436 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.436 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.436 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.436 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416157+0000", 2024-09-15T21:47:44.437 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7590000000000003e-06 2024-09-15T21:47:44.437 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.437 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.437 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.437 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416767+0000", 2024-09-15T21:47:44.437 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00061034900000000005 2024-09-15T21:47:44.437 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.437 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.437 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.437 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416782+0000", 2024-09-15T21:47:44.437 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.483e-05 2024-09-15T21:47:44.437 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.438 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.438 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.438 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422002+0000", 2024-09-15T21:47:44.438 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052202639999999996 2024-09-15T21:47:44.438 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.438 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.438 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.438 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422028+0000", 2024-09-15T21:47:44.438 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5423e-05 2024-09-15T21:47:44.438 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.438 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.438 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.438 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.439 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.439 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-15T21:47:44.439 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:44.439 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 75.980182826999993, 2024-09-15T21:47:44.439 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055107769999999997, 2024-09-15T21:47:44.439 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.439 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.439 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.439 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.439 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.439 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.439 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:44.439 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.440 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.440 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.440 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.440 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:44.440 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.440 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.440 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.440 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.440 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416543+0000", 2024-09-15T21:47:44.440 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5400000000000001e-07 2024-09-15T21:47:44.440 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.440 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.440 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.441 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416548+0000", 2024-09-15T21:47:44.441 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8659999999999997e-06 2024-09-15T21:47:44.441 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.441 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.441 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.441 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416555+0000", 2024-09-15T21:47:44.441 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7190000000000001e-06 2024-09-15T21:47:44.441 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.441 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.441 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.441 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416560+0000", 2024-09-15T21:47:44.441 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3270000000000002e-06 2024-09-15T21:47:44.441 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.442 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.442 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.442 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416916+0000", 2024-09-15T21:47:44.442 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00035661200000000002 2024-09-15T21:47:44.442 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.442 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.442 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.442 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416932+0000", 2024-09-15T21:47:44.442 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5824e-05 2024-09-15T21:47:44.442 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.442 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.442 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.442 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422030+0000", 2024-09-15T21:47:44.443 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050975229999999996 2024-09-15T21:47:44.443 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.443 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.443 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.443 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422053+0000", 2024-09-15T21:47:44.443 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3252e-05 2024-09-15T21:47:44.443 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.443 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.443 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.443 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.443 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.443 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-15T21:47:44.443 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:44.444 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 75.979784327999994, 2024-09-15T21:47:44.444 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051422289999999999, 2024-09-15T21:47:44.444 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.444 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.444 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.444 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.444 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.444 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:44.444 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.444 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.445 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.445 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:44.445 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.445 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.445 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.445 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:44.445 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2699999999999999e-07 2024-09-15T21:47:44.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.445 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.445 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.445 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416946+0000", 2024-09-15T21:47:44.445 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1390000000000003e-06 2024-09-15T21:47:44.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.446 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.446 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.446 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416957+0000", 2024-09-15T21:47:44.446 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.044e-05 2024-09-15T21:47:44.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.446 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.446 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.446 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416963+0000", 2024-09-15T21:47:44.446 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5320000000000003e-06 2024-09-15T21:47:44.447 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.447 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.447 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.447 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417090+0000", 2024-09-15T21:47:44.447 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126549 2024-09-15T21:47:44.447 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.447 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.448 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.448 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417114+0000", 2024-09-15T21:47:44.448 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3736000000000001e-05 2024-09-15T21:47:44.448 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.448 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.448 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.448 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422055+0000", 2024-09-15T21:47:44.448 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049410410000000002 2024-09-15T21:47:44.448 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.448 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.448 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.448 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422083+0000", 2024-09-15T21:47:44.448 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8164999999999999e-05 2024-09-15T21:47:44.449 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.449 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.449 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.449 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.449 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.449 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:44.449 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:44.449 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 69.935292356000005, 2024-09-15T21:47:44.449 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007521186, 2024-09-15T21:47:44.449 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.449 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.449 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.450 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.450 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.450 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.450 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:44.450 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.450 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.450 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.450 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.450 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:44.450 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.450 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.450 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.450 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.450 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:44.451 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9999999999999998e-07 2024-09-15T21:47:44.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.451 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.451 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461443+0000", 2024-09-15T21:47:44.451 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0448000000000001e-05 2024-09-15T21:47:44.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.451 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.451 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462146+0000", 2024-09-15T21:47:44.451 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00070223499999999995 2024-09-15T21:47:44.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.452 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.452 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462154+0000", 2024-09-15T21:47:44.452 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9060000000000002e-06 2024-09-15T21:47:44.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.452 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.452 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462254+0000", 2024-09-15T21:47:44.452 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9897999999999998e-05 2024-09-15T21:47:44.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.452 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.452 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462330+0000", 2024-09-15T21:47:44.453 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6551000000000006e-05 2024-09-15T21:47:44.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.453 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.453 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:44.453 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462454+0000", 2024-09-15T21:47:44.453 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000124213 2024-09-15T21:47:44.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.453 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.453 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:44.453 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463681+0000", 2024-09-15T21:47:44.453 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012265729999999999 2024-09-15T21:47:44.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.453 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.454 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:44.454 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468817+0000", 2024-09-15T21:47:44.454 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051361430000000001 2024-09-15T21:47:44.454 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.454 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.454 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.454 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468892+0000", 2024-09-15T21:47:44.454 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5480000000000002e-05 2024-09-15T21:47:44.454 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.454 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.454 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.454 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468954+0000", 2024-09-15T21:47:44.454 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1339000000000003e-05 2024-09-15T21:47:44.455 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.455 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.455 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.455 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.455 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.455 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:44.455 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:44.455 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 69.935275740999998, 2024-09-15T21:47:44.455 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077296149999999996, 2024-09-15T21:47:44.455 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.455 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.455 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.455 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.455 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.456 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.456 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:44.456 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.456 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.456 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.456 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:44.456 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.457 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.457 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461450+0000", 2024-09-15T21:47:44.457 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0499999999999999e-07 2024-09-15T21:47:44.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.457 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.457 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461452+0000", 2024-09-15T21:47:44.457 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.007e-06 2024-09-15T21:47:44.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.458 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.458 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462209+0000", 2024-09-15T21:47:44.458 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075781000000000004 2024-09-15T21:47:44.458 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.458 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.458 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.459 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462214+0000", 2024-09-15T21:47:44.459 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1420000000000001e-06 2024-09-15T21:47:44.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.459 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.459 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.459 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462681+0000", 2024-09-15T21:47:44.460 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046634200000000002 2024-09-15T21:47:44.460 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.460 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.460 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.460 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462772+0000", 2024-09-15T21:47:44.461 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1414000000000003e-05 2024-09-15T21:47:44.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.461 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.461 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:44.461 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462871+0000", 2024-09-15T21:47:44.461 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8455000000000005e-05 2024-09-15T21:47:44.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.462 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.462 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:44.462 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464138+0000", 2024-09-15T21:47:44.462 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012673910000000001 2024-09-15T21:47:44.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.462 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.462 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:44.463 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469079+0000", 2024-09-15T21:47:44.463 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049407799999999997 2024-09-15T21:47:44.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.463 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.463 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.463 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469141+0000", 2024-09-15T21:47:44.463 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2169999999999996e-05 2024-09-15T21:47:44.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.464 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.464 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.464 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469179+0000", 2024-09-15T21:47:44.464 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7799000000000002e-05 2024-09-15T21:47:44.464 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.465 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.465 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.465 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.465 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.465 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:44.466 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:44.466 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 63.921159932999998, 2024-09-15T21:47:44.466 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004990497, 2024-09-15T21:47:44.466 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.466 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.467 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.467 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.467 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.467 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:44.467 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.467 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.467 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.467 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:44.468 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.468 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.468 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:44.468 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4729999999999999e-06 2024-09-15T21:47:44.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.468 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.468 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475576+0000", 2024-09-15T21:47:44.468 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1749999999999994e-06 2024-09-15T21:47:44.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.469 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475619+0000", 2024-09-15T21:47:44.469 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3557000000000003e-05 2024-09-15T21:47:44.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.469 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475631+0000", 2024-09-15T21:47:44.469 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1739999999999999e-05 2024-09-15T21:47:44.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.470 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475770+0000", 2024-09-15T21:47:44.470 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013950599999999999 2024-09-15T21:47:44.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.470 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.470 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475887+0000", 2024-09-15T21:47:44.470 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011616 2024-09-15T21:47:44.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.470 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:44.470 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476018+0000", 2024-09-15T21:47:44.470 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013119499999999999 2024-09-15T21:47:44.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.471 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:44.471 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478442+0000", 2024-09-15T21:47:44.471 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024240870000000001 2024-09-15T21:47:44.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.471 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.471 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:44.471 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480457+0000", 2024-09-15T21:47:44.471 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020151779999999998 2024-09-15T21:47:44.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.471 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.471 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.471 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480511+0000", 2024-09-15T21:47:44.471 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3582000000000003e-05 2024-09-15T21:47:44.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.472 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.472 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.472 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480556+0000", 2024-09-15T21:47:44.472 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4844e-05 2024-09-15T21:47:44.472 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.472 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.472 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.473 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.473 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:44.473 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:44.473 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 63.921145615, 2024-09-15T21:47:44.473 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051124810000000003, 2024-09-15T21:47:44.473 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.474 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.474 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.474 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.474 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.474 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.474 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:44.474 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.474 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.474 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.474 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.474 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:44.474 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.474 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.475 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.475 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.475 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475580+0000", 2024-09-15T21:47:44.475 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7800000000000002e-07 2024-09-15T21:47:44.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.475 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.475 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.475 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475595+0000", 2024-09-15T21:47:44.475 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4919000000000001e-05 2024-09-15T21:47:44.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.475 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.475 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.475 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475662+0000", 2024-09-15T21:47:44.476 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7769999999999997e-05 2024-09-15T21:47:44.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.476 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.476 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.476 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475669+0000", 2024-09-15T21:47:44.476 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.703e-06 2024-09-15T21:47:44.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.476 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.476 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.476 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476781+0000", 2024-09-15T21:47:44.476 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011118300000000001 2024-09-15T21:47:44.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.476 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.477 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.477 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476863+0000", 2024-09-15T21:47:44.477 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2208999999999998e-05 2024-09-15T21:47:44.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.477 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.477 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:44.477 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477040+0000", 2024-09-15T21:47:44.477 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000176424 2024-09-15T21:47:44.478 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.478 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.478 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:44.478 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480154+0000", 2024-09-15T21:47:44.478 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031147589999999999 2024-09-15T21:47:44.478 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.478 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.478 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:44.478 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480596+0000", 2024-09-15T21:47:44.479 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044172699999999999 2024-09-15T21:47:44.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.479 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.479 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.479 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480652+0000", 2024-09-15T21:47:44.479 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6350000000000001e-05 2024-09-15T21:47:44.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.479 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.479 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.479 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480692+0000", 2024-09-15T21:47:44.480 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9412000000000001e-05 2024-09-15T21:47:44.480 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.480 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.480 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.480 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.480 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:6 2.9 2:941d59bc:::61f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:44.480 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:44.480 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.851935083000001, 2024-09-15T21:47:44.480 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057427529999999997, 2024-09-15T21:47:44.480 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.480 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.480 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.481 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.481 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.481 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.481 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:44.481 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.481 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.481 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.481 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:44.481 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.481 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.481 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.482 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:44.482 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.41e-07 2024-09-15T21:47:44.482 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.482 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.482 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.482 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544792+0000", 2024-09-15T21:47:44.482 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9929999999999998e-06 2024-09-15T21:47:44.482 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.483 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.483 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.483 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544885+0000", 2024-09-15T21:47:44.483 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2825000000000007e-05 2024-09-15T21:47:44.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.483 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.484 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.484 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544891+0000", 2024-09-15T21:47:44.484 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.93e-06 2024-09-15T21:47:44.484 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.484 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.484 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.484 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546766+0000", 2024-09-15T21:47:44.484 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018747569999999999 2024-09-15T21:47:44.484 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.484 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.485 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.485 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546864+0000", 2024-09-15T21:47:44.485 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8603000000000004e-05 2024-09-15T21:47:44.485 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.485 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.485 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:44.485 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547028+0000", 2024-09-15T21:47:44.485 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000163461 2024-09-15T21:47:44.485 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.485 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.485 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:44.485 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549499+0000", 2024-09-15T21:47:44.485 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024711749999999999 2024-09-15T21:47:44.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.486 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.486 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:44.486 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550417+0000", 2024-09-15T21:47:44.486 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00091817199999999998 2024-09-15T21:47:44.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.486 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.486 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.486 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550476+0000", 2024-09-15T21:47:44.486 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8635999999999999e-05 2024-09-15T21:47:44.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.486 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.486 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.487 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550533+0000", 2024-09-15T21:47:44.487 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6959999999999997e-05 2024-09-15T21:47:44.487 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.487 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.487 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.487 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.487 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:44.487 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:44.487 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.851920618000001, 2024-09-15T21:47:44.487 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004917121, 2024-09-15T21:47:44.487 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.487 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.487 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.488 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.488 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.488 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.488 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:44.488 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.488 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.488 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.488 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.488 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:44.488 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.488 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.488 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.489 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.489 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544805+0000", 2024-09-15T21:47:44.489 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4600000000000001e-07 2024-09-15T21:47:44.489 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.489 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.489 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.489 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544817+0000", 2024-09-15T21:47:44.489 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2023e-05 2024-09-15T21:47:44.489 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.489 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.489 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.489 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545000+0000", 2024-09-15T21:47:44.489 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018384400000000001 2024-09-15T21:47:44.490 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.490 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.490 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.490 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545007+0000", 2024-09-15T21:47:44.490 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1249999999999998e-06 2024-09-15T21:47:44.490 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.490 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.490 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.490 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545670+0000", 2024-09-15T21:47:44.490 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00066324999999999999 2024-09-15T21:47:44.490 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.490 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.490 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.490 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545749+0000", 2024-09-15T21:47:44.491 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9239000000000005e-05 2024-09-15T21:47:44.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.491 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.491 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:44.491 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545885+0000", 2024-09-15T21:47:44.491 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013586199999999999 2024-09-15T21:47:44.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.491 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.491 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:44.491 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547944+0000", 2024-09-15T21:47:44.491 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020594720000000001 2024-09-15T21:47:44.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.491 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.492 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:44.492 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549630+0000", 2024-09-15T21:47:44.492 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001685831 2024-09-15T21:47:44.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.492 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.492 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.492 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549681+0000", 2024-09-15T21:47:44.492 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1187000000000003e-05 2024-09-15T21:47:44.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.492 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.492 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.493 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549721+0000", 2024-09-15T21:47:44.493 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0042e-05 2024-09-15T21:47:44.493 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.493 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.493 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.493 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.493 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:44.493 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:44.493 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.851906870000001, 2024-09-15T21:47:44.493 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050615549999999997, 2024-09-15T21:47:44.493 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.493 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.494 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.494 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.494 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.494 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.494 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:44.494 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.494 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.494 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.494 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:44.494 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.495 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.495 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.495 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:44.495 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:47:44.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.495 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.495 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.495 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544821+0000", 2024-09-15T21:47:44.495 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4399999999999999e-06 2024-09-15T21:47:44.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.495 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.495 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.496 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545025+0000", 2024-09-15T21:47:44.496 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020379400000000001 2024-09-15T21:47:44.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.496 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.496 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.496 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545030+0000", 2024-09-15T21:47:44.496 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3109999999999996e-06 2024-09-15T21:47:44.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.496 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.497 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.497 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546196+0000", 2024-09-15T21:47:44.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001166261 2024-09-15T21:47:44.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.497 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.497 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.497 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546307+0000", 2024-09-15T21:47:44.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000110641 2024-09-15T21:47:44.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.497 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.497 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:44.497 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546422+0000", 2024-09-15T21:47:44.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000115399 2024-09-15T21:47:44.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.498 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:44.498 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547973+0000", 2024-09-15T21:47:44.498 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001551228 2024-09-15T21:47:44.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.498 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:44.498 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549815+0000", 2024-09-15T21:47:44.498 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001841685 2024-09-15T21:47:44.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.498 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.498 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549856+0000", 2024-09-15T21:47:44.499 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.091e-05 2024-09-15T21:47:44.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.499 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.499 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549880+0000", 2024-09-15T21:47:44.499 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3580000000000001e-05 2024-09-15T21:47:44.499 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.499 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.499 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.500 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:44.500 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:44.500 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.851902862000003, 2024-09-15T21:47:44.500 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056931389999999998, 2024-09-15T21:47:44.500 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.500 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.500 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.500 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.500 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.500 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.500 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:44.500 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.501 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.501 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.501 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:44.501 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.501 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.501 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.501 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.501 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:44.501 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:47:44.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.502 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.502 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.502 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544828+0000", 2024-09-15T21:47:44.502 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4369999999999998e-06 2024-09-15T21:47:44.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.502 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.502 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.502 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545041+0000", 2024-09-15T21:47:44.502 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000212695 2024-09-15T21:47:44.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.502 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.502 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.503 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545058+0000", 2024-09-15T21:47:44.503 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7717999999999999e-05 2024-09-15T21:47:44.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.503 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.503 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.503 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546958+0000", 2024-09-15T21:47:44.503 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018995539999999999 2024-09-15T21:47:44.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.503 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.503 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.504 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547057+0000", 2024-09-15T21:47:44.504 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9370999999999994e-05 2024-09-15T21:47:44.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.504 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.504 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:44.504 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547579+0000", 2024-09-15T21:47:44.504 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052198200000000004 2024-09-15T21:47:44.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.504 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.504 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:44.504 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549929+0000", 2024-09-15T21:47:44.504 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023494649999999998 2024-09-15T21:47:44.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.505 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.505 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:44.505 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550401+0000", 2024-09-15T21:47:44.505 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00047277300000000001 2024-09-15T21:47:44.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.505 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.505 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.505 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550458+0000", 2024-09-15T21:47:44.505 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6756999999999999e-05 2024-09-15T21:47:44.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.505 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.505 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.506 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550515+0000", 2024-09-15T21:47:44.506 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7093999999999999e-05 2024-09-15T21:47:44.506 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.506 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.506 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.506 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.506 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:26 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:44.506 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:44.506 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.851896205000003, 2024-09-15T21:47:44.506 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050112029999999997, 2024-09-15T21:47:44.506 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.507 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.507 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.507 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.507 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.507 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.507 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:44.507 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.507 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.507 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.507 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.507 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:44.508 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.508 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.508 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.508 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.508 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:44.508 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.36e-07 2024-09-15T21:47:44.508 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.508 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.508 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.508 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545086+0000", 2024-09-15T21:47:44.508 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025704699999999999 2024-09-15T21:47:44.508 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.509 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.509 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.509 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545119+0000", 2024-09-15T21:47:44.509 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2734999999999997e-05 2024-09-15T21:47:44.509 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.509 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.509 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.509 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545126+0000", 2024-09-15T21:47:44.509 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5599999999999996e-06 2024-09-15T21:47:44.509 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.509 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.510 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.510 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545253+0000", 2024-09-15T21:47:44.510 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012644500000000001 2024-09-15T21:47:44.510 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.510 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.510 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.511 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545366+0000", 2024-09-15T21:47:44.511 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011349999999999999 2024-09-15T21:47:44.511 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.511 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.511 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:44.511 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545673+0000", 2024-09-15T21:47:44.512 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030643599999999999 2024-09-15T21:47:44.512 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.512 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.512 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:44.512 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547924+0000", 2024-09-15T21:47:44.513 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0022516189999999998 2024-09-15T21:47:44.513 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.513 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.513 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:44.513 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549746+0000", 2024-09-15T21:47:44.513 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018217629999999999 2024-09-15T21:47:44.514 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.514 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.514 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.514 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549800+0000", 2024-09-15T21:47:44.514 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3412999999999998e-05 2024-09-15T21:47:44.514 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.515 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.515 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.515 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549840+0000", 2024-09-15T21:47:44.516 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0448999999999998e-05 2024-09-15T21:47:44.516 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.516 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.516 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.516 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.516 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.517 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:30 2.4 2:218afd09:::203.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:44.517 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:44.517 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 58.851635375999997, 2024-09-15T21:47:44.517 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055481230000000003, 2024-09-15T21:47:44.517 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.518 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.518 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:44.518 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.518 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.518 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.518 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:44.519 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.519 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.519 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.519 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.519 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:44.519 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.520 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.520 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.520 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.520 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545091+0000", 2024-09-15T21:47:44.520 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7000000000000003e-07 2024-09-15T21:47:44.521 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.521 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.521 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.521 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545094+0000", 2024-09-15T21:47:44.521 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9569999999999998e-06 2024-09-15T21:47:44.522 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.522 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.522 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.522 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545156+0000", 2024-09-15T21:47:44.522 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2277e-05 2024-09-15T21:47:44.522 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.523 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.523 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.523 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545162+0000", 2024-09-15T21:47:44.523 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9120000000000003e-06 2024-09-15T21:47:44.523 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.523 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.524 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.524 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545262+0000", 2024-09-15T21:47:44.524 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010001900000000001 2024-09-15T21:47:44.524 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.525 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.525 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.525 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545388+0000", 2024-09-15T21:47:44.525 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126243 2024-09-15T21:47:44.525 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.526 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.526 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.526 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550638+0000", 2024-09-15T21:47:44.526 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052497450000000001 2024-09-15T21:47:44.526 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.526 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.527 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.527 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.527 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.527 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:10 2.9 2:954a9a2b:::62b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:44.527 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:44.527 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 52.831951916000001, 2024-09-15T21:47:44.528 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048438860000000004, 2024-09-15T21:47:44.528 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:44.528 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:44.528 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:44.528 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:44.529 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.529 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:44.529 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:44.529 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.529 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.529 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.530 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:44.530 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:44.530 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:44.530 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.530 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.531 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:44.531 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:44.531 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0400000000000002e-07 2024-09-15T21:47:44.531 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.531 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.531 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:44.532 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564786+0000", 2024-09-15T21:47:44.532 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2412000000000001e-05 2024-09-15T21:47:44.532 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.532 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.532 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:44.532 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564991+0000", 2024-09-15T21:47:44.533 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020483900000000001 2024-09-15T21:47:44.533 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.533 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.533 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:44.533 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564998+0000", 2024-09-15T21:47:44.534 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2459999999999997e-06 2024-09-15T21:47:44.534 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.534 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.534 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:44.534 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565422+0000", 2024-09-15T21:47:44.534 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00042420100000000003 2024-09-15T21:47:44.535 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.535 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.535 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:44.535 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565515+0000", 2024-09-15T21:47:44.535 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2449999999999997e-05 2024-09-15T21:47:44.535 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.536 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.536 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:44.536 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565642+0000", 2024-09-15T21:47:44.536 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012743699999999999 2024-09-15T21:47:44.536 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.536 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.536 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:44.537 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566656+0000", 2024-09-15T21:47:44.537 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010138580000000001 2024-09-15T21:47:44.537 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.537 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.537 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:44.537 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569552+0000", 2024-09-15T21:47:44.537 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028966819999999998 2024-09-15T21:47:44.538 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.538 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.538 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:44.538 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569594+0000", 2024-09-15T21:47:44.538 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1235000000000001e-05 2024-09-15T21:47:44.538 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:44.538 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:44.538 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:44.538 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569617+0000", 2024-09-15T21:47:44.538 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3221999999999999e-05 2024-09-15T21:47:44.539 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.539 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.539 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.539 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:44.539 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:44.539 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:44.540 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:47:44.541 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:44.538+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:44.568 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:47:44.641 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:44.640+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.724+0000 7fd1b0d98640 1 Processor -- start 2024-09-15T21:47:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.724+0000 7fd1b0d98640 1 -- start start 2024-09-15T21:47:44.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.725+0000 7fd1b0d98640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd1ac152d30 0x7fd1ac0c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:44.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.726+0000 7fd1b0d98640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1ac06ff60 con 0x7fd1ac15c330 2024-09-15T21:47:44.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.726+0000 7fd1b0d98640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1ac0c8550 con 0x7fd1ac0c57a0 2024-09-15T21:47:44.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.726+0000 7fd1b0d98640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fd1ac0c8df0 con 0x7fd1ac152d30 2024-09-15T21:47:44.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.726+0000 7fd1aa59c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd1ac152d30 0x7fd1ac0c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:44.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.726+0000 7fd1aa59c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd1ac152d30 0x7fd1ac0c67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33808/0 (socket says 172.21.15.100:33808) 2024-09-15T21:47:44.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.726+0000 7fd1a3fff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd1ac15c330 0x7fd1ac159750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:47958/0 (socket says 172.21.15.100:47958) 2024-09-15T21:47:44.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.726+0000 7fd1a3fff640 1 -- 172.21.15.100:0/446834981 learned_addr learned my addr 172.21.15.100:0/446834981 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:44.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.726+0000 7fd1a9d9b640 1 -- 172.21.15.100:0/446834981 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4214655572 0 0) 0x7fd1ac0c8550 con 0x7fd1ac0c57a0 2024-09-15T21:47:44.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.726+0000 7fd1a9d9b640 1 -- 172.21.15.100:0/446834981 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd198003520 con 0x7fd1ac0c57a0 2024-09-15T21:47:44.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.726+0000 7fd1a9d9b640 1 -- 172.21.15.100:0/446834981 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 792544080 0 0) 0x7fd1ac06ff60 con 0x7fd1ac15c330 2024-09-15T21:47:44.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.726+0000 7fd1a9d9b640 1 -- 172.21.15.100:0/446834981 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd1ac0c8550 con 0x7fd1ac15c330 2024-09-15T21:47:44.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.726+0000 7fd1aa59c640 1 -- 172.21.15.100:0/446834981 >> v1:172.21.15.122:6789/0 conn(0x7fd1ac0c57a0 legacy=0x7fd1ac0c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:44.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.726+0000 7fd1aa59c640 1 -- 172.21.15.100:0/446834981 >> v1:172.21.15.100:6789/0 conn(0x7fd1ac15c330 legacy=0x7fd1ac159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:44.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.726+0000 7fd1aa59c640 1 -- 172.21.15.100:0/446834981 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ac0c8830 con 0x7fd1ac152d30 2024-09-15T21:47:44.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.727+0000 7fd1aa59c640 1 --2- 172.21.15.100:0/446834981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ac152d30 0x7fd1ac0c67b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0097a0 tx=0x7fd19c02a2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d042288c0de2c294 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.728+0000 7fd1a9d9b640 1 -- 172.21.15.100:0/446834981 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c037070 con 0x7fd1ac152d30 2024-09-15T21:47:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.728+0000 7fd1a9d9b640 1 -- 172.21.15.100:0/446834981 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd19c032640 con 0x7fd1ac152d30 2024-09-15T21:47:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.728+0000 7fd1a9d9b640 1 -- 172.21.15.100:0/446834981 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19c0329f0 con 0x7fd1ac152d30 2024-09-15T21:47:44.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.729+0000 7fd1b0d98640 1 -- 172.21.15.100:0/446834981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ac152d30 msgr2=0x7fd1ac0c67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.729+0000 7fd1b0d98640 1 --2- 172.21.15.100:0/446834981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ac152d30 0x7fd1ac0c67b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0097a0 tx=0x7fd19c02a2e0 comp rx=0 tx=0).stop 2024-09-15T21:47:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.729+0000 7fd1b0d98640 1 -- 172.21.15.100:0/446834981 shutdown_connections 2024-09-15T21:47:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.729+0000 7fd1b0d98640 1 --2- 172.21.15.100:0/446834981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ac152d30 0x7fd1ac0c67b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.729+0000 7fd1b0d98640 1 -- 172.21.15.100:0/446834981 >> 172.21.15.100:0/446834981 conn(0x7fd1ac0cb3a0 msgr2=0x7fd1ac0cb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.729+0000 7fd1b0d98640 1 -- 172.21.15.100:0/446834981 shutdown_connections 2024-09-15T21:47:44.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.729+0000 7fd1b0d98640 1 -- 172.21.15.100:0/446834981 wait complete. 2024-09-15T21:47:44.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.731+0000 7fd1b0d98640 1 Processor -- start 2024-09-15T21:47:44.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.731+0000 7fd1b0d98640 1 -- start start 2024-09-15T21:47:44.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.732+0000 7fd1b0d98640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd1ac0c57a0 0x7fd1ac158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:44.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.732+0000 7fd1b0d98640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1ac152d30 0x7fd1ac15b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:44.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.732+0000 7fd1b0d98640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ac15c330 0x7fd1ac0cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:44.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.732+0000 7fd1b0d98640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac159230 con 0x7fd1ac15c330 2024-09-15T21:47:44.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.732+0000 7fd1b0d98640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ac158f50 con 0x7fd1ac152d30 2024-09-15T21:47:44.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.732+0000 7fd1b0d98640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd1ac153110 con 0x7fd1ac0c57a0 2024-09-15T21:47:44.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.732+0000 7fd1a3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1ac152d30 0x7fd1ac15b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:44.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.732+0000 7fd1aad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ac15c330 0x7fd1ac0cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:44.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.732+0000 7fd1a3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1ac152d30 0x7fd1ac15b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:59786/0 (socket says 172.21.15.100:59786) 2024-09-15T21:47:44.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.732+0000 7fd1aad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ac15c330 0x7fd1ac0cd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33812/0 (socket says 172.21.15.100:33812) 2024-09-15T21:47:44.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.732+0000 7fd1a3fff640 1 -- 172.21.15.100:0/2738698581 learned_addr learned my addr 172.21.15.100:0/2738698581 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:44.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.732+0000 7fd1aa59c640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd1ac0c57a0 0x7fd1ac158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:44.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.733+0000 7fd1a3fff640 1 -- 172.21.15.100:0/2738698581 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd1ac0c57a0 msgr2=0x7fd1ac158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:44.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.733+0000 7fd1a3fff640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd1ac0c57a0 0x7fd1ac158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:44.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.733+0000 7fd1a3fff640 1 -- 172.21.15.100:0/2738698581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ac15c330 msgr2=0x7fd1ac0cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:44.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.733+0000 7fd1a3fff640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ac15c330 0x7fd1ac0cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:44.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.733+0000 7fd1a3fff640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ac15a910 con 0x7fd1ac152d30 2024-09-15T21:47:44.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.733+0000 7fd1aad9d640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ac15c330 0x7fd1ac0cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:44.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.733+0000 7fd1a3fff640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1ac152d30 0x7fd1ac15b0f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd19400c410 tx=0x7fd19400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:44.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.733+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194017070 con 0x7fd1ac152d30 2024-09-15T21:47:44.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.733+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd194005ce0 con 0x7fd1ac152d30 2024-09-15T21:47:44.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.733+0000 7fd1b0d98640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1ac158d10 con 0x7fd1ac152d30 2024-09-15T21:47:44.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.733+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1940072f0 con 0x7fd1ac152d30 2024-09-15T21:47:44.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.733+0000 7fd1b0d98640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1ac15bc50 con 0x7fd1ac152d30 2024-09-15T21:47:44.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.734+0000 7fd1b0d98640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fd1ac0c8110 con 0x7fd1ac152d30 2024-09-15T21:47:44.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.736+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd19401a070 con 0x7fd1ac152d30 2024-09-15T21:47:44.740 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.736+0000 7fd1a2ffd640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd18c06c2b0 0x7fd18c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:44.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.737+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5854+0+0 (secure 0 0 0) 0x7fd19408ab10 con 0x7fd1ac152d30 2024-09-15T21:47:44.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.737+0000 7fd1aa59c640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd18c06c2b0 0x7fd18c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:44.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.737+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v40) ==== 54+0+16 (secure 0 0 0) 0x7fd19408b0a0 con 0x7fd1ac152d30 2024-09-15T21:47:44.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.737+0000 7fd1b0d98640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fd1ac1eeaa0 0x7fd1ac156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:44.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.737+0000 7fd1b0d98640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd1ac0739d0 con 0x7fd1ac1eeaa0 2024-09-15T21:47:44.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.737+0000 7fd1aad9d640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fd1ac1eeaa0 0x7fd1ac156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:44.741 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:44.741 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:44.741 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:44.741 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:44.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.738+0000 7fd1aad9d640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fd1ac1eeaa0 0x7fd1ac156980 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:44.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.739+0000 7fd1aa59c640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd18c06c2b0 0x7fd18c06e770 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd19c034000 tx=0x7fd19c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:44.742 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.739+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1ac0739d0 con 0x7fd1ac1eeaa0 2024-09-15T21:47:44.743 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:44.741+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:44.750 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:47:44.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.766+0000 7fd1b0d98640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd1ac0736f0 con 0x7fd1ac1eeaa0 2024-09-15T21:47:44.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.769+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd1ac0736f0 con 0x7fd1ac1eeaa0 2024-09-15T21:47:44.771 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:44.771 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:44.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.769+0000 7fd1b0d98640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fd1ac15be20 0x7fd1ac155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:44.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.769+0000 7fd1b0d98640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fd1ac1563b0 con 0x7fd1ac15be20 2024-09-15T21:47:44.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.770+0000 7fd1aa59c640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fd1ac15be20 0x7fd1ac155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:44.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.770+0000 7fd1aa59c640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fd1ac15be20 0x7fd1ac155e70 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:44.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.771+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1ac1563b0 con 0x7fd1ac15be20 2024-09-15T21:47:44.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.794+0000 7fd1b0d98640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd1ac073130 con 0x7fd1ac15be20 2024-09-15T21:47:44.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.795+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd1ac073130 con 0x7fd1ac15be20 2024-09-15T21:47:44.798 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:44.799 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:44.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.796+0000 7fd1b0d98640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fd1ac1f2520 0x7fd1ac1f2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:44.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.796+0000 7fd1b0d98640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fd1ac1f2e60 con 0x7fd1ac1f2520 2024-09-15T21:47:44.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.796+0000 7fd1a3fff640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fd1ac1f2520 0x7fd1ac1f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:44.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.797+0000 7fd1a3fff640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fd1ac1f2520 0x7fd1ac1f2920 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:44.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.798+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1940616f0 con 0x7fd1ac1f2520 2024-09-15T21:47:44.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.823+0000 7fd1b0d98640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd1ac062880 con 0x7fd1ac1f2520 2024-09-15T21:47:44.825 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:47:44.828 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:44.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.826+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd1ac062880 con 0x7fd1ac1f2520 2024-09-15T21:47:44.829 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:44.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.826+0000 7fd1b0d98640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fd1ac063360 0x7fd1ac149930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:44.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.826+0000 7fd1b0d98640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fd1ac062880 con 0x7fd1ac063360 2024-09-15T21:47:44.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.826+0000 7fd1aad9d640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fd1ac063360 0x7fd1ac149930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:44.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.827+0000 7fd1aad9d640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fd1ac063360 0x7fd1ac149930 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:44.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.828+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1ac062880 con 0x7fd1ac063360 2024-09-15T21:47:44.844 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:44.842+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:44.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.855+0000 7fd1b0d98640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd1ac062880 con 0x7fd1ac063360 2024-09-15T21:47:44.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.857+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd1ac062880 con 0x7fd1ac063360 2024-09-15T21:47:44.859 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:44.860 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:44.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.858+0000 7fd1b0d98640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fd1ac14a6b0 0x7fd1ac02c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:44.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.858+0000 7fd1b0d98640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fd1ac15add0 con 0x7fd1ac14a6b0 2024-09-15T21:47:44.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.858+0000 7fd1aa59c640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fd1ac14a6b0 0x7fd1ac02c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:44.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.859+0000 7fd1aa59c640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fd1ac14a6b0 0x7fd1ac02c640 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:44.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.859+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1ac15add0 con 0x7fd1ac14a6b0 2024-09-15T21:47:44.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.886+0000 7fd1b0d98640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd1ac15add0 con 0x7fd1ac14a6b0 2024-09-15T21:47:44.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.888+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd1ac15add0 con 0x7fd1ac14a6b0 2024-09-15T21:47:44.890 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.888+0000 7fd1b0d98640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=41}) -- 0x7fd1ac204aa0 con 0x7fd1ac152d30 2024-09-15T21:47:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.888+0000 7fd1b0d98640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd1ac204c70 con 0x7fd1ac152d30 2024-09-15T21:47:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.889+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fd194058490 con 0x7fd1ac152d30 2024-09-15T21:47:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:47:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.889+0000 7fd1b0d98640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fd1ac02d4a0 0x7fd1ac203390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:44.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.889+0000 7fd1b0d98640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fd1ac208540 con 0x7fd1ac02d4a0 2024-09-15T21:47:44.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.890+0000 7fd1a3fff640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fd1ac02d4a0 0x7fd1ac203390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:44.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.890+0000 7fd1a3fff640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fd1ac02d4a0 0x7fd1ac203390 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:44.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.891+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1ac208540 con 0x7fd1ac02d4a0 2024-09-15T21:47:44.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.917+0000 7fd1b0d98640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd1ac2038d0 con 0x7fd1ac02d4a0 2024-09-15T21:47:44.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.919+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd1ac2038d0 con 0x7fd1ac02d4a0 2024-09-15T21:47:44.921 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:44.922 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:44.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.920+0000 7fd1b0d98640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fd1ac203f50 0x7fd1ac2043d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:44.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.920+0000 7fd1b0d98640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fd1ac20b6d0 con 0x7fd1ac203f50 2024-09-15T21:47:44.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.920+0000 7fd1aad9d640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fd1ac203f50 0x7fd1ac2043d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:44.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.920+0000 7fd1aad9d640 1 --2- 172.21.15.100:0/2738698581 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fd1ac203f50 0x7fd1ac2043d0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:44.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.921+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1ac20b6d0 con 0x7fd1ac203f50 2024-09-15T21:47:44.932 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:44.932 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:44.945 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:47:44.946 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:44.943+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:44.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.947+0000 7fd1b0d98640 1 -- 172.21.15.100:0/2738698581 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd1ac208e40 con 0x7fd1ac203f50 2024-09-15T21:47:44.951 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.950+0000 7fd1a2ffd640 1 -- 172.21.15.100:0/2738698581 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd1ac208e40 con 0x7fd1ac203f50 2024-09-15T21:47:44.952 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:44.952 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:44.956 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:47:44.957 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:47:44.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.963+0000 7f71752dd640 1 Processor -- start 2024-09-15T21:47:44.965 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.963+0000 7f71752dd640 1 -- start start 2024-09-15T21:47:44.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.964+0000 7f71752dd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f71700c6ff0 0x7f71700ca090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:44.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.964+0000 7f71752dd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f71700718b0 0x7f717006f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:44.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.964+0000 7f71752dd640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71700cd0c0 con 0x7f7170071ce0 2024-09-15T21:47:44.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.964+0000 7f71752dd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f71700cd360 con 0x7f71700c6ff0 2024-09-15T21:47:44.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.964+0000 7f71752dd640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f71700cd640 con 0x7f71700718b0 2024-09-15T21:47:44.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.965+0000 7f716effd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f71700c6ff0 0x7f71700ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:44.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.965+0000 7f716effd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f71700c6ff0 0x7f71700ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33818/0 (socket says 172.21.15.100:33818) 2024-09-15T21:47:44.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.965+0000 7f716effd640 1 -- 172.21.15.100:0/1421416347 learned_addr learned my addr 172.21.15.100:0/1421416347 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:44.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.965+0000 7f716e7fc640 1 --2- 172.21.15.100:0/1421416347 >> v2:172.21.15.122:3300/0 conn(0x7f71700718b0 0x7f717006f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:44.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.965+0000 7f716effd640 1 -- 172.21.15.100:0/1421416347 >> v2:172.21.15.122:3300/0 conn(0x7f71700718b0 msgr2=0x7f717006f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:44.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.965+0000 7f716effd640 1 --2- 172.21.15.100:0/1421416347 >> v2:172.21.15.122:3300/0 conn(0x7f71700718b0 0x7f717006f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:44.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.965+0000 7f716effd640 1 -- 172.21.15.100:0/1421416347 >> v1:172.21.15.122:6789/0 conn(0x7f7170071ce0 legacy=0x7f717015b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:44.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.965+0000 7f716effd640 1 -- 172.21.15.100:0/1421416347 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71700cd920 con 0x7f71700c6ff0 2024-09-15T21:47:44.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.966+0000 7f716effd640 1 --2- 172.21.15.100:0/1421416347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71700c6ff0 0x7f71700ca090 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7170070a20 tx=0x7f716002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c18f0926252ed82 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:44.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.966+0000 7f716dffb640 1 -- 172.21.15.100:0/1421416347 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160037070 con 0x7f71700c6ff0 2024-09-15T21:47:44.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.966+0000 7f716dffb640 1 -- 172.21.15.100:0/1421416347 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7160032440 con 0x7f71700c6ff0 2024-09-15T21:47:44.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.966+0000 7f716dffb640 1 -- 172.21.15.100:0/1421416347 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7160032810 con 0x7f71700c6ff0 2024-09-15T21:47:44.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.967+0000 7f71752dd640 1 -- 172.21.15.100:0/1421416347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71700c6ff0 msgr2=0x7f71700ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:44.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.967+0000 7f71752dd640 1 --2- 172.21.15.100:0/1421416347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71700c6ff0 0x7f71700ca090 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7170070a20 tx=0x7f716002a180 comp rx=0 tx=0).stop 2024-09-15T21:47:44.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.967+0000 7f71752dd640 1 -- 172.21.15.100:0/1421416347 shutdown_connections 2024-09-15T21:47:44.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.967+0000 7f71752dd640 1 --2- 172.21.15.100:0/1421416347 >> v2:172.21.15.122:3300/0 conn(0x7f71700718b0 0x7f717006f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:44.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.967+0000 7f71752dd640 1 --2- 172.21.15.100:0/1421416347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71700c6ff0 0x7f71700ca090 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:44.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.967+0000 7f71752dd640 1 -- 172.21.15.100:0/1421416347 >> 172.21.15.100:0/1421416347 conn(0x7f7170073820 msgr2=0x7f7170073c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:44.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.967+0000 7f71752dd640 1 -- 172.21.15.100:0/1421416347 shutdown_connections 2024-09-15T21:47:44.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.967+0000 7f71752dd640 1 -- 172.21.15.100:0/1421416347 wait complete. 2024-09-15T21:47:44.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.969+0000 7f71752dd640 1 Processor -- start 2024-09-15T21:47:44.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.969+0000 7f71752dd640 1 -- start start 2024-09-15T21:47:44.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.970+0000 7f71752dd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71700718b0 0x7f717015ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:44.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.971+0000 7f71752dd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7170071ce0 0x7f7170141d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:44.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.971+0000 7f716e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7170071ce0 0x7f7170141d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:44.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.971+0000 7f716e7fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7170071ce0 0x7f7170141d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:33834/0 (socket says 172.21.15.100:33834) 2024-09-15T21:47:44.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.971+0000 7f71752dd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f717015c8e0 0x7f7170142240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:44.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.971+0000 7f716e7fc640 1 -- 172.21.15.100:0/675577354 learned_addr learned my addr 172.21.15.100:0/675577354 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:44.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.971+0000 7f71752dd640 1 -- 172.21.15.100:0/675577354 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f71700c73d0 con 0x7f7170071ce0 2024-09-15T21:47:44.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.971+0000 7f716effd640 1 --2- 172.21.15.100:0/675577354 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71700718b0 0x7f717015ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:44.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.971+0000 7f71752dd640 1 -- 172.21.15.100:0/675577354 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71700c8630 con 0x7f717015c8e0 2024-09-15T21:47:44.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.971+0000 7f71752dd640 1 -- 172.21.15.100:0/675577354 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f71700c7550 con 0x7f71700718b0 2024-09-15T21:47:44.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.971+0000 7f716f7fe640 1 --2- 172.21.15.100:0/675577354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f717015c8e0 0x7f7170142240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:44.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.971+0000 7f716f7fe640 1 -- 172.21.15.100:0/675577354 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71700718b0 msgr2=0x7f717015ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:44.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.971+0000 7f716f7fe640 1 --2- 172.21.15.100:0/675577354 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71700718b0 0x7f717015ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:44.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.972+0000 7f716f7fe640 1 -- 172.21.15.100:0/675577354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7170071ce0 msgr2=0x7f7170141d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:44.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.972+0000 7f716f7fe640 1 --2- 172.21.15.100:0/675577354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7170071ce0 0x7f7170141d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:44.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.972+0000 7f716f7fe640 1 -- 172.21.15.100:0/675577354 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71701559e0 con 0x7f717015c8e0 2024-09-15T21:47:44.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.972+0000 7f716e7fc640 1 --2- 172.21.15.100:0/675577354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7170071ce0 0x7f7170141d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:44.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.972+0000 7f716f7fe640 1 --2- 172.21.15.100:0/675577354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f717015c8e0 0x7f7170142240 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f716800c780 tx=0x7f716800cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:44.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.972+0000 7f71577fe640 1 -- 172.21.15.100:0/675577354 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168017070 con 0x7f717015c8e0 2024-09-15T21:47:44.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.972+0000 7f71577fe640 1 -- 172.21.15.100:0/675577354 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7168003b90 con 0x7f717015c8e0 2024-09-15T21:47:44.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.972+0000 7f71577fe640 1 -- 172.21.15.100:0/675577354 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168005070 con 0x7f717015c8e0 2024-09-15T21:47:44.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.972+0000 7f71752dd640 1 -- 172.21.15.100:0/675577354 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7170155ca0 con 0x7f717015c8e0 2024-09-15T21:47:44.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.972+0000 7f71752dd640 1 -- 172.21.15.100:0/675577354 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71701e8140 con 0x7f717015c8e0 2024-09-15T21:47:44.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.974+0000 7f71752dd640 1 -- 172.21.15.100:0/675577354 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71700769a0 con 0x7f717015c8e0 2024-09-15T21:47:44.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.974+0000 7f71577fe640 1 -- 172.21.15.100:0/675577354 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7168005210 con 0x7f717015c8e0 2024-09-15T21:47:44.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.974+0000 7f71577fe640 1 --2- 172.21.15.100:0/675577354 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f715c06c2e0 0x7f715c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:44.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.975+0000 7f71577fe640 1 -- 172.21.15.100:0/675577354 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5854+0+0 (secure 0 0 0) 0x7f716808a930 con 0x7f717015c8e0 2024-09-15T21:47:44.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.975+0000 7f716effd640 1 --2- 172.21.15.100:0/675577354 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f715c06c2e0 0x7f715c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:44.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.975+0000 7f716effd640 1 --2- 172.21.15.100:0/675577354 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f715c06c2e0 0x7f715c06e7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7160034000 tx=0x7f716002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:44.981 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:44.979+0000 7f71577fe640 1 -- 172.21.15.100:0/675577354 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f71700769a0 con 0x7f717015c8e0 2024-09-15T21:47:45.046 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:45.045+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:45.062 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:45.062 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:45.071 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:47:45.157 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:45.155+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:45.248 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:45.246+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:45.267 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:45.268 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:45.277 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:47:45.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:45.321+0000 7f71752dd640 1 -- 172.21.15.100:0/675577354 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f71700ca090 con 0x7f717015c8e0 2024-09-15T21:47:45.349 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:45.348+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:45.428 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:45.428 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:45.437 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:47:45.451 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:45.449+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:45.552 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:45.550+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:45.586 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:45.587 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:45.596 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:47:45.652 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:45.651+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:45.711 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:45.712 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:45.721 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:47:45.753 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:45.751+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:45.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:45.819+0000 7f71577fe640 1 -- 172.21.15.100:0/675577354 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v41) ==== 91+0+0 (secure 0 0 0) 0x7f7168058920 con 0x7f717015c8e0 2024-09-15T21:47:45.821 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is unset 2024-09-15T21:47:45.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:45.822+0000 7f71752dd640 1 -- 172.21.15.100:0/675577354 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f715c06c2e0 msgr2=0x7f715c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:45.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:45.822+0000 7f71752dd640 1 --2- 172.21.15.100:0/675577354 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f715c06c2e0 0x7f715c06e7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7160034000 tx=0x7f716002e000 comp rx=0 tx=0).stop 2024-09-15T21:47:45.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:45.822+0000 7f71752dd640 1 -- 172.21.15.100:0/675577354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f717015c8e0 msgr2=0x7f7170142240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:45.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:45.822+0000 7f71752dd640 1 --2- 172.21.15.100:0/675577354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f717015c8e0 0x7f7170142240 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f716800c780 tx=0x7f716800cc40 comp rx=0 tx=0).stop 2024-09-15T21:47:45.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:45.822+0000 7f71752dd640 1 -- 172.21.15.100:0/675577354 shutdown_connections 2024-09-15T21:47:45.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:45.822+0000 7f71752dd640 1 --2- 172.21.15.100:0/675577354 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f715c06c2e0 0x7f715c06e7a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:45.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:45.822+0000 7f71752dd640 1 --2- 172.21.15.100:0/675577354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f717015c8e0 0x7f7170142240 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:45.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:45.822+0000 7f71752dd640 1 --2- 172.21.15.100:0/675577354 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7170071ce0 0x7f7170141d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:45.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:45.822+0000 7f71752dd640 1 --2- 172.21.15.100:0/675577354 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71700718b0 0x7f717015ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:45.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:45.822+0000 7f71752dd640 1 -- 172.21.15.100:0/675577354 >> 172.21.15.100:0/675577354 conn(0x7f7170073820 msgr2=0x7f71700c51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:45.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:45.822+0000 7f71752dd640 1 -- 172.21.15.100:0/675577354 shutdown_connections 2024-09-15T21:47:45.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:45.822+0000 7f71752dd640 1 -- 172.21.15.100:0/675577354 wait complete. 2024-09-15T21:47:45.867 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:45.867 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:45.877 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:47:45.881 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:45.879+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:45.956 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:45.954+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:45.981 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:45.981 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:45.989 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:47:46.057 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:46.056+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:46.093 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:46.093 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:46.102 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:47:46.158 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:46.157+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:46.203 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:46.203 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:46.212 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:47:46.259 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:46.257+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:46.315 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:46.315 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:46.324 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:47:46.367 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:46.366+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:46.427 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:46.427 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:46.436 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:47:46.460 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:46.459+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:46.546 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-15T21:47:46.546 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:47:46.546 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-15T21:47:46.547 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.181830 2024-09-15T21:47:46.547 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0.18183040809386986 2024-09-15T21:47:46.561 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:46.560+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:46.579 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:46.579 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:46.588 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:47:46.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.652+0000 7f83ca65e640 1 Processor -- start 2024-09-15T21:47:46.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.652+0000 7f83ca65e640 1 -- start start 2024-09-15T21:47:46.655 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.653+0000 7f83ca65e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f83c4155680 0x7f83c415b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:46.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.653+0000 7f83ca65e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f83c40cb910 0x7f83c40cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:46.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.653+0000 7f83ca65e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83c415b7f0 con 0x7f83c41552a0 2024-09-15T21:47:46.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.653+0000 7f83ca65e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f83c415bab0 con 0x7f83c4155680 2024-09-15T21:47:46.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.653+0000 7f83ca65e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f83c415bd90 con 0x7f83c40cb910 2024-09-15T21:47:46.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.653+0000 7f83c37fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f83c4155680 0x7f83c415b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:46.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.653+0000 7f83c37fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f83c4155680 0x7f83c415b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57436/0 (socket says 172.21.15.100:57436) 2024-09-15T21:47:46.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.653+0000 7f83c37fe640 1 -- 172.21.15.100:0/3768426323 learned_addr learned my addr 172.21.15.100:0/3768426323 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:46.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.653+0000 7f83c8bfb640 1 --2- 172.21.15.100:0/3768426323 >> v2:172.21.15.122:3300/0 conn(0x7f83c40cb910 0x7f83c40cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:46.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.654+0000 7f83c37fe640 1 -- 172.21.15.100:0/3768426323 >> v2:172.21.15.122:3300/0 conn(0x7f83c40cb910 msgr2=0x7f83c40cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:46.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.654+0000 7f83c37fe640 1 --2- 172.21.15.100:0/3768426323 >> v2:172.21.15.122:3300/0 conn(0x7f83c40cb910 0x7f83c40cbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:46.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.654+0000 7f83c37fe640 1 -- 172.21.15.100:0/3768426323 >> v1:172.21.15.122:6789/0 conn(0x7f83c41552a0 legacy=0x7f83c40c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:46.656 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.654+0000 7f83c37fe640 1 -- 172.21.15.100:0/3768426323 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c415c070 con 0x7f83c4155680 2024-09-15T21:47:46.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.654+0000 7f83c37fe640 1 --2- 172.21.15.100:0/3768426323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83c4155680 0x7f83c415b0f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f83c40b3510 tx=0x7f83b802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a481bf0df757b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:46.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.655+0000 7f83c2ffd640 1 -- 172.21.15.100:0/3768426323 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8037070 con 0x7f83c4155680 2024-09-15T21:47:46.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.655+0000 7f83c2ffd640 1 -- 172.21.15.100:0/3768426323 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b802ae50 con 0x7f83c4155680 2024-09-15T21:47:46.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.655+0000 7f83c2ffd640 1 -- 172.21.15.100:0/3768426323 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b8032860 con 0x7f83c4155680 2024-09-15T21:47:46.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.655+0000 7f83ca65e640 1 -- 172.21.15.100:0/3768426323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83c4155680 msgr2=0x7f83c415b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:46.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.655+0000 7f83ca65e640 1 --2- 172.21.15.100:0/3768426323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83c4155680 0x7f83c415b0f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f83c40b3510 tx=0x7f83b802a300 comp rx=0 tx=0).stop 2024-09-15T21:47:46.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.655+0000 7f83ca65e640 1 -- 172.21.15.100:0/3768426323 shutdown_connections 2024-09-15T21:47:46.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.655+0000 7f83ca65e640 1 --2- 172.21.15.100:0/3768426323 >> v2:172.21.15.122:3300/0 conn(0x7f83c40cb910 0x7f83c40cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:46.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.655+0000 7f83ca65e640 1 --2- 172.21.15.100:0/3768426323 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83c4155680 0x7f83c415b0f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:46.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.655+0000 7f83ca65e640 1 -- 172.21.15.100:0/3768426323 >> 172.21.15.100:0/3768426323 conn(0x7f83c406d9a0 msgr2=0x7f83c406ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:46.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.655+0000 7f83ca65e640 1 -- 172.21.15.100:0/3768426323 shutdown_connections 2024-09-15T21:47:46.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.655+0000 7f83ca65e640 1 -- 172.21.15.100:0/3768426323 wait complete. 2024-09-15T21:47:46.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.657+0000 7f83ca65e640 1 Processor -- start 2024-09-15T21:47:46.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.657+0000 7f83ca65e640 1 -- start start 2024-09-15T21:47:46.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.658+0000 7f83ca65e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83c40cb910 0x7f83c41e8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:46.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.658+0000 7f83ca65e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83c41552a0 0x7f83c415a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:46.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.658+0000 7f83ca65e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83c4155680 0x7f83c414e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:46.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.658+0000 7f83ca65e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f83c40c61f0 con 0x7f83c41552a0 2024-09-15T21:47:46.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.658+0000 7f83ca65e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f83c40c6820 con 0x7f83c40cb910 2024-09-15T21:47:46.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.658+0000 7f83ca65e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f83c40c6d40 con 0x7f83c4155680 2024-09-15T21:47:46.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.658+0000 7f83c37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83c41552a0 0x7f83c415a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:46.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.658+0000 7f83c37fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83c41552a0 0x7f83c415a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57438/0 (socket says 172.21.15.100:57438) 2024-09-15T21:47:46.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.658+0000 7f83c37fe640 1 -- 172.21.15.100:0/1389423667 learned_addr learned my addr 172.21.15.100:0/1389423667 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:46.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.658+0000 7f83c3fff640 1 --2- 172.21.15.100:0/1389423667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83c40cb910 0x7f83c41e8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:46.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.658+0000 7f83c8bfb640 1 --2- 172.21.15.100:0/1389423667 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83c4155680 0x7f83c414e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:46.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.658+0000 7f83c3fff640 1 -- 172.21.15.100:0/1389423667 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83c4155680 msgr2=0x7f83c414e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:46.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.658+0000 7f83c3fff640 1 --2- 172.21.15.100:0/1389423667 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83c4155680 0x7f83c414e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:46.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.658+0000 7f83c3fff640 1 -- 172.21.15.100:0/1389423667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83c41552a0 msgr2=0x7f83c415a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:46.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.658+0000 7f83c3fff640 1 --2- 172.21.15.100:0/1389423667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83c41552a0 0x7f83c415a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:46.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.658+0000 7f83c3fff640 1 -- 172.21.15.100:0/1389423667 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c41f1e20 con 0x7f83c40cb910 2024-09-15T21:47:46.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.658+0000 7f83c37fe640 1 --2- 172.21.15.100:0/1389423667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83c41552a0 0x7f83c415a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:46.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.658+0000 7f83c8bfb640 1 --2- 172.21.15.100:0/1389423667 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83c4155680 0x7f83c414e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:47:46.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.659+0000 7f83c3fff640 1 --2- 172.21.15.100:0/1389423667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83c40cb910 0x7f83c41e8280 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f83b400c340 tx=0x7f83b400c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:46.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.659+0000 7f83c0ff9640 1 -- 172.21.15.100:0/1389423667 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b40052d0 con 0x7f83c40cb910 2024-09-15T21:47:46.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.659+0000 7f83c0ff9640 1 -- 172.21.15.100:0/1389423667 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83b4005490 con 0x7f83c40cb910 2024-09-15T21:47:46.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.659+0000 7f83ca65e640 1 -- 172.21.15.100:0/1389423667 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c41f20e0 con 0x7f83c40cb910 2024-09-15T21:47:46.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.659+0000 7f83c0ff9640 1 -- 172.21.15.100:0/1389423667 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b4007260 con 0x7f83c40cb910 2024-09-15T21:47:46.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.659+0000 7f83ca65e640 1 -- 172.21.15.100:0/1389423667 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c4159620 con 0x7f83c40cb910 2024-09-15T21:47:46.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.660+0000 7f83ca65e640 1 -- 172.21.15.100:0/1389423667 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83c40cbda0 con 0x7f83c40cb910 2024-09-15T21:47:46.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.661+0000 7f83c0ff9640 1 -- 172.21.15.100:0/1389423667 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f83b4007400 con 0x7f83c40cb910 2024-09-15T21:47:46.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.661+0000 7f83c0ff9640 1 --2- 172.21.15.100:0/1389423667 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f83b006c2b0 0x7f83b006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:46.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.662+0000 7f83c0ff9640 1 -- 172.21.15.100:0/1389423667 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5854+0+0 (secure 0 0 0) 0x7f83b408a8f0 con 0x7f83c40cb910 2024-09-15T21:47:46.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.662+0000 7f83c37fe640 1 --2- 172.21.15.100:0/1389423667 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f83b006c2b0 0x7f83b006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:46.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.662+0000 7f83c37fe640 1 --2- 172.21.15.100:0/1389423667 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f83b006c2b0 0x7f83b006e770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f83b8035040 tx=0x7f83b802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:46.666 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:46.664+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:46.667 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:46.666+0000 7f83c0ff9640 1 -- 172.21.15.100:0/1389423667 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f83b4058960 con 0x7f83c40cb910 2024-09-15T21:47:46.733 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:46.733 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:46.740 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:47:46.784 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:46.782+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:46.868 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:46.866+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:46.876 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T21:47:46.880 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T21:47:46.907 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:46.907 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:46.920 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:47:46.957 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:47:46.969 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:46.968+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:47.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.032+0000 7f83ca65e640 1 -- 172.21.15.100:0/1389423667 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 2, "weight": 0.18183040809386986} v 0) -- 0x7f83c40c7980 con 0x7f83c40cb910 2024-09-15T21:47:47.070 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:47.069+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:47.122 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.121+0000 7f56b0e53640 1 Processor -- start 2024-09-15T21:47:47.123 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.121+0000 7f56b0e53640 1 -- start start 2024-09-15T21:47:47.124 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.122+0000 7f56b0e53640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f56ac15c330 0x7f56ac0c67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.122+0000 7f56b0e53640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f56ac0c57a0 0x7f56ac0c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.122+0000 7f56b0e53640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56ac06ff60 con 0x7f56ac152d30 2024-09-15T21:47:47.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.123+0000 7f56aa59c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f56ac152d30 0x7f56ac159750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42112/0 (socket says 172.21.15.100:42112) 2024-09-15T21:47:47.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.123+0000 7f56aa59c640 1 -- 172.21.15.100:0/1191360004 learned_addr learned my addr 172.21.15.100:0/1191360004 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:47.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.123+0000 7f56a9d9b640 1 --2- 172.21.15.100:0/1191360004 >> v2:172.21.15.122:3300/0 conn(0x7f56ac15c330 0x7f56ac0c67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.125 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.123+0000 7f56b0e53640 1 -- 172.21.15.100:0/1191360004 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f56ac0c8df0 con 0x7f56ac0c57a0 2024-09-15T21:47:47.125 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:47.125 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:47.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.123+0000 7f56b0e53640 1 -- 172.21.15.100:0/1191360004 --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f56ac0c8550 con 0x7f56ac15c330 2024-09-15T21:47:47.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.123+0000 7f56aad9d640 1 --2- 172.21.15.100:0/1191360004 >> v2:172.21.15.100:3300/0 conn(0x7f56ac0c57a0 0x7f56ac0c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.123+0000 7f56a959a640 1 -- 172.21.15.100:0/1191360004 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2315404084 0 0) 0x7f56ac06ff60 con 0x7f56ac152d30 2024-09-15T21:47:47.129 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.123+0000 7f56a959a640 1 -- 172.21.15.100:0/1191360004 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5694003520 con 0x7f56ac152d30 2024-09-15T21:47:47.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56a959a640 1 -- 172.21.15.100:0/1191360004 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 594773276 0 0) 0x7f5694003520 con 0x7f56ac152d30 2024-09-15T21:47:47.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56a959a640 1 -- 172.21.15.100:0/1191360004 >> v2:172.21.15.122:3300/0 conn(0x7f56ac15c330 msgr2=0x7f56ac0c67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:47.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56a959a640 1 --2- 172.21.15.100:0/1191360004 >> v2:172.21.15.122:3300/0 conn(0x7f56ac15c330 0x7f56ac0c67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:47.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56a959a640 1 -- 172.21.15.100:0/1191360004 >> v2:172.21.15.100:3300/0 conn(0x7f56ac0c57a0 msgr2=0x7f56ac0c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:47.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56a959a640 1 --2- 172.21.15.100:0/1191360004 >> v2:172.21.15.100:3300/0 conn(0x7f56ac0c57a0 0x7f56ac0c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:47.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56a959a640 1 -- 172.21.15.100:0/1191360004 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ac0c8830 con 0x7f56ac152d30 2024-09-15T21:47:47.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56a959a640 1 -- 172.21.15.100:0/1191360004 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7f569c001670 con 0x7f56ac152d30 2024-09-15T21:47:47.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56a959a640 1 -- 172.21.15.100:0/1191360004 >> v1:172.21.15.100:6789/0 conn(0x7f56ac152d30 legacy=0x7f56ac159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:47.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56a959a640 1 --2- 172.21.15.100:0/1191360004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5694003a00 0x7f5694003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56a959a640 1 --2- 172.21.15.100:0/1191360004 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5694004800 0x7f5694004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56a959a640 1 --2- 172.21.15.100:0/1191360004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56940051c0 0x7f569400fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56a959a640 1 -- 172.21.15.100:0/1191360004 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac0c8df0 con 0x7f56940051c0 2024-09-15T21:47:47.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56a959a640 1 -- 172.21.15.100:0/1191360004 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac0c8550 con 0x7f5694003a00 2024-09-15T21:47:47.130 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56a959a640 1 -- 172.21.15.100:0/1191360004 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5694010450 con 0x7f5694004800 2024-09-15T21:47:47.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56aad9d640 1 --2- 172.21.15.100:0/1191360004 >> v2:172.21.15.100:3300/0 conn(0x7f56ac0c57a0 0x7f56ac0c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:47:47.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56aa59c640 1 --2- 172.21.15.100:0/1191360004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5694003a00 0x7f5694003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56a9d9b640 1 --2- 172.21.15.100:0/1191360004 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5694004800 0x7f5694004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56aad9d640 1 --2- 172.21.15.100:0/1191360004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56940051c0 0x7f569400fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56aa59c640 1 -- 172.21.15.100:0/1191360004 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5694004800 msgr2=0x7f5694004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:47.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56aa59c640 1 --2- 172.21.15.100:0/1191360004 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5694004800 0x7f5694004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:47.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56aa59c640 1 -- 172.21.15.100:0/1191360004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56940051c0 msgr2=0x7f569400fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:47.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56aa59c640 1 --2- 172.21.15.100:0/1191360004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56940051c0 0x7f569400fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:47.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56aa59c640 1 -- 172.21.15.100:0/1191360004 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5694003520 con 0x7f5694003a00 2024-09-15T21:47:47.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.124+0000 7f56aad9d640 1 --2- 172.21.15.100:0/1191360004 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56940051c0 0x7f569400fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:47:47.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.125+0000 7f56aad9d640 1 -- 172.21.15.100:0/1191360004 reap_dead start 2024-09-15T21:47:47.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.126+0000 7f56aa59c640 1 --2- 172.21.15.100:0/1191360004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5694003a00 0x7f5694003e20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f569c00c160 tx=0x7f5694005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4863094f1a656eaf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:47.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.126+0000 7f56a959a640 1 -- 172.21.15.100:0/1191360004 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c007c80 con 0x7f5694003a00 2024-09-15T21:47:47.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.126+0000 7f56a959a640 1 -- 172.21.15.100:0/1191360004 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f569c005ad0 con 0x7f5694003a00 2024-09-15T21:47:47.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.126+0000 7f56a959a640 1 -- 172.21.15.100:0/1191360004 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c007070 con 0x7f5694003a00 2024-09-15T21:47:47.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.127+0000 7f56b0e53640 1 -- 172.21.15.100:0/1191360004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5694003a00 msgr2=0x7f5694003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:47.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.127+0000 7f56b0e53640 1 --2- 172.21.15.100:0/1191360004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5694003a00 0x7f5694003e20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f569c00c160 tx=0x7f5694005830 comp rx=0 tx=0).stop 2024-09-15T21:47:47.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.127+0000 7f56b0e53640 1 -- 172.21.15.100:0/1191360004 shutdown_connections 2024-09-15T21:47:47.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.127+0000 7f56b0e53640 1 --2- 172.21.15.100:0/1191360004 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5694003a00 0x7f5694003e20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:47.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.127+0000 7f56b0e53640 1 -- 172.21.15.100:0/1191360004 >> 172.21.15.100:0/1191360004 conn(0x7f56ac0cb3a0 msgr2=0x7f56ac0cb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:47.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.127+0000 7f56b0e53640 1 -- 172.21.15.100:0/1191360004 shutdown_connections 2024-09-15T21:47:47.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.127+0000 7f56b0e53640 1 -- 172.21.15.100:0/1191360004 wait complete. 2024-09-15T21:47:47.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.129+0000 7f56b0e53640 1 Processor -- start 2024-09-15T21:47:47.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.129+0000 7f56b0e53640 1 -- start start 2024-09-15T21:47:47.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.130+0000 7f56b0e53640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5694003a00 0x7f56ac1521b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.130+0000 7f56b0e53640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56ac14fda0 0x7f56ac1499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.130+0000 7f56b0e53640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56ac15a4c0 0x7f56ac15a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.131+0000 7f56aa59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5694003a00 0x7f56ac1521b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.131+0000 7f56aa59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5694003a00 0x7f56ac1521b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:34092/0 (socket says 172.21.15.100:34092) 2024-09-15T21:47:47.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.131+0000 7f56a9d9b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56ac14fda0 0x7f56ac1499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.131+0000 7f56a9d9b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56ac14fda0 0x7f56ac1499b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:58026/0 (socket says 172.21.15.100:58026) 2024-09-15T21:47:47.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.131+0000 7f56aad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56ac15a4c0 0x7f56ac15a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.131+0000 7f56aad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56ac15a4c0 0x7f56ac15a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57464/0 (socket says 172.21.15.100:57464) 2024-09-15T21:47:47.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.131+0000 7f56b0e53640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac159230 con 0x7f56ac15a4c0 2024-09-15T21:47:47.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.131+0000 7f56b0e53640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac158f50 con 0x7f5694003a00 2024-09-15T21:47:47.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.131+0000 7f56aa59c640 1 -- 172.21.15.100:0/4068771697 learned_addr learned my addr 172.21.15.100:0/4068771697 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:47.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.131+0000 7f56b0e53640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f56ac153110 con 0x7f56ac14fda0 2024-09-15T21:47:47.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.131+0000 7f56aa59c640 1 -- 172.21.15.100:0/4068771697 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56ac14fda0 msgr2=0x7f56ac1499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:47.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.131+0000 7f56aa59c640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f56ac14fda0 0x7f56ac1499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:47.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.131+0000 7f56aa59c640 1 -- 172.21.15.100:0/4068771697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56ac15a4c0 msgr2=0x7f56ac15a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:47.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.131+0000 7f56aa59c640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56ac15a4c0 0x7f56ac15a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:47.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.131+0000 7f56aa59c640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ac1470c0 con 0x7f5694003a00 2024-09-15T21:47:47.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.131+0000 7f56aad9d640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56ac15a4c0 0x7f56ac15a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:47.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.132+0000 7f56aa59c640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5694003a00 0x7f56ac1521b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f569c02acb0 tx=0x7f56ac0c9ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:47.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.132+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c03f070 con 0x7f5694003a00 2024-09-15T21:47:47.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.132+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f569c001430 con 0x7f5694003a00 2024-09-15T21:47:47.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.132+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f569c007070 con 0x7f5694003a00 2024-09-15T21:47:47.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.132+0000 7f56b0e53640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ac15ae80 con 0x7f5694003a00 2024-09-15T21:47:47.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.132+0000 7f56b0e53640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ac150550 con 0x7f5694003a00 2024-09-15T21:47:47.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.133+0000 7f56b0e53640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f56ac0c8110 con 0x7f5694003a00 2024-09-15T21:47:47.138 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:47:47.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.134+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f569c042070 con 0x7f5694003a00 2024-09-15T21:47:47.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.135+0000 7f569affd640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f568c06c2e0 0x7f568c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.135+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5854+0+0 (secure 0 0 0) 0x7f569c0ab260 con 0x7f5694003a00 2024-09-15T21:47:47.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.135+0000 7f56a9d9b640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f568c06c2e0 0x7f568c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.135+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v41) ==== 54+0+16 (secure 0 0 0) 0x7f569c049020 con 0x7f5694003a00 2024-09-15T21:47:47.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.135+0000 7f5698ff9640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f5684001650 0x7f5684003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.136+0000 7f5698ff9640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5684006bb0 con 0x7f5684001650 2024-09-15T21:47:47.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.136+0000 7f56aad9d640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f5684001650 0x7f5684003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.136+0000 7f56a9d9b640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f568c06c2e0 0x7f568c06e7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f56ac1ea4a0 tx=0x7f56a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:47.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.136+0000 7f56aad9d640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f5684001650 0x7f5684003b10 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:47.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.137+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5684006bb0 con 0x7f5684001650 2024-09-15T21:47:47.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.165+0000 7f5698ff9640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5684005c00 con 0x7f5684001650 2024-09-15T21:47:47.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.168+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5684005c00 con 0x7f5684001650 2024-09-15T21:47:47.169 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:47.170 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:47.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.168+0000 7f5698ff9640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f5684004190 0x7f5684004640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.168+0000 7f5698ff9640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f5684004b80 con 0x7f5684004190 2024-09-15T21:47:47.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.168+0000 7f56aa59c640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f5684004190 0x7f5684004640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.169+0000 7f56aa59c640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f5684004190 0x7f5684004640 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:47.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.170+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5684004b80 con 0x7f5684004190 2024-09-15T21:47:47.171 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:47.170+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:47.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.198+0000 7f5698ff9640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5684005090 con 0x7f5684004190 2024-09-15T21:47:47.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.200+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5684005090 con 0x7f5684004190 2024-09-15T21:47:47.202 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:47.203 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:47.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.201+0000 7f5698ff9640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f5684009340 0x7f568400b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.201+0000 7f5698ff9640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f568400e5b0 con 0x7f5684009340 2024-09-15T21:47:47.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.201+0000 7f56a9d9b640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f5684009340 0x7f568400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.201+0000 7f56a9d9b640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f5684009340 0x7f568400b730 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:47.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.202+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f568400e5b0 con 0x7f5684009340 2024-09-15T21:47:47.231 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.229+0000 7f5698ff9640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f568400c000 con 0x7f5684009340 2024-09-15T21:47:47.234 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.232+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f568400c000 con 0x7f5684009340 2024-09-15T21:47:47.234 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:47.235 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:47.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.232+0000 7f5698ff9640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f568400c1b0 0x7f568400c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.232+0000 7f5698ff9640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f568400cb50 con 0x7f568400c1b0 2024-09-15T21:47:47.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.233+0000 7f56aad9d640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f568400c1b0 0x7f568400c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.233+0000 7f56aad9d640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f568400c1b0 0x7f568400c610 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:47.236 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.234+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f568400cb50 con 0x7f568400c1b0 2024-09-15T21:47:47.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.260+0000 7f5698ff9640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f568400d060 con 0x7f568400c1b0 2024-09-15T21:47:47.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.262+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f568400d060 con 0x7f568400c1b0 2024-09-15T21:47:47.265 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:47.266 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:47.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.263+0000 7f5698ff9640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f5684010800 0x7f5684012bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.263+0000 7f5698ff9640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f56840159c0 con 0x7f5684010800 2024-09-15T21:47:47.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.263+0000 7f56aa59c640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f5684010800 0x7f5684012bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.263+0000 7f56aa59c640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f5684010800 0x7f5684012bf0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:47.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.264+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f56840159c0 con 0x7f5684010800 2024-09-15T21:47:47.273 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:47.271+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:47.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.291+0000 7f5698ff9640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5684013130 con 0x7f5684010800 2024-09-15T21:47:47.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.293+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5684013130 con 0x7f5684010800 2024-09-15T21:47:47.295 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.296 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:32 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'18, mlcod=17'14)", 2024-09-15T21:47:47.297 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:47.297 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 92.935544293999996, 2024-09-15T21:47:47.297 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052297760000000002, 2024-09-15T21:47:47.297 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.297 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.297 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:47.297 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.298 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.298 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.298 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:47.298 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.298 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.298 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.298 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.298 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:47.298 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.299 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.299 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.299 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.299 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:47.299 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.06e-07 2024-09-15T21:47:47.299 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.300 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.300 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.300 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356780+0000", 2024-09-15T21:47:47.300 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4208999999999994e-05 2024-09-15T21:47:47.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.300 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.300 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.300 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356789+0000", 2024-09-15T21:47:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2800000000000003e-06 2024-09-15T21:47:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356793+0000", 2024-09-15T21:47:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:47:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.301 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356838+0000", 2024-09-15T21:47:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4851999999999997e-05 2024-09-15T21:47:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356854+0000", 2024-09-15T21:47:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6124999999999999e-05 2024-09-15T21:47:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.302 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.361892+0000", 2024-09-15T21:47:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050379140000000001 2024-09-15T21:47:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.361935+0000", 2024-09-15T21:47:47.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3217999999999999e-05 2024-09-15T21:47:47.304 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.304 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.304 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:35 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'21, mlcod=17'15)", 2024-09-15T21:47:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 92.933507164000005, 2024-09-15T21:47:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055652039999999998, 2024-09-15T21:47:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.306 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358744+0000", 2024-09-15T21:47:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4399999999999998e-07 2024-09-15T21:47:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358773+0000", 2024-09-15T21:47:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8847e-05 2024-09-15T21:47:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358783+0000", 2024-09-15T21:47:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0299000000000001e-05 2024-09-15T21:47:47.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358796+0000", 2024-09-15T21:47:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3499000000000001e-05 2024-09-15T21:47:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358850+0000", 2024-09-15T21:47:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3952000000000001e-05 2024-09-15T21:47:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358872+0000", 2024-09-15T21:47:47.308 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1424999999999999e-05 2024-09-15T21:47:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364273+0000", 2024-09-15T21:47:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054011299999999996 2024-09-15T21:47:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364308+0000", 2024-09-15T21:47:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5108000000000003e-05 2024-09-15T21:47:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.309 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:36 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'22, mlcod=17'15)", 2024-09-15T21:47:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 92.932712973999998, 2024-09-15T21:47:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047904530000000001, 2024-09-15T21:47:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.310 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6000000000000004e-07 2024-09-15T21:47:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359542+0000", 2024-09-15T21:47:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3510000000000004e-06 2024-09-15T21:47:47.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359552+0000", 2024-09-15T21:47:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0263e-05 2024-09-15T21:47:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359557+0000", 2024-09-15T21:47:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0479999999999998e-06 2024-09-15T21:47:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.312 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359593+0000", 2024-09-15T21:47:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6151e-05 2024-09-15T21:47:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359608+0000", 2024-09-15T21:47:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4609e-05 2024-09-15T21:47:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364310+0000", 2024-09-15T21:47:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047025510000000001 2024-09-15T21:47:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364327+0000", 2024-09-15T21:47:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6920000000000001e-05 2024-09-15T21:47:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=18'85)", 2024-09-15T21:47:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 91.394167038999996, 2024-09-15T21:47:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057443499999999996, 2024-09-15T21:47:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.314 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8899999999999999e-07 2024-09-15T21:47:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898167+0000", 2024-09-15T21:47:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3171999999999996e-05 2024-09-15T21:47:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898177+0000", 2024-09-15T21:47:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0866000000000001e-05 2024-09-15T21:47:47.316 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898185+0000", 2024-09-15T21:47:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2490000000000004e-06 2024-09-15T21:47:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898224+0000", 2024-09-15T21:47:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9237999999999999e-05 2024-09-15T21:47:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898238+0000", 2024-09-15T21:47:47.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.366e-05 2024-09-15T21:47:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.903803+0000", 2024-09-15T21:47:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005565291 2024-09-15T21:47:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.903827+0000", 2024-09-15T21:47:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4284999999999999e-05 2024-09-15T21:47:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:132 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'90, mlcod=18'85)", 2024-09-15T21:47:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 91.393675314999996, 2024-09-15T21:47:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055085469999999999, 2024-09-15T21:47:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9299999999999999e-07 2024-09-15T21:47:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898579+0000", 2024-09-15T21:47:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6650000000000001e-06 2024-09-15T21:47:47.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898586+0000", 2024-09-15T21:47:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8429999999999993e-06 2024-09-15T21:47:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898592+0000", 2024-09-15T21:47:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3990000000000003e-06 2024-09-15T21:47:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898621+0000", 2024-09-15T21:47:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9019000000000001e-05 2024-09-15T21:47:47.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898630+0000", 2024-09-15T21:47:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5540000000000003e-06 2024-09-15T21:47:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904047+0000", 2024-09-15T21:47:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054167169999999997 2024-09-15T21:47:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904083+0000", 2024-09-15T21:47:47.322 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5957000000000001e-05 2024-09-15T21:47:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-15T21:47:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 91.393086166000003, 2024-09-15T21:47:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049424780000000001, 2024-09-15T21:47:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.323 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899166+0000", 2024-09-15T21:47:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0779999999999998e-06 2024-09-15T21:47:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.324 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899174+0000", 2024-09-15T21:47:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.729e-06 2024-09-15T21:47:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899181+0000", 2024-09-15T21:47:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4080000000000001e-06 2024-09-15T21:47:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899185+0000", 2024-09-15T21:47:47.325 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0840000000000002e-06 2024-09-15T21:47:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899240+0000", 2024-09-15T21:47:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4874999999999999e-05 2024-09-15T21:47:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899257+0000", 2024-09-15T21:47:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7549000000000001e-05 2024-09-15T21:47:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904085+0000", 2024-09-15T21:47:47.326 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048272640000000004 2024-09-15T21:47:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904106+0000", 2024-09-15T21:47:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1491000000000001e-05 2024-09-15T21:47:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 78.878862889999994, 2024-09-15T21:47:47.327 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046608459999999997, 2024-09-15T21:47:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:47.328 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413391+0000", 2024-09-15T21:47:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9899999999999999e-06 2024-09-15T21:47:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413399+0000", 2024-09-15T21:47:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6310000000000003e-06 2024-09-15T21:47:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.329 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413444+0000", 2024-09-15T21:47:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5052000000000002e-05 2024-09-15T21:47:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413453+0000", 2024-09-15T21:47:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8869999999999998e-06 2024-09-15T21:47:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413576+0000", 2024-09-15T21:47:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012358700000000001 2024-09-15T21:47:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.330 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413679+0000", 2024-09-15T21:47:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000103318 2024-09-15T21:47:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413785+0000", 2024-09-15T21:47:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010561 2024-09-15T21:47:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:47.331 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416276+0000", 2024-09-15T21:47:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002490754 2024-09-15T21:47:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417949+0000", 2024-09-15T21:47:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016733519999999999 2024-09-15T21:47:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418005+0000", 2024-09-15T21:47:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5747e-05 2024-09-15T21:47:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.332 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418048+0000", 2024-09-15T21:47:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2917999999999998e-05 2024-09-15T21:47:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 78.878848419999997, 2024-09-15T21:47:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051419999999999999, 2024-09-15T21:47:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.333 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.334 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413402+0000", 2024-09-15T21:47:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6899999999999998e-07 2024-09-15T21:47:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413405+0000", 2024-09-15T21:47:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8109999999999999e-06 2024-09-15T21:47:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413517+0000", 2024-09-15T21:47:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000112592 2024-09-15T21:47:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413527+0000", 2024-09-15T21:47:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0144e-05 2024-09-15T21:47:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414155+0000", 2024-09-15T21:47:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00062767200000000002 2024-09-15T21:47:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414239+0000", 2024-09-15T21:47:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3607000000000001e-05 2024-09-15T21:47:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.336 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414368+0000", 2024-09-15T21:47:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000129049 2024-09-15T21:47:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416332+0000", 2024-09-15T21:47:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019645109999999999 2024-09-15T21:47:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418446+0000", 2024-09-15T21:47:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021137640000000002 2024-09-15T21:47:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.337 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418503+0000", 2024-09-15T21:47:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7166999999999997e-05 2024-09-15T21:47:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418543+0000", 2024-09-15T21:47:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0314000000000001e-05 2024-09-15T21:47:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:47.338 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 78.878835229000003, 2024-09-15T21:47:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057945979999999998, 2024-09-15T21:47:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.339 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9499999999999998e-07 2024-09-15T21:47:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413417+0000", 2024-09-15T21:47:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3910000000000001e-06 2024-09-15T21:47:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.340 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414570+0000", 2024-09-15T21:47:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011525890000000001 2024-09-15T21:47:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414577+0000", 2024-09-15T21:47:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1840000000000002e-06 2024-09-15T21:47:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414733+0000", 2024-09-15T21:47:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015563900000000001 2024-09-15T21:47:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.341 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414831+0000", 2024-09-15T21:47:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8138000000000001e-05 2024-09-15T21:47:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414956+0000", 2024-09-15T21:47:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000124604 2024-09-15T21:47:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418804+0000", 2024-09-15T21:47:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038484299999999999 2024-09-15T21:47:47.342 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419124+0000", 2024-09-15T21:47:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00032017699999999999 2024-09-15T21:47:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419171+0000", 2024-09-15T21:47:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6520000000000002e-05 2024-09-15T21:47:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419209+0000", 2024-09-15T21:47:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8531000000000001e-05 2024-09-15T21:47:47.343 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 78.878828132999999, 2024-09-15T21:47:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059183300000000003, 2024-09-15T21:47:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.344 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3000000000000002e-07 2024-09-15T21:47:47.345 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413426+0000", 2024-09-15T21:47:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7110000000000001e-06 2024-09-15T21:47:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414621+0000", 2024-09-15T21:47:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001195022 2024-09-15T21:47:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414625+0000", 2024-09-15T21:47:47.346 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6360000000000002e-06 2024-09-15T21:47:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415269+0000", 2024-09-15T21:47:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00064315399999999995 2024-09-15T21:47:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415356+0000", 2024-09-15T21:47:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7410999999999994e-05 2024-09-15T21:47:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419340+0000", 2024-09-15T21:47:47.347 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0039840659999999996 2024-09-15T21:47:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-15T21:47:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 78.878520542000004, 2024-09-15T21:47:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053379719999999999, 2024-09-15T21:47:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.348 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.349 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413730+0000", 2024-09-15T21:47:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1379999999999999e-06 2024-09-15T21:47:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413740+0000", 2024-09-15T21:47:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4830000000000005e-06 2024-09-15T21:47:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413751+0000", 2024-09-15T21:47:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1228e-05 2024-09-15T21:47:47.350 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413758+0000", 2024-09-15T21:47:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1289999999999998e-06 2024-09-15T21:47:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414038+0000", 2024-09-15T21:47:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00027935999999999998 2024-09-15T21:47:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414055+0000", 2024-09-15T21:47:47.351 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7325000000000001e-05 2024-09-15T21:47:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419028+0000", 2024-09-15T21:47:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049730720000000003 2024-09-15T21:47:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419067+0000", 2024-09-15T21:47:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9236999999999997e-05 2024-09-15T21:47:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.352 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 72.836597275000003, 2024-09-15T21:47:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.92695576499999999, 2024-09-15T21:47:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.353 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8500000000000002e-07 2024-09-15T21:47:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455655+0000", 2024-09-15T21:47:47.354 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.401e-06 2024-09-15T21:47:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.380965+0000", 2024-09-15T21:47:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.92530946000000003 2024-09-15T21:47:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.380981+0000", 2024-09-15T21:47:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6209e-05 2024-09-15T21:47:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.355 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381086+0000", 2024-09-15T21:47:47.356 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105011 2024-09-15T21:47:47.356 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.356 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.356 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.356 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:47.357 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381154+0000", 2024-09-15T21:47:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7596999999999997e-05 2024-09-15T21:47:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381249+0000", 2024-09-15T21:47:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5354999999999998e-05 2024-09-15T21:47:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.357 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381844+0000", 2024-09-15T21:47:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00059518600000000004 2024-09-15T21:47:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382531+0000", 2024-09-15T21:47:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00068671499999999996 2024-09-15T21:47:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382570+0000", 2024-09-15T21:47:47.358 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9373999999999999e-05 2024-09-15T21:47:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382608+0000", 2024-09-15T21:47:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8071999999999997e-05 2024-09-15T21:47:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 66.816688016000001, 2024-09-15T21:47:47.359 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047117319999999997, 2024-09-15T21:47:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:47.360 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4700000000000001e-07 2024-09-15T21:47:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475566+0000", 2024-09-15T21:47:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1669999999999999e-06 2024-09-15T21:47:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.361 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476316+0000", 2024-09-15T21:47:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075022999999999995 2024-09-15T21:47:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476324+0000", 2024-09-15T21:47:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.259e-06 2024-09-15T21:47:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476373+0000", 2024-09-15T21:47:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8807999999999998e-05 2024-09-15T21:47:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.362 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476446+0000", 2024-09-15T21:47:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3138000000000003e-05 2024-09-15T21:47:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476593+0000", 2024-09-15T21:47:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00014680199999999999 2024-09-15T21:47:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480189+0000", 2024-09-15T21:47:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035961370000000001 2024-09-15T21:47:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.363 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480198+0000", 2024-09-15T21:47:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.6649999999999992e-06 2024-09-15T21:47:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480235+0000", 2024-09-15T21:47:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6943999999999997e-05 2024-09-15T21:47:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480274+0000", 2024-09-15T21:47:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9035000000000001e-05 2024-09-15T21:47:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.364 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 66.816683013000002, 2024-09-15T21:47:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046830109999999999, 2024-09-15T21:47:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.365 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.53e-07 2024-09-15T21:47:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.366 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475570+0000", 2024-09-15T21:47:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7470000000000001e-06 2024-09-15T21:47:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476353+0000", 2024-09-15T21:47:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00078312300000000004 2024-09-15T21:47:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476359+0000", 2024-09-15T21:47:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.117e-06 2024-09-15T21:47:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.367 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476412+0000", 2024-09-15T21:47:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2398999999999998e-05 2024-09-15T21:47:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476524+0000", 2024-09-15T21:47:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011293 2024-09-15T21:47:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476909+0000", 2024-09-15T21:47:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00038490099999999999 2024-09-15T21:47:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.368 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479761+0000", 2024-09-15T21:47:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002851737 2024-09-15T21:47:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480154+0000", 2024-09-15T21:47:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00039307399999999998 2024-09-15T21:47:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480203+0000", 2024-09-15T21:47:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8418000000000003e-05 2024-09-15T21:47:47.369 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480250+0000", 2024-09-15T21:47:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7311999999999997e-05 2024-09-15T21:47:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 66.816678851000006, 2024-09-15T21:47:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005035398, 2024-09-15T21:47:47.370 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.371 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.22e-07 2024-09-15T21:47:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475574+0000", 2024-09-15T21:47:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.593e-06 2024-09-15T21:47:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476579+0000", 2024-09-15T21:47:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010053989999999999 2024-09-15T21:47:47.372 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476588+0000", 2024-09-15T21:47:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4530000000000002e-06 2024-09-15T21:47:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477723+0000", 2024-09-15T21:47:47.373 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001134962 2024-09-15T21:47:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477801+0000", 2024-09-15T21:47:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8228999999999999e-05 2024-09-15T21:47:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477946+0000", 2024-09-15T21:47:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00014553400000000001 2024-09-15T21:47:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:47.374 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480258+0000", 2024-09-15T21:47:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023112039999999999 2024-09-15T21:47:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480504+0000", 2024-09-15T21:47:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024667500000000002 2024-09-15T21:47:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480556+0000", 2024-09-15T21:47:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2114999999999998e-05 2024-09-15T21:47:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.375 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480606+0000", 2024-09-15T21:47:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0012000000000002e-05 2024-09-15T21:47:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 66.816674921000001, 2024-09-15T21:47:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050672490000000002, 2024-09-15T21:47:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.376 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.377 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475577+0000", 2024-09-15T21:47:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2290000000000002e-06 2024-09-15T21:47:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476605+0000", 2024-09-15T21:47:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010278780000000001 2024-09-15T21:47:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.378 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476611+0000", 2024-09-15T21:47:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5600000000000001e-06 2024-09-15T21:47:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477053+0000", 2024-09-15T21:47:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044162800000000001 2024-09-15T21:47:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477180+0000", 2024-09-15T21:47:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000127129 2024-09-15T21:47:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.379 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477288+0000", 2024-09-15T21:47:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010789999999999999 2024-09-15T21:47:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480288+0000", 2024-09-15T21:47:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003000052 2024-09-15T21:47:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480568+0000", 2024-09-15T21:47:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028026899999999998 2024-09-15T21:47:47.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480609+0000", 2024-09-15T21:47:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1437999999999999e-05 2024-09-15T21:47:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480642+0000", 2024-09-15T21:47:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2879999999999997e-05 2024-09-15T21:47:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.381 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:20 2.1f 2:fdd45aa0:::61c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 66.816671249999999, 2024-09-15T21:47:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053112589999999996, 2024-09-15T21:47:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.382 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475581+0000", 2024-09-15T21:47:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1009999999999999e-06 2024-09-15T21:47:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.383 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476628+0000", 2024-09-15T21:47:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010465380000000001 2024-09-15T21:47:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476633+0000", 2024-09-15T21:47:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9869999999999999e-06 2024-09-15T21:47:47.384 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.293+0000 7f5698ff9640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=42}) -- 0x7f5684015c20 con 0x7f5694003a00 2024-09-15T21:47:47.385 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.293+0000 7f5698ff9640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5684015df0 con 0x7f5694003a00 2024-09-15T21:47:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477618+0000", 2024-09-15T21:47:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00098539299999999999 2024-09-15T21:47:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.385 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477696+0000", 2024-09-15T21:47:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8509000000000003e-05 2024-09-15T21:47:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477824+0000", 2024-09-15T21:47:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000127062 2024-09-15T21:47:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480294+0000", 2024-09-15T21:47:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024703939999999999 2024-09-15T21:47:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.386 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480788+0000", 2024-09-15T21:47:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00049443800000000004 2024-09-15T21:47:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480845+0000", 2024-09-15T21:47:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6762000000000003e-05 2024-09-15T21:47:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480890+0000", 2024-09-15T21:47:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4789000000000001e-05 2024-09-15T21:47:47.387 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 66.816667778999999, 2024-09-15T21:47:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007191546, 2024-09-15T21:47:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.388 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9999999999999999e-07 2024-09-15T21:47:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.389 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475585+0000", 2024-09-15T21:47:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.384e-06 2024-09-15T21:47:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476665+0000", 2024-09-15T21:47:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010797140000000001 2024-09-15T21:47:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476671+0000", 2024-09-15T21:47:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3940000000000001e-06 2024-09-15T21:47:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.390 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478048+0000", 2024-09-15T21:47:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013767530000000001 2024-09-15T21:47:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478177+0000", 2024-09-15T21:47:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012898099999999999 2024-09-15T21:47:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482774+0000", 2024-09-15T21:47:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045970200000000003 2024-09-15T21:47:47.391 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 61.747438113999998, 2024-09-15T21:47:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055444480000000004, 2024-09-15T21:47:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.392 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.393 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4200000000000002e-07 2024-09-15T21:47:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544814+0000", 2024-09-15T21:47:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9980000000000002e-06 2024-09-15T21:47:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544872+0000", 2024-09-15T21:47:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8405000000000001e-05 2024-09-15T21:47:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544877+0000", 2024-09-15T21:47:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.143e-06 2024-09-15T21:47:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545750+0000", 2024-09-15T21:47:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00087310000000000003 2024-09-15T21:47:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545850+0000", 2024-09-15T21:47:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9939999999999995e-05 2024-09-15T21:47:47.395 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.396 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.396 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:47.396 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546014+0000", 2024-09-15T21:47:47.396 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00016408500000000001 2024-09-15T21:47:47.396 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.396 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.396 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:47.396 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.548748+0000", 2024-09-15T21:47:47.396 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027339539999999998 2024-09-15T21:47:47.396 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.396 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.396 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:47.396 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550265+0000", 2024-09-15T21:47:47.396 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015170019999999999 2024-09-15T21:47:47.397 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.397 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.397 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:47.397 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550315+0000", 2024-09-15T21:47:47.397 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9867999999999998e-05 2024-09-15T21:47:47.397 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.397 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.397 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.397 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550356+0000", 2024-09-15T21:47:47.397 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1711000000000001e-05 2024-09-15T21:47:47.397 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.397 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.397 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.397 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.398 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.398 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:47.398 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:47.398 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 61.747434912999999, 2024-09-15T21:47:47.398 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058729619999999998, 2024-09-15T21:47:47.398 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:47.398 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:47.398 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:47.398 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:47.398 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.398 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:47.398 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:47.398 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.398 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.399 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.399 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:47.399 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:47.399 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:47.399 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.399 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.399 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:47.399 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:47.399 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0599999999999999e-07 2024-09-15T21:47:47.399 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.399 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.399 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:47.399 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544821+0000", 2024-09-15T21:47:47.400 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4079999999999997e-06 2024-09-15T21:47:47.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.400 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.400 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:47.400 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544939+0000", 2024-09-15T21:47:47.400 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118532 2024-09-15T21:47:47.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.400 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.400 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:47.400 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544946+0000", 2024-09-15T21:47:47.400 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9260000000000003e-06 2024-09-15T21:47:47.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.400 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.400 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:47.401 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545011+0000", 2024-09-15T21:47:47.401 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4572000000000005e-05 2024-09-15T21:47:47.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.401 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.401 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:47.401 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545134+0000", 2024-09-15T21:47:47.401 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012309899999999999 2024-09-15T21:47:47.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.401 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.401 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:47.401 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545271+0000", 2024-09-15T21:47:47.401 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013746599999999999 2024-09-15T21:47:47.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.401 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.401 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:47.402 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550243+0000", 2024-09-15T21:47:47.402 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049714529999999998 2024-09-15T21:47:47.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.402 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.402 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:47.402 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550605+0000", 2024-09-15T21:47:47.402 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00036260699999999998 2024-09-15T21:47:47.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.402 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.402 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:47.402 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550662+0000", 2024-09-15T21:47:47.402 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6573000000000002e-05 2024-09-15T21:47:47.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:47.402 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:47.403 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:47.403 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550688+0000", 2024-09-15T21:47:47.403 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6120000000000001e-05 2024-09-15T21:47:47.403 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.403 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.403 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.403 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:47.403 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:47.403 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.294+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f569c0b2070 con 0x7f5694003a00 2024-09-15T21:47:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:47:47.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.294+0000 7f5698ff9640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f5684013c10 0x7f5684014060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.294+0000 7f5698ff9640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f56840159f0 con 0x7f5684013c10 2024-09-15T21:47:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.294+0000 7f56a9d9b640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f5684013c10 0x7f5684014060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.294+0000 7f56a9d9b640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f5684013c10 0x7f5684014060 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:47.405 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.295+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f56840159f0 con 0x7f5684013c10 2024-09-15T21:47:47.406 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:47:47.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.322+0000 7f5698ff9640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f5684014710 con 0x7f5684013c10 2024-09-15T21:47:47.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.324+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f5684014710 con 0x7f5684013c10 2024-09-15T21:47:47.407 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.324+0000 7f5698ff9640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f5684014e30 0x7f5684015290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.324+0000 7f5698ff9640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f568401c9d0 con 0x7f5684014e30 2024-09-15T21:47:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.325+0000 7f56aad9d640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f5684014e30 0x7f5684015290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.325+0000 7f56aad9d640 1 --2- 172.21.15.100:0/4068771697 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f5684014e30 0x7f5684015290 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.326+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f568401c9d0 con 0x7f5684014e30 2024-09-15T21:47:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.352+0000 7f5698ff9640 1 -- 172.21.15.100:0/4068771697 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f568401b850 con 0x7f5684014e30 2024-09-15T21:47:47.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.354+0000 7f569affd640 1 -- 172.21.15.100:0/4068771697 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f568401b850 con 0x7f5684014e30 2024-09-15T21:47:47.409 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:47.409 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:47:47.409 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:47.410 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:47:47.410 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:47.405+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:47.507 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:47.506+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:47.609 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:47.607+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:47.612 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:47.612 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:47.612 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:47.612 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:47.625 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:47:47.710 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:47.708+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:47.811 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:47.809+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:47.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.827+0000 7f83c0ff9640 1 -- 172.21.15.100:0/1389423667 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 2, "weight": 0.18183040809386986}]=0 set osd.2 primary-affinity to 0.18183 (8119162) v42) ==== 155+0+0 (secure 0 0 0) 0x7f83b405cc60 con 0x7f83c40cb910 2024-09-15T21:47:47.830 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.2 primary-affinity to 0.18183 (8119162) 2024-09-15T21:47:47.830 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:47.830 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:47.830 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:47.830 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:47.830 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:47.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.831+0000 7f83ca65e640 1 -- 172.21.15.100:0/1389423667 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f83b006c2b0 msgr2=0x7f83b006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:47.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.831+0000 7f83ca65e640 1 --2- 172.21.15.100:0/1389423667 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f83b006c2b0 0x7f83b006e770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f83b8035040 tx=0x7f83b802e000 comp rx=0 tx=0).stop 2024-09-15T21:47:47.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.831+0000 7f83ca65e640 1 -- 172.21.15.100:0/1389423667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83c40cb910 msgr2=0x7f83c41e8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:47.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.831+0000 7f83ca65e640 1 --2- 172.21.15.100:0/1389423667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83c40cb910 0x7f83c41e8280 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f83b400c340 tx=0x7f83b400c800 comp rx=0 tx=0).stop 2024-09-15T21:47:47.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.832+0000 7f83ca65e640 1 -- 172.21.15.100:0/1389423667 shutdown_connections 2024-09-15T21:47:47.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.832+0000 7f83ca65e640 1 --2- 172.21.15.100:0/1389423667 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f83b006c2b0 0x7f83b006e770 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:47.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.832+0000 7f83ca65e640 1 --2- 172.21.15.100:0/1389423667 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f83c4155680 0x7f83c414e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:47.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.832+0000 7f83ca65e640 1 --2- 172.21.15.100:0/1389423667 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83c41552a0 0x7f83c415a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:47.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.832+0000 7f83ca65e640 1 --2- 172.21.15.100:0/1389423667 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83c40cb910 0x7f83c41e8280 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:47.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.832+0000 7f83ca65e640 1 -- 172.21.15.100:0/1389423667 >> 172.21.15.100:0/1389423667 conn(0x7f83c406d9a0 msgr2=0x7f83c414fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:47.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.832+0000 7f83ca65e640 1 -- 172.21.15.100:0/1389423667 shutdown_connections 2024-09-15T21:47:47.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.832+0000 7f83ca65e640 1 -- 172.21.15.100:0/1389423667 wait complete. 2024-09-15T21:47:47.840 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:47:47.846 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:47:47.936 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:47.934+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:47.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.952+0000 7fdc68aa6640 1 Processor -- start 2024-09-15T21:47:47.955 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.953+0000 7fdc68aa6640 1 -- start start 2024-09-15T21:47:47.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.954+0000 7fdc68aa6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdc640cb7d0 0x7fdc640cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.954+0000 7fdc68aa6640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fdc6406d0e0 0x7fdc640cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.954+0000 7fdc68aa6640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc6415e7a0 con 0x7fdc64155440 2024-09-15T21:47:47.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.954+0000 7fdc68aa6640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fdc6415b470 con 0x7fdc6406d0e0 2024-09-15T21:47:47.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.954+0000 7fdc68aa6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fdc64155820 con 0x7fdc640cb7d0 2024-09-15T21:47:47.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.954+0000 7fdc62d9d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fdc6406d0e0 0x7fdc640cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.954+0000 7fdc61d9b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdc640cb7d0 0x7fdc640cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.954+0000 7fdc62d9d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fdc6406d0e0 0x7fdc640cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:34106/0 (socket says 172.21.15.100:34106) 2024-09-15T21:47:47.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.954+0000 7fdc62d9d640 1 -- 172.21.15.100:0/347746337 learned_addr learned my addr 172.21.15.100:0/347746337 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:47.960 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.954+0000 7fdc62d9d640 1 -- 172.21.15.100:0/347746337 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdc640cb7d0 msgr2=0x7fdc640cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:47.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.954+0000 7fdc62d9d640 1 --2- 172.21.15.100:0/347746337 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdc640cb7d0 0x7fdc640cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:47.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.954+0000 7fdc62d9d640 1 -- 172.21.15.100:0/347746337 >> v1:172.21.15.122:6789/0 conn(0x7fdc64155440 legacy=0x7fdc640cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:47.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.954+0000 7fdc62d9d640 1 -- 172.21.15.100:0/347746337 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc6415ea40 con 0x7fdc6406d0e0 2024-09-15T21:47:47.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.954+0000 7fdc61d9b640 1 --2- 172.21.15.100:0/347746337 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdc640cb7d0 0x7fdc640cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:47.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.955+0000 7fdc62d9d640 1 --2- 172.21.15.100:0/347746337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc6406d0e0 0x7fdc640cc720 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009e50 tx=0x7fdc5c02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3e963ea0f9af916 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:47.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.955+0000 7fdc6159a640 1 -- 172.21.15.100:0/347746337 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c036070 con 0x7fdc6406d0e0 2024-09-15T21:47:47.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.955+0000 7fdc6159a640 1 -- 172.21.15.100:0/347746337 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc5c031600 con 0x7fdc6406d0e0 2024-09-15T21:47:47.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.955+0000 7fdc6159a640 1 -- 172.21.15.100:0/347746337 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c0319d0 con 0x7fdc6406d0e0 2024-09-15T21:47:47.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.956+0000 7fdc68aa6640 1 -- 172.21.15.100:0/347746337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc6406d0e0 msgr2=0x7fdc640cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:47.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.956+0000 7fdc68aa6640 1 --2- 172.21.15.100:0/347746337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc6406d0e0 0x7fdc640cc720 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c009e50 tx=0x7fdc5c02a680 comp rx=0 tx=0).stop 2024-09-15T21:47:47.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.956+0000 7fdc68aa6640 1 -- 172.21.15.100:0/347746337 shutdown_connections 2024-09-15T21:47:47.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.956+0000 7fdc68aa6640 1 --2- 172.21.15.100:0/347746337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc6406d0e0 0x7fdc640cc720 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:47.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.956+0000 7fdc68aa6640 1 --2- 172.21.15.100:0/347746337 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdc640cb7d0 0x7fdc640cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:47.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.956+0000 7fdc68aa6640 1 -- 172.21.15.100:0/347746337 >> 172.21.15.100:0/347746337 conn(0x7fdc640680d0 msgr2=0x7fdc6406d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:47.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.956+0000 7fdc68aa6640 1 -- 172.21.15.100:0/347746337 shutdown_connections 2024-09-15T21:47:47.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.956+0000 7fdc68aa6640 1 -- 172.21.15.100:0/347746337 wait complete. 2024-09-15T21:47:47.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.958+0000 7fdc68aa6640 1 Processor -- start 2024-09-15T21:47:47.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.958+0000 7fdc68aa6640 1 -- start start 2024-09-15T21:47:47.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.959+0000 7fdc68aa6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc6406d0e0 0x7fdc64150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.959+0000 7fdc68aa6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc640cb7d0 0x7fdc640b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.959+0000 7fdc68aa6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdc64155440 0x7fdc640cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.959+0000 7fdc68aa6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6415f2e0 con 0x7fdc640cb7d0 2024-09-15T21:47:47.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.959+0000 7fdc68aa6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6415f000 con 0x7fdc6406d0e0 2024-09-15T21:47:47.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.959+0000 7fdc68aa6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fdc6415b070 con 0x7fdc64155440 2024-09-15T21:47:47.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.959+0000 7fdc61d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc640cb7d0 0x7fdc640b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.959+0000 7fdc61d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc640cb7d0 0x7fdc640b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57468/0 (socket says 172.21.15.100:57468) 2024-09-15T21:47:47.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.959+0000 7fdc61d9b640 1 -- 172.21.15.100:0/2695998836 learned_addr learned my addr 172.21.15.100:0/2695998836 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:47.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.959+0000 7fdc62d9d640 1 --2- 172.21.15.100:0/2695998836 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdc64155440 0x7fdc640cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.959+0000 7fdc6259c640 1 --2- 172.21.15.100:0/2695998836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc6406d0e0 0x7fdc64150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.959+0000 7fdc61d9b640 1 -- 172.21.15.100:0/2695998836 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdc64155440 msgr2=0x7fdc640cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:47.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.959+0000 7fdc61d9b640 1 --2- 172.21.15.100:0/2695998836 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdc64155440 0x7fdc640cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:47.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.959+0000 7fdc61d9b640 1 -- 172.21.15.100:0/2695998836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc6406d0e0 msgr2=0x7fdc64150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:47.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.959+0000 7fdc61d9b640 1 --2- 172.21.15.100:0/2695998836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc6406d0e0 0x7fdc64150be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:47.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.959+0000 7fdc61d9b640 1 -- 172.21.15.100:0/2695998836 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc640b5d90 con 0x7fdc640cb7d0 2024-09-15T21:47:47.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.959+0000 7fdc6259c640 1 --2- 172.21.15.100:0/2695998836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc6406d0e0 0x7fdc64150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:47:47.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.960+0000 7fdc61d9b640 1 --2- 172.21.15.100:0/2695998836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc640cb7d0 0x7fdc640b3f80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800ec10 tx=0x7fdc5800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:47.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.960+0000 7fdc52ffd640 1 -- 172.21.15.100:0/2695998836 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc58017070 con 0x7fdc640cb7d0 2024-09-15T21:47:47.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.960+0000 7fdc52ffd640 1 -- 172.21.15.100:0/2695998836 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc5800cc30 con 0x7fdc640cb7d0 2024-09-15T21:47:47.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.960+0000 7fdc68aa6640 1 -- 172.21.15.100:0/2695998836 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc640b3470 con 0x7fdc640cb7d0 2024-09-15T21:47:47.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.961+0000 7fdc52ffd640 1 -- 172.21.15.100:0/2695998836 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc58013470 con 0x7fdc640cb7d0 2024-09-15T21:47:47.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.961+0000 7fdc68aa6640 1 -- 172.21.15.100:0/2695998836 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc641543f0 con 0x7fdc640cb7d0 2024-09-15T21:47:47.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.962+0000 7fdc52ffd640 1 -- 172.21.15.100:0/2695998836 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdc5801a070 con 0x7fdc640cb7d0 2024-09-15T21:47:47.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.962+0000 7fdc68aa6640 1 -- 172.21.15.100:0/2695998836 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc640caed0 con 0x7fdc640cb7d0 2024-09-15T21:47:47.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.963+0000 7fdc52ffd640 1 --2- 172.21.15.100:0/2695998836 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fdc4406c3b0 0x7fdc4406e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:47.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.963+0000 7fdc52ffd640 1 -- 172.21.15.100:0/2695998836 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5886+0+0 (secure 0 0 0) 0x7fdc5808b470 con 0x7fdc640cb7d0 2024-09-15T21:47:47.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.963+0000 7fdc6259c640 1 --2- 172.21.15.100:0/2695998836 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fdc4406c3b0 0x7fdc4406e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:47.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.964+0000 7fdc6259c640 1 --2- 172.21.15.100:0/2695998836 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fdc4406c3b0 0x7fdc4406e870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdc54001700 tx=0x7fdc54009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:47.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:47.967+0000 7fdc52ffd640 1 -- 172.21.15.100:0/2695998836 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdc58059b10 con 0x7fdc640cb7d0 2024-09-15T21:47:48.013 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:48.012+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:48.023 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:48.023 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:48.023 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:48.023 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:48.023 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.024 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:48.024 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:48.024 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 79.607738230999999, 2024-09-15T21:47:48.024 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006101291, 2024-09-15T21:47:48.024 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.024 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.024 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.024 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.024 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.024 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.024 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:48.024 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.024 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.025 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.025 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.025 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:48.025 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.025 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.025 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.025 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413116+0000", 2024-09-15T21:47:48.025 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9700000000000003e-07 2024-09-15T21:47:48.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.025 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.025 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.025 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413118+0000", 2024-09-15T21:47:48.026 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2910000000000002e-06 2024-09-15T21:47:48.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.026 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.026 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.026 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413220+0000", 2024-09-15T21:47:48.026 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102056 2024-09-15T21:47:48.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.026 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.026 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.026 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413225+0000", 2024-09-15T21:47:48.026 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0490000000000003e-06 2024-09-15T21:47:48.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.026 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.027 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.027 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414808+0000", 2024-09-15T21:47:48.027 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015826900000000001 2024-09-15T21:47:48.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.027 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.027 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.027 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414907+0000", 2024-09-15T21:47:48.027 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9600999999999997e-05 2024-09-15T21:47:48.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.027 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.027 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:48.027 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415054+0000", 2024-09-15T21:47:48.027 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000146532 2024-09-15T21:47:48.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.028 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.028 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.028 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418056+0000", 2024-09-15T21:47:48.028 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030020519999999998 2024-09-15T21:47:48.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.028 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.028 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.028 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419128+0000", 2024-09-15T21:47:48.028 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001071671 2024-09-15T21:47:48.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.028 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.028 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.028 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419184+0000", 2024-09-15T21:47:48.028 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6125000000000002e-05 2024-09-15T21:47:48.029 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.029 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.029 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.029 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419217+0000", 2024-09-15T21:47:48.029 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2926999999999998e-05 2024-09-15T21:47:48.029 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.029 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.029 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.029 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.029 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.029 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:48.029 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:48.029 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 79.607734704999999, 2024-09-15T21:47:48.029 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059975130000000003, 2024-09-15T21:47:48.030 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.030 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.030 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.030 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.030 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.030 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.030 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:48.030 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.030 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.030 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.030 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:48.030 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.031 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.031 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.031 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:48.031 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6199999999999999e-07 2024-09-15T21:47:48.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.031 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.031 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.031 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413121+0000", 2024-09-15T21:47:48.031 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2000000000000001e-06 2024-09-15T21:47:48.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.032 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.032 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.032 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413292+0000", 2024-09-15T21:47:48.032 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017034400000000001 2024-09-15T21:47:48.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.032 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.032 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.032 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413300+0000", 2024-09-15T21:47:48.032 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8539999999999997e-06 2024-09-15T21:47:48.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.032 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.032 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.032 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414299+0000", 2024-09-15T21:47:48.033 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0009992829999999999 2024-09-15T21:47:48.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.033 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.033 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.033 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414390+0000", 2024-09-15T21:47:48.033 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1237999999999996e-05 2024-09-15T21:47:48.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.033 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.033 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:48.033 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414490+0000", 2024-09-15T21:47:48.033 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9438999999999994e-05 2024-09-15T21:47:48.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.033 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.033 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.034 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417393+0000", 2024-09-15T21:47:48.034 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002903306 2024-09-15T21:47:48.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.034 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.034 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419023+0000", 2024-09-15T21:47:48.034 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001630292 2024-09-15T21:47:48.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.034 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.034 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419067+0000", 2024-09-15T21:47:48.034 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4017999999999998e-05 2024-09-15T21:47:48.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.035 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.035 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419116+0000", 2024-09-15T21:47:48.035 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9277000000000003e-05 2024-09-15T21:47:48.035 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.035 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.035 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.035 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.035 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:48.035 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:48.035 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 79.605036507999998, 2024-09-15T21:47:48.035 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060879280000000003, 2024-09-15T21:47:48.035 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.035 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.036 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.036 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.036 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.036 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.036 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:48.036 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.036 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.036 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.036 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.036 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:48.036 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.036 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.036 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.036 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.037 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415818+0000", 2024-09-15T21:47:48.037 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:47:48.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.037 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.037 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.037 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415854+0000", 2024-09-15T21:47:48.037 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6706999999999999e-05 2024-09-15T21:47:48.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.037 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.037 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.037 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415860+0000", 2024-09-15T21:47:48.037 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0789999999999997e-06 2024-09-15T21:47:48.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.037 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.038 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.038 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415865+0000", 2024-09-15T21:47:48.038 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4580000000000003e-06 2024-09-15T21:47:48.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.038 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.038 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.038 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416092+0000", 2024-09-15T21:47:48.038 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00022739400000000001 2024-09-15T21:47:48.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.038 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.038 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.038 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416108+0000", 2024-09-15T21:47:48.038 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.613e-05 2024-09-15T21:47:48.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.039 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.039 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.039 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421866+0000", 2024-09-15T21:47:48.039 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005757585 2024-09-15T21:47:48.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.039 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.039 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.039 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421905+0000", 2024-09-15T21:47:48.040 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9214000000000001e-05 2024-09-15T21:47:48.040 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.040 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.040 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.040 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.040 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.040 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-15T21:47:48.040 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:48.040 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 79.604976347999994, 2024-09-15T21:47:48.040 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060637570000000003, 2024-09-15T21:47:48.040 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.040 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.040 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.041 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.041 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.041 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.041 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:48.041 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.041 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.041 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.041 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:48.041 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.042 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.042 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.042 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.042 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415878+0000", 2024-09-15T21:47:48.042 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.34e-07 2024-09-15T21:47:48.042 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.042 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.042 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.042 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415881+0000", 2024-09-15T21:47:48.042 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.354e-06 2024-09-15T21:47:48.042 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.042 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.042 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.043 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415887+0000", 2024-09-15T21:47:48.043 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:47:48.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.043 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.043 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.043 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415891+0000", 2024-09-15T21:47:48.043 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2690000000000001e-06 2024-09-15T21:47:48.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.043 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.043 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.043 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416267+0000", 2024-09-15T21:47:48.043 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00037614799999999998 2024-09-15T21:47:48.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.043 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.044 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.044 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416288+0000", 2024-09-15T21:47:48.044 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0604000000000001e-05 2024-09-15T21:47:48.044 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.044 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.044 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.044 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421908+0000", 2024-09-15T21:47:48.044 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056203390000000002 2024-09-15T21:47:48.044 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.044 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.044 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.044 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421941+0000", 2024-09-15T21:47:48.044 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3079e-05 2024-09-15T21:47:48.044 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.045 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.045 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.045 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.045 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.045 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:48.045 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:48.045 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 79.604957920999993, 2024-09-15T21:47:48.045 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006080284, 2024-09-15T21:47:48.045 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.045 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.045 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.045 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.045 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.045 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.046 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:48.046 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.046 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.046 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.046 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.046 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:48.046 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.046 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.046 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.046 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.046 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:48.046 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6899999999999999e-07 2024-09-15T21:47:48.046 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.046 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.047 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.047 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415898+0000", 2024-09-15T21:47:48.047 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.951e-06 2024-09-15T21:47:48.047 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.047 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.047 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.047 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415920+0000", 2024-09-15T21:47:48.047 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1651999999999999e-05 2024-09-15T21:47:48.047 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.047 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.047 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.047 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415925+0000", 2024-09-15T21:47:48.047 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.463e-06 2024-09-15T21:47:48.047 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.048 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.048 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.048 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416449+0000", 2024-09-15T21:47:48.048 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052384000000000003 2024-09-15T21:47:48.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.048 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.048 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.048 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416466+0000", 2024-09-15T21:47:48.048 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7108999999999998e-05 2024-09-15T21:47:48.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.048 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.049 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.049 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421943+0000", 2024-09-15T21:47:48.049 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054771569999999999 2024-09-15T21:47:48.049 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.049 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.049 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.049 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421976+0000", 2024-09-15T21:47:48.049 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2943e-05 2024-09-15T21:47:48.049 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.049 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.049 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.049 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.049 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.049 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-15T21:47:48.050 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:48.050 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 79.604918685000001, 2024-09-15T21:47:48.050 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060643340000000002, 2024-09-15T21:47:48.050 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.050 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.050 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.050 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.050 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.050 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.050 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:48.050 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.050 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.050 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.050 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.051 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:48.051 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.051 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.051 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.051 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.051 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:48.051 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4200000000000002e-07 2024-09-15T21:47:48.051 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.051 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.051 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.051 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415939+0000", 2024-09-15T21:47:48.052 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.439e-06 2024-09-15T21:47:48.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.052 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.052 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.052 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415945+0000", 2024-09-15T21:47:48.052 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7740000000000001e-06 2024-09-15T21:47:48.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.052 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.052 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.052 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415948+0000", 2024-09-15T21:47:48.052 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.737e-06 2024-09-15T21:47:48.053 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.053 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.053 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.053 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416605+0000", 2024-09-15T21:47:48.053 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00065721699999999998 2024-09-15T21:47:48.053 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.053 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.053 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.053 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416620+0000", 2024-09-15T21:47:48.053 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4564e-05 2024-09-15T21:47:48.053 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.053 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.053 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.054 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421977+0000", 2024-09-15T21:47:48.054 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053572369999999999 2024-09-15T21:47:48.054 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.054 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.054 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.054 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421999+0000", 2024-09-15T21:47:48.054 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2024000000000001e-05 2024-09-15T21:47:48.054 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.054 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.054 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.054 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.054 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.054 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-15T21:47:48.054 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:48.055 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 79.604718026, 2024-09-15T21:47:48.055 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058922669999999996, 2024-09-15T21:47:48.055 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.055 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.055 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.055 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.055 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.055 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.055 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:48.055 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.055 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.055 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.055 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.055 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:48.056 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.056 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.056 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.056 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.056 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:48.056 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6499999999999999e-07 2024-09-15T21:47:48.056 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.056 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.056 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.056 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416142+0000", 2024-09-15T21:47:48.056 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3929999999999999e-06 2024-09-15T21:47:48.056 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.056 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.056 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.057 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416151+0000", 2024-09-15T21:47:48.057 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5840000000000001e-06 2024-09-15T21:47:48.057 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.057 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.057 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.057 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416157+0000", 2024-09-15T21:47:48.057 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7590000000000003e-06 2024-09-15T21:47:48.057 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.057 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.057 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.057 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416767+0000", 2024-09-15T21:47:48.057 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00061034900000000005 2024-09-15T21:47:48.057 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.057 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.058 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.058 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416782+0000", 2024-09-15T21:47:48.058 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.483e-05 2024-09-15T21:47:48.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.058 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.058 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.058 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422002+0000", 2024-09-15T21:47:48.058 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052202639999999996 2024-09-15T21:47:48.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.058 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.058 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.058 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422028+0000", 2024-09-15T21:47:48.058 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5423e-05 2024-09-15T21:47:48.058 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.058 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.059 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.059 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.059 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.059 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-15T21:47:48.059 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:48.059 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 79.604311525, 2024-09-15T21:47:48.059 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055107769999999997, 2024-09-15T21:47:48.059 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.059 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.059 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.059 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.059 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.060 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.060 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:48.060 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.060 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.060 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.060 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.060 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:48.060 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.060 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.060 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.060 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.061 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416543+0000", 2024-09-15T21:47:48.061 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5400000000000001e-07 2024-09-15T21:47:48.061 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.061 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.061 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.061 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416548+0000", 2024-09-15T21:47:48.061 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8659999999999997e-06 2024-09-15T21:47:48.061 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.061 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.061 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.062 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416555+0000", 2024-09-15T21:47:48.062 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7190000000000001e-06 2024-09-15T21:47:48.062 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.062 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.062 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.062 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416560+0000", 2024-09-15T21:47:48.062 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3270000000000002e-06 2024-09-15T21:47:48.062 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.062 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.063 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.063 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416916+0000", 2024-09-15T21:47:48.063 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00035661200000000002 2024-09-15T21:47:48.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.063 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.063 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.063 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416932+0000", 2024-09-15T21:47:48.063 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5824e-05 2024-09-15T21:47:48.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.063 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.063 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.063 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422030+0000", 2024-09-15T21:47:48.063 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050975229999999996 2024-09-15T21:47:48.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.064 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.064 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.064 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422053+0000", 2024-09-15T21:47:48.064 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3252e-05 2024-09-15T21:47:48.064 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.064 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.064 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.064 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.064 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.064 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-15T21:47:48.064 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:48.064 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 79.603913026000001, 2024-09-15T21:47:48.064 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051422289999999999, 2024-09-15T21:47:48.064 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.065 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.065 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.065 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.065 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.065 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.065 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:48.065 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.065 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.065 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.065 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.065 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:48.065 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.065 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.065 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.066 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.066 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:48.066 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2699999999999999e-07 2024-09-15T21:47:48.066 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.066 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.066 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.066 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416946+0000", 2024-09-15T21:47:48.066 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1390000000000003e-06 2024-09-15T21:47:48.066 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.066 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.066 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.066 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416957+0000", 2024-09-15T21:47:48.066 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.044e-05 2024-09-15T21:47:48.066 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.067 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.067 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.067 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416963+0000", 2024-09-15T21:47:48.067 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5320000000000003e-06 2024-09-15T21:47:48.067 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.067 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.067 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.067 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417090+0000", 2024-09-15T21:47:48.067 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126549 2024-09-15T21:47:48.067 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.068 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.068 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.069 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417114+0000", 2024-09-15T21:47:48.070 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3736000000000001e-05 2024-09-15T21:47:48.070 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.070 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.070 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.070 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422055+0000", 2024-09-15T21:47:48.071 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049410410000000002 2024-09-15T21:47:48.071 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.071 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.071 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.071 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422083+0000", 2024-09-15T21:47:48.071 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8164999999999999e-05 2024-09-15T21:47:48.072 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.072 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.072 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.072 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.072 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.072 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.073 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:48.073 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 73.559421053999998, 2024-09-15T21:47:48.073 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007521186, 2024-09-15T21:47:48.073 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.073 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.077 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.077 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.077 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.077 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.077 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:48.077 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.078 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.078 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.078 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.078 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:48.078 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.080 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.080 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.081 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.081 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:48.081 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9999999999999998e-07 2024-09-15T21:47:48.081 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.081 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.081 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.082 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461443+0000", 2024-09-15T21:47:48.082 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0448000000000001e-05 2024-09-15T21:47:48.082 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.082 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.082 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.083 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462146+0000", 2024-09-15T21:47:48.084 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00070223499999999995 2024-09-15T21:47:48.084 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.084 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.084 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.084 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462154+0000", 2024-09-15T21:47:48.084 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9060000000000002e-06 2024-09-15T21:47:48.084 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.084 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.084 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.084 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462254+0000", 2024-09-15T21:47:48.085 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9897999999999998e-05 2024-09-15T21:47:48.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.085 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.085 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.085 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462330+0000", 2024-09-15T21:47:48.085 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6551000000000006e-05 2024-09-15T21:47:48.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.085 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.085 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:48.085 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462454+0000", 2024-09-15T21:47:48.085 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000124213 2024-09-15T21:47:48.086 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.086 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.086 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.086 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463681+0000", 2024-09-15T21:47:48.086 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012265729999999999 2024-09-15T21:47:48.086 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.086 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.087 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.087 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468817+0000", 2024-09-15T21:47:48.087 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051361430000000001 2024-09-15T21:47:48.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.087 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.087 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.087 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468892+0000", 2024-09-15T21:47:48.087 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5480000000000002e-05 2024-09-15T21:47:48.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.087 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.087 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.087 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468954+0000", 2024-09-15T21:47:48.087 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1339000000000003e-05 2024-09-15T21:47:48.087 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.088 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.088 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.089 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.089 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.089 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.089 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:48.089 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 73.559404439000005, 2024-09-15T21:47:48.089 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077296149999999996, 2024-09-15T21:47:48.089 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.089 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.089 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.089 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.089 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.089 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.089 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:48.090 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.090 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.090 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.090 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.090 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:48.090 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.090 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.090 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.090 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.090 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461450+0000", 2024-09-15T21:47:48.090 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0499999999999999e-07 2024-09-15T21:47:48.090 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.090 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.090 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.091 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461452+0000", 2024-09-15T21:47:48.091 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.007e-06 2024-09-15T21:47:48.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.091 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.091 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.091 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462209+0000", 2024-09-15T21:47:48.091 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075781000000000004 2024-09-15T21:47:48.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.091 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.091 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.091 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462214+0000", 2024-09-15T21:47:48.091 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1420000000000001e-06 2024-09-15T21:47:48.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.091 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.092 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.092 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462681+0000", 2024-09-15T21:47:48.092 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046634200000000002 2024-09-15T21:47:48.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.092 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.092 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.092 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462772+0000", 2024-09-15T21:47:48.092 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1414000000000003e-05 2024-09-15T21:47:48.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.092 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.092 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:48.092 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462871+0000", 2024-09-15T21:47:48.092 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8455000000000005e-05 2024-09-15T21:47:48.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.093 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.093 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.093 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464138+0000", 2024-09-15T21:47:48.093 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012673910000000001 2024-09-15T21:47:48.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.093 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.093 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.093 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469079+0000", 2024-09-15T21:47:48.093 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049407799999999997 2024-09-15T21:47:48.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.094 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.094 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.094 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469141+0000", 2024-09-15T21:47:48.094 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2169999999999996e-05 2024-09-15T21:47:48.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.094 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.094 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.094 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469179+0000", 2024-09-15T21:47:48.094 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7799000000000002e-05 2024-09-15T21:47:48.094 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.094 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.094 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.094 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.095 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.095 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:48.095 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 67.545288631000005, 2024-09-15T21:47:48.095 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004990497, 2024-09-15T21:47:48.095 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.095 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.095 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.095 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.095 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.095 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.095 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:48.095 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.095 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.096 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.096 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.096 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:48.096 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.096 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.096 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.096 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.096 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:48.096 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4729999999999999e-06 2024-09-15T21:47:48.096 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.096 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.096 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.096 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475576+0000", 2024-09-15T21:47:48.096 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1749999999999994e-06 2024-09-15T21:47:48.097 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.097 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.097 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.097 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475619+0000", 2024-09-15T21:47:48.097 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3557000000000003e-05 2024-09-15T21:47:48.097 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.097 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.097 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.097 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475631+0000", 2024-09-15T21:47:48.097 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1739999999999999e-05 2024-09-15T21:47:48.098 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.098 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.098 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.098 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475770+0000", 2024-09-15T21:47:48.098 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013950599999999999 2024-09-15T21:47:48.098 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.098 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.098 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.098 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475887+0000", 2024-09-15T21:47:48.098 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011616 2024-09-15T21:47:48.098 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.098 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.098 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:48.099 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476018+0000", 2024-09-15T21:47:48.099 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013119499999999999 2024-09-15T21:47:48.099 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.099 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.099 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.099 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478442+0000", 2024-09-15T21:47:48.099 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024240870000000001 2024-09-15T21:47:48.099 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.099 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.099 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.099 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480457+0000", 2024-09-15T21:47:48.099 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020151779999999998 2024-09-15T21:47:48.100 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.100 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.100 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.100 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480511+0000", 2024-09-15T21:47:48.100 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3582000000000003e-05 2024-09-15T21:47:48.100 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.101 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.101 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.101 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480556+0000", 2024-09-15T21:47:48.101 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4844e-05 2024-09-15T21:47:48.101 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.101 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.102 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.102 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.102 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.102 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.102 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:48.102 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 67.545274312999993, 2024-09-15T21:47:48.102 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051124810000000003, 2024-09-15T21:47:48.103 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.103 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.103 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.103 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.103 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.103 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.103 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:48.103 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.104 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.104 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.104 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.104 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:48.104 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.104 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.104 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.105 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.105 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475580+0000", 2024-09-15T21:47:48.105 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7800000000000002e-07 2024-09-15T21:47:48.105 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.105 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.105 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.105 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475595+0000", 2024-09-15T21:47:48.105 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4919000000000001e-05 2024-09-15T21:47:48.105 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.105 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.105 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.105 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475662+0000", 2024-09-15T21:47:48.105 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7769999999999997e-05 2024-09-15T21:47:48.105 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.106 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.106 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.106 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475669+0000", 2024-09-15T21:47:48.106 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.703e-06 2024-09-15T21:47:48.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.106 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.106 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.106 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476781+0000", 2024-09-15T21:47:48.106 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011118300000000001 2024-09-15T21:47:48.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.106 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.106 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.107 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476863+0000", 2024-09-15T21:47:48.107 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2208999999999998e-05 2024-09-15T21:47:48.107 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.107 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.107 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:48.107 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477040+0000", 2024-09-15T21:47:48.107 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000176424 2024-09-15T21:47:48.107 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.107 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.107 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.107 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480154+0000", 2024-09-15T21:47:48.107 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031147589999999999 2024-09-15T21:47:48.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.108 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.108 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.108 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480596+0000", 2024-09-15T21:47:48.108 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044172699999999999 2024-09-15T21:47:48.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.108 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.108 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.108 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480652+0000", 2024-09-15T21:47:48.108 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6350000000000001e-05 2024-09-15T21:47:48.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.108 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.108 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.109 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480692+0000", 2024-09-15T21:47:48.109 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9412000000000001e-05 2024-09-15T21:47:48.109 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.109 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.109 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.109 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.109 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.109 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:6 2.9 2:941d59bc:::61f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.109 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:48.109 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 62.476063781000001, 2024-09-15T21:47:48.109 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057427529999999997, 2024-09-15T21:47:48.109 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.109 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.109 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.110 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.110 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.110 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.110 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:48.110 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.110 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.110 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.110 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.111 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:48.111 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.111 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.111 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.111 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.111 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:48.111 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.41e-07 2024-09-15T21:47:48.111 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.111 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.111 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.111 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544792+0000", 2024-09-15T21:47:48.112 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9929999999999998e-06 2024-09-15T21:47:48.112 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.112 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.112 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.112 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544885+0000", 2024-09-15T21:47:48.112 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2825000000000007e-05 2024-09-15T21:47:48.112 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.112 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.112 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.112 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544891+0000", 2024-09-15T21:47:48.112 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.93e-06 2024-09-15T21:47:48.112 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.113 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.113 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.113 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546766+0000", 2024-09-15T21:47:48.113 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018747569999999999 2024-09-15T21:47:48.113 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.113 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.113 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.113 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546864+0000", 2024-09-15T21:47:48.113 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8603000000000004e-05 2024-09-15T21:47:48.113 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.113 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.113 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:48.113 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547028+0000", 2024-09-15T21:47:48.113 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000163461 2024-09-15T21:47:48.114 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.114 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.114 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.114 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549499+0000", 2024-09-15T21:47:48.114 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024711749999999999 2024-09-15T21:47:48.114 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.114 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.114 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.114 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550417+0000", 2024-09-15T21:47:48.115 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00091817199999999998 2024-09-15T21:47:48.115 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.115 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.115 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.115 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550476+0000", 2024-09-15T21:47:48.115 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8635999999999999e-05 2024-09-15T21:47:48.115 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.115 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.115 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.115 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550533+0000", 2024-09-15T21:47:48.116 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6959999999999997e-05 2024-09-15T21:47:48.116 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.116 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.116 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.116 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.116 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.116 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.116 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:48.116 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 62.476049316000001, 2024-09-15T21:47:48.116 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004917121, 2024-09-15T21:47:48.116 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.116 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.116 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.117 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.117 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.117 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.117 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:48.117 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.117 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.117 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.117 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.117 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:48.117 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.117 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.117 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.117 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.117 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544805+0000", 2024-09-15T21:47:48.118 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4600000000000001e-07 2024-09-15T21:47:48.118 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.118 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.118 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.118 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544817+0000", 2024-09-15T21:47:48.118 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2023e-05 2024-09-15T21:47:48.118 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.118 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.118 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.118 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545000+0000", 2024-09-15T21:47:48.118 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018384400000000001 2024-09-15T21:47:48.118 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.118 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.119 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.119 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545007+0000", 2024-09-15T21:47:48.119 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1249999999999998e-06 2024-09-15T21:47:48.119 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.119 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.119 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.119 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545670+0000", 2024-09-15T21:47:48.119 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00066324999999999999 2024-09-15T21:47:48.119 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.119 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.120 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.120 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545749+0000", 2024-09-15T21:47:48.120 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9239000000000005e-05 2024-09-15T21:47:48.120 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.120 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.120 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:48.121 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545885+0000", 2024-09-15T21:47:48.121 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013586199999999999 2024-09-15T21:47:48.121 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.121 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.121 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.121 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547944+0000", 2024-09-15T21:47:48.121 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020594720000000001 2024-09-15T21:47:48.121 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.121 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.121 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.121 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549630+0000", 2024-09-15T21:47:48.122 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001685831 2024-09-15T21:47:48.122 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.122 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.122 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.122 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549681+0000", 2024-09-15T21:47:48.122 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1187000000000003e-05 2024-09-15T21:47:48.122 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.122 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.122 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.122 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549721+0000", 2024-09-15T21:47:48.123 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0042e-05 2024-09-15T21:47:48.123 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.123 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.123 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.123 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.123 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.123 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.123 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:48.123 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 62.476035568, 2024-09-15T21:47:48.123 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050615549999999997, 2024-09-15T21:47:48.123 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.123 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.124 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.124 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.124 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.124 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.124 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:48.124 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.124 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.125 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.125 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.125 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:48.125 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.125 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.125 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.125 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.125 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:48.126 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:47:48.126 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.126 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.126 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.126 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544821+0000", 2024-09-15T21:47:48.126 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4399999999999999e-06 2024-09-15T21:47:48.126 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.126 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.126 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.126 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545025+0000", 2024-09-15T21:47:48.127 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020379400000000001 2024-09-15T21:47:48.127 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.127 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.127 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.127 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545030+0000", 2024-09-15T21:47:48.127 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3109999999999996e-06 2024-09-15T21:47:48.128 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.128 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.128 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.128 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546196+0000", 2024-09-15T21:47:48.128 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001166261 2024-09-15T21:47:48.128 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.128 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.128 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.128 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546307+0000", 2024-09-15T21:47:48.128 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000110641 2024-09-15T21:47:48.128 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.129 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.129 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:48.129 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546422+0000", 2024-09-15T21:47:48.129 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000115399 2024-09-15T21:47:48.129 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.129 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.129 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.129 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547973+0000", 2024-09-15T21:47:48.129 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001551228 2024-09-15T21:47:48.129 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.129 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.129 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.129 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549815+0000", 2024-09-15T21:47:48.129 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001841685 2024-09-15T21:47:48.130 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.130 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.130 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.130 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549856+0000", 2024-09-15T21:47:48.130 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.091e-05 2024-09-15T21:47:48.130 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.130 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.130 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.130 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549880+0000", 2024-09-15T21:47:48.130 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3580000000000001e-05 2024-09-15T21:47:48.130 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.130 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.131 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.131 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.131 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.131 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.131 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:48.131 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 62.476031560000003, 2024-09-15T21:47:48.131 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056931389999999998, 2024-09-15T21:47:48.131 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.132 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.132 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.132 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.132 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.132 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.132 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:48.133 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.133 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.133 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.133 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.133 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:48.133 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.134 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.134 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.134 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.134 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:48.134 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:47:48.134 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.134 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.135 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.135 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544828+0000", 2024-09-15T21:47:48.135 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4369999999999998e-06 2024-09-15T21:47:48.135 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.135 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.135 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.136 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545041+0000", 2024-09-15T21:47:48.136 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000212695 2024-09-15T21:47:48.136 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.136 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.136 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.136 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545058+0000", 2024-09-15T21:47:48.137 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7717999999999999e-05 2024-09-15T21:47:48.137 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.137 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.137 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.137 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546958+0000", 2024-09-15T21:47:48.137 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018995539999999999 2024-09-15T21:47:48.137 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.137 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.137 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.137 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547057+0000", 2024-09-15T21:47:48.137 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9370999999999994e-05 2024-09-15T21:47:48.137 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.138 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.138 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:48.138 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547579+0000", 2024-09-15T21:47:48.138 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052198200000000004 2024-09-15T21:47:48.138 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.138 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.138 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.138 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549929+0000", 2024-09-15T21:47:48.138 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023494649999999998 2024-09-15T21:47:48.138 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.138 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.138 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.138 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550401+0000", 2024-09-15T21:47:48.139 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00047277300000000001 2024-09-15T21:47:48.139 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.139 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.139 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.139 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550458+0000", 2024-09-15T21:47:48.139 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6756999999999999e-05 2024-09-15T21:47:48.139 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.139 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.139 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.139 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550515+0000", 2024-09-15T21:47:48.139 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7093999999999999e-05 2024-09-15T21:47:48.139 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.139 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.139 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.140 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.140 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.140 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:26 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.140 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:48.140 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 62.476024903000003, 2024-09-15T21:47:48.140 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050112029999999997, 2024-09-15T21:47:48.140 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.140 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.140 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.141 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.141 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.141 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.141 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:48.141 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.141 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.141 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.141 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.141 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:48.141 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.141 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.141 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.141 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.141 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:48.142 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.36e-07 2024-09-15T21:47:48.142 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.142 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.142 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.142 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545086+0000", 2024-09-15T21:47:48.142 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025704699999999999 2024-09-15T21:47:48.142 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.142 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.142 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.142 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545119+0000", 2024-09-15T21:47:48.142 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2734999999999997e-05 2024-09-15T21:47:48.142 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.142 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.143 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.143 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545126+0000", 2024-09-15T21:47:48.143 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5599999999999996e-06 2024-09-15T21:47:48.143 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.143 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.143 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.143 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545253+0000", 2024-09-15T21:47:48.143 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012644500000000001 2024-09-15T21:47:48.143 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.143 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.143 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.143 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545366+0000", 2024-09-15T21:47:48.143 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011349999999999999 2024-09-15T21:47:48.143 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.144 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.144 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:48.144 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545673+0000", 2024-09-15T21:47:48.144 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030643599999999999 2024-09-15T21:47:48.144 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.144 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.144 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.144 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547924+0000", 2024-09-15T21:47:48.144 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0022516189999999998 2024-09-15T21:47:48.144 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.144 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.144 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.144 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549746+0000", 2024-09-15T21:47:48.144 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018217629999999999 2024-09-15T21:47:48.145 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.145 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.145 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.145 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549800+0000", 2024-09-15T21:47:48.145 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3412999999999998e-05 2024-09-15T21:47:48.145 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.145 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.145 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.145 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549840+0000", 2024-09-15T21:47:48.145 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0448999999999998e-05 2024-09-15T21:47:48.145 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.145 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.145 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.145 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.146 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.146 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:30 2.4 2:218afd09:::203.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.146 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:48.146 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 62.475764073999997, 2024-09-15T21:47:48.146 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055481230000000003, 2024-09-15T21:47:48.146 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.146 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.146 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:48.146 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.146 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.146 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.146 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:48.146 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.146 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.147 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.147 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.147 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:48.147 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.147 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.147 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.147 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.147 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545091+0000", 2024-09-15T21:47:48.147 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7000000000000003e-07 2024-09-15T21:47:48.147 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.148 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.148 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.148 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545094+0000", 2024-09-15T21:47:48.148 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9569999999999998e-06 2024-09-15T21:47:48.148 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.148 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.148 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.149 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545156+0000", 2024-09-15T21:47:48.149 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2277e-05 2024-09-15T21:47:48.149 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.149 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.149 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.149 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545162+0000", 2024-09-15T21:47:48.149 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9120000000000003e-06 2024-09-15T21:47:48.150 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.150 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.150 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.150 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545262+0000", 2024-09-15T21:47:48.150 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010001900000000001 2024-09-15T21:47:48.150 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.151 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.151 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.151 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545388+0000", 2024-09-15T21:47:48.151 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126243 2024-09-15T21:47:48.151 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.151 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.151 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.152 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550638+0000", 2024-09-15T21:47:48.152 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052497450000000001 2024-09-15T21:47:48.152 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.152 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.152 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.152 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.153 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.153 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:10 2.9 2:954a9a2b:::62b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.153 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:48.153 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 56.456080614000001, 2024-09-15T21:47:48.153 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048438860000000004, 2024-09-15T21:47:48.154 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.154 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.154 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.154 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.154 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.154 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.155 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:48.155 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.155 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.155 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.155 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.155 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:48.156 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.156 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.156 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.156 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.156 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:48.156 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0400000000000002e-07 2024-09-15T21:47:48.157 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.157 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.157 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.157 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564786+0000", 2024-09-15T21:47:48.157 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2412000000000001e-05 2024-09-15T21:47:48.157 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.158 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.158 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.158 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564991+0000", 2024-09-15T21:47:48.158 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020483900000000001 2024-09-15T21:47:48.158 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.158 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.159 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.159 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564998+0000", 2024-09-15T21:47:48.159 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2459999999999997e-06 2024-09-15T21:47:48.159 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.159 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.159 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.160 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565422+0000", 2024-09-15T21:47:48.160 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00042420100000000003 2024-09-15T21:47:48.160 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.160 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.160 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.160 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565515+0000", 2024-09-15T21:47:48.161 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2449999999999997e-05 2024-09-15T21:47:48.161 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.161 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.161 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:48.161 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565642+0000", 2024-09-15T21:47:48.162 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012743699999999999 2024-09-15T21:47:48.162 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.162 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.162 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.162 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566656+0000", 2024-09-15T21:47:48.162 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010138580000000001 2024-09-15T21:47:48.163 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.163 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.163 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.163 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569552+0000", 2024-09-15T21:47:48.163 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028966819999999998 2024-09-15T21:47:48.163 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.164 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.164 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.164 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569594+0000", 2024-09-15T21:47:48.164 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1235000000000001e-05 2024-09-15T21:47:48.164 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.164 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.165 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.165 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569617+0000", 2024-09-15T21:47:48.165 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3221999999999999e-05 2024-09-15T21:47:48.165 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.165 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.165 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.166 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.166 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.166 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:48.169 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:47:48.169 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:48.166+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:48.174 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:48.172+0000 7fdc68aa6640 1 -- 172.21.15.100:0/2695998836 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7fdc640b44c0 con 0x7fdc640cb7d0 2024-09-15T21:47:48.269 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:48.267+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:48.281 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:48.282 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:48.282 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:48.282 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:48.291 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:47:48.370 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:48.368+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:48.407 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:48.407 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:48.407 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:48.407 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:48.407 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:48.417 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:47:48.502 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:48.500+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:48.565 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:48.565 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:48.565 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:48.565 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:48.565 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.565 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:48.566 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:48.566 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 80.149227680999999, 2024-09-15T21:47:48.566 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00598016, 2024-09-15T21:47:48.566 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.566 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.566 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.566 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.566 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.566 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.566 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:48.566 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.566 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.566 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.567 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.567 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:48.567 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.567 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.567 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.567 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.567 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413339+0000", 2024-09-15T21:47:48.567 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6279999999999998e-06 2024-09-15T21:47:48.567 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.567 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.567 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.567 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413354+0000", 2024-09-15T21:47:48.567 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4877e-05 2024-09-15T21:47:48.567 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.568 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.568 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.568 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413418+0000", 2024-09-15T21:47:48.568 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4274000000000002e-05 2024-09-15T21:47:48.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.568 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.568 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.568 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413428+0000", 2024-09-15T21:47:48.568 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5899999999999997e-06 2024-09-15T21:47:48.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.568 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.568 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.568 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413563+0000", 2024-09-15T21:47:48.569 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013521900000000001 2024-09-15T21:47:48.569 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.569 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.569 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.569 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413678+0000", 2024-09-15T21:47:48.569 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000114904 2024-09-15T21:47:48.569 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.569 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.569 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:48.569 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413789+0000", 2024-09-15T21:47:48.569 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011069300000000001 2024-09-15T21:47:48.569 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.569 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.569 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.570 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417603+0000", 2024-09-15T21:47:48.570 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038138460000000001 2024-09-15T21:47:48.570 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.570 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.570 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.570 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419220+0000", 2024-09-15T21:47:48.570 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001617678 2024-09-15T21:47:48.570 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.570 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.570 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.571 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419275+0000", 2024-09-15T21:47:48.571 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4391999999999997e-05 2024-09-15T21:47:48.571 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.571 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.571 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.571 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419316+0000", 2024-09-15T21:47:48.571 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1059000000000001e-05 2024-09-15T21:47:48.571 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.571 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.571 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.572 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.572 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.572 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:48.572 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:48.572 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 80.149205553000002, 2024-09-15T21:47:48.572 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0087926470000000007, 2024-09-15T21:47:48.572 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.572 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.572 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.572 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.572 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.572 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.572 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:48.573 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.573 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.573 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.573 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:48.573 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.574 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.574 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.574 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:48.574 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4099999999999999e-07 2024-09-15T21:47:48.574 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.574 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.574 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.574 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413361+0000", 2024-09-15T21:47:48.574 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6189999999999998e-06 2024-09-15T21:47:48.574 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.575 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.575 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.575 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413461+0000", 2024-09-15T21:47:48.575 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000100053 2024-09-15T21:47:48.575 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.575 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.575 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.575 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413466+0000", 2024-09-15T21:47:48.576 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3850000000000001e-06 2024-09-15T21:47:48.576 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.576 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.576 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.576 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414073+0000", 2024-09-15T21:47:48.576 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000606724 2024-09-15T21:47:48.577 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.577 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.577 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.577 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414143+0000", 2024-09-15T21:47:48.577 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9727000000000005e-05 2024-09-15T21:47:48.577 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.577 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.577 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:48.578 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414259+0000", 2024-09-15T21:47:48.578 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000116178 2024-09-15T21:47:48.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.579 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.579 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.579 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417610+0000", 2024-09-15T21:47:48.579 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003351176 2024-09-15T21:47:48.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.579 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.579 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.579 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422058+0000", 2024-09-15T21:47:48.579 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004447599 2024-09-15T21:47:48.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.580 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.580 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.580 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422111+0000", 2024-09-15T21:47:48.580 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3606999999999997e-05 2024-09-15T21:47:48.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.580 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.580 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.581 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422150+0000", 2024-09-15T21:47:48.581 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9137999999999997e-05 2024-09-15T21:47:48.581 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.581 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.581 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.581 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.581 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.582 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:48.582 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:48.582 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 80.149201122999997, 2024-09-15T21:47:48.582 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0089009769999999992, 2024-09-15T21:47:48.582 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.582 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.582 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.582 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.583 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.583 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.583 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:48.583 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.583 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.583 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.583 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.583 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:48.583 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.584 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.584 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.584 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413363+0000", 2024-09-15T21:47:48.584 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3999999999999997e-07 2024-09-15T21:47:48.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.584 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.584 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.584 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413365+0000", 2024-09-15T21:47:48.585 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.216e-06 2024-09-15T21:47:48.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.585 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.585 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.585 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413481+0000", 2024-09-15T21:47:48.585 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000116311 2024-09-15T21:47:48.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.585 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.586 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.586 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413488+0000", 2024-09-15T21:47:48.586 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0110000000000003e-06 2024-09-15T21:47:48.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.586 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.586 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.586 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414452+0000", 2024-09-15T21:47:48.587 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00096385399999999999 2024-09-15T21:47:48.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.587 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.587 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.587 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414545+0000", 2024-09-15T21:47:48.587 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.3024999999999998e-05 2024-09-15T21:47:48.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.587 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.587 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:48.587 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414644+0000", 2024-09-15T21:47:48.587 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9098000000000006e-05 2024-09-15T21:47:48.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.587 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.587 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.588 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417667+0000", 2024-09-15T21:47:48.588 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030229409999999999 2024-09-15T21:47:48.588 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.588 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.588 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.588 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422200+0000", 2024-09-15T21:47:48.588 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045327880000000003 2024-09-15T21:47:48.588 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.588 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.588 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.588 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422228+0000", 2024-09-15T21:47:48.589 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8109000000000001e-05 2024-09-15T21:47:48.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.589 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.589 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.589 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422263+0000", 2024-09-15T21:47:48.590 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5284000000000003e-05 2024-09-15T21:47:48.590 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.590 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.590 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.590 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.591 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.591 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:48.591 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:48.591 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 80.149197537000006, 2024-09-15T21:47:48.591 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0089846820000000008, 2024-09-15T21:47:48.592 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.592 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.592 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.592 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.592 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.593 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.593 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:48.593 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.593 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.593 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.594 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:48.594 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.594 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.594 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.594 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.594 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:48.595 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7599999999999998e-07 2024-09-15T21:47:48.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.595 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.595 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.595 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413368+0000", 2024-09-15T21:47:48.595 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.058e-06 2024-09-15T21:47:48.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.596 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.596 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.596 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413832+0000", 2024-09-15T21:47:48.596 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046398700000000001 2024-09-15T21:47:48.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.597 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.597 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.597 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413841+0000", 2024-09-15T21:47:48.597 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1710000000000008e-06 2024-09-15T21:47:48.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.597 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.597 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.598 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414869+0000", 2024-09-15T21:47:48.598 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001027584 2024-09-15T21:47:48.598 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.598 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.598 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.598 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414946+0000", 2024-09-15T21:47:48.599 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7177999999999999e-05 2024-09-15T21:47:48.599 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.599 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.599 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:48.599 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415054+0000", 2024-09-15T21:47:48.599 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010807 2024-09-15T21:47:48.600 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.600 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.600 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.600 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417672+0000", 2024-09-15T21:47:48.600 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002618015 2024-09-15T21:47:48.600 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.600 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.601 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.601 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422295+0000", 2024-09-15T21:47:48.601 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046233300000000001 2024-09-15T21:47:48.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.601 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.601 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.601 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422330+0000", 2024-09-15T21:47:48.601 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4476e-05 2024-09-15T21:47:48.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.601 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.601 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.601 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422351+0000", 2024-09-15T21:47:48.601 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0537e-05 2024-09-15T21:47:48.601 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.602 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.602 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.602 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.602 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.602 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:48.602 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:48.602 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 80.149194176999998, 2024-09-15T21:47:48.602 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0090873700000000009, 2024-09-15T21:47:48.602 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.602 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.603 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.603 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.603 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.603 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.603 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:48.603 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.603 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.603 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.603 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.603 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:48.604 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.604 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.604 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.604 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:48.604 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6100000000000002e-07 2024-09-15T21:47:48.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.604 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.604 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.604 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413371+0000", 2024-09-15T21:47:48.604 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.764e-06 2024-09-15T21:47:48.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.604 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.604 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.605 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413859+0000", 2024-09-15T21:47:48.605 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00048751900000000001 2024-09-15T21:47:48.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.605 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.605 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.605 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413864+0000", 2024-09-15T21:47:48.605 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6200000000000004e-06 2024-09-15T21:47:48.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.605 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.605 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.606 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415253+0000", 2024-09-15T21:47:48.606 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013884050000000001 2024-09-15T21:47:48.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.606 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.606 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.606 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415364+0000", 2024-09-15T21:47:48.606 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000111119 2024-09-15T21:47:48.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.606 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.606 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:48.606 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415453+0000", 2024-09-15T21:47:48.606 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8641999999999996e-05 2024-09-15T21:47:48.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.607 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.607 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.607 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417676+0000", 2024-09-15T21:47:48.607 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00222368 2024-09-15T21:47:48.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.607 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.607 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.607 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422395+0000", 2024-09-15T21:47:48.608 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047191150000000003 2024-09-15T21:47:48.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.608 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.608 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.608 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422434+0000", 2024-09-15T21:47:48.608 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8525999999999997e-05 2024-09-15T21:47:48.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.608 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.608 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.608 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422457+0000", 2024-09-15T21:47:48.609 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2719000000000001e-05 2024-09-15T21:47:48.609 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.609 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.609 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.609 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.609 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:48.609 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:48.609 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 80.149191154999997, 2024-09-15T21:47:48.609 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0092139749999999992, 2024-09-15T21:47:48.609 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.609 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.609 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.609 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.610 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.610 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.610 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:48.610 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.610 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.610 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.610 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.610 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:48.610 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.610 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.610 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.610 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.610 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:48.610 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3799999999999999e-07 2024-09-15T21:47:48.611 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.611 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.611 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.611 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413375+0000", 2024-09-15T21:47:48.611 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1959999999999998e-06 2024-09-15T21:47:48.611 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.611 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.611 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.611 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413876+0000", 2024-09-15T21:47:48.611 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00050134500000000005 2024-09-15T21:47:48.611 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.611 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.611 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.611 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413881+0000", 2024-09-15T21:47:48.612 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3580000000000002e-06 2024-09-15T21:47:48.612 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.612 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.612 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.612 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415669+0000", 2024-09-15T21:47:48.612 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0017875090000000001 2024-09-15T21:47:48.612 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.612 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.612 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.612 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415737+0000", 2024-09-15T21:47:48.612 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8386000000000006e-05 2024-09-15T21:47:48.612 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.612 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.612 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:48.613 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415857+0000", 2024-09-15T21:47:48.613 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119906 2024-09-15T21:47:48.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.613 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.613 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.613 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421848+0000", 2024-09-15T21:47:48.613 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059903880000000001 2024-09-15T21:47:48.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.613 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.613 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.613 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422513+0000", 2024-09-15T21:47:48.613 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00066575599999999999 2024-09-15T21:47:48.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.613 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.613 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.614 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422549+0000", 2024-09-15T21:47:48.614 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5719999999999997e-05 2024-09-15T21:47:48.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.614 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.614 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.614 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422586+0000", 2024-09-15T21:47:48.614 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7172999999999998e-05 2024-09-15T21:47:48.614 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.614 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.614 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.614 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.614 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:48.614 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:48.615 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 80.149185333000005, 2024-09-15T21:47:48.615 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.009324087, 2024-09-15T21:47:48.615 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.615 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.615 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.615 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.615 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.615 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.615 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:48.615 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.615 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.615 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.615 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.615 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:48.616 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.616 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.616 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.616 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.616 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:48.616 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8900000000000001e-07 2024-09-15T21:47:48.616 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.616 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.616 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.616 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413381+0000", 2024-09-15T21:47:48.616 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1959999999999998e-06 2024-09-15T21:47:48.616 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.616 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.616 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.616 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413895+0000", 2024-09-15T21:47:48.617 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051466000000000001 2024-09-15T21:47:48.617 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.617 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.617 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.617 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413900+0000", 2024-09-15T21:47:48.617 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6090000000000003e-06 2024-09-15T21:47:48.617 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.617 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.617 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.617 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416081+0000", 2024-09-15T21:47:48.617 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021816090000000002 2024-09-15T21:47:48.617 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.617 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.617 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.618 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416173+0000", 2024-09-15T21:47:48.618 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1334000000000003e-05 2024-09-15T21:47:48.618 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.618 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.618 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:48.618 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416264+0000", 2024-09-15T21:47:48.618 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0864000000000003e-05 2024-09-15T21:47:48.618 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.618 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.618 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.618 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421854+0000", 2024-09-15T21:47:48.618 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055902240000000004 2024-09-15T21:47:48.618 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.618 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.619 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.619 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422629+0000", 2024-09-15T21:47:48.619 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00077550700000000004 2024-09-15T21:47:48.619 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.619 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.619 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.619 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422678+0000", 2024-09-15T21:47:48.619 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8810999999999997e-05 2024-09-15T21:47:48.619 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.619 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.619 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.619 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422702+0000", 2024-09-15T21:47:48.619 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3983999999999999e-05 2024-09-15T21:47:48.619 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.619 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.620 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.620 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.620 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.620 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:48.620 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:48.620 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 80.149181850999994, 2024-09-15T21:47:48.620 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0094270580000000003, 2024-09-15T21:47:48.620 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.620 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.620 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.620 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.620 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.620 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.620 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:48.621 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.621 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.621 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.621 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:48.621 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.621 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.621 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.621 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:48.621 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7500000000000001e-07 2024-09-15T21:47:48.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.621 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.621 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.621 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413386+0000", 2024-09-15T21:47:48.622 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5619999999999997e-06 2024-09-15T21:47:48.622 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.622 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.622 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.622 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413918+0000", 2024-09-15T21:47:48.622 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00053162499999999998 2024-09-15T21:47:48.622 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.622 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.622 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.622 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413923+0000", 2024-09-15T21:47:48.622 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9280000000000001e-06 2024-09-15T21:47:48.622 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.622 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.623 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.623 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416492+0000", 2024-09-15T21:47:48.623 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0025687140000000002 2024-09-15T21:47:48.623 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.623 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.623 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.623 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416558+0000", 2024-09-15T21:47:48.623 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6699000000000006e-05 2024-09-15T21:47:48.623 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.623 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.623 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:48.623 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416691+0000", 2024-09-15T21:47:48.623 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013263199999999999 2024-09-15T21:47:48.623 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.623 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.624 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.624 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421879+0000", 2024-09-15T21:47:48.624 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051881089999999998 2024-09-15T21:47:48.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.624 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.624 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.624 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422742+0000", 2024-09-15T21:47:48.624 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000862845 2024-09-15T21:47:48.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.624 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.624 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.624 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422786+0000", 2024-09-15T21:47:48.624 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4332000000000002e-05 2024-09-15T21:47:48.625 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.625 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.625 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.625 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422809+0000", 2024-09-15T21:47:48.625 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2337e-05 2024-09-15T21:47:48.625 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.625 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.625 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.625 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.625 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.625 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:4 2.10 2:0f65c9b8:::60a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.625 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:48.625 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 74.106895682000001, 2024-09-15T21:47:48.625 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058710300000000002, 2024-09-15T21:47:48.626 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.626 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.626 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.626 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.626 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.626 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.626 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:48.626 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.626 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.626 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.626 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.626 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:48.626 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.626 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.627 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.627 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.627 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:48.627 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0800000000000005e-07 2024-09-15T21:47:48.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.627 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.627 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.627 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455672+0000", 2024-09-15T21:47:48.627 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3900000000000002e-06 2024-09-15T21:47:48.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.627 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.627 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.627 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456538+0000", 2024-09-15T21:47:48.627 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00086613100000000002 2024-09-15T21:47:48.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.628 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.628 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.628 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456545+0000", 2024-09-15T21:47:48.628 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4830000000000002e-06 2024-09-15T21:47:48.628 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.628 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.628 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.628 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456594+0000", 2024-09-15T21:47:48.628 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8429e-05 2024-09-15T21:47:48.628 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.628 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.628 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.628 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456675+0000", 2024-09-15T21:47:48.629 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1516999999999999e-05 2024-09-15T21:47:48.629 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.629 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.629 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:48.629 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456772+0000", 2024-09-15T21:47:48.629 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6523999999999999e-05 2024-09-15T21:47:48.629 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.629 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.629 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.629 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.460299+0000", 2024-09-15T21:47:48.629 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035274859999999998 2024-09-15T21:47:48.629 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.629 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.629 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.630 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461451+0000", 2024-09-15T21:47:48.630 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011518100000000001 2024-09-15T21:47:48.630 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.630 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.630 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.630 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461498+0000", 2024-09-15T21:47:48.630 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7406e-05 2024-09-15T21:47:48.630 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.630 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.630 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.630 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461539+0000", 2024-09-15T21:47:48.630 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0346000000000003e-05 2024-09-15T21:47:48.630 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.630 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.631 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.631 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.631 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.631 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:31 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.631 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:48.631 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 74.106890419999999, 2024-09-15T21:47:48.631 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0078158979999999999, 2024-09-15T21:47:48.631 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.631 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.631 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:48.631 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.631 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.631 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.631 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:48.631 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.632 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.632 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.632 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.632 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:48.632 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.632 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.632 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.632 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.632 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:48.632 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0499999999999999e-07 2024-09-15T21:47:48.632 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.632 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.632 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.632 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455675+0000", 2024-09-15T21:47:48.633 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.376e-06 2024-09-15T21:47:48.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.633 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.633 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.633 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456964+0000", 2024-09-15T21:47:48.633 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001289204 2024-09-15T21:47:48.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.633 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.633 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.633 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456970+0000", 2024-09-15T21:47:48.633 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2129999999999996e-06 2024-09-15T21:47:48.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.633 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.633 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.633 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457032+0000", 2024-09-15T21:47:48.634 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2209e-05 2024-09-15T21:47:48.634 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.634 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.634 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.634 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457134+0000", 2024-09-15T21:47:48.634 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010204000000000001 2024-09-15T21:47:48.634 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.634 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.634 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.634 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463489+0000", 2024-09-15T21:47:48.634 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0063545509999999999 2024-09-15T21:47:48.634 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.634 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.634 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.635 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.635 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.635 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.635 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:48.635 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 68.087130655999999, 2024-09-15T21:47:48.635 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057594079999999997, 2024-09-15T21:47:48.635 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.635 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.635 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.635 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.635 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.635 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.635 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:48.635 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.636 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.636 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.636 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.636 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:48.636 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.636 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.636 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.636 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.636 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:48.636 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1999999999999999e-07 2024-09-15T21:47:48.636 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.636 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.636 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.636 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475437+0000", 2024-09-15T21:47:48.637 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5429999999999998e-06 2024-09-15T21:47:48.637 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.637 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.637 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.637 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475686+0000", 2024-09-15T21:47:48.637 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000248908 2024-09-15T21:47:48.637 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.637 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.637 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.637 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475695+0000", 2024-09-15T21:47:48.637 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.9740000000000008e-06 2024-09-15T21:47:48.637 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.637 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.637 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.638 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476291+0000", 2024-09-15T21:47:48.638 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00059651500000000004 2024-09-15T21:47:48.638 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.638 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.638 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.638 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476366+0000", 2024-09-15T21:47:48.638 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4531999999999997e-05 2024-09-15T21:47:48.638 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.638 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.638 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:48.638 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476491+0000", 2024-09-15T21:47:48.638 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012527299999999999 2024-09-15T21:47:48.638 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.639 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.639 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.639 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480600+0000", 2024-09-15T21:47:48.639 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0041085870000000003 2024-09-15T21:47:48.639 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.639 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.639 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.639 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481089+0000", 2024-09-15T21:47:48.639 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000489037 2024-09-15T21:47:48.639 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.639 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.639 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.639 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481152+0000", 2024-09-15T21:47:48.639 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3136e-05 2024-09-15T21:47:48.639 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.640 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.640 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.640 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481192+0000", 2024-09-15T21:47:48.640 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0182999999999998e-05 2024-09-15T21:47:48.640 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.640 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.640 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.640 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.640 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.640 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:17 2.10 2:0e9da45e:::61b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.640 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:48.640 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 68.087124994999996, 2024-09-15T21:47:48.640 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058311830000000002, 2024-09-15T21:47:48.640 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.641 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.641 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.641 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.641 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.641 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.641 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:48.641 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.641 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.641 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.641 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.641 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:48.641 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.641 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.641 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.642 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.642 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475439+0000", 2024-09-15T21:47:48.642 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1300000000000001e-07 2024-09-15T21:47:48.642 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.642 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.642 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.642 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475441+0000", 2024-09-15T21:47:48.642 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.419e-06 2024-09-15T21:47:48.642 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.643 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.643 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.643 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:48.644 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000269187 2024-09-15T21:47:48.644 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.644 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.644 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.644 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475715+0000", 2024-09-15T21:47:48.644 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3660000000000002e-06 2024-09-15T21:47:48.644 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.644 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.644 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.644 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476671+0000", 2024-09-15T21:47:48.644 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000956507 2024-09-15T21:47:48.644 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.644 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.645 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.645 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477500+0000", 2024-09-15T21:47:48.645 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00082897400000000001 2024-09-15T21:47:48.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.645 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.645 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:48.645 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477625+0000", 2024-09-15T21:47:48.645 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012513100000000001 2024-09-15T21:47:48.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.645 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.645 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.646 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480520+0000", 2024-09-15T21:47:48.646 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002894715 2024-09-15T21:47:48.646 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.646 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.646 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.646 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481184+0000", 2024-09-15T21:47:48.647 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000663801 2024-09-15T21:47:48.647 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.647 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.647 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.647 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481232+0000", 2024-09-15T21:47:48.647 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8457e-05 2024-09-15T21:47:48.647 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.648 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.648 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.648 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481270+0000", 2024-09-15T21:47:48.648 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7313e-05 2024-09-15T21:47:48.648 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.648 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.649 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.649 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.649 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.649 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.649 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:48.649 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 68.087121119000003, 2024-09-15T21:47:48.650 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0075605530000000002, 2024-09-15T21:47:48.650 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.650 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.650 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.650 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.650 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.651 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.651 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:48.651 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.651 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.651 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.651 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.651 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:48.652 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.652 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.652 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.652 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.652 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:48.652 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.53e-07 2024-09-15T21:47:48.653 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.653 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.653 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.653 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475445+0000", 2024-09-15T21:47:48.653 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7360000000000001e-06 2024-09-15T21:47:48.654 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.654 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.654 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.654 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475735+0000", 2024-09-15T21:47:48.654 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028975799999999999 2024-09-15T21:47:48.654 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.655 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.655 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475740+0000", 2024-09-15T21:47:48.655 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8250000000000004e-06 2024-09-15T21:47:48.655 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.656 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.656 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478479+0000", 2024-09-15T21:47:48.656 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002739525 2024-09-15T21:47:48.656 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.656 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.656 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.657 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478556+0000", 2024-09-15T21:47:48.657 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6213999999999999e-05 2024-09-15T21:47:48.657 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.657 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.657 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:48.657 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478672+0000", 2024-09-15T21:47:48.658 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011605 2024-09-15T21:47:48.658 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.658 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.658 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.658 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481323+0000", 2024-09-15T21:47:48.658 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0026518819999999999 2024-09-15T21:47:48.659 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.659 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.659 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.659 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482918+0000", 2024-09-15T21:47:48.659 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001594105 2024-09-15T21:47:48.659 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.659 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.660 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.660 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482965+0000", 2024-09-15T21:47:48.660 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6950000000000003e-05 2024-09-15T21:47:48.660 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.660 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.660 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.661 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483003+0000", 2024-09-15T21:47:48.661 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8254999999999999e-05 2024-09-15T21:47:48.661 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.661 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.661 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.661 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.662 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.662 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.662 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:48.662 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 68.087117062999994, 2024-09-15T21:47:48.662 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076582890000000004, 2024-09-15T21:47:48.662 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.663 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.663 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.663 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.663 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.663 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.663 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:48.664 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.664 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.664 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.664 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.664 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:48.664 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.665 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.665 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.665 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475447+0000", 2024-09-15T21:47:48.665 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4699999999999998e-07 2024-09-15T21:47:48.665 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.666 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.666 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.666 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475449+0000", 2024-09-15T21:47:48.666 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.176e-06 2024-09-15T21:47:48.666 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.666 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.667 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.667 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475752+0000", 2024-09-15T21:47:48.667 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030374200000000002 2024-09-15T21:47:48.667 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.667 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.667 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.668 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475757+0000", 2024-09-15T21:47:48.668 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2209999999999999e-06 2024-09-15T21:47:48.668 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.668 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.668 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.668 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477441+0000", 2024-09-15T21:47:48.669 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016843489999999999 2024-09-15T21:47:48.669 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.669 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.669 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.669 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477567+0000", 2024-09-15T21:47:48.669 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012552099999999999 2024-09-15T21:47:48.670 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.670 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.670 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:48.670 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477712+0000", 2024-09-15T21:47:48.670 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000145427 2024-09-15T21:47:48.671 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.671 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.671 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.671 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482935+0000", 2024-09-15T21:47:48.671 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052231200000000004 2024-09-15T21:47:48.671 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.672 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.672 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.672 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483015+0000", 2024-09-15T21:47:48.672 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9634000000000004e-05 2024-09-15T21:47:48.672 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.672 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.673 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.674 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483065+0000", 2024-09-15T21:47:48.674 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9869e-05 2024-09-15T21:47:48.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.674 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.675 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.675 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483105+0000", 2024-09-15T21:47:48.675 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9983e-05 2024-09-15T21:47:48.675 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.675 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.675 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.676 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.676 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.676 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.676 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:48.677 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 68.087110522000003, 2024-09-15T21:47:48.677 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076459379999999997, 2024-09-15T21:47:48.677 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.677 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.677 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.678 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.678 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.678 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.678 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:48.678 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.679 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.679 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.679 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:48.679 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.680 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.680 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.680 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:48.680 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.58e-07 2024-09-15T21:47:48.680 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.680 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.681 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.681 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475455+0000", 2024-09-15T21:47:48.681 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.075e-06 2024-09-15T21:47:48.681 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.681 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.682 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.682 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475769+0000", 2024-09-15T21:47:48.682 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00031336999999999999 2024-09-15T21:47:48.682 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.682 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.682 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.683 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475773+0000", 2024-09-15T21:47:48.683 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8080000000000004e-06 2024-09-15T21:47:48.683 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.683 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.683 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.683 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478950+0000", 2024-09-15T21:47:48.683 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031763870000000001 2024-09-15T21:47:48.683 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.683 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.683 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.684 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479067+0000", 2024-09-15T21:47:48.684 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000117543 2024-09-15T21:47:48.684 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.684 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.684 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:48.684 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479196+0000", 2024-09-15T21:47:48.684 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000128987 2024-09-15T21:47:48.684 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.684 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.684 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.685 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481366+0000", 2024-09-15T21:47:48.685 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021699089999999998 2024-09-15T21:47:48.685 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.685 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.685 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.685 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483015+0000", 2024-09-15T21:47:48.685 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016486719999999999 2024-09-15T21:47:48.685 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.685 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.685 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.686 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483061+0000", 2024-09-15T21:47:48.686 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5729999999999998e-05 2024-09-15T21:47:48.686 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.686 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.686 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.686 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483099+0000", 2024-09-15T21:47:48.686 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8099000000000002e-05 2024-09-15T21:47:48.686 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.686 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.686 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.687 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.687 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.687 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:48.687 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 68.087063412999996, 2024-09-15T21:47:48.687 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077290859999999996, 2024-09-15T21:47:48.687 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.687 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.687 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.688 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.688 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.688 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.688 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:48.688 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.688 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.688 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.688 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.688 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:48.689 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.689 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.689 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.689 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.689 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475501+0000", 2024-09-15T21:47:48.689 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1900000000000002e-07 2024-09-15T21:47:48.689 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.689 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.689 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.689 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475504+0000", 2024-09-15T21:47:48.690 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.168e-06 2024-09-15T21:47:48.690 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.690 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.690 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.690 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477037+0000", 2024-09-15T21:47:48.690 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001533202 2024-09-15T21:47:48.690 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.690 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.690 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.690 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477050+0000", 2024-09-15T21:47:48.691 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3132999999999999e-05 2024-09-15T21:47:48.691 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.691 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.691 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.691 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479477+0000", 2024-09-15T21:47:48.691 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002426489 2024-09-15T21:47:48.691 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.691 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.691 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.692 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479553+0000", 2024-09-15T21:47:48.692 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6583999999999997e-05 2024-09-15T21:47:48.692 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.692 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.692 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:48.692 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479637+0000", 2024-09-15T21:47:48.692 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4172999999999995e-05 2024-09-15T21:47:48.692 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.692 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.692 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.693 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483108+0000", 2024-09-15T21:47:48.693 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003470356 2024-09-15T21:47:48.693 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.693 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.693 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.693 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483160+0000", 2024-09-15T21:47:48.693 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2507999999999999e-05 2024-09-15T21:47:48.693 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.694 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.694 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.694 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483202+0000", 2024-09-15T21:47:48.694 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1758999999999998e-05 2024-09-15T21:47:48.694 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.694 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.694 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.694 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483229+0000", 2024-09-15T21:47:48.694 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6995000000000002e-05 2024-09-15T21:47:48.695 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.695 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.695 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.695 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.695 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.695 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.695 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:48.695 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 68.087058455999994, 2024-09-15T21:47:48.695 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.55145717800000005, 2024-09-15T21:47:48.695 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.695 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.695 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:48.695 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.695 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.696 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.696 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:48.696 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.696 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.696 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.696 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:48.696 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.696 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.696 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.696 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:48.696 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.65e-07 2024-09-15T21:47:48.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.697 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.697 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.697 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475507+0000", 2024-09-15T21:47:48.697 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4699999999999999e-06 2024-09-15T21:47:48.697 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.697 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.697 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.697 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.025935+0000", 2024-09-15T21:47:48.697 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.55042854200000002 2024-09-15T21:47:48.697 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.697 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.697 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.697 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.025965+0000", 2024-09-15T21:47:48.697 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9995999999999999e-05 2024-09-15T21:47:48.698 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.698 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.698 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.698 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026016+0000", 2024-09-15T21:47:48.698 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1310000000000002e-05 2024-09-15T21:47:48.698 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.698 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.698 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.698 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026122+0000", 2024-09-15T21:47:48.698 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105605 2024-09-15T21:47:48.698 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.698 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.698 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.698 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026962+0000", 2024-09-15T21:47:48.698 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00083998999999999999 2024-09-15T21:47:48.699 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.699 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.699 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.699 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.699 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.699 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:5 2.e 2:75ee5fd8:::629.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.699 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:48.699 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 56.997852090999999, 2024-09-15T21:47:48.699 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0071955470000000001, 2024-09-15T21:47:48.699 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.699 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.699 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.699 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.699 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.700 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.700 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:48.700 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.700 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.700 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.700 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.700 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:48.700 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.700 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.700 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.700 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.700 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564713+0000", 2024-09-15T21:47:48.700 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.866e-06 2024-09-15T21:47:48.700 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.701 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.701 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.701 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564721+0000", 2024-09-15T21:47:48.701 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3299999999999999e-06 2024-09-15T21:47:48.701 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.701 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.701 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.701 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564762+0000", 2024-09-15T21:47:48.701 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0707999999999997e-05 2024-09-15T21:47:48.701 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.701 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.701 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.701 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564774+0000", 2024-09-15T21:47:48.701 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1883e-05 2024-09-15T21:47:48.702 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.702 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.702 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.702 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564877+0000", 2024-09-15T21:47:48.702 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010254700000000001 2024-09-15T21:47:48.702 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.702 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.702 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.702 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565066+0000", 2024-09-15T21:47:48.702 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018933700000000001 2024-09-15T21:47:48.702 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.702 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.702 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:48.702 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565164+0000", 2024-09-15T21:47:48.703 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7633999999999994e-05 2024-09-15T21:47:48.703 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.703 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.703 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.703 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569030+0000", 2024-09-15T21:47:48.703 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038669070000000002 2024-09-15T21:47:48.703 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.703 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.703 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.703 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571818+0000", 2024-09-15T21:47:48.703 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027878930000000001 2024-09-15T21:47:48.703 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.703 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.703 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.704 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571868+0000", 2024-09-15T21:47:48.704 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9880999999999999e-05 2024-09-15T21:47:48.704 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.704 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.704 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.704 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571907+0000", 2024-09-15T21:47:48.704 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8560999999999999e-05 2024-09-15T21:47:48.704 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.704 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.704 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.704 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.704 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.704 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:6 2.e 2:75ee5fd8:::629.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.704 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:48.705 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 56.997837963999999, 2024-09-15T21:47:48.705 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073433220000000002, 2024-09-15T21:47:48.705 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.705 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.705 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.705 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.705 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.705 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.705 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:48.705 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.705 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.705 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.705 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.705 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:48.706 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.706 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.706 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.706 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.706 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564726+0000", 2024-09-15T21:47:48.706 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7399999999999999e-07 2024-09-15T21:47:48.706 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.706 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.706 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.706 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564728+0000", 2024-09-15T21:47:48.706 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.447e-06 2024-09-15T21:47:48.706 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.706 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.706 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.706 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565446+0000", 2024-09-15T21:47:48.707 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00071774999999999996 2024-09-15T21:47:48.707 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.707 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.707 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.707 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565453+0000", 2024-09-15T21:47:48.707 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3239999999999997e-06 2024-09-15T21:47:48.707 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.707 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.707 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.707 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565515+0000", 2024-09-15T21:47:48.707 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1768999999999998e-05 2024-09-15T21:47:48.707 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.707 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.707 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.708 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565591+0000", 2024-09-15T21:47:48.708 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5873999999999999e-05 2024-09-15T21:47:48.708 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.708 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.708 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:48.708 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565709+0000", 2024-09-15T21:47:48.708 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011838899999999999 2024-09-15T21:47:48.708 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.708 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.708 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.708 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569039+0000", 2024-09-15T21:47:48.708 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0033292170000000002 2024-09-15T21:47:48.708 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.708 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.709 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.709 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571977+0000", 2024-09-15T21:47:48.709 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0029387829999999999 2024-09-15T21:47:48.709 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.709 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.709 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.709 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.572032+0000", 2024-09-15T21:47:48.709 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4540999999999998e-05 2024-09-15T21:47:48.709 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.709 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.709 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.709 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.572069+0000", 2024-09-15T21:47:48.709 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6854000000000003e-05 2024-09-15T21:47:48.709 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.710 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.710 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.710 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.710 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.710 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:2 2.1 2:82971f97:::404.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:48.710 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:48.710 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 56.997833997000001, 2024-09-15T21:47:48.710 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007245357, 2024-09-15T21:47:48.710 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:48.710 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:48.710 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:48.710 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:48.710 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.710 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:48.711 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:48.711 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.711 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.711 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.711 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:48.711 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:48.711 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:48.711 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.711 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.711 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:48.712 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564730+0000", 2024-09-15T21:47:48.712 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2999999999999999e-07 2024-09-15T21:47:48.712 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.712 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.712 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:48.712 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564731+0000", 2024-09-15T21:47:48.712 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.367e-06 2024-09-15T21:47:48.712 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.712 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.712 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:48.712 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565484+0000", 2024-09-15T21:47:48.712 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075311599999999996 2024-09-15T21:47:48.712 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.712 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.713 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:48.713 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565490+0000", 2024-09-15T21:47:48.713 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8100000000000003e-06 2024-09-15T21:47:48.713 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.713 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.713 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:48.713 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565828+0000", 2024-09-15T21:47:48.713 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000337836 2024-09-15T21:47:48.713 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.713 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.713 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:48.713 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565964+0000", 2024-09-15T21:47:48.713 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013641599999999999 2024-09-15T21:47:48.713 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.714 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.714 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:48.714 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566071+0000", 2024-09-15T21:47:48.714 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000107126 2024-09-15T21:47:48.714 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.714 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.714 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:48.714 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571361+0000", 2024-09-15T21:47:48.714 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052896210000000004 2024-09-15T21:47:48.714 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.714 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.714 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:48.714 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571874+0000", 2024-09-15T21:47:48.714 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051320599999999999 2024-09-15T21:47:48.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.715 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.715 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:48.715 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571936+0000", 2024-09-15T21:47:48.715 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1650999999999996e-05 2024-09-15T21:47:48.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:48.715 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:48.715 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:48.715 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571975+0000", 2024-09-15T21:47:48.715 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8977999999999998e-05 2024-09-15T21:47:48.716 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.716 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.716 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.716 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:48.716 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:48.716 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:48.717 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:47:48.718 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:48.716+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:48.819 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:48.817+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:48.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:48.833+0000 7fdc52ffd640 1 -- 172.21.15.100:0/2695998836 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v43) ==== 101+0+0 (secure 0 0 0) 0x7fdc58059410 con 0x7fdc640cb7d0 2024-09-15T21:47:48.836 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is unset 2024-09-15T21:47:48.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:48.836+0000 7fdc68aa6640 1 -- 172.21.15.100:0/2695998836 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fdc4406c3b0 msgr2=0x7fdc4406e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:48.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:48.836+0000 7fdc68aa6640 1 --2- 172.21.15.100:0/2695998836 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fdc4406c3b0 0x7fdc4406e870 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdc54001700 tx=0x7fdc54009040 comp rx=0 tx=0).stop 2024-09-15T21:47:48.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:48.836+0000 7fdc68aa6640 1 -- 172.21.15.100:0/2695998836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc640cb7d0 msgr2=0x7fdc640b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:48.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:48.836+0000 7fdc68aa6640 1 --2- 172.21.15.100:0/2695998836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc640cb7d0 0x7fdc640b3f80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdc5800ec10 tx=0x7fdc5800c520 comp rx=0 tx=0).stop 2024-09-15T21:47:48.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:48.836+0000 7fdc68aa6640 1 -- 172.21.15.100:0/2695998836 shutdown_connections 2024-09-15T21:47:48.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:48.836+0000 7fdc68aa6640 1 --2- 172.21.15.100:0/2695998836 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fdc4406c3b0 0x7fdc4406e870 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:48.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:48.836+0000 7fdc68aa6640 1 --2- 172.21.15.100:0/2695998836 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdc64155440 0x7fdc640cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:48.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:48.836+0000 7fdc68aa6640 1 --2- 172.21.15.100:0/2695998836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc640cb7d0 0x7fdc640b3f80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:48.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:48.836+0000 7fdc68aa6640 1 --2- 172.21.15.100:0/2695998836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc6406d0e0 0x7fdc64150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:48.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:48.836+0000 7fdc68aa6640 1 -- 172.21.15.100:0/2695998836 >> 172.21.15.100:0/2695998836 conn(0x7fdc640680d0 msgr2=0x7fdc6414ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:48.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:48.836+0000 7fdc68aa6640 1 -- 172.21.15.100:0/2695998836 shutdown_connections 2024-09-15T21:47:48.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:48.836+0000 7fdc68aa6640 1 -- 172.21.15.100:0/2695998836 wait complete. 2024-09-15T21:47:48.919 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:48.917+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:48.925 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:48.925 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:48.925 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:48.925 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:48.937 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:47:49.021 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:49.020+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:49.114 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:49.114 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:49.114 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:49.114 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:49.114 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:49.121 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:49.120+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:49.123 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:47:49.222 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:49.220+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:49.284 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:49.285 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:49.285 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:49.285 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:49.285 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.285 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:49.285 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:47:49.285 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 80.868412461999995, 2024-09-15T21:47:49.285 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047486860000000002, 2024-09-15T21:47:49.285 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.285 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.286 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:49.286 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.286 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.286 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:47:49.286 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.286 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.286 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:47:49.286 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.286 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.286 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.287 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.287 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413816+0000", 2024-09-15T21:47:49.287 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4810000000000003e-06 2024-09-15T21:47:49.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.287 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.287 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.287 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413827+0000", 2024-09-15T21:47:49.287 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.187e-05 2024-09-15T21:47:49.287 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.287 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.287 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.287 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413862+0000", 2024-09-15T21:47:49.288 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4041000000000002e-05 2024-09-15T21:47:49.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.288 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.288 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413871+0000", 2024-09-15T21:47:49.288 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8919999999999995e-06 2024-09-15T21:47:49.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.288 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.288 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413997+0000", 2024-09-15T21:47:49.288 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012601299999999999 2024-09-15T21:47:49.288 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.288 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.289 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:49.289 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414100+0000", 2024-09-15T21:47:49.289 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102555 2024-09-15T21:47:49.289 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.289 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.289 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.289 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418558+0000", 2024-09-15T21:47:49.289 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044578339999999999 2024-09-15T21:47:49.289 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.289 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.290 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.290 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.290 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.290 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:49.290 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:47:49.290 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 80.864767647999997, 2024-09-15T21:47:49.290 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045423590000000002, 2024-09-15T21:47:49.290 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.290 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.291 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:49.291 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.291 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.291 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.291 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:47:49.291 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.291 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.291 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.291 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:47:49.291 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.291 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.292 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.292 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.292 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417455+0000", 2024-09-15T21:47:49.292 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9100000000000003e-07 2024-09-15T21:47:49.292 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.292 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.292 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.292 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417463+0000", 2024-09-15T21:47:49.292 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3750000000000003e-06 2024-09-15T21:47:49.292 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.292 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.292 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.293 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417475+0000", 2024-09-15T21:47:49.293 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2473000000000001e-05 2024-09-15T21:47:49.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.293 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.293 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.293 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417482+0000", 2024-09-15T21:47:49.293 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4010000000000002e-06 2024-09-15T21:47:49.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.293 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.293 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.293 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417524+0000", 2024-09-15T21:47:49.293 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1870999999999999e-05 2024-09-15T21:47:49.293 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.293 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.294 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:49.294 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417542+0000", 2024-09-15T21:47:49.294 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8224e-05 2024-09-15T21:47:49.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.294 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.294 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:49.294 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421955+0000", 2024-09-15T21:47:49.294 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044130740000000003 2024-09-15T21:47:49.294 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.294 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.294 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.294 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421996+0000", 2024-09-15T21:47:49.294 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.125e-05 2024-09-15T21:47:49.294 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.295 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.295 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.295 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.295 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.295 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 7 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:49.296 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:49.296 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 80.864506843000001, 2024-09-15T21:47:49.296 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043086519999999996, 2024-09-15T21:47:49.296 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.296 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.296 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:47:49.296 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.297 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.297 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.297 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:49.297 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.297 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.297 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.298 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.299 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:49.299 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.299 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.299 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.299 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.299 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:49.300 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.58e-07 2024-09-15T21:47:49.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.300 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.300 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.300 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417722+0000", 2024-09-15T21:47:49.300 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4729999999999997e-06 2024-09-15T21:47:49.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.300 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.300 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.300 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417737+0000", 2024-09-15T21:47:49.300 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5155e-05 2024-09-15T21:47:49.300 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.300 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.300 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.301 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417752+0000", 2024-09-15T21:47:49.301 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4863e-05 2024-09-15T21:47:49.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.301 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.301 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417819+0000", 2024-09-15T21:47:49.301 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7680000000000003e-05 2024-09-15T21:47:49.301 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.301 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.301 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.301 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422023+0000", 2024-09-15T21:47:49.301 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042040230000000003 2024-09-15T21:47:49.301 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.301 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.302 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.302 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.302 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.302 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:25 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:49.302 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:47:49.302 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 74.825017813000002, 2024-09-15T21:47:49.302 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070572830000000001, 2024-09-15T21:47:49.302 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.302 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.302 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:49.302 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.302 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.302 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.302 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:47:49.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:47:49.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.303 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457205+0000", 2024-09-15T21:47:49.303 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6899999999999999e-06 2024-09-15T21:47:49.303 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.303 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.303 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.304 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457211+0000", 2024-09-15T21:47:49.304 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.626e-06 2024-09-15T21:47:49.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.304 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.304 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.304 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457244+0000", 2024-09-15T21:47:49.304 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2691999999999998e-05 2024-09-15T21:47:49.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.304 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.304 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.304 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457251+0000", 2024-09-15T21:47:49.304 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3839999999999999e-06 2024-09-15T21:47:49.304 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.304 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.305 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.305 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457309+0000", 2024-09-15T21:47:49.305 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8023999999999999e-05 2024-09-15T21:47:49.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.305 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:49.305 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457413+0000", 2024-09-15T21:47:49.305 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010331999999999999 2024-09-15T21:47:49.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.305 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.305 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:49.305 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:49.305 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118439 2024-09-15T21:47:49.305 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.306 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:49.306 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463246+0000", 2024-09-15T21:47:49.306 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057148219999999996 2024-09-15T21:47:49.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.306 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:49.306 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463428+0000", 2024-09-15T21:47:49.306 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018218299999999999 2024-09-15T21:47:49.306 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.306 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.306 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:49.306 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464212+0000", 2024-09-15T21:47:49.306 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00078402799999999998 2024-09-15T21:47:49.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.307 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.307 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.307 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464261+0000", 2024-09-15T21:47:49.307 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9075e-05 2024-09-15T21:47:49.307 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.307 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.307 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.307 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.309 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.309 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:28 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:49.309 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:49.309 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 74.825007786, 2024-09-15T21:47:49.309 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0072141449999999999, 2024-09-15T21:47:49.309 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.309 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.309 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:49.309 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.309 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.309 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.310 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:49.310 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.310 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.310 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.310 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.310 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:49.310 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.310 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.311 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.311 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.311 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:49.311 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:47:49.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.311 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.311 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.311 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457217+0000", 2024-09-15T21:47:49.311 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7259999999999998e-06 2024-09-15T21:47:49.311 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.311 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.312 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457842+0000", 2024-09-15T21:47:49.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00062482599999999998 2024-09-15T21:47:49.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.312 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457848+0000", 2024-09-15T21:47:49.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7089999999999996e-06 2024-09-15T21:47:49.312 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.312 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.312 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.312 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457895+0000", 2024-09-15T21:47:49.312 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6317999999999999e-05 2024-09-15T21:47:49.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.313 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:49.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458013+0000", 2024-09-15T21:47:49.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011848100000000001 2024-09-15T21:47:49.313 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.313 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.313 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.313 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464428+0000", 2024-09-15T21:47:49.313 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0064146990000000003 2024-09-15T21:47:49.313 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.313 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.313 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.314 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.314 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.314 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:29 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:49.314 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:47:49.314 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 74.825003343000006, 2024-09-15T21:47:49.314 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073658980000000001, 2024-09-15T21:47:49.314 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.314 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.314 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:49.314 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.314 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.314 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.314 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:47:49.314 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.315 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.315 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:47:49.315 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.315 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.315 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.315 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457219+0000", 2024-09-15T21:47:49.315 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2000000000000001e-07 2024-09-15T21:47:49.315 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.316 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.316 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.316 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457226+0000", 2024-09-15T21:47:49.316 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.956e-06 2024-09-15T21:47:49.316 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.316 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.316 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.316 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458266+0000", 2024-09-15T21:47:49.316 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001040476 2024-09-15T21:47:49.316 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458273+0000", 2024-09-15T21:47:49.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0690000000000004e-06 2024-09-15T21:47:49.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458319+0000", 2024-09-15T21:47:49.317 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5668999999999999e-05 2024-09-15T21:47:49.317 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.317 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.317 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:49.317 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458415+0000", 2024-09-15T21:47:49.318 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6701999999999997e-05 2024-09-15T21:47:49.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.318 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.318 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464584+0000", 2024-09-15T21:47:49.318 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061687060000000004 2024-09-15T21:47:49.318 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.318 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.318 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.318 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.318 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.318 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.1 epoch 22/19 rep_tid 12 entries 22'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.10:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:49.318 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:47:49.319 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 74.823573336999999, 2024-09-15T21:47:49.319 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0039881550000000002, 2024-09-15T21:47:49.319 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.319 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.319 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:47:49.319 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.319 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.319 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:47:49.319 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.319 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.319 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.319 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.319 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:47:49.320 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.320 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.320 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.320 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458649+0000", 2024-09-15T21:47:49.320 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9599999999999999e-07 2024-09-15T21:47:49.320 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.320 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.320 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.321 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458656+0000", 2024-09-15T21:47:49.321 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2629999999999997e-06 2024-09-15T21:47:49.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.321 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.321 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.321 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458670+0000", 2024-09-15T21:47:49.321 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4166e-05 2024-09-15T21:47:49.321 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.321 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.321 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.321 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458682+0000", 2024-09-15T21:47:49.322 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1865000000000001e-05 2024-09-15T21:47:49.322 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.322 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.322 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.322 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458749+0000", 2024-09-15T21:47:49.322 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6507000000000005e-05 2024-09-15T21:47:49.322 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.322 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.322 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.322 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462636+0000", 2024-09-15T21:47:49.322 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038877579999999998 2024-09-15T21:47:49.322 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.322 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.322 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.323 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.323 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.1.10:12 2.12 e22/19 2:48cd7578:::60e.00000000:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:49.323 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:49.323 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 74.823259004999997, 2024-09-15T21:47:49.323 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042479500000000003, 2024-09-15T21:47:49.323 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.323 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.323 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:49.323 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.323 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.323 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.324 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:49.324 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.324 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.324 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.324 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:49.324 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.324 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.324 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.325 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.325 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:49.325 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2900000000000004e-07 2024-09-15T21:47:49.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.325 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.325 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458968+0000", 2024-09-15T21:47:49.325 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2039999999999996e-06 2024-09-15T21:47:49.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.325 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.325 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.325 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458977+0000", 2024-09-15T21:47:49.325 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.6179999999999995e-06 2024-09-15T21:47:49.325 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.326 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.326 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.326 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458983+0000", 2024-09-15T21:47:49.326 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9510000000000003e-06 2024-09-15T21:47:49.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.326 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.326 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.326 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459479+0000", 2024-09-15T21:47:49.326 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000496274 2024-09-15T21:47:49.326 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.327 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.327 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:49.327 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459496+0000", 2024-09-15T21:47:49.327 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6538e-05 2024-09-15T21:47:49.327 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.327 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.327 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:49.327 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463169+0000", 2024-09-15T21:47:49.327 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003672832 2024-09-15T21:47:49.327 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.327 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.327 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.327 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463211+0000", 2024-09-15T21:47:49.328 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2004e-05 2024-09-15T21:47:49.328 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.328 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.328 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.328 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.328 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.328 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.12 epoch 22/19 rep_tid 2 entries 22'3 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.10:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:49.328 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:47:49.328 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 74.822579450999996, 2024-09-15T21:47:49.328 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035859609999999999, 2024-09-15T21:47:49.328 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.328 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.328 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:47:49.329 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.329 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.329 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.329 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:47:49.329 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.329 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.329 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.329 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:47:49.329 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.329 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.329 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.329 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.330 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459643+0000", 2024-09-15T21:47:49.330 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2200000000000004e-07 2024-09-15T21:47:49.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.330 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.330 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459650+0000", 2024-09-15T21:47:49.330 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4479999999999997e-06 2024-09-15T21:47:49.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.330 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.330 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459665+0000", 2024-09-15T21:47:49.330 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4586999999999999e-05 2024-09-15T21:47:49.330 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.330 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.331 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.331 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459676+0000", 2024-09-15T21:47:49.331 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1673e-05 2024-09-15T21:47:49.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.331 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.331 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.331 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459732+0000", 2024-09-15T21:47:49.331 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5612999999999998e-05 2024-09-15T21:47:49.331 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.332 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.332 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.332 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463228+0000", 2024-09-15T21:47:49.332 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0034961179999999999 2024-09-15T21:47:49.332 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.332 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.332 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.332 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.332 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.333 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:34 2.1b 2:db457a8a:::201.00000001:head [write 0~588] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:49.333 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:47:49.333 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 74.817367814999997, 2024-09-15T21:47:49.333 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070946129999999996, 2024-09-15T21:47:49.333 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.333 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.333 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:49.333 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.333 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.333 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.333 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:47:49.333 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.333 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.334 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.334 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.334 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:47:49.334 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.334 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.334 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.334 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464855+0000", 2024-09-15T21:47:49.334 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5880000000000001e-06 2024-09-15T21:47:49.334 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.334 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.334 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464863+0000", 2024-09-15T21:47:49.335 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0879999999999995e-06 2024-09-15T21:47:49.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464889+0000", 2024-09-15T21:47:49.335 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5729e-05 2024-09-15T21:47:49.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.335 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.335 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464898+0000", 2024-09-15T21:47:49.335 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0389999999999993e-06 2024-09-15T21:47:49.335 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.335 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.336 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.336 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464962+0000", 2024-09-15T21:47:49.336 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3567999999999999e-05 2024-09-15T21:47:49.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.336 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:49.336 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.465041+0000", 2024-09-15T21:47:49.336 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8889999999999999e-05 2024-09-15T21:47:49.336 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.336 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.336 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:49.336 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.465173+0000", 2024-09-15T21:47:49.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013226899999999999 2024-09-15T21:47:49.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.337 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.337 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:49.337 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.466319+0000", 2024-09-15T21:47:49.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011463039999999999 2024-09-15T21:47:49.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.337 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.337 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:49.337 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471825+0000", 2024-09-15T21:47:49.337 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055062160000000004 2024-09-15T21:47:49.337 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.337 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.338 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:49.338 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471886+0000", 2024-09-15T21:47:49.338 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0503e-05 2024-09-15T21:47:49.338 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.338 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.338 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.338 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471948+0000", 2024-09-15T21:47:49.338 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2419e-05 2024-09-15T21:47:49.338 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.338 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.338 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.338 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.338 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.339 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:49.339 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:47:49.339 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 68.806709651000006, 2024-09-15T21:47:49.339 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047360500000000003, 2024-09-15T21:47:49.339 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.339 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.339 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:49.339 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.339 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.339 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.339 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:47:49.339 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.339 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.339 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.340 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.340 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:47:49.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.340 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.340 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475514+0000", 2024-09-15T21:47:49.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0990000000000001e-06 2024-09-15T21:47:49.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.340 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.340 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.340 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475524+0000", 2024-09-15T21:47:49.340 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5850000000000006e-06 2024-09-15T21:47:49.340 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.341 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.341 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.341 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475557+0000", 2024-09-15T21:47:49.341 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3090000000000003e-05 2024-09-15T21:47:49.341 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.341 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.341 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.341 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475568+0000", 2024-09-15T21:47:49.341 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1116e-05 2024-09-15T21:47:49.341 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.341 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.341 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.342 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475649+0000", 2024-09-15T21:47:49.342 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0815999999999993e-05 2024-09-15T21:47:49.342 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.342 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.342 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:49.342 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475805+0000", 2024-09-15T21:47:49.342 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015667200000000001 2024-09-15T21:47:49.342 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.342 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.342 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:49.342 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475944+0000", 2024-09-15T21:47:49.342 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000138651 2024-09-15T21:47:49.342 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.342 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.343 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:49.343 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476900+0000", 2024-09-15T21:47:49.343 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00095588699999999999 2024-09-15T21:47:49.343 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.343 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.343 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:49.343 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480153+0000", 2024-09-15T21:47:49.343 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0032535200000000002 2024-09-15T21:47:49.343 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.343 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.343 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:49.343 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480203+0000", 2024-09-15T21:47:49.343 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9932999999999998e-05 2024-09-15T21:47:49.343 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.344 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.344 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.344 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480248+0000", 2024-09-15T21:47:49.344 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4681000000000001e-05 2024-09-15T21:47:49.344 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.344 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.344 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.344 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.344 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.344 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:49.344 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:49.344 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 68.806693710000005, 2024-09-15T21:47:49.344 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049206709999999997, 2024-09-15T21:47:49.344 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.345 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.345 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:49.345 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.345 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.345 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.345 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:49.345 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.345 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.345 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.345 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.345 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:49.345 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.345 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.345 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.346 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.346 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:49.346 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1699999999999999e-07 2024-09-15T21:47:49.346 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.346 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.346 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.346 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475531+0000", 2024-09-15T21:47:49.346 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.05e-06 2024-09-15T21:47:49.346 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.346 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.346 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.346 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475743+0000", 2024-09-15T21:47:49.346 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021138099999999999 2024-09-15T21:47:49.347 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.347 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.347 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.347 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475751+0000", 2024-09-15T21:47:49.347 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7770000000000002e-06 2024-09-15T21:47:49.347 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.347 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.347 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.347 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476272+0000", 2024-09-15T21:47:49.347 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052158399999999996 2024-09-15T21:47:49.347 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.347 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.347 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:49.347 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476348+0000", 2024-09-15T21:47:49.348 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5963000000000004e-05 2024-09-15T21:47:49.348 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.348 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.348 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:49.348 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476474+0000", 2024-09-15T21:47:49.348 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012638800000000001 2024-09-15T21:47:49.348 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.348 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.348 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:49.348 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478194+0000", 2024-09-15T21:47:49.348 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001719768 2024-09-15T21:47:49.348 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.348 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.349 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:49.349 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480358+0000", 2024-09-15T21:47:49.349 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021637340000000001 2024-09-15T21:47:49.349 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.349 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.349 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:49.349 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480411+0000", 2024-09-15T21:47:49.349 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2720999999999999e-05 2024-09-15T21:47:49.349 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.349 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.350 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.350 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480449+0000", 2024-09-15T21:47:49.350 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7888e-05 2024-09-15T21:47:49.350 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.350 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.350 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.350 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.350 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.350 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:49.350 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:49.350 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 68.806688921000003, 2024-09-15T21:47:49.350 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005215478, 2024-09-15T21:47:49.350 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.350 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.351 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:49.351 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.351 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.351 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.351 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:49.351 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.351 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.351 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.351 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.351 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:49.351 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.351 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.351 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.352 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.352 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:49.352 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5199999999999998e-07 2024-09-15T21:47:49.352 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.352 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.352 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.352 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475535+0000", 2024-09-15T21:47:49.352 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9520000000000001e-06 2024-09-15T21:47:49.352 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.352 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.352 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.352 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475785+0000", 2024-09-15T21:47:49.352 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024966099999999998 2024-09-15T21:47:49.353 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.353 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.353 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.353 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475799+0000", 2024-09-15T21:47:49.353 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4649e-05 2024-09-15T21:47:49.353 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.353 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.353 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.353 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475856+0000", 2024-09-15T21:47:49.353 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6956000000000002e-05 2024-09-15T21:47:49.353 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.353 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.353 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:49.353 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476016+0000", 2024-09-15T21:47:49.354 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000159457 2024-09-15T21:47:49.354 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.354 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.354 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.354 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480748+0000", 2024-09-15T21:47:49.354 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047324510000000004 2024-09-15T21:47:49.354 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.354 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.354 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.354 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.354 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.354 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.2.12:1 2.1 e22/19 2:85bbe569:::202.00000000:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:49.354 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:47:49.354 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 68.803886449999993, 2024-09-15T21:47:49.355 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045423649999999996, 2024-09-15T21:47:49.355 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.355 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.355 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:49.355 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.355 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.355 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.355 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:47:49.355 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.355 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.355 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.355 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.355 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:47:49.355 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.356 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.356 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.356 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.356 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478336+0000", 2024-09-15T21:47:49.356 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4499999999999996e-07 2024-09-15T21:47:49.356 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.356 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.356 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.356 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478342+0000", 2024-09-15T21:47:49.356 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5579999999999997e-06 2024-09-15T21:47:49.356 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.356 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.356 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.357 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478356+0000", 2024-09-15T21:47:49.357 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3567000000000001e-05 2024-09-15T21:47:49.357 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.357 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.357 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.357 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478364+0000", 2024-09-15T21:47:49.357 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7889999999999995e-06 2024-09-15T21:47:49.357 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.357 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.357 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.357 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478416+0000", 2024-09-15T21:47:49.357 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2188000000000002e-05 2024-09-15T21:47:49.357 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.358 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.358 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:49.358 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478433+0000", 2024-09-15T21:47:49.358 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6674e-05 2024-09-15T21:47:49.358 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.358 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.358 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:49.358 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482832+0000", 2024-09-15T21:47:49.358 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043996110000000003 2024-09-15T21:47:49.358 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.358 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.358 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.358 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482878+0000", 2024-09-15T21:47:49.359 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5232999999999999e-05 2024-09-15T21:47:49.359 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.359 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.359 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.359 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.359 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.359 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:25 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:49.359 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:47:49.359 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 57.717026410999999, 2024-09-15T21:47:49.359 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058459619999999997, 2024-09-15T21:47:49.359 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.359 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.360 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:49.360 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.360 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.360 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.360 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:47:49.360 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.360 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.360 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.360 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.360 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:47:49.360 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.360 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.360 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.361 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.361 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565197+0000", 2024-09-15T21:47:49.361 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7889999999999999e-06 2024-09-15T21:47:49.361 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.361 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.361 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.361 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565208+0000", 2024-09-15T21:47:49.361 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1358999999999999e-05 2024-09-15T21:47:49.361 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.361 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.361 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.361 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565237+0000", 2024-09-15T21:47:49.361 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8626e-05 2024-09-15T21:47:49.362 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.362 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.362 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.362 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565249+0000", 2024-09-15T21:47:49.362 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1897e-05 2024-09-15T21:47:49.362 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.362 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.362 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.362 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565332+0000", 2024-09-15T21:47:49.362 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2943999999999997e-05 2024-09-15T21:47:49.362 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.362 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.363 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:49.363 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565452+0000", 2024-09-15T21:47:49.363 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119992 2024-09-15T21:47:49.363 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.363 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.363 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:49.363 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565610+0000", 2024-09-15T21:47:49.363 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015845099999999999 2024-09-15T21:47:49.363 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.363 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.363 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:49.363 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.568999+0000", 2024-09-15T21:47:49.363 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0033887190000000001 2024-09-15T21:47:49.364 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.364 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.364 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:49.364 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.570979+0000", 2024-09-15T21:47:49.364 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019804919999999999 2024-09-15T21:47:49.364 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.364 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.364 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:49.364 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571014+0000", 2024-09-15T21:47:49.364 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4331e-05 2024-09-15T21:47:49.364 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.364 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.365 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.365 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571041+0000", 2024-09-15T21:47:49.365 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7362e-05 2024-09-15T21:47:49.365 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.365 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.365 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.365 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.365 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.365 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:5 2.e e22/19 2:75ee5fd8:::629.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:47:49.365 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:47:49.365 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 57.716350765000001, 2024-09-15T21:47:49.365 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057947119999999996, 2024-09-15T21:47:49.365 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.366 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.366 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:49.366 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.366 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.366 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.366 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:47:49.366 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.366 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.366 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.366 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.366 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:47:49.366 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.366 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.367 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.367 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.367 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565872+0000", 2024-09-15T21:47:49.367 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9800000000000002e-07 2024-09-15T21:47:49.367 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.367 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.367 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.367 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565880+0000", 2024-09-15T21:47:49.367 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7619999999999995e-06 2024-09-15T21:47:49.367 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.367 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.367 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.367 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565890+0000", 2024-09-15T21:47:49.368 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0648999999999999e-05 2024-09-15T21:47:49.368 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.368 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.368 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.368 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565897+0000", 2024-09-15T21:47:49.369 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.153e-06 2024-09-15T21:47:49.369 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.369 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.369 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.369 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565966+0000", 2024-09-15T21:47:49.369 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9046000000000003e-05 2024-09-15T21:47:49.369 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.369 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.369 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:49.369 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565985+0000", 2024-09-15T21:47:49.369 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8695e-05 2024-09-15T21:47:49.369 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.370 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.370 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:49.370 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571625+0000", 2024-09-15T21:47:49.370 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056397139999999997 2024-09-15T21:47:49.370 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.370 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.370 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.370 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571666+0000", 2024-09-15T21:47:49.370 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0695000000000003e-05 2024-09-15T21:47:49.370 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.370 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.370 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.370 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.370 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.371 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:6 2.e e22/19 2:75ee5fd8:::629.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:47:49.371 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:47:49.371 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 57.716288517000002, 2024-09-15T21:47:49.371 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057607429999999996, 2024-09-15T21:47:49.371 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.371 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.371 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:49.371 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.371 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.371 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.371 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:47:49.371 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.372 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.372 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.372 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.372 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:47:49.372 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.372 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.372 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.372 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.372 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565934+0000", 2024-09-15T21:47:49.372 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6199999999999998e-07 2024-09-15T21:47:49.372 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.373 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.373 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.373 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565938+0000", 2024-09-15T21:47:49.373 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4970000000000003e-06 2024-09-15T21:47:49.373 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.373 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.373 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.373 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565945+0000", 2024-09-15T21:47:49.373 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5069999999999997e-06 2024-09-15T21:47:49.373 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.374 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.374 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.374 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565948+0000", 2024-09-15T21:47:49.374 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:47:49.374 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.374 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.374 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.374 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566200+0000", 2024-09-15T21:47:49.374 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025170200000000002 2024-09-15T21:47:49.374 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.374 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.374 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:49.374 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566218+0000", 2024-09-15T21:47:49.375 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7543000000000001e-05 2024-09-15T21:47:49.375 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.375 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.375 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:49.375 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571668+0000", 2024-09-15T21:47:49.375 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054502999999999999 2024-09-15T21:47:49.375 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.375 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.375 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.375 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571694+0000", 2024-09-15T21:47:49.375 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5979e-05 2024-09-15T21:47:49.375 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.375 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.375 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.376 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.376 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.376 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:2 2.1 e22/19 2:82971f97:::404.00000000:head v 22'5, mlcod=22'3)", 2024-09-15T21:47:49.376 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:47:49.376 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 57.716080581, 2024-09-15T21:47:49.376 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056016019999999998, 2024-09-15T21:47:49.376 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.376 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.376 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:49.376 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.376 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.376 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.376 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:47:49.377 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.377 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.377 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.377 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.377 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:47:49.377 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.377 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.377 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.377 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.377 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566142+0000", 2024-09-15T21:47:49.377 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:47:49.377 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.377 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.378 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.378 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566149+0000", 2024-09-15T21:47:49.378 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7510000000000008e-06 2024-09-15T21:47:49.378 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.378 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.378 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.378 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566161+0000", 2024-09-15T21:47:49.378 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1259e-05 2024-09-15T21:47:49.378 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.378 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.378 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.379 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566168+0000", 2024-09-15T21:47:49.379 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.255e-06 2024-09-15T21:47:49.379 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.379 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.379 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.379 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566227+0000", 2024-09-15T21:47:49.379 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8961e-05 2024-09-15T21:47:49.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.380 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.380 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:49.380 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566247+0000", 2024-09-15T21:47:49.380 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0536000000000001e-05 2024-09-15T21:47:49.380 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.380 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.380 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:49.380 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571694+0000", 2024-09-15T21:47:49.381 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005446762 2024-09-15T21:47:49.381 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.381 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.382 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.382 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571743+0000", 2024-09-15T21:47:49.382 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8396999999999998e-05 2024-09-15T21:47:49.382 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.382 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.382 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.382 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.382 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.382 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:17 2.14 e22/19 2:2eebc8f4:::62f.00000000:head v 22'9, mlcod=22'7)", 2024-09-15T21:47:49.382 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:47:49.382 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 57.716064484, 2024-09-15T21:47:49.382 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056532860000000004, 2024-09-15T21:47:49.383 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.383 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.383 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:49.383 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.383 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.383 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.383 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:47:49.383 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.383 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.383 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.383 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.383 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:47:49.384 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.384 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.384 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.384 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.384 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566158+0000", 2024-09-15T21:47:49.384 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.92e-07 2024-09-15T21:47:49.384 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.384 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.384 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.384 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566172+0000", 2024-09-15T21:47:49.384 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3976e-05 2024-09-15T21:47:49.384 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.384 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.385 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.385 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566407+0000", 2024-09-15T21:47:49.385 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023483900000000001 2024-09-15T21:47:49.385 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.385 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.385 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.385 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566416+0000", 2024-09-15T21:47:49.385 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0780000000000002e-06 2024-09-15T21:47:49.385 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.385 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.385 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.385 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566476+0000", 2024-09-15T21:47:49.385 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9908999999999998e-05 2024-09-15T21:47:49.385 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.386 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.386 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:49.386 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566493+0000", 2024-09-15T21:47:49.386 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7567e-05 2024-09-15T21:47:49.386 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.386 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.386 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:49.386 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571762+0000", 2024-09-15T21:47:49.386 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052682959999999996 2024-09-15T21:47:49.386 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.386 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.386 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.386 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571810+0000", 2024-09-15T21:47:49.387 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8829000000000003e-05 2024-09-15T21:47:49.387 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.387 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.387 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.387 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.387 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.387 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:18 2.14 e22/19 2:2eebc8f4:::62f.00000000:head v 22'10, mlcod=22'7)", 2024-09-15T21:47:49.387 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:47:49.387 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 57.715775409000003, 2024-09-15T21:47:49.387 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053992579999999997, 2024-09-15T21:47:49.387 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:49.387 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:49.387 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:49.387 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:49.388 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.388 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:49.388 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:47:49.388 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.388 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.388 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.388 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:49.388 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:47:49.388 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:49.388 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.388 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.388 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:49.388 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566447+0000", 2024-09-15T21:47:49.389 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2e-07 2024-09-15T21:47:49.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.389 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.389 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:49.389 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566452+0000", 2024-09-15T21:47:49.389 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0390000000000004e-06 2024-09-15T21:47:49.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.389 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.389 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:49.389 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566460+0000", 2024-09-15T21:47:49.389 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1389999999999995e-06 2024-09-15T21:47:49.389 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.389 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.389 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:49.390 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566473+0000", 2024-09-15T21:47:49.390 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3093e-05 2024-09-15T21:47:49.390 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.390 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.390 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:49.390 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566688+0000", 2024-09-15T21:47:49.390 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021537100000000001 2024-09-15T21:47:49.390 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.390 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.390 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:49.390 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566706+0000", 2024-09-15T21:47:49.390 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7802999999999999e-05 2024-09-15T21:47:49.390 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.390 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.391 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:49.391 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571813+0000", 2024-09-15T21:47:49.391 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051065140000000004 2024-09-15T21:47:49.391 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:49.391 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:49.391 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:49.391 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571845+0000", 2024-09-15T21:47:49.391 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2879000000000001e-05 2024-09-15T21:47:49.391 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.391 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.391 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.391 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:49.391 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:49.391 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:49.393 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:47:49.393 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:49.391+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:49.410 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:47:49.493 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:49.491+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:49.510 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:49.510 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:49.510 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:49.510 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:49.519 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:47:49.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.530+0000 7f0703a77640 1 Processor -- start 2024-09-15T21:47:49.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.531+0000 7f0703a77640 1 -- start start 2024-09-15T21:47:49.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.531+0000 7f0703a77640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f06fc06cf60 0x7f06fc06d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:49.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.531+0000 7f0703a77640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f06fc06d880 0x7f06fc154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:49.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.531+0000 7f0703a77640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06fc15e5c0 con 0x7f06fc06fd80 2024-09-15T21:47:49.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.531+0000 7f0703a77640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f06fc15f450 con 0x7f06fc06cf60 2024-09-15T21:47:49.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.531+0000 7f0703a77640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f06fc15e840 con 0x7f06fc06d880 2024-09-15T21:47:49.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.532+0000 7f0701813640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f06fc06cf60 0x7f06fc06d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:49.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.532+0000 7f0701813640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f06fc06cf60 0x7f06fc06d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57480/0 (socket says 172.21.15.100:57480) 2024-09-15T21:47:49.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.532+0000 7f0701813640 1 -- 172.21.15.100:0/687655502 learned_addr learned my addr 172.21.15.100:0/687655502 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:49.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.532+0000 7f0702014640 1 --2- 172.21.15.100:0/687655502 >> v2:172.21.15.122:3300/0 conn(0x7f06fc06d880 0x7f06fc154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:49.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.532+0000 7f0702014640 1 -- 172.21.15.100:0/687655502 >> v2:172.21.15.100:3300/0 conn(0x7f06fc06cf60 msgr2=0x7f06fc06d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:49.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.532+0000 7f0702014640 1 --2- 172.21.15.100:0/687655502 >> v2:172.21.15.100:3300/0 conn(0x7f06fc06cf60 0x7f06fc06d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:49.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.532+0000 7f0702014640 1 -- 172.21.15.100:0/687655502 >> v1:172.21.15.122:6789/0 conn(0x7f06fc06fd80 legacy=0x7f06fc0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:49.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.532+0000 7f0702014640 1 -- 172.21.15.100:0/687655502 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06fc15eb20 con 0x7f06fc06d880 2024-09-15T21:47:49.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.533+0000 7f0702014640 1 --2- 172.21.15.100:0/687655502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06fc06d880 0x7f06fc154ae0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f06fc15c140 tx=0x7f06f402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=443be835e125cc8d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:49.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.533+0000 7f0700811640 1 -- 172.21.15.100:0/687655502 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4007740 con 0x7f06fc06d880 2024-09-15T21:47:49.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.533+0000 7f0700811640 1 -- 172.21.15.100:0/687655502 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f40078e0 con 0x7f06fc06d880 2024-09-15T21:47:49.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.533+0000 7f0700811640 1 -- 172.21.15.100:0/687655502 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4007cb0 con 0x7f06fc06d880 2024-09-15T21:47:49.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.533+0000 7f0703a77640 1 -- 172.21.15.100:0/687655502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06fc06d880 msgr2=0x7f06fc154ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:49.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.533+0000 7f0703a77640 1 --2- 172.21.15.100:0/687655502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06fc06d880 0x7f06fc154ae0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f06fc15c140 tx=0x7f06f402a710 comp rx=0 tx=0).stop 2024-09-15T21:47:49.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.533+0000 7f0703a77640 1 -- 172.21.15.100:0/687655502 shutdown_connections 2024-09-15T21:47:49.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.533+0000 7f0703a77640 1 --2- 172.21.15.100:0/687655502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06fc06d880 0x7f06fc154ae0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:49.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.533+0000 7f0703a77640 1 --2- 172.21.15.100:0/687655502 >> v2:172.21.15.100:3300/0 conn(0x7f06fc06cf60 0x7f06fc06d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:49.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.533+0000 7f0703a77640 1 -- 172.21.15.100:0/687655502 >> 172.21.15.100:0/687655502 conn(0x7f06fc0cb3a0 msgr2=0x7f06fc0cb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:49.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.534+0000 7f0703a77640 1 -- 172.21.15.100:0/687655502 shutdown_connections 2024-09-15T21:47:49.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.534+0000 7f0703a77640 1 -- 172.21.15.100:0/687655502 wait complete. 2024-09-15T21:47:49.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.535+0000 7f0703a77640 1 Processor -- start 2024-09-15T21:47:49.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.535+0000 7f0703a77640 1 -- start start 2024-09-15T21:47:49.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f0703a77640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f06fc06cf60 0x7f06fc0c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:49.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f0703a77640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06fc06d880 0x7f06fc1f0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:49.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f0703a77640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06fc06fd80 0x7f06fc15a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:49.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f0703a77640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc0c9310 con 0x7f06fc06d880 2024-09-15T21:47:49.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f0703a77640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f06fc15f0e0 con 0x7f06fc06fd80 2024-09-15T21:47:49.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f0703a77640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f06fc0710c0 con 0x7f06fc06cf60 2024-09-15T21:47:49.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f0702014640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06fc06fd80 0x7f06fc15a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:49.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f0701813640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f06fc06cf60 0x7f06fc0c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:49.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f0702014640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06fc06fd80 0x7f06fc15a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:34138/0 (socket says 172.21.15.100:34138) 2024-09-15T21:47:49.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f0702014640 1 -- 172.21.15.100:0/3118622139 learned_addr learned my addr 172.21.15.100:0/3118622139 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:49.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f0701813640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f06fc06cf60 0x7f06fc0c8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:58070/0 (socket says 172.21.15.100:58070) 2024-09-15T21:47:49.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f0701012640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06fc06d880 0x7f06fc1f0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:49.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f0702014640 1 -- 172.21.15.100:0/3118622139 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f06fc06cf60 msgr2=0x7f06fc0c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:49.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f0702014640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f06fc06cf60 0x7f06fc0c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:49.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f0702014640 1 -- 172.21.15.100:0/3118622139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06fc06d880 msgr2=0x7f06fc1f0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:49.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f0702014640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06fc06d880 0x7f06fc1f0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:49.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f0702014640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06fc0c7ab0 con 0x7f06fc06fd80 2024-09-15T21:47:49.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f0701813640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f06fc06cf60 0x7f06fc0c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:49.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f0702014640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06fc06fd80 0x7f06fc15a250 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009e50 tx=0x7f06f4004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:49.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.536+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4040070 con 0x7f06fc06fd80 2024-09-15T21:47:49.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.537+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06f4031bf0 con 0x7f06fc06fd80 2024-09-15T21:47:49.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.537+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06f4009450 con 0x7f06fc06fd80 2024-09-15T21:47:49.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.537+0000 7f0703a77640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06fc1f1810 con 0x7f06fc06fd80 2024-09-15T21:47:49.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.537+0000 7f0703a77640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06fc1f1dd0 con 0x7f06fc06fd80 2024-09-15T21:47:49.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.538+0000 7f0703a77640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f06fc15afd0 con 0x7f06fc06fd80 2024-09-15T21:47:49.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.539+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f06f4039040 con 0x7f06fc06fd80 2024-09-15T21:47:49.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.539+0000 7f06f27fc640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f06e006c2b0 0x7f06e006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:49.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.539+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5886+0+0 (secure 0 0 0) 0x7f06f40acaf0 con 0x7f06fc06fd80 2024-09-15T21:47:49.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.539+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v43) ==== 54+0+16 (secure 0 0 0) 0x7f06f40b1240 con 0x7f06fc06fd80 2024-09-15T21:47:49.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.539+0000 7f0701813640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f06e006c2b0 0x7f06e006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:49.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.540+0000 7f06dffff640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f06d8001650 0x7f06d8003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:49.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.540+0000 7f06dffff640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f06d8006bb0 con 0x7f06d8001650 2024-09-15T21:47:49.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.540+0000 7f0701012640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f06d8001650 0x7f06d8003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:49.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.540+0000 7f0701813640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f06e006c2b0 0x7f06e006e770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009fc0 tx=0x7f06f8009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:49.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.540+0000 7f0701012640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f06d8001650 0x7f06d8003b10 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:49.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.541+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f06d8006bb0 con 0x7f06d8001650 2024-09-15T21:47:49.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.555+0000 7f06dffff640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f06d8005c60 con 0x7f06d8001650 2024-09-15T21:47:49.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.557+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f06d8005c60 con 0x7f06d8001650 2024-09-15T21:47:49.559 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:49.560 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:49.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.557+0000 7f06dffff640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f06d8004190 0x7f06d8004640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:49.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.557+0000 7f06dffff640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f06d8004c00 con 0x7f06d8004190 2024-09-15T21:47:49.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.558+0000 7f0701813640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f06d8004190 0x7f06d8004640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:49.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.558+0000 7f0701813640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f06d8004190 0x7f06d8004640 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:49.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.559+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f06d8004c00 con 0x7f06d8004190 2024-09-15T21:47:49.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.586+0000 7f06dffff640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f06d8005170 con 0x7f06d8004190 2024-09-15T21:47:49.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.588+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f06d8005170 con 0x7f06d8004190 2024-09-15T21:47:49.589 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:49.590 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:49.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.588+0000 7f06dffff640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f06d8009370 0x7f06d800b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:49.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.588+0000 7f06dffff640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f06d800e6f0 con 0x7f06d8009370 2024-09-15T21:47:49.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.588+0000 7f0701012640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f06d8009370 0x7f06d800b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:49.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.588+0000 7f0701012640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f06d8009370 0x7f06d800b760 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:49.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.589+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f06d800e6f0 con 0x7f06d8009370 2024-09-15T21:47:49.594 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:49.592+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:49.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.615+0000 7f06dffff640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f06d800d760 con 0x7f06d8009370 2024-09-15T21:47:49.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.618+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f06d800d760 con 0x7f06d8009370 2024-09-15T21:47:49.620 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:49.621 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:49.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.618+0000 7f06dffff640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f06d800bcf0 0x7f06d800c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:49.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.618+0000 7f06dffff640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f06d800c6c0 con 0x7f06d800bcf0 2024-09-15T21:47:49.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.618+0000 7f0702014640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f06d800bcf0 0x7f06d800c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:49.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.619+0000 7f0702014640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f06d800bcf0 0x7f06d800c180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:49.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.620+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f06d800c6c0 con 0x7f06d800bcf0 2024-09-15T21:47:49.624 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:49.624 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:49.624 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:49.624 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:49.624 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:49.633 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:47:49.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.647+0000 7f06dffff640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f06d800cbd0 con 0x7f06d800bcf0 2024-09-15T21:47:49.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.649+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f06d800cbd0 con 0x7f06d800bcf0 2024-09-15T21:47:49.651 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:49.652 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:49.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.649+0000 7f06dffff640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f06d800cf90 0x7f06d8012950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:49.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.649+0000 7f06dffff640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f06d8015860 con 0x7f06d800cf90 2024-09-15T21:47:49.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.650+0000 7f0701813640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f06d800cf90 0x7f06d8012950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:49.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.650+0000 7f0701813640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f06d800cf90 0x7f06d8012950 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:49.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.651+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f06d8015860 con 0x7f06d800cf90 2024-09-15T21:47:49.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.677+0000 7f06dffff640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f06d8012e90 con 0x7f06d800cf90 2024-09-15T21:47:49.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.679+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f06d8012e90 con 0x7f06d800cf90 2024-09-15T21:47:49.681 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:49.682 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:49.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.679+0000 7f06dffff640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=44}) -- 0x7f06d8014a00 con 0x7f06fc06fd80 2024-09-15T21:47:49.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.679+0000 7f06dffff640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f06d8012e90 con 0x7f06fc06fd80 2024-09-15T21:47:49.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.679+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f06f40a9050 con 0x7f06fc06fd80 2024-09-15T21:47:49.682 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:47:49.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.680+0000 7f06dffff640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f06d80132f0 0x7f06d8013740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:49.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.680+0000 7f06dffff640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f06d8013e30 con 0x7f06d80132f0 2024-09-15T21:47:49.682 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.680+0000 7f0701012640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f06d80132f0 0x7f06d8013740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:49.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.680+0000 7f0701012640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f06d80132f0 0x7f06d8013740 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:49.683 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.681+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f06d8013e30 con 0x7f06d80132f0 2024-09-15T21:47:49.697 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:49.695+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:49.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.708+0000 7f06dffff640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f06d8014070 con 0x7f06d80132f0 2024-09-15T21:47:49.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.710+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f06d8014070 con 0x7f06d80132f0 2024-09-15T21:47:49.712 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:49.713 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:49.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.710+0000 7f06dffff640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f06d8017ab0 0x7f06d8019ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:49.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.710+0000 7f06dffff640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f06d801ccf0 con 0x7f06d8017ab0 2024-09-15T21:47:49.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.710+0000 7f0702014640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f06d8017ab0 0x7f06d8019ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:49.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.710+0000 7f0702014640 1 --2- 172.21.15.100:0/3118622139 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f06d8017ab0 0x7f06d8019ea0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:49.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.711+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f06d801ccf0 con 0x7f06d8017ab0 2024-09-15T21:47:49.737 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:49.738 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:49.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.738+0000 7f06dffff640 1 -- 172.21.15.100:0/3118622139 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f06d801a3e0 con 0x7f06d8017ab0 2024-09-15T21:47:49.741 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:49.740+0000 7f06f27fc640 1 -- 172.21.15.100:0/3118622139 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f06d801a3e0 con 0x7f06d8017ab0 2024-09-15T21:47:49.741 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:49.742 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:49.746 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:47:49.747 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:47:49.747 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:47:49.797 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:49.795+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:49.891 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:49.891 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:49.898 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:49.896+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:49.900 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:47:49.999 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:49.997+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:50.003 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:50.003 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:50.012 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:47:50.100 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:50.098+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:50.113 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:50.113 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:50.122 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:47:50.201 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:50.199+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:50.226 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:50.226 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:50.235 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:47:50.302 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:50.300+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:50.338 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:50.338 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:50.347 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:47:50.433 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:50.431+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:50.492 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:50.492 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:50.501 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:47:50.504 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:50.502+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:50.605 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:50.603+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:50.665 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:50.665 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:50.677 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:47:50.705 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:50.704+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:50.806 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:50.805+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:50.827 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:50.827 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:50.839 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:47:50.857 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:47:50.923 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:50.922+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:51.008 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:51.007+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:51.031 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:51.031 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:51.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.036+0000 7f413695b640 1 Processor -- start 2024-09-15T21:47:51.038 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.037+0000 7f413695b640 1 -- start start 2024-09-15T21:47:51.040 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:47:51.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.038+0000 7f413695b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f4130155440 0x7f41300cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:51.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.038+0000 7f413695b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f41300cb7d0 0x7f41300cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:51.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.038+0000 7f413695b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f413015e7a0 con 0x7f413006d0e0 2024-09-15T21:47:51.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.038+0000 7f413695b640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f413015b470 con 0x7f4130155440 2024-09-15T21:47:51.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.038+0000 7f413695b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4130155820 con 0x7f41300cb7d0 2024-09-15T21:47:51.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.038+0000 7f4134ef8640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f413006d0e0 0x7f41300cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42128/0 (socket says 172.21.15.100:42128) 2024-09-15T21:47:51.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.038+0000 7f4134ef8640 1 -- 172.21.15.100:0/1870984549 learned_addr learned my addr 172.21.15.100:0/1870984549 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:51.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.038+0000 7f412ffff640 1 --2- 172.21.15.100:0/1870984549 >> v2:172.21.15.122:3300/0 conn(0x7f4130155440 0x7f41300cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:51.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.038+0000 7f412f7fe640 1 --2- 172.21.15.100:0/1870984549 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f41300cb7d0 0x7f41300cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:51.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.039+0000 7f412ffff640 1 -- 172.21.15.100:0/1870984549 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f41300cb7d0 msgr2=0x7f41300cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:51.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.039+0000 7f412ffff640 1 --2- 172.21.15.100:0/1870984549 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f41300cb7d0 0x7f41300cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:51.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.039+0000 7f412ffff640 1 -- 172.21.15.100:0/1870984549 >> v1:172.21.15.100:6789/0 conn(0x7f413006d0e0 legacy=0x7f41300cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:51.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.039+0000 7f412effd640 1 -- 172.21.15.100:0/1870984549 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 859988383 0 0) 0x7f413015e7a0 con 0x7f413006d0e0 2024-09-15T21:47:51.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.039+0000 7f412ffff640 1 -- 172.21.15.100:0/1870984549 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f413015ea40 con 0x7f4130155440 2024-09-15T21:47:51.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.039+0000 7f412f7fe640 1 --2- 172.21.15.100:0/1870984549 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f41300cb7d0 0x7f41300cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:51.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.040+0000 7f412ffff640 1 --2- 172.21.15.100:0/1870984549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4130155440 0x7f41300cd420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f41200097a0 tx=0x7f412002a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f56f5db9ef2b13a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:51.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.040+0000 7f412effd640 1 -- 172.21.15.100:0/1870984549 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120037070 con 0x7f4130155440 2024-09-15T21:47:51.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.040+0000 7f412effd640 1 -- 172.21.15.100:0/1870984549 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f412002ae40 con 0x7f4130155440 2024-09-15T21:47:51.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.040+0000 7f412effd640 1 -- 172.21.15.100:0/1870984549 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120032660 con 0x7f4130155440 2024-09-15T21:47:51.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.041+0000 7f413695b640 1 -- 172.21.15.100:0/1870984549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4130155440 msgr2=0x7f41300cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:51.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.041+0000 7f413695b640 1 --2- 172.21.15.100:0/1870984549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4130155440 0x7f41300cd420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f41200097a0 tx=0x7f412002a0f0 comp rx=0 tx=0).stop 2024-09-15T21:47:51.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.041+0000 7f413695b640 1 -- 172.21.15.100:0/1870984549 shutdown_connections 2024-09-15T21:47:51.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.041+0000 7f413695b640 1 --2- 172.21.15.100:0/1870984549 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f41300cb7d0 0x7f41300cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:51.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.041+0000 7f413695b640 1 --2- 172.21.15.100:0/1870984549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4130155440 0x7f41300cd420 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:51.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.041+0000 7f413695b640 1 -- 172.21.15.100:0/1870984549 >> 172.21.15.100:0/1870984549 conn(0x7f41300680d0 msgr2=0x7f413006d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:51.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.041+0000 7f413695b640 1 -- 172.21.15.100:0/1870984549 shutdown_connections 2024-09-15T21:47:51.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.041+0000 7f413695b640 1 -- 172.21.15.100:0/1870984549 wait complete. 2024-09-15T21:47:51.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.043+0000 7f413695b640 1 Processor -- start 2024-09-15T21:47:51.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.043+0000 7f413695b640 1 -- start start 2024-09-15T21:47:51.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.045+0000 7f413695b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f413006d0e0 0x7f41301f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:51.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.045+0000 7f413695b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41300cb7d0 0x7f41300cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:51.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.045+0000 7f412f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41300cb7d0 0x7f41300cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:51.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.045+0000 7f412f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41300cb7d0 0x7f41300cfc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57508/0 (socket says 172.21.15.100:57508) 2024-09-15T21:47:51.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.045+0000 7f412ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f413006d0e0 0x7f41301f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:51.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.045+0000 7f412f7fe640 1 -- 172.21.15.100:0/3190321628 learned_addr learned my addr 172.21.15.100:0/3190321628 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:51.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.045+0000 7f413695b640 1 --2- 172.21.15.100:0/3190321628 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4130155440 0x7f41300b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:51.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.046+0000 7f413695b640 1 -- 172.21.15.100:0/3190321628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f413015f2e0 con 0x7f41300cb7d0 2024-09-15T21:47:51.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.046+0000 7f413695b640 1 -- 172.21.15.100:0/3190321628 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f413015f000 con 0x7f413006d0e0 2024-09-15T21:47:51.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.046+0000 7f4134ef8640 1 --2- 172.21.15.100:0/3190321628 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4130155440 0x7f41300b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:51.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.046+0000 7f413695b640 1 -- 172.21.15.100:0/3190321628 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f413015b070 con 0x7f4130155440 2024-09-15T21:47:51.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.046+0000 7f412f7fe640 1 -- 172.21.15.100:0/3190321628 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4130155440 msgr2=0x7f41300b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:51.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.046+0000 7f412f7fe640 1 --2- 172.21.15.100:0/3190321628 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4130155440 0x7f41300b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:51.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.046+0000 7f412f7fe640 1 -- 172.21.15.100:0/3190321628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f413006d0e0 msgr2=0x7f41301f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:51.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.046+0000 7f412f7fe640 1 --2- 172.21.15.100:0/3190321628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f413006d0e0 0x7f41301f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:51.049 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.046+0000 7f412f7fe640 1 -- 172.21.15.100:0/3190321628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41301f10d0 con 0x7f41300cb7d0 2024-09-15T21:47:51.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.047+0000 7f412f7fe640 1 --2- 172.21.15.100:0/3190321628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41300cb7d0 0x7f41300cfc20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f412400ebc0 tx=0x7f412400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:51.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.047+0000 7f412cff9640 1 -- 172.21.15.100:0/3190321628 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4124017070 con 0x7f41300cb7d0 2024-09-15T21:47:51.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.047+0000 7f412cff9640 1 -- 172.21.15.100:0/3190321628 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f412400edd0 con 0x7f41300cb7d0 2024-09-15T21:47:51.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.047+0000 7f412cff9640 1 -- 172.21.15.100:0/3190321628 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412400ce50 con 0x7f41300cb7d0 2024-09-15T21:47:51.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.047+0000 7f413695b640 1 -- 172.21.15.100:0/3190321628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41301f1390 con 0x7f41300cb7d0 2024-09-15T21:47:51.050 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.048+0000 7f413695b640 1 -- 172.21.15.100:0/3190321628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41301f1950 con 0x7f41300cb7d0 2024-09-15T21:47:51.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.049+0000 7f412cff9640 1 -- 172.21.15.100:0/3190321628 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f412401a070 con 0x7f41300cb7d0 2024-09-15T21:47:51.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.049+0000 7f413695b640 1 -- 172.21.15.100:0/3190321628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41300b3940 con 0x7f41300cb7d0 2024-09-15T21:47:51.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.050+0000 7f412cff9640 1 --2- 172.21.15.100:0/3190321628 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f411c06c380 0x7f411c06e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:51.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.050+0000 7f412cff9640 1 -- 172.21.15.100:0/3190321628 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5886+0+0 (secure 0 0 0) 0x7f412408b470 con 0x7f41300cb7d0 2024-09-15T21:47:51.052 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.051+0000 7f412ffff640 1 --2- 172.21.15.100:0/3190321628 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f411c06c380 0x7f411c06e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:51.053 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.052+0000 7f412ffff640 1 --2- 172.21.15.100:0/3190321628 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f411c06c380 0x7f411c06e840 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4120034000 tx=0x7f412002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:51.061 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.059+0000 7f412cff9640 1 -- 172.21.15.100:0/3190321628 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4124059a80 con 0x7f41300cb7d0 2024-09-15T21:47:51.109 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:51.107+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:51.155 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:51.155 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:51.164 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:47:51.249 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:51.247+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:51.310 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:51.310 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:51.310 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:51.308+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:51.319 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:47:51.411 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:51.410+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:51.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.422+0000 7f413695b640 1 -- 172.21.15.100:0/3190321628 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7f41300caed0 con 0x7f41300cb7d0 2024-09-15T21:47:51.492 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:51.492 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:51.501 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:47:51.543 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:51.541+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:51.614 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:51.612+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:51.655 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:51.655 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:51.664 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:47:51.715 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:51.713+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:51.749 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:47:51.784 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:51.784 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:51.796 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:47:51.837 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:51.835+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:51.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.838+0000 7f412cff9640 1 -- 172.21.15.100:0/3190321628 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v44) ==== 87+0+0 (secure 0 0 0) 0x7f4124059380 con 0x7f41300cb7d0 2024-09-15T21:47:51.840 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is set 2024-09-15T21:47:51.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.841+0000 7f413695b640 1 -- 172.21.15.100:0/3190321628 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f411c06c380 msgr2=0x7f411c06e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:51.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.841+0000 7f413695b640 1 --2- 172.21.15.100:0/3190321628 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f411c06c380 0x7f411c06e840 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4120034000 tx=0x7f412002e000 comp rx=0 tx=0).stop 2024-09-15T21:47:51.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.841+0000 7f413695b640 1 -- 172.21.15.100:0/3190321628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41300cb7d0 msgr2=0x7f41300cfc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:51.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.841+0000 7f413695b640 1 --2- 172.21.15.100:0/3190321628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41300cb7d0 0x7f41300cfc20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f412400ebc0 tx=0x7f412400c520 comp rx=0 tx=0).stop 2024-09-15T21:47:51.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.841+0000 7f413695b640 1 -- 172.21.15.100:0/3190321628 shutdown_connections 2024-09-15T21:47:51.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.841+0000 7f413695b640 1 --2- 172.21.15.100:0/3190321628 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f411c06c380 0x7f411c06e840 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4120034000 tx=0x7f412002e000 comp rx=0 tx=0).stop 2024-09-15T21:47:51.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.841+0000 7f413695b640 1 --2- 172.21.15.100:0/3190321628 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4130155440 0x7f41300b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:51.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.841+0000 7f413695b640 1 --2- 172.21.15.100:0/3190321628 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f41300cb7d0 0x7f41300cfc20 secure :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f412400ebc0 tx=0x7f412400c520 comp rx=0 tx=0).stop 2024-09-15T21:47:51.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.841+0000 7f413695b640 1 --2- 172.21.15.100:0/3190321628 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f413006d0e0 0x7f41301f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:51.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.841+0000 7f413695b640 1 -- 172.21.15.100:0/3190321628 >> 172.21.15.100:0/3190321628 conn(0x7f41300680d0 msgr2=0x7f413014a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:51.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.842+0000 7f413695b640 1 -- 172.21.15.100:0/3190321628 shutdown_connections 2024-09-15T21:47:51.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.842+0000 7f413695b640 1 -- 172.21.15.100:0/3190321628 wait complete. 2024-09-15T21:47:51.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.881+0000 7fbadc331640 1 Processor -- start 2024-09-15T21:47:51.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.881+0000 7fbadc331640 1 -- start start 2024-09-15T21:47:51.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.883+0000 7fbadc331640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fbad4155030 0x7fbad415ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:51.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.883+0000 7fbadc331640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbad4072080 0x7fbad406f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:51.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.883+0000 7fbadc331640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbad40ca070 con 0x7fbad4155410 2024-09-15T21:47:51.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.883+0000 7fbadc331640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fbad415bfb0 con 0x7fbad4072080 2024-09-15T21:47:51.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.883+0000 7fbadc331640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fbad4072550 con 0x7fbad4155030 2024-09-15T21:47:51.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.883+0000 7fbad98cc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbad4072080 0x7fbad406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:51.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.883+0000 7fbada0cd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fbad4155030 0x7fbad415ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:51.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.883+0000 7fbad98cc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbad4072080 0x7fbad406f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57522/0 (socket says 172.21.15.100:57522) 2024-09-15T21:47:51.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.883+0000 7fbad98cc640 1 -- 172.21.15.100:0/866729838 learned_addr learned my addr 172.21.15.100:0/866729838 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:51.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.883+0000 7fbad98cc640 1 -- 172.21.15.100:0/866729838 >> v2:172.21.15.122:3300/0 conn(0x7fbad4155030 msgr2=0x7fbad415ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:51.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.883+0000 7fbad98cc640 1 --2- 172.21.15.100:0/866729838 >> v2:172.21.15.122:3300/0 conn(0x7fbad4155030 0x7fbad415ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:51.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.883+0000 7fbad98cc640 1 -- 172.21.15.100:0/866729838 >> v1:172.21.15.100:6789/0 conn(0x7fbad4155410 legacy=0x7fbad406be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:51.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.883+0000 7fbad90cb640 1 -- 172.21.15.100:0/866729838 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4065966177 0 0) 0x7fbad40ca070 con 0x7fbad4155410 2024-09-15T21:47:51.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.883+0000 7fbad98cc640 1 -- 172.21.15.100:0/866729838 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad415e5e0 con 0x7fbad4072080 2024-09-15T21:47:51.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.884+0000 7fbad98cc640 1 --2- 172.21.15.100:0/866729838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbad4072080 0x7fbad406f910 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbad40cea00 tx=0x7fbac402a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2a4366c4b8a319 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:51.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.884+0000 7fbad90cb640 1 -- 172.21.15.100:0/866729838 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4037070 con 0x7fbad4072080 2024-09-15T21:47:51.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.884+0000 7fbad90cb640 1 -- 172.21.15.100:0/866729838 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac4032640 con 0x7fbad4072080 2024-09-15T21:47:51.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.884+0000 7fbad90cb640 1 -- 172.21.15.100:0/866729838 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac402ae60 con 0x7fbad4072080 2024-09-15T21:47:51.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.885+0000 7fbadc331640 1 -- 172.21.15.100:0/866729838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbad4072080 msgr2=0x7fbad406f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:51.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.885+0000 7fbadc331640 1 --2- 172.21.15.100:0/866729838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbad4072080 0x7fbad406f910 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbad40cea00 tx=0x7fbac402a310 comp rx=0 tx=0).stop 2024-09-15T21:47:51.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.885+0000 7fbadc331640 1 -- 172.21.15.100:0/866729838 shutdown_connections 2024-09-15T21:47:51.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.885+0000 7fbadc331640 1 --2- 172.21.15.100:0/866729838 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbad4072080 0x7fbad406f910 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:51.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.885+0000 7fbadc331640 1 --2- 172.21.15.100:0/866729838 >> v2:172.21.15.122:3300/0 conn(0x7fbad4155030 0x7fbad415ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:51.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.885+0000 7fbadc331640 1 -- 172.21.15.100:0/866729838 >> 172.21.15.100:0/866729838 conn(0x7fbad40cd9a0 msgr2=0x7fbad40cdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:51.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.886+0000 7fbadc331640 1 -- 172.21.15.100:0/866729838 shutdown_connections 2024-09-15T21:47:51.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.886+0000 7fbadc331640 1 -- 172.21.15.100:0/866729838 wait complete. 2024-09-15T21:47:51.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.888+0000 7fbadc331640 1 Processor -- start 2024-09-15T21:47:51.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.888+0000 7fbadc331640 1 -- start start 2024-09-15T21:47:51.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.889+0000 7fbadc331640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbad4072080 0x7fbad4158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:51.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.889+0000 7fbadc331640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbad4155030 0x7fbad40cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:51.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.889+0000 7fbadc331640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbad4155410 0x7fbad41f0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:51.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.890+0000 7fbadc331640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbad415ee80 con 0x7fbad4155030 2024-09-15T21:47:51.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.890+0000 7fbad98cc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbad4155030 0x7fbad40cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:51.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.890+0000 7fbada0cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbad4072080 0x7fbad4158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:51.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.890+0000 7fbad98cc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbad4155030 0x7fbad40cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57528/0 (socket says 172.21.15.100:57528) 2024-09-15T21:47:51.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.890+0000 7fbad98cc640 1 -- 172.21.15.100:0/662332273 learned_addr learned my addr 172.21.15.100:0/662332273 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:51.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.890+0000 7fbada8ce640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbad4155410 0x7fbad41f0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:51.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.890+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbad415eba0 con 0x7fbad4072080 2024-09-15T21:47:51.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.890+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbad415f160 con 0x7fbad4155410 2024-09-15T21:47:51.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.890+0000 7fbad98cc640 1 -- 172.21.15.100:0/662332273 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbad4155410 msgr2=0x7fbad41f0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:51.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.890+0000 7fbad98cc640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbad4155410 0x7fbad41f0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:51.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.890+0000 7fbad98cc640 1 -- 172.21.15.100:0/662332273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbad4072080 msgr2=0x7fbad4158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:51.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.890+0000 7fbad98cc640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbad4072080 0x7fbad4158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:51.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.890+0000 7fbad98cc640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad4150190 con 0x7fbad4155030 2024-09-15T21:47:51.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.891+0000 7fbad98cc640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbad4155030 0x7fbad40cd0d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbac4000c00 tx=0x7fbac40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:51.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.891+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac4037040 con 0x7fbad4155030 2024-09-15T21:47:51.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.891+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbac4042070 con 0x7fbad4155030 2024-09-15T21:47:51.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.891+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbac402ae60 con 0x7fbad4155030 2024-09-15T21:47:51.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.891+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbad415e090 con 0x7fbad4155030 2024-09-15T21:47:51.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.891+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbad41506e0 con 0x7fbad4155030 2024-09-15T21:47:51.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.892+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fbad415f440 con 0x7fbad4155030 2024-09-15T21:47:51.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.893+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbac40309b0 con 0x7fbad4155030 2024-09-15T21:47:51.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.893+0000 7fbacaffd640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbab806c2e0 0x7fbab806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:51.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.894+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5886+0+0 (secure 0 0 0) 0x7fbac40ab950 con 0x7fbad4155030 2024-09-15T21:47:51.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.894+0000 7fbada0cd640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbab806c2e0 0x7fbab806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:51.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.894+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v44) ==== 54+0+16 (secure 0 0 0) 0x7fbac40b0240 con 0x7fbad4155030 2024-09-15T21:47:51.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.894+0000 7fbadc331640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fbad414fb70 0x7fbad40c6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:51.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.894+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbad41510f0 con 0x7fbad414fb70 2024-09-15T21:47:51.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.894+0000 7fbada8ce640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fbad414fb70 0x7fbad40c6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:51.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.894+0000 7fbada0cd640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbab806c2e0 0x7fbab806e7a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009aa0 tx=0x7fbad0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:51.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.894+0000 7fbada8ce640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fbad414fb70 0x7fbad40c6640 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:51.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.895+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbad41510f0 con 0x7fbad414fb70 2024-09-15T21:47:51.917 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:51.915+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:51.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.922+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbad4153ce0 con 0x7fbad414fb70 2024-09-15T21:47:51.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.924+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbad4153ce0 con 0x7fbad414fb70 2024-09-15T21:47:51.926 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:51.927 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:51.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.924+0000 7fbadc331640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fbad414e980 0x7fbad415c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:51.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.924+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fbad40c9760 con 0x7fbad414e980 2024-09-15T21:47:51.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.925+0000 7fbada0cd640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fbad414e980 0x7fbad415c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:51.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.925+0000 7fbada0cd640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fbad414e980 0x7fbad415c6c0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:51.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.926+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbad40c9760 con 0x7fbad414e980 2024-09-15T21:47:51.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.940+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbad4150920 con 0x7fbad414e980 2024-09-15T21:47:51.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.942+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbad4150920 con 0x7fbad414e980 2024-09-15T21:47:51.944 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:51.945 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:51.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.942+0000 7fbadc331640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fbad414cf20 0x7fbad414d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:51.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.942+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fbad414d880 con 0x7fbad414cf20 2024-09-15T21:47:51.945 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.942+0000 7fbad98cc640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fbad414cf20 0x7fbad414d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:51.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.943+0000 7fbad98cc640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fbad414cf20 0x7fbad414d340 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:51.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.944+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbad414d880 con 0x7fbad414cf20 2024-09-15T21:47:51.959 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.957+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbad4062630 con 0x7fbad414cf20 2024-09-15T21:47:51.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.960+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbad4062630 con 0x7fbad414cf20 2024-09-15T21:47:51.961 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:51.962 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:51.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.960+0000 7fbadc331640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fbad4063110 0x7fbad4072f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:51.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.960+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fbad4063570 con 0x7fbad4063110 2024-09-15T21:47:51.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.960+0000 7fbada8ce640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fbad4063110 0x7fbad4072f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:51.962 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.960+0000 7fbada8ce640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fbad4063110 0x7fbad4072f20 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:51.963 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.961+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbad4063570 con 0x7fbad4063110 2024-09-15T21:47:51.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.975+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbad4073730 con 0x7fbad4063110 2024-09-15T21:47:51.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.976+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbad4073730 con 0x7fbad4063110 2024-09-15T21:47:51.979 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:51.980 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:51.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.977+0000 7fbadc331640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fbad402c640 0x7fbad402ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:51.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.977+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fbad4073e50 con 0x7fbad402c640 2024-09-15T21:47:51.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.977+0000 7fbada0cd640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fbad402c640 0x7fbad402ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:51.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.977+0000 7fbada0cd640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fbad402c640 0x7fbad402ca80 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:51.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.978+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbad4073e50 con 0x7fbad402c640 2024-09-15T21:47:51.993 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.991+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbad4073e50 con 0x7fbad402c640 2024-09-15T21:47:51.995 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.993+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbad4073e50 con 0x7fbad402c640 2024-09-15T21:47:51.995 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:51.996 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:51.996 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.994+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=45}) -- 0x7fbad402cfc0 con 0x7fbad4155030 2024-09-15T21:47:51.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.994+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbad4073e50 con 0x7fbad4155030 2024-09-15T21:47:51.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.994+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbac4079390 con 0x7fbad4155030 2024-09-15T21:47:51.997 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:47:51.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.994+0000 7fbadc331640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fbad402d860 0x7fbad4207cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:51.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.994+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fbad420aaa0 con 0x7fbad402d860 2024-09-15T21:47:51.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.995+0000 7fbad98cc640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fbad402d860 0x7fbad4207cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:51.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.995+0000 7fbad98cc640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fbad402d860 0x7fbad4207cd0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:51.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:51.996+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbad420aaa0 con 0x7fbad402d860 2024-09-15T21:47:52.002 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:52.002 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:52.002 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:52.002 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:52.002 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.002 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:52.003 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:52.003 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 83.585662300999999, 2024-09-15T21:47:52.003 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00598016, 2024-09-15T21:47:52.003 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.003 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.003 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.003 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.003 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.003 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.003 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:52.003 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.004 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.004 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.004 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:47:52.004 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.004 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.004 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.004 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413339+0000", 2024-09-15T21:47:52.004 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6279999999999998e-06 2024-09-15T21:47:52.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.005 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.005 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.005 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413354+0000", 2024-09-15T21:47:52.005 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4877e-05 2024-09-15T21:47:52.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.005 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.005 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.005 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413418+0000", 2024-09-15T21:47:52.005 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4274000000000002e-05 2024-09-15T21:47:52.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.005 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.005 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.005 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413428+0000", 2024-09-15T21:47:52.006 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5899999999999997e-06 2024-09-15T21:47:52.006 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.006 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.006 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.006 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413563+0000", 2024-09-15T21:47:52.006 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013521900000000001 2024-09-15T21:47:52.006 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.006 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.006 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.006 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413678+0000", 2024-09-15T21:47:52.006 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000114904 2024-09-15T21:47:52.006 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.006 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.006 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:52.007 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413789+0000", 2024-09-15T21:47:52.007 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011069300000000001 2024-09-15T21:47:52.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.007 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.007 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.007 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417603+0000", 2024-09-15T21:47:52.007 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038138460000000001 2024-09-15T21:47:52.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.007 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.007 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.007 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419220+0000", 2024-09-15T21:47:52.007 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001617678 2024-09-15T21:47:52.008 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.008 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.008 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.008 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419275+0000", 2024-09-15T21:47:52.008 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4391999999999997e-05 2024-09-15T21:47:52.008 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.008 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.008 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.008 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419316+0000", 2024-09-15T21:47:52.008 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1059000000000001e-05 2024-09-15T21:47:52.008 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.008 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.008 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.008 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.009 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.009 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:52.009 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:52.009 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 83.585640173000002, 2024-09-15T21:47:52.009 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0087926470000000007, 2024-09-15T21:47:52.009 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.009 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.009 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.009 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.009 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.009 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.010 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:52.010 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.010 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.010 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.010 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.010 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:52.010 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.010 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.010 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.010 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.010 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:47:52.010 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4099999999999999e-07 2024-09-15T21:47:52.010 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.010 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.011 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.011 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413361+0000", 2024-09-15T21:47:52.011 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6189999999999998e-06 2024-09-15T21:47:52.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.011 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.011 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.011 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413461+0000", 2024-09-15T21:47:52.011 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000100053 2024-09-15T21:47:52.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.011 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.011 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.011 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413466+0000", 2024-09-15T21:47:52.011 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3850000000000001e-06 2024-09-15T21:47:52.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.012 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.012 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.012 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414073+0000", 2024-09-15T21:47:52.012 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000606724 2024-09-15T21:47:52.012 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.012 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.012 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.012 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414143+0000", 2024-09-15T21:47:52.012 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9727000000000005e-05 2024-09-15T21:47:52.012 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.012 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.012 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:52.012 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414259+0000", 2024-09-15T21:47:52.012 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000116178 2024-09-15T21:47:52.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.013 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.013 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.013 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417610+0000", 2024-09-15T21:47:52.013 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003351176 2024-09-15T21:47:52.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.013 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.013 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.013 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422058+0000", 2024-09-15T21:47:52.013 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004447599 2024-09-15T21:47:52.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.013 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.013 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.013 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422111+0000", 2024-09-15T21:47:52.013 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3606999999999997e-05 2024-09-15T21:47:52.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.014 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.014 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.014 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422150+0000", 2024-09-15T21:47:52.014 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9137999999999997e-05 2024-09-15T21:47:52.014 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.014 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.014 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.014 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.015 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:52.015 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:52.015 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 83.585635742999997, 2024-09-15T21:47:52.015 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0089009769999999992, 2024-09-15T21:47:52.015 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.015 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.015 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.015 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.015 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.015 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.015 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:52.015 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.015 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.016 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.016 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.016 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:47:52.016 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.016 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.016 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.016 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.016 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413363+0000", 2024-09-15T21:47:52.016 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3999999999999997e-07 2024-09-15T21:47:52.016 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.016 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.016 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.016 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413365+0000", 2024-09-15T21:47:52.016 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.216e-06 2024-09-15T21:47:52.017 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.017 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.017 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.017 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413481+0000", 2024-09-15T21:47:52.017 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000116311 2024-09-15T21:47:52.017 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.017 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.017 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.017 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413488+0000", 2024-09-15T21:47:52.017 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0110000000000003e-06 2024-09-15T21:47:52.017 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.017 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.017 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.017 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414452+0000", 2024-09-15T21:47:52.018 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00096385399999999999 2024-09-15T21:47:52.018 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.018 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.018 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.018 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414545+0000", 2024-09-15T21:47:52.018 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.3024999999999998e-05 2024-09-15T21:47:52.018 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.018 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.018 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:52.018 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414644+0000", 2024-09-15T21:47:52.019 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9098000000000006e-05 2024-09-15T21:47:52.019 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.019 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.019 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.019 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417667+0000", 2024-09-15T21:47:52.019 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030229409999999999 2024-09-15T21:47:52.019 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.019 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.019 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.019 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422200+0000", 2024-09-15T21:47:52.019 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045327880000000003 2024-09-15T21:47:52.019 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.020 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.020 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.020 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422228+0000", 2024-09-15T21:47:52.020 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8109000000000001e-05 2024-09-15T21:47:52.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.020 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.020 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.020 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422263+0000", 2024-09-15T21:47:52.020 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5284000000000003e-05 2024-09-15T21:47:52.020 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.020 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.020 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.021 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.021 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:52.021 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:52.021 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 83.585632157000006, 2024-09-15T21:47:52.021 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0089846820000000008, 2024-09-15T21:47:52.021 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.021 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.021 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.021 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.021 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.021 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.021 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:52.022 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.022 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.022 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.022 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.022 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:52.022 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.022 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.022 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.022 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.022 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:47:52.022 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7599999999999998e-07 2024-09-15T21:47:52.022 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.022 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.023 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.023 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413368+0000", 2024-09-15T21:47:52.023 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.058e-06 2024-09-15T21:47:52.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.023 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.023 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.023 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413832+0000", 2024-09-15T21:47:52.023 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046398700000000001 2024-09-15T21:47:52.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.023 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.023 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.023 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413841+0000", 2024-09-15T21:47:52.023 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1710000000000008e-06 2024-09-15T21:47:52.024 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.024 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.024 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.024 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414869+0000", 2024-09-15T21:47:52.024 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001027584 2024-09-15T21:47:52.024 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.024 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.024 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.024 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414946+0000", 2024-09-15T21:47:52.024 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7177999999999999e-05 2024-09-15T21:47:52.024 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.024 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.025 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:52.025 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415054+0000", 2024-09-15T21:47:52.025 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010807 2024-09-15T21:47:52.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.025 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.025 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.025 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417672+0000", 2024-09-15T21:47:52.025 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002618015 2024-09-15T21:47:52.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.025 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.025 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.025 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422295+0000", 2024-09-15T21:47:52.026 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046233300000000001 2024-09-15T21:47:52.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.026 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.026 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.026 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422330+0000", 2024-09-15T21:47:52.026 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4476e-05 2024-09-15T21:47:52.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.026 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.026 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.026 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422351+0000", 2024-09-15T21:47:52.026 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0537e-05 2024-09-15T21:47:52.026 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.026 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.026 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.027 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.027 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:52.027 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:52.027 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 83.585628796999998, 2024-09-15T21:47:52.027 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0090873700000000009, 2024-09-15T21:47:52.027 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.027 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.027 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.027 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.027 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.027 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.028 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:52.028 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.028 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.028 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.028 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:52.028 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.028 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.028 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.028 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:47:52.028 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6100000000000002e-07 2024-09-15T21:47:52.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.028 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.029 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.029 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413371+0000", 2024-09-15T21:47:52.029 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.764e-06 2024-09-15T21:47:52.029 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.029 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.029 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.029 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413859+0000", 2024-09-15T21:47:52.029 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00048751900000000001 2024-09-15T21:47:52.029 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.029 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.029 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.029 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413864+0000", 2024-09-15T21:47:52.029 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6200000000000004e-06 2024-09-15T21:47:52.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.030 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.030 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.030 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415253+0000", 2024-09-15T21:47:52.030 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013884050000000001 2024-09-15T21:47:52.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.030 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.030 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.030 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415364+0000", 2024-09-15T21:47:52.030 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000111119 2024-09-15T21:47:52.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.030 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.030 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:52.031 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415453+0000", 2024-09-15T21:47:52.031 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8641999999999996e-05 2024-09-15T21:47:52.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.031 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.031 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.031 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417676+0000", 2024-09-15T21:47:52.031 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00222368 2024-09-15T21:47:52.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.031 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.031 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.031 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422395+0000", 2024-09-15T21:47:52.031 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047191150000000003 2024-09-15T21:47:52.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.032 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.032 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.032 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422434+0000", 2024-09-15T21:47:52.032 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8525999999999997e-05 2024-09-15T21:47:52.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.032 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.032 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.032 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422457+0000", 2024-09-15T21:47:52.032 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2719000000000001e-05 2024-09-15T21:47:52.032 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.032 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.032 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.033 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.033 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:52.033 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:52.033 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 83.585625774999997, 2024-09-15T21:47:52.033 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0092139749999999992, 2024-09-15T21:47:52.033 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.033 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.033 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.033 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.033 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.033 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.033 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:52.034 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.034 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.034 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:52.034 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.034 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.034 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:47:52.034 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3799999999999999e-07 2024-09-15T21:47:52.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.035 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.035 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413375+0000", 2024-09-15T21:47:52.035 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1959999999999998e-06 2024-09-15T21:47:52.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.035 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.035 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.035 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413876+0000", 2024-09-15T21:47:52.035 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00050134500000000005 2024-09-15T21:47:52.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.035 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.035 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.035 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413881+0000", 2024-09-15T21:47:52.036 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3580000000000002e-06 2024-09-15T21:47:52.036 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.036 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.036 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.036 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415669+0000", 2024-09-15T21:47:52.036 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0017875090000000001 2024-09-15T21:47:52.036 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.036 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.036 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.036 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415737+0000", 2024-09-15T21:47:52.036 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8386000000000006e-05 2024-09-15T21:47:52.036 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.036 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.036 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:52.037 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415857+0000", 2024-09-15T21:47:52.037 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119906 2024-09-15T21:47:52.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.037 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.037 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.037 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421848+0000", 2024-09-15T21:47:52.037 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059903880000000001 2024-09-15T21:47:52.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.037 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.037 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.037 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422513+0000", 2024-09-15T21:47:52.037 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00066575599999999999 2024-09-15T21:47:52.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.038 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.038 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.038 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422549+0000", 2024-09-15T21:47:52.038 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5719999999999997e-05 2024-09-15T21:47:52.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.038 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.038 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.038 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422586+0000", 2024-09-15T21:47:52.038 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7172999999999998e-05 2024-09-15T21:47:52.038 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.038 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.038 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.038 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.039 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:52.039 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:52.039 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 83.585619953000005, 2024-09-15T21:47:52.039 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.009324087, 2024-09-15T21:47:52.039 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.039 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.039 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.039 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.039 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.039 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.039 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:52.039 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.039 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.040 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.040 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:52.040 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.040 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.040 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.040 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.040 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:47:52.040 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8900000000000001e-07 2024-09-15T21:47:52.040 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.040 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.040 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.040 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413381+0000", 2024-09-15T21:47:52.041 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1959999999999998e-06 2024-09-15T21:47:52.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.041 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.041 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.041 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413895+0000", 2024-09-15T21:47:52.041 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051466000000000001 2024-09-15T21:47:52.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.041 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.041 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.041 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413900+0000", 2024-09-15T21:47:52.041 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6090000000000003e-06 2024-09-15T21:47:52.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.041 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.041 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.042 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416081+0000", 2024-09-15T21:47:52.042 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021816090000000002 2024-09-15T21:47:52.042 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.042 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.042 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.042 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416173+0000", 2024-09-15T21:47:52.042 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1334000000000003e-05 2024-09-15T21:47:52.042 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.042 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.042 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:52.042 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416264+0000", 2024-09-15T21:47:52.042 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0864000000000003e-05 2024-09-15T21:47:52.042 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.042 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.043 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.043 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421854+0000", 2024-09-15T21:47:52.043 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055902240000000004 2024-09-15T21:47:52.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.043 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.043 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.043 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422629+0000", 2024-09-15T21:47:52.043 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00077550700000000004 2024-09-15T21:47:52.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.043 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.043 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.043 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422678+0000", 2024-09-15T21:47:52.043 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8810999999999997e-05 2024-09-15T21:47:52.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.043 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.044 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.044 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422702+0000", 2024-09-15T21:47:52.044 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3983999999999999e-05 2024-09-15T21:47:52.044 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.044 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.044 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.044 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.044 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.044 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:52.044 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:52.044 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 83.585616470999994, 2024-09-15T21:47:52.044 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0094270580000000003, 2024-09-15T21:47:52.044 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.045 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.045 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.045 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.045 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.045 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.045 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:52.045 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.045 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.045 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.045 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.045 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:52.045 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.045 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.046 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.046 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.046 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:47:52.046 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7500000000000001e-07 2024-09-15T21:47:52.046 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.046 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.046 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.046 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413386+0000", 2024-09-15T21:47:52.046 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5619999999999997e-06 2024-09-15T21:47:52.046 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.046 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.046 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.046 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413918+0000", 2024-09-15T21:47:52.046 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00053162499999999998 2024-09-15T21:47:52.047 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.047 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.047 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.047 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413923+0000", 2024-09-15T21:47:52.047 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9280000000000001e-06 2024-09-15T21:47:52.047 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.047 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.047 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.047 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416492+0000", 2024-09-15T21:47:52.047 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0025687140000000002 2024-09-15T21:47:52.047 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.047 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.047 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.047 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416558+0000", 2024-09-15T21:47:52.047 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6699000000000006e-05 2024-09-15T21:47:52.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.048 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.048 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:52.048 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416691+0000", 2024-09-15T21:47:52.048 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013263199999999999 2024-09-15T21:47:52.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.048 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.048 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.048 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421879+0000", 2024-09-15T21:47:52.048 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051881089999999998 2024-09-15T21:47:52.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.048 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.049 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.049 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422742+0000", 2024-09-15T21:47:52.049 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000862845 2024-09-15T21:47:52.049 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.049 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.049 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.049 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422786+0000", 2024-09-15T21:47:52.049 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4332000000000002e-05 2024-09-15T21:47:52.049 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.049 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.049 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.049 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422809+0000", 2024-09-15T21:47:52.049 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2337e-05 2024-09-15T21:47:52.049 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.050 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.050 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.050 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.050 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.050 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:4 2.10 2:0f65c9b8:::60a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.050 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:52.050 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 77.543330302000001, 2024-09-15T21:47:52.050 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058710300000000002, 2024-09-15T21:47:52.050 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.050 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.051 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.051 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.051 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.051 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.051 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:52.051 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.051 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.051 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.051 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.051 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:52.051 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.051 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.051 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.051 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.052 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:47:52.052 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0800000000000005e-07 2024-09-15T21:47:52.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.052 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.052 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.052 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455672+0000", 2024-09-15T21:47:52.052 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3900000000000002e-06 2024-09-15T21:47:52.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.052 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.052 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.052 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456538+0000", 2024-09-15T21:47:52.052 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00086613100000000002 2024-09-15T21:47:52.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.052 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.053 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.053 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456545+0000", 2024-09-15T21:47:52.053 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4830000000000002e-06 2024-09-15T21:47:52.053 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.053 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.053 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.053 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456594+0000", 2024-09-15T21:47:52.053 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8429e-05 2024-09-15T21:47:52.053 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.053 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.053 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.053 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456675+0000", 2024-09-15T21:47:52.053 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1516999999999999e-05 2024-09-15T21:47:52.053 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.053 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.054 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:52.054 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456772+0000", 2024-09-15T21:47:52.054 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6523999999999999e-05 2024-09-15T21:47:52.054 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.054 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.054 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.054 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.460299+0000", 2024-09-15T21:47:52.054 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035274859999999998 2024-09-15T21:47:52.054 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.054 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.054 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.054 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461451+0000", 2024-09-15T21:47:52.054 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011518100000000001 2024-09-15T21:47:52.054 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.054 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.055 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.055 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461498+0000", 2024-09-15T21:47:52.055 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7406e-05 2024-09-15T21:47:52.055 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.055 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.055 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.055 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461539+0000", 2024-09-15T21:47:52.055 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0346000000000003e-05 2024-09-15T21:47:52.055 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.055 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.055 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.055 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.055 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.055 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:31 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.056 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:52.056 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 77.543325039999999, 2024-09-15T21:47:52.056 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0078158979999999999, 2024-09-15T21:47:52.056 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.056 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.056 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:52.056 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.056 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.056 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.056 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:52.056 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.056 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.056 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.056 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.056 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:52.057 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.057 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.057 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.057 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.057 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:47:52.057 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0499999999999999e-07 2024-09-15T21:47:52.057 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.057 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.057 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.057 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455675+0000", 2024-09-15T21:47:52.057 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.376e-06 2024-09-15T21:47:52.057 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.057 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.057 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.058 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456964+0000", 2024-09-15T21:47:52.058 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001289204 2024-09-15T21:47:52.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.058 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.058 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.058 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456970+0000", 2024-09-15T21:47:52.058 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2129999999999996e-06 2024-09-15T21:47:52.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.058 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.058 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.058 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457032+0000", 2024-09-15T21:47:52.058 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2209e-05 2024-09-15T21:47:52.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.058 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.059 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.059 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457134+0000", 2024-09-15T21:47:52.059 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010204000000000001 2024-09-15T21:47:52.059 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.059 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.059 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.059 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463489+0000", 2024-09-15T21:47:52.059 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0063545509999999999 2024-09-15T21:47:52.059 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.059 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.059 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.059 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.059 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.059 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.060 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:52.060 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 71.523565275999999, 2024-09-15T21:47:52.060 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057594079999999997, 2024-09-15T21:47:52.060 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.060 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.060 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.060 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.060 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.060 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.060 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:52.060 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.060 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.060 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.060 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.061 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:52.061 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.061 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.061 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.061 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.061 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:47:52.061 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1999999999999999e-07 2024-09-15T21:47:52.061 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.061 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.061 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.061 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475437+0000", 2024-09-15T21:47:52.061 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5429999999999998e-06 2024-09-15T21:47:52.062 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.062 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.062 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.062 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475686+0000", 2024-09-15T21:47:52.062 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000248908 2024-09-15T21:47:52.062 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.062 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.062 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.062 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475695+0000", 2024-09-15T21:47:52.062 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.9740000000000008e-06 2024-09-15T21:47:52.062 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.062 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.062 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.062 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476291+0000", 2024-09-15T21:47:52.063 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00059651500000000004 2024-09-15T21:47:52.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.063 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.063 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.063 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476366+0000", 2024-09-15T21:47:52.063 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4531999999999997e-05 2024-09-15T21:47:52.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.063 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.063 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:52.063 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476491+0000", 2024-09-15T21:47:52.063 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012527299999999999 2024-09-15T21:47:52.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.064 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.064 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.064 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480600+0000", 2024-09-15T21:47:52.064 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0041085870000000003 2024-09-15T21:47:52.064 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.064 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.064 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.064 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481089+0000", 2024-09-15T21:47:52.065 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000489037 2024-09-15T21:47:52.065 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.065 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.065 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.065 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481152+0000", 2024-09-15T21:47:52.065 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3136e-05 2024-09-15T21:47:52.065 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.065 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.065 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.065 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481192+0000", 2024-09-15T21:47:52.065 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0182999999999998e-05 2024-09-15T21:47:52.065 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.065 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.066 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.066 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.066 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.066 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:17 2.10 2:0e9da45e:::61b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.066 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:52.066 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 71.523559614999996, 2024-09-15T21:47:52.066 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058311830000000002, 2024-09-15T21:47:52.066 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.066 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.066 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.066 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.066 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.066 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.067 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:52.067 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.067 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.067 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.067 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.067 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:47:52.067 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.067 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.067 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.067 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.067 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475439+0000", 2024-09-15T21:47:52.067 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1300000000000001e-07 2024-09-15T21:47:52.067 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.067 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.067 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.068 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475441+0000", 2024-09-15T21:47:52.068 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.419e-06 2024-09-15T21:47:52.068 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.068 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.068 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.068 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:52.068 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000269187 2024-09-15T21:47:52.068 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.068 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.068 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.068 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475715+0000", 2024-09-15T21:47:52.068 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3660000000000002e-06 2024-09-15T21:47:52.068 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.068 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.069 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.069 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476671+0000", 2024-09-15T21:47:52.069 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000956507 2024-09-15T21:47:52.069 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.069 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.069 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.069 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477500+0000", 2024-09-15T21:47:52.069 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00082897400000000001 2024-09-15T21:47:52.069 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.069 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.069 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:52.069 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477625+0000", 2024-09-15T21:47:52.069 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012513100000000001 2024-09-15T21:47:52.069 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.069 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.070 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.070 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480520+0000", 2024-09-15T21:47:52.070 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002894715 2024-09-15T21:47:52.070 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.070 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.070 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.070 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481184+0000", 2024-09-15T21:47:52.070 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000663801 2024-09-15T21:47:52.070 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.070 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.070 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.070 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481232+0000", 2024-09-15T21:47:52.070 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8457e-05 2024-09-15T21:47:52.070 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.070 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.071 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.071 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481270+0000", 2024-09-15T21:47:52.071 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7313e-05 2024-09-15T21:47:52.071 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.071 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.071 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.071 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.071 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.071 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.071 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:52.071 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 71.523555739000003, 2024-09-15T21:47:52.071 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0075605530000000002, 2024-09-15T21:47:52.071 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.071 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.072 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.072 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.072 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.072 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.072 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:52.072 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.072 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.072 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.072 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.072 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:52.072 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.072 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.072 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.073 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.073 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:47:52.073 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.53e-07 2024-09-15T21:47:52.073 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.073 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.073 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.073 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475445+0000", 2024-09-15T21:47:52.073 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7360000000000001e-06 2024-09-15T21:47:52.073 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.073 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.073 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.073 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475735+0000", 2024-09-15T21:47:52.073 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028975799999999999 2024-09-15T21:47:52.074 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.074 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.074 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.074 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475740+0000", 2024-09-15T21:47:52.074 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8250000000000004e-06 2024-09-15T21:47:52.074 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.074 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.074 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.074 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478479+0000", 2024-09-15T21:47:52.074 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002739525 2024-09-15T21:47:52.074 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.074 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.074 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.074 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478556+0000", 2024-09-15T21:47:52.075 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6213999999999999e-05 2024-09-15T21:47:52.075 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.075 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.075 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:52.075 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478672+0000", 2024-09-15T21:47:52.075 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011605 2024-09-15T21:47:52.075 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.075 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.075 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.075 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481323+0000", 2024-09-15T21:47:52.075 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0026518819999999999 2024-09-15T21:47:52.075 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.075 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.075 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.076 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482918+0000", 2024-09-15T21:47:52.076 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001594105 2024-09-15T21:47:52.076 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.076 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.076 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.076 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482965+0000", 2024-09-15T21:47:52.076 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6950000000000003e-05 2024-09-15T21:47:52.076 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.076 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.076 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.076 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483003+0000", 2024-09-15T21:47:52.076 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8254999999999999e-05 2024-09-15T21:47:52.076 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.077 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.077 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.077 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.077 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.077 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.077 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:52.077 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 71.523551682999994, 2024-09-15T21:47:52.077 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076582890000000004, 2024-09-15T21:47:52.077 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.077 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.077 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.077 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.077 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.078 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.078 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:52.078 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.078 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.078 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.078 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.078 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:47:52.078 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.078 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.078 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.078 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.078 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475447+0000", 2024-09-15T21:47:52.078 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4699999999999998e-07 2024-09-15T21:47:52.079 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.079 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.079 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.079 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475449+0000", 2024-09-15T21:47:52.079 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.176e-06 2024-09-15T21:47:52.079 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.079 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.079 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.079 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475752+0000", 2024-09-15T21:47:52.079 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030374200000000002 2024-09-15T21:47:52.079 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.079 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.080 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.080 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475757+0000", 2024-09-15T21:47:52.080 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2209999999999999e-06 2024-09-15T21:47:52.080 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.080 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.080 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.080 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477441+0000", 2024-09-15T21:47:52.080 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016843489999999999 2024-09-15T21:47:52.080 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.080 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.080 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.080 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477567+0000", 2024-09-15T21:47:52.080 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012552099999999999 2024-09-15T21:47:52.081 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.081 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.081 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:52.081 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477712+0000", 2024-09-15T21:47:52.081 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000145427 2024-09-15T21:47:52.081 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.081 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.081 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.081 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482935+0000", 2024-09-15T21:47:52.081 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052231200000000004 2024-09-15T21:47:52.082 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.082 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.082 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.082 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483015+0000", 2024-09-15T21:47:52.082 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9634000000000004e-05 2024-09-15T21:47:52.082 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.082 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.082 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.082 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483065+0000", 2024-09-15T21:47:52.082 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9869e-05 2024-09-15T21:47:52.082 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.082 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.082 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.082 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483105+0000", 2024-09-15T21:47:52.083 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9983e-05 2024-09-15T21:47:52.083 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.083 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.083 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.083 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.083 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.083 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.083 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:52.083 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 71.523545142000003, 2024-09-15T21:47:52.083 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076459379999999997, 2024-09-15T21:47:52.083 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.083 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.083 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.084 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.084 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.084 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.084 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:52.084 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.084 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.084 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.084 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.084 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:52.084 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.084 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.084 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.084 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.085 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:47:52.085 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.58e-07 2024-09-15T21:47:52.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.085 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.085 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.085 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475455+0000", 2024-09-15T21:47:52.085 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.075e-06 2024-09-15T21:47:52.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.085 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.085 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.085 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475769+0000", 2024-09-15T21:47:52.085 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00031336999999999999 2024-09-15T21:47:52.086 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.086 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.086 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.086 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475773+0000", 2024-09-15T21:47:52.086 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8080000000000004e-06 2024-09-15T21:47:52.086 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.086 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.086 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.086 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478950+0000", 2024-09-15T21:47:52.086 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031763870000000001 2024-09-15T21:47:52.086 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.086 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.087 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.087 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479067+0000", 2024-09-15T21:47:52.087 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000117543 2024-09-15T21:47:52.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.087 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.087 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:52.087 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479196+0000", 2024-09-15T21:47:52.087 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000128987 2024-09-15T21:47:52.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.087 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.087 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.087 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481366+0000", 2024-09-15T21:47:52.087 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021699089999999998 2024-09-15T21:47:52.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.088 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.088 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.088 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483015+0000", 2024-09-15T21:47:52.088 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016486719999999999 2024-09-15T21:47:52.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.088 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.088 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.088 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483061+0000", 2024-09-15T21:47:52.088 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5729999999999998e-05 2024-09-15T21:47:52.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.088 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.088 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.089 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483099+0000", 2024-09-15T21:47:52.089 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8099000000000002e-05 2024-09-15T21:47:52.089 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.089 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.089 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.089 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.089 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.089 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.090 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:52.090 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 71.523498032999996, 2024-09-15T21:47:52.090 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077290859999999996, 2024-09-15T21:47:52.090 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.090 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.090 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.090 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.090 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.090 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.090 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:52.090 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.091 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.091 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.091 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:47:52.091 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.091 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.091 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.091 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475501+0000", 2024-09-15T21:47:52.091 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1900000000000002e-07 2024-09-15T21:47:52.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.091 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.092 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.092 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475504+0000", 2024-09-15T21:47:52.092 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.168e-06 2024-09-15T21:47:52.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.092 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.092 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.092 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477037+0000", 2024-09-15T21:47:52.092 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001533202 2024-09-15T21:47:52.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.092 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.092 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.092 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477050+0000", 2024-09-15T21:47:52.092 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3132999999999999e-05 2024-09-15T21:47:52.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.093 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.093 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.093 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479477+0000", 2024-09-15T21:47:52.093 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002426489 2024-09-15T21:47:52.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.093 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.093 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.093 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479553+0000", 2024-09-15T21:47:52.093 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6583999999999997e-05 2024-09-15T21:47:52.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.093 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.093 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:47:52.093 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479637+0000", 2024-09-15T21:47:52.094 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4172999999999995e-05 2024-09-15T21:47:52.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.094 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.094 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.094 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483108+0000", 2024-09-15T21:47:52.094 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003470356 2024-09-15T21:47:52.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.094 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.094 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.094 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483160+0000", 2024-09-15T21:47:52.094 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2507999999999999e-05 2024-09-15T21:47:52.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.094 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.094 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.094 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483202+0000", 2024-09-15T21:47:52.095 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1758999999999998e-05 2024-09-15T21:47:52.095 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.095 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.095 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.095 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483229+0000", 2024-09-15T21:47:52.095 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6995000000000002e-05 2024-09-15T21:47:52.095 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.095 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.095 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.095 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.095 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.095 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.095 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:52.095 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 71.523493075999994, 2024-09-15T21:47:52.095 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.55145717800000005, 2024-09-15T21:47:52.096 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.096 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.096 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:52.096 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.096 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.096 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.096 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:52.096 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.096 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.096 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.096 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.096 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:52.096 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.097 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.097 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.097 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.097 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:47:52.097 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.65e-07 2024-09-15T21:47:52.097 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.097 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.097 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.097 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475507+0000", 2024-09-15T21:47:52.097 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4699999999999999e-06 2024-09-15T21:47:52.097 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.097 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.097 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.097 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.025935+0000", 2024-09-15T21:47:52.098 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.55042854200000002 2024-09-15T21:47:52.098 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.098 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.098 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.098 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.025965+0000", 2024-09-15T21:47:52.098 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9995999999999999e-05 2024-09-15T21:47:52.098 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.098 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.098 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.098 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026016+0000", 2024-09-15T21:47:52.098 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1310000000000002e-05 2024-09-15T21:47:52.098 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.098 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.098 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.098 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026122+0000", 2024-09-15T21:47:52.099 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105605 2024-09-15T21:47:52.099 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.099 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.099 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.099 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026962+0000", 2024-09-15T21:47:52.099 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00083998999999999999 2024-09-15T21:47:52.099 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.099 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.099 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.099 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.099 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.099 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:5 2.e 2:75ee5fd8:::629.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.099 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:52.099 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 60.434286710999999, 2024-09-15T21:47:52.100 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0071955470000000001, 2024-09-15T21:47:52.100 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.100 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.100 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.100 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.100 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.100 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.100 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:52.100 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.100 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.100 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.100 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.100 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:47:52.100 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.101 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.101 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.101 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.101 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564713+0000", 2024-09-15T21:47:52.101 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.866e-06 2024-09-15T21:47:52.101 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.101 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.101 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.101 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564721+0000", 2024-09-15T21:47:52.101 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3299999999999999e-06 2024-09-15T21:47:52.101 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.101 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.101 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.101 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564762+0000", 2024-09-15T21:47:52.102 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0707999999999997e-05 2024-09-15T21:47:52.102 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.102 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.102 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.102 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564774+0000", 2024-09-15T21:47:52.102 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1883e-05 2024-09-15T21:47:52.102 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.102 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.102 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.102 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564877+0000", 2024-09-15T21:47:52.102 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010254700000000001 2024-09-15T21:47:52.102 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.102 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.102 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.102 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565066+0000", 2024-09-15T21:47:52.103 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018933700000000001 2024-09-15T21:47:52.103 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.103 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.103 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:52.103 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565164+0000", 2024-09-15T21:47:52.103 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7633999999999994e-05 2024-09-15T21:47:52.103 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.103 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.103 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.103 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569030+0000", 2024-09-15T21:47:52.103 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038669070000000002 2024-09-15T21:47:52.103 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.103 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.103 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.104 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571818+0000", 2024-09-15T21:47:52.104 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027878930000000001 2024-09-15T21:47:52.104 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.104 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.104 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.104 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571868+0000", 2024-09-15T21:47:52.104 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9880999999999999e-05 2024-09-15T21:47:52.104 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.104 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.104 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.104 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571907+0000", 2024-09-15T21:47:52.105 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8560999999999999e-05 2024-09-15T21:47:52.105 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.105 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.105 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.105 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.105 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.105 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:6 2.e 2:75ee5fd8:::629.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.105 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:52.105 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 60.434272583999999, 2024-09-15T21:47:52.105 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073433220000000002, 2024-09-15T21:47:52.105 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.105 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.105 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.106 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.106 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.106 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.106 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:52.106 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.106 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.106 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.106 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:47:52.106 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.106 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.106 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.107 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564726+0000", 2024-09-15T21:47:52.107 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7399999999999999e-07 2024-09-15T21:47:52.107 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.107 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.107 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.107 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564728+0000", 2024-09-15T21:47:52.107 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.447e-06 2024-09-15T21:47:52.107 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.107 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.107 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.107 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565446+0000", 2024-09-15T21:47:52.107 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00071774999999999996 2024-09-15T21:47:52.107 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.107 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.108 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.108 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565453+0000", 2024-09-15T21:47:52.108 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3239999999999997e-06 2024-09-15T21:47:52.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.108 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.108 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.108 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565515+0000", 2024-09-15T21:47:52.108 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1768999999999998e-05 2024-09-15T21:47:52.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.108 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.108 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.108 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565591+0000", 2024-09-15T21:47:52.108 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5873999999999999e-05 2024-09-15T21:47:52.109 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.109 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.109 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:52.109 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565709+0000", 2024-09-15T21:47:52.109 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011838899999999999 2024-09-15T21:47:52.109 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.109 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.109 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.109 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569039+0000", 2024-09-15T21:47:52.109 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0033292170000000002 2024-09-15T21:47:52.109 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.109 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.110 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.110 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571977+0000", 2024-09-15T21:47:52.110 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0029387829999999999 2024-09-15T21:47:52.110 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.110 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.110 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.110 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.572032+0000", 2024-09-15T21:47:52.110 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4540999999999998e-05 2024-09-15T21:47:52.110 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.110 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.110 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.110 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.572069+0000", 2024-09-15T21:47:52.110 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6854000000000003e-05 2024-09-15T21:47:52.110 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.110 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.111 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.111 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.111 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.111 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:2 2.1 2:82971f97:::404.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.111 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:52.111 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 60.434268617000001, 2024-09-15T21:47:52.111 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007245357, 2024-09-15T21:47:52.111 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.111 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.111 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.111 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.111 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.111 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.111 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:52.111 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.111 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.112 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.112 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.112 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:47:52.112 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.112 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.112 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.112 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.112 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564730+0000", 2024-09-15T21:47:52.112 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2999999999999999e-07 2024-09-15T21:47:52.112 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.112 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.112 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.112 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564731+0000", 2024-09-15T21:47:52.112 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.367e-06 2024-09-15T21:47:52.113 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.113 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.113 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.113 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565484+0000", 2024-09-15T21:47:52.113 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075311599999999996 2024-09-15T21:47:52.113 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.113 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.113 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.113 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565490+0000", 2024-09-15T21:47:52.113 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8100000000000003e-06 2024-09-15T21:47:52.113 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.113 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.113 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.113 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565828+0000", 2024-09-15T21:47:52.114 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000337836 2024-09-15T21:47:52.114 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.114 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.114 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.114 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565964+0000", 2024-09-15T21:47:52.114 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013641599999999999 2024-09-15T21:47:52.114 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.114 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.114 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:52.114 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566071+0000", 2024-09-15T21:47:52.114 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000107126 2024-09-15T21:47:52.114 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.114 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.115 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.115 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571361+0000", 2024-09-15T21:47:52.115 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052896210000000004 2024-09-15T21:47:52.115 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.115 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.115 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.115 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571874+0000", 2024-09-15T21:47:52.115 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051320599999999999 2024-09-15T21:47:52.115 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.115 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.115 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.115 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571936+0000", 2024-09-15T21:47:52.115 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1650999999999996e-05 2024-09-15T21:47:52.115 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.116 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.116 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.116 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571975+0000", 2024-09-15T21:47:52.116 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8977999999999998e-05 2024-09-15T21:47:52.116 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.116 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.116 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.116 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.116 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.116 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:52.117 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:52.009+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbad4208210 con 0x7fbad402d860 2024-09-15T21:47:52.118 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:47:52.118 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:52.011+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbad4208210 con 0x7fbad402d860 2024-09-15T21:47:52.119 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:52.120 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:52.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:52.012+0000 7fbadc331640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fbad4208930 0x7fbad420ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:52.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:52.012+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fbad420dc00 con 0x7fbad4208930 2024-09-15T21:47:52.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:52.012+0000 7fbada8ce640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fbad4208930 0x7fbad420ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:52.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:52.013+0000 7fbada8ce640 1 --2- 172.21.15.100:0/662332273 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fbad4208930 0x7fbad420ad20 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:52.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:52.013+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbad420dc00 con 0x7fbad4208930 2024-09-15T21:47:52.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:52.038+0000 7fbadc331640 1 -- 172.21.15.100:0/662332273 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fbad402d4e0 con 0x7fbad4208930 2024-09-15T21:47:52.120 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:52.041+0000 7fbacaffd640 1 -- 172.21.15.100:0/662332273 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fbad402d4e0 con 0x7fbad4208930 2024-09-15T21:47:52.121 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:52.121 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:47:52.121 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:52.122 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:47:52.122 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:52.116+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:52.219 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:52.217+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:52.268 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:52.268 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:52.268 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:52.269 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:52.277 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:47:52.320 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:52.318+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:52.421 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:52.419+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:52.432 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:52.432 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:52.432 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:52.432 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:52.433 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:52.442 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:47:52.522 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:52.520+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:52.551 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:52.551 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:52.551 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:52.551 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:52.551 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.551 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:52.551 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:47:52.551 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 84.133860534999997, 2024-09-15T21:47:52.551 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047486860000000002, 2024-09-15T21:47:52.552 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.552 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.552 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:52.552 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.552 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.552 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.552 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:47:52.552 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.552 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.552 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.552 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.552 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:47:52.552 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.552 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.553 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.553 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.553 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413816+0000", 2024-09-15T21:47:52.553 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4810000000000003e-06 2024-09-15T21:47:52.553 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.553 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.553 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.553 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413827+0000", 2024-09-15T21:47:52.553 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.187e-05 2024-09-15T21:47:52.553 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.553 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.553 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.553 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413862+0000", 2024-09-15T21:47:52.554 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4041000000000002e-05 2024-09-15T21:47:52.554 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.554 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.554 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.554 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413871+0000", 2024-09-15T21:47:52.554 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8919999999999995e-06 2024-09-15T21:47:52.554 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.554 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.554 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.554 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413997+0000", 2024-09-15T21:47:52.554 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012601299999999999 2024-09-15T21:47:52.554 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.554 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.554 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.554 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414100+0000", 2024-09-15T21:47:52.555 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102555 2024-09-15T21:47:52.555 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.555 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.555 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.555 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418558+0000", 2024-09-15T21:47:52.555 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044578339999999999 2024-09-15T21:47:52.555 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.555 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.555 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.555 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.555 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.555 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:52.555 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:47:52.555 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 84.130215720999999, 2024-09-15T21:47:52.556 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045423590000000002, 2024-09-15T21:47:52.556 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.556 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.556 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.556 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.556 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.556 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.556 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:47:52.556 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.556 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.557 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.557 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.557 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:47:52.557 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.557 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.557 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.557 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.557 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417455+0000", 2024-09-15T21:47:52.557 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9100000000000003e-07 2024-09-15T21:47:52.557 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.557 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.557 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.557 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417463+0000", 2024-09-15T21:47:52.558 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3750000000000003e-06 2024-09-15T21:47:52.558 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.558 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.558 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.558 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417475+0000", 2024-09-15T21:47:52.558 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2473000000000001e-05 2024-09-15T21:47:52.558 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.558 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.558 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.558 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417482+0000", 2024-09-15T21:47:52.558 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4010000000000002e-06 2024-09-15T21:47:52.558 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.558 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.559 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.559 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417524+0000", 2024-09-15T21:47:52.559 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1870999999999999e-05 2024-09-15T21:47:52.559 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.559 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.559 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.559 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417542+0000", 2024-09-15T21:47:52.559 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8224e-05 2024-09-15T21:47:52.559 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.559 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.559 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.560 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421955+0000", 2024-09-15T21:47:52.560 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044130740000000003 2024-09-15T21:47:52.560 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.560 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.560 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.560 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421996+0000", 2024-09-15T21:47:52.560 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.125e-05 2024-09-15T21:47:52.560 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.560 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.560 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.560 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.560 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.561 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 7 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:52.561 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:52.561 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 84.129954916000003, 2024-09-15T21:47:52.561 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043086519999999996, 2024-09-15T21:47:52.561 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.561 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.561 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:47:52.561 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.561 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.561 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.562 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:52.562 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.562 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.562 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.562 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.562 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:52.562 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.562 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.562 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.562 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.562 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:47:52.562 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.58e-07 2024-09-15T21:47:52.562 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.563 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.563 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.563 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417722+0000", 2024-09-15T21:47:52.563 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4729999999999997e-06 2024-09-15T21:47:52.563 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.563 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.563 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.563 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417737+0000", 2024-09-15T21:47:52.563 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5155e-05 2024-09-15T21:47:52.563 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.563 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.564 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.564 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417752+0000", 2024-09-15T21:47:52.564 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4863e-05 2024-09-15T21:47:52.564 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.564 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.564 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.564 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417819+0000", 2024-09-15T21:47:52.564 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7680000000000003e-05 2024-09-15T21:47:52.564 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.564 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.564 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.565 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422023+0000", 2024-09-15T21:47:52.565 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042040230000000003 2024-09-15T21:47:52.565 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.565 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.565 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.565 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.565 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.565 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:25 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.565 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:47:52.565 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 78.090465886000004, 2024-09-15T21:47:52.565 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070572830000000001, 2024-09-15T21:47:52.565 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.566 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.566 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.566 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.566 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.566 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.566 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:47:52.566 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.566 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.566 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.567 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.567 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:47:52.567 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.567 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.567 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.567 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.567 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457205+0000", 2024-09-15T21:47:52.567 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6899999999999999e-06 2024-09-15T21:47:52.567 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.568 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.568 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.568 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457211+0000", 2024-09-15T21:47:52.568 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.626e-06 2024-09-15T21:47:52.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.569 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.569 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.569 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457244+0000", 2024-09-15T21:47:52.569 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2691999999999998e-05 2024-09-15T21:47:52.569 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.569 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.569 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.569 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457251+0000", 2024-09-15T21:47:52.569 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3839999999999999e-06 2024-09-15T21:47:52.569 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.569 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.569 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.570 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457309+0000", 2024-09-15T21:47:52.570 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8023999999999999e-05 2024-09-15T21:47:52.570 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.570 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.570 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.570 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457413+0000", 2024-09-15T21:47:52.570 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010331999999999999 2024-09-15T21:47:52.570 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.570 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.570 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:52.570 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:52.570 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118439 2024-09-15T21:47:52.570 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.571 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.571 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.571 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463246+0000", 2024-09-15T21:47:52.571 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057148219999999996 2024-09-15T21:47:52.571 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.571 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.571 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.571 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463428+0000", 2024-09-15T21:47:52.571 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018218299999999999 2024-09-15T21:47:52.571 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.571 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.571 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.571 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464212+0000", 2024-09-15T21:47:52.571 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00078402799999999998 2024-09-15T21:47:52.572 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.572 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.572 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.572 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464261+0000", 2024-09-15T21:47:52.572 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9075e-05 2024-09-15T21:47:52.572 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.572 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.572 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.572 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.572 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.572 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:28 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.572 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:52.572 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 78.090455859000002, 2024-09-15T21:47:52.573 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0072141449999999999, 2024-09-15T21:47:52.573 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.573 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.573 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:52.573 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.573 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.573 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.573 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:52.573 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.573 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.573 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.573 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:52.573 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.574 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.574 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.574 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.574 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:47:52.574 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:47:52.574 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.574 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.574 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.574 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457217+0000", 2024-09-15T21:47:52.574 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7259999999999998e-06 2024-09-15T21:47:52.574 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.574 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.575 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.575 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457842+0000", 2024-09-15T21:47:52.575 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00062482599999999998 2024-09-15T21:47:52.575 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.575 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.575 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.575 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457848+0000", 2024-09-15T21:47:52.575 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7089999999999996e-06 2024-09-15T21:47:52.575 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.575 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.575 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.575 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457895+0000", 2024-09-15T21:47:52.575 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6317999999999999e-05 2024-09-15T21:47:52.575 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.576 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.576 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.576 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458013+0000", 2024-09-15T21:47:52.576 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011848100000000001 2024-09-15T21:47:52.576 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.576 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.576 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.576 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464428+0000", 2024-09-15T21:47:52.576 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0064146990000000003 2024-09-15T21:47:52.576 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.576 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.576 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.576 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.576 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.577 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:29 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.577 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:47:52.577 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 78.090451415999993, 2024-09-15T21:47:52.577 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073658980000000001, 2024-09-15T21:47:52.577 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.577 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.577 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:52.577 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.577 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.577 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.577 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:47:52.577 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.577 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.577 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.578 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.578 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:47:52.578 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.578 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.578 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.578 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457219+0000", 2024-09-15T21:47:52.578 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2000000000000001e-07 2024-09-15T21:47:52.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.578 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.578 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.578 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457226+0000", 2024-09-15T21:47:52.578 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.956e-06 2024-09-15T21:47:52.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.579 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.579 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.579 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458266+0000", 2024-09-15T21:47:52.579 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001040476 2024-09-15T21:47:52.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.579 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.579 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.579 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458273+0000", 2024-09-15T21:47:52.579 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0690000000000004e-06 2024-09-15T21:47:52.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.579 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.579 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.580 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458319+0000", 2024-09-15T21:47:52.580 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5668999999999999e-05 2024-09-15T21:47:52.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.580 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.580 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.580 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458415+0000", 2024-09-15T21:47:52.580 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6701999999999997e-05 2024-09-15T21:47:52.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.580 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.580 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.580 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464584+0000", 2024-09-15T21:47:52.580 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061687060000000004 2024-09-15T21:47:52.580 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.580 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.581 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.581 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.581 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.581 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.1 epoch 22/19 rep_tid 12 entries 22'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.10:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:52.581 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:47:52.581 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 78.089021410000001, 2024-09-15T21:47:52.581 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0039881550000000002, 2024-09-15T21:47:52.581 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.581 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.581 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:47:52.581 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.581 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.581 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.581 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:47:52.582 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.582 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.582 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.582 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:47:52.582 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.582 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.582 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.582 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458649+0000", 2024-09-15T21:47:52.582 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9599999999999999e-07 2024-09-15T21:47:52.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.582 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.583 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.583 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458656+0000", 2024-09-15T21:47:52.583 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2629999999999997e-06 2024-09-15T21:47:52.583 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.583 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.583 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.583 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458670+0000", 2024-09-15T21:47:52.583 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4166e-05 2024-09-15T21:47:52.583 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.583 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.583 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.583 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458682+0000", 2024-09-15T21:47:52.583 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1865000000000001e-05 2024-09-15T21:47:52.583 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.584 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.584 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.584 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458749+0000", 2024-09-15T21:47:52.584 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6507000000000005e-05 2024-09-15T21:47:52.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.584 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.584 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.584 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462636+0000", 2024-09-15T21:47:52.584 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038877579999999998 2024-09-15T21:47:52.584 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.584 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.584 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.584 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.585 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.1.10:12 2.12 e22/19 2:48cd7578:::60e.00000000:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:52.585 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:52.585 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 78.088707077999999, 2024-09-15T21:47:52.585 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042479500000000003, 2024-09-15T21:47:52.585 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.585 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.585 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.585 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.585 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.585 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.585 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:52.585 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.585 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.586 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.586 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:52.586 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.586 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.586 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.586 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:47:52.586 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2900000000000004e-07 2024-09-15T21:47:52.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.586 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.588 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.588 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458968+0000", 2024-09-15T21:47:52.588 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2039999999999996e-06 2024-09-15T21:47:52.588 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.588 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.588 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.588 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458977+0000", 2024-09-15T21:47:52.588 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.6179999999999995e-06 2024-09-15T21:47:52.588 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.588 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.589 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.589 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458983+0000", 2024-09-15T21:47:52.589 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9510000000000003e-06 2024-09-15T21:47:52.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.589 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.589 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.589 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459479+0000", 2024-09-15T21:47:52.589 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000496274 2024-09-15T21:47:52.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.589 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.589 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.589 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459496+0000", 2024-09-15T21:47:52.589 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6538e-05 2024-09-15T21:47:52.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.590 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.590 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.590 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463169+0000", 2024-09-15T21:47:52.590 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003672832 2024-09-15T21:47:52.590 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.590 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.590 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.590 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463211+0000", 2024-09-15T21:47:52.590 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2004e-05 2024-09-15T21:47:52.590 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.590 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.590 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.590 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.590 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.591 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.12 epoch 22/19 rep_tid 2 entries 22'3 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.10:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:52.591 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:47:52.591 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 78.088027523999997, 2024-09-15T21:47:52.591 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035859609999999999, 2024-09-15T21:47:52.591 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.591 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.591 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:47:52.591 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.591 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.591 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.591 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:47:52.591 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.592 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.592 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.592 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:47:52.592 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.592 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.592 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.592 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459643+0000", 2024-09-15T21:47:52.592 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2200000000000004e-07 2024-09-15T21:47:52.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.593 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.593 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.593 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459650+0000", 2024-09-15T21:47:52.593 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4479999999999997e-06 2024-09-15T21:47:52.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.593 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.593 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.593 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459665+0000", 2024-09-15T21:47:52.593 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4586999999999999e-05 2024-09-15T21:47:52.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.593 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.593 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.593 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459676+0000", 2024-09-15T21:47:52.593 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1673e-05 2024-09-15T21:47:52.594 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.594 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.594 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.594 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459732+0000", 2024-09-15T21:47:52.594 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5612999999999998e-05 2024-09-15T21:47:52.594 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.594 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.594 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.594 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463228+0000", 2024-09-15T21:47:52.594 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0034961179999999999 2024-09-15T21:47:52.594 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.595 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.595 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.595 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.595 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:34 2.1b 2:db457a8a:::201.00000001:head [write 0~588] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.595 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:47:52.595 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 78.082815887999999, 2024-09-15T21:47:52.595 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070946129999999996, 2024-09-15T21:47:52.595 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.595 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.595 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.595 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.595 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.595 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.595 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:47:52.596 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.596 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.596 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.596 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:47:52.596 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.596 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.596 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.596 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464855+0000", 2024-09-15T21:47:52.596 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5880000000000001e-06 2024-09-15T21:47:52.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.597 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.597 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.597 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464863+0000", 2024-09-15T21:47:52.597 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0879999999999995e-06 2024-09-15T21:47:52.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.597 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.597 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.597 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464889+0000", 2024-09-15T21:47:52.598 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5729e-05 2024-09-15T21:47:52.598 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.598 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.598 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.598 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464898+0000", 2024-09-15T21:47:52.598 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0389999999999993e-06 2024-09-15T21:47:52.598 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.598 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.598 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.598 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464962+0000", 2024-09-15T21:47:52.598 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3567999999999999e-05 2024-09-15T21:47:52.599 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.599 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.599 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.599 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.465041+0000", 2024-09-15T21:47:52.599 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8889999999999999e-05 2024-09-15T21:47:52.599 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.599 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.599 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:52.599 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.465173+0000", 2024-09-15T21:47:52.599 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013226899999999999 2024-09-15T21:47:52.599 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.600 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.600 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.600 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.466319+0000", 2024-09-15T21:47:52.600 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011463039999999999 2024-09-15T21:47:52.600 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.600 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.600 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.600 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471825+0000", 2024-09-15T21:47:52.600 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055062160000000004 2024-09-15T21:47:52.600 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.600 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.601 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.601 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471886+0000", 2024-09-15T21:47:52.601 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0503e-05 2024-09-15T21:47:52.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.601 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.601 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.601 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471948+0000", 2024-09-15T21:47:52.601 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2419e-05 2024-09-15T21:47:52.601 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.601 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.602 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.602 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.602 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.602 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.602 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:47:52.602 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 72.072157723999993, 2024-09-15T21:47:52.602 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047360500000000003, 2024-09-15T21:47:52.603 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.603 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.603 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.603 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.603 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.603 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.603 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:47:52.603 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.603 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.603 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.604 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.604 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:47:52.604 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.604 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.604 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.604 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475514+0000", 2024-09-15T21:47:52.604 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0990000000000001e-06 2024-09-15T21:47:52.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.604 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.605 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.605 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475524+0000", 2024-09-15T21:47:52.605 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5850000000000006e-06 2024-09-15T21:47:52.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.605 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.605 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.605 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475557+0000", 2024-09-15T21:47:52.606 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3090000000000003e-05 2024-09-15T21:47:52.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.606 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.606 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.606 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475568+0000", 2024-09-15T21:47:52.606 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1116e-05 2024-09-15T21:47:52.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.607 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.607 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.607 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475649+0000", 2024-09-15T21:47:52.607 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0815999999999993e-05 2024-09-15T21:47:52.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.607 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.607 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.607 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475805+0000", 2024-09-15T21:47:52.607 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015667200000000001 2024-09-15T21:47:52.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.608 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.608 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:52.608 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475944+0000", 2024-09-15T21:47:52.608 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000138651 2024-09-15T21:47:52.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.608 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.608 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.608 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476900+0000", 2024-09-15T21:47:52.608 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00095588699999999999 2024-09-15T21:47:52.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.608 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.608 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.608 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480153+0000", 2024-09-15T21:47:52.608 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0032535200000000002 2024-09-15T21:47:52.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.609 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.609 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.609 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480203+0000", 2024-09-15T21:47:52.609 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9932999999999998e-05 2024-09-15T21:47:52.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.609 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.609 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.609 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480248+0000", 2024-09-15T21:47:52.609 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4681000000000001e-05 2024-09-15T21:47:52.609 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.609 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.609 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.610 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.610 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.610 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:52.610 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 72.072141783000006, 2024-09-15T21:47:52.610 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049206709999999997, 2024-09-15T21:47:52.610 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.610 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.610 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.610 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.610 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.610 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.610 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:52.610 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.610 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.611 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.611 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.611 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:52.611 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.611 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.611 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.611 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.611 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:47:52.611 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1699999999999999e-07 2024-09-15T21:47:52.611 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.611 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.611 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.611 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475531+0000", 2024-09-15T21:47:52.611 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.05e-06 2024-09-15T21:47:52.612 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.612 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.612 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.612 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475743+0000", 2024-09-15T21:47:52.612 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021138099999999999 2024-09-15T21:47:52.612 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.612 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.612 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.612 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475751+0000", 2024-09-15T21:47:52.612 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7770000000000002e-06 2024-09-15T21:47:52.612 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.612 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.612 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.612 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476272+0000", 2024-09-15T21:47:52.613 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052158399999999996 2024-09-15T21:47:52.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.613 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.613 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.613 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476348+0000", 2024-09-15T21:47:52.613 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5963000000000004e-05 2024-09-15T21:47:52.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.613 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.613 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:52.613 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476474+0000", 2024-09-15T21:47:52.613 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012638800000000001 2024-09-15T21:47:52.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.613 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.614 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.614 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478194+0000", 2024-09-15T21:47:52.614 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001719768 2024-09-15T21:47:52.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.614 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.614 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.614 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480358+0000", 2024-09-15T21:47:52.614 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021637340000000001 2024-09-15T21:47:52.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.614 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.614 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.614 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480411+0000", 2024-09-15T21:47:52.614 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2720999999999999e-05 2024-09-15T21:47:52.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.615 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.615 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.615 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480449+0000", 2024-09-15T21:47:52.615 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7888e-05 2024-09-15T21:47:52.615 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.615 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.615 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.615 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.615 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.615 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.615 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:52.616 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 72.072136994000005, 2024-09-15T21:47:52.616 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005215478, 2024-09-15T21:47:52.616 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.616 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.616 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:52.616 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.616 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.616 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.616 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:52.616 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.616 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.616 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.617 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.617 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:52.617 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.617 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.617 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.617 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.617 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:47:52.617 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5199999999999998e-07 2024-09-15T21:47:52.617 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.617 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.617 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.617 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475535+0000", 2024-09-15T21:47:52.617 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9520000000000001e-06 2024-09-15T21:47:52.617 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.618 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.618 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.618 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475785+0000", 2024-09-15T21:47:52.618 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024966099999999998 2024-09-15T21:47:52.618 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.618 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.618 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.618 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475799+0000", 2024-09-15T21:47:52.618 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4649e-05 2024-09-15T21:47:52.618 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.618 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.618 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.618 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475856+0000", 2024-09-15T21:47:52.619 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6956000000000002e-05 2024-09-15T21:47:52.619 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.619 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.619 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.619 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476016+0000", 2024-09-15T21:47:52.619 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000159457 2024-09-15T21:47:52.619 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.619 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.619 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.619 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480748+0000", 2024-09-15T21:47:52.619 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047324510000000004 2024-09-15T21:47:52.619 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.620 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.620 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.620 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.620 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.620 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.2.12:1 2.1 e22/19 2:85bbe569:::202.00000000:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:52.620 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:47:52.620 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 72.069334522999995, 2024-09-15T21:47:52.620 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045423649999999996, 2024-09-15T21:47:52.620 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.620 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.620 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.621 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.621 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.621 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.621 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:47:52.621 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.621 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.621 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.621 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:47:52.621 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.621 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.621 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.622 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478336+0000", 2024-09-15T21:47:52.622 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4499999999999996e-07 2024-09-15T21:47:52.622 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.622 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.622 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.622 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478342+0000", 2024-09-15T21:47:52.622 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5579999999999997e-06 2024-09-15T21:47:52.622 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.622 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.622 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.622 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478356+0000", 2024-09-15T21:47:52.622 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3567000000000001e-05 2024-09-15T21:47:52.622 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.622 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.622 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.623 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478364+0000", 2024-09-15T21:47:52.623 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7889999999999995e-06 2024-09-15T21:47:52.623 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.623 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.623 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.623 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478416+0000", 2024-09-15T21:47:52.623 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2188000000000002e-05 2024-09-15T21:47:52.623 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.623 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.623 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.623 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478433+0000", 2024-09-15T21:47:52.623 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6674e-05 2024-09-15T21:47:52.623 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.623 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.624 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.624 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482832+0000", 2024-09-15T21:47:52.624 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043996110000000003 2024-09-15T21:47:52.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.624 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.624 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.624 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482878+0000", 2024-09-15T21:47:52.624 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5232999999999999e-05 2024-09-15T21:47:52.624 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.624 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.624 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.624 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.625 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:25 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:52.625 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:47:52.625 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 60.982474484000001, 2024-09-15T21:47:52.625 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058459619999999997, 2024-09-15T21:47:52.625 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.625 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.625 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.625 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.625 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.625 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.625 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:47:52.626 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.626 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.626 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.626 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.626 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:47:52.626 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.626 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.626 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.626 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.626 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565197+0000", 2024-09-15T21:47:52.626 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7889999999999999e-06 2024-09-15T21:47:52.626 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.626 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.626 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.627 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565208+0000", 2024-09-15T21:47:52.627 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1358999999999999e-05 2024-09-15T21:47:52.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.627 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.627 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.627 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565237+0000", 2024-09-15T21:47:52.627 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8626e-05 2024-09-15T21:47:52.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.627 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.627 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.627 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565249+0000", 2024-09-15T21:47:52.627 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1897e-05 2024-09-15T21:47:52.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.627 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.628 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.628 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565332+0000", 2024-09-15T21:47:52.628 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2943999999999997e-05 2024-09-15T21:47:52.628 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.628 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.628 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.628 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565452+0000", 2024-09-15T21:47:52.628 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119992 2024-09-15T21:47:52.628 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.629 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.629 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:52.629 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565610+0000", 2024-09-15T21:47:52.629 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015845099999999999 2024-09-15T21:47:52.629 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.629 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.629 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:52.629 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.568999+0000", 2024-09-15T21:47:52.629 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0033887190000000001 2024-09-15T21:47:52.629 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.629 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.629 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:52.629 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.570979+0000", 2024-09-15T21:47:52.630 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019804919999999999 2024-09-15T21:47:52.630 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.630 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.630 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.630 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571014+0000", 2024-09-15T21:47:52.630 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4331e-05 2024-09-15T21:47:52.630 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.630 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.630 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.631 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571041+0000", 2024-09-15T21:47:52.631 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7362e-05 2024-09-15T21:47:52.631 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.631 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.631 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.631 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.631 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.631 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:5 2.e e22/19 2:75ee5fd8:::629.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:47:52.631 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:47:52.631 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 60.981798838000003, 2024-09-15T21:47:52.631 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057947119999999996, 2024-09-15T21:47:52.631 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.631 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.632 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.632 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.632 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.632 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.632 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:47:52.632 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.632 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.632 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.632 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.633 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:47:52.633 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.633 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.633 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.633 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565872+0000", 2024-09-15T21:47:52.633 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9800000000000002e-07 2024-09-15T21:47:52.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.633 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.633 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.633 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565880+0000", 2024-09-15T21:47:52.633 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7619999999999995e-06 2024-09-15T21:47:52.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.634 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.634 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.634 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565890+0000", 2024-09-15T21:47:52.634 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0648999999999999e-05 2024-09-15T21:47:52.634 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.634 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.634 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.634 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565897+0000", 2024-09-15T21:47:52.634 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.153e-06 2024-09-15T21:47:52.634 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.635 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.635 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.635 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565966+0000", 2024-09-15T21:47:52.635 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9046000000000003e-05 2024-09-15T21:47:52.635 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.635 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.635 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.635 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565985+0000", 2024-09-15T21:47:52.635 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8695e-05 2024-09-15T21:47:52.635 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.635 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.635 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.636 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571625+0000", 2024-09-15T21:47:52.636 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056397139999999997 2024-09-15T21:47:52.636 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.636 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.636 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.636 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571666+0000", 2024-09-15T21:47:52.636 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0695000000000003e-05 2024-09-15T21:47:52.636 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.636 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.636 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.636 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.636 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.637 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:6 2.e e22/19 2:75ee5fd8:::629.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:47:52.637 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:47:52.637 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 60.981736589999997, 2024-09-15T21:47:52.637 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057607429999999996, 2024-09-15T21:47:52.637 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.637 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.637 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.637 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.637 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.637 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.638 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:47:52.638 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.638 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.638 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.638 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.638 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:47:52.638 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.638 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.638 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.638 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.639 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565934+0000", 2024-09-15T21:47:52.639 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6199999999999998e-07 2024-09-15T21:47:52.639 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.639 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.639 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.639 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565938+0000", 2024-09-15T21:47:52.639 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4970000000000003e-06 2024-09-15T21:47:52.639 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.639 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.639 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.640 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565945+0000", 2024-09-15T21:47:52.640 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5069999999999997e-06 2024-09-15T21:47:52.640 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.640 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.640 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.640 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565948+0000", 2024-09-15T21:47:52.640 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:47:52.640 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.640 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.640 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.641 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566200+0000", 2024-09-15T21:47:52.641 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025170200000000002 2024-09-15T21:47:52.641 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.641 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.641 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.641 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566218+0000", 2024-09-15T21:47:52.641 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7543000000000001e-05 2024-09-15T21:47:52.641 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.641 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.641 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.641 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571668+0000", 2024-09-15T21:47:52.641 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054502999999999999 2024-09-15T21:47:52.641 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.641 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.642 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.642 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571694+0000", 2024-09-15T21:47:52.642 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5979e-05 2024-09-15T21:47:52.642 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.642 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.642 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.642 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.642 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.642 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:2 2.1 e22/19 2:82971f97:::404.00000000:head v 22'5, mlcod=22'3)", 2024-09-15T21:47:52.642 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:47:52.642 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 60.981528654000002, 2024-09-15T21:47:52.642 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056016019999999998, 2024-09-15T21:47:52.642 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.642 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.643 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.643 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.643 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.643 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.643 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:47:52.643 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.643 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.643 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.643 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.643 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:47:52.643 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.643 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.643 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.643 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.644 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566142+0000", 2024-09-15T21:47:52.644 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:47:52.644 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.644 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.644 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.644 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566149+0000", 2024-09-15T21:47:52.644 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7510000000000008e-06 2024-09-15T21:47:52.644 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.644 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.644 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.644 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566161+0000", 2024-09-15T21:47:52.644 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1259e-05 2024-09-15T21:47:52.644 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.645 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.645 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.645 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566168+0000", 2024-09-15T21:47:52.645 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.255e-06 2024-09-15T21:47:52.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.645 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.645 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.645 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566227+0000", 2024-09-15T21:47:52.645 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8961e-05 2024-09-15T21:47:52.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.645 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.645 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.645 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566247+0000", 2024-09-15T21:47:52.646 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0536000000000001e-05 2024-09-15T21:47:52.646 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.646 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.646 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.646 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571694+0000", 2024-09-15T21:47:52.646 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005446762 2024-09-15T21:47:52.646 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.646 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.646 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.646 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571743+0000", 2024-09-15T21:47:52.646 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8396999999999998e-05 2024-09-15T21:47:52.646 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.646 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.647 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.647 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.647 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.647 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:17 2.14 e22/19 2:2eebc8f4:::62f.00000000:head v 22'9, mlcod=22'7)", 2024-09-15T21:47:52.647 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:47:52.647 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 60.981512557000002, 2024-09-15T21:47:52.647 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056532860000000004, 2024-09-15T21:47:52.647 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.647 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.647 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.647 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.647 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.647 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.648 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:47:52.648 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.648 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.648 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.648 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.648 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:47:52.648 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.648 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.648 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.648 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.648 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566158+0000", 2024-09-15T21:47:52.648 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.92e-07 2024-09-15T21:47:52.648 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.649 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.649 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.649 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566172+0000", 2024-09-15T21:47:52.649 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3976e-05 2024-09-15T21:47:52.649 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.649 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.649 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.649 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566407+0000", 2024-09-15T21:47:52.649 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023483900000000001 2024-09-15T21:47:52.649 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.649 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.649 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.649 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566416+0000", 2024-09-15T21:47:52.649 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0780000000000002e-06 2024-09-15T21:47:52.650 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.650 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.650 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.650 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566476+0000", 2024-09-15T21:47:52.650 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9908999999999998e-05 2024-09-15T21:47:52.650 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.650 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.650 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.650 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566493+0000", 2024-09-15T21:47:52.650 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7567e-05 2024-09-15T21:47:52.650 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.650 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.650 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.651 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571762+0000", 2024-09-15T21:47:52.651 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052682959999999996 2024-09-15T21:47:52.651 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.651 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.651 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.651 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571810+0000", 2024-09-15T21:47:52.651 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8829000000000003e-05 2024-09-15T21:47:52.651 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.651 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.651 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.651 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.651 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.651 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:18 2.14 e22/19 2:2eebc8f4:::62f.00000000:head v 22'10, mlcod=22'7)", 2024-09-15T21:47:52.651 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:47:52.652 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 60.981223481999997, 2024-09-15T21:47:52.652 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053992579999999997, 2024-09-15T21:47:52.652 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:52.652 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:52.652 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:52.652 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:52.652 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.652 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:52.652 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:47:52.652 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.652 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.652 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.652 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:52.652 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:47:52.653 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:52.653 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.653 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.653 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:52.653 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566447+0000", 2024-09-15T21:47:52.653 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2e-07 2024-09-15T21:47:52.653 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.653 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.653 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:52.653 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566452+0000", 2024-09-15T21:47:52.653 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0390000000000004e-06 2024-09-15T21:47:52.653 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.653 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.654 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:52.654 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566460+0000", 2024-09-15T21:47:52.654 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1389999999999995e-06 2024-09-15T21:47:52.654 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.654 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.654 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:52.654 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566473+0000", 2024-09-15T21:47:52.654 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3093e-05 2024-09-15T21:47:52.654 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.654 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.654 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:52.654 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566688+0000", 2024-09-15T21:47:52.654 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021537100000000001 2024-09-15T21:47:52.654 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.655 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:52.655 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566706+0000", 2024-09-15T21:47:52.655 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7802999999999999e-05 2024-09-15T21:47:52.655 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.655 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:52.655 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571813+0000", 2024-09-15T21:47:52.655 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051065140000000004 2024-09-15T21:47:52.655 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:52.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:52.655 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:52.655 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571845+0000", 2024-09-15T21:47:52.655 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2879000000000001e-05 2024-09-15T21:47:52.656 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.656 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.656 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.656 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:52.656 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:52.656 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:52.657 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:47:52.658 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:52.655+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:52.758 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:52.756+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:52.779 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:52.779 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:52.779 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:52.779 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:52.789 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:47:52.852 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-15T21:47:52.852 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:47:52.853 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-15T21:47:52.853 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-15T21:47:52.853 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-15T21:47:52.853 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-15T21:47:52.859 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:52.858+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:52.899 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:52.899 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:52.899 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:52.899 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:52.899 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:52.908 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:47:52.958 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:52.959 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-15T21:47:52.959 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:52.960 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:52.958+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:53.053 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:53.053 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:53.053 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:53.053 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:53.053 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.054 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:53.054 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:47:53.054 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.637734222999995, 2024-09-15T21:47:53.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061533810000000003, 2024-09-15T21:47:53.054 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.054 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.054 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.054 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.054 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.054 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:47:53.054 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.054 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.054 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:47:53.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413085+0000", 2024-09-15T21:47:53.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8390000000000003e-06 2024-09-15T21:47:53.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.055 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.055 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.055 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413092+0000", 2024-09-15T21:47:53.055 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3479999999999998e-06 2024-09-15T21:47:53.055 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413109+0000", 2024-09-15T21:47:53.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7192000000000001e-05 2024-09-15T21:47:53.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413118+0000", 2024-09-15T21:47:53.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.935e-06 2024-09-15T21:47:53.056 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.056 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.056 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.056 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413219+0000", 2024-09-15T21:47:53.056 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010121 2024-09-15T21:47:53.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413381+0000", 2024-09-15T21:47:53.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016148399999999999 2024-09-15T21:47:53.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:53.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413525+0000", 2024-09-15T21:47:53.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014428100000000001 2024-09-15T21:47:53.057 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.057 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.057 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.057 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415866+0000", 2024-09-15T21:47:53.057 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023409300000000002 2024-09-15T21:47:53.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419156+0000", 2024-09-15T21:47:53.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032903580000000002 2024-09-15T21:47:53.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419199+0000", 2024-09-15T21:47:53.058 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2171e-05 2024-09-15T21:47:53.058 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.058 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.058 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.058 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419232+0000", 2024-09-15T21:47:53.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3633000000000001e-05 2024-09-15T21:47:53.059 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.059 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.059 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.059 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.059 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.059 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.4:28 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-15T21:47:53.059 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:47:53.059 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 84.636470713999998, 2024-09-15T21:47:53.059 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002614594, 2024-09-15T21:47:53.059 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.060 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.060 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.060 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:47:53.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.060 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.060 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.060 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:47:53.060 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.060 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414344+0000", 2024-09-15T21:47:53.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6589999999999999e-06 2024-09-15T21:47:53.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414355+0000", 2024-09-15T21:47:53.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0604999999999999e-05 2024-09-15T21:47:53.061 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.061 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.061 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.061 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414370+0000", 2024-09-15T21:47:53.061 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-15T21:47:53.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414379+0000", 2024-09-15T21:47:53.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2449999999999998e-06 2024-09-15T21:47:53.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.062 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414477+0000", 2024-09-15T21:47:53.062 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8968000000000005e-05 2024-09-15T21:47:53.062 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.062 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.062 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414504+0000", 2024-09-15T21:47:53.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6899000000000001e-05 2024-09-15T21:47:53.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416837+0000", 2024-09-15T21:47:53.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023327130000000001 2024-09-15T21:47:53.063 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.063 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.063 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.063 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416957+0000", 2024-09-15T21:47:53.063 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000119796 2024-09-15T21:47:53.063 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.063 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.063 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.064 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.064 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.1.10:8 2.b e22/19 2:d70a01c6:::60c.00000000:head v 22'4, mlcod=22'1)", 2024-09-15T21:47:53.064 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:47:53.064 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 78.592891261999995, 2024-09-15T21:47:53.064 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027633890000000002, 2024-09-15T21:47:53.064 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.064 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.064 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.064 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.064 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.064 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.064 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:47:53.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:47:53.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.065 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457923+0000", 2024-09-15T21:47:53.065 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.136e-06 2024-09-15T21:47:53.065 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.065 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.065 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457931+0000", 2024-09-15T21:47:53.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1529999999999997e-06 2024-09-15T21:47:53.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457946+0000", 2024-09-15T21:47:53.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4482e-05 2024-09-15T21:47:53.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.066 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.066 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457956+0000", 2024-09-15T21:47:53.066 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0774999999999999e-05 2024-09-15T21:47:53.066 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.066 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458016+0000", 2024-09-15T21:47:53.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9247999999999999e-05 2024-09-15T21:47:53.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458036+0000", 2024-09-15T21:47:53.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0920999999999998e-05 2024-09-15T21:47:53.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.067 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.067 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.067 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460636+0000", 2024-09-15T21:47:53.067 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025992089999999999 2024-09-15T21:47:53.067 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.068 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.068 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460685+0000", 2024-09-15T21:47:53.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9465000000000002e-05 2024-09-15T21:47:53.068 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.068 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.068 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.068 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.068 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.068 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.18 epoch 22/19 rep_tid 5 entries 22'1 (0'0) error 2:1d00e16c:::201.00000005:head by mds.1.10:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:53.068 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:47:53.068 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 78.591199130999996, 2024-09-15T21:47:53.068 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003509599, 2024-09-15T21:47:53.068 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.069 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.069 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:53.069 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:47:53.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.069 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.069 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:47:53.069 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.069 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.069 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459615+0000", 2024-09-15T21:47:53.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4200000000000005e-07 2024-09-15T21:47:53.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459626+0000", 2024-09-15T21:47:53.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1456e-05 2024-09-15T21:47:53.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.070 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.070 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459650+0000", 2024-09-15T21:47:53.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.429e-05 2024-09-15T21:47:53.070 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459664+0000", 2024-09-15T21:47:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3805000000000001e-05 2024-09-15T21:47:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459784+0000", 2024-09-15T21:47:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000119614 2024-09-15T21:47:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463123+0000", 2024-09-15T21:47:53.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033395920000000002 2024-09-15T21:47:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:7 2.13 e22/19 2:cdf3ac81:::616.00000000:head v 22'5, mlcod=22'3)", 2024-09-15T21:47:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:47:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 72.573418352999994, 2024-09-15T21:47:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002138615, 2024-09-15T21:47:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:47:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:47:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477396+0000", 2024-09-15T21:47:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4190000000000001e-06 2024-09-15T21:47:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477405+0000", 2024-09-15T21:47:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2320000000000007e-06 2024-09-15T21:47:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477419+0000", 2024-09-15T21:47:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3589e-05 2024-09-15T21:47:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477428+0000", 2024-09-15T21:47:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4779999999999997e-06 2024-09-15T21:47:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477493+0000", 2024-09-15T21:47:53.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4546000000000002e-05 2024-09-15T21:47:53.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477511+0000", 2024-09-15T21:47:53.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7793000000000001e-05 2024-09-15T21:47:53.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479481+0000", 2024-09-15T21:47:53.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019701660000000002 2024-09-15T21:47:53.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479533+0000", 2024-09-15T21:47:53.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2392000000000003e-05 2024-09-15T21:47:53.076 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.076 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.076 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.076 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:26 2.11 e22/19 2:893cd530:::mds2_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-15T21:47:53.076 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:53.076 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 72.573271426999995, 2024-09-15T21:47:53.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020450450000000001, 2024-09-15T21:47:53.076 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.077 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.077 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.077 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:53.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:53.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:47:53.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9699999999999995e-07 2024-09-15T21:47:53.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477555+0000", 2024-09-15T21:47:53.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3144e-05 2024-09-15T21:47:53.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477565+0000", 2024-09-15T21:47:53.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8579999999999995e-06 2024-09-15T21:47:53.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477573+0000", 2024-09-15T21:47:53.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4869999999999996e-06 2024-09-15T21:47:53.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477632+0000", 2024-09-15T21:47:53.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9178999999999997e-05 2024-09-15T21:47:53.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477654+0000", 2024-09-15T21:47:53.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2234000000000001e-05 2024-09-15T21:47:53.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479526+0000", 2024-09-15T21:47:53.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018717600000000001 2024-09-15T21:47:53.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479587+0000", 2024-09-15T21:47:53.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0686000000000001e-05 2024-09-15T21:47:53.080 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.080 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.080 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.080 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:22 2.1a e22/19 2:5fd68f60:::61d.00000000:head v 22'3, mlcod=0'0)", 2024-09-15T21:47:53.081 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:53.081 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 72.572911371999993, 2024-09-15T21:47:53.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023694380000000002, 2024-09-15T21:47:53.081 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.081 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.081 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.081 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:53.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:53.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:47:53.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4899999999999995e-07 2024-09-15T21:47:53.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477910+0000", 2024-09-15T21:47:53.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:47:53.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477920+0000", 2024-09-15T21:47:53.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9250000000000007e-06 2024-09-15T21:47:53.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.083 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.083 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477927+0000", 2024-09-15T21:47:53.083 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0940000000000002e-06 2024-09-15T21:47:53.083 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.083 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477985+0000", 2024-09-15T21:47:53.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8451e-05 2024-09-15T21:47:53.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478005+0000", 2024-09-15T21:47:53.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0064e-05 2024-09-15T21:47:53.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480223+0000", 2024-09-15T21:47:53.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022177910000000002 2024-09-15T21:47:53.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480271+0000", 2024-09-15T21:47:53.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8164999999999997e-05 2024-09-15T21:47:53.085 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.085 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.085 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.085 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:8 2.13 e22/19 2:cdf3ac81:::616.00000000:head v 22'6, mlcod=22'3)", 2024-09-15T21:47:53.085 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:47:53.085 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 72.572712413000005, 2024-09-15T21:47:53.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021760249999999998, 2024-09-15T21:47:53.085 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.086 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.086 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.086 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:47:53.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:47:53.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478102+0000", 2024-09-15T21:47:53.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3099999999999996e-07 2024-09-15T21:47:53.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478124+0000", 2024-09-15T21:47:53.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2790999999999999e-05 2024-09-15T21:47:53.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478136+0000", 2024-09-15T21:47:53.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1922999999999999e-05 2024-09-15T21:47:53.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478144+0000", 2024-09-15T21:47:53.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9300000000000003e-06 2024-09-15T21:47:53.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478192+0000", 2024-09-15T21:47:53.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8155999999999998e-05 2024-09-15T21:47:53.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478209+0000", 2024-09-15T21:47:53.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6816e-05 2024-09-15T21:47:53.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480243+0000", 2024-09-15T21:47:53.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020336199999999999 2024-09-15T21:47:53.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480277+0000", 2024-09-15T21:47:53.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3858e-05 2024-09-15T21:47:53.089 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.089 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.089 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.090 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.11 epoch 22/19 rep_tid 19 entries 22'2 (0'0) error 2:8975f766:::202.00000001:head by mds.2.12:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:53.090 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:53.090 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 72.572586474999994, 2024-09-15T21:47:53.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023882299999999999, 2024-09-15T21:47:53.090 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.090 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.090 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:53.090 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.090 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.090 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:53.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:53.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:47:53.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2399999999999998e-07 2024-09-15T21:47:53.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478236+0000", 2024-09-15T21:47:53.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5299999999999996e-06 2024-09-15T21:47:53.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478248+0000", 2024-09-15T21:47:53.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2806000000000001e-05 2024-09-15T21:47:53.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478264+0000", 2024-09-15T21:47:53.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5468000000000002e-05 2024-09-15T21:47:53.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478339+0000", 2024-09-15T21:47:53.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4851000000000006e-05 2024-09-15T21:47:53.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480615+0000", 2024-09-15T21:47:53.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022760509999999999 2024-09-15T21:47:53.093 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.093 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.093 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.094 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:3 2.1a e22/19 2:5fdc322d:::61e.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:47:53.094 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:47:53.094 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 67.504401873000006, 2024-09-15T21:47:53.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023961220000000001, 2024-09-15T21:47:53.094 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.094 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.094 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.094 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:47:53.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:47:53.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546413+0000", 2024-09-15T21:47:53.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.812e-06 2024-09-15T21:47:53.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546424+0000", 2024-09-15T21:47:53.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1313e-05 2024-09-15T21:47:53.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546444+0000", 2024-09-15T21:47:53.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9706000000000001e-05 2024-09-15T21:47:53.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546453+0000", 2024-09-15T21:47:53.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:47:53.096 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546519+0000", 2024-09-15T21:47:53.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5710999999999995e-05 2024-09-15T21:47:53.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546539+0000", 2024-09-15T21:47:53.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0275000000000002e-05 2024-09-15T21:47:53.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548763+0000", 2024-09-15T21:47:53.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002223226 2024-09-15T21:47:53.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548807+0000", 2024-09-15T21:47:53.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4598000000000002e-05 2024-09-15T21:47:53.098 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.098 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.098 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.098 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:19 2.11 e22/19 2:890f50f7:::626.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:47:53.098 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:47:53.098 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 67.504104713000004, 2024-09-15T21:47:53.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022671280000000002, 2024-09-15T21:47:53.099 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.099 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.099 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.099 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:47:53.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:47:53.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546709+0000", 2024-09-15T21:47:53.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4499999999999997e-07 2024-09-15T21:47:53.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546723+0000", 2024-09-15T21:47:53.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4623999999999999e-05 2024-09-15T21:47:53.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546734+0000", 2024-09-15T21:47:53.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0331e-05 2024-09-15T21:47:53.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546741+0000", 2024-09-15T21:47:53.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.075e-06 2024-09-15T21:47:53.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546954+0000", 2024-09-15T21:47:53.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021275600000000001 2024-09-15T21:47:53.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546971+0000", 2024-09-15T21:47:53.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7503000000000002e-05 2024-09-15T21:47:53.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548926+0000", 2024-09-15T21:47:53.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019550129999999998 2024-09-15T21:47:53.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548975+0000", 2024-09-15T21:47:53.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9280999999999998e-05 2024-09-15T21:47:53.103 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.103 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.103 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.103 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.d epoch 22/19 rep_tid 18 entries 22'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.14:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:47:53.103 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:47:53.103 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 67.504098362999997, 2024-09-15T21:47:53.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021738489999999998, 2024-09-15T21:47:53.104 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.104 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.104 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:53.104 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:47:53.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:47:53.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546716+0000", 2024-09-15T21:47:53.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.615e-06 2024-09-15T21:47:53.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546730+0000", 2024-09-15T21:47:53.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3519999999999999e-05 2024-09-15T21:47:53.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546753+0000", 2024-09-15T21:47:53.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3643999999999999e-05 2024-09-15T21:47:53.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546770+0000", 2024-09-15T21:47:53.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6518999999999999e-05 2024-09-15T21:47:53.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546900+0000", 2024-09-15T21:47:53.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013049099999999999 2024-09-15T21:47:53.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548888+0000", 2024-09-15T21:47:53.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019880599999999998 2024-09-15T21:47:53.109 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.109 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.109 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.109 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:4 2.1a e22/19 2:5fdc322d:::61e.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:47:53.109 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:47:53.109 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 67.503989715000003, 2024-09-15T21:47:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002968769, 2024-09-15T21:47:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:47:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:47:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546824+0000", 2024-09-15T21:47:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6499999999999999e-07 2024-09-15T21:47:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546831+0000", 2024-09-15T21:47:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9940000000000003e-06 2024-09-15T21:47:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546840+0000", 2024-09-15T21:47:53.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3689999999999996e-06 2024-09-15T21:47:53.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546847+0000", 2024-09-15T21:47:53.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0589999999999997e-06 2024-09-15T21:47:53.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547709+0000", 2024-09-15T21:47:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00086144699999999999 2024-09-15T21:47:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547730+0000", 2024-09-15T21:47:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1542000000000001e-05 2024-09-15T21:47:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549750+0000", 2024-09-15T21:47:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002020165 2024-09-15T21:47:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549792+0000", 2024-09-15T21:47:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1628000000000002e-05 2024-09-15T21:47:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:20 2.11 e22/19 2:890f50f7:::626.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:47:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 67.502965106999994, 2024-09-15T21:47:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017734579999999999, 2024-09-15T21:47:53.114 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.115 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.115 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.115 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:53.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:53.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:47:53.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:47:53.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547856+0000", 2024-09-15T21:47:53.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7999999999999999e-06 2024-09-15T21:47:53.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547867+0000", 2024-09-15T21:47:53.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0971999999999999e-05 2024-09-15T21:47:53.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547881+0000", 2024-09-15T21:47:53.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4168999999999999e-05 2024-09-15T21:47:53.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547935+0000", 2024-09-15T21:47:53.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3721999999999998e-05 2024-09-15T21:47:53.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547976+0000", 2024-09-15T21:47:53.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1248999999999998e-05 2024-09-15T21:47:53.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549559+0000", 2024-09-15T21:47:53.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001582815 2024-09-15T21:47:53.118 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.118 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.118 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.118 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549621+0000", 2024-09-15T21:47:53.118 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2156999999999995e-05 2024-09-15T21:47:53.119 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.119 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.119 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.119 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.119 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:13 2.1a e22/19 2:594e21d0:::623.00000000:head v 22'6, mlcod=22'2)", 2024-09-15T21:47:53.119 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:47:53.119 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 67.502900111000002, 2024-09-15T21:47:53.119 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019085300000000001, 2024-09-15T21:47:53.119 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.119 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.119 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.119 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.119 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:47:53.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:47:53.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547914+0000", 2024-09-15T21:47:53.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1500000000000004e-07 2024-09-15T21:47:53.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547920+0000", 2024-09-15T21:47:53.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5239999999999997e-06 2024-09-15T21:47:53.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547930+0000", 2024-09-15T21:47:53.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5109999999999992e-06 2024-09-15T21:47:53.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547937+0000", 2024-09-15T21:47:53.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5410000000000003e-06 2024-09-15T21:47:53.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548518+0000", 2024-09-15T21:47:53.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00058074299999999999 2024-09-15T21:47:53.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548538+0000", 2024-09-15T21:47:53.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9959e-05 2024-09-15T21:47:53.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549795+0000", 2024-09-15T21:47:53.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012570120000000001 2024-09-15T21:47:53.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549821+0000", 2024-09-15T21:47:53.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6625e-05 2024-09-15T21:47:53.123 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.123 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.123 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.123 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:23 2.18 e22/19 2:1c1db442:::103.00000000.inode:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:53.123 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:47:53.124 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 67.502838693000001, 2024-09-15T21:47:53.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002713198, 2024-09-15T21:47:53.124 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.124 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.124 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.124 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:47:53.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:47:53.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547975+0000", 2024-09-15T21:47:53.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3900000000000004e-07 2024-09-15T21:47:53.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547983+0000", 2024-09-15T21:47:53.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6690000000000006e-06 2024-09-15T21:47:53.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547994+0000", 2024-09-15T21:47:53.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1503e-05 2024-09-15T21:47:53.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548002+0000", 2024-09-15T21:47:53.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1340000000000004e-06 2024-09-15T21:47:53.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548098+0000", 2024-09-15T21:47:53.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5791000000000005e-05 2024-09-15T21:47:53.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548118+0000", 2024-09-15T21:47:53.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0129e-05 2024-09-15T21:47:53.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550643+0000", 2024-09-15T21:47:53.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002524728 2024-09-15T21:47:53.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550688+0000", 2024-09-15T21:47:53.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4404999999999999e-05 2024-09-15T21:47:53.128 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.128 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.128 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.128 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:14 2.1a e22/19 2:594e21d0:::623.00000000:head v 22'7, mlcod=22'2)", 2024-09-15T21:47:53.128 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:47:53.128 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 67.502823733, 2024-09-15T21:47:53.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024358579999999999, 2024-09-15T21:47:53.128 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.128 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.128 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.128 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:47:53.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:47:53.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547990+0000", 2024-09-15T21:47:53.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5799999999999999e-07 2024-09-15T21:47:53.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548011+0000", 2024-09-15T21:47:53.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0942999999999999e-05 2024-09-15T21:47:53.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548020+0000", 2024-09-15T21:47:53.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2380000000000003e-06 2024-09-15T21:47:53.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548027+0000", 2024-09-15T21:47:53.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6429999999999998e-06 2024-09-15T21:47:53.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548848+0000", 2024-09-15T21:47:53.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082167999999999998 2024-09-15T21:47:53.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548868+0000", 2024-09-15T21:47:53.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9791000000000001e-05 2024-09-15T21:47:53.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550387+0000", 2024-09-15T21:47:53.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001519322 2024-09-15T21:47:53.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550425+0000", 2024-09-15T21:47:53.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7682999999999998e-05 2024-09-15T21:47:53.132 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.132 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.132 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.133 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:35 2.f 2:f426ad4f:::mds2_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:53.133 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:47:53.133 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 67.312712047999995, 2024-09-15T21:47:53.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017328949999999999, 2024-09-15T21:47:53.133 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.133 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.133 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.133 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:47:53.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:47:53.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738103+0000", 2024-09-15T21:47:53.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0020000000000001e-06 2024-09-15T21:47:53.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738112+0000", 2024-09-15T21:47:53.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2220000000000008e-06 2024-09-15T21:47:53.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738127+0000", 2024-09-15T21:47:53.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5016e-05 2024-09-15T21:47:53.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738137+0000", 2024-09-15T21:47:53.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0301e-05 2024-09-15T21:47:53.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738216+0000", 2024-09-15T21:47:53.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8356000000000007e-05 2024-09-15T21:47:53.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738340+0000", 2024-09-15T21:47:53.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012372800000000001 2024-09-15T21:47:53.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:53.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738453+0000", 2024-09-15T21:47:53.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000113785 2024-09-15T21:47:53.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739115+0000", 2024-09-15T21:47:53.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00066178500000000002 2024-09-15T21:47:53.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739745+0000", 2024-09-15T21:47:53.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00063017900000000003 2024-09-15T21:47:53.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.137 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.137 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739783+0000", 2024-09-15T21:47:53.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7941000000000001e-05 2024-09-15T21:47:53.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739834+0000", 2024-09-15T21:47:53.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.058e-05 2024-09-15T21:47:53.138 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.138 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.138 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.138 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.4.16:13 2.13 e22/19 2:c81eaac8:::62d.00000000:head v 22'8, mlcod=22'6)", 2024-09-15T21:47:53.138 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:47:53.138 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 61.485306989000001, 2024-09-15T21:47:53.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002151778, 2024-09-15T21:47:53.139 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.139 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.139 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.139 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:47:53.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:47:53.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565507+0000", 2024-09-15T21:47:53.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3340000000000001e-06 2024-09-15T21:47:53.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565518+0000", 2024-09-15T21:47:53.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0339999999999999e-05 2024-09-15T21:47:53.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565536+0000", 2024-09-15T21:47:53.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8647e-05 2024-09-15T21:47:53.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565545+0000", 2024-09-15T21:47:53.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1430000000000004e-06 2024-09-15T21:47:53.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565611+0000", 2024-09-15T21:47:53.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5129000000000006e-05 2024-09-15T21:47:53.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565634+0000", 2024-09-15T21:47:53.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3173e-05 2024-09-15T21:47:53.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567561+0000", 2024-09-15T21:47:53.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019268180000000001 2024-09-15T21:47:53.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567658+0000", 2024-09-15T21:47:53.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7194000000000005e-05 2024-09-15T21:47:53.143 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.143 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.143 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.143 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.d epoch 22/19 rep_tid 24 entries 22'3 (0'0) error 2:b2057508:::204.00000002:head by mds.4.16:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 22'1)", 2024-09-15T21:47:53.143 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:47:53.143 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 61.485146127, 2024-09-15T21:47:53.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026915049999999999, 2024-09-15T21:47:53.143 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.143 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.143 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:47:53.143 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:47:53.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:47:53.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565668+0000", 2024-09-15T21:47:53.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.113e-06 2024-09-15T21:47:53.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565684+0000", 2024-09-15T21:47:53.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5943999999999998e-05 2024-09-15T21:47:53.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565705+0000", 2024-09-15T21:47:53.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0761999999999999e-05 2024-09-15T21:47:53.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565724+0000", 2024-09-15T21:47:53.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9208e-05 2024-09-15T21:47:53.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565817+0000", 2024-09-15T21:47:53.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3218000000000001e-05 2024-09-15T21:47:53.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568358+0000", 2024-09-15T21:47:53.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00254126 2024-09-15T21:47:53.146 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.147 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.147 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.147 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.147 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.147 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:53.148 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:47:53.149 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:53.146+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:53.248 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:53.247+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:53.253 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:53.253 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:53.253 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:53.253 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:53.263 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:47:53.350 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:53.348+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:53.408 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:53.408 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:53.409 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:53.409 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:53.409 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:53.416 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:47:53.451 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:53.449+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:53.552 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:53.551+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:53.560 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:53.560 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:53.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:53.560 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:53.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.560 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:53.560 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:53.560 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.144039555000006, 2024-09-15T21:47:53.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0050344960000000003, 2024-09-15T21:47:53.560 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.560 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.561 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.561 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:53.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:53.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:53.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0199999999999998e-07 2024-09-15T21:47:53.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413470+0000", 2024-09-15T21:47:53.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4329999999999998e-06 2024-09-15T21:47:53.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413579+0000", 2024-09-15T21:47:53.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108593 2024-09-15T21:47:53.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413587+0000", 2024-09-15T21:47:53.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3100000000000001e-06 2024-09-15T21:47:53.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413691+0000", 2024-09-15T21:47:53.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010357 2024-09-15T21:47:53.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413887+0000", 2024-09-15T21:47:53.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019666799999999999 2024-09-15T21:47:53.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:53.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414027+0000", 2024-09-15T21:47:53.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013996700000000001 2024-09-15T21:47:53.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416360+0000", 2024-09-15T21:47:53.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002332204 2024-09-15T21:47:53.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418437+0000", 2024-09-15T21:47:53.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002077279 2024-09-15T21:47:53.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418469+0000", 2024-09-15T21:47:53.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2359e-05 2024-09-15T21:47:53.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418502+0000", 2024-09-15T21:47:53.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2811000000000001e-05 2024-09-15T21:47:53.565 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.565 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.565 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.566 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:53.566 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:53.566 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.144035514999999, 2024-09-15T21:47:53.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0053538780000000003, 2024-09-15T21:47:53.566 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.566 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.566 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.566 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:53.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:53.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:53.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2000000000000001e-07 2024-09-15T21:47:53.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413474+0000", 2024-09-15T21:47:53.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-15T21:47:53.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413612+0000", 2024-09-15T21:47:53.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000137727 2024-09-15T21:47:53.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413618+0000", 2024-09-15T21:47:53.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.241e-06 2024-09-15T21:47:53.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414443+0000", 2024-09-15T21:47:53.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082447 2024-09-15T21:47:53.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414523+0000", 2024-09-15T21:47:53.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0059e-05 2024-09-15T21:47:53.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:53.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414653+0000", 2024-09-15T21:47:53.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013031699999999999 2024-09-15T21:47:53.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416366+0000", 2024-09-15T21:47:53.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001712917 2024-09-15T21:47:53.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418769+0000", 2024-09-15T21:47:53.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024031529999999999 2024-09-15T21:47:53.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418802+0000", 2024-09-15T21:47:53.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2925000000000001e-05 2024-09-15T21:47:53.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418825+0000", 2024-09-15T21:47:53.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3204e-05 2024-09-15T21:47:53.571 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.571 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.571 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.571 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:53.571 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:53.571 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.144027493999999, 2024-09-15T21:47:53.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059708640000000002, 2024-09-15T21:47:53.571 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.571 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.571 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.572 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:53.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:53.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:53.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7300000000000002e-07 2024-09-15T21:47:53.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413482+0000", 2024-09-15T21:47:53.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2359999999999999e-06 2024-09-15T21:47:53.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414323+0000", 2024-09-15T21:47:53.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00084095399999999999 2024-09-15T21:47:53.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414332+0000", 2024-09-15T21:47:53.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0880000000000001e-06 2024-09-15T21:47:53.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415439+0000", 2024-09-15T21:47:53.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011071869999999999 2024-09-15T21:47:53.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415554+0000", 2024-09-15T21:47:53.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114499 2024-09-15T21:47:53.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:53.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415682+0000", 2024-09-15T21:47:53.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012802500000000001 2024-09-15T21:47:53.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.417957+0000", 2024-09-15T21:47:53.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022746720000000002 2024-09-15T21:47:53.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.575 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.575 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419385+0000", 2024-09-15T21:47:53.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001428826 2024-09-15T21:47:53.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419420+0000", 2024-09-15T21:47:53.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4875000000000001e-05 2024-09-15T21:47:53.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419450+0000", 2024-09-15T21:47:53.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0128999999999999e-05 2024-09-15T21:47:53.576 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.576 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.576 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.577 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:53.577 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:53.578 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.144023676000003, 2024-09-15T21:47:53.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0087186220000000005, 2024-09-15T21:47:53.578 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.578 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.578 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.578 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:53.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:53.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413484+0000", 2024-09-15T21:47:53.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-15T21:47:53.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413487+0000", 2024-09-15T21:47:53.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.534e-06 2024-09-15T21:47:53.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414353+0000", 2024-09-15T21:47:53.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00086616999999999998 2024-09-15T21:47:53.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414359+0000", 2024-09-15T21:47:53.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3279999999999996e-06 2024-09-15T21:47:53.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416041+0000", 2024-09-15T21:47:53.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001682467 2024-09-15T21:47:53.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416158+0000", 2024-09-15T21:47:53.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117242 2024-09-15T21:47:53.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:53.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416284+0000", 2024-09-15T21:47:53.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012509000000000001 2024-09-15T21:47:53.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.417992+0000", 2024-09-15T21:47:53.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017087059999999999 2024-09-15T21:47:53.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422142+0000", 2024-09-15T21:47:53.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041501180000000004 2024-09-15T21:47:53.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422176+0000", 2024-09-15T21:47:53.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4109999999999997e-05 2024-09-15T21:47:53.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422202+0000", 2024-09-15T21:47:53.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5585e-05 2024-09-15T21:47:53.583 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.583 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.583 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.583 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:53.583 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:53.583 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 85.144018251000006, 2024-09-15T21:47:53.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0088000370000000001, 2024-09-15T21:47:53.584 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.584 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.584 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:53.584 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:53.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:53.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:53.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:53.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413491+0000", 2024-09-15T21:47:53.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.773e-06 2024-09-15T21:47:53.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414372+0000", 2024-09-15T21:47:53.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00088122000000000005 2024-09-15T21:47:53.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414378+0000", 2024-09-15T21:47:53.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4380000000000003e-06 2024-09-15T21:47:53.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416631+0000", 2024-09-15T21:47:53.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022530319999999999 2024-09-15T21:47:53.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416765+0000", 2024-09-15T21:47:53.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013423499999999999 2024-09-15T21:47:53.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422289+0000", 2024-09-15T21:47:53.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055240530000000001 2024-09-15T21:47:53.588 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.588 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.588 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.589 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:53.589 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:53.589 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.102449931999999, 2024-09-15T21:47:53.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056900609999999997, 2024-09-15T21:47:53.589 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.589 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.589 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.590 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:53.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:53.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455058+0000", 2024-09-15T21:47:53.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.51e-07 2024-09-15T21:47:53.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455061+0000", 2024-09-15T21:47:53.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.145e-06 2024-09-15T21:47:53.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455158+0000", 2024-09-15T21:47:53.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7101000000000004e-05 2024-09-15T21:47:53.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455165+0000", 2024-09-15T21:47:53.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1019999999999999e-06 2024-09-15T21:47:53.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456433+0000", 2024-09-15T21:47:53.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012678259999999999 2024-09-15T21:47:53.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456517+0000", 2024-09-15T21:47:53.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4373e-05 2024-09-15T21:47:53.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:53.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456677+0000", 2024-09-15T21:47:53.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015963300000000001 2024-09-15T21:47:53.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458636+0000", 2024-09-15T21:47:53.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019594030000000002 2024-09-15T21:47:53.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460665+0000", 2024-09-15T21:47:53.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020283549999999999 2024-09-15T21:47:53.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460707+0000", 2024-09-15T21:47:53.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2398000000000003e-05 2024-09-15T21:47:53.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460747+0000", 2024-09-15T21:47:53.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0274000000000001e-05 2024-09-15T21:47:53.595 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.595 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.595 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.595 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:53.595 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:53.595 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.102442737999993, 2024-09-15T21:47:53.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086420030000000005, 2024-09-15T21:47:53.595 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.595 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.595 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.595 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:53.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:53.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455065+0000", 2024-09-15T21:47:53.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3700000000000001e-07 2024-09-15T21:47:53.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455068+0000", 2024-09-15T21:47:53.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.915e-06 2024-09-15T21:47:53.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455366+0000", 2024-09-15T21:47:53.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00029825400000000002 2024-09-15T21:47:53.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455374+0000", 2024-09-15T21:47:53.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6240000000000008e-06 2024-09-15T21:47:53.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456199+0000", 2024-09-15T21:47:53.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082448500000000002 2024-09-15T21:47:53.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457034+0000", 2024-09-15T21:47:53.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00083553399999999997 2024-09-15T21:47:53.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:53.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457171+0000", 2024-09-15T21:47:53.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013653600000000001 2024-09-15T21:47:53.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462663+0000", 2024-09-15T21:47:53.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054924420000000002 2024-09-15T21:47:53.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463613+0000", 2024-09-15T21:47:53.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094971499999999996 2024-09-15T21:47:53.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463663+0000", 2024-09-15T21:47:53.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9636999999999999e-05 2024-09-15T21:47:53.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463706+0000", 2024-09-15T21:47:53.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3523999999999999e-05 2024-09-15T21:47:53.608 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.608 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.608 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.609 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:53.609 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:53.609 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.102438210000003, 2024-09-15T21:47:53.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090408420000000003, 2024-09-15T21:47:53.609 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.609 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.609 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.609 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:53.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:53.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:53.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.79e-07 2024-09-15T21:47:53.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455072+0000", 2024-09-15T21:47:53.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6189999999999998e-06 2024-09-15T21:47:53.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455403+0000", 2024-09-15T21:47:53.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033167299999999999 2024-09-15T21:47:53.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455420+0000", 2024-09-15T21:47:53.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6518999999999999e-05 2024-09-15T21:47:53.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457448+0000", 2024-09-15T21:47:53.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020275710000000001 2024-09-15T21:47:53.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:53.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3844999999999994e-05 2024-09-15T21:47:53.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:53.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457684+0000", 2024-09-15T21:47:53.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015239 2024-09-15T21:47:53.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462678+0000", 2024-09-15T21:47:53.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049939279999999999 2024-09-15T21:47:53.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464017+0000", 2024-09-15T21:47:53.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013397140000000001 2024-09-15T21:47:53.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464071+0000", 2024-09-15T21:47:53.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3735e-05 2024-09-15T21:47:53.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464110+0000", 2024-09-15T21:47:53.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8569000000000003e-05 2024-09-15T21:47:53.615 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.615 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.615 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.615 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:53.615 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:53.615 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.102426597000004, 2024-09-15T21:47:53.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091802889999999995, 2024-09-15T21:47:53.615 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.615 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.615 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.615 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:53.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:53.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455081+0000", 2024-09-15T21:47:53.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:53.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455084+0000", 2024-09-15T21:47:53.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7810000000000001e-06 2024-09-15T21:47:53.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455536+0000", 2024-09-15T21:47:53.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00045219100000000001 2024-09-15T21:47:53.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455583+0000", 2024-09-15T21:47:53.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7308999999999997e-05 2024-09-15T21:47:53.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457998+0000", 2024-09-15T21:47:53.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024154290000000002 2024-09-15T21:47:53.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458147+0000", 2024-09-15T21:47:53.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014834599999999999 2024-09-15T21:47:53.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:53.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458265+0000", 2024-09-15T21:47:53.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118458 2024-09-15T21:47:53.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464134+0000", 2024-09-15T21:47:53.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005869038 2024-09-15T21:47:53.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464187+0000", 2024-09-15T21:47:53.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2343999999999999e-05 2024-09-15T21:47:53.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464222+0000", 2024-09-15T21:47:53.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5447000000000001e-05 2024-09-15T21:47:53.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464261+0000", 2024-09-15T21:47:53.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8659999999999999e-05 2024-09-15T21:47:53.623 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.623 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.623 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.624 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:53.624 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:53.624 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 73.081804218000002, 2024-09-15T21:47:53.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048468290000000004, 2024-09-15T21:47:53.625 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.625 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.625 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.625 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:53.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:53.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:53.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.35e-07 2024-09-15T21:47:53.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475706+0000", 2024-09-15T21:47:53.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.994e-06 2024-09-15T21:47:53.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475770+0000", 2024-09-15T21:47:53.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3904000000000003e-05 2024-09-15T21:47:53.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475775+0000", 2024-09-15T21:47:53.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2490000000000001e-06 2024-09-15T21:47:53.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476502+0000", 2024-09-15T21:47:53.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072697300000000001 2024-09-15T21:47:53.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476611+0000", 2024-09-15T21:47:53.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0001091 2024-09-15T21:47:53.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:53.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476807+0000", 2024-09-15T21:47:53.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000195892 2024-09-15T21:47:53.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479678+0000", 2024-09-15T21:47:53.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028704709999999999 2024-09-15T21:47:53.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480444+0000", 2024-09-15T21:47:53.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00076602099999999998 2024-09-15T21:47:53.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480502+0000", 2024-09-15T21:47:53.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8313999999999998e-05 2024-09-15T21:47:53.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480550+0000", 2024-09-15T21:47:53.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7676000000000002e-05 2024-09-15T21:47:53.631 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.631 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.631 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.632 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:53.632 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:53.632 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 73.081797042999995, 2024-09-15T21:47:53.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051250929999999998, 2024-09-15T21:47:53.633 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.633 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.633 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:53.633 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:53.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:53.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:53.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1e-07 2024-09-15T21:47:53.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475723+0000", 2024-09-15T21:47:53.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2758000000000001e-05 2024-09-15T21:47:53.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475972+0000", 2024-09-15T21:47:53.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024886299999999999 2024-09-15T21:47:53.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475979+0000", 2024-09-15T21:47:53.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0230000000000004e-06 2024-09-15T21:47:53.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476560+0000", 2024-09-15T21:47:53.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00058097599999999998 2024-09-15T21:47:53.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476689+0000", 2024-09-15T21:47:53.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129486 2024-09-15T21:47:53.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480835+0000", 2024-09-15T21:47:53.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041457769999999998 2024-09-15T21:47:53.637 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.637 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.638 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.638 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:53.638 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:53.638 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.012881817999997, 2024-09-15T21:47:53.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054420570000000001, 2024-09-15T21:47:53.638 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.638 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.639 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.639 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:53.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:53.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544626+0000", 2024-09-15T21:47:53.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8100000000000003e-07 2024-09-15T21:47:53.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544629+0000", 2024-09-15T21:47:53.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9869999999999999e-06 2024-09-15T21:47:53.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544775+0000", 2024-09-15T21:47:53.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014610599999999999 2024-09-15T21:47:53.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544782+0000", 2024-09-15T21:47:53.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4109999999999999e-06 2024-09-15T21:47:53.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545617+0000", 2024-09-15T21:47:53.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00083468300000000002 2024-09-15T21:47:53.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545690+0000", 2024-09-15T21:47:53.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3545000000000001e-05 2024-09-15T21:47:53.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:53.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545803+0000", 2024-09-15T21:47:53.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000112083 2024-09-15T21:47:53.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548353+0000", 2024-09-15T21:47:53.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025507220000000001 2024-09-15T21:47:53.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549964+0000", 2024-09-15T21:47:53.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001610382 2024-09-15T21:47:53.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550021+0000", 2024-09-15T21:47:53.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6898999999999999e-05 2024-09-15T21:47:53.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.645 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.645 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550067+0000", 2024-09-15T21:47:53.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758000000000002e-05 2024-09-15T21:47:53.645 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.645 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.645 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.646 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:53.646 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:53.646 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.012876539999993, 2024-09-15T21:47:53.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056323399999999996, 2024-09-15T21:47:53.646 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.646 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.646 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.647 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:53.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:53.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:53.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:47:53.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544634+0000", 2024-09-15T21:47:53.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.58e-06 2024-09-15T21:47:53.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544800+0000", 2024-09-15T21:47:53.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016576899999999999 2024-09-15T21:47:53.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544808+0000", 2024-09-15T21:47:53.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5179999999999998e-06 2024-09-15T21:47:53.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546075+0000", 2024-09-15T21:47:53.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012676969999999999 2024-09-15T21:47:53.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546179+0000", 2024-09-15T21:47:53.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000103935 2024-09-15T21:47:53.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:53.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546286+0000", 2024-09-15T21:47:53.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106758 2024-09-15T21:47:53.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548528+0000", 2024-09-15T21:47:53.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022420510000000001 2024-09-15T21:47:53.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550119+0000", 2024-09-15T21:47:53.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015907079999999999 2024-09-15T21:47:53.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550202+0000", 2024-09-15T21:47:53.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3374000000000005e-05 2024-09-15T21:47:53.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550263+0000", 2024-09-15T21:47:53.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0643999999999997e-05 2024-09-15T21:47:53.654 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.654 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.654 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.655 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:53.655 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:53.655 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.012871568999998, 2024-09-15T21:47:53.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061905470000000002, 2024-09-15T21:47:53.655 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.655 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.655 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.655 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:53.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:53.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:53.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6899999999999999e-07 2024-09-15T21:47:53.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544638+0000", 2024-09-15T21:47:53.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3920000000000001e-06 2024-09-15T21:47:53.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544819+0000", 2024-09-15T21:47:53.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018065099999999999 2024-09-15T21:47:53.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544825+0000", 2024-09-15T21:47:53.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7239999999999997e-06 2024-09-15T21:47:53.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546583+0000", 2024-09-15T21:47:53.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017588219999999999 2024-09-15T21:47:53.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546665+0000", 2024-09-15T21:47:53.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1693000000000005e-05 2024-09-15T21:47:53.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:53.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546808+0000", 2024-09-15T21:47:53.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142993 2024-09-15T21:47:53.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548533+0000", 2024-09-15T21:47:53.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017246 2024-09-15T21:47:53.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.661 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.661 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550606+0000", 2024-09-15T21:47:53.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020733969999999998 2024-09-15T21:47:53.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550783+0000", 2024-09-15T21:47:53.662 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000176565 2024-09-15T21:47:53.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550826+0000", 2024-09-15T21:47:53.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3441e-05 2024-09-15T21:47:53.663 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.663 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.663 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.664 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:53.664 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:53.664 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.012863847999995, 2024-09-15T21:47:53.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052813010000000004, 2024-09-15T21:47:53.665 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.665 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.665 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.665 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:53.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:53.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:53.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3799999999999999e-07 2024-09-15T21:47:53.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544647+0000", 2024-09-15T21:47:53.668 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6569999999999999e-06 2024-09-15T21:47:53.668 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544858+0000", 2024-09-15T21:47:53.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021069900000000001 2024-09-15T21:47:53.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544862+0000", 2024-09-15T21:47:53.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4900000000000002e-06 2024-09-15T21:47:53.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.670 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545610+0000", 2024-09-15T21:47:53.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00074786999999999998 2024-09-15T21:47:53.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545706+0000", 2024-09-15T21:47:53.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6271e-05 2024-09-15T21:47:53.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:53.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545834+0000", 2024-09-15T21:47:53.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012759499999999999 2024-09-15T21:47:53.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.674 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.674 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548554+0000", 2024-09-15T21:47:53.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027199339999999998 2024-09-15T21:47:53.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549818+0000", 2024-09-15T21:47:53.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012642199999999999 2024-09-15T21:47:53.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549884+0000", 2024-09-15T21:47:53.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5843e-05 2024-09-15T21:47:53.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549925+0000", 2024-09-15T21:47:53.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0484000000000001e-05 2024-09-15T21:47:53.677 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.677 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.677 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.678 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:53.678 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:53.678 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 68.012855170999998, 2024-09-15T21:47:53.678 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051106110000000001, 2024-09-15T21:47:53.678 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.679 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.679 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.679 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:53.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:53.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:53.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2700000000000001e-07 2024-09-15T21:47:53.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544667+0000", 2024-09-15T21:47:53.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4746000000000001e-05 2024-09-15T21:47:53.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545021+0000", 2024-09-15T21:47:53.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035412900000000001 2024-09-15T21:47:53.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545052+0000", 2024-09-15T21:47:53.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1158000000000002e-05 2024-09-15T21:47:53.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.684 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546050+0000", 2024-09-15T21:47:53.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00099782800000000008 2024-09-15T21:47:53.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546291+0000", 2024-09-15T21:47:53.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024093500000000001 2024-09-15T21:47:53.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:53.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546434+0000", 2024-09-15T21:47:53.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014308600000000001 2024-09-15T21:47:53.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548583+0000", 2024-09-15T21:47:53.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021491029999999999 2024-09-15T21:47:53.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549695+0000", 2024-09-15T21:47:53.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011116419999999999 2024-09-15T21:47:53.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549726+0000", 2024-09-15T21:47:53.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1195000000000002e-05 2024-09-15T21:47:53.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549763+0000", 2024-09-15T21:47:53.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6562e-05 2024-09-15T21:47:53.691 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.691 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.692 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.692 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:17 2.14 2:2eebc8f4:::62f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:53.692 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:53.692 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 61.992425558999997, 2024-09-15T21:47:53.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071004800000000002, 2024-09-15T21:47:53.693 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.693 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.693 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.693 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:53.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:53.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565083+0000", 2024-09-15T21:47:53.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.689e-06 2024-09-15T21:47:53.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.696 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565090+0000", 2024-09-15T21:47:53.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5930000000000002e-06 2024-09-15T21:47:53.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.697 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565119+0000", 2024-09-15T21:47:53.697 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8785999999999998e-05 2024-09-15T21:47:53.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565126+0000", 2024-09-15T21:47:53.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0110000000000003e-06 2024-09-15T21:47:53.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565237+0000", 2024-09-15T21:47:53.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011161 2024-09-15T21:47:53.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565383+0000", 2024-09-15T21:47:53.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014626900000000001 2024-09-15T21:47:53.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:53.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565520+0000", 2024-09-15T21:47:53.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013676399999999999 2024-09-15T21:47:53.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569909+0000", 2024-09-15T21:47:53.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043890149999999996 2024-09-15T21:47:53.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572096+0000", 2024-09-15T21:47:53.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021866559999999999 2024-09-15T21:47:53.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572144+0000", 2024-09-15T21:47:53.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7725000000000001e-05 2024-09-15T21:47:53.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572182+0000", 2024-09-15T21:47:53.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8362000000000003e-05 2024-09-15T21:47:53.701 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.701 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.701 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.701 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:53.701 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:53.701 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 61.992414461000003, 2024-09-15T21:47:53.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072110280000000004, 2024-09-15T21:47:53.701 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.701 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.701 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.702 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:53.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:53.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:53.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4299999999999999e-07 2024-09-15T21:47:53.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565096+0000", 2024-09-15T21:47:53.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4319999999999999e-06 2024-09-15T21:47:53.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565152+0000", 2024-09-15T21:47:53.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5686999999999997e-05 2024-09-15T21:47:53.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565157+0000", 2024-09-15T21:47:53.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2700000000000004e-06 2024-09-15T21:47:53.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565847+0000", 2024-09-15T21:47:53.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068937299999999996 2024-09-15T21:47:53.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565940+0000", 2024-09-15T21:47:53.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3573999999999996e-05 2024-09-15T21:47:53.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:53.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566085+0000", 2024-09-15T21:47:53.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000144902 2024-09-15T21:47:53.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570595+0000", 2024-09-15T21:47:53.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045103469999999996 2024-09-15T21:47:53.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572224+0000", 2024-09-15T21:47:53.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016282930000000001 2024-09-15T21:47:53.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572260+0000", 2024-09-15T21:47:53.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6103999999999999e-05 2024-09-15T21:47:53.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572304+0000", 2024-09-15T21:47:53.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3803000000000001e-05 2024-09-15T21:47:53.706 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.706 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.707 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.707 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:53.707 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:53.707 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 61.992409995999999, 2024-09-15T21:47:53.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055995389999999997, 2024-09-15T21:47:53.707 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.707 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.707 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.707 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:53.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:53.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:53.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9999999999999999e-07 2024-09-15T21:47:53.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:53.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2399999999999999e-06 2024-09-15T21:47:53.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.709 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.709 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565280+0000", 2024-09-15T21:47:53.709 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017929400000000001 2024-09-15T21:47:53.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.710 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.710 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565292+0000", 2024-09-15T21:47:53.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2193e-05 2024-09-15T21:47:53.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565357+0000", 2024-09-15T21:47:53.711 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4684000000000006e-05 2024-09-15T21:47:53.711 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.711 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.711 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.711 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565499+0000", 2024-09-15T21:47:53.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142032 2024-09-15T21:47:53.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:53.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565634+0000", 2024-09-15T21:47:53.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000135613 2024-09-15T21:47:53.712 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.712 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.712 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.712 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569941+0000", 2024-09-15T21:47:53.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043066789999999999 2024-09-15T21:47:53.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570585+0000", 2024-09-15T21:47:53.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00064364700000000001 2024-09-15T21:47:53.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570647+0000", 2024-09-15T21:47:53.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2173000000000003e-05 2024-09-15T21:47:53.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570697+0000", 2024-09-15T21:47:53.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9784000000000003e-05 2024-09-15T21:47:53.714 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.714 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.714 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.714 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:53.714 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:53.714 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 61.992405765999997, 2024-09-15T21:47:53.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066548939999999997, 2024-09-15T21:47:53.714 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:53.715 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:53.715 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:53.715 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:53.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:53.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:53.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:53.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:53.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:53.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:53.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565102+0000", 2024-09-15T21:47:53.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-15T21:47:53.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:53.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565103+0000", 2024-09-15T21:47:53.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6700000000000001e-06 2024-09-15T21:47:53.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:53.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565608+0000", 2024-09-15T21:47:53.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050518299999999998 2024-09-15T21:47:53.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:53.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565616+0000", 2024-09-15T21:47:53.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8199999999999997e-06 2024-09-15T21:47:53.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:53.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566598+0000", 2024-09-15T21:47:53.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00098176300000000008 2024-09-15T21:47:53.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:53.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566871+0000", 2024-09-15T21:47:53.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027312799999999998 2024-09-15T21:47:53.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:53.718 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567015+0000", 2024-09-15T21:47:53.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014428499999999999 2024-09-15T21:47:53.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.718 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:53.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570729+0000", 2024-09-15T21:47:53.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037132459999999999 2024-09-15T21:47:53.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:53.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571667+0000", 2024-09-15T21:47:53.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00093862299999999997 2024-09-15T21:47:53.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:53.719 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571718+0000", 2024-09-15T21:47:53.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1146999999999997e-05 2024-09-15T21:47:53.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:53.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:53.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:53.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571756+0000", 2024-09-15T21:47:53.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7821000000000003e-05 2024-09-15T21:47:53.720 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.720 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.720 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.720 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:53.720 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:53.720 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:53.722 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:47:53.722 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:53.720+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:53.822 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:53.821+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:53.862 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:47:53.868 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:53.868 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:53.869 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:53.869 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:53.878 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:47:53.961 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:53.960+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:53.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.962+0000 7f1fe24fa640 1 Processor -- start 2024-09-15T21:47:53.964 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.963+0000 7f1fe24fa640 1 -- start start 2024-09-15T21:47:53.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.963+0000 7f1fe24fa640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1fdc155440 0x7f1fdc0cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:53.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.963+0000 7f1fe24fa640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1fdc0cb7d0 0x7f1fdc0cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:53.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.964+0000 7f1fe24fa640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fdc15e7a0 con 0x7f1fdc06d0e0 2024-09-15T21:47:53.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.964+0000 7f1fe24fa640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f1fdc15b470 con 0x7f1fdc0cb7d0 2024-09-15T21:47:53.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.964+0000 7f1fe24fa640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f1fdc155820 con 0x7f1fdc155440 2024-09-15T21:47:53.966 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.964+0000 7f1fdb7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1fdc0cb7d0 0x7f1fdc0cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:53.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.964+0000 7f1fdbfff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1fdc155440 0x7f1fdc0cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:53.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.964+0000 7f1fdb7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1fdc0cb7d0 0x7f1fdc0cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57536/0 (socket says 172.21.15.100:57536) 2024-09-15T21:47:53.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.964+0000 7f1fdb7fe640 1 -- 172.21.15.100:0/3405134511 learned_addr learned my addr 172.21.15.100:0/3405134511 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:53.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.964+0000 7f1fdb7fe640 1 -- 172.21.15.100:0/3405134511 >> v2:172.21.15.122:3300/0 conn(0x7f1fdc155440 msgr2=0x7f1fdc0cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:53.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.964+0000 7f1fdb7fe640 1 --2- 172.21.15.100:0/3405134511 >> v2:172.21.15.122:3300/0 conn(0x7f1fdc155440 0x7f1fdc0cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:53.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.964+0000 7f1fdb7fe640 1 -- 172.21.15.100:0/3405134511 >> v1:172.21.15.122:6789/0 conn(0x7f1fdc06d0e0 legacy=0x7f1fdc0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:53.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.964+0000 7f1fdb7fe640 1 -- 172.21.15.100:0/3405134511 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fdc15ea40 con 0x7f1fdc0cb7d0 2024-09-15T21:47:53.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.964+0000 7f1fdaffd640 1 -- 172.21.15.100:0/3405134511 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2841200266 0 0) 0x7f1fdc15e7a0 con 0x7f1fdc06d0e0 2024-09-15T21:47:53.967 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.964+0000 7f1fdbfff640 1 --2- 172.21.15.100:0/3405134511 >> v2:172.21.15.122:3300/0 conn(0x7f1fdc155440 0x7f1fdc0cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:53.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.965+0000 7f1fdb7fe640 1 --2- 172.21.15.100:0/3405134511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fdc0cb7d0 0x7f1fdc0cd960 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc0ce040 tx=0x7f1fd002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231847539184ffef server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:53.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.965+0000 7f1fdaffd640 1 -- 172.21.15.100:0/3405134511 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0037070 con 0x7f1fdc0cb7d0 2024-09-15T21:47:53.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.965+0000 7f1fdaffd640 1 -- 172.21.15.100:0/3405134511 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fd0032440 con 0x7f1fdc0cb7d0 2024-09-15T21:47:53.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.965+0000 7f1fdaffd640 1 -- 172.21.15.100:0/3405134511 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0032810 con 0x7f1fdc0cb7d0 2024-09-15T21:47:53.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.966+0000 7f1fe24fa640 1 -- 172.21.15.100:0/3405134511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fdc0cb7d0 msgr2=0x7f1fdc0cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:53.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.966+0000 7f1fe24fa640 1 --2- 172.21.15.100:0/3405134511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fdc0cb7d0 0x7f1fdc0cd960 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1fdc0ce040 tx=0x7f1fd002a1e0 comp rx=0 tx=0).stop 2024-09-15T21:47:53.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.966+0000 7f1fe24fa640 1 -- 172.21.15.100:0/3405134511 shutdown_connections 2024-09-15T21:47:53.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.966+0000 7f1fe24fa640 1 --2- 172.21.15.100:0/3405134511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fdc0cb7d0 0x7f1fdc0cd960 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:53.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.966+0000 7f1fe24fa640 1 --2- 172.21.15.100:0/3405134511 >> v2:172.21.15.122:3300/0 conn(0x7f1fdc155440 0x7f1fdc0cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:53.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.966+0000 7f1fe24fa640 1 -- 172.21.15.100:0/3405134511 >> 172.21.15.100:0/3405134511 conn(0x7f1fdc0680d0 msgr2=0x7f1fdc06d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:53.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.966+0000 7f1fe24fa640 1 -- 172.21.15.100:0/3405134511 shutdown_connections 2024-09-15T21:47:53.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.966+0000 7f1fe24fa640 1 -- 172.21.15.100:0/3405134511 wait complete. 2024-09-15T21:47:53.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.967+0000 7f1fe24fa640 1 Processor -- start 2024-09-15T21:47:53.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.968+0000 7f1fe24fa640 1 -- start start 2024-09-15T21:47:53.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.968+0000 7f1fe24fa640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fdc06d0e0 0x7f1fdc0c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:53.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.968+0000 7f1fe24fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fdc0cb7d0 0x7f1fdc149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:53.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.968+0000 7f1fe24fa640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1fdc155440 0x7f1fdc1f0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:53.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.968+0000 7f1fe24fa640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc15f2e0 con 0x7f1fdc06d0e0 2024-09-15T21:47:53.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.968+0000 7f1fe24fa640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc15f000 con 0x7f1fdc0cb7d0 2024-09-15T21:47:53.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.968+0000 7f1fe24fa640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1fdc15b070 con 0x7f1fdc155440 2024-09-15T21:47:53.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.968+0000 7f1fdbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fdc06d0e0 0x7f1fdc0c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:53.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.968+0000 7f1fdbfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fdc06d0e0 0x7f1fdc0c9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57548/0 (socket says 172.21.15.100:57548) 2024-09-15T21:47:53.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.968+0000 7f1fdbfff640 1 -- 172.21.15.100:0/4273327908 learned_addr learned my addr 172.21.15.100:0/4273327908 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:53.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.968+0000 7f1fdb7fe640 1 --2- 172.21.15.100:0/4273327908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fdc0cb7d0 0x7f1fdc149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:53.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.969+0000 7f1fe0a97640 1 --2- 172.21.15.100:0/4273327908 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1fdc155440 0x7f1fdc1f0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:53.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.969+0000 7f1fdb7fe640 1 -- 172.21.15.100:0/4273327908 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1fdc155440 msgr2=0x7f1fdc1f0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:53.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.969+0000 7f1fdb7fe640 1 --2- 172.21.15.100:0/4273327908 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1fdc155440 0x7f1fdc1f0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:53.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.969+0000 7f1fdb7fe640 1 -- 172.21.15.100:0/4273327908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fdc06d0e0 msgr2=0x7f1fdc0c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:53.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.969+0000 7f1fdb7fe640 1 --2- 172.21.15.100:0/4273327908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fdc06d0e0 0x7f1fdc0c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:53.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.969+0000 7f1fdb7fe640 1 -- 172.21.15.100:0/4273327908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fdc15c750 con 0x7f1fdc0cb7d0 2024-09-15T21:47:53.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.969+0000 7f1fdb7fe640 1 --2- 172.21.15.100:0/4273327908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fdc0cb7d0 0x7f1fdc149bf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0031000 tx=0x7f1fd00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:53.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.969+0000 7f1fd8ff9640 1 -- 172.21.15.100:0/4273327908 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0041030 con 0x7f1fdc0cb7d0 2024-09-15T21:47:53.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.969+0000 7f1fd8ff9640 1 -- 172.21.15.100:0/4273327908 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fd0030440 con 0x7f1fdc0cb7d0 2024-09-15T21:47:53.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.969+0000 7f1fd8ff9640 1 -- 172.21.15.100:0/4273327908 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0037040 con 0x7f1fdc0cb7d0 2024-09-15T21:47:53.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.969+0000 7f1fe24fa640 1 -- 172.21.15.100:0/4273327908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fdc1f4350 con 0x7f1fdc0cb7d0 2024-09-15T21:47:53.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.969+0000 7f1fe24fa640 1 -- 172.21.15.100:0/4273327908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fdc1f4910 con 0x7f1fdc0cb7d0 2024-09-15T21:47:53.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.971+0000 7f1fe24fa640 1 -- 172.21.15.100:0/4273327908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fdc0b3940 con 0x7f1fdc0cb7d0 2024-09-15T21:47:53.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.971+0000 7f1fd8ff9640 1 -- 172.21.15.100:0/4273327908 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1fd0044070 con 0x7f1fdc0cb7d0 2024-09-15T21:47:53.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.972+0000 7f1fd8ff9640 1 --2- 172.21.15.100:0/4273327908 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1fc806c2e0 0x7f1fc806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:53.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.972+0000 7f1fd8ff9640 1 -- 172.21.15.100:0/4273327908 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5886+0+0 (secure 0 0 0) 0x7f1fd00ab860 con 0x7f1fdc0cb7d0 2024-09-15T21:47:53.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.972+0000 7f1fdbfff640 1 --2- 172.21.15.100:0/4273327908 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1fc806c2e0 0x7f1fc806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:53.976 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.972+0000 7f1fdbfff640 1 --2- 172.21.15.100:0/4273327908 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1fc806c2e0 0x7f1fc806e7a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc009560 tx=0x7f1fcc009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:53.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:53.976+0000 7f1fd8ff9640 1 -- 172.21.15.100:0/4273327908 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1fd0079960 con 0x7f1fdc0cb7d0 2024-09-15T21:47:54.022 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:54.022 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:54.022 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:54.022 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:54.022 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:54.023 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:54.022+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:54.031 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:47:54.123 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:47:54.124 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:54.123+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:54.173 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.171+0000 7f1fe24fa640 1 -- 172.21.15.100:0/4273327908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f1fdc0c5d10 con 0x7f1fdc0cb7d0 2024-09-15T21:47:54.175 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:47:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 100.937679613, 2024-09-15T21:47:54.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0944283340000001, 2024-09-15T21:47:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2024-09-15T21:47:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234360+0000", 2024-09-15T21:47:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3710000000000001e-06 2024-09-15T21:47:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234374+0000", 2024-09-15T21:47:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4003e-05 2024-09-15T21:47:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234392+0000", 2024-09-15T21:47:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7294000000000001e-05 2024-09-15T21:47:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234405+0000", 2024-09-15T21:47:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2961e-05 2024-09-15T21:47:54.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234483+0000", 2024-09-15T21:47:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8850000000000006e-05 2024-09-15T21:47:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-15T21:47:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234491+0000", 2024-09-15T21:47:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3529999999999997e-06 2024-09-15T21:47:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327253+0000", 2024-09-15T21:47:54.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.092761962 2024-09-15T21:47:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327456+0000", 2024-09-15T21:47:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000202893 2024-09-15T21:47:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327652+0000", 2024-09-15T21:47:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019613000000000001 2024-09-15T21:47:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328368+0000", 2024-09-15T21:47:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000716582 2024-09-15T21:47:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328682+0000", 2024-09-15T21:47:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000313825 2024-09-15T21:47:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328743+0000", 2024-09-15T21:47:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0571e-05 2024-09-15T21:47:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328787+0000", 2024-09-15T21:47:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4539000000000002e-05 2024-09-15T21:47:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 99.816514210999998, 2024-09-15T21:47:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0089551270000000002, 2024-09-15T21:47:54.182 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 35 2024-09-15T21:47:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355525+0000", 2024-09-15T21:47:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:47:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355606+0000", 2024-09-15T21:47:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1723000000000003e-05 2024-09-15T21:47:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355624+0000", 2024-09-15T21:47:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7645000000000001e-05 2024-09-15T21:47:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355629+0000", 2024-09-15T21:47:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7319999999999998e-06 2024-09-15T21:47:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358315+0000", 2024-09-15T21:47:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026858659999999999 2024-09-15T21:47:54.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358373+0000", 2024-09-15T21:47:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8593e-05 2024-09-15T21:47:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358521+0000", 2024-09-15T21:47:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014817500000000001 2024-09-15T21:47:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360353+0000", 2024-09-15T21:47:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018317100000000001 2024-09-15T21:47:54.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364424+0000", 2024-09-15T21:47:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040707970000000001 2024-09-15T21:47:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364452+0000", 2024-09-15T21:47:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7937999999999999e-05 2024-09-15T21:47:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364479+0000", 2024-09-15T21:47:54.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7744999999999999e-05 2024-09-15T21:47:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 99.816430248000003, 2024-09-15T21:47:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090267869999999997, 2024-09-15T21:47:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:54.188 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 36 2024-09-15T21:47:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.189 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355609+0000", 2024-09-15T21:47:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3099999999999998e-07 2024-09-15T21:47:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355612+0000", 2024-09-15T21:47:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1489999999999998e-06 2024-09-15T21:47:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355637+0000", 2024-09-15T21:47:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.508e-05 2024-09-15T21:47:54.190 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355640+0000", 2024-09-15T21:47:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7930000000000002e-06 2024-09-15T21:47:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359195+0000", 2024-09-15T21:47:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035555529999999999 2024-09-15T21:47:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359283+0000", 2024-09-15T21:47:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8042999999999997e-05 2024-09-15T21:47:54.191 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359419+0000", 2024-09-15T21:47:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013576699999999999 2024-09-15T21:47:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.192 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360361+0000", 2024-09-15T21:47:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094171000000000001 2024-09-15T21:47:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364536+0000", 2024-09-15T21:47:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00417484 2024-09-15T21:47:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364590+0000", 2024-09-15T21:47:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4355999999999999e-05 2024-09-15T21:47:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.193 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364635+0000", 2024-09-15T21:47:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5065000000000003e-05 2024-09-15T21:47:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.194 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 99.816425643000002, 2024-09-15T21:47:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091162750000000001, 2024-09-15T21:47:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 37 2024-09-15T21:47:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.195 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:54.196 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:47:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355615+0000", 2024-09-15T21:47:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-15T21:47:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355646+0000", 2024-09-15T21:47:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0391000000000001e-05 2024-09-15T21:47:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.197 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355648+0000", 2024-09-15T21:47:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6309999999999999e-06 2024-09-15T21:47:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359770+0000", 2024-09-15T21:47:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041222899999999998 2024-09-15T21:47:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359854+0000", 2024-09-15T21:47:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3399000000000006e-05 2024-09-15T21:47:54.198 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359988+0000", 2024-09-15T21:47:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013442200000000001 2024-09-15T21:47:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360524+0000", 2024-09-15T21:47:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00053582900000000004 2024-09-15T21:47:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.199 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364673+0000", 2024-09-15T21:47:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041493199999999997 2024-09-15T21:47:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364704+0000", 2024-09-15T21:47:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0380999999999999e-05 2024-09-15T21:47:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.200 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364729+0000", 2024-09-15T21:47:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5335999999999999e-05 2024-09-15T21:47:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 99.747708404999997, 2024-09-15T21:47:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0026270020000001, 2024-09-15T21:47:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.201 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 88 2024-09-15T21:47:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:54.202 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424331+0000", 2024-09-15T21:47:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4299999999999998e-07 2024-09-15T21:47:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424333+0000", 2024-09-15T21:47:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2359999999999999e-06 2024-09-15T21:47:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.203 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422895+0000", 2024-09-15T21:47:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99856184199999998 2024-09-15T21:47:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422902+0000", 2024-09-15T21:47:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4440000000000002e-06 2024-09-15T21:47:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423053+0000", 2024-09-15T21:47:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000150958 2024-09-15T21:47:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.204 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423160+0000", 2024-09-15T21:47:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106774 2024-09-15T21:47:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423344+0000", 2024-09-15T21:47:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018424199999999999 2024-09-15T21:47:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.424730+0000", 2024-09-15T21:47:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001386155 2024-09-15T21:47:54.205 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426756+0000", 2024-09-15T21:47:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020261150000000002 2024-09-15T21:47:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426906+0000", 2024-09-15T21:47:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000149176 2024-09-15T21:47:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426957+0000", 2024-09-15T21:47:54.206 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1616999999999997e-05 2024-09-15T21:47:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.617635207000006, 2024-09-15T21:47:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095037790000000004, 2024-09-15T21:47:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:54.207 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 114 2024-09-15T21:47:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.208 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554404+0000", 2024-09-15T21:47:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2399999999999999e-07 2024-09-15T21:47:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554407+0000", 2024-09-15T21:47:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-15T21:47:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554632+0000", 2024-09-15T21:47:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022568999999999999 2024-09-15T21:47:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.209 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554640+0000", 2024-09-15T21:47:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7970000000000001e-06 2024-09-15T21:47:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561325+0000", 2024-09-15T21:47:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066848020000000001 2024-09-15T21:47:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561402+0000", 2024-09-15T21:47:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6851e-05 2024-09-15T21:47:54.210 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561530+0000", 2024-09-15T21:47:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012863999999999999 2024-09-15T21:47:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563144+0000", 2024-09-15T21:47:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016132849999999999 2024-09-15T21:47:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563800+0000", 2024-09-15T21:47:54.211 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00065586099999999999 2024-09-15T21:47:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563857+0000", 2024-09-15T21:47:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.702e-05 2024-09-15T21:47:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563907+0000", 2024-09-15T21:47:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0479000000000002e-05 2024-09-15T21:47:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.212 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.617630210000002, 2024-09-15T21:47:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011813423999999999, 2024-09-15T21:47:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:54.213 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 115 2024-09-15T21:47:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554409+0000", 2024-09-15T21:47:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1100000000000002e-07 2024-09-15T21:47:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554498+0000", 2024-09-15T21:47:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9642999999999995e-05 2024-09-15T21:47:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554656+0000", 2024-09-15T21:47:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000158114 2024-09-15T21:47:54.215 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554661+0000", 2024-09-15T21:47:54.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0730000000000004e-06 2024-09-15T21:47:54.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561862+0000", 2024-09-15T21:47:54.216 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072002999999999998 2024-09-15T21:47:54.216 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.216 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.216 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.216 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561939+0000", 2024-09-15T21:47:54.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7265e-05 2024-09-15T21:47:54.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562073+0000", 2024-09-15T21:47:54.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013371600000000001 2024-09-15T21:47:54.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.217 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563196+0000", 2024-09-15T21:47:54.217 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011231679999999999 2024-09-15T21:47:54.217 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.217 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.217 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566124+0000", 2024-09-15T21:47:54.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002928403 2024-09-15T21:47:54.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566183+0000", 2024-09-15T21:47:54.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8328000000000001e-05 2024-09-15T21:47:54.218 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.218 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.218 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.218 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566222+0000", 2024-09-15T21:47:54.218 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9103000000000002e-05 2024-09-15T21:47:54.218 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.218 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.219 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.219 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.219 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.219 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:54.219 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:54.219 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.617536770000001, 2024-09-15T21:47:54.219 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01181426, 2024-09-15T21:47:54.219 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.219 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.219 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.219 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:54.219 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:54.219 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:54.220 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 116 2024-09-15T21:47:54.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.220 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:54.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.220 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.220 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:54.220 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.220 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.220 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554503+0000", 2024-09-15T21:47:54.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8000000000000005e-07 2024-09-15T21:47:54.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554506+0000", 2024-09-15T21:47:54.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5899999999999999e-06 2024-09-15T21:47:54.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.221 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.221 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.221 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554674+0000", 2024-09-15T21:47:54.221 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016763600000000001 2024-09-15T21:47:54.221 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554678+0000", 2024-09-15T21:47:54.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5310000000000003e-06 2024-09-15T21:47:54.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562714+0000", 2024-09-15T21:47:54.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.008035258 2024-09-15T21:47:54.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.222 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.222 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.222 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562765+0000", 2024-09-15T21:47:54.222 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1093e-05 2024-09-15T21:47:54.222 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562845+0000", 2024-09-15T21:47:54.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0427999999999996e-05 2024-09-15T21:47:54.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563943+0000", 2024-09-15T21:47:54.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097647 2024-09-15T21:47:54.223 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.223 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.223 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.223 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566232+0000", 2024-09-15T21:47:54.223 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022890850000000002 2024-09-15T21:47:54.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566279+0000", 2024-09-15T21:47:54.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6692e-05 2024-09-15T21:47:54.224 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.224 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.224 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.224 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566316+0000", 2024-09-15T21:47:54.224 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.752e-05 2024-09-15T21:47:54.224 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.224 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.224 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.225 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.225 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.225 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:54.225 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:54.225 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.617530670999997, 2024-09-15T21:47:54.225 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012232125, 2024-09-15T21:47:54.225 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.225 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.225 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.225 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:54.225 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:54.225 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:54.225 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 117 2024-09-15T21:47:54.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.226 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:54.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:54.226 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.226 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.226 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.226 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.226 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:54.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7700000000000001e-07 2024-09-15T21:47:54.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554511+0000", 2024-09-15T21:47:54.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8430000000000001e-06 2024-09-15T21:47:54.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.227 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554690+0000", 2024-09-15T21:47:54.227 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017856299999999999 2024-09-15T21:47:54.227 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.227 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.227 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554698+0000", 2024-09-15T21:47:54.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6200000000000005e-06 2024-09-15T21:47:54.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563224+0000", 2024-09-15T21:47:54.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0085262640000000004 2024-09-15T21:47:54.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.228 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.228 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.228 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563342+0000", 2024-09-15T21:47:54.228 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117687 2024-09-15T21:47:54.228 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563472+0000", 2024-09-15T21:47:54.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012957999999999999 2024-09-15T21:47:54.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566367+0000", 2024-09-15T21:47:54.229 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028952280000000001 2024-09-15T21:47:54.229 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.229 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.229 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.229 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566680+0000", 2024-09-15T21:47:54.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031345100000000002 2024-09-15T21:47:54.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566714+0000", 2024-09-15T21:47:54.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3701000000000001e-05 2024-09-15T21:47:54.230 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.230 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.230 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.230 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566740+0000", 2024-09-15T21:47:54.230 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5911e-05 2024-09-15T21:47:54.230 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.231 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.231 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.231 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.231 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.231 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:54.231 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:54.231 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.274638589000006, 2024-09-15T21:47:54.231 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069806590000000002, 2024-09-15T21:47:54.231 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.231 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.231 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.231 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:54.231 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:54.231 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:54.232 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 134 2024-09-15T21:47:54.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.232 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:54.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.232 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.232 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.232 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:54.232 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.232 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897401+0000", 2024-09-15T21:47:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0500000000000003e-07 2024-09-15T21:47:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897492+0000", 2024-09-15T21:47:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1770999999999999e-05 2024-09-15T21:47:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897750+0000", 2024-09-15T21:47:54.233 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025778299999999998 2024-09-15T21:47:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897756+0000", 2024-09-15T21:47:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2140000000000001e-06 2024-09-15T21:47:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899312+0000", 2024-09-15T21:47:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001556144 2024-09-15T21:47:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.234 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899372+0000", 2024-09-15T21:47:54.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0015000000000001e-05 2024-09-15T21:47:54.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899444+0000", 2024-09-15T21:47:54.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1923999999999998e-05 2024-09-15T21:47:54.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.235 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901354+0000", 2024-09-15T21:47:54.235 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019092530000000001 2024-09-15T21:47:54.235 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.235 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.235 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904327+0000", 2024-09-15T21:47:54.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029734520000000001 2024-09-15T21:47:54.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904360+0000", 2024-09-15T21:47:54.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3216000000000002e-05 2024-09-15T21:47:54.236 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.236 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.236 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.236 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904381+0000", 2024-09-15T21:47:54.236 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0282e-05 2024-09-15T21:47:54.236 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.237 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.237 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.237 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.237 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:54.237 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:54.237 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.274542732, 2024-09-15T21:47:54.237 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070121059999999997, 2024-09-15T21:47:54.237 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.237 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.237 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.237 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:54.237 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:54.238 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:54.238 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 135 2024-09-15T21:47:54.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.238 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:54.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.238 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.238 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:54.238 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897497+0000", 2024-09-15T21:47:54.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4799999999999997e-07 2024-09-15T21:47:54.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897500+0000", 2024-09-15T21:47:54.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2949999999999998e-06 2024-09-15T21:47:54.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.239 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.239 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897781+0000", 2024-09-15T21:47:54.239 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028064300000000002 2024-09-15T21:47:54.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897786+0000", 2024-09-15T21:47:54.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5169999999999999e-06 2024-09-15T21:47:54.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899857+0000", 2024-09-15T21:47:54.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020705340000000002 2024-09-15T21:47:54.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.240 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.240 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899932+0000", 2024-09-15T21:47:54.240 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5840999999999994e-05 2024-09-15T21:47:54.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900005+0000", 2024-09-15T21:47:54.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2716e-05 2024-09-15T21:47:54.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902296+0000", 2024-09-15T21:47:54.241 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022912010000000001 2024-09-15T21:47:54.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.241 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.241 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904424+0000", 2024-09-15T21:47:54.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021278600000000001 2024-09-15T21:47:54.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904472+0000", 2024-09-15T21:47:54.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7586000000000001e-05 2024-09-15T21:47:54.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.242 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.242 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.242 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904508+0000", 2024-09-15T21:47:54.242 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6164999999999997e-05 2024-09-15T21:47:54.242 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.242 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.242 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.243 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.243 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:54.243 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:54.243 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.274518240999996, 2024-09-15T21:47:54.243 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070611429999999998, 2024-09-15T21:47:54.243 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.243 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.243 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.243 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:54.243 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:54.243 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:54.243 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 136 2024-09-15T21:47:54.243 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.243 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:54.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:54.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.244 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.244 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.244 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.244 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897521+0000", 2024-09-15T21:47:54.244 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2900000000000004e-07 2024-09-15T21:47:54.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.245 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897527+0000", 2024-09-15T21:47:54.245 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8490000000000002e-06 2024-09-15T21:47:54.245 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.245 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.245 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897799+0000", 2024-09-15T21:47:54.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027219499999999998 2024-09-15T21:47:54.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897804+0000", 2024-09-15T21:47:54.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4219999999999999e-06 2024-09-15T21:47:54.246 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.246 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.246 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.246 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900248+0000", 2024-09-15T21:47:54.246 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002443457 2024-09-15T21:47:54.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900306+0000", 2024-09-15T21:47:54.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8263000000000001e-05 2024-09-15T21:47:54.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.247 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900385+0000", 2024-09-15T21:47:54.247 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9161999999999999e-05 2024-09-15T21:47:54.247 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.247 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.247 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902300+0000", 2024-09-15T21:47:54.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001914947 2024-09-15T21:47:54.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904536+0000", 2024-09-15T21:47:54.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022354419999999998 2024-09-15T21:47:54.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.248 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.248 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.248 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904562+0000", 2024-09-15T21:47:54.248 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6378999999999999e-05 2024-09-15T21:47:54.248 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.249 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.249 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904581+0000", 2024-09-15T21:47:54.249 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9497999999999999e-05 2024-09-15T21:47:54.249 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.249 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.249 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.249 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.249 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.249 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:54.249 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:54.250 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.274509674000001, 2024-09-15T21:47:54.250 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007180306, 2024-09-15T21:47:54.250 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.250 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.250 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.250 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:54.250 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:54.250 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:54.250 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 137 2024-09-15T21:47:54.250 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.250 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.250 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.250 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.250 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:54.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:54.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.251 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.251 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.251 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.251 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:54.251 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1399999999999998e-07 2024-09-15T21:47:54.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897612+0000", 2024-09-15T21:47:54.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2607000000000004e-05 2024-09-15T21:47:54.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897815+0000", 2024-09-15T21:47:54.252 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020352400000000001 2024-09-15T21:47:54.252 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.252 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.252 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.252 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897821+0000", 2024-09-15T21:47:54.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4670000000000003e-06 2024-09-15T21:47:54.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900665+0000", 2024-09-15T21:47:54.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028443689999999998 2024-09-15T21:47:54.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.253 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900714+0000", 2024-09-15T21:47:54.253 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8812e-05 2024-09-15T21:47:54.253 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.253 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.253 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900799+0000", 2024-09-15T21:47:54.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5129000000000004e-05 2024-09-15T21:47:54.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903413+0000", 2024-09-15T21:47:54.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026142890000000001 2024-09-15T21:47:54.254 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.254 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.254 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.254 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904625+0000", 2024-09-15T21:47:54.254 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012121149999999999 2024-09-15T21:47:54.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904673+0000", 2024-09-15T21:47:54.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7540999999999998e-05 2024-09-15T21:47:54.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.255 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.255 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.255 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904709+0000", 2024-09-15T21:47:54.255 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6139000000000001e-05 2024-09-15T21:47:54.255 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.255 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.255 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.255 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.256 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.256 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:54.256 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:54.256 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.274423718999998, 2024-09-15T21:47:54.256 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071727450000000003, 2024-09-15T21:47:54.256 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.256 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.256 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.256 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:54.256 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:54.256 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:54.256 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 138 2024-09-15T21:47:54.256 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.256 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:54.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:54.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.257 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.257 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.257 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.257 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:54.257 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:47:54.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897619+0000", 2024-09-15T21:47:54.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3869999999999999e-06 2024-09-15T21:47:54.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897832+0000", 2024-09-15T21:47:54.258 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021329500000000001 2024-09-15T21:47:54.258 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.258 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.258 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.258 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897837+0000", 2024-09-15T21:47:54.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3730000000000001e-06 2024-09-15T21:47:54.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901391+0000", 2024-09-15T21:47:54.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003553601 2024-09-15T21:47:54.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.259 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901466+0000", 2024-09-15T21:47:54.259 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5130999999999996e-05 2024-09-15T21:47:54.259 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.259 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.259 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901601+0000", 2024-09-15T21:47:54.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134797 2024-09-15T21:47:54.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903417+0000", 2024-09-15T21:47:54.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001816156 2024-09-15T21:47:54.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.260 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.260 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.260 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904741+0000", 2024-09-15T21:47:54.260 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013241049999999999 2024-09-15T21:47:54.260 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904767+0000", 2024-09-15T21:47:54.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5876000000000001e-05 2024-09-15T21:47:54.261 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.261 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.261 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.261 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904788+0000", 2024-09-15T21:47:54.261 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0449999999999999e-05 2024-09-15T21:47:54.261 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.262 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.262 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.262 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.262 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.262 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:54.262 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:54.262 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.274418065999996, 2024-09-15T21:47:54.262 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072970250000000004, 2024-09-15T21:47:54.262 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.262 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.262 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.262 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:54.262 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:54.262 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:54.263 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 139 2024-09-15T21:47:54.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.263 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:54.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.263 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.263 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.263 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:54.263 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.263 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897621+0000", 2024-09-15T21:47:54.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3099999999999999e-07 2024-09-15T21:47:54.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897625+0000", 2024-09-15T21:47:54.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.698e-06 2024-09-15T21:47:54.264 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.264 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.264 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.264 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897848+0000", 2024-09-15T21:47:54.264 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022375699999999999 2024-09-15T21:47:54.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897853+0000", 2024-09-15T21:47:54.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7709999999999997e-06 2024-09-15T21:47:54.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901929+0000", 2024-09-15T21:47:54.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040757780000000004 2024-09-15T21:47:54.265 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.265 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.265 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.265 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901976+0000", 2024-09-15T21:47:54.265 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6751e-05 2024-09-15T21:47:54.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902054+0000", 2024-09-15T21:47:54.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8856999999999994e-05 2024-09-15T21:47:54.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.266 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.266 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903421+0000", 2024-09-15T21:47:54.266 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00136619 2024-09-15T21:47:54.266 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.266 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904827+0000", 2024-09-15T21:47:54.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0014067369999999999 2024-09-15T21:47:54.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904881+0000", 2024-09-15T21:47:54.267 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3520999999999998e-05 2024-09-15T21:47:54.267 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.267 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.267 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.267 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904918+0000", 2024-09-15T21:47:54.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6634000000000002e-05 2024-09-15T21:47:54.268 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.268 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.268 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.268 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.268 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.268 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:54.268 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:54.268 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.274412237000007, 2024-09-15T21:47:54.268 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097221730000000006, 2024-09-15T21:47:54.268 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.268 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.268 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.269 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:54.269 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:54.269 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:54.269 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 140 2024-09-15T21:47:54.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.269 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.269 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.269 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:54.269 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.269 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.269 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:54.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897627+0000", 2024-09-15T21:47:54.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-15T21:47:54.270 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.270 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.270 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.270 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897698+0000", 2024-09-15T21:47:54.270 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1722000000000002e-05 2024-09-15T21:47:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897863+0000", 2024-09-15T21:47:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016479099999999999 2024-09-15T21:47:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897868+0000", 2024-09-15T21:47:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4950000000000002e-06 2024-09-15T21:47:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.271 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902335+0000", 2024-09-15T21:47:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004467632 2024-09-15T21:47:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902414+0000", 2024-09-15T21:47:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8688000000000003e-05 2024-09-15T21:47:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.272 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902535+0000", 2024-09-15T21:47:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012066900000000001 2024-09-15T21:47:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903714+0000", 2024-09-15T21:47:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011794520000000001 2024-09-15T21:47:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907115+0000", 2024-09-15T21:47:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034008879999999999 2024-09-15T21:47:54.273 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907250+0000", 2024-09-15T21:47:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134657 2024-09-15T21:47:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907348+0000", 2024-09-15T21:47:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8838000000000005e-05 2024-09-15T21:47:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.274 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.274337156000001, 2024-09-15T21:47:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097618050000000001, 2024-09-15T21:47:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:54.275 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 141 2024-09-15T21:47:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.276 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897702+0000", 2024-09-15T21:47:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6199999999999998e-07 2024-09-15T21:47:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.277 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897705+0000", 2024-09-15T21:47:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0180000000000002e-06 2024-09-15T21:47:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897889+0000", 2024-09-15T21:47:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000183678 2024-09-15T21:47:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.278 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897894+0000", 2024-09-15T21:47:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8930000000000004e-06 2024-09-15T21:47:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903099+0000", 2024-09-15T21:47:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052056280000000003 2024-09-15T21:47:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903151+0000", 2024-09-15T21:47:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2163000000000002e-05 2024-09-15T21:47:54.279 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903220+0000", 2024-09-15T21:47:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8483000000000001e-05 2024-09-15T21:47:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904794+0000", 2024-09-15T21:47:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001574521 2024-09-15T21:47:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.280 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907405+0000", 2024-09-15T21:47:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026110339999999999 2024-09-15T21:47:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907440+0000", 2024-09-15T21:47:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4564999999999999e-05 2024-09-15T21:47:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.281 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907463+0000", 2024-09-15T21:47:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3260000000000001e-05 2024-09-15T21:47:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:22 2.8 2:1470b2f8:::613.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.716896829000007, 2024-09-15T21:47:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007454937, 2024-09-15T21:47:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.283 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:54.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:54.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.283 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.283 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:54.283 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6800000000000002e-07 2024-09-15T21:47:54.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455145+0000", 2024-09-15T21:47:54.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4470000000000002e-06 2024-09-15T21:47:54.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.284 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456425+0000", 2024-09-15T21:47:54.284 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012798200000000001 2024-09-15T21:47:54.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.284 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456433+0000", 2024-09-15T21:47:54.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7389999999999999e-06 2024-09-15T21:47:54.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.285 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:54.285 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097865 2024-09-15T21:47:54.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.285 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457614+0000", 2024-09-15T21:47:54.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3560000000000006e-05 2024-09-15T21:47:54.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457998+0000", 2024-09-15T21:47:54.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00038372699999999999 2024-09-15T21:47:54.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.286 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.286 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460488+0000", 2024-09-15T21:47:54.286 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002489855 2024-09-15T21:47:54.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462512+0000", 2024-09-15T21:47:54.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020235269999999998 2024-09-15T21:47:54.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462554+0000", 2024-09-15T21:47:54.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2472000000000003e-05 2024-09-15T21:47:54.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.287 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.287 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462597+0000", 2024-09-15T21:47:54.287 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2657000000000002e-05 2024-09-15T21:47:54.288 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.288 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.288 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.288 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:2 2.5 2:a6e36b3f:::401.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:54.288 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:54.288 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 79.716891860000004, 2024-09-15T21:47:54.288 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074484590000000002, 2024-09-15T21:47:54.288 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.288 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.288 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.288 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:54.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.289 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:54.289 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.289 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:54.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:47:54.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455150+0000", 2024-09-15T21:47:54.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7319999999999998e-06 2024-09-15T21:47:54.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.290 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.290 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456449+0000", 2024-09-15T21:47:54.290 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012991490000000001 2024-09-15T21:47:54.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456454+0000", 2024-09-15T21:47:54.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0390000000000004e-06 2024-09-15T21:47:54.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456513+0000", 2024-09-15T21:47:54.291 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8768999999999999e-05 2024-09-15T21:47:54.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.291 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.291 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456628+0000", 2024-09-15T21:47:54.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115641 2024-09-15T21:47:54.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456741+0000", 2024-09-15T21:47:54.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000113094 2024-09-15T21:47:54.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.292 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.292 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458520+0000", 2024-09-15T21:47:54.292 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001778993 2024-09-15T21:47:54.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462494+0000", 2024-09-15T21:47:54.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039737009999999996 2024-09-15T21:47:54.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462549+0000", 2024-09-15T21:47:54.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4537000000000003e-05 2024-09-15T21:47:54.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.293 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.293 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462595+0000", 2024-09-15T21:47:54.293 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6511000000000002e-05 2024-09-15T21:47:54.294 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.294 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.294 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.294 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.294 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.294 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:27 2.5 2:a5fa685f:::504.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:54.294 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:54.294 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 62.607199475999998, 2024-09-15T21:47:54.294 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071804599999999996, 2024-09-15T21:47:54.295 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:54.295 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:54.295 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:54.295 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:54.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:54.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:54.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.295 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.295 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.295 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:54.295 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:54.295 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:54.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:54.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:54.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-15T21:47:54.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:54.296 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564841+0000", 2024-09-15T21:47:54.296 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-15T21:47:54.296 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.296 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.296 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:54.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565126+0000", 2024-09-15T21:47:54.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028469900000000002 2024-09-15T21:47:54.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:54.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565133+0000", 2024-09-15T21:47:54.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9480000000000002e-06 2024-09-15T21:47:54.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.297 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:54.297 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568006+0000", 2024-09-15T21:47:54.297 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028730330000000001 2024-09-15T21:47:54.297 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.297 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:54.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568119+0000", 2024-09-15T21:47:54.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011325899999999999 2024-09-15T21:47:54.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:54.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568235+0000", 2024-09-15T21:47:54.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115868 2024-09-15T21:47:54.298 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.298 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.298 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:54.298 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569668+0000", 2024-09-15T21:47:54.298 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001433099 2024-09-15T21:47:54.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:54.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571926+0000", 2024-09-15T21:47:54.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022572410000000001 2024-09-15T21:47:54.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:54.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571981+0000", 2024-09-15T21:47:54.299 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5383000000000002e-05 2024-09-15T21:47:54.299 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:54.299 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:54.299 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:54.299 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572020+0000", 2024-09-15T21:47:54.300 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8574e-05 2024-09-15T21:47:54.300 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.300 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.300 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.300 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:54.300 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:54.300 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:54.303 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:47:54.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.269+0000 7f9b588ba640 1 Processor -- start 2024-09-15T21:47:54.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.269+0000 7f9b588ba640 1 -- start start 2024-09-15T21:47:54.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.271+0000 7f9b588ba640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9b54155670 0x7f9b5415e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:54.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.271+0000 7f9b588ba640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9b5415bb60 0x7f9b5406d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:54.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.271+0000 7f9b588ba640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b5415edb0 con 0x7f9b54155290 2024-09-15T21:47:54.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.271+0000 7f9b588ba640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f9b5415f090 con 0x7f9b54155670 2024-09-15T21:47:54.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.271+0000 7f9b588ba640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f9b5415f370 con 0x7f9b5415bb60 2024-09-15T21:47:54.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.271+0000 7f9b51d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9b54155670 0x7f9b5415e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:54.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.271+0000 7f9b51d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9b54155670 0x7f9b5415e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57558/0 (socket says 172.21.15.100:57558) 2024-09-15T21:47:54.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.271+0000 7f9b51d9b640 1 -- 172.21.15.100:0/3449646882 learned_addr learned my addr 172.21.15.100:0/3449646882 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:54.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.271+0000 7f9b52d9d640 1 --2- 172.21.15.100:0/3449646882 >> v2:172.21.15.122:3300/0 conn(0x7f9b5415bb60 0x7f9b5406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:54.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.271+0000 7f9b51d9b640 1 -- 172.21.15.100:0/3449646882 >> v2:172.21.15.122:3300/0 conn(0x7f9b5415bb60 msgr2=0x7f9b5406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:54.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.271+0000 7f9b51d9b640 1 --2- 172.21.15.100:0/3449646882 >> v2:172.21.15.122:3300/0 conn(0x7f9b5415bb60 0x7f9b5406d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:54.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.271+0000 7f9b51d9b640 1 -- 172.21.15.100:0/3449646882 >> v1:172.21.15.122:6789/0 conn(0x7f9b54155290 legacy=0x7f9b540c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:54.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.271+0000 7f9b51d9b640 1 -- 172.21.15.100:0/3449646882 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b540c5db0 con 0x7f9b54155670 2024-09-15T21:47:54.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.272+0000 7f9b51d9b640 1 --2- 172.21.15.100:0/3449646882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b54155670 0x7f9b5415e690 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9b540cde70 tx=0x7f9b4802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab822cd730994f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:54.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.272+0000 7f9b5159a640 1 -- 172.21.15.100:0/3449646882 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48037070 con 0x7f9b54155670 2024-09-15T21:47:54.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.272+0000 7f9b5159a640 1 -- 172.21.15.100:0/3449646882 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b48032440 con 0x7f9b54155670 2024-09-15T21:47:54.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.272+0000 7f9b5159a640 1 -- 172.21.15.100:0/3449646882 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48032810 con 0x7f9b54155670 2024-09-15T21:47:54.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.272+0000 7f9b588ba640 1 -- 172.21.15.100:0/3449646882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b54155670 msgr2=0x7f9b5415e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:54.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.272+0000 7f9b588ba640 1 --2- 172.21.15.100:0/3449646882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b54155670 0x7f9b5415e690 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9b540cde70 tx=0x7f9b4802a270 comp rx=0 tx=0).stop 2024-09-15T21:47:54.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.273+0000 7f9b588ba640 1 -- 172.21.15.100:0/3449646882 shutdown_connections 2024-09-15T21:47:54.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.273+0000 7f9b588ba640 1 --2- 172.21.15.100:0/3449646882 >> v2:172.21.15.122:3300/0 conn(0x7f9b5415bb60 0x7f9b5406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:54.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.273+0000 7f9b588ba640 1 --2- 172.21.15.100:0/3449646882 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b54155670 0x7f9b5415e690 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:54.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.273+0000 7f9b588ba640 1 -- 172.21.15.100:0/3449646882 >> 172.21.15.100:0/3449646882 conn(0x7f9b54073310 msgr2=0x7f9b54073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:54.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.273+0000 7f9b588ba640 1 -- 172.21.15.100:0/3449646882 shutdown_connections 2024-09-15T21:47:54.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.273+0000 7f9b588ba640 1 -- 172.21.15.100:0/3449646882 wait complete. 2024-09-15T21:47:54.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.275+0000 7f9b588ba640 1 Processor -- start 2024-09-15T21:47:54.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.275+0000 7f9b588ba640 1 -- start start 2024-09-15T21:47:54.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.276+0000 7f9b588ba640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9b54155290 0x7f9b540d1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:54.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.276+0000 7f9b588ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b54155670 0x7f9b5414e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:54.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.276+0000 7f9b588ba640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b5415bb60 0x7f9b5414ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:54.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.276+0000 7f9b588ba640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5415c590 con 0x7f9b5415bb60 2024-09-15T21:47:54.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.276+0000 7f9b588ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9b540b39e0 con 0x7f9b54155670 2024-09-15T21:47:54.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.276+0000 7f9b588ba640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9b540c6cc0 con 0x7f9b54155290 2024-09-15T21:47:54.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.276+0000 7f9b51d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b54155670 0x7f9b5414e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:54.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.276+0000 7f9b51d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b54155670 0x7f9b5414e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:34216/0 (socket says 172.21.15.100:34216) 2024-09-15T21:47:54.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.276+0000 7f9b51d9b640 1 -- 172.21.15.100:0/427499224 learned_addr learned my addr 172.21.15.100:0/427499224 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:54.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.276+0000 7f9b5259c640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9b54155290 0x7f9b540d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:54.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.277+0000 7f9b52d9d640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b5415bb60 0x7f9b5414ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:54.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.277+0000 7f9b51d9b640 1 -- 172.21.15.100:0/427499224 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9b54155290 msgr2=0x7f9b540d1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:54.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.277+0000 7f9b51d9b640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9b54155290 0x7f9b540d1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:54.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.277+0000 7f9b51d9b640 1 -- 172.21.15.100:0/427499224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b5415bb60 msgr2=0x7f9b5414ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:54.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.277+0000 7f9b51d9b640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9b5415bb60 0x7f9b5414ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:54.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.277+0000 7f9b51d9b640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b54156ed0 con 0x7f9b54155670 2024-09-15T21:47:54.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.277+0000 7f9b5259c640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9b54155290 0x7f9b540d1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:54.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.277+0000 7f9b51d9b640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b54155670 0x7f9b5414e590 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9b48000c00 tx=0x7f9b480046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:54.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.277+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48041020 con 0x7f9b54155670 2024-09-15T21:47:54.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.277+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b480306c0 con 0x7f9b54155670 2024-09-15T21:47:54.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.277+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48037040 con 0x7f9b54155670 2024-09-15T21:47:54.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.277+0000 7f9b588ba640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b540cf0e0 con 0x7f9b54155670 2024-09-15T21:47:54.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.278+0000 7f9b588ba640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b540cfda0 con 0x7f9b54155670 2024-09-15T21:47:54.312 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.279+0000 7f9b588ba640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f9b5415bf70 con 0x7f9b54155670 2024-09-15T21:47:54.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.279+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9b48044020 con 0x7f9b54155670 2024-09-15T21:47:54.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.279+0000 7f9b42ffd640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9b3406c2b0 0x7f9b3406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:54.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.279+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5886+0+0 (secure 0 0 0) 0x7f9b480ab970 con 0x7f9b54155670 2024-09-15T21:47:54.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.280+0000 7f9b5259c640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9b3406c2b0 0x7f9b3406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:54.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.280+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v44) ==== 54+0+16 (secure 0 0 0) 0x7f9b4804b070 con 0x7f9b54155670 2024-09-15T21:47:54.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.280+0000 7f9b588ba640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f9b540cffe0 0x7f9b5415c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:54.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.280+0000 7f9b588ba640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b54155a50 con 0x7f9b540cffe0 2024-09-15T21:47:54.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.280+0000 7f9b5259c640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9b3406c2b0 0x7f9b3406e770 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9b44000f60 tx=0x7f9b44009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:54.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.280+0000 7f9b52d9d640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f9b540cffe0 0x7f9b5415c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:54.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.281+0000 7f9b52d9d640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f9b540cffe0 0x7f9b5415c710 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:54.313 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.281+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b54155a50 con 0x7f9b540cffe0 2024-09-15T21:47:54.314 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:54.300+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:54.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.308+0000 7f9b588ba640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9b540b31f0 con 0x7f9b540cffe0 2024-09-15T21:47:54.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.311+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9b540b31f0 con 0x7f9b540cffe0 2024-09-15T21:47:54.315 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:54.316 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:54.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.311+0000 7f9b588ba640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f9b540ce960 0x7f9b54158580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:54.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.311+0000 7f9b588ba640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f9b54073c10 con 0x7f9b540ce960 2024-09-15T21:47:54.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.311+0000 7f9b5259c640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f9b540ce960 0x7f9b54158580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:54.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.312+0000 7f9b5259c640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f9b540ce960 0x7f9b54158580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:54.316 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.313+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b54073c10 con 0x7f9b540ce960 2024-09-15T21:47:54.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.340+0000 7f9b588ba640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9b5415de80 con 0x7f9b540ce960 2024-09-15T21:47:54.344 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.342+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9b5415de80 con 0x7f9b540ce960 2024-09-15T21:47:54.344 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:54.345 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:54.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.342+0000 7f9b588ba640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f9b541f4bd0 0x7f9b541f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:54.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.342+0000 7f9b588ba640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f9b5415d8c0 con 0x7f9b541f4bd0 2024-09-15T21:47:54.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.343+0000 7f9b51d9b640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f9b541f4bd0 0x7f9b541f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:54.345 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.343+0000 7f9b51d9b640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f9b541f4bd0 0x7f9b541f5010 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:54.346 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.344+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b5415d8c0 con 0x7f9b541f4bd0 2024-09-15T21:47:54.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.371+0000 7f9b588ba640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9b54062630 con 0x7f9b541f4bd0 2024-09-15T21:47:54.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.373+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9b54062630 con 0x7f9b541f4bd0 2024-09-15T21:47:54.376 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:54.376 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:54.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.374+0000 7f9b588ba640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f9b54063110 0x7f9b5414f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:54.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.374+0000 7f9b588ba640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f9b54063510 con 0x7f9b54063110 2024-09-15T21:47:54.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.374+0000 7f9b52d9d640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f9b54063110 0x7f9b5414f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:54.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.375+0000 7f9b52d9d640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f9b54063110 0x7f9b5414f510 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:54.377 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.376+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b54063510 con 0x7f9b54063110 2024-09-15T21:47:54.402 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:54.401+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:54.404 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.402+0000 7f9b588ba640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9b54063510 con 0x7f9b54063110 2024-09-15T21:47:54.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.405+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9b54063510 con 0x7f9b54063110 2024-09-15T21:47:54.407 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:54.407 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:54.407 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.405+0000 7f9b588ba640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f9b541501b0 0x7f9b5402c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.405+0000 7f9b588ba640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f9b54063510 con 0x7f9b541501b0 2024-09-15T21:47:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.406+0000 7f9b5259c640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f9b541501b0 0x7f9b5402c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.406+0000 7f9b5259c640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f9b541501b0 0x7f9b5402c640 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:54.408 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.407+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b54063510 con 0x7f9b541501b0 2024-09-15T21:47:54.423 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:54.423 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:54.432 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:47:54.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.433+0000 7f9b588ba640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9b54063510 con 0x7f9b541501b0 2024-09-15T21:47:54.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.435+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9b54063510 con 0x7f9b541501b0 2024-09-15T21:47:54.437 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.435+0000 7f9b588ba640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=45}) -- 0x7f9b542071b0 con 0x7f9b54155670 2024-09-15T21:47:54.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.435+0000 7f9b588ba640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b54063510 con 0x7f9b54155670 2024-09-15T21:47:54.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.436+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7f9b48079940 con 0x7f9b54155670 2024-09-15T21:47:54.439 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:47:54.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.436+0000 7f9b588ba640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f9b5402d450 0x7f9b54205aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:54.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.436+0000 7f9b588ba640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f9b5402da10 con 0x7f9b5402d450 2024-09-15T21:47:54.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.436+0000 7f9b51d9b640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f9b5402d450 0x7f9b54205aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:54.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.437+0000 7f9b51d9b640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f9b5402d450 0x7f9b54205aa0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:54.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.437+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b5402da10 con 0x7f9b5402d450 2024-09-15T21:47:54.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.463+0000 7f9b588ba640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9b54074130 con 0x7f9b5402d450 2024-09-15T21:47:54.466 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.465+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9b54074130 con 0x7f9b5402d450 2024-09-15T21:47:54.467 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:54.467 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:54.467 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.465+0000 7f9b588ba640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f9b542064f0 0x7f9b54206950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:54.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.465+0000 7f9b588ba640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f9b5420dc30 con 0x7f9b542064f0 2024-09-15T21:47:54.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.465+0000 7f9b52d9d640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f9b542064f0 0x7f9b54206950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:54.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.466+0000 7f9b52d9d640 1 --2- 172.21.15.100:0/427499224 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f9b542064f0 0x7f9b54206950 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:54.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.466+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b5420dc30 con 0x7f9b542064f0 2024-09-15T21:47:54.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.493+0000 7f9b588ba640 1 -- 172.21.15.100:0/427499224 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f9b5420b3a0 con 0x7f9b542064f0 2024-09-15T21:47:54.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.495+0000 7f9b42ffd640 1 -- 172.21.15.100:0/427499224 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f9b5420b3a0 con 0x7f9b542064f0 2024-09-15T21:47:54.497 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:54.497 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:54.501 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:47:54.502 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:47:54.503 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:54.501+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:54.567 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:54.567 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:54.578 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:47:54.623 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:54.621+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:54.684 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:54.684 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:54.693 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:47:54.736 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:54.734+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:54.806 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:54.804+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:54.842 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:54.842 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:54.852 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:47:54.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.901+0000 7f1fd8ff9640 1 -- 172.21.15.100:0/4273327908 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v45) ==== 97+0+0 (secure 0 0 0) 0x7f1fd007dc60 con 0x7f1fdc0cb7d0 2024-09-15T21:47:54.903 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is set 2024-09-15T21:47:54.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.903+0000 7f1fe24fa640 1 -- 172.21.15.100:0/4273327908 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1fc806c2e0 msgr2=0x7f1fc806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:54.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.903+0000 7f1fe24fa640 1 --2- 172.21.15.100:0/4273327908 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1fc806c2e0 0x7f1fc806e7a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc009560 tx=0x7f1fcc009190 comp rx=0 tx=0).stop 2024-09-15T21:47:54.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.903+0000 7f1fe24fa640 1 -- 172.21.15.100:0/4273327908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fdc0cb7d0 msgr2=0x7f1fdc149bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:54.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.903+0000 7f1fe24fa640 1 --2- 172.21.15.100:0/4273327908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fdc0cb7d0 0x7f1fdc149bf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0031000 tx=0x7f1fd00046c0 comp rx=0 tx=0).stop 2024-09-15T21:47:54.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.903+0000 7f1fe24fa640 1 -- 172.21.15.100:0/4273327908 shutdown_connections 2024-09-15T21:47:54.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.903+0000 7f1fe24fa640 1 --2- 172.21.15.100:0/4273327908 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1fc806c2e0 0x7f1fc806e7a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:54.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.903+0000 7f1fe24fa640 1 --2- 172.21.15.100:0/4273327908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fdc0cb7d0 0x7f1fdc149bf0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:54.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.903+0000 7f1fe24fa640 1 --2- 172.21.15.100:0/4273327908 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1fdc155440 0x7f1fdc1f0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:54.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.903+0000 7f1fe24fa640 1 --2- 172.21.15.100:0/4273327908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fdc06d0e0 0x7f1fdc0c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:54.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.903+0000 7f1fe24fa640 1 -- 172.21.15.100:0/4273327908 >> 172.21.15.100:0/4273327908 conn(0x7f1fdc0680d0 msgr2=0x7f1fdc0c5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:54.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.903+0000 7f1fe24fa640 1 -- 172.21.15.100:0/4273327908 shutdown_connections 2024-09-15T21:47:54.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:54.903+0000 7f1fe24fa640 1 -- 172.21.15.100:0/4273327908 wait complete. 2024-09-15T21:47:54.907 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:54.905+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:54.962 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:54.962 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:54.971 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:47:55.013 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:55.011+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:55.109 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:55.108+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:55.144 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:55.144 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:55.153 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:47:55.237 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:55.236+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:55.300 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:55.300 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:55.310 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:47:55.311 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:55.310+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:55.412 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:55.411+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:55.455 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:55.456 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:55.465 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:47:55.514 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:55.512+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:55.615 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:55.613+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:55.616 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:55.616 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:55.626 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:47:55.715 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:55.713+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:55.731 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:55.731 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:55.740 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:47:55.816 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:55.815+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:55.888 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:55.888 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:55.897 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:47:55.917 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:55.915+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:56.004 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:56.005 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:56.014 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:47:56.017 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:56.016+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:56.119 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:56.117+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:56.127 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:56.127 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:56.137 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:47:56.220 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:56.219+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:56.281 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:56.281 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:56.290 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:47:56.332 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:56.330+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:56.422 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:56.420+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:56.434 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:56.434 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:56.443 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:47:56.503 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:47:56.546 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:56.547 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:56.547 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:56.546+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:56.555 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:47:56.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.610+0000 7f4c8291f640 1 Processor -- start 2024-09-15T21:47:56.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.610+0000 7f4c8291f640 1 -- start start 2024-09-15T21:47:56.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.610+0000 7f4c8291f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4c7c155290 0x7f4c7c15e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:56.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.610+0000 7f4c8291f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c7c155670 0x7f4c7c15bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:56.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.611+0000 7f4c8291f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c7c15edb0 con 0x7f4c7c0c65b0 2024-09-15T21:47:56.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.611+0000 7f4c8291f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f4c7c15c0a0 con 0x7f4c7c155290 2024-09-15T21:47:56.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.611+0000 7f4c8291f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4c7c15f090 con 0x7f4c7c155670 2024-09-15T21:47:56.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.611+0000 7f4c80ebc640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f4c7c0c65b0 0x7f4c7c06d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57398/0 (socket says 172.21.15.100:57398) 2024-09-15T21:47:56.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.611+0000 7f4c80ebc640 1 -- 172.21.15.100:0/2966107463 learned_addr learned my addr 172.21.15.100:0/2966107463 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:56.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.611+0000 7f4c7bfff640 1 --2- 172.21.15.100:0/2966107463 >> v2:172.21.15.100:3300/0 conn(0x7f4c7c155290 0x7f4c7c15e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:56.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.611+0000 7f4c7b7fe640 1 --2- 172.21.15.100:0/2966107463 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c7c155670 0x7f4c7c15bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:56.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.611+0000 7f4c7b7fe640 1 -- 172.21.15.100:0/2966107463 >> v2:172.21.15.100:3300/0 conn(0x7f4c7c155290 msgr2=0x7f4c7c15e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:56.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.611+0000 7f4c7b7fe640 1 --2- 172.21.15.100:0/2966107463 >> v2:172.21.15.100:3300/0 conn(0x7f4c7c155290 0x7f4c7c15e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:56.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.611+0000 7f4c7b7fe640 1 -- 172.21.15.100:0/2966107463 >> v1:172.21.15.100:6789/0 conn(0x7f4c7c0c65b0 legacy=0x7f4c7c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:56.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.611+0000 7f4c7affd640 1 -- 172.21.15.100:0/2966107463 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2379110425 0 0) 0x7f4c7c15edb0 con 0x7f4c7c0c65b0 2024-09-15T21:47:56.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.611+0000 7f4c7b7fe640 1 -- 172.21.15.100:0/2966107463 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7c15f370 con 0x7f4c7c155670 2024-09-15T21:47:56.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.611+0000 7f4c7bfff640 1 --2- 172.21.15.100:0/2966107463 >> v2:172.21.15.100:3300/0 conn(0x7f4c7c155290 0x7f4c7c15e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:56.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.612+0000 7f4c7b7fe640 1 --2- 172.21.15.100:0/2966107463 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c7c155670 0x7f4c7c15bb60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4c700097a0 tx=0x7f4c7002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9818e62a34fcaf58 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:56.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.613+0000 7f4c7affd640 1 -- 172.21.15.100:0/2966107463 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c70037070 con 0x7f4c7c155670 2024-09-15T21:47:56.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.613+0000 7f4c7affd640 1 -- 172.21.15.100:0/2966107463 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c70032440 con 0x7f4c7c155670 2024-09-15T21:47:56.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.613+0000 7f4c7affd640 1 -- 172.21.15.100:0/2966107463 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c70032810 con 0x7f4c7c155670 2024-09-15T21:47:56.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.613+0000 7f4c8291f640 1 -- 172.21.15.100:0/2966107463 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c7c155670 msgr2=0x7f4c7c15bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:56.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.613+0000 7f4c8291f640 1 --2- 172.21.15.100:0/2966107463 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c7c155670 0x7f4c7c15bb60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4c700097a0 tx=0x7f4c7002a180 comp rx=0 tx=0).stop 2024-09-15T21:47:56.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.614+0000 7f4c8291f640 1 -- 172.21.15.100:0/2966107463 shutdown_connections 2024-09-15T21:47:56.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.614+0000 7f4c8291f640 1 --2- 172.21.15.100:0/2966107463 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c7c155670 0x7f4c7c15bb60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:56.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.614+0000 7f4c8291f640 1 --2- 172.21.15.100:0/2966107463 >> v2:172.21.15.100:3300/0 conn(0x7f4c7c155290 0x7f4c7c15e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:56.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.614+0000 7f4c8291f640 1 -- 172.21.15.100:0/2966107463 >> 172.21.15.100:0/2966107463 conn(0x7f4c7c073310 msgr2=0x7f4c7c073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:56.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.614+0000 7f4c8291f640 1 -- 172.21.15.100:0/2966107463 shutdown_connections 2024-09-15T21:47:56.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.614+0000 7f4c8291f640 1 -- 172.21.15.100:0/2966107463 wait complete. 2024-09-15T21:47:56.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.615+0000 7f4c8291f640 1 Processor -- start 2024-09-15T21:47:56.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.616+0000 7f4c8291f640 1 -- start start 2024-09-15T21:47:56.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.617+0000 7f4c8291f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c7c0c65b0 0x7f4c7c14e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:56.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.617+0000 7f4c8291f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c7c155290 0x7f4c7c0b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:56.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.617+0000 7f4c8291f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c7c155670 0x7f4c7c149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:56.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.617+0000 7f4c8291f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c0b39e0 con 0x7f4c7c155290 2024-09-15T21:47:56.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.617+0000 7f4c8291f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4c7c0c6090 con 0x7f4c7c155670 2024-09-15T21:47:56.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.617+0000 7f4c8291f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4c7c15c590 con 0x7f4c7c0c65b0 2024-09-15T21:47:56.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.617+0000 7f4c7b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c7c155290 0x7f4c7c0b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:56.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.617+0000 7f4c7bfff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c7c0c65b0 0x7f4c7c14e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:56.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.617+0000 7f4c7b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c7c155290 0x7f4c7c0b5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57662/0 (socket says 172.21.15.100:57662) 2024-09-15T21:47:56.621 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.617+0000 7f4c7b7fe640 1 -- 172.21.15.100:0/430280603 learned_addr learned my addr 172.21.15.100:0/430280603 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:56.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.617+0000 7f4c80ebc640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c7c155670 0x7f4c7c149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:56.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.617+0000 7f4c7bfff640 1 -- 172.21.15.100:0/430280603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c7c155670 msgr2=0x7f4c7c149770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:56.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.617+0000 7f4c7bfff640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c7c155670 0x7f4c7c149770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:56.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.617+0000 7f4c7bfff640 1 -- 172.21.15.100:0/430280603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c7c155290 msgr2=0x7f4c7c0b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:56.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.617+0000 7f4c7bfff640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c7c155290 0x7f4c7c0b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:56.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.617+0000 7f4c7bfff640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7c14b3d0 con 0x7f4c7c0c65b0 2024-09-15T21:47:56.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.617+0000 7f4c7b7fe640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4c7c155290 0x7f4c7c0b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:56.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.617+0000 7f4c80ebc640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c7c155670 0x7f4c7c149770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:56.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.618+0000 7f4c7bfff640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4c7c0c65b0 0x7f4c7c14e080 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00e930 tx=0x7f4c6c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:56.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.618+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c017070 con 0x7f4c7c0c65b0 2024-09-15T21:47:56.622 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.618+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c6c00ca00 con 0x7f4c7c0c65b0 2024-09-15T21:47:56.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.618+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c00cdf0 con 0x7f4c7c0c65b0 2024-09-15T21:47:56.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.619+0000 7f4c8291f640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7c148a60 con 0x7f4c7c0c65b0 2024-09-15T21:47:56.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.619+0000 7f4c8291f640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c7c14dc60 con 0x7f4c7c0c65b0 2024-09-15T21:47:56.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.619+0000 7f4c8291f640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f4c7c15bb60 con 0x7f4c7c0c65b0 2024-09-15T21:47:56.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.620+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4c6c01a070 con 0x7f4c7c0c65b0 2024-09-15T21:47:56.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.621+0000 7f4c78ff9640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4c6806c2b0 0x7f4c6806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:56.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.621+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(45..45 src has 1..45) ==== 5886+0+0 (secure 0 0 0) 0x7f4c6c08afe0 con 0x7f4c7c0c65b0 2024-09-15T21:47:56.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.621+0000 7f4c7b7fe640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4c6806c2b0 0x7f4c6806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:56.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.621+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v45) ==== 54+0+16 (secure 0 0 0) 0x7f4c6c08d600 con 0x7f4c7c0c65b0 2024-09-15T21:47:56.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.621+0000 7f4c627fc640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f4c54001650 0x7f4c54003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:56.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.621+0000 7f4c7b7fe640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4c6806c2b0 0x7f4c6806e770 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4c7002a1c0 tx=0x7f4c7002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:56.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.622+0000 7f4c80ebc640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f4c54001650 0x7f4c54003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:56.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.622+0000 7f4c627fc640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4c54006bb0 con 0x7f4c54001650 2024-09-15T21:47:56.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.622+0000 7f4c80ebc640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f4c54001650 0x7f4c54003b10 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:56.626 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.623+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c54006bb0 con 0x7f4c54001650 2024-09-15T21:47:56.626 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:56.622+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:56.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.637+0000 7f4c627fc640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4c54005c60 con 0x7f4c54001650 2024-09-15T21:47:56.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.638+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c54005c60 con 0x7f4c54001650 2024-09-15T21:47:56.642 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:56.643 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:56.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.639+0000 7f4c627fc640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f4c54004190 0x7f4c54004640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:56.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.639+0000 7f4c627fc640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f4c54004c00 con 0x7f4c54004190 2024-09-15T21:47:56.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.639+0000 7f4c7bfff640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f4c54004190 0x7f4c54004640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:56.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.639+0000 7f4c7bfff640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f4c54004190 0x7f4c54004640 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:56.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.640+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c54004c00 con 0x7f4c54004190 2024-09-15T21:47:56.660 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:56.661 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:56.668 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:47:56.669 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.667+0000 7f4c627fc640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4c54005140 con 0x7f4c54004190 2024-09-15T21:47:56.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.668+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c54005140 con 0x7f4c54004190 2024-09-15T21:47:56.670 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:56.671 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:56.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.669+0000 7f4c627fc640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f4c54009370 0x7f4c5400b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:56.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.669+0000 7f4c627fc640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f4c5400e6c0 con 0x7f4c54009370 2024-09-15T21:47:56.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.669+0000 7f4c7b7fe640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f4c54009370 0x7f4c5400b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:56.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.669+0000 7f4c7b7fe640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f4c54009370 0x7f4c5400b760 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:56.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.670+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c5400e6c0 con 0x7f4c54009370 2024-09-15T21:47:56.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.696+0000 7f4c627fc640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4c5400d6b0 con 0x7f4c54009370 2024-09-15T21:47:56.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.697+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c5400d6b0 con 0x7f4c54009370 2024-09-15T21:47:56.699 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.698+0000 7f4c627fc640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f4c5400e130 0x7f4c5400bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.698+0000 7f4c627fc640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f4c5400c480 con 0x7f4c5400e130 2024-09-15T21:47:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.698+0000 7f4c80ebc640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f4c5400e130 0x7f4c5400bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.698+0000 7f4c80ebc640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f4c5400e130 0x7f4c5400bca0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:56.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.699+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c5400c480 con 0x7f4c5400e130 2024-09-15T21:47:56.725 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:56.724+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:56.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.725+0000 7f4c627fc640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4c5400c990 con 0x7f4c5400e130 2024-09-15T21:47:56.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.727+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c5400c990 con 0x7f4c5400e130 2024-09-15T21:47:56.729 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.728+0000 7f4c627fc640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f4c5400cf90 0x7f4c54012890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.728+0000 7f4c627fc640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f4c54015730 con 0x7f4c5400cf90 2024-09-15T21:47:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.728+0000 7f4c7bfff640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f4c5400cf90 0x7f4c54012890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:56.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.728+0000 7f4c7bfff640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f4c5400cf90 0x7f4c54012890 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:56.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.729+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c54015730 con 0x7f4c5400cf90 2024-09-15T21:47:56.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.742+0000 7f4c627fc640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4c5400c700 con 0x7f4c5400cf90 2024-09-15T21:47:56.746 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.744+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c5400c700 con 0x7f4c5400cf90 2024-09-15T21:47:56.746 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:56.747 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:56.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.744+0000 7f4c627fc640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=46}) -- 0x7f4c540159f0 con 0x7f4c7c0c65b0 2024-09-15T21:47:56.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.744+0000 7f4c627fc640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4c54015bc0 con 0x7f4c7c0c65b0 2024-09-15T21:47:56.747 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.745+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4c6c021040 con 0x7f4c7c0c65b0 2024-09-15T21:47:56.747 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:47:56.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.745+0000 7f4c627fc640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f4c540136a0 0x7f4c54013af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:56.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.745+0000 7f4c627fc640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f4c54015480 con 0x7f4c540136a0 2024-09-15T21:47:56.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.745+0000 7f4c7b7fe640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f4c540136a0 0x7f4c54013af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:56.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.745+0000 7f4c7b7fe640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f4c540136a0 0x7f4c54013af0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:56.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.746+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c54015480 con 0x7f4c540136a0 2024-09-15T21:47:56.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.760+0000 7f4c627fc640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4c540141a0 con 0x7f4c540136a0 2024-09-15T21:47:56.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.761+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c540141a0 con 0x7f4c540136a0 2024-09-15T21:47:56.763 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:56.764 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:56.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.761+0000 7f4c627fc640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f4c540148c0 0x7f4c54014d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:56.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.761+0000 7f4c627fc640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f4c5401c7e0 con 0x7f4c540148c0 2024-09-15T21:47:56.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.762+0000 7f4c80ebc640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f4c540148c0 0x7f4c54014d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:56.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.762+0000 7f4c80ebc640 1 --2- 172.21.15.100:0/430280603 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f4c540148c0 0x7f4c54014d40 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:56.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.763+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c5401c7e0 con 0x7f4c540148c0 2024-09-15T21:47:56.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.776+0000 7f4c627fc640 1 -- 172.21.15.100:0/430280603 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7f4c5401b660 con 0x7f4c540148c0 2024-09-15T21:47:56.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:56.778+0000 7f4c78ff9640 1 -- 172.21.15.100:0/430280603 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f4c5401b660 con 0x7f4c540148c0 2024-09-15T21:47:56.780 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:56.780 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:56.785 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:47:56.785 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:47:56.815 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 88.398690823999999, 2024-09-15T21:47:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0050344960000000003, 2024-09-15T21:47:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.815 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.816 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:47:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0199999999999998e-07 2024-09-15T21:47:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413470+0000", 2024-09-15T21:47:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4329999999999998e-06 2024-09-15T21:47:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413579+0000", 2024-09-15T21:47:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108593 2024-09-15T21:47:56.817 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413587+0000", 2024-09-15T21:47:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3100000000000001e-06 2024-09-15T21:47:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413691+0000", 2024-09-15T21:47:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010357 2024-09-15T21:47:56.818 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413887+0000", 2024-09-15T21:47:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019666799999999999 2024-09-15T21:47:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414027+0000", 2024-09-15T21:47:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013996700000000001 2024-09-15T21:47:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416360+0000", 2024-09-15T21:47:56.819 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002332204 2024-09-15T21:47:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418437+0000", 2024-09-15T21:47:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002077279 2024-09-15T21:47:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418469+0000", 2024-09-15T21:47:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2359e-05 2024-09-15T21:47:56.820 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418502+0000", 2024-09-15T21:47:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2811000000000001e-05 2024-09-15T21:47:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:56.821 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 88.398686784000006, 2024-09-15T21:47:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0053538780000000003, 2024-09-15T21:47:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.822 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:47:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2000000000000001e-07 2024-09-15T21:47:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413474+0000", 2024-09-15T21:47:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-15T21:47:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.823 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413612+0000", 2024-09-15T21:47:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000137727 2024-09-15T21:47:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413618+0000", 2024-09-15T21:47:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.241e-06 2024-09-15T21:47:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414443+0000", 2024-09-15T21:47:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082447 2024-09-15T21:47:56.824 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414523+0000", 2024-09-15T21:47:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0059e-05 2024-09-15T21:47:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414653+0000", 2024-09-15T21:47:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013031699999999999 2024-09-15T21:47:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:56.825 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416366+0000", 2024-09-15T21:47:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001712917 2024-09-15T21:47:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418769+0000", 2024-09-15T21:47:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024031529999999999 2024-09-15T21:47:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418802+0000", 2024-09-15T21:47:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2925000000000001e-05 2024-09-15T21:47:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.826 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418825+0000", 2024-09-15T21:47:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3204e-05 2024-09-15T21:47:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 88.398678763000007, 2024-09-15T21:47:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059708640000000002, 2024-09-15T21:47:56.827 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:56.828 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:47:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7300000000000002e-07 2024-09-15T21:47:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413482+0000", 2024-09-15T21:47:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2359999999999999e-06 2024-09-15T21:47:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.829 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414323+0000", 2024-09-15T21:47:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00084095399999999999 2024-09-15T21:47:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414332+0000", 2024-09-15T21:47:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0880000000000001e-06 2024-09-15T21:47:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415439+0000", 2024-09-15T21:47:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011071869999999999 2024-09-15T21:47:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.830 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415554+0000", 2024-09-15T21:47:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114499 2024-09-15T21:47:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415682+0000", 2024-09-15T21:47:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012802500000000001 2024-09-15T21:47:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.417957+0000", 2024-09-15T21:47:56.831 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022746720000000002 2024-09-15T21:47:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419385+0000", 2024-09-15T21:47:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001428826 2024-09-15T21:47:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419420+0000", 2024-09-15T21:47:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4875000000000001e-05 2024-09-15T21:47:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.832 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419450+0000", 2024-09-15T21:47:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0128999999999999e-05 2024-09-15T21:47:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 88.398674944999996, 2024-09-15T21:47:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0087186220000000005, 2024-09-15T21:47:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.833 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:47:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.834 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413484+0000", 2024-09-15T21:47:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-15T21:47:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413487+0000", 2024-09-15T21:47:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.534e-06 2024-09-15T21:47:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.835 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414353+0000", 2024-09-15T21:47:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00086616999999999998 2024-09-15T21:47:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414359+0000", 2024-09-15T21:47:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3279999999999996e-06 2024-09-15T21:47:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416041+0000", 2024-09-15T21:47:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001682467 2024-09-15T21:47:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.836 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416158+0000", 2024-09-15T21:47:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117242 2024-09-15T21:47:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416284+0000", 2024-09-15T21:47:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012509000000000001 2024-09-15T21:47:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.417992+0000", 2024-09-15T21:47:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017087059999999999 2024-09-15T21:47:56.837 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422142+0000", 2024-09-15T21:47:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041501180000000004 2024-09-15T21:47:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422176+0000", 2024-09-15T21:47:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4109999999999997e-05 2024-09-15T21:47:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422202+0000", 2024-09-15T21:47:56.838 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5585e-05 2024-09-15T21:47:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 88.398669519999999, 2024-09-15T21:47:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0088000370000000001, 2024-09-15T21:47:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.839 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.840 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:47:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413491+0000", 2024-09-15T21:47:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.773e-06 2024-09-15T21:47:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414372+0000", 2024-09-15T21:47:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00088122000000000005 2024-09-15T21:47:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.841 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414378+0000", 2024-09-15T21:47:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4380000000000003e-06 2024-09-15T21:47:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416631+0000", 2024-09-15T21:47:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022530319999999999 2024-09-15T21:47:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416765+0000", 2024-09-15T21:47:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013423499999999999 2024-09-15T21:47:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.842 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422289+0000", 2024-09-15T21:47:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055240530000000001 2024-09-15T21:47:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.357101201000006, 2024-09-15T21:47:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056900609999999997, 2024-09-15T21:47:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:47:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.844 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455058+0000", 2024-09-15T21:47:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.51e-07 2024-09-15T21:47:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455061+0000", 2024-09-15T21:47:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.145e-06 2024-09-15T21:47:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.845 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455158+0000", 2024-09-15T21:47:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7101000000000004e-05 2024-09-15T21:47:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455165+0000", 2024-09-15T21:47:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1019999999999999e-06 2024-09-15T21:47:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456433+0000", 2024-09-15T21:47:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012678259999999999 2024-09-15T21:47:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.846 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456517+0000", 2024-09-15T21:47:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4373e-05 2024-09-15T21:47:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456677+0000", 2024-09-15T21:47:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015963300000000001 2024-09-15T21:47:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458636+0000", 2024-09-15T21:47:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019594030000000002 2024-09-15T21:47:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.847 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460665+0000", 2024-09-15T21:47:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020283549999999999 2024-09-15T21:47:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460707+0000", 2024-09-15T21:47:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2398000000000003e-05 2024-09-15T21:47:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460747+0000", 2024-09-15T21:47:56.848 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0274000000000001e-05 2024-09-15T21:47:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.357094007000001, 2024-09-15T21:47:56.849 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086420030000000005, 2024-09-15T21:47:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:47:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.850 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455065+0000", 2024-09-15T21:47:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3700000000000001e-07 2024-09-15T21:47:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455068+0000", 2024-09-15T21:47:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.915e-06 2024-09-15T21:47:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455366+0000", 2024-09-15T21:47:56.851 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00029825400000000002 2024-09-15T21:47:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455374+0000", 2024-09-15T21:47:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6240000000000008e-06 2024-09-15T21:47:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456199+0000", 2024-09-15T21:47:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082448500000000002 2024-09-15T21:47:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.852 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457034+0000", 2024-09-15T21:47:56.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00083553399999999997 2024-09-15T21:47:56.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:56.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457171+0000", 2024-09-15T21:47:56.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013653600000000001 2024-09-15T21:47:56.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.853 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:56.853 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462663+0000", 2024-09-15T21:47:56.853 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054924420000000002 2024-09-15T21:47:56.853 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.853 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:56.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463613+0000", 2024-09-15T21:47:56.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094971499999999996 2024-09-15T21:47:56.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:56.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463663+0000", 2024-09-15T21:47:56.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9636999999999999e-05 2024-09-15T21:47:56.854 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.854 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.854 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.854 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463706+0000", 2024-09-15T21:47:56.854 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3523999999999999e-05 2024-09-15T21:47:56.854 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.855 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.855 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.855 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.855 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:56.855 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:56.855 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.357089478999995, 2024-09-15T21:47:56.855 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090408420000000003, 2024-09-15T21:47:56.855 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.855 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.855 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:56.855 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.855 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:56.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:56.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.856 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.856 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.856 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:47:56.856 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.79e-07 2024-09-15T21:47:56.856 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455072+0000", 2024-09-15T21:47:56.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6189999999999998e-06 2024-09-15T21:47:56.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.857 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455403+0000", 2024-09-15T21:47:56.857 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033167299999999999 2024-09-15T21:47:56.857 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.857 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.857 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455420+0000", 2024-09-15T21:47:56.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6518999999999999e-05 2024-09-15T21:47:56.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457448+0000", 2024-09-15T21:47:56.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020275710000000001 2024-09-15T21:47:56.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.858 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.858 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.858 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:56.858 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3844999999999994e-05 2024-09-15T21:47:56.858 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:56.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457684+0000", 2024-09-15T21:47:56.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015239 2024-09-15T21:47:56.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:56.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462678+0000", 2024-09-15T21:47:56.859 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049939279999999999 2024-09-15T21:47:56.859 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.859 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.859 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:56.859 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464017+0000", 2024-09-15T21:47:56.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013397140000000001 2024-09-15T21:47:56.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:56.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464071+0000", 2024-09-15T21:47:56.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3735e-05 2024-09-15T21:47:56.860 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.860 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.860 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.860 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464110+0000", 2024-09-15T21:47:56.860 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8569000000000003e-05 2024-09-15T21:47:56.860 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.860 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.861 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.861 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.861 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:56.861 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:56.861 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.357077865999997, 2024-09-15T21:47:56.861 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091802889999999995, 2024-09-15T21:47:56.861 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.861 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.861 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:56.861 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.861 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.861 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:56.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:47:56.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.862 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455081+0000", 2024-09-15T21:47:56.862 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:56.862 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.862 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.862 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.863 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455084+0000", 2024-09-15T21:47:56.863 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7810000000000001e-06 2024-09-15T21:47:56.863 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.864 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455536+0000", 2024-09-15T21:47:56.864 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00045219100000000001 2024-09-15T21:47:56.864 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.864 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.864 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455583+0000", 2024-09-15T21:47:56.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7308999999999997e-05 2024-09-15T21:47:56.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.865 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457998+0000", 2024-09-15T21:47:56.865 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024154290000000002 2024-09-15T21:47:56.865 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.865 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.865 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458147+0000", 2024-09-15T21:47:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014834599999999999 2024-09-15T21:47:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458265+0000", 2024-09-15T21:47:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118458 2024-09-15T21:47:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464134+0000", 2024-09-15T21:47:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005869038 2024-09-15T21:47:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.866 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:56.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464187+0000", 2024-09-15T21:47:56.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2343999999999999e-05 2024-09-15T21:47:56.867 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.867 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.867 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:56.867 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464222+0000", 2024-09-15T21:47:56.867 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5447000000000001e-05 2024-09-15T21:47:56.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.868 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.868 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464261+0000", 2024-09-15T21:47:56.868 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8659999999999999e-05 2024-09-15T21:47:56.868 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.868 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.868 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.868 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.868 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.869 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:56.869 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:56.869 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.336455486999995, 2024-09-15T21:47:56.869 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048468290000000004, 2024-09-15T21:47:56.869 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.869 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.869 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:56.869 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.869 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.869 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.869 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:56.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:56.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.870 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:47:56.870 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.35e-07 2024-09-15T21:47:56.870 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.870 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.870 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475706+0000", 2024-09-15T21:47:56.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.994e-06 2024-09-15T21:47:56.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475770+0000", 2024-09-15T21:47:56.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3904000000000003e-05 2024-09-15T21:47:56.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.871 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.871 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475775+0000", 2024-09-15T21:47:56.871 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2490000000000001e-06 2024-09-15T21:47:56.871 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.871 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476502+0000", 2024-09-15T21:47:56.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072697300000000001 2024-09-15T21:47:56.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476611+0000", 2024-09-15T21:47:56.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0001091 2024-09-15T21:47:56.872 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.872 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.872 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:56.872 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476807+0000", 2024-09-15T21:47:56.872 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000195892 2024-09-15T21:47:56.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:56.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479678+0000", 2024-09-15T21:47:56.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028704709999999999 2024-09-15T21:47:56.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:56.873 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480444+0000", 2024-09-15T21:47:56.873 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00076602099999999998 2024-09-15T21:47:56.873 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.873 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.873 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:56.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480502+0000", 2024-09-15T21:47:56.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8313999999999998e-05 2024-09-15T21:47:56.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.874 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.874 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480550+0000", 2024-09-15T21:47:56.874 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7676000000000002e-05 2024-09-15T21:47:56.874 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.874 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.874 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.874 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.874 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.874 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:56.874 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:56.875 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.336448312000002, 2024-09-15T21:47:56.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051250929999999998, 2024-09-15T21:47:56.875 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.875 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.875 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:47:56.875 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.875 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:56.875 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.875 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.875 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.875 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:56.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:47:56.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1e-07 2024-09-15T21:47:56.876 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.876 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.876 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.876 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475723+0000", 2024-09-15T21:47:56.876 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2758000000000001e-05 2024-09-15T21:47:56.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475972+0000", 2024-09-15T21:47:56.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024886299999999999 2024-09-15T21:47:56.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.877 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475979+0000", 2024-09-15T21:47:56.877 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0230000000000004e-06 2024-09-15T21:47:56.877 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.877 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.877 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476560+0000", 2024-09-15T21:47:56.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00058097599999999998 2024-09-15T21:47:56.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476689+0000", 2024-09-15T21:47:56.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129486 2024-09-15T21:47:56.878 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.878 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.878 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.878 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480835+0000", 2024-09-15T21:47:56.878 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041457769999999998 2024-09-15T21:47:56.878 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.878 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.879 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.879 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.879 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:56.879 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:56.879 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.267533087000004, 2024-09-15T21:47:56.879 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054420570000000001, 2024-09-15T21:47:56.879 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.879 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.879 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:56.879 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.879 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.879 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.879 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:56.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:47:56.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.880 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.880 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544626+0000", 2024-09-15T21:47:56.880 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8100000000000003e-07 2024-09-15T21:47:56.880 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.880 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544629+0000", 2024-09-15T21:47:56.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9869999999999999e-06 2024-09-15T21:47:56.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544775+0000", 2024-09-15T21:47:56.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014610599999999999 2024-09-15T21:47:56.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.881 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.881 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.881 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544782+0000", 2024-09-15T21:47:56.881 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4109999999999999e-06 2024-09-15T21:47:56.881 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545617+0000", 2024-09-15T21:47:56.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00083468300000000002 2024-09-15T21:47:56.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545690+0000", 2024-09-15T21:47:56.882 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3545000000000001e-05 2024-09-15T21:47:56.882 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.882 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.882 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:56.882 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545803+0000", 2024-09-15T21:47:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000112083 2024-09-15T21:47:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548353+0000", 2024-09-15T21:47:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025507220000000001 2024-09-15T21:47:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549964+0000", 2024-09-15T21:47:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001610382 2024-09-15T21:47:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.883 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550021+0000", 2024-09-15T21:47:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6898999999999999e-05 2024-09-15T21:47:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550067+0000", 2024-09-15T21:47:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758000000000002e-05 2024-09-15T21:47:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:56.884 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.267527809000001, 2024-09-15T21:47:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056323399999999996, 2024-09-15T21:47:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.885 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:56.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.886 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:47:56.886 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:47:56.886 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.886 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.886 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.886 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544634+0000", 2024-09-15T21:47:56.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.58e-06 2024-09-15T21:47:56.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.887 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544800+0000", 2024-09-15T21:47:56.887 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016576899999999999 2024-09-15T21:47:56.887 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.887 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.887 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544808+0000", 2024-09-15T21:47:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5179999999999998e-06 2024-09-15T21:47:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546075+0000", 2024-09-15T21:47:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012676969999999999 2024-09-15T21:47:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546179+0000", 2024-09-15T21:47:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000103935 2024-09-15T21:47:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.888 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546286+0000", 2024-09-15T21:47:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106758 2024-09-15T21:47:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548528+0000", 2024-09-15T21:47:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022420510000000001 2024-09-15T21:47:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550119+0000", 2024-09-15T21:47:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015907079999999999 2024-09-15T21:47:56.889 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:56.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550202+0000", 2024-09-15T21:47:56.890 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3374000000000005e-05 2024-09-15T21:47:56.890 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.890 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.890 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.890 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550263+0000", 2024-09-15T21:47:56.892 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0643999999999997e-05 2024-09-15T21:47:56.892 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.892 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.892 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.892 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.892 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.892 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:56.892 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.267522838000005, 2024-09-15T21:47:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061905470000000002, 2024-09-15T21:47:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.893 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:47:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6899999999999999e-07 2024-09-15T21:47:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544638+0000", 2024-09-15T21:47:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3920000000000001e-06 2024-09-15T21:47:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.894 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544819+0000", 2024-09-15T21:47:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018065099999999999 2024-09-15T21:47:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544825+0000", 2024-09-15T21:47:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7239999999999997e-06 2024-09-15T21:47:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546583+0000", 2024-09-15T21:47:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017588219999999999 2024-09-15T21:47:56.895 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546665+0000", 2024-09-15T21:47:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1693000000000005e-05 2024-09-15T21:47:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546808+0000", 2024-09-15T21:47:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142993 2024-09-15T21:47:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548533+0000", 2024-09-15T21:47:56.896 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017246 2024-09-15T21:47:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550606+0000", 2024-09-15T21:47:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020733969999999998 2024-09-15T21:47:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550783+0000", 2024-09-15T21:47:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000176565 2024-09-15T21:47:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.897 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550826+0000", 2024-09-15T21:47:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3441e-05 2024-09-15T21:47:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.267515117000002, 2024-09-15T21:47:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052813010000000004, 2024-09-15T21:47:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:56.898 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:47:56.899 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3799999999999999e-07 2024-09-15T21:47:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544647+0000", 2024-09-15T21:47:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6569999999999999e-06 2024-09-15T21:47:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544858+0000", 2024-09-15T21:47:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021069900000000001 2024-09-15T21:47:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.900 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544862+0000", 2024-09-15T21:47:56.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4900000000000002e-06 2024-09-15T21:47:56.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545610+0000", 2024-09-15T21:47:56.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00074786999999999998 2024-09-15T21:47:56.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.901 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.901 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545706+0000", 2024-09-15T21:47:56.901 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6271e-05 2024-09-15T21:47:56.901 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.901 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:56.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545834+0000", 2024-09-15T21:47:56.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012759499999999999 2024-09-15T21:47:56.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:56.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548554+0000", 2024-09-15T21:47:56.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027199339999999998 2024-09-15T21:47:56.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.902 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.902 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:56.902 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549818+0000", 2024-09-15T21:47:56.902 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012642199999999999 2024-09-15T21:47:56.902 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:56.903 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549884+0000", 2024-09-15T21:47:56.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5843e-05 2024-09-15T21:47:56.903 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.903 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.903 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.903 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549925+0000", 2024-09-15T21:47:56.903 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0484000000000001e-05 2024-09-15T21:47:56.903 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.903 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.903 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.904 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.904 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.904 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:56.904 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:56.904 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.267506440000005, 2024-09-15T21:47:56.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051106110000000001, 2024-09-15T21:47:56.905 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.905 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.905 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:56.905 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:56.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.905 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.905 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.905 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.905 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:56.905 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:47:56.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2700000000000001e-07 2024-09-15T21:47:56.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.906 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544667+0000", 2024-09-15T21:47:56.906 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4746000000000001e-05 2024-09-15T21:47:56.906 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.906 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.906 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545021+0000", 2024-09-15T21:47:56.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035412900000000001 2024-09-15T21:47:56.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545052+0000", 2024-09-15T21:47:56.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1158000000000002e-05 2024-09-15T21:47:56.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.907 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.907 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.907 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546050+0000", 2024-09-15T21:47:56.907 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00099782800000000008 2024-09-15T21:47:56.907 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.908 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546291+0000", 2024-09-15T21:47:56.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024093500000000001 2024-09-15T21:47:56.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:56.908 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546434+0000", 2024-09-15T21:47:56.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014308600000000001 2024-09-15T21:47:56.908 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.908 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.908 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:56.908 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548583+0000", 2024-09-15T21:47:56.908 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021491029999999999 2024-09-15T21:47:56.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.909 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:56.909 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549695+0000", 2024-09-15T21:47:56.909 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011116419999999999 2024-09-15T21:47:56.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.909 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:56.909 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549726+0000", 2024-09-15T21:47:56.909 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1195000000000002e-05 2024-09-15T21:47:56.909 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.909 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.909 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.910 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549763+0000", 2024-09-15T21:47:56.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6562e-05 2024-09-15T21:47:56.910 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.910 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.910 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.910 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.910 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.910 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:17 2.14 2:2eebc8f4:::62f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:56.910 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:56.910 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 65.247076828000004, 2024-09-15T21:47:56.910 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071004800000000002, 2024-09-15T21:47:56.910 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.910 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.910 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:56.911 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:56.911 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:47:56.911 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.911 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.911 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.911 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.911 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565083+0000", 2024-09-15T21:47:56.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.689e-06 2024-09-15T21:47:56.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.912 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.912 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.912 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565090+0000", 2024-09-15T21:47:56.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5930000000000002e-06 2024-09-15T21:47:56.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.912 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.912 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.912 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565119+0000", 2024-09-15T21:47:56.912 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8785999999999998e-05 2024-09-15T21:47:56.912 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.912 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.912 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.913 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565126+0000", 2024-09-15T21:47:56.913 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0110000000000003e-06 2024-09-15T21:47:56.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.913 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.913 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565237+0000", 2024-09-15T21:47:56.913 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011161 2024-09-15T21:47:56.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.913 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.913 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565383+0000", 2024-09-15T21:47:56.913 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014626900000000001 2024-09-15T21:47:56.913 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.913 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.914 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:56.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565520+0000", 2024-09-15T21:47:56.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013676399999999999 2024-09-15T21:47:56.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.914 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:56.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569909+0000", 2024-09-15T21:47:56.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043890149999999996 2024-09-15T21:47:56.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.914 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.914 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:56.914 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572096+0000", 2024-09-15T21:47:56.914 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021866559999999999 2024-09-15T21:47:56.914 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.915 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.915 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:56.915 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572144+0000", 2024-09-15T21:47:56.915 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7725000000000001e-05 2024-09-15T21:47:56.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.915 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.915 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.915 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572182+0000", 2024-09-15T21:47:56.915 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8362000000000003e-05 2024-09-15T21:47:56.915 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.915 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.915 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.915 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.915 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.916 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:56.916 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:56.916 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 65.247065730000003, 2024-09-15T21:47:56.916 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072110280000000004, 2024-09-15T21:47:56.916 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.916 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.916 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:56.916 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.916 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.916 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.916 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:56.916 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.917 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.917 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:56.917 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.917 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.917 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:47:56.917 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4299999999999999e-07 2024-09-15T21:47:56.917 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.917 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.917 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.917 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565096+0000", 2024-09-15T21:47:56.918 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4319999999999999e-06 2024-09-15T21:47:56.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.918 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.918 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565152+0000", 2024-09-15T21:47:56.918 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5686999999999997e-05 2024-09-15T21:47:56.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.918 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.918 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565157+0000", 2024-09-15T21:47:56.918 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2700000000000004e-06 2024-09-15T21:47:56.918 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.918 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.918 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.919 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565847+0000", 2024-09-15T21:47:56.919 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068937299999999996 2024-09-15T21:47:56.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.919 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.919 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565940+0000", 2024-09-15T21:47:56.919 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3573999999999996e-05 2024-09-15T21:47:56.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.919 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:47:56.919 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566085+0000", 2024-09-15T21:47:56.919 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000144902 2024-09-15T21:47:56.919 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.919 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:56.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570595+0000", 2024-09-15T21:47:56.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045103469999999996 2024-09-15T21:47:56.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:56.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572224+0000", 2024-09-15T21:47:56.920 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016282930000000001 2024-09-15T21:47:56.920 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.920 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.920 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:56.920 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572260+0000", 2024-09-15T21:47:56.921 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6103999999999999e-05 2024-09-15T21:47:56.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.921 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.921 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572304+0000", 2024-09-15T21:47:56.921 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3803000000000001e-05 2024-09-15T21:47:56.921 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.921 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.921 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.921 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.921 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.921 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:56.921 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:56.922 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 65.247061264999999, 2024-09-15T21:47:56.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055995389999999997, 2024-09-15T21:47:56.922 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.922 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.922 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:56.922 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.922 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:56.922 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.922 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.922 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.922 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.922 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:56.923 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.923 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.923 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.923 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:47:56.923 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9999999999999999e-07 2024-09-15T21:47:56.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.923 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.923 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.923 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:56.923 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2399999999999999e-06 2024-09-15T21:47:56.923 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.923 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.923 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.924 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565280+0000", 2024-09-15T21:47:56.924 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017929400000000001 2024-09-15T21:47:56.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.924 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.924 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565292+0000", 2024-09-15T21:47:56.924 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2193e-05 2024-09-15T21:47:56.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.924 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.924 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565357+0000", 2024-09-15T21:47:56.924 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4684000000000006e-05 2024-09-15T21:47:56.924 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.924 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.925 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.925 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565499+0000", 2024-09-15T21:47:56.925 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142032 2024-09-15T21:47:56.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.925 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.925 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:56.925 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565634+0000", 2024-09-15T21:47:56.925 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000135613 2024-09-15T21:47:56.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.925 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.925 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:56.925 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569941+0000", 2024-09-15T21:47:56.925 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043066789999999999 2024-09-15T21:47:56.925 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.926 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.926 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:56.926 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570585+0000", 2024-09-15T21:47:56.926 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00064364700000000001 2024-09-15T21:47:56.926 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.926 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.926 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:56.926 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570647+0000", 2024-09-15T21:47:56.926 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2173000000000003e-05 2024-09-15T21:47:56.926 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.926 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.926 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.926 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570697+0000", 2024-09-15T21:47:56.926 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9784000000000003e-05 2024-09-15T21:47:56.927 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.927 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.927 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.927 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.927 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.927 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:56.927 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:56.927 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 65.247057034999997, 2024-09-15T21:47:56.927 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066548939999999997, 2024-09-15T21:47:56.927 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:56.927 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:56.927 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:56.927 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:56.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.928 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:56.928 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:56.928 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.928 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:56.928 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:47:56.928 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:56.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.928 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.928 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:56.928 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565102+0000", 2024-09-15T21:47:56.928 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-15T21:47:56.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.929 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:56.929 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565103+0000", 2024-09-15T21:47:56.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6700000000000001e-06 2024-09-15T21:47:56.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.929 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:56.929 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565608+0000", 2024-09-15T21:47:56.929 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050518299999999998 2024-09-15T21:47:56.929 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.929 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.929 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:56.929 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565616+0000", 2024-09-15T21:47:56.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8199999999999997e-06 2024-09-15T21:47:56.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.930 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:56.930 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566598+0000", 2024-09-15T21:47:56.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00098176300000000008 2024-09-15T21:47:56.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.930 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:56.930 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566871+0000", 2024-09-15T21:47:56.930 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027312799999999998 2024-09-15T21:47:56.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.930 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.930 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:56.931 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567015+0000", 2024-09-15T21:47:56.931 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014428499999999999 2024-09-15T21:47:56.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.931 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:56.931 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570729+0000", 2024-09-15T21:47:56.931 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037132459999999999 2024-09-15T21:47:56.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.931 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:56.931 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571667+0000", 2024-09-15T21:47:56.931 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00093862299999999997 2024-09-15T21:47:56.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.931 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.932 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:56.932 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571718+0000", 2024-09-15T21:47:56.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1146999999999997e-05 2024-09-15T21:47:56.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:56.932 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:56.932 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:56.932 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571756+0000", 2024-09-15T21:47:56.932 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7821000000000003e-05 2024-09-15T21:47:56.932 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.932 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.932 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.932 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:56.932 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:56.933 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:56.933 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:47:56.934 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:56.932+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:56.935 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:47:57.034 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:57.033+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:57.039 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:57.039 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:57.039 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:47:57.039 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:57.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.038+0000 7f046b9bd640 1 Processor -- start 2024-09-15T21:47:57.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.039+0000 7f046b9bd640 1 -- start start 2024-09-15T21:47:57.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.039+0000 7f046b9bd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0464155440 0x7f04640cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:57.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.040+0000 7f046b9bd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f04640cc8f0 0x7f04640cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:57.041 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.040+0000 7f046b9bd640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f046415e7a0 con 0x7f04640cb7d0 2024-09-15T21:47:57.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.040+0000 7f046b9bd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f046415b470 con 0x7f0464155440 2024-09-15T21:47:57.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.040+0000 7f046b9bd640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f0464155820 con 0x7f04640cc8f0 2024-09-15T21:47:57.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.040+0000 7f0469759640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0464155440 0x7f04640cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:57.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.040+0000 7f0469759640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0464155440 0x7f04640cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57676/0 (socket says 172.21.15.100:57676) 2024-09-15T21:47:57.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.040+0000 7f0469759640 1 -- 172.21.15.100:0/1796336746 learned_addr learned my addr 172.21.15.100:0/1796336746 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:57.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.040+0000 7f0469f5a640 1 --2- 172.21.15.100:0/1796336746 >> v2:172.21.15.122:3300/0 conn(0x7f04640cc8f0 0x7f04640cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:57.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.040+0000 7f0469f5a640 1 -- 172.21.15.100:0/1796336746 >> v2:172.21.15.100:3300/0 conn(0x7f0464155440 msgr2=0x7f04640cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:57.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.040+0000 7f0469f5a640 1 --2- 172.21.15.100:0/1796336746 >> v2:172.21.15.100:3300/0 conn(0x7f0464155440 0x7f04640cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:57.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.040+0000 7f0469f5a640 1 -- 172.21.15.100:0/1796336746 >> v1:172.21.15.122:6789/0 conn(0x7f04640cb7d0 legacy=0x7f04640cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:57.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.040+0000 7f0469f5a640 1 -- 172.21.15.100:0/1796336746 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046415ea40 con 0x7f04640cc8f0 2024-09-15T21:47:57.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.040+0000 7f0469759640 1 --2- 172.21.15.100:0/1796336746 >> v2:172.21.15.100:3300/0 conn(0x7f0464155440 0x7f04640cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:57.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.041+0000 7f0469f5a640 1 --2- 172.21.15.100:0/1796336746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04640cc8f0 0x7f04640cd960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f04640ce140 tx=0x7f045c02a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9428787209578a78 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:57.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.041+0000 7f045bfff640 1 -- 172.21.15.100:0/1796336746 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c007540 con 0x7f04640cc8f0 2024-09-15T21:47:57.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.041+0000 7f045bfff640 1 -- 172.21.15.100:0/1796336746 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f045c0076e0 con 0x7f04640cc8f0 2024-09-15T21:47:57.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.041+0000 7f045bfff640 1 -- 172.21.15.100:0/1796336746 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c007ab0 con 0x7f04640cc8f0 2024-09-15T21:47:57.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.042+0000 7f046b9bd640 1 -- 172.21.15.100:0/1796336746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04640cc8f0 msgr2=0x7f04640cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:57.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.042+0000 7f046b9bd640 1 --2- 172.21.15.100:0/1796336746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04640cc8f0 0x7f04640cd960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f04640ce140 tx=0x7f045c02a620 comp rx=0 tx=0).stop 2024-09-15T21:47:57.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.042+0000 7f046b9bd640 1 -- 172.21.15.100:0/1796336746 shutdown_connections 2024-09-15T21:47:57.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.042+0000 7f046b9bd640 1 --2- 172.21.15.100:0/1796336746 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04640cc8f0 0x7f04640cd960 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:57.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.042+0000 7f046b9bd640 1 --2- 172.21.15.100:0/1796336746 >> v2:172.21.15.100:3300/0 conn(0x7f0464155440 0x7f04640cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:57.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.042+0000 7f046b9bd640 1 -- 172.21.15.100:0/1796336746 >> 172.21.15.100:0/1796336746 conn(0x7f04640680d0 msgr2=0x7f046406d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:57.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.042+0000 7f046b9bd640 1 -- 172.21.15.100:0/1796336746 shutdown_connections 2024-09-15T21:47:57.045 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.042+0000 7f046b9bd640 1 -- 172.21.15.100:0/1796336746 wait complete. 2024-09-15T21:47:57.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.043+0000 7f046b9bd640 1 Processor -- start 2024-09-15T21:47:57.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.043+0000 7f046b9bd640 1 -- start start 2024-09-15T21:47:57.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.044+0000 7f046b9bd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04640cb7d0 0x7f0464153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:57.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.044+0000 7f046b9bd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f04640cc8f0 0x7f04641f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:57.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.044+0000 7f046b9bd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0464155440 0x7f04641f19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:57.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.044+0000 7f046b9bd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f046415f2e0 con 0x7f0464155440 2024-09-15T21:47:57.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.044+0000 7f046b9bd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f046415f000 con 0x7f04640cb7d0 2024-09-15T21:47:57.047 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.044+0000 7f046b9bd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f046415b070 con 0x7f04640cc8f0 2024-09-15T21:47:57.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.044+0000 7f0469f5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0464155440 0x7f04641f19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:57.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.044+0000 7f0469f5a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0464155440 0x7f04641f19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57680/0 (socket says 172.21.15.100:57680) 2024-09-15T21:47:57.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.044+0000 7f0469f5a640 1 -- 172.21.15.100:0/3002241708 learned_addr learned my addr 172.21.15.100:0/3002241708 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:57.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.045+0000 7f0469f5a640 1 -- 172.21.15.100:0/3002241708 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f04640cc8f0 msgr2=0x7f04641f14b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:47:57.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.045+0000 7f0469f5a640 1 --2- 172.21.15.100:0/3002241708 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f04640cc8f0 0x7f04641f14b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:57.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.045+0000 7f0469f5a640 1 -- 172.21.15.100:0/3002241708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04640cb7d0 msgr2=0x7f0464153f20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T21:47:57.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.045+0000 7f0469f5a640 1 --2- 172.21.15.100:0/3002241708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04640cb7d0 0x7f0464153f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:57.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.045+0000 7f0469f5a640 1 -- 172.21.15.100:0/3002241708 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f046415aac0 con 0x7f0464155440 2024-09-15T21:47:57.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.045+0000 7f0469f5a640 1 --2- 172.21.15.100:0/3002241708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0464155440 0x7f04641f19f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f045c009e50 tx=0x7f045c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:57.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.045+0000 7f0459ffb640 1 -- 172.21.15.100:0/3002241708 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c040070 con 0x7f0464155440 2024-09-15T21:47:57.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.045+0000 7f046b9bd640 1 -- 172.21.15.100:0/3002241708 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f046415ad80 con 0x7f0464155440 2024-09-15T21:47:57.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.045+0000 7f0459ffb640 1 -- 172.21.15.100:0/3002241708 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f045c031a30 con 0x7f0464155440 2024-09-15T21:47:57.048 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.045+0000 7f0459ffb640 1 -- 172.21.15.100:0/3002241708 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c009450 con 0x7f0464155440 2024-09-15T21:47:57.050 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:47:57.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.046+0000 7f046b9bd640 1 -- 172.21.15.100:0/3002241708 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04640b5d90 con 0x7f0464155440 2024-09-15T21:47:57.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.047+0000 7f046b9bd640 1 -- 172.21.15.100:0/3002241708 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04640b3940 con 0x7f0464155440 2024-09-15T21:47:57.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.047+0000 7f0459ffb640 1 -- 172.21.15.100:0/3002241708 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f045c039040 con 0x7f0464155440 2024-09-15T21:47:57.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.048+0000 7f0459ffb640 1 --2- 172.21.15.100:0/3002241708 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f044806c2b0 0x7f044806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:57.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.048+0000 7f0459ffb640 1 -- 172.21.15.100:0/3002241708 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5886+0+0 (secure 0 0 0) 0x7f045c0acb90 con 0x7f0464155440 2024-09-15T21:47:57.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.048+0000 7f0469759640 1 --2- 172.21.15.100:0/3002241708 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f044806c2b0 0x7f044806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:57.051 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.048+0000 7f0469759640 1 --2- 172.21.15.100:0/3002241708 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f044806c2b0 0x7f044806e770 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0460009960 tx=0x7f0460009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:57.058 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.056+0000 7f0459ffb640 1 -- 172.21.15.100:0/3002241708 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f045c07b250 con 0x7f0464155440 2024-09-15T21:47:57.139 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:57.138+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:57.194 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:57.195 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:47:57.195 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:47:57.195 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:47:57.195 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:57.203 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:47:57.239 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.237+0000 7f046b9bd640 1 -- 172.21.15.100:0/3002241708 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f04640caeb0 con 0x7f0464155440 2024-09-15T21:47:57.243 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:57.242+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:57.338 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:57.336+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:57.348 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:47:57.348 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:47:57.348 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:47:57.348 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:47:57.348 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.348 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:47:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 104.110347253, 2024-09-15T21:47:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0944283340000001, 2024-09-15T21:47:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:57.349 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2024-09-15T21:47:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.350 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:47:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234360+0000", 2024-09-15T21:47:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3710000000000001e-06 2024-09-15T21:47:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.351 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234374+0000", 2024-09-15T21:47:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4003e-05 2024-09-15T21:47:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234392+0000", 2024-09-15T21:47:57.352 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7294000000000001e-05 2024-09-15T21:47:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234405+0000", 2024-09-15T21:47:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2961e-05 2024-09-15T21:47:57.353 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.354 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234483+0000", 2024-09-15T21:47:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8850000000000006e-05 2024-09-15T21:47:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-15T21:47:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234491+0000", 2024-09-15T21:47:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3529999999999997e-06 2024-09-15T21:47:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327253+0000", 2024-09-15T21:47:57.355 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.092761962 2024-09-15T21:47:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327456+0000", 2024-09-15T21:47:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000202893 2024-09-15T21:47:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327652+0000", 2024-09-15T21:47:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019613000000000001 2024-09-15T21:47:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.356 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328368+0000", 2024-09-15T21:47:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000716582 2024-09-15T21:47:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328682+0000", 2024-09-15T21:47:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000313825 2024-09-15T21:47:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328743+0000", 2024-09-15T21:47:57.357 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0571e-05 2024-09-15T21:47:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328787+0000", 2024-09-15T21:47:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4539000000000002e-05 2024-09-15T21:47:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 102.989181851, 2024-09-15T21:47:57.358 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0089551270000000002, 2024-09-15T21:47:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 35 2024-09-15T21:47:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.359 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:47:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355525+0000", 2024-09-15T21:47:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:47:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.360 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355606+0000", 2024-09-15T21:47:57.361 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1723000000000003e-05 2024-09-15T21:47:57.361 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.361 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355624+0000", 2024-09-15T21:47:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7645000000000001e-05 2024-09-15T21:47:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355629+0000", 2024-09-15T21:47:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7319999999999998e-06 2024-09-15T21:47:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.362 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358315+0000", 2024-09-15T21:47:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026858659999999999 2024-09-15T21:47:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358373+0000", 2024-09-15T21:47:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8593e-05 2024-09-15T21:47:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358521+0000", 2024-09-15T21:47:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014817500000000001 2024-09-15T21:47:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.363 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360353+0000", 2024-09-15T21:47:57.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018317100000000001 2024-09-15T21:47:57.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364424+0000", 2024-09-15T21:47:57.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040707970000000001 2024-09-15T21:47:57.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.364 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.364 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.364 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364452+0000", 2024-09-15T21:47:57.364 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7937999999999999e-05 2024-09-15T21:47:57.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.365 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.365 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364479+0000", 2024-09-15T21:47:57.365 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7744999999999999e-05 2024-09-15T21:47:57.365 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.365 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.365 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.365 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 102.989097888, 2024-09-15T21:47:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090267869999999997, 2024-09-15T21:47:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 36 2024-09-15T21:47:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.366 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:47:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355609+0000", 2024-09-15T21:47:57.367 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3099999999999998e-07 2024-09-15T21:47:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355612+0000", 2024-09-15T21:47:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1489999999999998e-06 2024-09-15T21:47:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355637+0000", 2024-09-15T21:47:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.508e-05 2024-09-15T21:47:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.368 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355640+0000", 2024-09-15T21:47:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7930000000000002e-06 2024-09-15T21:47:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359195+0000", 2024-09-15T21:47:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035555529999999999 2024-09-15T21:47:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359283+0000", 2024-09-15T21:47:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8042999999999997e-05 2024-09-15T21:47:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.369 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359419+0000", 2024-09-15T21:47:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013576699999999999 2024-09-15T21:47:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360361+0000", 2024-09-15T21:47:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094171000000000001 2024-09-15T21:47:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364536+0000", 2024-09-15T21:47:57.370 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00417484 2024-09-15T21:47:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364590+0000", 2024-09-15T21:47:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4355999999999999e-05 2024-09-15T21:47:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364635+0000", 2024-09-15T21:47:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5065000000000003e-05 2024-09-15T21:47:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.371 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 102.989093283, 2024-09-15T21:47:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091162750000000001, 2024-09-15T21:47:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 37 2024-09-15T21:47:57.372 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.373 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:47:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:47:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355615+0000", 2024-09-15T21:47:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-15T21:47:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355646+0000", 2024-09-15T21:47:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0391000000000001e-05 2024-09-15T21:47:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.374 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355648+0000", 2024-09-15T21:47:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6309999999999999e-06 2024-09-15T21:47:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359770+0000", 2024-09-15T21:47:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041222899999999998 2024-09-15T21:47:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.375 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359854+0000", 2024-09-15T21:47:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3399000000000006e-05 2024-09-15T21:47:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359988+0000", 2024-09-15T21:47:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013442200000000001 2024-09-15T21:47:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360524+0000", 2024-09-15T21:47:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00053582900000000004 2024-09-15T21:47:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.376 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364673+0000", 2024-09-15T21:47:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041493199999999997 2024-09-15T21:47:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364704+0000", 2024-09-15T21:47:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0380999999999999e-05 2024-09-15T21:47:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364729+0000", 2024-09-15T21:47:57.377 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5335999999999999e-05 2024-09-15T21:47:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:47:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 102.920376045, 2024-09-15T21:47:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0026270020000001, 2024-09-15T21:47:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:57.378 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 88 2024-09-15T21:47:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:47:57.379 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424331+0000", 2024-09-15T21:47:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4299999999999998e-07 2024-09-15T21:47:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424333+0000", 2024-09-15T21:47:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2359999999999999e-06 2024-09-15T21:47:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.380 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422895+0000", 2024-09-15T21:47:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99856184199999998 2024-09-15T21:47:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422902+0000", 2024-09-15T21:47:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4440000000000002e-06 2024-09-15T21:47:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423053+0000", 2024-09-15T21:47:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000150958 2024-09-15T21:47:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.381 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423160+0000", 2024-09-15T21:47:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106774 2024-09-15T21:47:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423344+0000", 2024-09-15T21:47:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018424199999999999 2024-09-15T21:47:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.424730+0000", 2024-09-15T21:47:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001386155 2024-09-15T21:47:57.382 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426756+0000", 2024-09-15T21:47:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020261150000000002 2024-09-15T21:47:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426906+0000", 2024-09-15T21:47:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000149176 2024-09-15T21:47:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426957+0000", 2024-09-15T21:47:57.383 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1616999999999997e-05 2024-09-15T21:47:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 101.79030284700001, 2024-09-15T21:47:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095037790000000004, 2024-09-15T21:47:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.384 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 114 2024-09-15T21:47:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:47:57.385 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554404+0000", 2024-09-15T21:47:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2399999999999999e-07 2024-09-15T21:47:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554407+0000", 2024-09-15T21:47:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-15T21:47:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.386 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554632+0000", 2024-09-15T21:47:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022568999999999999 2024-09-15T21:47:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554640+0000", 2024-09-15T21:47:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7970000000000001e-06 2024-09-15T21:47:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.387 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561325+0000", 2024-09-15T21:47:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066848020000000001 2024-09-15T21:47:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561402+0000", 2024-09-15T21:47:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6851e-05 2024-09-15T21:47:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.388 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561530+0000", 2024-09-15T21:47:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012863999999999999 2024-09-15T21:47:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563144+0000", 2024-09-15T21:47:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016132849999999999 2024-09-15T21:47:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563800+0000", 2024-09-15T21:47:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00065586099999999999 2024-09-15T21:47:57.389 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563857+0000", 2024-09-15T21:47:57.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.702e-05 2024-09-15T21:47:57.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.390 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.390 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563907+0000", 2024-09-15T21:47:57.390 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0479000000000002e-05 2024-09-15T21:47:57.390 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.390 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.390 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.390 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.390 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.391 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:57.391 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:57.391 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 101.79029785, 2024-09-15T21:47:57.391 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011813423999999999, 2024-09-15T21:47:57.391 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.391 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.391 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.391 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:57.391 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:57.391 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:57.391 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 115 2024-09-15T21:47:57.391 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.391 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.391 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:57.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:47:57.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.392 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.392 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.392 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.392 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554409+0000", 2024-09-15T21:47:57.392 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1100000000000002e-07 2024-09-15T21:47:57.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554498+0000", 2024-09-15T21:47:57.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9642999999999995e-05 2024-09-15T21:47:57.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554656+0000", 2024-09-15T21:47:57.393 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000158114 2024-09-15T21:47:57.393 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.393 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.393 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.393 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554661+0000", 2024-09-15T21:47:57.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0730000000000004e-06 2024-09-15T21:47:57.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561862+0000", 2024-09-15T21:47:57.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072002999999999998 2024-09-15T21:47:57.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.394 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561939+0000", 2024-09-15T21:47:57.394 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7265e-05 2024-09-15T21:47:57.394 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.394 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.394 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562073+0000", 2024-09-15T21:47:57.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013371600000000001 2024-09-15T21:47:57.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563196+0000", 2024-09-15T21:47:57.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011231679999999999 2024-09-15T21:47:57.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.395 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.395 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.395 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566124+0000", 2024-09-15T21:47:57.395 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002928403 2024-09-15T21:47:57.395 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566183+0000", 2024-09-15T21:47:57.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8328000000000001e-05 2024-09-15T21:47:57.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.396 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.396 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.396 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566222+0000", 2024-09-15T21:47:57.396 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9103000000000002e-05 2024-09-15T21:47:57.396 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.396 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.396 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.396 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.397 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.397 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:57.397 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:57.397 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 101.79020441, 2024-09-15T21:47:57.397 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01181426, 2024-09-15T21:47:57.397 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.397 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.397 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.397 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:57.397 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:57.397 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:57.397 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 116 2024-09-15T21:47:57.397 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.398 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:57.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:47:57.398 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.398 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.398 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.398 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.398 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554503+0000", 2024-09-15T21:47:57.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8000000000000005e-07 2024-09-15T21:47:57.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554506+0000", 2024-09-15T21:47:57.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5899999999999999e-06 2024-09-15T21:47:57.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.399 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554674+0000", 2024-09-15T21:47:57.399 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016763600000000001 2024-09-15T21:47:57.399 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.399 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.399 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554678+0000", 2024-09-15T21:47:57.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5310000000000003e-06 2024-09-15T21:47:57.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562714+0000", 2024-09-15T21:47:57.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.008035258 2024-09-15T21:47:57.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.400 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.400 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.400 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562765+0000", 2024-09-15T21:47:57.400 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1093e-05 2024-09-15T21:47:57.400 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562845+0000", 2024-09-15T21:47:57.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0427999999999996e-05 2024-09-15T21:47:57.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563943+0000", 2024-09-15T21:47:57.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097647 2024-09-15T21:47:57.401 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.401 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.401 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.401 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566232+0000", 2024-09-15T21:47:57.401 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022890850000000002 2024-09-15T21:47:57.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566279+0000", 2024-09-15T21:47:57.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6692e-05 2024-09-15T21:47:57.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.402 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.402 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.402 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566316+0000", 2024-09-15T21:47:57.402 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.752e-05 2024-09-15T21:47:57.402 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.402 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.402 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.402 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.403 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.403 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:57.403 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:57.403 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 101.790198311, 2024-09-15T21:47:57.403 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012232125, 2024-09-15T21:47:57.403 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.403 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.403 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.403 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:57.403 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:57.403 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:57.403 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 117 2024-09-15T21:47:57.403 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.404 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:57.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:57.404 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.404 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.404 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.404 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.404 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:47:57.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7700000000000001e-07 2024-09-15T21:47:57.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554511+0000", 2024-09-15T21:47:57.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8430000000000001e-06 2024-09-15T21:47:57.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.405 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.405 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554690+0000", 2024-09-15T21:47:57.405 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017856299999999999 2024-09-15T21:47:57.405 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.405 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554698+0000", 2024-09-15T21:47:57.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6200000000000005e-06 2024-09-15T21:47:57.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.406 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.406 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563224+0000", 2024-09-15T21:47:57.406 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0085262640000000004 2024-09-15T21:47:57.406 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563342+0000", 2024-09-15T21:47:57.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117687 2024-09-15T21:47:57.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563472+0000", 2024-09-15T21:47:57.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012957999999999999 2024-09-15T21:47:57.407 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.407 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.407 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.407 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566367+0000", 2024-09-15T21:47:57.407 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028952280000000001 2024-09-15T21:47:57.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566680+0000", 2024-09-15T21:47:57.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031345100000000002 2024-09-15T21:47:57.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.408 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566714+0000", 2024-09-15T21:47:57.408 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3701000000000001e-05 2024-09-15T21:47:57.408 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.408 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.408 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.409 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566740+0000", 2024-09-15T21:47:57.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5911e-05 2024-09-15T21:47:57.409 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.409 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.409 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.409 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.409 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.409 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:57.409 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:57.409 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 101.44730622900001, 2024-09-15T21:47:57.409 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069806590000000002, 2024-09-15T21:47:57.409 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.409 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.410 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.410 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:57.410 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:57.410 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:57.410 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 134 2024-09-15T21:47:57.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.410 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.410 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:57.410 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.410 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.410 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.410 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:47:57.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897401+0000", 2024-09-15T21:47:57.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0500000000000003e-07 2024-09-15T21:47:57.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.411 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.411 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.411 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897492+0000", 2024-09-15T21:47:57.411 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1770999999999999e-05 2024-09-15T21:47:57.411 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897750+0000", 2024-09-15T21:47:57.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025778299999999998 2024-09-15T21:47:57.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897756+0000", 2024-09-15T21:47:57.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2140000000000001e-06 2024-09-15T21:47:57.412 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.412 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.412 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.412 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899312+0000", 2024-09-15T21:47:57.412 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001556144 2024-09-15T21:47:57.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899372+0000", 2024-09-15T21:47:57.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0015000000000001e-05 2024-09-15T21:47:57.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899444+0000", 2024-09-15T21:47:57.413 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1923999999999998e-05 2024-09-15T21:47:57.413 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.413 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.413 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.413 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901354+0000", 2024-09-15T21:47:57.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019092530000000001 2024-09-15T21:47:57.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904327+0000", 2024-09-15T21:47:57.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029734520000000001 2024-09-15T21:47:57.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.414 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904360+0000", 2024-09-15T21:47:57.414 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3216000000000002e-05 2024-09-15T21:47:57.414 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.414 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.414 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.415 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904381+0000", 2024-09-15T21:47:57.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0282e-05 2024-09-15T21:47:57.415 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.415 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.415 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.415 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.415 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.415 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:57.415 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:57.415 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 101.447210372, 2024-09-15T21:47:57.415 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070121059999999997, 2024-09-15T21:47:57.415 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.415 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.416 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.416 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:57.416 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:57.416 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:57.416 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 135 2024-09-15T21:47:57.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.416 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.416 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.416 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:57.416 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.416 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.416 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:47:57.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897497+0000", 2024-09-15T21:47:57.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4799999999999997e-07 2024-09-15T21:47:57.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.417 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.417 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.417 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897500+0000", 2024-09-15T21:47:57.417 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2949999999999998e-06 2024-09-15T21:47:57.417 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897781+0000", 2024-09-15T21:47:57.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028064300000000002 2024-09-15T21:47:57.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897786+0000", 2024-09-15T21:47:57.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5169999999999999e-06 2024-09-15T21:47:57.418 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.418 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.418 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.418 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899857+0000", 2024-09-15T21:47:57.418 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020705340000000002 2024-09-15T21:47:57.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899932+0000", 2024-09-15T21:47:57.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5840999999999994e-05 2024-09-15T21:47:57.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.419 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900005+0000", 2024-09-15T21:47:57.419 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2716e-05 2024-09-15T21:47:57.419 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.419 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.419 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902296+0000", 2024-09-15T21:47:57.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022912010000000001 2024-09-15T21:47:57.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904424+0000", 2024-09-15T21:47:57.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021278600000000001 2024-09-15T21:47:57.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.420 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.420 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.420 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904472+0000", 2024-09-15T21:47:57.420 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7586000000000001e-05 2024-09-15T21:47:57.420 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.421 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.421 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904508+0000", 2024-09-15T21:47:57.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6164999999999997e-05 2024-09-15T21:47:57.421 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.421 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.421 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.421 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.421 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.421 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:57.421 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:57.421 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 101.447185881, 2024-09-15T21:47:57.421 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070611429999999998, 2024-09-15T21:47:57.421 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.422 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.422 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.422 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:57.422 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:57.422 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:57.422 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 136 2024-09-15T21:47:57.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.422 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.422 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.422 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:57.422 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.422 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.422 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:47:57.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897521+0000", 2024-09-15T21:47:57.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2900000000000004e-07 2024-09-15T21:47:57.423 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.423 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.423 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.423 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897527+0000", 2024-09-15T21:47:57.423 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8490000000000002e-06 2024-09-15T21:47:57.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897799+0000", 2024-09-15T21:47:57.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027219499999999998 2024-09-15T21:47:57.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.424 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897804+0000", 2024-09-15T21:47:57.424 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4219999999999999e-06 2024-09-15T21:47:57.424 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.424 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.424 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900248+0000", 2024-09-15T21:47:57.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002443457 2024-09-15T21:47:57.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900306+0000", 2024-09-15T21:47:57.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8263000000000001e-05 2024-09-15T21:47:57.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.425 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.425 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900385+0000", 2024-09-15T21:47:57.425 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9161999999999999e-05 2024-09-15T21:47:57.425 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.425 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902300+0000", 2024-09-15T21:47:57.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001914947 2024-09-15T21:47:57.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904536+0000", 2024-09-15T21:47:57.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022354419999999998 2024-09-15T21:47:57.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.426 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.426 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.426 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904562+0000", 2024-09-15T21:47:57.426 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6378999999999999e-05 2024-09-15T21:47:57.426 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.427 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.427 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904581+0000", 2024-09-15T21:47:57.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9497999999999999e-05 2024-09-15T21:47:57.427 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.427 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.427 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.427 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.427 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.427 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:57.427 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:57.427 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 101.447177314, 2024-09-15T21:47:57.427 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007180306, 2024-09-15T21:47:57.427 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.428 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.428 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.428 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:57.428 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:57.428 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:57.428 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 137 2024-09-15T21:47:57.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.428 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.428 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.428 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.428 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:57.428 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.428 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:57.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:47:57.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1399999999999998e-07 2024-09-15T21:47:57.429 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.429 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.429 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.429 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897612+0000", 2024-09-15T21:47:57.429 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2607000000000004e-05 2024-09-15T21:47:57.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897815+0000", 2024-09-15T21:47:57.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020352400000000001 2024-09-15T21:47:57.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897821+0000", 2024-09-15T21:47:57.430 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4670000000000003e-06 2024-09-15T21:47:57.430 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.430 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.430 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.430 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900665+0000", 2024-09-15T21:47:57.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028443689999999998 2024-09-15T21:47:57.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900714+0000", 2024-09-15T21:47:57.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8812e-05 2024-09-15T21:47:57.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.431 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900799+0000", 2024-09-15T21:47:57.431 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5129000000000004e-05 2024-09-15T21:47:57.431 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.431 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.431 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903413+0000", 2024-09-15T21:47:57.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026142890000000001 2024-09-15T21:47:57.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904625+0000", 2024-09-15T21:47:57.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012121149999999999 2024-09-15T21:47:57.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.432 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.432 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904673+0000", 2024-09-15T21:47:57.432 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7540999999999998e-05 2024-09-15T21:47:57.432 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.432 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.433 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.433 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904709+0000", 2024-09-15T21:47:57.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6139000000000001e-05 2024-09-15T21:47:57.433 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.433 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.433 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.433 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.433 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.433 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:57.433 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:57.433 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 101.447091359, 2024-09-15T21:47:57.433 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071727450000000003, 2024-09-15T21:47:57.433 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.433 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.434 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.434 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:57.434 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:57.434 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:57.434 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 138 2024-09-15T21:47:57.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.434 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.434 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:57.434 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.434 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.434 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.434 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:57.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:47:57.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:47:57.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.435 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.435 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897619+0000", 2024-09-15T21:47:57.435 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3869999999999999e-06 2024-09-15T21:47:57.435 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.435 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897832+0000", 2024-09-15T21:47:57.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021329500000000001 2024-09-15T21:47:57.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897837+0000", 2024-09-15T21:47:57.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3730000000000001e-06 2024-09-15T21:47:57.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.436 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.436 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.436 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901391+0000", 2024-09-15T21:47:57.436 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003553601 2024-09-15T21:47:57.436 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901466+0000", 2024-09-15T21:47:57.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5130999999999996e-05 2024-09-15T21:47:57.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901601+0000", 2024-09-15T21:47:57.437 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134797 2024-09-15T21:47:57.437 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.437 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.437 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.437 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903417+0000", 2024-09-15T21:47:57.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001816156 2024-09-15T21:47:57.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904741+0000", 2024-09-15T21:47:57.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013241049999999999 2024-09-15T21:47:57.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.438 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904767+0000", 2024-09-15T21:47:57.438 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5876000000000001e-05 2024-09-15T21:47:57.438 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.438 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.438 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.439 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904788+0000", 2024-09-15T21:47:57.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0449999999999999e-05 2024-09-15T21:47:57.439 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.439 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.439 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.439 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.439 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.439 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:57.439 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:57.439 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 101.447085706, 2024-09-15T21:47:57.439 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072970250000000004, 2024-09-15T21:47:57.439 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.439 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.439 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.440 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:57.440 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:57.440 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:57.440 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 139 2024-09-15T21:47:57.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.440 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.440 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:57.440 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.440 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.440 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.440 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:47:57.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897621+0000", 2024-09-15T21:47:57.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3099999999999999e-07 2024-09-15T21:47:57.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.441 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.441 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897625+0000", 2024-09-15T21:47:57.441 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.698e-06 2024-09-15T21:47:57.441 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.441 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897848+0000", 2024-09-15T21:47:57.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022375699999999999 2024-09-15T21:47:57.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897853+0000", 2024-09-15T21:47:57.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7709999999999997e-06 2024-09-15T21:47:57.442 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.442 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.442 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.442 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901929+0000", 2024-09-15T21:47:57.442 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040757780000000004 2024-09-15T21:47:57.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901976+0000", 2024-09-15T21:47:57.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6751e-05 2024-09-15T21:47:57.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902054+0000", 2024-09-15T21:47:57.443 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8856999999999994e-05 2024-09-15T21:47:57.443 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.443 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.443 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.443 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903421+0000", 2024-09-15T21:47:57.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00136619 2024-09-15T21:47:57.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904827+0000", 2024-09-15T21:47:57.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0014067369999999999 2024-09-15T21:47:57.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.444 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904881+0000", 2024-09-15T21:47:57.444 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3520999999999998e-05 2024-09-15T21:47:57.444 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.444 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.444 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.445 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904918+0000", 2024-09-15T21:47:57.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6634000000000002e-05 2024-09-15T21:47:57.445 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.445 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.445 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.445 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.445 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.445 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:57.445 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:57.445 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 101.44707987699999, 2024-09-15T21:47:57.445 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097221730000000006, 2024-09-15T21:47:57.445 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.445 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.445 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.446 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:57.446 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:57.446 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:57.446 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 140 2024-09-15T21:47:57.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.446 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.446 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:57.446 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.446 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.446 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.446 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:47:57.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897627+0000", 2024-09-15T21:47:57.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-15T21:47:57.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.447 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.447 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897698+0000", 2024-09-15T21:47:57.447 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1722000000000002e-05 2024-09-15T21:47:57.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897863+0000", 2024-09-15T21:47:57.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016479099999999999 2024-09-15T21:47:57.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897868+0000", 2024-09-15T21:47:57.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4950000000000002e-06 2024-09-15T21:47:57.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.448 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.448 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902335+0000", 2024-09-15T21:47:57.448 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004467632 2024-09-15T21:47:57.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902414+0000", 2024-09-15T21:47:57.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8688000000000003e-05 2024-09-15T21:47:57.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902535+0000", 2024-09-15T21:47:57.449 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012066900000000001 2024-09-15T21:47:57.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.449 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.449 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.449 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903714+0000", 2024-09-15T21:47:57.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011794520000000001 2024-09-15T21:47:57.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907115+0000", 2024-09-15T21:47:57.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034008879999999999 2024-09-15T21:47:57.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.450 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907250+0000", 2024-09-15T21:47:57.450 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134657 2024-09-15T21:47:57.450 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.450 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.451 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907348+0000", 2024-09-15T21:47:57.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8838000000000005e-05 2024-09-15T21:47:57.451 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.451 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.451 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.451 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.451 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.451 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:47:57.451 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:57.451 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 101.447004796, 2024-09-15T21:47:57.451 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097618050000000001, 2024-09-15T21:47:57.451 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.451 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.452 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.452 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:47:57.452 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:47:57.452 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:47:57.452 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 141 2024-09-15T21:47:57.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.452 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.452 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:57.452 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.452 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:47:57.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897702+0000", 2024-09-15T21:47:57.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6199999999999998e-07 2024-09-15T21:47:57.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.453 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.453 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.453 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897705+0000", 2024-09-15T21:47:57.453 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0180000000000002e-06 2024-09-15T21:47:57.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897889+0000", 2024-09-15T21:47:57.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000183678 2024-09-15T21:47:57.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897894+0000", 2024-09-15T21:47:57.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8930000000000004e-06 2024-09-15T21:47:57.454 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.454 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.454 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903099+0000", 2024-09-15T21:47:57.454 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052056280000000003 2024-09-15T21:47:57.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903151+0000", 2024-09-15T21:47:57.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2163000000000002e-05 2024-09-15T21:47:57.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903220+0000", 2024-09-15T21:47:57.455 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8483000000000001e-05 2024-09-15T21:47:57.455 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.455 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.455 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.455 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904794+0000", 2024-09-15T21:47:57.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001574521 2024-09-15T21:47:57.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907405+0000", 2024-09-15T21:47:57.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026110339999999999 2024-09-15T21:47:57.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.456 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907440+0000", 2024-09-15T21:47:57.456 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4564999999999999e-05 2024-09-15T21:47:57.456 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.456 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.456 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.457 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907463+0000", 2024-09-15T21:47:57.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3260000000000001e-05 2024-09-15T21:47:57.457 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.457 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.457 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.457 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.457 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.457 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:22 2.8 2:1470b2f8:::613.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:57.457 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:57.457 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.889564469000007, 2024-09-15T21:47:57.457 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007454937, 2024-09-15T21:47:57.457 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.457 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.457 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.458 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:57.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:57.458 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.458 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.458 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.458 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.458 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:47:57.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6800000000000002e-07 2024-09-15T21:47:57.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455145+0000", 2024-09-15T21:47:57.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4470000000000002e-06 2024-09-15T21:47:57.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.459 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.459 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456425+0000", 2024-09-15T21:47:57.459 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012798200000000001 2024-09-15T21:47:57.459 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.459 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456433+0000", 2024-09-15T21:47:57.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7389999999999999e-06 2024-09-15T21:47:57.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:47:57.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097865 2024-09-15T21:47:57.460 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.460 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.460 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.460 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457614+0000", 2024-09-15T21:47:57.460 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3560000000000006e-05 2024-09-15T21:47:57.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457998+0000", 2024-09-15T21:47:57.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00038372699999999999 2024-09-15T21:47:57.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460488+0000", 2024-09-15T21:47:57.461 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002489855 2024-09-15T21:47:57.461 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.461 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.461 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.461 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462512+0000", 2024-09-15T21:47:57.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020235269999999998 2024-09-15T21:47:57.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462554+0000", 2024-09-15T21:47:57.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2472000000000003e-05 2024-09-15T21:47:57.462 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.462 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.462 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.462 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462597+0000", 2024-09-15T21:47:57.462 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2657000000000002e-05 2024-09-15T21:47:57.462 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.462 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.463 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.463 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.463 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:2 2.5 2:a6e36b3f:::401.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:57.463 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:57.463 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 82.889559500000004, 2024-09-15T21:47:57.463 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074484590000000002, 2024-09-15T21:47:57.463 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.463 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.463 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.463 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.463 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.463 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.463 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:57.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:57.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.464 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:47:57.464 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:47:57.464 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.464 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.464 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455150+0000", 2024-09-15T21:47:57.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7319999999999998e-06 2024-09-15T21:47:57.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.465 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.465 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456449+0000", 2024-09-15T21:47:57.465 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012991490000000001 2024-09-15T21:47:57.465 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.465 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456454+0000", 2024-09-15T21:47:57.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0390000000000004e-06 2024-09-15T21:47:57.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456513+0000", 2024-09-15T21:47:57.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8768999999999999e-05 2024-09-15T21:47:57.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.466 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.466 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.466 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456628+0000", 2024-09-15T21:47:57.466 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115641 2024-09-15T21:47:57.466 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456741+0000", 2024-09-15T21:47:57.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000113094 2024-09-15T21:47:57.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458520+0000", 2024-09-15T21:47:57.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001778993 2024-09-15T21:47:57.467 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.467 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.467 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.467 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462494+0000", 2024-09-15T21:47:57.467 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039737009999999996 2024-09-15T21:47:57.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462549+0000", 2024-09-15T21:47:57.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4537000000000003e-05 2024-09-15T21:47:57.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.468 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.468 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.468 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462595+0000", 2024-09-15T21:47:57.468 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6511000000000002e-05 2024-09-15T21:47:57.468 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.468 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.468 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.468 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.469 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:27 2.5 2:a5fa685f:::504.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:57.469 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:57.469 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 65.779867116000005, 2024-09-15T21:47:57.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071804599999999996, 2024-09-15T21:47:57.469 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:47:57.469 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:47:57.469 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.469 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:47:57.469 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.469 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:47:57.469 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:57.469 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:47:57.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:57.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:47:57.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:47:57.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:47:57.470 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-15T21:47:57.470 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.470 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.470 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:47:57.470 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564841+0000", 2024-09-15T21:47:57.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-15T21:47:57.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:47:57.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565126+0000", 2024-09-15T21:47:57.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028469900000000002 2024-09-15T21:47:57.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.471 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565133+0000", 2024-09-15T21:47:57.471 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9480000000000002e-06 2024-09-15T21:47:57.471 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.471 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.471 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:47:57.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568006+0000", 2024-09-15T21:47:57.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028730330000000001 2024-09-15T21:47:57.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:47:57.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568119+0000", 2024-09-15T21:47:57.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011325899999999999 2024-09-15T21:47:57.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.472 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:47:57.472 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568235+0000", 2024-09-15T21:47:57.472 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115868 2024-09-15T21:47:57.472 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.472 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:47:57.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569668+0000", 2024-09-15T21:47:57.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001433099 2024-09-15T21:47:57.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571926+0000", 2024-09-15T21:47:57.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022572410000000001 2024-09-15T21:47:57.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.473 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.473 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:47:57.473 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571981+0000", 2024-09-15T21:47:57.473 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5383000000000002e-05 2024-09-15T21:47:57.473 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:47:57.474 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:47:57.474 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:47:57.474 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572020+0000", 2024-09-15T21:47:57.474 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8574e-05 2024-09-15T21:47:57.474 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.474 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.474 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.474 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:47:57.474 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:47:57.474 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:47:57.476 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:47:57.476 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:57.474+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:57.577 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:57.575+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:57.639 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:57.639 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:57.639 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:57.639 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:57.652 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:47:57.677 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:57.676+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:57.779 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:57.777+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:57.799 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:57.800 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:57.800 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:57.800 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:57.800 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:57.809 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:47:57.879 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:57.878+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:57.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.903+0000 7f0459ffb640 1 -- 172.21.15.100:0/3002241708 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v46) ==== 91+0+0 (secure 0 0 0) 0x7f045c03ae50 con 0x7f0464155440 2024-09-15T21:47:57.906 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is unset 2024-09-15T21:47:57.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.906+0000 7f046b9bd640 1 -- 172.21.15.100:0/3002241708 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f044806c2b0 msgr2=0x7f044806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:57.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.906+0000 7f046b9bd640 1 --2- 172.21.15.100:0/3002241708 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f044806c2b0 0x7f044806e770 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0460009960 tx=0x7f0460009240 comp rx=0 tx=0).stop 2024-09-15T21:47:57.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.906+0000 7f046b9bd640 1 -- 172.21.15.100:0/3002241708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0464155440 msgr2=0x7f04641f19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:57.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.906+0000 7f046b9bd640 1 --2- 172.21.15.100:0/3002241708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0464155440 0x7f04641f19f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f045c009e50 tx=0x7f045c004a40 comp rx=0 tx=0).stop 2024-09-15T21:47:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.908+0000 7f046b9bd640 1 -- 172.21.15.100:0/3002241708 shutdown_connections 2024-09-15T21:47:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.908+0000 7f046b9bd640 1 --2- 172.21.15.100:0/3002241708 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f044806c2b0 0x7f044806e770 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.908+0000 7f046b9bd640 1 --2- 172.21.15.100:0/3002241708 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0464155440 0x7f04641f19f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:57.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.908+0000 7f046b9bd640 1 --2- 172.21.15.100:0/3002241708 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f04640cc8f0 0x7f04641f14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.908+0000 7f046b9bd640 1 --2- 172.21.15.100:0/3002241708 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04640cb7d0 0x7f0464153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.908+0000 7f046b9bd640 1 -- 172.21.15.100:0/3002241708 >> 172.21.15.100:0/3002241708 conn(0x7f04640680d0 msgr2=0x7f046414b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.910+0000 7f046b9bd640 1 -- 172.21.15.100:0/3002241708 shutdown_connections 2024-09-15T21:47:57.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:57.910+0000 7f046b9bd640 1 -- 172.21.15.100:0/3002241708 wait complete. 2024-09-15T21:47:57.962 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:57.962 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:57.962 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:57.962 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:57.962 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.963 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:32 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'18, mlcod=17'14)", 2024-09-15T21:47:57.963 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:57.963 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 103.602142734, 2024-09-15T21:47:57.963 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052297760000000002, 2024-09-15T21:47:57.963 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:57.963 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:57.963 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.963 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:57.963 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.963 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:57.963 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:57.963 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:57.963 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.963 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.964 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:57.964 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:57.964 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:57.964 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.964 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.964 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:57.964 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:47:57.964 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.06e-07 2024-09-15T21:47:57.964 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.964 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.964 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:57.964 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356780+0000", 2024-09-15T21:47:57.964 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4208999999999994e-05 2024-09-15T21:47:57.964 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.965 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.965 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:57.965 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356789+0000", 2024-09-15T21:47:57.965 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2800000000000003e-06 2024-09-15T21:47:57.965 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.965 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.965 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.965 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356793+0000", 2024-09-15T21:47:57.965 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:47:57.965 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.965 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.965 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:57.965 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356838+0000", 2024-09-15T21:47:57.966 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4851999999999997e-05 2024-09-15T21:47:57.966 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.966 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.966 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:57.966 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356854+0000", 2024-09-15T21:47:57.966 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6124999999999999e-05 2024-09-15T21:47:57.966 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.966 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.966 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:57.966 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.361892+0000", 2024-09-15T21:47:57.966 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050379140000000001 2024-09-15T21:47:57.966 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.966 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.966 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:57.967 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.361935+0000", 2024-09-15T21:47:57.967 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3217999999999999e-05 2024-09-15T21:47:57.967 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:57.967 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:57.967 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:57.967 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.967 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.967 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:35 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'21, mlcod=17'15)", 2024-09-15T21:47:57.967 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:57.967 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 103.60010560400001, 2024-09-15T21:47:57.967 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055652039999999998, 2024-09-15T21:47:57.967 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:57.967 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:57.967 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.968 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:57.968 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.968 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:57.968 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:57.968 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:57.968 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.968 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.968 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:57.968 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:47:57.968 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:57.969 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.969 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.969 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:57.969 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358744+0000", 2024-09-15T21:47:57.969 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4399999999999998e-07 2024-09-15T21:47:57.969 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.970 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.970 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:57.970 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358773+0000", 2024-09-15T21:47:57.970 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8847e-05 2024-09-15T21:47:57.970 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.970 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.970 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:57.970 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358783+0000", 2024-09-15T21:47:57.970 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0299000000000001e-05 2024-09-15T21:47:57.970 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.970 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.970 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.970 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358796+0000", 2024-09-15T21:47:57.970 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3499000000000001e-05 2024-09-15T21:47:57.971 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.971 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.971 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:57.971 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358850+0000", 2024-09-15T21:47:57.971 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3952000000000001e-05 2024-09-15T21:47:57.971 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.971 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.971 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:57.971 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358872+0000", 2024-09-15T21:47:57.971 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1424999999999999e-05 2024-09-15T21:47:57.971 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.971 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.971 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:57.971 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364273+0000", 2024-09-15T21:47:57.972 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054011299999999996 2024-09-15T21:47:57.972 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.972 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.972 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:57.972 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364308+0000", 2024-09-15T21:47:57.972 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5108000000000003e-05 2024-09-15T21:47:57.972 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:57.972 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:57.972 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:57.972 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.972 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.972 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:36 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'22, mlcod=17'15)", 2024-09-15T21:47:57.972 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:57.972 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 103.599311414, 2024-09-15T21:47:57.973 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047904530000000001, 2024-09-15T21:47:57.973 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:57.973 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:57.973 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.973 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:57.973 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.973 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:57.973 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:57.973 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:57.973 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.973 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.973 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:57.973 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:57.973 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:57.974 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.974 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.974 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:57.974 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:47:57.974 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6000000000000004e-07 2024-09-15T21:47:57.974 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.974 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.974 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:57.974 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359542+0000", 2024-09-15T21:47:57.974 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3510000000000004e-06 2024-09-15T21:47:57.974 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.974 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.974 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:57.974 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359552+0000", 2024-09-15T21:47:57.974 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0263e-05 2024-09-15T21:47:57.975 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.975 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.975 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.975 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359557+0000", 2024-09-15T21:47:57.975 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0479999999999998e-06 2024-09-15T21:47:57.975 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.975 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.975 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:57.975 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359593+0000", 2024-09-15T21:47:57.975 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6151e-05 2024-09-15T21:47:57.975 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.975 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.975 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:57.975 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359608+0000", 2024-09-15T21:47:57.976 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4609e-05 2024-09-15T21:47:57.976 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.976 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.976 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:57.976 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364310+0000", 2024-09-15T21:47:57.976 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047025510000000001 2024-09-15T21:47:57.976 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.976 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.976 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:57.976 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364327+0000", 2024-09-15T21:47:57.976 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6920000000000001e-05 2024-09-15T21:47:57.976 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:57.976 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:57.976 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:57.977 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.977 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.977 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=18'85)", 2024-09-15T21:47:57.977 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:57.977 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 102.060765479, 2024-09-15T21:47:57.977 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057443499999999996, 2024-09-15T21:47:57.977 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:57.977 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:57.977 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.977 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:57.977 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.977 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:57.977 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:57.978 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:57.978 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.978 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.978 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:57.978 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:57.978 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:57.978 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.978 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.978 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:57.978 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:47:57.978 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8899999999999999e-07 2024-09-15T21:47:57.978 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.978 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.978 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:57.978 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898167+0000", 2024-09-15T21:47:57.979 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3171999999999996e-05 2024-09-15T21:47:57.979 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.979 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.979 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:57.979 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898177+0000", 2024-09-15T21:47:57.979 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0866000000000001e-05 2024-09-15T21:47:57.979 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.979 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.979 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.980 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898185+0000", 2024-09-15T21:47:57.980 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2490000000000004e-06 2024-09-15T21:47:57.980 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.980 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.980 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:57.980 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898224+0000", 2024-09-15T21:47:57.980 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9237999999999999e-05 2024-09-15T21:47:57.980 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.980 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.980 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:57.980 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898238+0000", 2024-09-15T21:47:57.980 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.366e-05 2024-09-15T21:47:57.980 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.980 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.981 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:57.981 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.903803+0000", 2024-09-15T21:47:57.981 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005565291 2024-09-15T21:47:57.981 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.981 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.981 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:57.981 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.903827+0000", 2024-09-15T21:47:57.981 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4284999999999999e-05 2024-09-15T21:47:57.981 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:57.981 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:57.981 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:57.981 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.981 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.981 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:132 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'90, mlcod=18'85)", 2024-09-15T21:47:57.982 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:57.982 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 102.060273755, 2024-09-15T21:47:57.982 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055085469999999999, 2024-09-15T21:47:57.982 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:57.982 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:57.982 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.982 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:57.982 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.982 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:57.982 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:57.982 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:57.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.982 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.982 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:57.983 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:57.983 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:57.983 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.983 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.983 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:57.983 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:47:57.983 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9299999999999999e-07 2024-09-15T21:47:57.983 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.983 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.983 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:57.984 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898579+0000", 2024-09-15T21:47:57.984 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6650000000000001e-06 2024-09-15T21:47:57.984 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.984 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.984 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:57.984 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898586+0000", 2024-09-15T21:47:57.984 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8429999999999993e-06 2024-09-15T21:47:57.984 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.984 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.984 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.984 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898592+0000", 2024-09-15T21:47:57.984 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3990000000000003e-06 2024-09-15T21:47:57.984 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.985 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.985 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:57.985 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898621+0000", 2024-09-15T21:47:57.985 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9019000000000001e-05 2024-09-15T21:47:57.985 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.985 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.985 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:57.985 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898630+0000", 2024-09-15T21:47:57.985 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5540000000000003e-06 2024-09-15T21:47:57.985 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.985 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.985 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:57.985 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904047+0000", 2024-09-15T21:47:57.985 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054167169999999997 2024-09-15T21:47:57.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.986 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.986 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:57.986 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904083+0000", 2024-09-15T21:47:57.986 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5957000000000001e-05 2024-09-15T21:47:57.986 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:57.986 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:57.986 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:57.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.986 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.986 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-15T21:47:57.986 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:57.986 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 102.059684606, 2024-09-15T21:47:57.986 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049424780000000001, 2024-09-15T21:47:57.987 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:57.987 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:57.987 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.987 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:57.987 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.987 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:57.987 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:57.987 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:57.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.987 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.987 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:57.987 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:47:57.987 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:57.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.988 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.988 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:57.988 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899166+0000", 2024-09-15T21:47:57.988 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0779999999999998e-06 2024-09-15T21:47:57.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.988 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.988 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:57.988 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899174+0000", 2024-09-15T21:47:57.988 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.729e-06 2024-09-15T21:47:57.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.988 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.988 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:57.988 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899181+0000", 2024-09-15T21:47:57.988 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4080000000000001e-06 2024-09-15T21:47:57.989 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.989 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.989 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.989 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899185+0000", 2024-09-15T21:47:57.989 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0840000000000002e-06 2024-09-15T21:47:57.989 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.989 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.989 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:57.989 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899240+0000", 2024-09-15T21:47:57.989 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4874999999999999e-05 2024-09-15T21:47:57.989 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.989 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.989 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:57.989 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899257+0000", 2024-09-15T21:47:57.989 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7549000000000001e-05 2024-09-15T21:47:57.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.990 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.990 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:57.990 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904085+0000", 2024-09-15T21:47:57.990 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048272640000000004 2024-09-15T21:47:57.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.990 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.990 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:57.990 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904106+0000", 2024-09-15T21:47:57.990 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1491000000000001e-05 2024-09-15T21:47:57.990 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:57.990 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:57.991 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:57.991 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.991 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.991 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:57.991 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:57.991 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 89.545461329999995, 2024-09-15T21:47:57.991 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046608459999999997, 2024-09-15T21:47:57.992 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:57.992 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:57.992 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.992 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:57.992 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.992 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:57.992 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:57.992 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:57.992 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.992 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.992 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:57.992 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:47:57.992 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:57.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.993 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.993 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:57.993 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413391+0000", 2024-09-15T21:47:57.993 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9899999999999999e-06 2024-09-15T21:47:57.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.993 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.993 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:57.993 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413399+0000", 2024-09-15T21:47:57.993 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6310000000000003e-06 2024-09-15T21:47:57.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.993 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.993 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:57.993 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413444+0000", 2024-09-15T21:47:57.994 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5052000000000002e-05 2024-09-15T21:47:57.994 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.994 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.994 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.994 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413453+0000", 2024-09-15T21:47:57.994 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8869999999999998e-06 2024-09-15T21:47:57.994 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.994 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.994 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:57.994 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413576+0000", 2024-09-15T21:47:57.994 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012358700000000001 2024-09-15T21:47:57.994 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.994 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.994 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:57.994 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413679+0000", 2024-09-15T21:47:57.995 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000103318 2024-09-15T21:47:57.995 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.995 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.995 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:57.995 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413785+0000", 2024-09-15T21:47:57.995 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010561 2024-09-15T21:47:57.995 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.995 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.995 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:57.995 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416276+0000", 2024-09-15T21:47:57.995 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002490754 2024-09-15T21:47:57.995 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.995 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.995 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:57.996 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417949+0000", 2024-09-15T21:47:57.996 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016733519999999999 2024-09-15T21:47:57.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.996 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.996 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:57.996 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418005+0000", 2024-09-15T21:47:57.996 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5747e-05 2024-09-15T21:47:57.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.996 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.996 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:57.996 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418048+0000", 2024-09-15T21:47:57.996 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2917999999999998e-05 2024-09-15T21:47:57.996 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:57.997 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:57.997 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:57.997 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.997 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.997 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:57.997 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:57.997 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 89.545446859999998, 2024-09-15T21:47:57.997 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051419999999999999, 2024-09-15T21:47:57.997 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:57.997 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:57.997 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:57.997 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:57.997 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.997 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:57.998 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:57.998 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:57.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.998 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.998 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:57.998 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:47:57.998 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:57.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.998 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.998 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:57.998 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413402+0000", 2024-09-15T21:47:57.998 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6899999999999998e-07 2024-09-15T21:47:57.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.998 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.999 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:57.999 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413405+0000", 2024-09-15T21:47:57.999 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8109999999999999e-06 2024-09-15T21:47:57.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.999 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.999 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:57.999 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413517+0000", 2024-09-15T21:47:57.999 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000112592 2024-09-15T21:47:57.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:57.999 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:57.999 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:57.999 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413527+0000", 2024-09-15T21:47:57.999 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0144e-05 2024-09-15T21:47:57.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.000 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.000 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.000 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414155+0000", 2024-09-15T21:47:58.000 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00062767200000000002 2024-09-15T21:47:58.000 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.000 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.000 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.000 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414239+0000", 2024-09-15T21:47:58.000 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3607000000000001e-05 2024-09-15T21:47:58.000 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.000 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.000 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:58.000 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414368+0000", 2024-09-15T21:47:58.000 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000129049 2024-09-15T21:47:58.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.001 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.001 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.001 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416332+0000", 2024-09-15T21:47:58.001 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019645109999999999 2024-09-15T21:47:58.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.001 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.001 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.001 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418446+0000", 2024-09-15T21:47:58.001 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021137640000000002 2024-09-15T21:47:58.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.001 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.001 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.001 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418503+0000", 2024-09-15T21:47:58.001 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7166999999999997e-05 2024-09-15T21:47:58.002 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.002 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.002 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.002 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418543+0000", 2024-09-15T21:47:58.002 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0314000000000001e-05 2024-09-15T21:47:58.002 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.002 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.002 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.002 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.002 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.002 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:58.002 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:58.002 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 89.545433669000005, 2024-09-15T21:47:58.002 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057945979999999998, 2024-09-15T21:47:58.003 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.003 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.003 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.003 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.003 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.003 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.003 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:58.003 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.003 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.003 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.003 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:58.003 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.004 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.004 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.004 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:47:58.004 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9499999999999998e-07 2024-09-15T21:47:58.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.004 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.004 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.004 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413417+0000", 2024-09-15T21:47:58.004 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3910000000000001e-06 2024-09-15T21:47:58.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.004 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.004 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.004 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414570+0000", 2024-09-15T21:47:58.004 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011525890000000001 2024-09-15T21:47:58.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.005 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.005 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.005 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414577+0000", 2024-09-15T21:47:58.005 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1840000000000002e-06 2024-09-15T21:47:58.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.005 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.005 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.005 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414733+0000", 2024-09-15T21:47:58.005 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015563900000000001 2024-09-15T21:47:58.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.005 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.005 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.005 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414831+0000", 2024-09-15T21:47:58.005 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8138000000000001e-05 2024-09-15T21:47:58.006 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.006 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.006 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:58.006 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414956+0000", 2024-09-15T21:47:58.006 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000124604 2024-09-15T21:47:58.006 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.006 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.006 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.006 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418804+0000", 2024-09-15T21:47:58.006 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038484299999999999 2024-09-15T21:47:58.006 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.006 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.006 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.007 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419124+0000", 2024-09-15T21:47:58.007 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00032017699999999999 2024-09-15T21:47:58.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.007 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.007 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.007 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419171+0000", 2024-09-15T21:47:58.007 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6520000000000002e-05 2024-09-15T21:47:58.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.007 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.007 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.007 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419209+0000", 2024-09-15T21:47:58.007 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8531000000000001e-05 2024-09-15T21:47:58.007 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.007 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.008 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.008 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.008 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.008 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:58.008 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:58.008 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 89.545426573, 2024-09-15T21:47:58.008 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059183300000000003, 2024-09-15T21:47:58.008 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.008 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.008 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:58.008 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.008 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.008 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.008 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:58.009 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.009 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.009 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.009 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.009 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:58.009 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.009 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.009 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.009 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.009 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:47:58.010 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3000000000000002e-07 2024-09-15T21:47:58.010 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.010 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.010 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.010 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413426+0000", 2024-09-15T21:47:58.010 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7110000000000001e-06 2024-09-15T21:47:58.010 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.010 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.010 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.010 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414621+0000", 2024-09-15T21:47:58.010 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001195022 2024-09-15T21:47:58.010 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.010 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.010 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.011 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414625+0000", 2024-09-15T21:47:58.011 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6360000000000002e-06 2024-09-15T21:47:58.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.011 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.011 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.011 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415269+0000", 2024-09-15T21:47:58.011 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00064315399999999995 2024-09-15T21:47:58.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.011 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.011 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.012 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415356+0000", 2024-09-15T21:47:58.012 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7410999999999994e-05 2024-09-15T21:47:58.012 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.012 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.012 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.012 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419340+0000", 2024-09-15T21:47:58.012 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0039840659999999996 2024-09-15T21:47:58.012 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.012 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.012 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.012 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.013 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.013 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-15T21:47:58.013 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:58.013 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 89.545118982000005, 2024-09-15T21:47:58.013 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053379719999999999, 2024-09-15T21:47:58.013 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.013 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.013 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.013 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.013 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.013 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.014 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:58.014 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.014 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.014 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.014 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:47:58.014 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.014 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.014 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.014 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413730+0000", 2024-09-15T21:47:58.014 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1379999999999999e-06 2024-09-15T21:47:58.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.015 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.015 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.015 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413740+0000", 2024-09-15T21:47:58.015 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4830000000000005e-06 2024-09-15T21:47:58.015 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.015 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.015 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.015 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413751+0000", 2024-09-15T21:47:58.015 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1228e-05 2024-09-15T21:47:58.015 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.015 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.015 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.015 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413758+0000", 2024-09-15T21:47:58.016 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1289999999999998e-06 2024-09-15T21:47:58.016 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.016 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.016 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.016 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414038+0000", 2024-09-15T21:47:58.016 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00027935999999999998 2024-09-15T21:47:58.016 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.016 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.016 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.016 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414055+0000", 2024-09-15T21:47:58.016 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7325000000000001e-05 2024-09-15T21:47:58.016 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.016 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.016 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.017 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419028+0000", 2024-09-15T21:47:58.017 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049730720000000003 2024-09-15T21:47:58.017 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.017 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.017 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.017 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419067+0000", 2024-09-15T21:47:58.017 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9236999999999997e-05 2024-09-15T21:47:58.017 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.017 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.017 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.017 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.017 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.017 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.017 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:58.018 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 83.503195715000004, 2024-09-15T21:47:58.018 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.92695576499999999, 2024-09-15T21:47:58.018 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.018 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.018 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.018 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.018 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.018 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.018 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:58.018 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.018 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.018 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.018 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.018 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:58.018 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.019 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.019 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.019 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.019 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:47:58.019 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8500000000000002e-07 2024-09-15T21:47:58.019 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.019 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.019 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.019 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455655+0000", 2024-09-15T21:47:58.019 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.401e-06 2024-09-15T21:47:58.019 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.019 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.019 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.019 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.380965+0000", 2024-09-15T21:47:58.020 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.92530946000000003 2024-09-15T21:47:58.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.020 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.020 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.020 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.380981+0000", 2024-09-15T21:47:58.020 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6209e-05 2024-09-15T21:47:58.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.020 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.020 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.020 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381086+0000", 2024-09-15T21:47:58.020 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105011 2024-09-15T21:47:58.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.020 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.020 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.021 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381154+0000", 2024-09-15T21:47:58.021 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7596999999999997e-05 2024-09-15T21:47:58.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.021 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.021 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:58.021 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381249+0000", 2024-09-15T21:47:58.021 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5354999999999998e-05 2024-09-15T21:47:58.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.021 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.021 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.021 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381844+0000", 2024-09-15T21:47:58.021 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00059518600000000004 2024-09-15T21:47:58.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.022 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.022 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.022 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382531+0000", 2024-09-15T21:47:58.022 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00068671499999999996 2024-09-15T21:47:58.022 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.022 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.022 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.022 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382570+0000", 2024-09-15T21:47:58.022 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9373999999999999e-05 2024-09-15T21:47:58.022 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.022 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.022 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.022 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382608+0000", 2024-09-15T21:47:58.022 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8071999999999997e-05 2024-09-15T21:47:58.022 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.023 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.023 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.023 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.023 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.023 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:58.023 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 77.483286456000002, 2024-09-15T21:47:58.023 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047117319999999997, 2024-09-15T21:47:58.023 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.023 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.023 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.023 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.023 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.023 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.024 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:58.024 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.024 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.024 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.024 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.024 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:58.024 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.024 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.024 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.024 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.024 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:47:58.024 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4700000000000001e-07 2024-09-15T21:47:58.024 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.025 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.025 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.025 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475566+0000", 2024-09-15T21:47:58.025 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1669999999999999e-06 2024-09-15T21:47:58.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.025 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.025 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.025 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476316+0000", 2024-09-15T21:47:58.025 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075022999999999995 2024-09-15T21:47:58.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.025 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.025 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.025 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476324+0000", 2024-09-15T21:47:58.025 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.259e-06 2024-09-15T21:47:58.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.026 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.026 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.026 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476373+0000", 2024-09-15T21:47:58.026 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8807999999999998e-05 2024-09-15T21:47:58.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.026 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.026 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.026 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476446+0000", 2024-09-15T21:47:58.026 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3138000000000003e-05 2024-09-15T21:47:58.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.026 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.026 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:58.026 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476593+0000", 2024-09-15T21:47:58.026 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00014680199999999999 2024-09-15T21:47:58.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.027 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.027 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.027 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480189+0000", 2024-09-15T21:47:58.027 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035961370000000001 2024-09-15T21:47:58.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.027 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.027 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.027 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480198+0000", 2024-09-15T21:47:58.027 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.6649999999999992e-06 2024-09-15T21:47:58.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.027 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.027 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.027 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480235+0000", 2024-09-15T21:47:58.028 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6943999999999997e-05 2024-09-15T21:47:58.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.028 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.028 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.028 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480274+0000", 2024-09-15T21:47:58.028 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9035000000000001e-05 2024-09-15T21:47:58.028 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.028 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.028 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.028 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.028 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.028 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:58.028 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 77.483281453000004, 2024-09-15T21:47:58.029 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046830109999999999, 2024-09-15T21:47:58.029 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.029 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.029 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.029 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.029 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.029 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.029 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:58.029 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.029 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.029 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.029 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.029 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:58.029 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.030 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.030 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.030 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:58.030 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.53e-07 2024-09-15T21:47:58.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.030 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.030 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.030 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475570+0000", 2024-09-15T21:47:58.030 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7470000000000001e-06 2024-09-15T21:47:58.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.030 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.030 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.030 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476353+0000", 2024-09-15T21:47:58.031 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00078312300000000004 2024-09-15T21:47:58.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.031 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.031 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.031 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476359+0000", 2024-09-15T21:47:58.031 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.117e-06 2024-09-15T21:47:58.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.031 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.031 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.031 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476412+0000", 2024-09-15T21:47:58.031 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2398999999999998e-05 2024-09-15T21:47:58.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.031 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.031 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.032 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476524+0000", 2024-09-15T21:47:58.032 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011293 2024-09-15T21:47:58.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.032 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.032 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:58.032 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476909+0000", 2024-09-15T21:47:58.032 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00038490099999999999 2024-09-15T21:47:58.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.032 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.032 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.032 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479761+0000", 2024-09-15T21:47:58.032 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002851737 2024-09-15T21:47:58.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.032 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.033 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.033 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480154+0000", 2024-09-15T21:47:58.033 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00039307399999999998 2024-09-15T21:47:58.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.033 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.033 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.033 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480203+0000", 2024-09-15T21:47:58.033 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8418000000000003e-05 2024-09-15T21:47:58.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.033 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.033 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.033 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480250+0000", 2024-09-15T21:47:58.033 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7311999999999997e-05 2024-09-15T21:47:58.033 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.034 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.034 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.034 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.034 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:58.034 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 77.483277290999993, 2024-09-15T21:47:58.034 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005035398, 2024-09-15T21:47:58.034 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.034 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.034 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.034 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.035 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.035 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:58.035 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.035 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.035 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.035 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:58.035 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.035 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.035 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.035 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:47:58.035 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.22e-07 2024-09-15T21:47:58.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.035 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.036 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.036 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475574+0000", 2024-09-15T21:47:58.036 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.593e-06 2024-09-15T21:47:58.036 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.036 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.036 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.036 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476579+0000", 2024-09-15T21:47:58.036 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010053989999999999 2024-09-15T21:47:58.036 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.036 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.036 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.036 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476588+0000", 2024-09-15T21:47:58.036 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4530000000000002e-06 2024-09-15T21:47:58.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.037 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.037 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.037 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477723+0000", 2024-09-15T21:47:58.037 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001134962 2024-09-15T21:47:58.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.037 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.037 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.037 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477801+0000", 2024-09-15T21:47:58.037 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8228999999999999e-05 2024-09-15T21:47:58.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.037 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.037 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:58.037 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477946+0000", 2024-09-15T21:47:58.038 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00014553400000000001 2024-09-15T21:47:58.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.038 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.038 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.038 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480258+0000", 2024-09-15T21:47:58.038 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023112039999999999 2024-09-15T21:47:58.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.038 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.038 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.038 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480504+0000", 2024-09-15T21:47:58.038 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024667500000000002 2024-09-15T21:47:58.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.038 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.038 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.038 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480556+0000", 2024-09-15T21:47:58.039 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2114999999999998e-05 2024-09-15T21:47:58.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.039 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.039 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.039 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480606+0000", 2024-09-15T21:47:58.039 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0012000000000002e-05 2024-09-15T21:47:58.039 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.039 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.039 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.039 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.039 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.039 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:58.039 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 77.483273361000002, 2024-09-15T21:47:58.040 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050672490000000002, 2024-09-15T21:47:58.040 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.040 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.040 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.040 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.040 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.040 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.040 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:58.040 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.040 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.040 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.040 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.040 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:58.040 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.041 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.041 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.041 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:47:58.041 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:58.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.041 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.041 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.041 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475577+0000", 2024-09-15T21:47:58.041 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2290000000000002e-06 2024-09-15T21:47:58.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.041 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.041 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.041 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476605+0000", 2024-09-15T21:47:58.042 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010278780000000001 2024-09-15T21:47:58.042 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.042 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.042 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.042 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476611+0000", 2024-09-15T21:47:58.042 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5600000000000001e-06 2024-09-15T21:47:58.042 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.042 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.042 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.042 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477053+0000", 2024-09-15T21:47:58.042 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044162800000000001 2024-09-15T21:47:58.042 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.042 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.042 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.043 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477180+0000", 2024-09-15T21:47:58.043 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000127129 2024-09-15T21:47:58.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.043 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.043 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:58.043 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477288+0000", 2024-09-15T21:47:58.043 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010789999999999999 2024-09-15T21:47:58.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.043 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.043 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.043 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480288+0000", 2024-09-15T21:47:58.043 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003000052 2024-09-15T21:47:58.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.043 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.044 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.044 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480568+0000", 2024-09-15T21:47:58.044 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028026899999999998 2024-09-15T21:47:58.044 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.044 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.044 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.044 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480609+0000", 2024-09-15T21:47:58.044 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1437999999999999e-05 2024-09-15T21:47:58.044 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.044 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.044 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.044 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480642+0000", 2024-09-15T21:47:58.044 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2879999999999997e-05 2024-09-15T21:47:58.044 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.045 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.045 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.045 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.045 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.045 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:20 2.1f 2:fdd45aa0:::61c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.045 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:58.045 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 77.48326969, 2024-09-15T21:47:58.045 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053112589999999996, 2024-09-15T21:47:58.045 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.045 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.045 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.045 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.045 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.045 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.045 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:58.046 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.046 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.046 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.046 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.046 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:58.046 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.046 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.046 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.046 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.046 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:58.046 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:47:58.046 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.046 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.046 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.047 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475581+0000", 2024-09-15T21:47:58.047 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1009999999999999e-06 2024-09-15T21:47:58.047 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.047 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.047 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.047 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476628+0000", 2024-09-15T21:47:58.047 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010465380000000001 2024-09-15T21:47:58.047 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.047 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.047 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.047 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476633+0000", 2024-09-15T21:47:58.047 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9869999999999999e-06 2024-09-15T21:47:58.047 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.047 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.048 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.048 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477618+0000", 2024-09-15T21:47:58.048 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00098539299999999999 2024-09-15T21:47:58.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.048 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.048 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.048 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477696+0000", 2024-09-15T21:47:58.048 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8509000000000003e-05 2024-09-15T21:47:58.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.049 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.049 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:58.049 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477824+0000", 2024-09-15T21:47:58.049 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000127062 2024-09-15T21:47:58.049 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.049 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.049 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.049 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480294+0000", 2024-09-15T21:47:58.049 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024703939999999999 2024-09-15T21:47:58.049 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.050 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.050 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.050 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480788+0000", 2024-09-15T21:47:58.050 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00049443800000000004 2024-09-15T21:47:58.050 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.050 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.050 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.050 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480845+0000", 2024-09-15T21:47:58.050 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6762000000000003e-05 2024-09-15T21:47:58.050 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.051 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.051 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.051 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480890+0000", 2024-09-15T21:47:58.051 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4789000000000001e-05 2024-09-15T21:47:58.051 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.051 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.051 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.051 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.051 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.051 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.051 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:58.051 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 77.483266219000001, 2024-09-15T21:47:58.051 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007191546, 2024-09-15T21:47:58.052 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.052 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.052 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:58.052 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.052 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.052 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.052 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:58.052 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.052 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.052 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.052 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:58.052 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.053 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.053 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.053 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:47:58.053 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9999999999999999e-07 2024-09-15T21:47:58.053 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.053 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.053 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.053 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475585+0000", 2024-09-15T21:47:58.053 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.384e-06 2024-09-15T21:47:58.053 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.053 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.053 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.053 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476665+0000", 2024-09-15T21:47:58.053 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010797140000000001 2024-09-15T21:47:58.054 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.054 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.054 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.054 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476671+0000", 2024-09-15T21:47:58.054 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3940000000000001e-06 2024-09-15T21:47:58.054 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.054 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.054 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.054 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478048+0000", 2024-09-15T21:47:58.054 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013767530000000001 2024-09-15T21:47:58.054 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.054 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.054 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.054 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478177+0000", 2024-09-15T21:47:58.055 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012898099999999999 2024-09-15T21:47:58.055 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.055 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.055 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.055 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482774+0000", 2024-09-15T21:47:58.055 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045970200000000003 2024-09-15T21:47:58.055 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.055 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.055 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.055 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.055 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.055 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.055 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:58.055 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 72.414036554000006, 2024-09-15T21:47:58.056 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055444480000000004, 2024-09-15T21:47:58.056 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.056 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.056 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.056 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.056 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.056 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.056 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:58.056 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.056 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.056 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.056 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.056 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:58.056 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.057 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.057 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.057 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.057 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:47:58.057 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4200000000000002e-07 2024-09-15T21:47:58.057 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.057 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.057 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.057 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544814+0000", 2024-09-15T21:47:58.057 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9980000000000002e-06 2024-09-15T21:47:58.057 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.057 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.057 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.057 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544872+0000", 2024-09-15T21:47:58.057 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8405000000000001e-05 2024-09-15T21:47:58.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.058 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.058 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.058 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544877+0000", 2024-09-15T21:47:58.058 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.143e-06 2024-09-15T21:47:58.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.058 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.058 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.058 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545750+0000", 2024-09-15T21:47:58.058 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00087310000000000003 2024-09-15T21:47:58.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.058 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.058 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.058 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545850+0000", 2024-09-15T21:47:58.059 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9939999999999995e-05 2024-09-15T21:47:58.059 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.059 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.059 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:58.059 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546014+0000", 2024-09-15T21:47:58.059 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00016408500000000001 2024-09-15T21:47:58.059 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.059 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.059 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.059 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.548748+0000", 2024-09-15T21:47:58.059 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027339539999999998 2024-09-15T21:47:58.059 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.059 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.059 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.060 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550265+0000", 2024-09-15T21:47:58.060 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015170019999999999 2024-09-15T21:47:58.060 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.060 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.060 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.060 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550315+0000", 2024-09-15T21:47:58.060 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9867999999999998e-05 2024-09-15T21:47:58.060 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.060 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.060 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.060 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550356+0000", 2024-09-15T21:47:58.060 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1711000000000001e-05 2024-09-15T21:47:58.060 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.060 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.061 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.061 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.061 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.061 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.061 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:58.061 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 72.414033352999994, 2024-09-15T21:47:58.061 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058729619999999998, 2024-09-15T21:47:58.061 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.061 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.061 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.061 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.061 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.062 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.062 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:58.062 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.062 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.062 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.062 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.062 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:58.062 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.062 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.062 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.062 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.063 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:47:58.063 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0599999999999999e-07 2024-09-15T21:47:58.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.063 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.063 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.063 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544821+0000", 2024-09-15T21:47:58.063 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4079999999999997e-06 2024-09-15T21:47:58.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.063 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.063 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.063 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544939+0000", 2024-09-15T21:47:58.063 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118532 2024-09-15T21:47:58.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.063 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.063 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.064 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544946+0000", 2024-09-15T21:47:58.064 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9260000000000003e-06 2024-09-15T21:47:58.064 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.064 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.064 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.064 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545011+0000", 2024-09-15T21:47:58.064 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4572000000000005e-05 2024-09-15T21:47:58.064 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.064 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.064 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.064 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545134+0000", 2024-09-15T21:47:58.064 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012309899999999999 2024-09-15T21:47:58.064 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.065 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.065 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:58.065 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545271+0000", 2024-09-15T21:47:58.065 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013746599999999999 2024-09-15T21:47:58.065 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.065 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.065 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.065 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550243+0000", 2024-09-15T21:47:58.065 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049714529999999998 2024-09-15T21:47:58.065 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.065 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.065 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.065 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550605+0000", 2024-09-15T21:47:58.065 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00036260699999999998 2024-09-15T21:47:58.065 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.066 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.066 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.066 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550662+0000", 2024-09-15T21:47:58.066 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6573000000000002e-05 2024-09-15T21:47:58.066 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.066 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.066 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.066 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550688+0000", 2024-09-15T21:47:58.066 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6120000000000001e-05 2024-09-15T21:47:58.066 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.066 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.066 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.066 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.066 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.067 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:58.067 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-15T21:47:58.067 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:47:58.067 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:47:58.067 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 7 2024-09-15T21:47:58.069 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:47:58.070 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:58.067+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:58.169 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:58.168+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:58.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.260+0000 7f6fb2f31640 1 Processor -- start 2024-09-15T21:47:58.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.260+0000 7f6fb2f31640 1 -- start start 2024-09-15T21:47:58.263 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:58.263 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:58.263 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:58.263 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:58.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.261+0000 7f6fb2f31640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6fac155440 0x7f6fac0cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:58.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.261+0000 7f6fb2f31640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6fac0cc8f0 0x7f6fac0cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:58.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.262+0000 7f6fb2f31640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fac15e7a0 con 0x7f6fac0cb7d0 2024-09-15T21:47:58.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.262+0000 7f6fb2f31640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f6fac15b470 con 0x7f6fac0cc8f0 2024-09-15T21:47:58.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.262+0000 7f6fb2f31640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6fac155820 con 0x7f6fac155440 2024-09-15T21:47:58.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.262+0000 7f6fabfff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f6fac0cb7d0 0x7f6fac0cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57414/0 (socket says 172.21.15.100:57414) 2024-09-15T21:47:58.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.262+0000 7f6fabfff640 1 -- 172.21.15.100:0/3292608670 learned_addr learned my addr 172.21.15.100:0/3292608670 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:47:58.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.262+0000 7f6fb0ccd640 1 --2- 172.21.15.100:0/3292608670 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6fac155440 0x7f6fac0cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:58.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.262+0000 7f6fb14ce640 1 --2- 172.21.15.100:0/3292608670 >> v2:172.21.15.122:3300/0 conn(0x7f6fac0cc8f0 0x7f6fac0cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:58.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.262+0000 7f6fb14ce640 1 -- 172.21.15.100:0/3292608670 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6fac155440 msgr2=0x7f6fac0cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:58.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.262+0000 7f6fb14ce640 1 --2- 172.21.15.100:0/3292608670 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6fac155440 0x7f6fac0cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:58.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.262+0000 7f6fb14ce640 1 -- 172.21.15.100:0/3292608670 >> v1:172.21.15.100:6789/0 conn(0x7f6fac0cb7d0 legacy=0x7f6fac0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:58.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.262+0000 7f6fb14ce640 1 -- 172.21.15.100:0/3292608670 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fac15ea40 con 0x7f6fac0cc8f0 2024-09-15T21:47:58.266 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.263+0000 7f6fb14ce640 1 --2- 172.21.15.100:0/3292608670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fac0cc8f0 0x7f6fac0cd960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009d90 tx=0x7f6fa402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=795d98517a0180cb server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:58.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.264+0000 7f6fab7fe640 1 -- 172.21.15.100:0/3292608670 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4007740 con 0x7f6fac0cc8f0 2024-09-15T21:47:58.267 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.264+0000 7f6fab7fe640 1 -- 172.21.15.100:0/3292608670 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fa40078e0 con 0x7f6fac0cc8f0 2024-09-15T21:47:58.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.264+0000 7f6fab7fe640 1 -- 172.21.15.100:0/3292608670 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4007cb0 con 0x7f6fac0cc8f0 2024-09-15T21:47:58.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.265+0000 7f6fb2f31640 1 -- 172.21.15.100:0/3292608670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fac0cc8f0 msgr2=0x7f6fac0cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:58.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.265+0000 7f6fb2f31640 1 --2- 172.21.15.100:0/3292608670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fac0cc8f0 0x7f6fac0cd960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009d90 tx=0x7f6fa402a660 comp rx=0 tx=0).stop 2024-09-15T21:47:58.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.265+0000 7f6fb2f31640 1 -- 172.21.15.100:0/3292608670 shutdown_connections 2024-09-15T21:47:58.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.265+0000 7f6fb2f31640 1 --2- 172.21.15.100:0/3292608670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fac0cc8f0 0x7f6fac0cd960 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:58.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.265+0000 7f6fb2f31640 1 --2- 172.21.15.100:0/3292608670 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6fac155440 0x7f6fac0cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:58.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.265+0000 7f6fb2f31640 1 -- 172.21.15.100:0/3292608670 >> 172.21.15.100:0/3292608670 conn(0x7f6fac0680d0 msgr2=0x7f6fac06d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:58.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.265+0000 7f6fb2f31640 1 -- 172.21.15.100:0/3292608670 shutdown_connections 2024-09-15T21:47:58.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.265+0000 7f6fb2f31640 1 -- 172.21.15.100:0/3292608670 wait complete. 2024-09-15T21:47:58.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.267+0000 7f6fb2f31640 1 Processor -- start 2024-09-15T21:47:58.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.267+0000 7f6fb2f31640 1 -- start start 2024-09-15T21:47:58.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.268+0000 7f6fb2f31640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fac0cb7d0 0x7f6fac152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:58.270 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:47:58.268+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:47:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.268+0000 7f6fb0ccd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fac0cb7d0 0x7f6fac152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.268+0000 7f6fb0ccd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fac0cb7d0 0x7f6fac152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57684/0 (socket says 172.21.15.100:57684) 2024-09-15T21:47:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.268+0000 7f6fb0ccd640 1 -- 172.21.15.100:0/3486289395 learned_addr learned my addr 172.21.15.100:0/3486289395 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.268+0000 7f6fb2f31640 1 --2- 172.21.15.100:0/3486289395 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6fac0cc8f0 0x7f6fac15a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.269+0000 7f6fb2f31640 1 --2- 172.21.15.100:0/3486289395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fac155440 0x7f6fac148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:58.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.269+0000 7f6fb2f31640 1 -- 172.21.15.100:0/3486289395 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac15f2e0 con 0x7f6fac0cb7d0 2024-09-15T21:47:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.269+0000 7f6fb2f31640 1 -- 172.21.15.100:0/3486289395 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac15f000 con 0x7f6fac155440 2024-09-15T21:47:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.269+0000 7f6fb2f31640 1 -- 172.21.15.100:0/3486289395 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6fac15b070 con 0x7f6fac0cc8f0 2024-09-15T21:47:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.269+0000 7f6fabfff640 1 --2- 172.21.15.100:0/3486289395 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6fac0cc8f0 0x7f6fac15a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.269+0000 7f6fb14ce640 1 --2- 172.21.15.100:0/3486289395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fac155440 0x7f6fac148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.269+0000 7f6fabfff640 1 -- 172.21.15.100:0/3486289395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fac155440 msgr2=0x7f6fac148dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.269+0000 7f6fabfff640 1 --2- 172.21.15.100:0/3486289395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fac155440 0x7f6fac148dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.269+0000 7f6fabfff640 1 -- 172.21.15.100:0/3486289395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fac0cb7d0 msgr2=0x7f6fac152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.269+0000 7f6fabfff640 1 --2- 172.21.15.100:0/3486289395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fac0cb7d0 0x7f6fac152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.269+0000 7f6fabfff640 1 -- 172.21.15.100:0/3486289395 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fac152620 con 0x7f6fac0cc8f0 2024-09-15T21:47:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.269+0000 7f6fb0ccd640 1 --2- 172.21.15.100:0/3486289395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fac0cb7d0 0x7f6fac152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:47:58.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.270+0000 7f6fabfff640 1 --2- 172.21.15.100:0/3486289395 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6fac0cc8f0 0x7f6fac15a2e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0cd3d0 tx=0x7f6f9800c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.270+0000 7f6fa97fa640 1 -- 172.21.15.100:0/3486289395 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98017070 con 0x7f6fac0cc8f0 2024-09-15T21:47:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.270+0000 7f6fa97fa640 1 -- 172.21.15.100:0/3486289395 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f98005ce0 con 0x7f6fac0cc8f0 2024-09-15T21:47:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.270+0000 7f6fa97fa640 1 -- 172.21.15.100:0/3486289395 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f98007310 con 0x7f6fac0cc8f0 2024-09-15T21:47:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.271+0000 7f6fb2f31640 1 -- 172.21.15.100:0/3486289395 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fac0b5b60 con 0x7f6fac0cc8f0 2024-09-15T21:47:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.271+0000 7f6fb2f31640 1 -- 172.21.15.100:0/3486289395 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fac0d0740 con 0x7f6fac0cc8f0 2024-09-15T21:47:58.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.272+0000 7f6fb2f31640 1 -- 172.21.15.100:0/3486289395 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fac0b3940 con 0x7f6fac0cc8f0 2024-09-15T21:47:58.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.273+0000 7f6fa97fa640 1 -- 172.21.15.100:0/3486289395 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6f9801a070 con 0x7f6fac0cc8f0 2024-09-15T21:47:58.276 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:47:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.274+0000 7f6fa97fa640 1 --2- 172.21.15.100:0/3486289395 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6f9006c2b0 0x7f6f9006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.274+0000 7f6fb0ccd640 1 --2- 172.21.15.100:0/3486289395 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6f9006c2b0 0x7f6f9006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.274+0000 7f6fa97fa640 1 -- 172.21.15.100:0/3486289395 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(46..46 src has 1..46) ==== 5886+0+0 (secure 0 0 0) 0x7f6f9808a640 con 0x7f6fac0cc8f0 2024-09-15T21:47:58.277 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.275+0000 7f6fb0ccd640 1 --2- 172.21.15.100:0/3486289395 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6f9006c2b0 0x7f6f9006e770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00112c0 tx=0x7f6fa0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:58.281 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.279+0000 7f6fa97fa640 1 -- 172.21.15.100:0/3486289395 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6f980586b0 con 0x7f6fac0cc8f0 2024-09-15T21:47:58.371 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:47:58.370+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:47:58.451 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:58.452 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:58.452 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:47:58.452 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:47:58.452 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:58.465 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:47:58.507 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:58.505+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:58.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.553+0000 7f6fb2f31640 1 -- 172.21.15.100:0/3486289395 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd out", "ids": ["7"]} v 0) -- 0x7f6fac1527f0 con 0x7f6fac0cc8f0 2024-09-15T21:47:58.556 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-09-15T21:47:58.554+0000 7f81c6dea640 -1 mon.a@0(leader).osd e46 definitely_dead 0 2024-09-15T21:47:58.574 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:58.572+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:58.672 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:58.672 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:47:58.672 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:47:58.672 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:47:58.673 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.673 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:58.673 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:58.673 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 90.256678268000002, 2024-09-15T21:47:58.673 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006101291, 2024-09-15T21:47:58.673 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.673 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.673 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.673 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.673 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.673 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.673 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:58.673 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.674 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.674 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.674 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:47:58.674 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.674 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.674 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.674 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413116+0000", 2024-09-15T21:47:58.674 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9700000000000003e-07 2024-09-15T21:47:58.674 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.674 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.674 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.674 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413118+0000", 2024-09-15T21:47:58.675 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2910000000000002e-06 2024-09-15T21:47:58.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.675 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.675 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.675 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413220+0000", 2024-09-15T21:47:58.675 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102056 2024-09-15T21:47:58.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.675 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.675 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.675 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413225+0000", 2024-09-15T21:47:58.675 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0490000000000003e-06 2024-09-15T21:47:58.675 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.675 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.675 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.676 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414808+0000", 2024-09-15T21:47:58.676 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015826900000000001 2024-09-15T21:47:58.676 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.676 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.676 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.676 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414907+0000", 2024-09-15T21:47:58.676 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9600999999999997e-05 2024-09-15T21:47:58.676 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.676 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.676 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:58.676 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415054+0000", 2024-09-15T21:47:58.676 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000146532 2024-09-15T21:47:58.676 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.676 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.677 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.677 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418056+0000", 2024-09-15T21:47:58.677 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030020519999999998 2024-09-15T21:47:58.677 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.677 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.677 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.677 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419128+0000", 2024-09-15T21:47:58.677 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001071671 2024-09-15T21:47:58.677 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.677 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.677 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.677 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419184+0000", 2024-09-15T21:47:58.677 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6125000000000002e-05 2024-09-15T21:47:58.677 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.678 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.678 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.678 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419217+0000", 2024-09-15T21:47:58.678 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2926999999999998e-05 2024-09-15T21:47:58.678 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.678 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.678 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.678 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.678 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.678 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:47:58.678 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:58.678 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 90.256674742000001, 2024-09-15T21:47:58.678 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059975130000000003, 2024-09-15T21:47:58.678 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.679 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.679 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.679 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.679 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.679 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.679 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:58.679 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.679 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.679 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.679 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:58.679 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.679 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.679 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.679 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.680 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:47:58.680 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6199999999999999e-07 2024-09-15T21:47:58.680 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.680 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.680 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.680 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413121+0000", 2024-09-15T21:47:58.680 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2000000000000001e-06 2024-09-15T21:47:58.680 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.680 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.680 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.680 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413292+0000", 2024-09-15T21:47:58.680 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017034400000000001 2024-09-15T21:47:58.680 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.681 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.681 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.681 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413300+0000", 2024-09-15T21:47:58.681 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8539999999999997e-06 2024-09-15T21:47:58.681 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.681 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.681 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.681 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414299+0000", 2024-09-15T21:47:58.681 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0009992829999999999 2024-09-15T21:47:58.681 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.681 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.681 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.681 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414390+0000", 2024-09-15T21:47:58.681 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1237999999999996e-05 2024-09-15T21:47:58.681 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.682 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.682 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:47:58.682 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414490+0000", 2024-09-15T21:47:58.682 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9438999999999994e-05 2024-09-15T21:47:58.682 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.682 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.682 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.682 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417393+0000", 2024-09-15T21:47:58.682 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002903306 2024-09-15T21:47:58.682 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.682 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.682 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.682 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419023+0000", 2024-09-15T21:47:58.682 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001630292 2024-09-15T21:47:58.682 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.683 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.683 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.683 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419067+0000", 2024-09-15T21:47:58.683 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4017999999999998e-05 2024-09-15T21:47:58.683 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.683 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.683 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.683 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419116+0000", 2024-09-15T21:47:58.683 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9277000000000003e-05 2024-09-15T21:47:58.683 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.683 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.683 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.683 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.683 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.684 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-15T21:47:58.684 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:58.684 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 90.253976545, 2024-09-15T21:47:58.684 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060879280000000003, 2024-09-15T21:47:58.684 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.684 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.684 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.684 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.684 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.684 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.684 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:58.684 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.684 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.684 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.685 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.685 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:47:58.685 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.685 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.685 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.685 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.685 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415818+0000", 2024-09-15T21:47:58.685 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:47:58.685 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.685 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.685 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.685 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415854+0000", 2024-09-15T21:47:58.685 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6706999999999999e-05 2024-09-15T21:47:58.685 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415860+0000", 2024-09-15T21:47:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0789999999999997e-06 2024-09-15T21:47:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415865+0000", 2024-09-15T21:47:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4580000000000003e-06 2024-09-15T21:47:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416092+0000", 2024-09-15T21:47:58.686 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00022739400000000001 2024-09-15T21:47:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416108+0000", 2024-09-15T21:47:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.613e-05 2024-09-15T21:47:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421866+0000", 2024-09-15T21:47:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005757585 2024-09-15T21:47:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.687 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421905+0000", 2024-09-15T21:47:58.688 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9214000000000001e-05 2024-09-15T21:47:58.688 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.688 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.688 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.688 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.688 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.688 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-15T21:47:58.688 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:58.688 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 90.253916384999997, 2024-09-15T21:47:58.689 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060637570000000003, 2024-09-15T21:47:58.689 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.689 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.689 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.689 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.689 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.689 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.689 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:58.689 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.689 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.689 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.689 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.689 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:47:58.689 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.690 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.690 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.690 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.690 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415878+0000", 2024-09-15T21:47:58.690 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.34e-07 2024-09-15T21:47:58.690 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.690 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.690 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.690 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415881+0000", 2024-09-15T21:47:58.690 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.354e-06 2024-09-15T21:47:58.690 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.690 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.690 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.690 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415887+0000", 2024-09-15T21:47:58.691 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:47:58.691 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.691 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.691 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.691 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415891+0000", 2024-09-15T21:47:58.691 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2690000000000001e-06 2024-09-15T21:47:58.691 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.691 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.691 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.691 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416267+0000", 2024-09-15T21:47:58.691 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00037614799999999998 2024-09-15T21:47:58.691 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.691 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.691 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.692 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416288+0000", 2024-09-15T21:47:58.692 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0604000000000001e-05 2024-09-15T21:47:58.692 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.692 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.692 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.692 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421908+0000", 2024-09-15T21:47:58.692 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056203390000000002 2024-09-15T21:47:58.692 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.692 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.692 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.692 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421941+0000", 2024-09-15T21:47:58.692 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3079e-05 2024-09-15T21:47:58.692 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.692 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.693 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.693 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.693 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.693 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-15T21:47:58.693 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:58.693 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 90.253897957999996, 2024-09-15T21:47:58.693 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006080284, 2024-09-15T21:47:58.693 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.693 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.693 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.693 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.693 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.693 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.693 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:58.694 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.694 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.694 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.694 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.694 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:58.694 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.694 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.694 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.694 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.695 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:47:58.695 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6899999999999999e-07 2024-09-15T21:47:58.695 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.695 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.695 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.695 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415898+0000", 2024-09-15T21:47:58.695 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.951e-06 2024-09-15T21:47:58.695 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.695 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.695 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.695 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415920+0000", 2024-09-15T21:47:58.695 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1651999999999999e-05 2024-09-15T21:47:58.695 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.696 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.696 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.696 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415925+0000", 2024-09-15T21:47:58.696 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.463e-06 2024-09-15T21:47:58.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.696 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.696 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.696 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416449+0000", 2024-09-15T21:47:58.696 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052384000000000003 2024-09-15T21:47:58.696 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.696 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.696 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.696 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416466+0000", 2024-09-15T21:47:58.696 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7108999999999998e-05 2024-09-15T21:47:58.697 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.697 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.697 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.697 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421943+0000", 2024-09-15T21:47:58.697 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054771569999999999 2024-09-15T21:47:58.697 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.697 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.697 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.697 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421976+0000", 2024-09-15T21:47:58.697 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2943e-05 2024-09-15T21:47:58.697 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.697 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.697 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.698 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.698 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.698 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-15T21:47:58.698 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:58.698 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 90.253858722000004, 2024-09-15T21:47:58.698 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060643340000000002, 2024-09-15T21:47:58.698 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.698 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.698 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.698 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.698 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.698 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.698 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:58.698 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.699 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.699 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.699 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.699 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:58.699 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.699 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.699 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.699 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.699 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:47:58.699 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4200000000000002e-07 2024-09-15T21:47:58.699 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.699 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.699 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.699 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415939+0000", 2024-09-15T21:47:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.439e-06 2024-09-15T21:47:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415945+0000", 2024-09-15T21:47:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7740000000000001e-06 2024-09-15T21:47:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415948+0000", 2024-09-15T21:47:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.737e-06 2024-09-15T21:47:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.700 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416605+0000", 2024-09-15T21:47:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00065721699999999998 2024-09-15T21:47:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416620+0000", 2024-09-15T21:47:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4564e-05 2024-09-15T21:47:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421977+0000", 2024-09-15T21:47:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053572369999999999 2024-09-15T21:47:58.701 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421999+0000", 2024-09-15T21:47:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2024000000000001e-05 2024-09-15T21:47:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-15T21:47:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 90.253658063000003, 2024-09-15T21:47:58.702 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058922669999999996, 2024-09-15T21:47:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.703 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:47:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6499999999999999e-07 2024-09-15T21:47:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416142+0000", 2024-09-15T21:47:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3929999999999999e-06 2024-09-15T21:47:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.704 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416151+0000", 2024-09-15T21:47:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5840000000000001e-06 2024-09-15T21:47:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416157+0000", 2024-09-15T21:47:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7590000000000003e-06 2024-09-15T21:47:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416767+0000", 2024-09-15T21:47:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00061034900000000005 2024-09-15T21:47:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.705 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416782+0000", 2024-09-15T21:47:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.483e-05 2024-09-15T21:47:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422002+0000", 2024-09-15T21:47:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052202639999999996 2024-09-15T21:47:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422028+0000", 2024-09-15T21:47:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5423e-05 2024-09-15T21:47:58.706 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-15T21:47:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 90.253251562000003, 2024-09-15T21:47:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055107769999999997, 2024-09-15T21:47:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.707 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:47:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.708 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416543+0000", 2024-09-15T21:47:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5400000000000001e-07 2024-09-15T21:47:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416548+0000", 2024-09-15T21:47:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8659999999999997e-06 2024-09-15T21:47:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416555+0000", 2024-09-15T21:47:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7190000000000001e-06 2024-09-15T21:47:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.709 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416560+0000", 2024-09-15T21:47:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3270000000000002e-06 2024-09-15T21:47:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416916+0000", 2024-09-15T21:47:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00035661200000000002 2024-09-15T21:47:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416932+0000", 2024-09-15T21:47:58.710 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5824e-05 2024-09-15T21:47:58.711 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.711 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.711 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.711 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422030+0000", 2024-09-15T21:47:58.711 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050975229999999996 2024-09-15T21:47:58.711 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.711 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.711 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.711 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422053+0000", 2024-09-15T21:47:58.711 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3252e-05 2024-09-15T21:47:58.711 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.711 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.711 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.711 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.712 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.712 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-15T21:47:58.712 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:58.712 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 90.252853063000003, 2024-09-15T21:47:58.712 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051422289999999999, 2024-09-15T21:47:58.712 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.712 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.712 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.712 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.712 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.712 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.712 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:58.712 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.712 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.713 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.713 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.713 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:58.713 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.713 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.713 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.713 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.713 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:47:58.713 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2699999999999999e-07 2024-09-15T21:47:58.713 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.713 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.713 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.713 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416946+0000", 2024-09-15T21:47:58.713 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1390000000000003e-06 2024-09-15T21:47:58.714 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.714 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.714 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.714 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416957+0000", 2024-09-15T21:47:58.714 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.044e-05 2024-09-15T21:47:58.714 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.714 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.714 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.714 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416963+0000", 2024-09-15T21:47:58.714 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5320000000000003e-06 2024-09-15T21:47:58.714 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.714 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.714 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.714 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417090+0000", 2024-09-15T21:47:58.715 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126549 2024-09-15T21:47:58.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.715 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.715 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.715 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417114+0000", 2024-09-15T21:47:58.715 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3736000000000001e-05 2024-09-15T21:47:58.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.715 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.715 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.715 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422055+0000", 2024-09-15T21:47:58.715 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049410410000000002 2024-09-15T21:47:58.715 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.716 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.716 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.716 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422083+0000", 2024-09-15T21:47:58.716 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8164999999999999e-05 2024-09-15T21:47:58.716 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.716 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.716 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.716 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.716 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.716 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.716 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:58.717 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 84.208361091, 2024-09-15T21:47:58.717 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007521186, 2024-09-15T21:47:58.717 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.717 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.717 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.717 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.717 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.717 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.717 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:58.717 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.717 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.717 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.717 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.717 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:58.718 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.718 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.718 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.718 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.718 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:47:58.718 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9999999999999998e-07 2024-09-15T21:47:58.718 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.718 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.718 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.718 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461443+0000", 2024-09-15T21:47:58.718 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0448000000000001e-05 2024-09-15T21:47:58.718 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.718 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.718 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.719 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462146+0000", 2024-09-15T21:47:58.719 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00070223499999999995 2024-09-15T21:47:58.719 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.719 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.719 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.719 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462154+0000", 2024-09-15T21:47:58.719 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9060000000000002e-06 2024-09-15T21:47:58.719 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.719 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.719 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.719 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462254+0000", 2024-09-15T21:47:58.719 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9897999999999998e-05 2024-09-15T21:47:58.719 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.719 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.719 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.720 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462330+0000", 2024-09-15T21:47:58.720 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6551000000000006e-05 2024-09-15T21:47:58.720 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.720 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.720 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:58.720 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462454+0000", 2024-09-15T21:47:58.720 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000124213 2024-09-15T21:47:58.720 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.720 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.720 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.720 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463681+0000", 2024-09-15T21:47:58.720 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012265729999999999 2024-09-15T21:47:58.720 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.721 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.721 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.721 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468817+0000", 2024-09-15T21:47:58.721 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051361430000000001 2024-09-15T21:47:58.721 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.721 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.721 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.721 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468892+0000", 2024-09-15T21:47:58.721 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5480000000000002e-05 2024-09-15T21:47:58.721 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.721 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.721 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.721 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468954+0000", 2024-09-15T21:47:58.721 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1339000000000003e-05 2024-09-15T21:47:58.722 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.722 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.722 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.722 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.722 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.722 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.722 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:58.722 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 84.208344475999994, 2024-09-15T21:47:58.722 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077296149999999996, 2024-09-15T21:47:58.722 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.722 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.722 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.722 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.722 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.722 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.723 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:58.723 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.723 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.723 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.723 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.723 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:47:58.723 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.723 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.723 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.723 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.723 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461450+0000", 2024-09-15T21:47:58.723 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0499999999999999e-07 2024-09-15T21:47:58.723 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.724 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.724 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.724 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461452+0000", 2024-09-15T21:47:58.724 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.007e-06 2024-09-15T21:47:58.724 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.724 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.724 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.724 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462209+0000", 2024-09-15T21:47:58.724 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075781000000000004 2024-09-15T21:47:58.724 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.724 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.724 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.724 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462214+0000", 2024-09-15T21:47:58.725 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1420000000000001e-06 2024-09-15T21:47:58.725 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.725 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.725 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.725 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462681+0000", 2024-09-15T21:47:58.725 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046634200000000002 2024-09-15T21:47:58.725 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.725 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.725 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.725 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462772+0000", 2024-09-15T21:47:58.726 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1414000000000003e-05 2024-09-15T21:47:58.726 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.726 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.726 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:58.726 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462871+0000", 2024-09-15T21:47:58.726 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8455000000000005e-05 2024-09-15T21:47:58.726 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.726 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.726 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.726 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464138+0000", 2024-09-15T21:47:58.726 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012673910000000001 2024-09-15T21:47:58.726 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.726 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.727 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.727 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469079+0000", 2024-09-15T21:47:58.727 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049407799999999997 2024-09-15T21:47:58.727 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.727 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.727 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.727 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469141+0000", 2024-09-15T21:47:58.727 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2169999999999996e-05 2024-09-15T21:47:58.727 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.727 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.727 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.727 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469179+0000", 2024-09-15T21:47:58.727 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7799000000000002e-05 2024-09-15T21:47:58.727 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.727 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.728 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.728 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.728 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.728 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.728 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:58.728 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 78.194228667999994, 2024-09-15T21:47:58.728 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004990497, 2024-09-15T21:47:58.728 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.728 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.728 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.728 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.728 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.728 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.729 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:58.729 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.729 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.729 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.729 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.729 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:47:58.729 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.729 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.729 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.730 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.730 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:47:58.730 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4729999999999999e-06 2024-09-15T21:47:58.730 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.730 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.730 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.730 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475576+0000", 2024-09-15T21:47:58.730 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1749999999999994e-06 2024-09-15T21:47:58.730 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.730 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.730 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.730 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475619+0000", 2024-09-15T21:47:58.730 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3557000000000003e-05 2024-09-15T21:47:58.730 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.731 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.731 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.731 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475631+0000", 2024-09-15T21:47:58.731 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1739999999999999e-05 2024-09-15T21:47:58.732 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.732 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.732 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.732 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475770+0000", 2024-09-15T21:47:58.732 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013950599999999999 2024-09-15T21:47:58.732 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.732 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.732 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475887+0000", 2024-09-15T21:47:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011616 2024-09-15T21:47:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476018+0000", 2024-09-15T21:47:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013119499999999999 2024-09-15T21:47:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478442+0000", 2024-09-15T21:47:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024240870000000001 2024-09-15T21:47:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.733 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.734 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.734 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480457+0000", 2024-09-15T21:47:58.734 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020151779999999998 2024-09-15T21:47:58.734 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.734 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.734 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.734 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480511+0000", 2024-09-15T21:47:58.734 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3582000000000003e-05 2024-09-15T21:47:58.734 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.734 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.734 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.734 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480556+0000", 2024-09-15T21:47:58.734 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4844e-05 2024-09-15T21:47:58.734 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.735 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.735 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.735 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.735 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.735 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.735 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:58.735 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 78.194214349999996, 2024-09-15T21:47:58.735 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051124810000000003, 2024-09-15T21:47:58.735 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.735 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.735 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.735 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.735 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.735 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.736 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:58.736 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.736 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.736 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.736 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.736 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:47:58.736 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.736 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.736 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.736 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.736 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475580+0000", 2024-09-15T21:47:58.736 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7800000000000002e-07 2024-09-15T21:47:58.736 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.736 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.736 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.737 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475595+0000", 2024-09-15T21:47:58.737 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4919000000000001e-05 2024-09-15T21:47:58.737 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.737 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.737 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.737 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475662+0000", 2024-09-15T21:47:58.737 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7769999999999997e-05 2024-09-15T21:47:58.737 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.737 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.737 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.737 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475669+0000", 2024-09-15T21:47:58.737 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.703e-06 2024-09-15T21:47:58.737 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.737 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.738 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.738 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476781+0000", 2024-09-15T21:47:58.738 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011118300000000001 2024-09-15T21:47:58.738 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.738 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.738 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.738 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476863+0000", 2024-09-15T21:47:58.738 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2208999999999998e-05 2024-09-15T21:47:58.738 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.738 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.738 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:58.738 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477040+0000", 2024-09-15T21:47:58.738 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000176424 2024-09-15T21:47:58.738 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.738 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.739 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.739 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480154+0000", 2024-09-15T21:47:58.739 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031147589999999999 2024-09-15T21:47:58.739 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.739 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.739 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.739 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480596+0000", 2024-09-15T21:47:58.739 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044172699999999999 2024-09-15T21:47:58.739 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.739 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.739 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.739 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480652+0000", 2024-09-15T21:47:58.739 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6350000000000001e-05 2024-09-15T21:47:58.739 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.740 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.740 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.740 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480692+0000", 2024-09-15T21:47:58.740 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9412000000000001e-05 2024-09-15T21:47:58.740 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.740 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.740 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.740 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.740 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.740 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:6 2.9 2:941d59bc:::61f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.740 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:58.740 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 73.125003817999996, 2024-09-15T21:47:58.740 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057427529999999997, 2024-09-15T21:47:58.740 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.740 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.741 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.741 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.741 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.741 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.741 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:58.741 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.741 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.741 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.741 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:58.741 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.741 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.741 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.741 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.742 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:47:58.742 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.41e-07 2024-09-15T21:47:58.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.742 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.742 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.742 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544792+0000", 2024-09-15T21:47:58.742 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9929999999999998e-06 2024-09-15T21:47:58.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.742 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.742 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.742 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544885+0000", 2024-09-15T21:47:58.742 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2825000000000007e-05 2024-09-15T21:47:58.742 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.742 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.743 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.743 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544891+0000", 2024-09-15T21:47:58.743 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.93e-06 2024-09-15T21:47:58.743 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.743 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.743 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.743 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546766+0000", 2024-09-15T21:47:58.743 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018747569999999999 2024-09-15T21:47:58.743 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.743 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.743 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.743 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546864+0000", 2024-09-15T21:47:58.743 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8603000000000004e-05 2024-09-15T21:47:58.743 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.743 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.744 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:58.744 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547028+0000", 2024-09-15T21:47:58.744 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000163461 2024-09-15T21:47:58.744 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.744 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.744 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.744 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549499+0000", 2024-09-15T21:47:58.744 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024711749999999999 2024-09-15T21:47:58.744 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.744 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.744 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.744 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550417+0000", 2024-09-15T21:47:58.744 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00091817199999999998 2024-09-15T21:47:58.744 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.745 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.745 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.745 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550476+0000", 2024-09-15T21:47:58.745 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8635999999999999e-05 2024-09-15T21:47:58.745 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.745 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.745 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.745 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550533+0000", 2024-09-15T21:47:58.745 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6959999999999997e-05 2024-09-15T21:47:58.745 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.745 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.745 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.746 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.746 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.746 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.746 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:58.746 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 73.124989353000004, 2024-09-15T21:47:58.746 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004917121, 2024-09-15T21:47:58.746 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.746 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.746 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.746 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.746 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.746 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.746 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:58.747 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.747 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.747 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.747 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.747 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:47:58.747 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.747 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.747 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.747 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.747 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544805+0000", 2024-09-15T21:47:58.747 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4600000000000001e-07 2024-09-15T21:47:58.747 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.747 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.748 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.748 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544817+0000", 2024-09-15T21:47:58.748 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2023e-05 2024-09-15T21:47:58.748 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.748 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.748 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.748 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545000+0000", 2024-09-15T21:47:58.748 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018384400000000001 2024-09-15T21:47:58.748 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.748 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.749 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.749 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545007+0000", 2024-09-15T21:47:58.749 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1249999999999998e-06 2024-09-15T21:47:58.749 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.749 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.749 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.749 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545670+0000", 2024-09-15T21:47:58.749 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00066324999999999999 2024-09-15T21:47:58.749 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.749 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.749 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.749 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545749+0000", 2024-09-15T21:47:58.749 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9239000000000005e-05 2024-09-15T21:47:58.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.750 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.750 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:58.750 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545885+0000", 2024-09-15T21:47:58.750 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013586199999999999 2024-09-15T21:47:58.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.750 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.750 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.750 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547944+0000", 2024-09-15T21:47:58.750 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020594720000000001 2024-09-15T21:47:58.750 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.750 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.750 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.750 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549630+0000", 2024-09-15T21:47:58.751 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001685831 2024-09-15T21:47:58.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.751 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.751 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.751 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549681+0000", 2024-09-15T21:47:58.751 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1187000000000003e-05 2024-09-15T21:47:58.751 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.751 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.751 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.751 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549721+0000", 2024-09-15T21:47:58.751 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0042e-05 2024-09-15T21:47:58.751 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.751 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.751 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.752 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.752 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.752 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.752 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:58.752 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 73.124975605000003, 2024-09-15T21:47:58.752 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050615549999999997, 2024-09-15T21:47:58.752 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.752 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.752 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.752 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.752 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.752 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.753 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:58.753 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.753 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.753 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.753 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.753 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:58.753 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.753 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.753 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.753 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.753 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:47:58.753 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:47:58.753 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.753 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.754 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.754 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544821+0000", 2024-09-15T21:47:58.754 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4399999999999999e-06 2024-09-15T21:47:58.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.754 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.754 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.754 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545025+0000", 2024-09-15T21:47:58.754 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020379400000000001 2024-09-15T21:47:58.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.754 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.754 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.754 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545030+0000", 2024-09-15T21:47:58.754 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3109999999999996e-06 2024-09-15T21:47:58.754 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.755 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.755 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.755 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546196+0000", 2024-09-15T21:47:58.755 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001166261 2024-09-15T21:47:58.755 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.755 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.755 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.755 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546307+0000", 2024-09-15T21:47:58.755 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000110641 2024-09-15T21:47:58.755 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.755 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.755 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:58.755 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546422+0000", 2024-09-15T21:47:58.755 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000115399 2024-09-15T21:47:58.756 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.756 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.756 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.756 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547973+0000", 2024-09-15T21:47:58.756 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001551228 2024-09-15T21:47:58.756 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.756 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.756 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.756 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549815+0000", 2024-09-15T21:47:58.756 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001841685 2024-09-15T21:47:58.756 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.756 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.756 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.757 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549856+0000", 2024-09-15T21:47:58.757 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.091e-05 2024-09-15T21:47:58.757 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.757 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.757 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.757 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549880+0000", 2024-09-15T21:47:58.757 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3580000000000001e-05 2024-09-15T21:47:58.757 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.757 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.757 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.757 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.757 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.757 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.757 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:58.758 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 73.124971596999998, 2024-09-15T21:47:58.758 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056931389999999998, 2024-09-15T21:47:58.758 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.758 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.758 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.758 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.758 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.758 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.758 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:58.758 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.758 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.758 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.758 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.758 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:58.759 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.759 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.759 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.759 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.759 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:47:58.759 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:47:58.759 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.759 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.759 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.759 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544828+0000", 2024-09-15T21:47:58.759 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4369999999999998e-06 2024-09-15T21:47:58.759 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.759 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.759 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.759 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545041+0000", 2024-09-15T21:47:58.760 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000212695 2024-09-15T21:47:58.760 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.760 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.760 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.760 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545058+0000", 2024-09-15T21:47:58.760 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7717999999999999e-05 2024-09-15T21:47:58.760 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.760 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.760 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.760 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546958+0000", 2024-09-15T21:47:58.760 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018995539999999999 2024-09-15T21:47:58.760 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.760 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.760 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.761 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547057+0000", 2024-09-15T21:47:58.761 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9370999999999994e-05 2024-09-15T21:47:58.761 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.761 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.761 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:47:58.761 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547579+0000", 2024-09-15T21:47:58.761 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052198200000000004 2024-09-15T21:47:58.761 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.761 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.761 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.761 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549929+0000", 2024-09-15T21:47:58.761 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023494649999999998 2024-09-15T21:47:58.761 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.762 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.762 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.762 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550401+0000", 2024-09-15T21:47:58.762 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00047277300000000001 2024-09-15T21:47:58.762 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.762 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.762 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.762 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550458+0000", 2024-09-15T21:47:58.762 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6756999999999999e-05 2024-09-15T21:47:58.762 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.762 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.762 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.762 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550515+0000", 2024-09-15T21:47:58.762 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7093999999999999e-05 2024-09-15T21:47:58.762 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.763 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.763 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.763 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.763 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.763 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:26 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.763 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:58.763 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 73.124964939999998, 2024-09-15T21:47:58.763 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050112029999999997, 2024-09-15T21:47:58.763 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.763 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.763 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.763 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.764 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.764 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.764 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:58.764 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.764 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.764 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.764 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.764 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:58.765 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.765 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.765 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.765 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.765 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:47:58.765 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.36e-07 2024-09-15T21:47:58.765 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.765 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.765 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.765 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545086+0000", 2024-09-15T21:47:58.765 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025704699999999999 2024-09-15T21:47:58.765 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.765 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.765 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.766 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545119+0000", 2024-09-15T21:47:58.766 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2734999999999997e-05 2024-09-15T21:47:58.766 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.766 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.766 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.766 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545126+0000", 2024-09-15T21:47:58.766 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5599999999999996e-06 2024-09-15T21:47:58.766 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.766 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.766 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.766 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545253+0000", 2024-09-15T21:47:58.766 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012644500000000001 2024-09-15T21:47:58.766 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.766 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.767 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.767 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545366+0000", 2024-09-15T21:47:58.767 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011349999999999999 2024-09-15T21:47:58.767 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.767 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.767 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:47:58.767 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545673+0000", 2024-09-15T21:47:58.767 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030643599999999999 2024-09-15T21:47:58.767 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.767 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.767 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.767 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547924+0000", 2024-09-15T21:47:58.768 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0022516189999999998 2024-09-15T21:47:58.768 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.768 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.768 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.768 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549746+0000", 2024-09-15T21:47:58.768 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018217629999999999 2024-09-15T21:47:58.768 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.768 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.768 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.768 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549800+0000", 2024-09-15T21:47:58.769 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3412999999999998e-05 2024-09-15T21:47:58.769 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.769 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.769 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.769 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549840+0000", 2024-09-15T21:47:58.769 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0448999999999998e-05 2024-09-15T21:47:58.769 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.769 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.769 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.769 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.769 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.769 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:30 2.4 2:218afd09:::203.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.769 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:58.770 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 73.124704111, 2024-09-15T21:47:58.770 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055481230000000003, 2024-09-15T21:47:58.770 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.770 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.770 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:47:58.770 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.770 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.770 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.770 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:58.770 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.770 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.770 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.770 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.770 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:47:58.771 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.771 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.771 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.771 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.771 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545091+0000", 2024-09-15T21:47:58.771 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7000000000000003e-07 2024-09-15T21:47:58.771 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.771 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.771 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.771 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545094+0000", 2024-09-15T21:47:58.771 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9569999999999998e-06 2024-09-15T21:47:58.771 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.771 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.771 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.771 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545156+0000", 2024-09-15T21:47:58.772 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2277e-05 2024-09-15T21:47:58.772 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.772 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.772 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.772 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545162+0000", 2024-09-15T21:47:58.772 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9120000000000003e-06 2024-09-15T21:47:58.772 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.772 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.772 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.772 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545262+0000", 2024-09-15T21:47:58.772 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010001900000000001 2024-09-15T21:47:58.772 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.772 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.772 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.773 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545388+0000", 2024-09-15T21:47:58.773 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126243 2024-09-15T21:47:58.773 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.773 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.773 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.773 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550638+0000", 2024-09-15T21:47:58.773 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052497450000000001 2024-09-15T21:47:58.773 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.773 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.773 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.773 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.773 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.773 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:10 2.9 2:954a9a2b:::62b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:47:58.774 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:58.774 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 67.105020651000004, 2024-09-15T21:47:58.774 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048438860000000004, 2024-09-15T21:47:58.774 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:47:58.774 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:47:58.774 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:47:58.774 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:47:58.774 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.774 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:47:58.774 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:58.774 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.774 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.774 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.774 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:47:58.775 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:58.775 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:47:58.775 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.775 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.775 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:47:58.775 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:47:58.775 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0400000000000002e-07 2024-09-15T21:47:58.775 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.775 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.775 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:47:58.775 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564786+0000", 2024-09-15T21:47:58.775 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2412000000000001e-05 2024-09-15T21:47:58.775 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.775 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.776 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:47:58.776 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564991+0000", 2024-09-15T21:47:58.776 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020483900000000001 2024-09-15T21:47:58.776 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.776 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.776 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:47:58.776 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564998+0000", 2024-09-15T21:47:58.776 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2459999999999997e-06 2024-09-15T21:47:58.776 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.776 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.776 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:47:58.776 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565422+0000", 2024-09-15T21:47:58.776 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00042420100000000003 2024-09-15T21:47:58.776 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.777 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.777 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:47:58.777 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565515+0000", 2024-09-15T21:47:58.777 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2449999999999997e-05 2024-09-15T21:47:58.777 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.777 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.777 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:47:58.777 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565642+0000", 2024-09-15T21:47:58.777 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012743699999999999 2024-09-15T21:47:58.777 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.777 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.777 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:47:58.777 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566656+0000", 2024-09-15T21:47:58.777 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010138580000000001 2024-09-15T21:47:58.778 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.778 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.778 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:47:58.778 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569552+0000", 2024-09-15T21:47:58.778 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028966819999999998 2024-09-15T21:47:58.778 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.778 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.778 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:47:58.778 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569594+0000", 2024-09-15T21:47:58.778 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1235000000000001e-05 2024-09-15T21:47:58.778 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:47:58.778 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:47:58.778 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:47:58.778 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569617+0000", 2024-09-15T21:47:58.778 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3221999999999999e-05 2024-09-15T21:47:58.779 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.779 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.779 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.779 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:47:58.779 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:47:58.779 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:58.780 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:47:58.781 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:58.778+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:58.786 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:47:58.883 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:58.880+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:58.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.913+0000 7f6fa97fa640 1 -- 172.21.15.100:0/3486289395 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd out", "ids": ["7"]}]=0 marked out osd.7. v47) ==== 87+0+0 (secure 0 0 0) 0x7f6f9805c9b0 con 0x7f6fac0cc8f0 2024-09-15T21:47:58.919 INFO:teuthology.orchestra.run.smithi100.stderr:marked out osd.7. 2024-09-15T21:47:58.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.917+0000 7f6fb2f31640 1 -- 172.21.15.100:0/3486289395 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6f9006c2b0 msgr2=0x7f6f9006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:58.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.917+0000 7f6fb2f31640 1 --2- 172.21.15.100:0/3486289395 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6f9006c2b0 0x7f6f9006e770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6fa00112c0 tx=0x7f6fa0009190 comp rx=0 tx=0).stop 2024-09-15T21:47:58.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.917+0000 7f6fb2f31640 1 -- 172.21.15.100:0/3486289395 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6fac0cc8f0 msgr2=0x7f6fac15a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:58.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.917+0000 7f6fb2f31640 1 --2- 172.21.15.100:0/3486289395 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6fac0cc8f0 0x7f6fac15a2e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0cd3d0 tx=0x7f6f9800c760 comp rx=0 tx=0).stop 2024-09-15T21:47:58.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.925+0000 7f6fb2f31640 1 -- 172.21.15.100:0/3486289395 shutdown_connections 2024-09-15T21:47:58.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.925+0000 7f6fb2f31640 1 --2- 172.21.15.100:0/3486289395 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6f9006c2b0 0x7f6f9006e770 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:58.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.925+0000 7f6fb2f31640 1 --2- 172.21.15.100:0/3486289395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fac155440 0x7f6fac148dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:58.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.925+0000 7f6fb2f31640 1 --2- 172.21.15.100:0/3486289395 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6fac0cc8f0 0x7f6fac15a2e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:58.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.925+0000 7f6fb2f31640 1 --2- 172.21.15.100:0/3486289395 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6fac0cb7d0 0x7f6fac152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:58.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.925+0000 7f6fb2f31640 1 -- 172.21.15.100:0/3486289395 >> 172.21.15.100:0/3486289395 conn(0x7f6fac0680d0 msgr2=0x7f6fac144710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:58.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.925+0000 7f6fb2f31640 1 -- 172.21.15.100:0/3486289395 shutdown_connections 2024-09-15T21:47:58.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.925+0000 7f6fb2f31640 1 -- 172.21.15.100:0/3486289395 wait complete. 2024-09-15T21:47:58.966 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:47:58.967 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:47:58.967 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:47:58.967 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:47:58.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.966+0000 7f2c9235b640 1 Processor -- start 2024-09-15T21:47:58.968 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.966+0000 7f2c9235b640 1 -- start start 2024-09-15T21:47:58.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.968+0000 7f2c9235b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2c8c15bee0 0x7f2c8c0c9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:58.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.968+0000 7f2c9235b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c8c0c9f20 con 0x7f2c8c152040 2024-09-15T21:47:58.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.968+0000 7f2c9235b640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c8c15b440 con 0x7f2c8c157d20 2024-09-15T21:47:58.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.968+0000 7f2c9235b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f2c8c0c7e90 con 0x7f2c8c15bee0 2024-09-15T21:47:58.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.968+0000 7f2c8b7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2c8c15bee0 0x7f2c8c0c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:58.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.968+0000 7f2c8b7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2c8c15bee0 0x7f2c8c0c9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57696/0 (socket says 172.21.15.100:57696) 2024-09-15T21:47:58.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.968+0000 7f2c8b7fe640 1 -- 172.21.15.100:0/3431174855 learned_addr learned my addr 172.21.15.100:0/3431174855 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:58.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.968+0000 7f2c8bfff640 1 --1- 172.21.15.100:0/3431174855 >> v1:172.21.15.100:6789/0 conn(0x7f2c8c152040 0x7f2c8c0c8af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57420/0 (socket says 172.21.15.100:57420) 2024-09-15T21:47:58.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.969+0000 7f2c8b7fe640 1 -- 172.21.15.100:0/3431174855 >> v1:172.21.15.122:6789/0 conn(0x7f2c8c157d20 legacy=0x7f2c8c072080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:47:58.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.969+0000 7f2c8affd640 1 -- 172.21.15.100:0/3431174855 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1166710731 0 0) 0x7f2c8c0c9f20 con 0x7f2c8c152040 2024-09-15T21:47:58.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.969+0000 7f2c8b7fe640 1 -- 172.21.15.100:0/3431174855 >> v1:172.21.15.100:6789/0 conn(0x7f2c8c152040 legacy=0x7f2c8c0c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:58.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.969+0000 7f2c8b7fe640 1 -- 172.21.15.100:0/3431174855 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8c0c9960 con 0x7f2c8c15bee0 2024-09-15T21:47:58.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.970+0000 7f2c8b7fe640 1 --2- 172.21.15.100:0/3431174855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c8c15bee0 0x7f2c8c0c9320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009870 tx=0x7f2c8002a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca7499d63073f12 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:58.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.970+0000 7f2c8affd640 1 -- 172.21.15.100:0/3431174855 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80037070 con 0x7f2c8c15bee0 2024-09-15T21:47:58.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.970+0000 7f2c8affd640 1 -- 172.21.15.100:0/3431174855 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c80032640 con 0x7f2c8c15bee0 2024-09-15T21:47:58.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.970+0000 7f2c8affd640 1 -- 172.21.15.100:0/3431174855 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80032a10 con 0x7f2c8c15bee0 2024-09-15T21:47:58.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.971+0000 7f2c9235b640 1 -- 172.21.15.100:0/3431174855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c8c15bee0 msgr2=0x7f2c8c0c9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:58.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.971+0000 7f2c9235b640 1 --2- 172.21.15.100:0/3431174855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c8c15bee0 0x7f2c8c0c9320 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009870 tx=0x7f2c8002a330 comp rx=0 tx=0).stop 2024-09-15T21:47:58.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.971+0000 7f2c9235b640 1 -- 172.21.15.100:0/3431174855 shutdown_connections 2024-09-15T21:47:58.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.971+0000 7f2c9235b640 1 --2- 172.21.15.100:0/3431174855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c8c15bee0 0x7f2c8c0c9320 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:58.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.971+0000 7f2c9235b640 1 -- 172.21.15.100:0/3431174855 >> 172.21.15.100:0/3431174855 conn(0x7f2c8c0cd9a0 msgr2=0x7f2c8c0cdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:47:58.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.971+0000 7f2c9235b640 1 -- 172.21.15.100:0/3431174855 shutdown_connections 2024-09-15T21:47:58.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.971+0000 7f2c9235b640 1 -- 172.21.15.100:0/3431174855 wait complete. 2024-09-15T21:47:58.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.973+0000 7f2c9235b640 1 Processor -- start 2024-09-15T21:47:58.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.973+0000 7f2c9235b640 1 -- start start 2024-09-15T21:47:58.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.974+0000 7f2c9235b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2c8c152040 0x7f2c8c15a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:58.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.974+0000 7f2c9235b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c8c157d20 0x7f2c8c15ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:58.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.975+0000 7f2c8b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c8c157d20 0x7f2c8c15ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:58.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.975+0000 7f2c8b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c8c157d20 0x7f2c8c15ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57710/0 (socket says 172.21.15.100:57710) 2024-09-15T21:47:58.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.975+0000 7f2c8b7fe640 1 -- 172.21.15.100:0/1187462788 learned_addr learned my addr 172.21.15.100:0/1187462788 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:47:58.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.975+0000 7f2c9235b640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c8c15bee0 0x7f2c8c0ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:58.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.975+0000 7f2c8bfff640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2c8c152040 0x7f2c8c15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:58.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.975+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c15ba50 con 0x7f2c8c157d20 2024-09-15T21:47:58.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.975+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c15c2e0 con 0x7f2c8c15bee0 2024-09-15T21:47:58.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.975+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2c8c15b6e0 con 0x7f2c8c152040 2024-09-15T21:47:58.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.975+0000 7f2c908f8640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c8c15bee0 0x7f2c8c0ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:58.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.975+0000 7f2c8b7fe640 1 -- 172.21.15.100:0/1187462788 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2c8c152040 msgr2=0x7f2c8c15a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:58.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.975+0000 7f2c8b7fe640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2c8c152040 0x7f2c8c15a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:58.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.975+0000 7f2c8b7fe640 1 -- 172.21.15.100:0/1187462788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c8c15bee0 msgr2=0x7f2c8c0ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:47:58.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.975+0000 7f2c8b7fe640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2c8c15bee0 0x7f2c8c0ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:47:58.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.975+0000 7f2c8b7fe640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8c0cbcd0 con 0x7f2c8c157d20 2024-09-15T21:47:58.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.976+0000 7f2c8b7fe640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c8c157d20 0x7f2c8c15ac80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009870 tx=0x7f2c800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:58.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.976+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80041020 con 0x7f2c8c157d20 2024-09-15T21:47:58.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.976+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c80030c70 con 0x7f2c8c157d20 2024-09-15T21:47:58.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.976+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80037040 con 0x7f2c8c157d20 2024-09-15T21:47:58.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.976+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8c0cbfb0 con 0x7f2c8c157d20 2024-09-15T21:47:58.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.976+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c8c0cc570 con 0x7f2c8c157d20 2024-09-15T21:47:58.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.977+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f2c8c0c4a80 con 0x7f2c8c157d20 2024-09-15T21:47:58.980 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:47:58.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.978+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2c80044020 con 0x7f2c8c157d20 2024-09-15T21:47:58.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.979+0000 7f2c88ff9640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2c7806c2b0 0x7f2c7806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:58.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.979+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5857+0+0 (secure 0 0 0) 0x7f2c800ac2a0 con 0x7f2c8c157d20 2024-09-15T21:47:58.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.979+0000 7f2c8bfff640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2c7806c2b0 0x7f2c7806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:58.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.979+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v47) ==== 54+0+16 (secure 0 0 0) 0x7f2c8004b030 con 0x7f2c8c157d20 2024-09-15T21:47:58.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.979+0000 7f2c9235b640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f2c8c0cc740 0x7f2c8c072080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:58.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.979+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2c8c149910 con 0x7f2c8c0cc740 2024-09-15T21:47:58.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.979+0000 7f2c908f8640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f2c8c0cc740 0x7f2c8c072080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:58.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.979+0000 7f2c8bfff640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2c7806c2b0 0x7f2c7806e770 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c000f60 tx=0x7f2c7c007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:58.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.980+0000 7f2c908f8640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f2c8c0cc740 0x7f2c8c072080 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:58.983 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:58.980+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c8c149910 con 0x7f2c8c0cc740 2024-09-15T21:47:58.984 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:47:58.982+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:47:59.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.007+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f2c8c073640 con 0x7f2c8c0cc740 2024-09-15T21:47:59.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.010+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2c8c073640 con 0x7f2c8c0cc740 2024-09-15T21:47:59.012 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:47:59.013 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:47:59.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.010+0000 7f2c9235b640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f2c8c159890 0x7f2c8c06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:59.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.011+0000 7f2c8bfff640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f2c8c159890 0x7f2c8c06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:59.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.011+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f2c8c15a150 con 0x7f2c8c159890 2024-09-15T21:47:59.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.011+0000 7f2c8bfff640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f2c8c159890 0x7f2c8c06bbe0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:59.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.012+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c8c15a150 con 0x7f2c8c159890 2024-09-15T21:47:59.042 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.040+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f2c8c159c70 con 0x7f2c8c159890 2024-09-15T21:47:59.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.043+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2c8c159c70 con 0x7f2c8c159890 2024-09-15T21:47:59.045 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:47:59.045 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:47:59.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.043+0000 7f2c9235b640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f2c8c06c630 0x7f2c8c062630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:59.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.043+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f2c8c06ca10 con 0x7f2c8c06c630 2024-09-15T21:47:59.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.043+0000 7f2c8b7fe640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f2c8c06c630 0x7f2c8c062630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:59.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.044+0000 7f2c8b7fe640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f2c8c06c630 0x7f2c8c062630 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:59.046 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.045+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c8c06ca10 con 0x7f2c8c06c630 2024-09-15T21:47:59.073 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.071+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f2c8c062e40 con 0x7f2c8c06c630 2024-09-15T21:47:59.075 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.073+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2c8c062e40 con 0x7f2c8c06c630 2024-09-15T21:47:59.075 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:47:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:47:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.074+0000 7f2c9235b640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f2c8c156fd0 0x7f2c8c1573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.074+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f2c8c157930 con 0x7f2c8c156fd0 2024-09-15T21:47:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.074+0000 7f2c908f8640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f2c8c156fd0 0x7f2c8c1573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:59.076 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.074+0000 7f2c908f8640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f2c8c156fd0 0x7f2c8c1573f0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:59.077 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.075+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c8c157930 con 0x7f2c8c156fd0 2024-09-15T21:47:59.085 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:59.084+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:59.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.102+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f2c8c02c640 con 0x7f2c8c156fd0 2024-09-15T21:47:59.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.103+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2c8c02c640 con 0x7f2c8c156fd0 2024-09-15T21:47:59.105 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:47:59.106 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:47:59.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.104+0000 7f2c9235b640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f2c8c02cd60 0x7f2c8c02d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:59.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.104+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f2c8c02d6e0 con 0x7f2c8c02cd60 2024-09-15T21:47:59.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.104+0000 7f2c8bfff640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f2c8c02cd60 0x7f2c8c02d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:59.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.104+0000 7f2c8bfff640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f2c8c02cd60 0x7f2c8c02d1a0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:59.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.105+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c8c02d6e0 con 0x7f2c8c02cd60 2024-09-15T21:47:59.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.132+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f2c8c02d6e0 con 0x7f2c8c02cd60 2024-09-15T21:47:59.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.134+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2c8c02d6e0 con 0x7f2c8c02cd60 2024-09-15T21:47:59.136 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:47:59.137 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:47:59.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.134+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=48}) -- 0x7f2c8c02d6e0 con 0x7f2c8c157d20 2024-09-15T21:47:59.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.134+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2c8c02d8f0 con 0x7f2c8c157d20 2024-09-15T21:47:59.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.134+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f2c800b3070 con 0x7f2c8c157d20 2024-09-15T21:47:59.137 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:47:59.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.134+0000 7f2c9235b640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f2c8c205190 0x7f2c8c2075a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:59.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.134+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f2c8c20a640 con 0x7f2c8c205190 2024-09-15T21:47:59.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.135+0000 7f2c8b7fe640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f2c8c205190 0x7f2c8c2075a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:59.137 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.135+0000 7f2c8b7fe640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f2c8c205190 0x7f2c8c2075a0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:59.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.136+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c8c20a640 con 0x7f2c8c205190 2024-09-15T21:47:59.142 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:59.142 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:59.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.150+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f2c8c20a610 con 0x7f2c8c205190 2024-09-15T21:47:59.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.152+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2c8c20a610 con 0x7f2c8c205190 2024-09-15T21:47:59.154 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:47:59.154 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:47:59.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.152+0000 7f2c9235b640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f2c8c209870 0x7f2c8c209cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:47:59.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.152+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f2c8c20da40 con 0x7f2c8c209870 2024-09-15T21:47:59.155 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:47:59.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.152+0000 7f2c908f8640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f2c8c209870 0x7f2c8c209cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:47:59.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.153+0000 7f2c908f8640 1 --2- 172.21.15.100:0/1187462788 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f2c8c209870 0x7f2c8c209cd0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:47:59.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.153+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2c8c20da40 con 0x7f2c8c209870 2024-09-15T21:47:59.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.167+0000 7f2c9235b640 1 -- 172.21.15.100:0/1187462788 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f2c8c207c20 con 0x7f2c8c209870 2024-09-15T21:47:59.171 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:47:59.171 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:47:59.169+0000 7f2c88ff9640 1 -- 172.21.15.100:0/1187462788 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f2c8c207c20 con 0x7f2c8c209870 2024-09-15T21:47:59.171 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:47:59.176 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:47:59.176 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:47:59.186 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:47:59.185+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:47:59.274 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:59.274 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:59.284 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:47:59.287 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:59.285+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:59.387 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:59.386+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:59.388 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:59.389 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:59.398 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:47:59.488 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:59.487+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:59.590 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:59.587+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:59.604 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:59.605 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:59.617 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:47:59.703 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:59.700+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:47:59.772 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:59.772 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:59.782 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:47:59.823 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:47:59.820+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:47:59.894 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:47:59.894 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:47:59.895 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:47:59.891+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:47:59.903 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:47:59.929 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:47:59.995 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:47:59.993+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:48:00.007 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:00.007 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:00.016 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:48:00.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.027+0000 7fbc11f03640 1 Processor -- start 2024-09-15T21:48:00.030 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.027+0000 7fbc11f03640 1 -- start start 2024-09-15T21:48:00.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.028+0000 7fbc11f03640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc0c159750 0x7fbc0c0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:00.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.028+0000 7fbc11f03640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbc0c0cc9f0 0x7fbc0c0ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:00.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.028+0000 7fbc11f03640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc0c163410 con 0x7fbc0c0b3710 2024-09-15T21:48:00.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.028+0000 7fbc11f03640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fbc0c15fbf0 con 0x7fbc0c0cc9f0 2024-09-15T21:48:00.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.028+0000 7fbc11f03640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbc0c0cbe50 con 0x7fbc0c159750 2024-09-15T21:48:00.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.028+0000 7fbc0bfff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fbc0c0cc9f0 0x7fbc0c0ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:00.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.028+0000 7fbc0affd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fbc0c0b3710 0x7fbc0c15f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57428/0 (socket says 172.21.15.100:57428) 2024-09-15T21:48:00.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.028+0000 7fbc0affd640 1 -- 172.21.15.100:0/2403483507 learned_addr learned my addr 172.21.15.100:0/2403483507 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:00.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.028+0000 7fbc0b7fe640 1 --2- 172.21.15.100:0/2403483507 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc0c159750 0x7fbc0c0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:00.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.029+0000 7fbc0bfff640 1 -- 172.21.15.100:0/2403483507 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc0c159750 msgr2=0x7fbc0c0cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:00.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.029+0000 7fbc0bfff640 1 --2- 172.21.15.100:0/2403483507 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc0c159750 0x7fbc0c0cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:00.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.029+0000 7fbc0bfff640 1 -- 172.21.15.100:0/2403483507 >> v1:172.21.15.100:6789/0 conn(0x7fbc0c0b3710 legacy=0x7fbc0c15f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:00.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.029+0000 7fbc0bfff640 1 -- 172.21.15.100:0/2403483507 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0c0cc080 con 0x7fbc0c0cc9f0 2024-09-15T21:48:00.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.029+0000 7fbc0a7fc640 1 -- 172.21.15.100:0/2403483507 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3296304694 0 0) 0x7fbc0c163410 con 0x7fbc0c0b3710 2024-09-15T21:48:00.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.030+0000 7fbc0bfff640 1 --2- 172.21.15.100:0/2403483507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc0c0cc9f0 0x7fbc0c0ccf40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbc04009d90 tx=0x7fbc0402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb7d4ae34f9f97 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:00.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.030+0000 7fbc0a7fc640 1 -- 172.21.15.100:0/2403483507 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04031600 con 0x7fbc0c0cc9f0 2024-09-15T21:48:00.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.030+0000 7fbc0a7fc640 1 -- 172.21.15.100:0/2403483507 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc040317a0 con 0x7fbc0c0cc9f0 2024-09-15T21:48:00.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.030+0000 7fbc0a7fc640 1 -- 172.21.15.100:0/2403483507 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04031b70 con 0x7fbc0c0cc9f0 2024-09-15T21:48:00.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.030+0000 7fbc11f03640 1 -- 172.21.15.100:0/2403483507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc0c0cc9f0 msgr2=0x7fbc0c0ccf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:00.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.030+0000 7fbc11f03640 1 --2- 172.21.15.100:0/2403483507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc0c0cc9f0 0x7fbc0c0ccf40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbc04009d90 tx=0x7fbc0402a5d0 comp rx=0 tx=0).stop 2024-09-15T21:48:00.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.031+0000 7fbc11f03640 1 -- 172.21.15.100:0/2403483507 shutdown_connections 2024-09-15T21:48:00.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.031+0000 7fbc11f03640 1 --2- 172.21.15.100:0/2403483507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc0c0cc9f0 0x7fbc0c0ccf40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:00.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.031+0000 7fbc11f03640 1 --2- 172.21.15.100:0/2403483507 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc0c159750 0x7fbc0c0cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:00.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.031+0000 7fbc11f03640 1 -- 172.21.15.100:0/2403483507 >> 172.21.15.100:0/2403483507 conn(0x7fbc0c0680d0 msgr2=0x7fbc0c06d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:00.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.031+0000 7fbc11f03640 1 -- 172.21.15.100:0/2403483507 shutdown_connections 2024-09-15T21:48:00.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.031+0000 7fbc11f03640 1 -- 172.21.15.100:0/2403483507 wait complete. 2024-09-15T21:48:00.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.033+0000 7fbc11f03640 1 Processor -- start 2024-09-15T21:48:00.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.033+0000 7fbc11f03640 1 -- start start 2024-09-15T21:48:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.034+0000 7fbc11f03640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc0c0b3710 0x7fbc0c0ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.034+0000 7fbc11f03640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc0c0cc9f0 0x7fbc0c1539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.034+0000 7fbc11f03640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc0c159750 0x7fbc0c0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.034+0000 7fbc11f03640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c1636b0 con 0x7fbc0c159750 2024-09-15T21:48:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.034+0000 7fbc11f03640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0c163990 con 0x7fbc0c0b3710 2024-09-15T21:48:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.034+0000 7fbc11f03640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fbc0c159b30 con 0x7fbc0c0cc9f0 2024-09-15T21:48:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.034+0000 7fbc0b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc0c0b3710 0x7fbc0c0ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.034+0000 7fbc0b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc0c0b3710 0x7fbc0c0ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:42068/0 (socket says 172.21.15.100:42068) 2024-09-15T21:48:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.034+0000 7fbc0affd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc0c0cc9f0 0x7fbc0c1539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.034+0000 7fbc0b7fe640 1 -- 172.21.15.100:0/3071690685 learned_addr learned my addr 172.21.15.100:0/3071690685 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.034+0000 7fbc0bfff640 1 --2- 172.21.15.100:0/3071690685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc0c159750 0x7fbc0c0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:00.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.035+0000 7fbc0b7fe640 1 -- 172.21.15.100:0/3071690685 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc0c0cc9f0 msgr2=0x7fbc0c1539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.035+0000 7fbc0b7fe640 1 --2- 172.21.15.100:0/3071690685 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc0c0cc9f0 0x7fbc0c1539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.035+0000 7fbc0b7fe640 1 -- 172.21.15.100:0/3071690685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc0c159750 msgr2=0x7fbc0c0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.035+0000 7fbc0b7fe640 1 --2- 172.21.15.100:0/3071690685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc0c159750 0x7fbc0c0723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.035+0000 7fbc0b7fe640 1 -- 172.21.15.100:0/3071690685 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0c0cee90 con 0x7fbc0c0b3710 2024-09-15T21:48:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.035+0000 7fbc0b7fe640 1 --2- 172.21.15.100:0/3071690685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc0c0b3710 0x7fbc0c0ca140 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc00ea10 tx=0x7fbbfc00eed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.035+0000 7fbbf3fff640 1 -- 172.21.15.100:0/3071690685 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc017070 con 0x7fbc0c0b3710 2024-09-15T21:48:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.035+0000 7fbc11f03640 1 -- 172.21.15.100:0/3071690685 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0c1f8320 con 0x7fbc0c0b3710 2024-09-15T21:48:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.035+0000 7fbbf3fff640 1 -- 172.21.15.100:0/3071690685 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbfc00ca00 con 0x7fbc0c0b3710 2024-09-15T21:48:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.035+0000 7fbbf3fff640 1 -- 172.21.15.100:0/3071690685 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc00cdf0 con 0x7fbc0c0b3710 2024-09-15T21:48:00.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.035+0000 7fbc11f03640 1 -- 172.21.15.100:0/3071690685 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc0c1f88e0 con 0x7fbc0c0b3710 2024-09-15T21:48:00.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.037+0000 7fbc11f03640 1 -- 172.21.15.100:0/3071690685 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc0c0caed0 con 0x7fbc0c0b3710 2024-09-15T21:48:00.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.037+0000 7fbbf3fff640 1 -- 172.21.15.100:0/3071690685 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fbbfc01a070 con 0x7fbc0c0b3710 2024-09-15T21:48:00.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.037+0000 7fbbf3fff640 1 --2- 172.21.15.100:0/3071690685 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbbf806c2b0 0x7fbbf806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:00.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.037+0000 7fbbf3fff640 1 -- 172.21.15.100:0/3071690685 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 6085+0+0 (secure 0 0 0) 0x7fbbfc08b8b0 con 0x7fbc0c0b3710 2024-09-15T21:48:00.043 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.038+0000 7fbc0affd640 1 --2- 172.21.15.100:0/3071690685 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbbf806c2b0 0x7fbbf806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:00.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.038+0000 7fbc0affd640 1 --2- 172.21.15.100:0/3071690685 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbbf806c2b0 0x7fbbf806e770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbc00007c30 tx=0x7fbc00007b80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:00.044 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.042+0000 7fbbf3fff640 1 -- 172.21.15.100:0/3071690685 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fbbfc059850 con 0x7fbc0c0b3710 2024-09-15T21:48:00.097 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:48:00.095+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:48:00.121 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:00.121 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:00.132 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:48:00.198 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:48:00.195+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:48:00.275 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:00.276 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:00.285 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:48:00.298 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:48:00.296+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:48:00.388 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:00.389 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:00.398 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:48:00.400 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:48:00.398+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:48:00.402 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.399+0000 7fbc11f03640 1 -- 172.21.15.100:0/3071690685 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7fbc0c0b41b0 con 0x7fbc0c0b3710 2024-09-15T21:48:00.501 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:48:00.499+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:48:00.501 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:00.501 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:00.511 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:48:00.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.559+0000 7fbbf3fff640 1 -- 172.21.15.100:0/3071690685 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v49) ==== 101+0+0 (secure 0 0 0) 0x7fbbfc05db50 con 0x7fbc0c0b3710 2024-09-15T21:48:00.562 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is unset 2024-09-15T21:48:00.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.563+0000 7fbc11f03640 1 -- 172.21.15.100:0/3071690685 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbbf806c2b0 msgr2=0x7fbbf806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:00.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.563+0000 7fbc11f03640 1 --2- 172.21.15.100:0/3071690685 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbbf806c2b0 0x7fbbf806e770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbc00007c30 tx=0x7fbc00007b80 comp rx=0 tx=0).stop 2024-09-15T21:48:00.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.563+0000 7fbc11f03640 1 -- 172.21.15.100:0/3071690685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc0c0b3710 msgr2=0x7fbc0c0ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:00.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.563+0000 7fbc11f03640 1 --2- 172.21.15.100:0/3071690685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc0c0b3710 0x7fbc0c0ca140 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc00ea10 tx=0x7fbbfc00eed0 comp rx=0 tx=0).stop 2024-09-15T21:48:00.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.563+0000 7fbc11f03640 1 -- 172.21.15.100:0/3071690685 shutdown_connections 2024-09-15T21:48:00.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.563+0000 7fbc11f03640 1 --2- 172.21.15.100:0/3071690685 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fbbf806c2b0 0x7fbbf806e770 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbc00007c30 tx=0x7fbc00007b80 comp rx=0 tx=0).stop 2024-09-15T21:48:00.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.563+0000 7fbc11f03640 1 --2- 172.21.15.100:0/3071690685 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbc0c159750 0x7fbc0c0723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:00.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.563+0000 7fbc11f03640 1 --2- 172.21.15.100:0/3071690685 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fbc0c0cc9f0 0x7fbc0c1539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:00.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.563+0000 7fbc11f03640 1 --2- 172.21.15.100:0/3071690685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc0c0b3710 0x7fbc0c0ca140 secure :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc00ea10 tx=0x7fbbfc00eed0 comp rx=0 tx=0).stop 2024-09-15T21:48:00.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.563+0000 7fbc11f03640 1 -- 172.21.15.100:0/3071690685 >> 172.21.15.100:0/3071690685 conn(0x7fbc0c0680d0 msgr2=0x7fbc0c0c7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:00.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.566+0000 7fbc11f03640 1 -- 172.21.15.100:0/3071690685 shutdown_connections 2024-09-15T21:48:00.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:00.566+0000 7fbc11f03640 1 -- 172.21.15.100:0/3071690685 wait complete. 2024-09-15T21:48:00.602 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:48:00.600+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:48:00.658 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:00.658 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:00.667 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:48:00.709 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:48:00.707+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:48:00.805 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:48:00.802+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:48:00.812 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:00.812 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:00.821 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:48:00.905 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:48:00.902+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:48:00.925 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:00.925 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:00.934 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:48:01.006 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:48:01.005+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:48:01.097 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:01.097 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:01.107 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:48:01.106+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:48:01.109 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:48:01.177 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:48:01.209 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:48:01.207+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:48:01.220 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:01.220 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:01.233 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:48:01.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.294+0000 7fdc7560e640 1 Processor -- start 2024-09-15T21:48:01.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.294+0000 7fdc7560e640 1 -- start start 2024-09-15T21:48:01.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.295+0000 7fdc7560e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdc70159580 0x7fdc70160010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:01.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.295+0000 7fdc7560e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdc70159960 0x7fdc70071f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:01.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.295+0000 7fdc7560e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fdc700724c0 0x7fdc70072950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:01.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.295+0000 7fdc6e7fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdc70159960 0x7fdc70071f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:01.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.295+0000 7fdc6e7fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdc70159960 0x7fdc70071f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57746/0 (socket says 172.21.15.100:57746) 2024-09-15T21:48:01.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.295+0000 7fdc6e7fc640 1 -- 172.21.15.100:0/2333872941 learned_addr learned my addr 172.21.15.100:0/2333872941 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:01.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.295+0000 7fdc7560e640 1 -- 172.21.15.100:0/2333872941 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fdc7015fb80 con 0x7fdc70159960 2024-09-15T21:48:01.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.295+0000 7fdc7560e640 1 -- 172.21.15.100:0/2333872941 --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fdc700ce5d0 con 0x7fdc700724c0 2024-09-15T21:48:01.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.295+0000 7fdc6effd640 1 --2- 172.21.15.100:0/2333872941 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdc70159580 0x7fdc70160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:01.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.296+0000 7fdc7560e640 1 -- 172.21.15.100:0/2333872941 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fdc700c87e0 con 0x7fdc70159580 2024-09-15T21:48:01.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.296+0000 7fdc6f7fe640 1 --2- 172.21.15.100:0/2333872941 >> v2:172.21.15.122:3300/0 conn(0x7fdc700724c0 0x7fdc70072950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:01.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.296+0000 7fdc6f7fe640 1 -- 172.21.15.100:0/2333872941 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdc70159580 msgr2=0x7fdc70160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:01.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.296+0000 7fdc6f7fe640 1 --2- 172.21.15.100:0/2333872941 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdc70159580 0x7fdc70160010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:01.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.296+0000 7fdc6f7fe640 1 -- 172.21.15.100:0/2333872941 >> v2:172.21.15.100:3300/0 conn(0x7fdc70159960 msgr2=0x7fdc70071f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:01.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.296+0000 7fdc6f7fe640 1 --2- 172.21.15.100:0/2333872941 >> v2:172.21.15.100:3300/0 conn(0x7fdc70159960 0x7fdc70071f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:01.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.296+0000 7fdc6f7fe640 1 -- 172.21.15.100:0/2333872941 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc700c8aa0 con 0x7fdc700724c0 2024-09-15T21:48:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.297+0000 7fdc6f7fe640 1 --2- 172.21.15.100:0/2333872941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc700724c0 0x7fdc70072950 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdc700715e0 tx=0x7fdc6802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a4d3bf22003dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:01.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.297+0000 7fdc6dffb640 1 -- 172.21.15.100:0/2333872941 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68031600 con 0x7fdc700724c0 2024-09-15T21:48:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.297+0000 7fdc6dffb640 1 -- 172.21.15.100:0/2333872941 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc680317a0 con 0x7fdc700724c0 2024-09-15T21:48:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.297+0000 7fdc6dffb640 1 -- 172.21.15.100:0/2333872941 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc68031b70 con 0x7fdc700724c0 2024-09-15T21:48:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.298+0000 7fdc7560e640 1 -- 172.21.15.100:0/2333872941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc700724c0 msgr2=0x7fdc70072950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.298+0000 7fdc7560e640 1 --2- 172.21.15.100:0/2333872941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc700724c0 0x7fdc70072950 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdc700715e0 tx=0x7fdc6802a5d0 comp rx=0 tx=0).stop 2024-09-15T21:48:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.298+0000 7fdc7560e640 1 -- 172.21.15.100:0/2333872941 shutdown_connections 2024-09-15T21:48:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.298+0000 7fdc7560e640 1 --2- 172.21.15.100:0/2333872941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc700724c0 0x7fdc70072950 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.298+0000 7fdc7560e640 1 --2- 172.21.15.100:0/2333872941 >> v2:172.21.15.100:3300/0 conn(0x7fdc70159960 0x7fdc70071f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.298+0000 7fdc7560e640 1 --2- 172.21.15.100:0/2333872941 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdc70159580 0x7fdc70160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.298+0000 7fdc7560e640 1 -- 172.21.15.100:0/2333872941 >> 172.21.15.100:0/2333872941 conn(0x7fdc700ca390 msgr2=0x7fdc700ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.298+0000 7fdc7560e640 1 -- 172.21.15.100:0/2333872941 shutdown_connections 2024-09-15T21:48:01.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.298+0000 7fdc7560e640 1 -- 172.21.15.100:0/2333872941 wait complete. 2024-09-15T21:48:01.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.300+0000 7fdc7560e640 1 Processor -- start 2024-09-15T21:48:01.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.300+0000 7fdc7560e640 1 -- start start 2024-09-15T21:48:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.301+0000 7fdc7560e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdc700724c0 0x7fdc700ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.301+0000 7fdc7560e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc70159580 0x7fdc700cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.301+0000 7fdc7560e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc70159960 0x7fdc70152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.301+0000 7fdc7560e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdc700c8d80 con 0x7fdc70159960 2024-09-15T21:48:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.301+0000 7fdc7560e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdc70162cf0 con 0x7fdc70159580 2024-09-15T21:48:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.301+0000 7fdc7560e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fdc700cb160 con 0x7fdc700724c0 2024-09-15T21:48:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.301+0000 7fdc6f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc70159960 0x7fdc70152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.301+0000 7fdc6f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc70159960 0x7fdc70152e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57748/0 (socket says 172.21.15.100:57748) 2024-09-15T21:48:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.301+0000 7fdc6f7fe640 1 -- 172.21.15.100:0/2700111762 learned_addr learned my addr 172.21.15.100:0/2700111762 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:01.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.301+0000 7fdc6e7fc640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc70159580 0x7fdc700cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.302+0000 7fdc6f7fe640 1 -- 172.21.15.100:0/2700111762 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdc700724c0 msgr2=0x7fdc700ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.302+0000 7fdc6f7fe640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdc700724c0 0x7fdc700ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.302+0000 7fdc6f7fe640 1 -- 172.21.15.100:0/2700111762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc70159580 msgr2=0x7fdc700cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.302+0000 7fdc6f7fe640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdc70159580 0x7fdc700cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.302+0000 7fdc6f7fe640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7015e9c0 con 0x7fdc70159960 2024-09-15T21:48:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.302+0000 7fdc6f7fe640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdc70159960 0x7fdc70152e20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdc68009e50 tx=0x7fdc68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.302+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6803f070 con 0x7fdc70159960 2024-09-15T21:48:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.302+0000 7fdc7560e640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7015e390 con 0x7fdc70159960 2024-09-15T21:48:01.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.302+0000 7fdc7560e640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc701e9990 con 0x7fdc70159960 2024-09-15T21:48:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.302+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc6803b820 con 0x7fdc70159960 2024-09-15T21:48:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.302+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6803bbf0 con 0x7fdc70159960 2024-09-15T21:48:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.304+0000 7fdc7560e640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fdc700cdb30 con 0x7fdc70159960 2024-09-15T21:48:01.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.304+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdc68044460 con 0x7fdc70159960 2024-09-15T21:48:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.305+0000 7fdc577fe640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fdc5c06c2e0 0x7fdc5c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.305+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 6085+0+0 (secure 0 0 0) 0x7fdc680ac060 con 0x7fdc70159960 2024-09-15T21:48:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.305+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v49) ==== 54+0+16 (secure 0 0 0) 0x7fdc700cdb30 con 0x7fdc70159960 2024-09-15T21:48:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.305+0000 7fdc6effd640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fdc5c06c2e0 0x7fdc5c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.305+0000 7fdc557fa640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fdc48001650 0x7fdc48003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:01.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.305+0000 7fdc557fa640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdc48006bb0 con 0x7fdc48001650 2024-09-15T21:48:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.306+0000 7fdc6e7fc640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fdc48001650 0x7fdc48003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.306+0000 7fdc6effd640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fdc5c06c2e0 0x7fdc5c06e7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdc60009a40 tx=0x7fdc60009500 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:01.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.306+0000 7fdc6e7fc640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fdc48001650 0x7fdc48003b10 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:01.310 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:48:01.307+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:48:01.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.307+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc48006bb0 con 0x7fdc48001650 2024-09-15T21:48:01.337 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.335+0000 7fdc557fa640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdc48005c60 con 0x7fdc48001650 2024-09-15T21:48:01.340 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.337+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdc48005c60 con 0x7fdc48001650 2024-09-15T21:48:01.340 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:48:01.341 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:48:01.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.338+0000 7fdc557fa640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fdc48004190 0x7fdc48004640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:01.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.338+0000 7fdc557fa640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fdc48004c00 con 0x7fdc48004190 2024-09-15T21:48:01.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.338+0000 7fdc6effd640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fdc48004190 0x7fdc48004640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:01.341 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.338+0000 7fdc6effd640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fdc48004190 0x7fdc48004640 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:01.342 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.339+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc48004c00 con 0x7fdc48004190 2024-09-15T21:48:01.370 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.367+0000 7fdc557fa640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdc480051a0 con 0x7fdc48004190 2024-09-15T21:48:01.373 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.370+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdc480051a0 con 0x7fdc48004190 2024-09-15T21:48:01.373 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:48:01.374 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:48:01.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.370+0000 7fdc557fa640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fdc48009370 0x7fdc4800b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:01.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.371+0000 7fdc6e7fc640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fdc48009370 0x7fdc4800b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:01.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.371+0000 7fdc557fa640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fdc4800e720 con 0x7fdc48009370 2024-09-15T21:48:01.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.371+0000 7fdc6e7fc640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fdc48009370 0x7fdc4800b760 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:01.375 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.372+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc4800e720 con 0x7fdc48009370 2024-09-15T21:48:01.393 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:01.393 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:48:01.393 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:48:01.393 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:48:01.393 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.394 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:32 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'18, mlcod=17'14)", 2024-09-15T21:48:01.394 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:48:01.394 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 107.03389073699999, 2024-09-15T21:48:01.394 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052297760000000002, 2024-09-15T21:48:01.394 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.394 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.394 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.394 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.394 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.394 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:48:01.394 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.394 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.394 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.395 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.395 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:48:01.395 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.395 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.395 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.395 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.395 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356706+0000", 2024-09-15T21:48:01.395 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.06e-07 2024-09-15T21:48:01.395 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.395 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.395 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.395 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356780+0000", 2024-09-15T21:48:01.395 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4208999999999994e-05 2024-09-15T21:48:01.396 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.396 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.396 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.396 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356789+0000", 2024-09-15T21:48:01.396 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2800000000000003e-06 2024-09-15T21:48:01.396 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.396 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.396 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.396 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356793+0000", 2024-09-15T21:48:01.396 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:48:01.396 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.396 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.396 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.396 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356838+0000", 2024-09-15T21:48:01.397 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4851999999999997e-05 2024-09-15T21:48:01.397 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.397 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.397 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.397 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.356854+0000", 2024-09-15T21:48:01.397 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6124999999999999e-05 2024-09-15T21:48:01.397 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.397 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.397 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.398 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.361892+0000", 2024-09-15T21:48:01.398 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050379140000000001 2024-09-15T21:48:01.398 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.398 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.398 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.398 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.361935+0000", 2024-09-15T21:48:01.398 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3217999999999999e-05 2024-09-15T21:48:01.398 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.398 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.398 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.398 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.398 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.398 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:35 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'21, mlcod=17'15)", 2024-09-15T21:48:01.398 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:48:01.399 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 107.031853607, 2024-09-15T21:48:01.399 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055652039999999998, 2024-09-15T21:48:01.399 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.399 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.399 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.400 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.400 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.400 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.400 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:48:01.400 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.400 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.400 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.400 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358743+0000", 2024-09-15T21:48:01.400 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.400 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.400 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.400 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.400 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358744+0000", 2024-09-15T21:48:01.401 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4399999999999998e-07 2024-09-15T21:48:01.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.401 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.401 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.401 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358773+0000", 2024-09-15T21:48:01.401 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8847e-05 2024-09-15T21:48:01.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.401 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.401 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.401 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358783+0000", 2024-09-15T21:48:01.401 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0299000000000001e-05 2024-09-15T21:48:01.401 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.401 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.401 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.402 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358796+0000", 2024-09-15T21:48:01.402 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3499000000000001e-05 2024-09-15T21:48:01.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.402 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.402 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.402 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358850+0000", 2024-09-15T21:48:01.402 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3952000000000001e-05 2024-09-15T21:48:01.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.402 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.402 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.402 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.358872+0000", 2024-09-15T21:48:01.402 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1424999999999999e-05 2024-09-15T21:48:01.402 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.403 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.403 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.403 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364273+0000", 2024-09-15T21:48:01.403 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054011299999999996 2024-09-15T21:48:01.403 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.403 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.403 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.403 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364308+0000", 2024-09-15T21:48:01.403 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5108000000000003e-05 2024-09-15T21:48:01.403 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.403 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.403 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.403 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.403 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.404 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:36 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'22, mlcod=17'15)", 2024-09-15T21:48:01.404 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:48:01.404 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 107.03105941699999, 2024-09-15T21:48:01.404 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047904530000000001, 2024-09-15T21:48:01.404 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.404 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.404 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.404 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.404 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.404 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.404 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:48:01.404 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.404 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.405 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.405 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:48:01.405 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.405 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.405 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.405 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359537+0000", 2024-09-15T21:48:01.405 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6000000000000004e-07 2024-09-15T21:48:01.405 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.405 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.405 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.405 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359542+0000", 2024-09-15T21:48:01.405 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3510000000000004e-06 2024-09-15T21:48:01.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.406 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.406 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359552+0000", 2024-09-15T21:48:01.406 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0263e-05 2024-09-15T21:48:01.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.406 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.406 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359557+0000", 2024-09-15T21:48:01.406 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0479999999999998e-06 2024-09-15T21:48:01.406 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.406 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.406 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.406 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359593+0000", 2024-09-15T21:48:01.407 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6151e-05 2024-09-15T21:48:01.407 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.407 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.407 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.407 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.359608+0000", 2024-09-15T21:48:01.407 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4609e-05 2024-09-15T21:48:01.407 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.407 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.407 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.407 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364310+0000", 2024-09-15T21:48:01.407 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047025510000000001 2024-09-15T21:48:01.407 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.407 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.408 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.408 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:14.364327+0000", 2024-09-15T21:48:01.408 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6920000000000001e-05 2024-09-15T21:48:01.408 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.408 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.408 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.408 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.408 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.408 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:131 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'89, mlcod=18'85)", 2024-09-15T21:48:01.408 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:48:01.408 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 105.49251348200001, 2024-09-15T21:48:01.408 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057443499999999996, 2024-09-15T21:48:01.408 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.409 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.409 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.409 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.409 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.409 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.409 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:48:01.409 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.409 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.409 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.409 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.409 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:48:01.409 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.409 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.409 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.410 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.410 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898083+0000", 2024-09-15T21:48:01.410 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8899999999999999e-07 2024-09-15T21:48:01.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.410 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.410 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.410 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898167+0000", 2024-09-15T21:48:01.410 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3171999999999996e-05 2024-09-15T21:48:01.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.410 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.410 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.410 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898177+0000", 2024-09-15T21:48:01.410 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0866000000000001e-05 2024-09-15T21:48:01.410 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.411 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.411 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898185+0000", 2024-09-15T21:48:01.411 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2490000000000004e-06 2024-09-15T21:48:01.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.411 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.411 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898224+0000", 2024-09-15T21:48:01.411 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9237999999999999e-05 2024-09-15T21:48:01.411 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.411 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.411 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.411 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898238+0000", 2024-09-15T21:48:01.412 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.366e-05 2024-09-15T21:48:01.412 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.412 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.412 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.412 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.903803+0000", 2024-09-15T21:48:01.412 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005565291 2024-09-15T21:48:01.412 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.412 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.412 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.412 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.903827+0000", 2024-09-15T21:48:01.412 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4284999999999999e-05 2024-09-15T21:48:01.412 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.412 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.413 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.413 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.413 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.413 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:132 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'90, mlcod=18'85)", 2024-09-15T21:48:01.413 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:48:01.413 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 105.49202175800001, 2024-09-15T21:48:01.413 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055085469999999999, 2024-09-15T21:48:01.413 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.413 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.413 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.413 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.413 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.413 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.414 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:48:01.414 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.414 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.414 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.414 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.414 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:48:01.414 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.414 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.414 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.414 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.414 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898575+0000", 2024-09-15T21:48:01.414 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9299999999999999e-07 2024-09-15T21:48:01.414 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.415 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.415 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.415 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898579+0000", 2024-09-15T21:48:01.415 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6650000000000001e-06 2024-09-15T21:48:01.415 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.415 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.415 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.415 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898586+0000", 2024-09-15T21:48:01.415 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8429999999999993e-06 2024-09-15T21:48:01.415 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.415 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.415 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.415 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898592+0000", 2024-09-15T21:48:01.415 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3990000000000003e-06 2024-09-15T21:48:01.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.416 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.416 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.416 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898621+0000", 2024-09-15T21:48:01.416 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9019000000000001e-05 2024-09-15T21:48:01.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.416 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.416 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.416 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.898630+0000", 2024-09-15T21:48:01.416 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5540000000000003e-06 2024-09-15T21:48:01.416 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.416 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.416 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.417 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904047+0000", 2024-09-15T21:48:01.417 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054167169999999997 2024-09-15T21:48:01.417 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.417 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.417 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.417 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904083+0000", 2024-09-15T21:48:01.417 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5957000000000001e-05 2024-09-15T21:48:01.417 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.417 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.417 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.417 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.417 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.417 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(client.4181.0:133 1.0 e18/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 18'91, mlcod=18'87)", 2024-09-15T21:48:01.417 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:48:01.418 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 105.491432609, 2024-09-15T21:48:01.418 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049424780000000001, 2024-09-15T21:48:01.418 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.418 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.418 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.418 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.418 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.418 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.418 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:48:01.418 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.419 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.419 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.419 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899164+0000", 2024-09-15T21:48:01.419 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.419 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.419 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.419 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899166+0000", 2024-09-15T21:48:01.419 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0779999999999998e-06 2024-09-15T21:48:01.419 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.419 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.419 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.420 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899174+0000", 2024-09-15T21:48:01.420 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.729e-06 2024-09-15T21:48:01.420 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.420 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.420 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.420 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899181+0000", 2024-09-15T21:48:01.420 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4080000000000001e-06 2024-09-15T21:48:01.420 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.421 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.421 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.421 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899185+0000", 2024-09-15T21:48:01.421 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0840000000000002e-06 2024-09-15T21:48:01.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.421 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.421 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.421 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899240+0000", 2024-09-15T21:48:01.421 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4874999999999999e-05 2024-09-15T21:48:01.421 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.421 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.422 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.422 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.899257+0000", 2024-09-15T21:48:01.422 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7549000000000001e-05 2024-09-15T21:48:01.422 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.422 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.422 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.422 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904085+0000", 2024-09-15T21:48:01.422 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048272640000000004 2024-09-15T21:48:01.422 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.422 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.422 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.422 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:15.904106+0000", 2024-09-15T21:48:01.422 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1491000000000001e-05 2024-09-15T21:48:01.422 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.423 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.423 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.423 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.423 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.423 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:4 2.1f 2:ff5b34d6:::1.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:01.423 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:48:01.423 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 92.977209333000005, 2024-09-15T21:48:01.423 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046608459999999997, 2024-09-15T21:48:01.423 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.423 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.423 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.423 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.423 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.423 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.424 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:48:01.424 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.424 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.424 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413387+0000", 2024-09-15T21:48:01.424 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.424 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.424 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.425 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.425 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413391+0000", 2024-09-15T21:48:01.425 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9899999999999999e-06 2024-09-15T21:48:01.425 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.425 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.425 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.425 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413399+0000", 2024-09-15T21:48:01.425 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6310000000000003e-06 2024-09-15T21:48:01.425 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.425 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.425 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.425 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413444+0000", 2024-09-15T21:48:01.425 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5052000000000002e-05 2024-09-15T21:48:01.425 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.426 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413453+0000", 2024-09-15T21:48:01.426 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8869999999999998e-06 2024-09-15T21:48:01.426 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.426 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413576+0000", 2024-09-15T21:48:01.426 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012358700000000001 2024-09-15T21:48:01.426 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.426 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.426 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.426 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413679+0000", 2024-09-15T21:48:01.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000103318 2024-09-15T21:48:01.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.427 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:01.427 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413785+0000", 2024-09-15T21:48:01.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010561 2024-09-15T21:48:01.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.427 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:01.427 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416276+0000", 2024-09-15T21:48:01.427 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002490754 2024-09-15T21:48:01.427 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.427 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.427 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:01.428 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417949+0000", 2024-09-15T21:48:01.428 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016733519999999999 2024-09-15T21:48:01.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.428 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.428 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.428 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418005+0000", 2024-09-15T21:48:01.428 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5747e-05 2024-09-15T21:48:01.428 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.428 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.428 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.428 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418048+0000", 2024-09-15T21:48:01.428 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2917999999999998e-05 2024-09-15T21:48:01.428 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.428 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.429 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.429 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.429 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.429 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:5 2.1f 2:ff5b34d6:::1.00000000:head [create,setxattr parent (30) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:01.429 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:48:01.429 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 92.977194862999994, 2024-09-15T21:48:01.429 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051419999999999999, 2024-09-15T21:48:01.429 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.429 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.429 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.430 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.430 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:48:01.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.430 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413401+0000", 2024-09-15T21:48:01.430 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.430 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.430 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.430 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.431 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413402+0000", 2024-09-15T21:48:01.431 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6899999999999998e-07 2024-09-15T21:48:01.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.431 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.431 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.431 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413405+0000", 2024-09-15T21:48:01.431 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8109999999999999e-06 2024-09-15T21:48:01.431 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.431 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.431 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.431 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413517+0000", 2024-09-15T21:48:01.432 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000112592 2024-09-15T21:48:01.432 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.432 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.432 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.432 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413527+0000", 2024-09-15T21:48:01.432 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0144e-05 2024-09-15T21:48:01.432 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.432 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.432 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.432 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414155+0000", 2024-09-15T21:48:01.432 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00062767200000000002 2024-09-15T21:48:01.432 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.432 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.433 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.433 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414239+0000", 2024-09-15T21:48:01.433 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3607000000000001e-05 2024-09-15T21:48:01.433 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.433 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.433 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:01.433 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414368+0000", 2024-09-15T21:48:01.433 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000129049 2024-09-15T21:48:01.433 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.433 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.433 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:01.433 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416332+0000", 2024-09-15T21:48:01.433 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019645109999999999 2024-09-15T21:48:01.434 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.434 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.434 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:01.434 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418446+0000", 2024-09-15T21:48:01.434 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021137640000000002 2024-09-15T21:48:01.434 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.434 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.434 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.434 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418503+0000", 2024-09-15T21:48:01.434 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7166999999999997e-05 2024-09-15T21:48:01.434 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.434 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.434 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.434 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418543+0000", 2024-09-15T21:48:01.435 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0314000000000001e-05 2024-09-15T21:48:01.435 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.435 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.435 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.435 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.435 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.435 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:26 2.1f 2:fe07bba1:::100.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:01.435 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:48:01.435 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 92.977181672, 2024-09-15T21:48:01.435 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057945979999999998, 2024-09-15T21:48:01.435 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.435 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.435 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.435 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.436 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.436 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.436 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:48:01.436 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.436 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.436 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.436 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.436 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:48:01.436 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.436 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.436 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.436 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.436 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413415+0000", 2024-09-15T21:48:01.436 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9499999999999998e-07 2024-09-15T21:48:01.437 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.437 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.437 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.437 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413417+0000", 2024-09-15T21:48:01.437 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3910000000000001e-06 2024-09-15T21:48:01.437 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.437 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.437 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.437 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414570+0000", 2024-09-15T21:48:01.437 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011525890000000001 2024-09-15T21:48:01.437 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.437 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.437 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.437 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414577+0000", 2024-09-15T21:48:01.438 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1840000000000002e-06 2024-09-15T21:48:01.438 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.438 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.438 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.438 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414733+0000", 2024-09-15T21:48:01.438 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015563900000000001 2024-09-15T21:48:01.438 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.438 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.438 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.438 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414831+0000", 2024-09-15T21:48:01.438 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8138000000000001e-05 2024-09-15T21:48:01.438 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.438 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.439 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:01.439 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414956+0000", 2024-09-15T21:48:01.439 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000124604 2024-09-15T21:48:01.439 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.439 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.439 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:01.439 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418804+0000", 2024-09-15T21:48:01.439 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038484299999999999 2024-09-15T21:48:01.439 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.439 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.439 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:01.439 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419124+0000", 2024-09-15T21:48:01.440 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00032017699999999999 2024-09-15T21:48:01.440 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.440 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.440 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.440 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419171+0000", 2024-09-15T21:48:01.440 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6520000000000002e-05 2024-09-15T21:48:01.440 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.440 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.440 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.440 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419209+0000", 2024-09-15T21:48:01.441 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8531000000000001e-05 2024-09-15T21:48:01.441 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.441 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.441 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.441 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.441 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.441 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:35 2.1f 2:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:01.441 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:48:01.441 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 92.977174575999996, 2024-09-15T21:48:01.441 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059183300000000003, 2024-09-15T21:48:01.441 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.441 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.441 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:48:01.441 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.442 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.442 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.442 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:48:01.442 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.442 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.442 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.442 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.442 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:48:01.442 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.442 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.442 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.442 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.442 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413422+0000", 2024-09-15T21:48:01.442 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3000000000000002e-07 2024-09-15T21:48:01.443 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.443 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.443 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.443 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413426+0000", 2024-09-15T21:48:01.443 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7110000000000001e-06 2024-09-15T21:48:01.443 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.443 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.443 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.443 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414621+0000", 2024-09-15T21:48:01.443 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001195022 2024-09-15T21:48:01.443 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.443 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.443 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.444 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414625+0000", 2024-09-15T21:48:01.444 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6360000000000002e-06 2024-09-15T21:48:01.444 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.444 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.444 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415269+0000", 2024-09-15T21:48:01.444 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00064315399999999995 2024-09-15T21:48:01.444 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.444 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.444 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.445 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415356+0000", 2024-09-15T21:48:01.445 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.7410999999999994e-05 2024-09-15T21:48:01.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.445 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.445 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.445 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419340+0000", 2024-09-15T21:48:01.445 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0039840659999999996 2024-09-15T21:48:01.445 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.445 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.445 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.445 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.445 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.445 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:2 2.f e22/19 2:f1f69726:::400.00000000:head v 22'1, mlcod=0'0)", 2024-09-15T21:48:01.445 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:48:01.446 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 92.976866985000001, 2024-09-15T21:48:01.446 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053379719999999999, 2024-09-15T21:48:01.446 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.446 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.446 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.446 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.446 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.446 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.446 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:48:01.446 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.446 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.446 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.446 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.446 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413729+0000", 2024-09-15T21:48:01.447 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.447 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.447 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.447 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.447 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413730+0000", 2024-09-15T21:48:01.447 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1379999999999999e-06 2024-09-15T21:48:01.447 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.447 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.447 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.447 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413740+0000", 2024-09-15T21:48:01.447 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.4830000000000005e-06 2024-09-15T21:48:01.447 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.447 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.448 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.448 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413751+0000", 2024-09-15T21:48:01.448 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1228e-05 2024-09-15T21:48:01.448 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.448 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.448 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.448 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413758+0000", 2024-09-15T21:48:01.448 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1289999999999998e-06 2024-09-15T21:48:01.448 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.448 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.448 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.448 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414038+0000", 2024-09-15T21:48:01.448 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00027935999999999998 2024-09-15T21:48:01.448 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.449 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.449 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.449 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414055+0000", 2024-09-15T21:48:01.449 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7325000000000001e-05 2024-09-15T21:48:01.449 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.449 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.449 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.449 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419028+0000", 2024-09-15T21:48:01.449 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049730720000000003 2024-09-15T21:48:01.449 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.449 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.449 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.449 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419067+0000", 2024-09-15T21:48:01.450 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9236999999999997e-05 2024-09-15T21:48:01.450 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.450 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.450 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.450 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.450 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.450 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:18 2.13 2:ca4dd7f5:::611.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:01.450 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:48:01.450 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 86.934943718, 2024-09-15T21:48:01.450 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.92695576499999999, 2024-09-15T21:48:01.450 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.450 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.450 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.450 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.451 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.451 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:48:01.451 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.451 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.451 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:48:01.451 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.451 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.451 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.451 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.451 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455653+0000", 2024-09-15T21:48:01.452 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8500000000000002e-07 2024-09-15T21:48:01.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.452 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.452 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455655+0000", 2024-09-15T21:48:01.452 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.401e-06 2024-09-15T21:48:01.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.452 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.452 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.380965+0000", 2024-09-15T21:48:01.452 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.92530946000000003 2024-09-15T21:48:01.452 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.452 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.453 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.453 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.380981+0000", 2024-09-15T21:48:01.453 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6209e-05 2024-09-15T21:48:01.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.453 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.453 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.453 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381086+0000", 2024-09-15T21:48:01.453 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105011 2024-09-15T21:48:01.453 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.453 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.453 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.453 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381154+0000", 2024-09-15T21:48:01.453 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7596999999999997e-05 2024-09-15T21:48:01.454 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.454 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.454 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:48:01.454 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381249+0000", 2024-09-15T21:48:01.454 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5354999999999998e-05 2024-09-15T21:48:01.454 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.454 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.454 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:01.454 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.381844+0000", 2024-09-15T21:48:01.454 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00059518600000000004 2024-09-15T21:48:01.454 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.454 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.454 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:01.455 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382531+0000", 2024-09-15T21:48:01.455 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00068671499999999996 2024-09-15T21:48:01.455 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.455 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.455 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.455 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382570+0000", 2024-09-15T21:48:01.455 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9373999999999999e-05 2024-09-15T21:48:01.455 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.455 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.455 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.455 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:35.382608+0000", 2024-09-15T21:48:01.455 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8071999999999997e-05 2024-09-15T21:48:01.455 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.455 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.456 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.456 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.456 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.456 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:6 2.1f 2:fa28013d:::615.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:01.456 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:48:01.456 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 80.915034458999997, 2024-09-15T21:48:01.456 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047117319999999997, 2024-09-15T21:48:01.456 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.456 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.456 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.456 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.456 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.456 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.457 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:48:01.457 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.457 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.457 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:48:01.457 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.457 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.457 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.457 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475562+0000", 2024-09-15T21:48:01.457 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4700000000000001e-07 2024-09-15T21:48:01.457 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.458 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.458 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.458 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475566+0000", 2024-09-15T21:48:01.458 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1669999999999999e-06 2024-09-15T21:48:01.458 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.458 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.458 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.458 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476316+0000", 2024-09-15T21:48:01.458 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075022999999999995 2024-09-15T21:48:01.458 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.458 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.458 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.458 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476324+0000", 2024-09-15T21:48:01.458 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.259e-06 2024-09-15T21:48:01.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.459 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.459 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.459 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476373+0000", 2024-09-15T21:48:01.459 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8807999999999998e-05 2024-09-15T21:48:01.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.459 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.459 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.459 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476446+0000", 2024-09-15T21:48:01.459 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3138000000000003e-05 2024-09-15T21:48:01.459 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.459 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.459 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:01.460 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476593+0000", 2024-09-15T21:48:01.460 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00014680199999999999 2024-09-15T21:48:01.460 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.460 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.460 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:01.460 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480189+0000", 2024-09-15T21:48:01.460 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035961370000000001 2024-09-15T21:48:01.460 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.460 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.460 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:01.460 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480198+0000", 2024-09-15T21:48:01.460 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.6649999999999992e-06 2024-09-15T21:48:01.460 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.460 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.461 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.461 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480235+0000", 2024-09-15T21:48:01.461 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6943999999999997e-05 2024-09-15T21:48:01.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.461 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.461 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.461 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480274+0000", 2024-09-15T21:48:01.461 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9035000000000001e-05 2024-09-15T21:48:01.461 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.461 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.461 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.461 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.461 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.461 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:7 2.13 2:cdf3ac81:::616.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:01.462 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:48:01.462 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 80.915029455999999, 2024-09-15T21:48:01.462 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046830109999999999, 2024-09-15T21:48:01.462 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.462 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.462 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.462 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.462 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.462 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.462 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:48:01.462 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.462 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.462 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.462 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.463 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:48:01.463 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.463 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.463 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.463 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.463 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:48:01.463 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.53e-07 2024-09-15T21:48:01.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.464 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.464 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.464 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475570+0000", 2024-09-15T21:48:01.464 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7470000000000001e-06 2024-09-15T21:48:01.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.464 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.464 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.464 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476353+0000", 2024-09-15T21:48:01.464 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00078312300000000004 2024-09-15T21:48:01.464 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.464 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.464 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.465 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476359+0000", 2024-09-15T21:48:01.465 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.117e-06 2024-09-15T21:48:01.465 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.465 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.465 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.465 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476412+0000", 2024-09-15T21:48:01.465 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2398999999999998e-05 2024-09-15T21:48:01.465 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.465 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.465 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.465 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476524+0000", 2024-09-15T21:48:01.465 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011293 2024-09-15T21:48:01.465 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.465 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.466 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:48:01.466 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476909+0000", 2024-09-15T21:48:01.466 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00038490099999999999 2024-09-15T21:48:01.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.466 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.466 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:01.466 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479761+0000", 2024-09-15T21:48:01.466 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002851737 2024-09-15T21:48:01.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.466 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.466 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:01.466 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480154+0000", 2024-09-15T21:48:01.466 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00039307399999999998 2024-09-15T21:48:01.466 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.467 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.467 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480203+0000", 2024-09-15T21:48:01.467 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8418000000000003e-05 2024-09-15T21:48:01.467 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.467 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.467 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480250+0000", 2024-09-15T21:48:01.467 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7311999999999997e-05 2024-09-15T21:48:01.467 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.467 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.467 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.467 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.467 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.468 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:8 2.13 2:cdf3ac81:::616.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:01.468 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:48:01.468 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 80.915025294000003, 2024-09-15T21:48:01.468 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005035398, 2024-09-15T21:48:01.468 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.468 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.468 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.468 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.468 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.468 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.468 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:48:01.468 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.468 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.469 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:48:01.469 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.469 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475571+0000", 2024-09-15T21:48:01.469 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.22e-07 2024-09-15T21:48:01.469 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.469 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.469 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.469 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475574+0000", 2024-09-15T21:48:01.469 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.593e-06 2024-09-15T21:48:01.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.470 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.470 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476579+0000", 2024-09-15T21:48:01.470 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010053989999999999 2024-09-15T21:48:01.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.470 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.470 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476588+0000", 2024-09-15T21:48:01.470 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4530000000000002e-06 2024-09-15T21:48:01.470 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.470 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.470 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.470 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477723+0000", 2024-09-15T21:48:01.471 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001134962 2024-09-15T21:48:01.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.471 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.471 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.471 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477801+0000", 2024-09-15T21:48:01.471 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8228999999999999e-05 2024-09-15T21:48:01.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.471 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.471 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:48:01.471 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477946+0000", 2024-09-15T21:48:01.471 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00014553400000000001 2024-09-15T21:48:01.471 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.471 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.471 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:01.472 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480258+0000", 2024-09-15T21:48:01.472 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023112039999999999 2024-09-15T21:48:01.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.472 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.472 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:01.472 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480504+0000", 2024-09-15T21:48:01.472 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024667500000000002 2024-09-15T21:48:01.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.472 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.472 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.472 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480556+0000", 2024-09-15T21:48:01.472 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2114999999999998e-05 2024-09-15T21:48:01.472 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.473 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.473 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.473 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480606+0000", 2024-09-15T21:48:01.473 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0012000000000002e-05 2024-09-15T21:48:01.473 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.473 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.473 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.473 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.473 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.473 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:19 2.1f 2:fdd45aa0:::61c.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:01.473 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:48:01.473 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 80.915021363999998, 2024-09-15T21:48:01.473 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050672490000000002, 2024-09-15T21:48:01.473 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:48:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:48:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.474 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.475 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.475 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475575+0000", 2024-09-15T21:48:01.475 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:48:01.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.475 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.475 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.475 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475577+0000", 2024-09-15T21:48:01.475 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2290000000000002e-06 2024-09-15T21:48:01.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.475 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.475 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.475 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476605+0000", 2024-09-15T21:48:01.475 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010278780000000001 2024-09-15T21:48:01.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.476 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.476 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.476 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476611+0000", 2024-09-15T21:48:01.476 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5600000000000001e-06 2024-09-15T21:48:01.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.476 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.476 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.476 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477053+0000", 2024-09-15T21:48:01.476 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044162800000000001 2024-09-15T21:48:01.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.476 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.476 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.476 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477180+0000", 2024-09-15T21:48:01.477 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000127129 2024-09-15T21:48:01.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.477 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.477 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:01.477 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477288+0000", 2024-09-15T21:48:01.477 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010789999999999999 2024-09-15T21:48:01.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.477 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.477 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:01.477 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480288+0000", 2024-09-15T21:48:01.477 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003000052 2024-09-15T21:48:01.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.477 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.477 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:01.478 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480568+0000", 2024-09-15T21:48:01.478 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028026899999999998 2024-09-15T21:48:01.478 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.478 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.478 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.478 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480609+0000", 2024-09-15T21:48:01.478 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1437999999999999e-05 2024-09-15T21:48:01.478 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.478 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.478 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.478 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480642+0000", 2024-09-15T21:48:01.478 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2879999999999997e-05 2024-09-15T21:48:01.479 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.479 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.479 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.479 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.479 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:20 2.1f 2:fdd45aa0:::61c.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:01.479 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:48:01.479 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 80.915017692999996, 2024-09-15T21:48:01.479 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053112589999999996, 2024-09-15T21:48:01.479 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.479 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.479 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.479 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.479 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.480 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.480 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:48:01.480 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.480 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.480 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.480 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:48:01.480 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.480 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.480 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.480 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:48:01.480 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:48:01.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.481 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.481 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.481 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475581+0000", 2024-09-15T21:48:01.481 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1009999999999999e-06 2024-09-15T21:48:01.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.481 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.481 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.481 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476628+0000", 2024-09-15T21:48:01.481 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010465380000000001 2024-09-15T21:48:01.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.481 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.481 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.481 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476633+0000", 2024-09-15T21:48:01.482 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9869999999999999e-06 2024-09-15T21:48:01.482 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.482 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.482 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.482 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477618+0000", 2024-09-15T21:48:01.482 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00098539299999999999 2024-09-15T21:48:01.482 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.482 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.482 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.482 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477696+0000", 2024-09-15T21:48:01.482 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8509000000000003e-05 2024-09-15T21:48:01.482 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.482 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.482 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:01.483 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477824+0000", 2024-09-15T21:48:01.483 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000127062 2024-09-15T21:48:01.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.483 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.483 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:01.483 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480294+0000", 2024-09-15T21:48:01.483 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024703939999999999 2024-09-15T21:48:01.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.483 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.483 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:01.483 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480788+0000", 2024-09-15T21:48:01.483 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00049443800000000004 2024-09-15T21:48:01.483 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.484 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.484 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.484 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480845+0000", 2024-09-15T21:48:01.484 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6762000000000003e-05 2024-09-15T21:48:01.484 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.484 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.484 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.484 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480890+0000", 2024-09-15T21:48:01.484 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4789000000000001e-05 2024-09-15T21:48:01.484 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.484 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.484 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.484 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.484 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.485 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:32 2.1f 2:fe09a0b8:::202.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:01.485 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:48:01.485 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 80.915014221999996, 2024-09-15T21:48:01.485 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007191546, 2024-09-15T21:48:01.485 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.485 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.485 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:48:01.485 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.485 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.485 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.485 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:48:01.485 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.485 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.486 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.486 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.486 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:48:01.486 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.487 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.487 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.487 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475582+0000", 2024-09-15T21:48:01.487 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9999999999999999e-07 2024-09-15T21:48:01.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.487 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.487 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.487 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475585+0000", 2024-09-15T21:48:01.487 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.384e-06 2024-09-15T21:48:01.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.487 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.487 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.488 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476665+0000", 2024-09-15T21:48:01.488 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010797140000000001 2024-09-15T21:48:01.488 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.488 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.488 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.488 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476671+0000", 2024-09-15T21:48:01.488 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3940000000000001e-06 2024-09-15T21:48:01.488 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.488 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.488 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.488 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478048+0000", 2024-09-15T21:48:01.488 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013767530000000001 2024-09-15T21:48:01.488 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.489 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.489 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.489 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478177+0000", 2024-09-15T21:48:01.489 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012898099999999999 2024-09-15T21:48:01.489 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.489 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.489 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.489 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482774+0000", 2024-09-15T21:48:01.489 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045970200000000003 2024-09-15T21:48:01.489 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.489 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.490 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.490 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.490 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.490 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:12 2.1f 2:ffe2d39d:::622.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:01.490 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:48:01.490 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 75.845784557000002, 2024-09-15T21:48:01.490 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055444480000000004, 2024-09-15T21:48:01.490 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.490 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.490 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.490 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.490 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.490 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.490 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:48:01.491 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.491 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.491 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.491 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:48:01.491 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.491 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.491 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.491 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544812+0000", 2024-09-15T21:48:01.491 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4200000000000002e-07 2024-09-15T21:48:01.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.491 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.491 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.492 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544814+0000", 2024-09-15T21:48:01.492 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9980000000000002e-06 2024-09-15T21:48:01.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.492 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.492 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.492 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544872+0000", 2024-09-15T21:48:01.492 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8405000000000001e-05 2024-09-15T21:48:01.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.492 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.492 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.492 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544877+0000", 2024-09-15T21:48:01.492 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.143e-06 2024-09-15T21:48:01.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.493 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.493 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.493 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545750+0000", 2024-09-15T21:48:01.493 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00087310000000000003 2024-09-15T21:48:01.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.493 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.493 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.493 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545850+0000", 2024-09-15T21:48:01.493 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9939999999999995e-05 2024-09-15T21:48:01.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.493 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.493 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:01.493 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546014+0000", 2024-09-15T21:48:01.494 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00016408500000000001 2024-09-15T21:48:01.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.494 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.494 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:01.494 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.548748+0000", 2024-09-15T21:48:01.494 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027339539999999998 2024-09-15T21:48:01.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.494 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.494 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:01.494 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550265+0000", 2024-09-15T21:48:01.494 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015170019999999999 2024-09-15T21:48:01.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.494 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.494 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.495 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550315+0000", 2024-09-15T21:48:01.495 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9867999999999998e-05 2024-09-15T21:48:01.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.495 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.495 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.495 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550356+0000", 2024-09-15T21:48:01.495 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1711000000000001e-05 2024-09-15T21:48:01.495 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.495 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.495 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.495 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.495 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:24 2.13 2:cbf94b35:::mds3_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:01.495 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:48:01.496 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 75.845781356000003, 2024-09-15T21:48:01.496 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058729619999999998, 2024-09-15T21:48:01.496 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.496 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.496 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.496 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.496 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.496 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.496 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:48:01.496 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.496 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.496 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.497 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:48:01.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.497 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.497 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.497 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544815+0000", 2024-09-15T21:48:01.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0599999999999999e-07 2024-09-15T21:48:01.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.497 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.497 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.497 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544821+0000", 2024-09-15T21:48:01.497 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4079999999999997e-06 2024-09-15T21:48:01.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.498 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.498 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544939+0000", 2024-09-15T21:48:01.498 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118532 2024-09-15T21:48:01.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.498 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.498 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544946+0000", 2024-09-15T21:48:01.498 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9260000000000003e-06 2024-09-15T21:48:01.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.498 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.498 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.498 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545011+0000", 2024-09-15T21:48:01.498 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4572000000000005e-05 2024-09-15T21:48:01.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.499 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.499 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545134+0000", 2024-09-15T21:48:01.499 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012309899999999999 2024-09-15T21:48:01.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.499 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:48:01.499 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545271+0000", 2024-09-15T21:48:01.499 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013746599999999999 2024-09-15T21:48:01.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.499 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.500 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:01.500 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550243+0000", 2024-09-15T21:48:01.500 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049714529999999998 2024-09-15T21:48:01.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.500 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.500 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:01.500 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550605+0000", 2024-09-15T21:48:01.500 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00036260699999999998 2024-09-15T21:48:01.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.500 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.501 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.501 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550662+0000", 2024-09-15T21:48:01.501 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6573000000000002e-05 2024-09-15T21:48:01.501 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.501 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.501 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.501 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550688+0000", 2024-09-15T21:48:01.501 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6120000000000001e-05 2024-09-15T21:48:01.501 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.501 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.501 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.501 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.501 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.502 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:01.503 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:48:01.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.399+0000 7fdc557fa640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdc4800d790 con 0x7fdc48009370 2024-09-15T21:48:01.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.402+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdc4800d790 con 0x7fdc48009370 2024-09-15T21:48:01.504 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:48:01.505 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:48:01.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.402+0000 7fdc557fa640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fdc4800bd20 0x7fdc4800c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:01.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.402+0000 7fdc557fa640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fdc4800e410 con 0x7fdc4800bd20 2024-09-15T21:48:01.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.402+0000 7fdc6f7fe640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fdc4800bd20 0x7fdc4800c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:01.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.402+0000 7fdc6f7fe640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fdc4800bd20 0x7fdc4800c1b0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:01.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.403+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc4800e410 con 0x7fdc4800bd20 2024-09-15T21:48:01.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.430+0000 7fdc557fa640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdc4800c800 con 0x7fdc4800bd20 2024-09-15T21:48:01.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.433+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdc4800c800 con 0x7fdc4800bd20 2024-09-15T21:48:01.506 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:48:01.507 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:48:01.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.433+0000 7fdc557fa640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fdc4800cfa0 0x7fdc48012980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:01.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.433+0000 7fdc557fa640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fdc48015860 con 0x7fdc4800cfa0 2024-09-15T21:48:01.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.433+0000 7fdc6effd640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fdc4800cfa0 0x7fdc48012980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:01.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.434+0000 7fdc6effd640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fdc4800cfa0 0x7fdc48012980 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:01.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.434+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc48015860 con 0x7fdc4800cfa0 2024-09-15T21:48:01.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.450+0000 7fdc557fa640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdc48012ec0 con 0x7fdc4800cfa0 2024-09-15T21:48:01.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.452+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdc48012ec0 con 0x7fdc4800cfa0 2024-09-15T21:48:01.508 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:48:01.509 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:48:01.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.452+0000 7fdc557fa640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=50}) -- 0x7fdc480147e0 con 0x7fdc70159960 2024-09-15T21:48:01.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.452+0000 7fdc557fa640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdc48012ec0 con 0x7fdc70159960 2024-09-15T21:48:01.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.452+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fdc680b1060 con 0x7fdc70159960 2024-09-15T21:48:01.509 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:48:01.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.452+0000 7fdc557fa640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fdc480130d0 0x7fdc48013540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:01.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.452+0000 7fdc557fa640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fdc48013a80 con 0x7fdc480130d0 2024-09-15T21:48:01.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.453+0000 7fdc6e7fc640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fdc480130d0 0x7fdc48013540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:01.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.453+0000 7fdc6e7fc640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fdc480130d0 0x7fdc48013540 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:01.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.454+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc48013a80 con 0x7fdc480130d0 2024-09-15T21:48:01.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.467+0000 7fdc557fa640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdc48013cc0 con 0x7fdc480130d0 2024-09-15T21:48:01.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.469+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdc48013cc0 con 0x7fdc480130d0 2024-09-15T21:48:01.510 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:48:01.511 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:48:01.502+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:48:01.511 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:48:01.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.469+0000 7fdc557fa640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fdc480141d0 0x7fdc48019ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:01.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.469+0000 7fdc557fa640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fdc4801c910 con 0x7fdc480141d0 2024-09-15T21:48:01.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.470+0000 7fdc6f7fe640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fdc480141d0 0x7fdc48019ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:01.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.470+0000 7fdc6f7fe640 1 --2- 172.21.15.100:0/2700111762 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fdc480141d0 0x7fdc48019ac0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:01.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.471+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdc4801c910 con 0x7fdc480141d0 2024-09-15T21:48:01.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.484+0000 7fdc557fa640 1 -- 172.21.15.100:0/2700111762 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fdc4801a080 con 0x7fdc480141d0 2024-09-15T21:48:01.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:01.486+0000 7fdc577fe640 1 -- 172.21.15.100:0/2700111762 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fdc4801a080 con 0x7fdc480141d0 2024-09-15T21:48:01.512 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:48:01.513 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:48:01.513 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:48:01.513 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:48:01.603 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:48:01.601+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:48:01.652 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:01.653 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:48:01.653 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:48:01.653 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:01.663 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:48:01.706 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:48:01.704+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:48:01.806 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:48:01.804+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:48:01.809 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:01.809 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:48:01.810 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:48:01.810 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:48:01.810 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:01.819 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:48:01.907 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:48:01.905+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:48:01.969 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:01.970 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:48:01.970 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:48:01.970 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:48:01.970 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.970 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:30 2.15 2:aa448500:::500.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:01.970 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:48:01.970 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 93.553898219000004, 2024-09-15T21:48:01.970 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006101291, 2024-09-15T21:48:01.970 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.970 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.970 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.970 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.970 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.971 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.971 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:48:01.971 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.971 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.971 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.971 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.971 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413115+0000", 2024-09-15T21:48:01.971 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.971 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.971 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.971 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.971 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413116+0000", 2024-09-15T21:48:01.971 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9700000000000003e-07 2024-09-15T21:48:01.971 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.972 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.972 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.972 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413118+0000", 2024-09-15T21:48:01.972 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2910000000000002e-06 2024-09-15T21:48:01.972 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.972 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.972 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.972 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413220+0000", 2024-09-15T21:48:01.972 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102056 2024-09-15T21:48:01.972 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.972 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.972 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.972 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413225+0000", 2024-09-15T21:48:01.973 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0490000000000003e-06 2024-09-15T21:48:01.973 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.973 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.973 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.973 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414808+0000", 2024-09-15T21:48:01.973 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0015826900000000001 2024-09-15T21:48:01.973 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.973 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.973 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.973 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414907+0000", 2024-09-15T21:48:01.973 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9600999999999997e-05 2024-09-15T21:48:01.973 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.973 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.974 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:48:01.974 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415054+0000", 2024-09-15T21:48:01.974 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000146532 2024-09-15T21:48:01.974 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.974 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.974 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:01.974 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418056+0000", 2024-09-15T21:48:01.974 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030020519999999998 2024-09-15T21:48:01.974 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.974 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.974 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:01.974 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419128+0000", 2024-09-15T21:48:01.975 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001071671 2024-09-15T21:48:01.975 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.975 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.975 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.975 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419184+0000", 2024-09-15T21:48:01.975 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6125000000000002e-05 2024-09-15T21:48:01.975 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.975 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.975 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.975 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419217+0000", 2024-09-15T21:48:01.975 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2926999999999998e-05 2024-09-15T21:48:01.976 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.976 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.976 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.976 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.976 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.976 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:31 2.d 2:b50e409b:::mds_snaptable:head [writefull 0~70] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:01.976 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:48:01.976 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 93.553894693000004, 2024-09-15T21:48:01.976 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059975130000000003, 2024-09-15T21:48:01.976 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.976 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.976 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.976 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.977 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.977 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.977 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:48:01.977 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.977 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.977 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.977 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.977 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:48:01.977 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.977 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.977 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.977 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.977 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413119+0000", 2024-09-15T21:48:01.978 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6199999999999999e-07 2024-09-15T21:48:01.978 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.978 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.978 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.978 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413121+0000", 2024-09-15T21:48:01.978 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2000000000000001e-06 2024-09-15T21:48:01.978 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.978 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.978 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.978 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413292+0000", 2024-09-15T21:48:01.978 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00017034400000000001 2024-09-15T21:48:01.978 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.978 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.978 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.979 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413300+0000", 2024-09-15T21:48:01.979 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8539999999999997e-06 2024-09-15T21:48:01.979 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.979 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.979 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.979 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414299+0000", 2024-09-15T21:48:01.979 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0009992829999999999 2024-09-15T21:48:01.979 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.979 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.979 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.979 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414390+0000", 2024-09-15T21:48:01.979 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1237999999999996e-05 2024-09-15T21:48:01.979 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.980 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.980 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 4", 2024-09-15T21:48:01.980 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414490+0000", 2024-09-15T21:48:01.980 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9438999999999994e-05 2024-09-15T21:48:01.980 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.980 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.980 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:01.980 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417393+0000", 2024-09-15T21:48:01.980 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002903306 2024-09-15T21:48:01.980 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.980 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.980 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:01.980 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419023+0000", 2024-09-15T21:48:01.981 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001630292 2024-09-15T21:48:01.981 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.981 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.981 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.981 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419067+0000", 2024-09-15T21:48:01.982 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4017999999999998e-05 2024-09-15T21:48:01.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.982 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.982 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.982 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419116+0000", 2024-09-15T21:48:01.982 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9277000000000003e-05 2024-09-15T21:48:01.982 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.982 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.982 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.982 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.982 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.982 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:7 2.10 e22/19 2:0d82a743:::600.00000000:head v 22'2, mlcod=0'0)", 2024-09-15T21:48:01.982 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:48:01.982 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 93.551196496000003, 2024-09-15T21:48:01.983 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060879280000000003, 2024-09-15T21:48:01.983 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.983 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.983 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.983 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.983 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.983 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.983 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:48:01.983 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.983 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.983 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.983 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.983 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415817+0000", 2024-09-15T21:48:01.984 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.984 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.984 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.984 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.984 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415818+0000", 2024-09-15T21:48:01.984 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:48:01.984 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.984 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.984 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.984 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415854+0000", 2024-09-15T21:48:01.984 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6706999999999999e-05 2024-09-15T21:48:01.984 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.984 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.985 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.985 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415860+0000", 2024-09-15T21:48:01.985 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0789999999999997e-06 2024-09-15T21:48:01.985 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.985 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.985 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.985 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415865+0000", 2024-09-15T21:48:01.985 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4580000000000003e-06 2024-09-15T21:48:01.985 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.985 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.985 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.985 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416092+0000", 2024-09-15T21:48:01.986 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00022739400000000001 2024-09-15T21:48:01.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.986 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.986 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.986 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416108+0000", 2024-09-15T21:48:01.986 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.613e-05 2024-09-15T21:48:01.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.986 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.986 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.986 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421866+0000", 2024-09-15T21:48:01.986 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005757585 2024-09-15T21:48:01.986 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.986 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.986 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.987 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421905+0000", 2024-09-15T21:48:01.987 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9214000000000001e-05 2024-09-15T21:48:01.987 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.987 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.987 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.987 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.987 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.987 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:8 2.10 e22/19 2:09401a81:::601.00000000:head v 22'3, mlcod=0'0)", 2024-09-15T21:48:01.987 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:48:01.987 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 93.551136335999999, 2024-09-15T21:48:01.987 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060637570000000003, 2024-09-15T21:48:01.987 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.987 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.988 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.988 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.988 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.988 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.988 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:48:01.988 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.988 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.988 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.988 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415877+0000", 2024-09-15T21:48:01.988 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.988 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.988 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.989 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.989 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415878+0000", 2024-09-15T21:48:01.989 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.34e-07 2024-09-15T21:48:01.989 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.989 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.989 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.989 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415881+0000", 2024-09-15T21:48:01.989 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.354e-06 2024-09-15T21:48:01.989 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.989 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.989 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.989 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415887+0000", 2024-09-15T21:48:01.989 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:48:01.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.990 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.990 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.990 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415891+0000", 2024-09-15T21:48:01.990 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2690000000000001e-06 2024-09-15T21:48:01.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.990 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.990 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.990 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416267+0000", 2024-09-15T21:48:01.990 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00037614799999999998 2024-09-15T21:48:01.990 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.990 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.990 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.990 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416288+0000", 2024-09-15T21:48:01.991 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0604000000000001e-05 2024-09-15T21:48:01.991 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.991 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.991 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.991 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421908+0000", 2024-09-15T21:48:01.991 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056203390000000002 2024-09-15T21:48:01.991 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.991 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.991 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.991 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421941+0000", 2024-09-15T21:48:01.991 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3079e-05 2024-09-15T21:48:01.991 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.991 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.992 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.992 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.992 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.992 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:9 2.10 e22/19 2:09401a81:::601.00000000:head v 22'4, mlcod=0'0)", 2024-09-15T21:48:01.992 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:48:01.992 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 93.551117908999998, 2024-09-15T21:48:01.992 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.006080284, 2024-09-15T21:48:01.992 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.992 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.992 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.992 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.992 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.992 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.992 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:48:01.993 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.993 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.993 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.993 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:48:01.993 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.993 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.993 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.993 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415896+0000", 2024-09-15T21:48:01.993 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6899999999999999e-07 2024-09-15T21:48:01.993 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.993 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.994 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.994 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415898+0000", 2024-09-15T21:48:01.994 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.951e-06 2024-09-15T21:48:01.994 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.994 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.994 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.994 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415920+0000", 2024-09-15T21:48:01.994 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1651999999999999e-05 2024-09-15T21:48:01.994 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.994 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.994 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.994 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415925+0000", 2024-09-15T21:48:01.994 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.463e-06 2024-09-15T21:48:01.994 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.995 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.995 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.995 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416449+0000", 2024-09-15T21:48:01.995 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052384000000000003 2024-09-15T21:48:01.995 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.995 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.995 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:01.995 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416466+0000", 2024-09-15T21:48:01.995 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7108999999999998e-05 2024-09-15T21:48:01.995 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.995 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.995 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:01.995 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421943+0000", 2024-09-15T21:48:01.995 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054771569999999999 2024-09-15T21:48:01.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.996 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.996 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:01.996 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421976+0000", 2024-09-15T21:48:01.996 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2943e-05 2024-09-15T21:48:01.996 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.996 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:01.996 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:01.996 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.996 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.997 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:10 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'5, mlcod=0'0)", 2024-09-15T21:48:01.997 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:48:01.997 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 93.551078673000006, 2024-09-15T21:48:01.997 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0060643340000000002, 2024-09-15T21:48:01.997 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:01.997 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:01.997 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:01.997 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:01.997 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.997 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:01.997 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:48:01.997 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.997 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.998 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.998 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:01.998 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:48:01.998 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:01.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.998 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.998 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:01.998 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415935+0000", 2024-09-15T21:48:01.998 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4200000000000002e-07 2024-09-15T21:48:01.998 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.998 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.998 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:01.998 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415939+0000", 2024-09-15T21:48:01.998 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.439e-06 2024-09-15T21:48:01.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.999 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.999 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:01.999 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415945+0000", 2024-09-15T21:48:01.999 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7740000000000001e-06 2024-09-15T21:48:01.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.999 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.999 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:01.999 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415948+0000", 2024-09-15T21:48:01.999 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.737e-06 2024-09-15T21:48:01.999 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:01.999 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:01.999 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:01.999 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416605+0000", 2024-09-15T21:48:02.000 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00065721699999999998 2024-09-15T21:48:02.000 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.000 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.000 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.000 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416620+0000", 2024-09-15T21:48:02.000 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4564e-05 2024-09-15T21:48:02.000 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.000 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.000 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.000 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421977+0000", 2024-09-15T21:48:02.000 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053572369999999999 2024-09-15T21:48:02.000 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.000 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.001 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.001 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421999+0000", 2024-09-15T21:48:02.001 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2024000000000001e-05 2024-09-15T21:48:02.001 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.001 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.001 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.001 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.001 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.001 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:11 2.10 e22/19 2:0bd6d154:::602.00000000:head v 22'6, mlcod=0'0)", 2024-09-15T21:48:02.001 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:48:02.001 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 93.550878014000006, 2024-09-15T21:48:02.001 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058922669999999996, 2024-09-15T21:48:02.001 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.002 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.002 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.002 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.002 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.002 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.002 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:48:02.002 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.002 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.002 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.002 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.002 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:48:02.002 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.002 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.003 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.003 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.003 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416136+0000", 2024-09-15T21:48:02.003 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6499999999999999e-07 2024-09-15T21:48:02.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.003 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.003 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.003 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416142+0000", 2024-09-15T21:48:02.003 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3929999999999999e-06 2024-09-15T21:48:02.003 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.003 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.003 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.003 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416151+0000", 2024-09-15T21:48:02.003 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5840000000000001e-06 2024-09-15T21:48:02.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.004 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.004 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.004 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416157+0000", 2024-09-15T21:48:02.004 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7590000000000003e-06 2024-09-15T21:48:02.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.004 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.004 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.004 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416767+0000", 2024-09-15T21:48:02.004 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00061034900000000005 2024-09-15T21:48:02.004 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.004 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.004 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.004 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416782+0000", 2024-09-15T21:48:02.005 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.483e-05 2024-09-15T21:48:02.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.005 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.005 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.005 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422002+0000", 2024-09-15T21:48:02.005 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052202639999999996 2024-09-15T21:48:02.005 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.005 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.005 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.005 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422028+0000", 2024-09-15T21:48:02.005 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5423e-05 2024-09-15T21:48:02.005 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.005 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.005 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.006 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.006 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.006 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:12 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'7, mlcod=0'0)", 2024-09-15T21:48:02.006 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:48:02.008 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 93.550471513000005, 2024-09-15T21:48:02.008 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055107769999999997, 2024-09-15T21:48:02.008 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.008 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.008 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.008 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.008 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.008 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:48:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416542+0000", 2024-09-15T21:48:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416543+0000", 2024-09-15T21:48:02.009 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5400000000000001e-07 2024-09-15T21:48:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416548+0000", 2024-09-15T21:48:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8659999999999997e-06 2024-09-15T21:48:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416555+0000", 2024-09-15T21:48:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7190000000000001e-06 2024-09-15T21:48:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.010 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416560+0000", 2024-09-15T21:48:02.011 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3270000000000002e-06 2024-09-15T21:48:02.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.011 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.011 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.011 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416916+0000", 2024-09-15T21:48:02.011 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00035661200000000002 2024-09-15T21:48:02.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.011 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.011 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.011 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416932+0000", 2024-09-15T21:48:02.012 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5824e-05 2024-09-15T21:48:02.012 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.012 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.012 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.012 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422030+0000", 2024-09-15T21:48:02.012 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050975229999999996 2024-09-15T21:48:02.012 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.012 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.012 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.012 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422053+0000", 2024-09-15T21:48:02.012 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3252e-05 2024-09-15T21:48:02.012 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.012 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.012 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.013 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.013 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:13 2.10 e22/19 2:0d89b25e:::603.00000000:head v 22'8, mlcod=0'0)", 2024-09-15T21:48:02.013 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:48:02.013 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 93.550073014000006, 2024-09-15T21:48:02.013 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051422289999999999, 2024-09-15T21:48:02.013 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.013 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.013 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.013 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.013 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.013 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.013 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:48:02.014 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.014 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.014 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.014 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:48:02.014 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.014 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.014 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.014 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416941+0000", 2024-09-15T21:48:02.014 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2699999999999999e-07 2024-09-15T21:48:02.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.014 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.014 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.015 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416946+0000", 2024-09-15T21:48:02.015 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1390000000000003e-06 2024-09-15T21:48:02.015 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.015 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.015 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.015 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416957+0000", 2024-09-15T21:48:02.015 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.044e-05 2024-09-15T21:48:02.015 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.015 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.015 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.015 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416963+0000", 2024-09-15T21:48:02.015 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5320000000000003e-06 2024-09-15T21:48:02.015 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.015 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.016 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.016 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417090+0000", 2024-09-15T21:48:02.016 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126549 2024-09-15T21:48:02.016 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.016 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.016 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.016 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417114+0000", 2024-09-15T21:48:02.016 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3736000000000001e-05 2024-09-15T21:48:02.016 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.016 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.017 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.017 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422055+0000", 2024-09-15T21:48:02.017 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049410410000000002 2024-09-15T21:48:02.017 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.017 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.017 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.017 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422083+0000", 2024-09-15T21:48:02.017 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8164999999999999e-05 2024-09-15T21:48:02.017 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.017 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.018 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.018 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.018 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.018 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:20 2.9 2:92e99a1d:::612.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.018 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:48:02.018 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 87.505581042000003, 2024-09-15T21:48:02.018 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007521186, 2024-09-15T21:48:02.018 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.018 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.018 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.018 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.019 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.019 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.019 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:48:02.019 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.019 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.019 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.019 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.019 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:48:02.019 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.019 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.019 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.019 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.019 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461433+0000", 2024-09-15T21:48:02.019 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9999999999999998e-07 2024-09-15T21:48:02.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.020 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.020 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.020 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461443+0000", 2024-09-15T21:48:02.020 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0448000000000001e-05 2024-09-15T21:48:02.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.020 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.020 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.020 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462146+0000", 2024-09-15T21:48:02.020 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00070223499999999995 2024-09-15T21:48:02.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.020 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.020 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.021 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462154+0000", 2024-09-15T21:48:02.021 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9060000000000002e-06 2024-09-15T21:48:02.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.021 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.021 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.021 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462254+0000", 2024-09-15T21:48:02.021 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9897999999999998e-05 2024-09-15T21:48:02.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.021 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.021 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.021 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462330+0000", 2024-09-15T21:48:02.021 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6551000000000006e-05 2024-09-15T21:48:02.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.022 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.022 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:02.022 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462454+0000", 2024-09-15T21:48:02.022 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000124213 2024-09-15T21:48:02.022 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.022 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.022 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.022 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463681+0000", 2024-09-15T21:48:02.022 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012265729999999999 2024-09-15T21:48:02.022 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.022 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.023 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.023 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468817+0000", 2024-09-15T21:48:02.023 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051361430000000001 2024-09-15T21:48:02.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.023 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.023 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.023 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468892+0000", 2024-09-15T21:48:02.023 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5480000000000002e-05 2024-09-15T21:48:02.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.023 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.023 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.023 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.468954+0000", 2024-09-15T21:48:02.024 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1339000000000003e-05 2024-09-15T21:48:02.024 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.024 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.024 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.024 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.024 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.024 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:27 2.9 2:91bd965c:::501.00000000:head [writefull 0~90] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.024 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:48:02.024 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 87.505564426999996, 2024-09-15T21:48:02.024 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077296149999999996, 2024-09-15T21:48:02.024 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.024 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.024 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.024 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.025 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.025 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.025 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:48:02.025 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.025 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.025 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.025 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461449+0000", 2024-09-15T21:48:02.025 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.025 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.025 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.025 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461450+0000", 2024-09-15T21:48:02.026 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0499999999999999e-07 2024-09-15T21:48:02.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.026 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.026 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.026 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461452+0000", 2024-09-15T21:48:02.026 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.007e-06 2024-09-15T21:48:02.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.026 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.026 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.026 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462209+0000", 2024-09-15T21:48:02.026 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075781000000000004 2024-09-15T21:48:02.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.026 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.026 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.027 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462214+0000", 2024-09-15T21:48:02.027 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1420000000000001e-06 2024-09-15T21:48:02.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.027 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.027 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.027 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462681+0000", 2024-09-15T21:48:02.027 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046634200000000002 2024-09-15T21:48:02.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.027 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.027 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.027 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462772+0000", 2024-09-15T21:48:02.027 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1414000000000003e-05 2024-09-15T21:48:02.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.027 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.028 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:02.028 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462871+0000", 2024-09-15T21:48:02.028 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8455000000000005e-05 2024-09-15T21:48:02.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.028 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.028 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.028 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464138+0000", 2024-09-15T21:48:02.028 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0012673910000000001 2024-09-15T21:48:02.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.028 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.028 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.028 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469079+0000", 2024-09-15T21:48:02.029 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049407799999999997 2024-09-15T21:48:02.029 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.029 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.029 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.029 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469141+0000", 2024-09-15T21:48:02.029 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2169999999999996e-05 2024-09-15T21:48:02.029 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.029 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.029 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.029 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.469179+0000", 2024-09-15T21:48:02.029 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7799000000000002e-05 2024-09-15T21:48:02.029 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.029 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.029 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.030 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.030 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:9 2.6 2:62978f0a:::617.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.030 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:48:02.030 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 81.491448618999996, 2024-09-15T21:48:02.030 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004990497, 2024-09-15T21:48:02.030 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.030 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.030 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.030 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.030 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.030 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.030 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:48:02.030 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.031 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.031 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.031 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475565+0000", 2024-09-15T21:48:02.031 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.031 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.031 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.031 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475567+0000", 2024-09-15T21:48:02.031 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4729999999999999e-06 2024-09-15T21:48:02.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.031 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.031 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.031 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475576+0000", 2024-09-15T21:48:02.032 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1749999999999994e-06 2024-09-15T21:48:02.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.032 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.032 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.032 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475619+0000", 2024-09-15T21:48:02.032 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3557000000000003e-05 2024-09-15T21:48:02.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.032 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.032 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.032 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475631+0000", 2024-09-15T21:48:02.032 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1739999999999999e-05 2024-09-15T21:48:02.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.032 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.033 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.033 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475770+0000", 2024-09-15T21:48:02.033 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013950599999999999 2024-09-15T21:48:02.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.033 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.033 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.033 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475887+0000", 2024-09-15T21:48:02.033 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011616 2024-09-15T21:48:02.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.033 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.033 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:48:02.033 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476018+0000", 2024-09-15T21:48:02.033 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013119499999999999 2024-09-15T21:48:02.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.034 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.034 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478442+0000", 2024-09-15T21:48:02.034 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024240870000000001 2024-09-15T21:48:02.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.034 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.034 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480457+0000", 2024-09-15T21:48:02.034 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020151779999999998 2024-09-15T21:48:02.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.034 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.034 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.034 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480511+0000", 2024-09-15T21:48:02.035 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3582000000000003e-05 2024-09-15T21:48:02.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.035 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.035 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.035 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480556+0000", 2024-09-15T21:48:02.035 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4844e-05 2024-09-15T21:48:02.035 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.035 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.035 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.035 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.035 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:10 2.6 2:62978f0a:::617.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.035 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:48:02.035 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 81.491434300999998, 2024-09-15T21:48:02.036 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051124810000000003, 2024-09-15T21:48:02.036 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.036 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.036 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.036 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.036 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.036 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.036 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:48:02.036 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.036 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.036 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.037 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.037 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475579+0000", 2024-09-15T21:48:02.037 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.037 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.037 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.037 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475580+0000", 2024-09-15T21:48:02.037 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7800000000000002e-07 2024-09-15T21:48:02.037 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.038 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.038 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.038 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475595+0000", 2024-09-15T21:48:02.038 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4919000000000001e-05 2024-09-15T21:48:02.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.038 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.038 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.038 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475662+0000", 2024-09-15T21:48:02.038 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7769999999999997e-05 2024-09-15T21:48:02.038 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.038 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.038 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.038 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475669+0000", 2024-09-15T21:48:02.039 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.703e-06 2024-09-15T21:48:02.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.039 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.039 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.039 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476781+0000", 2024-09-15T21:48:02.039 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011118300000000001 2024-09-15T21:48:02.039 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.039 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.039 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.039 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476863+0000", 2024-09-15T21:48:02.039 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2208999999999998e-05 2024-09-15T21:48:02.040 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.040 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.040 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:48:02.040 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477040+0000", 2024-09-15T21:48:02.040 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000176424 2024-09-15T21:48:02.040 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.040 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.040 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.041 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480154+0000", 2024-09-15T21:48:02.041 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031147589999999999 2024-09-15T21:48:02.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.041 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.041 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.041 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480596+0000", 2024-09-15T21:48:02.041 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00044172699999999999 2024-09-15T21:48:02.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.041 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.041 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.041 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480652+0000", 2024-09-15T21:48:02.041 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6350000000000001e-05 2024-09-15T21:48:02.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.042 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.042 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.042 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480692+0000", 2024-09-15T21:48:02.042 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9412000000000001e-05 2024-09-15T21:48:02.042 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.042 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.042 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.042 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.042 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.042 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:6 2.9 2:941d59bc:::61f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.042 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:48:02.042 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 76.422223768999999, 2024-09-15T21:48:02.042 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057427529999999997, 2024-09-15T21:48:02.043 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.043 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.043 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.043 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.043 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.043 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.043 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:48:02.043 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.043 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.043 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.043 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:48:02.043 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.043 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.044 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.044 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.044 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544790+0000", 2024-09-15T21:48:02.044 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.41e-07 2024-09-15T21:48:02.044 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.044 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.044 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.044 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544792+0000", 2024-09-15T21:48:02.044 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9929999999999998e-06 2024-09-15T21:48:02.044 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.044 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.044 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.044 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544885+0000", 2024-09-15T21:48:02.044 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2825000000000007e-05 2024-09-15T21:48:02.045 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.045 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.045 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.045 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544891+0000", 2024-09-15T21:48:02.045 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.93e-06 2024-09-15T21:48:02.045 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.045 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.045 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.045 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546766+0000", 2024-09-15T21:48:02.045 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018747569999999999 2024-09-15T21:48:02.045 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.045 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.045 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.046 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546864+0000", 2024-09-15T21:48:02.046 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8603000000000004e-05 2024-09-15T21:48:02.046 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.046 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.046 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:02.046 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547028+0000", 2024-09-15T21:48:02.046 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000163461 2024-09-15T21:48:02.046 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.046 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.046 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.046 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549499+0000", 2024-09-15T21:48:02.046 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0024711749999999999 2024-09-15T21:48:02.046 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.046 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.047 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.047 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550417+0000", 2024-09-15T21:48:02.047 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00091817199999999998 2024-09-15T21:48:02.047 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.047 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.047 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.047 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550476+0000", 2024-09-15T21:48:02.047 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8635999999999999e-05 2024-09-15T21:48:02.047 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.047 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.047 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.047 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550533+0000", 2024-09-15T21:48:02.047 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6959999999999997e-05 2024-09-15T21:48:02.048 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.048 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.048 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.048 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.048 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:8 2.6 2:6642f7a8:::620.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.048 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:48:02.048 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 76.422209304000006, 2024-09-15T21:48:02.048 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004917121, 2024-09-15T21:48:02.048 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.048 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.048 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.048 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.049 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.049 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.049 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:48:02.049 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.049 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.049 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.049 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.049 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544804+0000", 2024-09-15T21:48:02.049 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.050 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.050 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.050 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.050 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544805+0000", 2024-09-15T21:48:02.050 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4600000000000001e-07 2024-09-15T21:48:02.050 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.050 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.050 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.050 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544817+0000", 2024-09-15T21:48:02.050 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2023e-05 2024-09-15T21:48:02.050 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.051 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.051 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.051 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545000+0000", 2024-09-15T21:48:02.051 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018384400000000001 2024-09-15T21:48:02.051 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.051 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.051 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.051 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545007+0000", 2024-09-15T21:48:02.051 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1249999999999998e-06 2024-09-15T21:48:02.051 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.051 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.051 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.051 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545670+0000", 2024-09-15T21:48:02.051 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00066324999999999999 2024-09-15T21:48:02.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.052 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.052 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.052 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545749+0000", 2024-09-15T21:48:02.052 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9239000000000005e-05 2024-09-15T21:48:02.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.052 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.052 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:48:02.052 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545885+0000", 2024-09-15T21:48:02.052 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013586199999999999 2024-09-15T21:48:02.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.052 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.052 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.053 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547944+0000", 2024-09-15T21:48:02.053 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0020594720000000001 2024-09-15T21:48:02.053 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.053 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.053 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.053 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549630+0000", 2024-09-15T21:48:02.053 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001685831 2024-09-15T21:48:02.053 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.053 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.053 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.053 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549681+0000", 2024-09-15T21:48:02.053 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1187000000000003e-05 2024-09-15T21:48:02.053 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.054 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.054 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.054 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549721+0000", 2024-09-15T21:48:02.054 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0042e-05 2024-09-15T21:48:02.054 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.054 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.054 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.054 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.054 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.054 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:21 2.6 2:62eebe7c:::627.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.054 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:48:02.054 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 76.422195556000005, 2024-09-15T21:48:02.054 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050615549999999997, 2024-09-15T21:48:02.055 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.055 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.055 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.055 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.055 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.055 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.055 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:48:02.055 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.055 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.055 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.055 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.056 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:48:02.056 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.056 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.056 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.056 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.056 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544818+0000", 2024-09-15T21:48:02.056 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:48:02.056 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.056 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.056 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.056 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544821+0000", 2024-09-15T21:48:02.056 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4399999999999999e-06 2024-09-15T21:48:02.056 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.057 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.057 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.057 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545025+0000", 2024-09-15T21:48:02.057 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020379400000000001 2024-09-15T21:48:02.057 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.057 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.057 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.057 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545030+0000", 2024-09-15T21:48:02.057 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3109999999999996e-06 2024-09-15T21:48:02.057 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.057 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.057 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.057 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546196+0000", 2024-09-15T21:48:02.058 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001166261 2024-09-15T21:48:02.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.058 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.058 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.058 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546307+0000", 2024-09-15T21:48:02.058 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000110641 2024-09-15T21:48:02.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.058 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.058 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:48:02.058 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546422+0000", 2024-09-15T21:48:02.058 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000115399 2024-09-15T21:48:02.058 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.058 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.058 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.059 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547973+0000", 2024-09-15T21:48:02.059 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001551228 2024-09-15T21:48:02.059 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.059 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.059 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.059 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549815+0000", 2024-09-15T21:48:02.059 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001841685 2024-09-15T21:48:02.059 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.059 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.059 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.059 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549856+0000", 2024-09-15T21:48:02.059 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.091e-05 2024-09-15T21:48:02.059 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.060 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.060 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.060 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549880+0000", 2024-09-15T21:48:02.060 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3580000000000001e-05 2024-09-15T21:48:02.060 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.060 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.060 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.060 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.060 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.060 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:22 2.6 2:62eebe7c:::627.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.060 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:48:02.060 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 76.422191548000001, 2024-09-15T21:48:02.060 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056931389999999998, 2024-09-15T21:48:02.061 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.061 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.061 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.061 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.061 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.061 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.061 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:48:02.061 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.061 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.061 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.061 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.061 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:48:02.061 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.061 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.062 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.062 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.062 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544822+0000", 2024-09-15T21:48:02.062 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:48:02.062 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.062 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.062 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.062 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544828+0000", 2024-09-15T21:48:02.062 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4369999999999998e-06 2024-09-15T21:48:02.062 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.062 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.062 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.062 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545041+0000", 2024-09-15T21:48:02.063 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000212695 2024-09-15T21:48:02.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.063 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.063 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.063 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545058+0000", 2024-09-15T21:48:02.063 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7717999999999999e-05 2024-09-15T21:48:02.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.063 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.063 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.063 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.546958+0000", 2024-09-15T21:48:02.063 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018995539999999999 2024-09-15T21:48:02.063 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.063 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.063 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.064 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547057+0000", 2024-09-15T21:48:02.064 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9370999999999994e-05 2024-09-15T21:48:02.064 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.064 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.064 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:48:02.064 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547579+0000", 2024-09-15T21:48:02.064 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052198200000000004 2024-09-15T21:48:02.064 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.064 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.064 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.064 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549929+0000", 2024-09-15T21:48:02.064 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0023494649999999998 2024-09-15T21:48:02.064 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.064 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.065 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.065 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550401+0000", 2024-09-15T21:48:02.065 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00047277300000000001 2024-09-15T21:48:02.065 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.065 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.065 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.065 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550458+0000", 2024-09-15T21:48:02.065 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6756999999999999e-05 2024-09-15T21:48:02.065 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.065 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.065 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.065 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550515+0000", 2024-09-15T21:48:02.065 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.7093999999999999e-05 2024-09-15T21:48:02.066 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.066 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.066 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.066 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.066 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.066 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:26 2.15 2:afbd2f49:::103.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.066 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:48:02.066 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 76.422184891000001, 2024-09-15T21:48:02.066 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0050112029999999997, 2024-09-15T21:48:02.066 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.066 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.066 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.066 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.066 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.067 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.067 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:48:02.067 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.067 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.067 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.067 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.067 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:48:02.067 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.067 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.067 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.067 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.067 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.544829+0000", 2024-09-15T21:48:02.067 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.36e-07 2024-09-15T21:48:02.068 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.068 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.068 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.068 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545086+0000", 2024-09-15T21:48:02.068 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025704699999999999 2024-09-15T21:48:02.068 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.068 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.068 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.068 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545119+0000", 2024-09-15T21:48:02.068 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2734999999999997e-05 2024-09-15T21:48:02.068 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.068 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.068 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.069 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545126+0000", 2024-09-15T21:48:02.069 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5599999999999996e-06 2024-09-15T21:48:02.069 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.069 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.069 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.069 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545253+0000", 2024-09-15T21:48:02.069 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012644500000000001 2024-09-15T21:48:02.069 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.069 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.069 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.069 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545366+0000", 2024-09-15T21:48:02.069 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011349999999999999 2024-09-15T21:48:02.070 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.070 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.070 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 5", 2024-09-15T21:48:02.070 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545673+0000", 2024-09-15T21:48:02.070 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030643599999999999 2024-09-15T21:48:02.070 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.070 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.070 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.070 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.547924+0000", 2024-09-15T21:48:02.070 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0022516189999999998 2024-09-15T21:48:02.070 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.070 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.070 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.070 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549746+0000", 2024-09-15T21:48:02.071 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0018217629999999999 2024-09-15T21:48:02.071 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.071 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.071 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.071 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549800+0000", 2024-09-15T21:48:02.071 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3412999999999998e-05 2024-09-15T21:48:02.071 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.071 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.071 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.071 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.549840+0000", 2024-09-15T21:48:02.071 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0448999999999998e-05 2024-09-15T21:48:02.071 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.071 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.072 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.072 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.072 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.072 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.3.14:30 2.4 2:218afd09:::203.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.072 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:48:02.072 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 76.421924062000002, 2024-09-15T21:48:02.072 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055481230000000003, 2024-09-15T21:48:02.072 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.072 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.072 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:48:02.072 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.072 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.072 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.073 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:48:02.073 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.073 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.073 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.073 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.073 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545090+0000", 2024-09-15T21:48:02.073 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.073 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.073 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.073 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.073 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545091+0000", 2024-09-15T21:48:02.073 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7000000000000003e-07 2024-09-15T21:48:02.073 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.073 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.074 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.074 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545094+0000", 2024-09-15T21:48:02.074 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9569999999999998e-06 2024-09-15T21:48:02.074 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.074 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.074 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.074 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545156+0000", 2024-09-15T21:48:02.074 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2277e-05 2024-09-15T21:48:02.074 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.074 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.074 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.074 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545162+0000", 2024-09-15T21:48:02.074 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9120000000000003e-06 2024-09-15T21:48:02.074 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.075 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.075 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.075 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545262+0000", 2024-09-15T21:48:02.075 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010001900000000001 2024-09-15T21:48:02.075 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.075 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.075 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.075 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.545388+0000", 2024-09-15T21:48:02.075 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000126243 2024-09-15T21:48:02.075 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.075 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.075 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.075 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:45.550638+0000", 2024-09-15T21:48:02.075 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052497450000000001 2024-09-15T21:48:02.076 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.076 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.076 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.076 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.076 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.076 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:10 2.9 2:954a9a2b:::62b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.076 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:48:02.076 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 70.402240602000006, 2024-09-15T21:48:02.076 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0048438860000000004, 2024-09-15T21:48:02.076 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.076 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.076 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.076 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.077 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.077 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.077 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:48:02.077 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.077 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.077 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.077 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.077 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:48:02.077 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.077 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.077 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.077 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.077 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564773+0000", 2024-09-15T21:48:02.078 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0400000000000002e-07 2024-09-15T21:48:02.078 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.078 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.078 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.078 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564786+0000", 2024-09-15T21:48:02.078 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2412000000000001e-05 2024-09-15T21:48:02.078 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.078 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.078 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.078 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564991+0000", 2024-09-15T21:48:02.078 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00020483900000000001 2024-09-15T21:48:02.078 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.078 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.078 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.079 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564998+0000", 2024-09-15T21:48:02.079 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2459999999999997e-06 2024-09-15T21:48:02.079 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.079 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.079 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.079 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565422+0000", 2024-09-15T21:48:02.079 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00042420100000000003 2024-09-15T21:48:02.079 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.079 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.079 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.079 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565515+0000", 2024-09-15T21:48:02.079 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.2449999999999997e-05 2024-09-15T21:48:02.079 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.079 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.080 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:02.080 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565642+0000", 2024-09-15T21:48:02.080 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012743699999999999 2024-09-15T21:48:02.080 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.080 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.080 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.080 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566656+0000", 2024-09-15T21:48:02.080 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0010138580000000001 2024-09-15T21:48:02.080 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.080 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.080 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.080 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569552+0000", 2024-09-15T21:48:02.080 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0028966819999999998 2024-09-15T21:48:02.080 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.081 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.081 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.081 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569594+0000", 2024-09-15T21:48:02.081 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1235000000000001e-05 2024-09-15T21:48:02.081 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.081 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.081 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.081 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569617+0000", 2024-09-15T21:48:02.081 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3221999999999999e-05 2024-09-15T21:48:02.081 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.081 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.081 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.081 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.082 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.082 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:02.083 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:48:02.083 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:48:02.080+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:48:02.184 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:48:02.181+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:48:02.231 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:02.231 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:48:02.231 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:48:02.231 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:02.241 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:48:02.285 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:48:02.283+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:48:02.386 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:48:02.383+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:48:02.386 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:02.387 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:48:02.387 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:48:02.387 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:48:02.387 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:02.396 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:48:02.486 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:48:02.485+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:48:02.541 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:48:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:48:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:48:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:6 2.10 2:0d82a743:::600.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:48:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 94.125712974999999, 2024-09-15T21:48:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00598016, 2024-09-15T21:48:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.542 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:48:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413336+0000", 2024-09-15T21:48:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.543 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413339+0000", 2024-09-15T21:48:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6279999999999998e-06 2024-09-15T21:48:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413354+0000", 2024-09-15T21:48:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4877e-05 2024-09-15T21:48:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413418+0000", 2024-09-15T21:48:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4274000000000002e-05 2024-09-15T21:48:02.544 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.545 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.545 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.545 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413428+0000", 2024-09-15T21:48:02.545 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5899999999999997e-06 2024-09-15T21:48:02.545 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.545 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.545 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.545 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413563+0000", 2024-09-15T21:48:02.545 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013521900000000001 2024-09-15T21:48:02.545 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.545 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.545 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.545 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413678+0000", 2024-09-15T21:48:02.546 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000114904 2024-09-15T21:48:02.546 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.546 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.546 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:48:02.546 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413789+0000", 2024-09-15T21:48:02.546 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011069300000000001 2024-09-15T21:48:02.546 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.546 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.546 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.546 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417603+0000", 2024-09-15T21:48:02.546 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038138460000000001 2024-09-15T21:48:02.546 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.546 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.546 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.547 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419220+0000", 2024-09-15T21:48:02.547 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001617678 2024-09-15T21:48:02.547 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.547 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.547 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.547 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419275+0000", 2024-09-15T21:48:02.547 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4391999999999997e-05 2024-09-15T21:48:02.547 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.547 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.547 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.547 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.419316+0000", 2024-09-15T21:48:02.547 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1059000000000001e-05 2024-09-15T21:48:02.547 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.548 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.548 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.548 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.548 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.548 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:7 2.10 2:0d82a743:::600.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:02.548 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:48:02.548 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 94.125690847000001, 2024-09-15T21:48:02.548 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0087926470000000007, 2024-09-15T21:48:02.548 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.548 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.548 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.549 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.549 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.549 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.549 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:48:02.549 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.549 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.549 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.549 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.549 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:48:02.549 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.549 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.550 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.550 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.550 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413358+0000", 2024-09-15T21:48:02.550 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4099999999999999e-07 2024-09-15T21:48:02.550 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.550 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.550 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.550 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413361+0000", 2024-09-15T21:48:02.550 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6189999999999998e-06 2024-09-15T21:48:02.550 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.550 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.550 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.550 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413461+0000", 2024-09-15T21:48:02.551 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000100053 2024-09-15T21:48:02.551 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.551 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.551 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.551 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413466+0000", 2024-09-15T21:48:02.551 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3850000000000001e-06 2024-09-15T21:48:02.551 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.551 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.551 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.551 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414073+0000", 2024-09-15T21:48:02.551 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000606724 2024-09-15T21:48:02.551 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.551 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.551 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.552 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414143+0000", 2024-09-15T21:48:02.552 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9727000000000005e-05 2024-09-15T21:48:02.552 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.552 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.552 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:48:02.552 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414259+0000", 2024-09-15T21:48:02.552 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000116178 2024-09-15T21:48:02.552 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.552 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.552 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.552 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417610+0000", 2024-09-15T21:48:02.552 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003351176 2024-09-15T21:48:02.552 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.553 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.553 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.553 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422058+0000", 2024-09-15T21:48:02.553 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.004447599 2024-09-15T21:48:02.553 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.553 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.553 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.553 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422111+0000", 2024-09-15T21:48:02.553 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3606999999999997e-05 2024-09-15T21:48:02.553 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.553 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.553 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.553 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422150+0000", 2024-09-15T21:48:02.553 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9137999999999997e-05 2024-09-15T21:48:02.554 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.554 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.554 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.554 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.554 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.554 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:8 2.10 2:09401a81:::601.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:02.554 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:48:02.554 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 94.125686416999997, 2024-09-15T21:48:02.554 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0089009769999999992, 2024-09-15T21:48:02.554 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.554 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.554 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.554 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.554 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.555 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.555 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:48:02.555 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.555 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.555 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.555 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.555 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413362+0000", 2024-09-15T21:48:02.555 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.555 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.555 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.555 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.555 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413363+0000", 2024-09-15T21:48:02.555 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3999999999999997e-07 2024-09-15T21:48:02.555 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.555 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.556 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.556 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413365+0000", 2024-09-15T21:48:02.556 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.216e-06 2024-09-15T21:48:02.556 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.556 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.556 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.556 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413481+0000", 2024-09-15T21:48:02.556 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000116311 2024-09-15T21:48:02.556 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.556 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.556 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.556 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413488+0000", 2024-09-15T21:48:02.556 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0110000000000003e-06 2024-09-15T21:48:02.557 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.557 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.557 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.557 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414452+0000", 2024-09-15T21:48:02.557 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00096385399999999999 2024-09-15T21:48:02.557 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.557 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.557 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.557 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414545+0000", 2024-09-15T21:48:02.557 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.3024999999999998e-05 2024-09-15T21:48:02.557 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.557 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.557 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:48:02.557 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414644+0000", 2024-09-15T21:48:02.558 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9098000000000006e-05 2024-09-15T21:48:02.558 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.558 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.558 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.558 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417667+0000", 2024-09-15T21:48:02.558 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0030229409999999999 2024-09-15T21:48:02.558 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.558 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.558 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.558 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422200+0000", 2024-09-15T21:48:02.558 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045327880000000003 2024-09-15T21:48:02.558 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.558 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.558 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.559 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422228+0000", 2024-09-15T21:48:02.559 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8109000000000001e-05 2024-09-15T21:48:02.559 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.559 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.559 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.559 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422263+0000", 2024-09-15T21:48:02.559 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5284000000000003e-05 2024-09-15T21:48:02.559 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.559 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.559 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.559 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.559 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.559 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:9 2.10 2:09401a81:::601.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:02.560 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:48:02.560 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 94.125682831000006, 2024-09-15T21:48:02.560 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0089846820000000008, 2024-09-15T21:48:02.560 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.560 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.560 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.560 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.560 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.560 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.560 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:48:02.560 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.560 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.560 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.560 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.561 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:48:02.561 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.561 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.561 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.561 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.561 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413366+0000", 2024-09-15T21:48:02.561 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7599999999999998e-07 2024-09-15T21:48:02.561 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.561 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.561 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.561 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413368+0000", 2024-09-15T21:48:02.561 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.058e-06 2024-09-15T21:48:02.561 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.561 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.562 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.562 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413832+0000", 2024-09-15T21:48:02.562 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00046398700000000001 2024-09-15T21:48:02.562 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.562 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.562 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.562 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413841+0000", 2024-09-15T21:48:02.562 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1710000000000008e-06 2024-09-15T21:48:02.562 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.562 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.562 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.562 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414869+0000", 2024-09-15T21:48:02.562 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001027584 2024-09-15T21:48:02.562 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.563 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.563 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.563 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414946+0000", 2024-09-15T21:48:02.563 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7177999999999999e-05 2024-09-15T21:48:02.563 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.563 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.563 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:48:02.563 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415054+0000", 2024-09-15T21:48:02.563 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010807 2024-09-15T21:48:02.563 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.563 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.563 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.563 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417672+0000", 2024-09-15T21:48:02.563 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002618015 2024-09-15T21:48:02.563 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.564 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.564 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.564 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422295+0000", 2024-09-15T21:48:02.564 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0046233300000000001 2024-09-15T21:48:02.564 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.564 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.564 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.564 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422330+0000", 2024-09-15T21:48:02.564 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4476e-05 2024-09-15T21:48:02.564 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.564 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.564 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.564 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422351+0000", 2024-09-15T21:48:02.565 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0537e-05 2024-09-15T21:48:02.565 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.565 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.565 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.565 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.565 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.565 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:10 2.10 2:0bd6d154:::602.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:02.565 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:48:02.565 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 94.125679470999998, 2024-09-15T21:48:02.565 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0090873700000000009, 2024-09-15T21:48:02.565 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.565 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.565 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.565 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.566 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.566 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.566 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:48:02.566 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.566 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.566 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.566 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.566 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:48:02.566 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.566 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.566 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.566 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.566 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413369+0000", 2024-09-15T21:48:02.566 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6100000000000002e-07 2024-09-15T21:48:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413371+0000", 2024-09-15T21:48:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.764e-06 2024-09-15T21:48:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413859+0000", 2024-09-15T21:48:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00048751900000000001 2024-09-15T21:48:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.567 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413864+0000", 2024-09-15T21:48:02.568 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6200000000000004e-06 2024-09-15T21:48:02.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.568 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.568 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.568 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415253+0000", 2024-09-15T21:48:02.568 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0013884050000000001 2024-09-15T21:48:02.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.568 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.568 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.568 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415364+0000", 2024-09-15T21:48:02.568 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000111119 2024-09-15T21:48:02.568 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.568 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.568 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:48:02.569 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415453+0000", 2024-09-15T21:48:02.569 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.8641999999999996e-05 2024-09-15T21:48:02.569 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.569 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.569 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.569 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417676+0000", 2024-09-15T21:48:02.569 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00222368 2024-09-15T21:48:02.569 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.569 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.569 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.569 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422395+0000", 2024-09-15T21:48:02.569 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047191150000000003 2024-09-15T21:48:02.569 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.570 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.570 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.570 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422434+0000", 2024-09-15T21:48:02.570 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8525999999999997e-05 2024-09-15T21:48:02.570 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.570 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.570 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.570 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422457+0000", 2024-09-15T21:48:02.570 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2719000000000001e-05 2024-09-15T21:48:02.570 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.570 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.570 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.570 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.571 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.571 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:11 2.10 2:0bd6d154:::602.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:02.571 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:48:02.571 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 94.125676448999997, 2024-09-15T21:48:02.571 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0092139749999999992, 2024-09-15T21:48:02.571 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.571 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.571 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.571 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.571 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.571 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.571 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:48:02.571 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.571 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.572 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.572 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.572 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:48:02.572 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.572 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.572 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.572 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.572 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413372+0000", 2024-09-15T21:48:02.572 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3799999999999999e-07 2024-09-15T21:48:02.572 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.572 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.572 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.572 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413375+0000", 2024-09-15T21:48:02.573 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1959999999999998e-06 2024-09-15T21:48:02.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.573 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.573 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.573 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413876+0000", 2024-09-15T21:48:02.573 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00050134500000000005 2024-09-15T21:48:02.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.573 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.573 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.573 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413881+0000", 2024-09-15T21:48:02.573 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.3580000000000002e-06 2024-09-15T21:48:02.573 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.573 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.574 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.574 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415669+0000", 2024-09-15T21:48:02.574 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0017875090000000001 2024-09-15T21:48:02.574 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.574 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.574 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.574 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415737+0000", 2024-09-15T21:48:02.574 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8386000000000006e-05 2024-09-15T21:48:02.574 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.574 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.574 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:48:02.574 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.415857+0000", 2024-09-15T21:48:02.574 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119906 2024-09-15T21:48:02.574 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.575 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.575 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.575 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421848+0000", 2024-09-15T21:48:02.575 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0059903880000000001 2024-09-15T21:48:02.575 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.575 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.575 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.575 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422513+0000", 2024-09-15T21:48:02.575 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00066575599999999999 2024-09-15T21:48:02.575 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.575 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.575 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.575 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422549+0000", 2024-09-15T21:48:02.575 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5719999999999997e-05 2024-09-15T21:48:02.576 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.576 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.576 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.576 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422586+0000", 2024-09-15T21:48:02.576 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7172999999999998e-05 2024-09-15T21:48:02.576 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.576 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.576 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.576 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.576 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.576 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:12 2.10 2:0d89b25e:::603.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:02.576 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:48:02.576 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 94.125670627000005, 2024-09-15T21:48:02.576 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.009324087, 2024-09-15T21:48:02.577 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.577 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.577 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.577 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.577 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.577 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.577 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:48:02.577 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.577 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.577 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.577 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.577 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:48:02.578 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.578 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.578 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.578 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413378+0000", 2024-09-15T21:48:02.578 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8900000000000001e-07 2024-09-15T21:48:02.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.578 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.578 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.578 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413381+0000", 2024-09-15T21:48:02.578 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.1959999999999998e-06 2024-09-15T21:48:02.578 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.578 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413895+0000", 2024-09-15T21:48:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051466000000000001 2024-09-15T21:48:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413900+0000", 2024-09-15T21:48:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6090000000000003e-06 2024-09-15T21:48:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416081+0000", 2024-09-15T21:48:02.579 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021816090000000002 2024-09-15T21:48:02.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.580 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.580 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.580 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416173+0000", 2024-09-15T21:48:02.580 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.1334000000000003e-05 2024-09-15T21:48:02.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.580 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.580 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:48:02.580 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416264+0000", 2024-09-15T21:48:02.580 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0864000000000003e-05 2024-09-15T21:48:02.580 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.580 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.580 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.581 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421854+0000", 2024-09-15T21:48:02.581 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055902240000000004 2024-09-15T21:48:02.581 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.581 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.581 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.581 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422629+0000", 2024-09-15T21:48:02.581 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00077550700000000004 2024-09-15T21:48:02.581 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.581 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.581 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.582 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422678+0000", 2024-09-15T21:48:02.582 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8810999999999997e-05 2024-09-15T21:48:02.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.582 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.582 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.582 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422702+0000", 2024-09-15T21:48:02.582 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.3983999999999999e-05 2024-09-15T21:48:02.582 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.582 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.582 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.582 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.582 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.582 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:13 2.10 2:0d89b25e:::603.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:02.582 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:48:02.583 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 94.125667144999994, 2024-09-15T21:48:02.583 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0094270580000000003, 2024-09-15T21:48:02.583 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.583 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.583 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.583 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.583 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.583 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.583 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:48:02.583 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.583 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.583 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.583 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.584 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:48:02.584 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.584 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.584 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.584 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413382+0000", 2024-09-15T21:48:02.584 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7500000000000001e-07 2024-09-15T21:48:02.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.584 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.584 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.584 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413386+0000", 2024-09-15T21:48:02.584 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5619999999999997e-06 2024-09-15T21:48:02.584 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.585 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.585 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.585 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413918+0000", 2024-09-15T21:48:02.585 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00053162499999999998 2024-09-15T21:48:02.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.585 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.585 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.585 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413923+0000", 2024-09-15T21:48:02.585 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9280000000000001e-06 2024-09-15T21:48:02.585 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.585 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.585 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.585 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416492+0000", 2024-09-15T21:48:02.585 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0025687140000000002 2024-09-15T21:48:02.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.586 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.586 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.586 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416558+0000", 2024-09-15T21:48:02.586 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6699000000000006e-05 2024-09-15T21:48:02.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.586 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.586 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:48:02.586 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.416691+0000", 2024-09-15T21:48:02.586 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013263199999999999 2024-09-15T21:48:02.586 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.586 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.586 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.586 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421879+0000", 2024-09-15T21:48:02.587 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051881089999999998 2024-09-15T21:48:02.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.587 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.587 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.587 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422742+0000", 2024-09-15T21:48:02.587 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000862845 2024-09-15T21:48:02.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.587 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.587 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.587 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422786+0000", 2024-09-15T21:48:02.587 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4332000000000002e-05 2024-09-15T21:48:02.587 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.587 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.587 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.588 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422809+0000", 2024-09-15T21:48:02.588 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2337e-05 2024-09-15T21:48:02.588 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.588 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.588 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.588 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.588 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.588 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:4 2.10 2:0f65c9b8:::60a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.588 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:48:02.588 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 88.083380976000001, 2024-09-15T21:48:02.588 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058710300000000002, 2024-09-15T21:48:02.588 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.588 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.589 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.589 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.589 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.589 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.589 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:48:02.589 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.589 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.589 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.589 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:48:02.589 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.589 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.589 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.590 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.590 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455668+0000", 2024-09-15T21:48:02.590 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0800000000000005e-07 2024-09-15T21:48:02.590 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.590 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.590 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.590 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455672+0000", 2024-09-15T21:48:02.590 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3900000000000002e-06 2024-09-15T21:48:02.590 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.590 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.590 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.590 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456538+0000", 2024-09-15T21:48:02.590 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00086613100000000002 2024-09-15T21:48:02.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.591 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.591 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.591 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456545+0000", 2024-09-15T21:48:02.591 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4830000000000002e-06 2024-09-15T21:48:02.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.591 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.591 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.591 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456594+0000", 2024-09-15T21:48:02.591 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8429e-05 2024-09-15T21:48:02.591 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.591 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.591 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.592 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456675+0000", 2024-09-15T21:48:02.592 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1516999999999999e-05 2024-09-15T21:48:02.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.592 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.592 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:48:02.592 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456772+0000", 2024-09-15T21:48:02.592 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6523999999999999e-05 2024-09-15T21:48:02.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.592 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.592 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.592 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.460299+0000", 2024-09-15T21:48:02.592 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035274859999999998 2024-09-15T21:48:02.592 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.593 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.593 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.593 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461451+0000", 2024-09-15T21:48:02.593 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011518100000000001 2024-09-15T21:48:02.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.593 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.593 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.593 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461498+0000", 2024-09-15T21:48:02.593 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.7406e-05 2024-09-15T21:48:02.593 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.593 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.593 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.593 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.461539+0000", 2024-09-15T21:48:02.593 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0346000000000003e-05 2024-09-15T21:48:02.594 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.594 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.594 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.594 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.594 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.594 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:31 2.1 2:810cebc8:::201.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.594 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:48:02.594 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 88.083375713999999, 2024-09-15T21:48:02.594 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0078158979999999999, 2024-09-15T21:48:02.594 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.594 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.594 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:48:02.595 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.595 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.595 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.595 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:48:02.595 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.595 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.595 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.595 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:48:02.595 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.595 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.595 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.595 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.596 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455673+0000", 2024-09-15T21:48:02.596 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.0499999999999999e-07 2024-09-15T21:48:02.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.596 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.596 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.596 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.455675+0000", 2024-09-15T21:48:02.596 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.376e-06 2024-09-15T21:48:02.596 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.596 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.596 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.596 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456964+0000", 2024-09-15T21:48:02.596 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001289204 2024-09-15T21:48:02.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.597 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.597 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.597 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.456970+0000", 2024-09-15T21:48:02.597 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2129999999999996e-06 2024-09-15T21:48:02.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.597 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.597 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.597 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457032+0000", 2024-09-15T21:48:02.597 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2209e-05 2024-09-15T21:48:02.597 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.597 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.597 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.598 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457134+0000", 2024-09-15T21:48:02.598 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010204000000000001 2024-09-15T21:48:02.598 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.598 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.598 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.598 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463489+0000", 2024-09-15T21:48:02.598 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0063545509999999999 2024-09-15T21:48:02.598 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.598 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.598 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.598 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.598 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.598 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:14 2.1 2:81735765:::619.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.599 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:48:02.599 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 82.063615949999999, 2024-09-15T21:48:02.599 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057594079999999997, 2024-09-15T21:48:02.599 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.599 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.599 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.599 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.599 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.600 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.600 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:48:02.600 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.600 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.600 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.600 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.600 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:48:02.600 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.600 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.600 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.600 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.600 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475433+0000", 2024-09-15T21:48:02.601 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1999999999999999e-07 2024-09-15T21:48:02.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.601 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.601 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.601 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475437+0000", 2024-09-15T21:48:02.601 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5429999999999998e-06 2024-09-15T21:48:02.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.601 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.601 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.601 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475686+0000", 2024-09-15T21:48:02.601 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000248908 2024-09-15T21:48:02.601 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.601 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.601 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.602 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475695+0000", 2024-09-15T21:48:02.602 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.9740000000000008e-06 2024-09-15T21:48:02.602 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.602 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.602 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.602 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476291+0000", 2024-09-15T21:48:02.602 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00059651500000000004 2024-09-15T21:48:02.602 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.602 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.602 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.602 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476366+0000", 2024-09-15T21:48:02.602 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4531999999999997e-05 2024-09-15T21:48:02.603 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.603 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.603 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:48:02.603 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476491+0000", 2024-09-15T21:48:02.603 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012527299999999999 2024-09-15T21:48:02.603 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.603 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.603 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.603 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480600+0000", 2024-09-15T21:48:02.603 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0041085870000000003 2024-09-15T21:48:02.603 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.603 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.603 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.603 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481089+0000", 2024-09-15T21:48:02.604 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000489037 2024-09-15T21:48:02.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.604 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.604 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.604 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481152+0000", 2024-09-15T21:48:02.604 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3136e-05 2024-09-15T21:48:02.604 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.604 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.604 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.604 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481192+0000", 2024-09-15T21:48:02.604 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0182999999999998e-05 2024-09-15T21:48:02.604 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.604 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.605 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.605 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.605 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.605 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:17 2.10 2:0e9da45e:::61b.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.605 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:48:02.605 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 82.063610288999996, 2024-09-15T21:48:02.605 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058311830000000002, 2024-09-15T21:48:02.605 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.605 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.605 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.605 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.605 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.605 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.605 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:48:02.606 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.606 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.606 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.606 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475438+0000", 2024-09-15T21:48:02.606 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.606 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.606 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.606 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475439+0000", 2024-09-15T21:48:02.606 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.1300000000000001e-07 2024-09-15T21:48:02.606 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.606 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.606 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.607 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475441+0000", 2024-09-15T21:48:02.607 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.419e-06 2024-09-15T21:48:02.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.607 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.607 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.607 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:48:02.607 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000269187 2024-09-15T21:48:02.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.607 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.607 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.607 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475715+0000", 2024-09-15T21:48:02.607 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.3660000000000002e-06 2024-09-15T21:48:02.607 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.607 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.608 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.608 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476671+0000", 2024-09-15T21:48:02.608 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000956507 2024-09-15T21:48:02.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.608 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.608 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.608 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477500+0000", 2024-09-15T21:48:02.608 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00082897400000000001 2024-09-15T21:48:02.608 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.608 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.608 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:48:02.608 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477625+0000", 2024-09-15T21:48:02.608 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012513100000000001 2024-09-15T21:48:02.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.609 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.609 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.609 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480520+0000", 2024-09-15T21:48:02.609 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002894715 2024-09-15T21:48:02.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.609 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.609 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.609 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481184+0000", 2024-09-15T21:48:02.609 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000663801 2024-09-15T21:48:02.609 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.609 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.609 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481232+0000", 2024-09-15T21:48:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8457e-05 2024-09-15T21:48:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481270+0000", 2024-09-15T21:48:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7313e-05 2024-09-15T21:48:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.610 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:18 2.10 2:0e9da45e:::61b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:48:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 82.063606413000002, 2024-09-15T21:48:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0075605530000000002, 2024-09-15T21:48:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:48:02.611 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:48:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475442+0000", 2024-09-15T21:48:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.53e-07 2024-09-15T21:48:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.612 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475445+0000", 2024-09-15T21:48:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7360000000000001e-06 2024-09-15T21:48:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475735+0000", 2024-09-15T21:48:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00028975799999999999 2024-09-15T21:48:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475740+0000", 2024-09-15T21:48:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8250000000000004e-06 2024-09-15T21:48:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478479+0000", 2024-09-15T21:48:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002739525 2024-09-15T21:48:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478556+0000", 2024-09-15T21:48:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6213999999999999e-05 2024-09-15T21:48:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:48:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478672+0000", 2024-09-15T21:48:02.614 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011605 2024-09-15T21:48:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481323+0000", 2024-09-15T21:48:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0026518819999999999 2024-09-15T21:48:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482918+0000", 2024-09-15T21:48:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001594105 2024-09-15T21:48:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.615 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482965+0000", 2024-09-15T21:48:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6950000000000003e-05 2024-09-15T21:48:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483003+0000", 2024-09-15T21:48:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8254999999999999e-05 2024-09-15T21:48:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.616 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:1 2.1 2:85bbe569:::202.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:48:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 82.063602356999994, 2024-09-15T21:48:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076582890000000004, 2024-09-15T21:48:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:48:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.617 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.618 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475446+0000", 2024-09-15T21:48:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475447+0000", 2024-09-15T21:48:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.4699999999999998e-07 2024-09-15T21:48:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.620 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475449+0000", 2024-09-15T21:48:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.176e-06 2024-09-15T21:48:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475752+0000", 2024-09-15T21:48:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00030374200000000002 2024-09-15T21:48:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475757+0000", 2024-09-15T21:48:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2209999999999999e-06 2024-09-15T21:48:02.621 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.622 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.622 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.622 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477441+0000", 2024-09-15T21:48:02.622 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016843489999999999 2024-09-15T21:48:02.622 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.622 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.622 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.622 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477567+0000", 2024-09-15T21:48:02.622 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012552099999999999 2024-09-15T21:48:02.622 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.622 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.622 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:48:02.622 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477712+0000", 2024-09-15T21:48:02.623 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000145427 2024-09-15T21:48:02.623 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.623 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.623 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.623 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482935+0000", 2024-09-15T21:48:02.623 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052231200000000004 2024-09-15T21:48:02.623 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.623 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.623 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.623 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483015+0000", 2024-09-15T21:48:02.623 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.9634000000000004e-05 2024-09-15T21:48:02.623 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.623 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.624 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.624 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483065+0000", 2024-09-15T21:48:02.624 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9869e-05 2024-09-15T21:48:02.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.624 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.624 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.624 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483105+0000", 2024-09-15T21:48:02.624 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.9983e-05 2024-09-15T21:48:02.624 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.624 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.624 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.624 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.624 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.625 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:2 2.10 2:0daba267:::402.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.625 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:48:02.625 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 82.063595816000003, 2024-09-15T21:48:02.625 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0076459379999999997, 2024-09-15T21:48:02.625 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.625 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.625 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.625 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.625 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.625 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.625 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:48:02.625 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.625 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.626 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.626 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.626 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:48:02.626 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.626 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.626 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.626 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.626 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475453+0000", 2024-09-15T21:48:02.626 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.58e-07 2024-09-15T21:48:02.626 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.626 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.626 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.626 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475455+0000", 2024-09-15T21:48:02.626 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.075e-06 2024-09-15T21:48:02.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.627 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.627 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.627 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475769+0000", 2024-09-15T21:48:02.627 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00031336999999999999 2024-09-15T21:48:02.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.627 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.627 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.627 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475773+0000", 2024-09-15T21:48:02.627 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8080000000000004e-06 2024-09-15T21:48:02.627 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.627 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.627 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.628 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478950+0000", 2024-09-15T21:48:02.628 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0031763870000000001 2024-09-15T21:48:02.628 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.628 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.628 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.628 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479067+0000", 2024-09-15T21:48:02.628 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000117543 2024-09-15T21:48:02.628 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.628 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.628 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:48:02.628 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479196+0000", 2024-09-15T21:48:02.628 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000128987 2024-09-15T21:48:02.628 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.629 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.629 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.629 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.481366+0000", 2024-09-15T21:48:02.629 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021699089999999998 2024-09-15T21:48:02.629 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.629 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.629 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.629 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483015+0000", 2024-09-15T21:48:02.629 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0016486719999999999 2024-09-15T21:48:02.629 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.629 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.629 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.629 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483061+0000", 2024-09-15T21:48:02.630 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5729999999999998e-05 2024-09-15T21:48:02.630 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.630 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.630 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.630 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483099+0000", 2024-09-15T21:48:02.630 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8099000000000002e-05 2024-09-15T21:48:02.630 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.630 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.630 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.630 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.630 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.630 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:27 2.10 2:091e33df:::502.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.630 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:48:02.630 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 82.063548706999995, 2024-09-15T21:48:02.631 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0077290859999999996, 2024-09-15T21:48:02.631 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.631 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.631 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.631 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.631 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.631 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.631 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:48:02.631 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.631 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.631 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.631 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.631 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475500+0000", 2024-09-15T21:48:02.632 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.632 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.632 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.632 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.632 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475501+0000", 2024-09-15T21:48:02.632 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.1900000000000002e-07 2024-09-15T21:48:02.632 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.632 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.632 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.632 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475504+0000", 2024-09-15T21:48:02.632 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.168e-06 2024-09-15T21:48:02.632 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.632 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.633 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.633 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477037+0000", 2024-09-15T21:48:02.633 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001533202 2024-09-15T21:48:02.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.633 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.633 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.633 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.477050+0000", 2024-09-15T21:48:02.633 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3132999999999999e-05 2024-09-15T21:48:02.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.633 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.633 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.633 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479477+0000", 2024-09-15T21:48:02.633 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.002426489 2024-09-15T21:48:02.633 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.634 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.634 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.634 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479553+0000", 2024-09-15T21:48:02.634 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.6583999999999997e-05 2024-09-15T21:48:02.634 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.634 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.634 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 1", 2024-09-15T21:48:02.634 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.479637+0000", 2024-09-15T21:48:02.634 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.4172999999999995e-05 2024-09-15T21:48:02.634 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.634 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.634 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.634 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483108+0000", 2024-09-15T21:48:02.634 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003470356 2024-09-15T21:48:02.635 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.635 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.635 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.635 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483160+0000", 2024-09-15T21:48:02.635 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2507999999999999e-05 2024-09-15T21:48:02.635 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.635 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.635 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.635 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483202+0000", 2024-09-15T21:48:02.635 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1758999999999998e-05 2024-09-15T21:48:02.635 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.635 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.635 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.636 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.483229+0000", 2024-09-15T21:48:02.636 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.6995000000000002e-05 2024-09-15T21:48:02.636 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.636 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.636 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.636 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.636 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.636 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:31 2.10 2:0eee8b97:::202.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.636 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:48:02.636 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 82.063543749999994, 2024-09-15T21:48:02.636 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.55145717800000005, 2024-09-15T21:48:02.636 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.637 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.637 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:48:02.637 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.637 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.637 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.637 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:48:02.637 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.637 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.637 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.637 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.637 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:48:02.637 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.637 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.637 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.638 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.638 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475505+0000", 2024-09-15T21:48:02.638 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.65e-07 2024-09-15T21:48:02.638 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.638 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.638 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.638 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475507+0000", 2024-09-15T21:48:02.638 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4699999999999999e-06 2024-09-15T21:48:02.638 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.638 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.638 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.638 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.025935+0000", 2024-09-15T21:48:02.639 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.55042854200000002 2024-09-15T21:48:02.639 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.639 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.639 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.639 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.025965+0000", 2024-09-15T21:48:02.639 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.9995999999999999e-05 2024-09-15T21:48:02.639 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.639 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.639 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.639 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026016+0000", 2024-09-15T21:48:02.639 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.1310000000000002e-05 2024-09-15T21:48:02.639 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.639 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.639 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.640 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026122+0000", 2024-09-15T21:48:02.640 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000105605 2024-09-15T21:48:02.640 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.640 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.640 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.640 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:41.026962+0000", 2024-09-15T21:48:02.640 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00083998999999999999 2024-09-15T21:48:02.640 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.640 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.640 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.640 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.640 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.640 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:5 2.e 2:75ee5fd8:::629.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.641 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:48:02.641 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 70.974337384999998, 2024-09-15T21:48:02.641 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0071955470000000001, 2024-09-15T21:48:02.641 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.641 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.641 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.641 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.641 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.641 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.641 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:48:02.641 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.641 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.641 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.642 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.642 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564711+0000", 2024-09-15T21:48:02.642 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.642 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.642 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.642 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.642 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564713+0000", 2024-09-15T21:48:02.642 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.866e-06 2024-09-15T21:48:02.642 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.642 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.642 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.642 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564721+0000", 2024-09-15T21:48:02.642 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3299999999999999e-06 2024-09-15T21:48:02.642 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.643 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.643 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.643 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564762+0000", 2024-09-15T21:48:02.643 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0707999999999997e-05 2024-09-15T21:48:02.643 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.643 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.643 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.643 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564774+0000", 2024-09-15T21:48:02.643 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1883e-05 2024-09-15T21:48:02.643 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.643 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.643 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.643 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564877+0000", 2024-09-15T21:48:02.643 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010254700000000001 2024-09-15T21:48:02.644 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.644 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.644 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.644 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565066+0000", 2024-09-15T21:48:02.644 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018933700000000001 2024-09-15T21:48:02.644 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.644 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.644 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:48:02.644 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565164+0000", 2024-09-15T21:48:02.644 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.7633999999999994e-05 2024-09-15T21:48:02.644 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.644 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.644 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.644 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569030+0000", 2024-09-15T21:48:02.645 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038669070000000002 2024-09-15T21:48:02.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.645 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.645 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.645 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571818+0000", 2024-09-15T21:48:02.645 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0027878930000000001 2024-09-15T21:48:02.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.645 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.645 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.645 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571868+0000", 2024-09-15T21:48:02.645 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9880999999999999e-05 2024-09-15T21:48:02.645 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.645 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.646 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.646 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571907+0000", 2024-09-15T21:48:02.646 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8560999999999999e-05 2024-09-15T21:48:02.646 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.646 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.646 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.646 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.646 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.646 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:6 2.e 2:75ee5fd8:::629.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.646 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:48:02.646 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 70.974323257999998, 2024-09-15T21:48:02.646 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073433220000000002, 2024-09-15T21:48:02.646 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.647 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.647 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.647 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.647 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.647 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.647 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:48:02.647 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.647 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.647 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.647 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.647 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564725+0000", 2024-09-15T21:48:02.647 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.647 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.647 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.648 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.648 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564726+0000", 2024-09-15T21:48:02.648 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7399999999999999e-07 2024-09-15T21:48:02.648 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.648 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.648 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.648 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564728+0000", 2024-09-15T21:48:02.648 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.447e-06 2024-09-15T21:48:02.648 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.648 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.648 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.648 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565446+0000", 2024-09-15T21:48:02.648 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00071774999999999996 2024-09-15T21:48:02.648 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.649 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.649 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.649 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565453+0000", 2024-09-15T21:48:02.649 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3239999999999997e-06 2024-09-15T21:48:02.649 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.649 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.649 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.649 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565515+0000", 2024-09-15T21:48:02.649 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1768999999999998e-05 2024-09-15T21:48:02.649 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.649 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.649 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.649 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565591+0000", 2024-09-15T21:48:02.649 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5873999999999999e-05 2024-09-15T21:48:02.650 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.650 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.650 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:48:02.650 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565709+0000", 2024-09-15T21:48:02.650 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011838899999999999 2024-09-15T21:48:02.650 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.650 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.650 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.650 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.569039+0000", 2024-09-15T21:48:02.650 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0033292170000000002 2024-09-15T21:48:02.650 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.650 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.651 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.651 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571977+0000", 2024-09-15T21:48:02.651 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0029387829999999999 2024-09-15T21:48:02.651 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.651 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.651 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.651 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.572032+0000", 2024-09-15T21:48:02.651 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.4540999999999998e-05 2024-09-15T21:48:02.651 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.651 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.651 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.651 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.572069+0000", 2024-09-15T21:48:02.651 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.6854000000000003e-05 2024-09-15T21:48:02.651 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.652 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.652 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.652 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.652 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.652 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:2 2.1 2:82971f97:::404.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:02.652 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:48:02.652 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 70.974319291, 2024-09-15T21:48:02.652 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.007245357, 2024-09-15T21:48:02.652 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:02.652 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:02.652 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:02.652 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:02.652 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.652 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:02.653 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:48:02.653 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.653 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.653 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.653 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:02.653 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564729+0000", 2024-09-15T21:48:02.653 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:02.653 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.653 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.653 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:02.653 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564730+0000", 2024-09-15T21:48:02.653 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.2999999999999999e-07 2024-09-15T21:48:02.653 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.653 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.654 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:02.654 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.564731+0000", 2024-09-15T21:48:02.654 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.367e-06 2024-09-15T21:48:02.654 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.654 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.654 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:02.654 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565484+0000", 2024-09-15T21:48:02.654 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00075311599999999996 2024-09-15T21:48:02.654 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.654 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.654 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:02.654 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565490+0000", 2024-09-15T21:48:02.654 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8100000000000003e-06 2024-09-15T21:48:02.654 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.655 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:02.655 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565828+0000", 2024-09-15T21:48:02.655 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000337836 2024-09-15T21:48:02.655 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.655 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:02.655 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565964+0000", 2024-09-15T21:48:02.655 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013641599999999999 2024-09-15T21:48:02.655 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.655 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.655 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 3", 2024-09-15T21:48:02.655 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566071+0000", 2024-09-15T21:48:02.655 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000107126 2024-09-15T21:48:02.656 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.656 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.656 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:02.656 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571361+0000", 2024-09-15T21:48:02.656 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052896210000000004 2024-09-15T21:48:02.656 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.656 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.656 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:02.656 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571874+0000", 2024-09-15T21:48:02.656 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00051320599999999999 2024-09-15T21:48:02.656 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.656 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.656 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:02.656 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571936+0000", 2024-09-15T21:48:02.657 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.1650999999999996e-05 2024-09-15T21:48:02.657 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:02.657 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:02.657 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:02.657 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571975+0000", 2024-09-15T21:48:02.657 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8977999999999998e-05 2024-09-15T21:48:02.657 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.657 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.657 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.657 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:02.657 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:02.657 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:02.658 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:48:02.659 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:48:02.660 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:48:02.656+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:48:02.760 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:48:02.757+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:48:02.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.800+0000 7fe7b7ec9640 1 Processor -- start 2024-09-15T21:48:02.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.801+0000 7fe7b7ec9640 1 -- start start 2024-09-15T21:48:02.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.801+0000 7fe7b7ec9640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe7b00cb7d0 0x7fe7b00cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:02.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.801+0000 7fe7b7ec9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe7b006d0e0 0x7fe7b00cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:02.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.801+0000 7fe7b7ec9640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7b015e7a0 con 0x7fe7b0155440 2024-09-15T21:48:02.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.801+0000 7fe7b7ec9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fe7b015b470 con 0x7fe7b006d0e0 2024-09-15T21:48:02.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.801+0000 7fe7b7ec9640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b0155820 con 0x7fe7b00cb7d0 2024-09-15T21:48:02.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.801+0000 7fe7b6466640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe7b006d0e0 0x7fe7b00cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:02.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.802+0000 7fe7b5464640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe7b00cb7d0 0x7fe7b00cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:02.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.802+0000 7fe7b5c65640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fe7b0155440 0x7fe7b00cd420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.100:57776/0 (socket says 172.21.15.100:57776) 2024-09-15T21:48:02.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.802+0000 7fe7b6466640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fe7b006d0e0 0x7fe7b00cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57760/0 (socket says 172.21.15.100:57760) 2024-09-15T21:48:02.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.802+0000 7fe7b5c65640 1 -- 172.21.15.100:0/182236726 learned_addr learned my addr 172.21.15.100:0/182236726 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:02.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.802+0000 7fe7b4c63640 1 -- 172.21.15.100:0/182236726 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2331643945 0 0) 0x7fe7b015e7a0 con 0x7fe7b0155440 2024-09-15T21:48:02.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.802+0000 7fe7b6466640 1 -- 172.21.15.100:0/182236726 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe7b00cb7d0 msgr2=0x7fe7b00cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:02.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.802+0000 7fe7b6466640 1 --2- 172.21.15.100:0/182236726 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe7b00cb7d0 0x7fe7b00cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:02.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.802+0000 7fe7b6466640 1 -- 172.21.15.100:0/182236726 >> v1:172.21.15.122:6789/0 conn(0x7fe7b0155440 legacy=0x7fe7b00cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:02.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.802+0000 7fe7b6466640 1 -- 172.21.15.100:0/182236726 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b015ea40 con 0x7fe7b006d0e0 2024-09-15T21:48:02.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.803+0000 7fe7b6466640 1 --2- 172.21.15.100:0/182236726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7b006d0e0 0x7fe7b00cc720 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009d90 tx=0x7fe7a802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e42de1ef7392d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:02.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.803+0000 7fe7b4c63640 1 -- 172.21.15.100:0/182236726 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8031600 con 0x7fe7b006d0e0 2024-09-15T21:48:02.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.803+0000 7fe7b4c63640 1 -- 172.21.15.100:0/182236726 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a80317a0 con 0x7fe7b006d0e0 2024-09-15T21:48:02.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.803+0000 7fe7b4c63640 1 -- 172.21.15.100:0/182236726 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8031b70 con 0x7fe7b006d0e0 2024-09-15T21:48:02.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.804+0000 7fe7b7ec9640 1 -- 172.21.15.100:0/182236726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7b006d0e0 msgr2=0x7fe7b00cc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:02.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.804+0000 7fe7b7ec9640 1 --2- 172.21.15.100:0/182236726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7b006d0e0 0x7fe7b00cc720 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009d90 tx=0x7fe7a802a5d0 comp rx=0 tx=0).stop 2024-09-15T21:48:02.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.804+0000 7fe7b7ec9640 1 -- 172.21.15.100:0/182236726 shutdown_connections 2024-09-15T21:48:02.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.804+0000 7fe7b7ec9640 1 --2- 172.21.15.100:0/182236726 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7b006d0e0 0x7fe7b00cc720 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:02.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.804+0000 7fe7b7ec9640 1 --2- 172.21.15.100:0/182236726 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe7b00cb7d0 0x7fe7b00cc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:02.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.804+0000 7fe7b7ec9640 1 -- 172.21.15.100:0/182236726 >> 172.21.15.100:0/182236726 conn(0x7fe7b00680d0 msgr2=0x7fe7b006d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:02.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.804+0000 7fe7b7ec9640 1 -- 172.21.15.100:0/182236726 shutdown_connections 2024-09-15T21:48:02.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.804+0000 7fe7b7ec9640 1 -- 172.21.15.100:0/182236726 wait complete. 2024-09-15T21:48:02.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.806+0000 7fe7b7ec9640 1 Processor -- start 2024-09-15T21:48:02.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.806+0000 7fe7b7ec9640 1 -- start start 2024-09-15T21:48:02.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.807+0000 7fe7b7ec9640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe7b006d0e0 0x7fe7b0141290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:02.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.807+0000 7fe7b7ec9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7b00cb7d0 0x7fe7b0149c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:02.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.807+0000 7fe7b7ec9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7b0155440 0x7fe7b014a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:02.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.807+0000 7fe7b7ec9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b015f2e0 con 0x7fe7b0155440 2024-09-15T21:48:02.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.807+0000 7fe7b7ec9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b015f000 con 0x7fe7b00cb7d0 2024-09-15T21:48:02.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.807+0000 7fe7b7ec9640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b015b070 con 0x7fe7b006d0e0 2024-09-15T21:48:02.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.807+0000 7fe7b6466640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7b0155440 0x7fe7b014a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:02.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.807+0000 7fe7b6466640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7b0155440 0x7fe7b014a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57762/0 (socket says 172.21.15.100:57762) 2024-09-15T21:48:02.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.807+0000 7fe7b6466640 1 -- 172.21.15.100:0/1044801215 learned_addr learned my addr 172.21.15.100:0/1044801215 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:02.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.807+0000 7fe7b5c65640 1 --2- 172.21.15.100:0/1044801215 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe7b006d0e0 0x7fe7b0141290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:02.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.807+0000 7fe7b5464640 1 --2- 172.21.15.100:0/1044801215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7b00cb7d0 0x7fe7b0149c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:02.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.808+0000 7fe7b6466640 1 -- 172.21.15.100:0/1044801215 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe7b006d0e0 msgr2=0x7fe7b0141290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:02.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.808+0000 7fe7b6466640 1 --2- 172.21.15.100:0/1044801215 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe7b006d0e0 0x7fe7b0141290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:02.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.808+0000 7fe7b6466640 1 -- 172.21.15.100:0/1044801215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7b00cb7d0 msgr2=0x7fe7b0149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:02.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.808+0000 7fe7b6466640 1 --2- 172.21.15.100:0/1044801215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7b00cb7d0 0x7fe7b0149c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:02.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.808+0000 7fe7b6466640 1 -- 172.21.15.100:0/1044801215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b01417d0 con 0x7fe7b0155440 2024-09-15T21:48:02.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.808+0000 7fe7b6466640 1 --2- 172.21.15.100:0/1044801215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7b0155440 0x7fe7b014a1c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009d90 tx=0x7fe7a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:02.812 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:02.812 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:48:02.813 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:48:02.813 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:02.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.808+0000 7fe7a67fc640 1 -- 172.21.15.100:0/1044801215 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a803f070 con 0x7fe7b0155440 2024-09-15T21:48:02.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.808+0000 7fe7a67fc640 1 -- 172.21.15.100:0/1044801215 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7a8030be0 con 0x7fe7b0155440 2024-09-15T21:48:02.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.808+0000 7fe7b7ec9640 1 -- 172.21.15.100:0/1044801215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b01f4770 con 0x7fe7b0155440 2024-09-15T21:48:02.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.808+0000 7fe7a67fc640 1 -- 172.21.15.100:0/1044801215 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8009450 con 0x7fe7b0155440 2024-09-15T21:48:02.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.808+0000 7fe7b7ec9640 1 -- 172.21.15.100:0/1044801215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b01f4d30 con 0x7fe7b0155440 2024-09-15T21:48:02.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.809+0000 7fe7b7ec9640 1 -- 172.21.15.100:0/1044801215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b00b3940 con 0x7fe7b0155440 2024-09-15T21:48:02.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.810+0000 7fe7a67fc640 1 -- 172.21.15.100:0/1044801215 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe7a8003990 con 0x7fe7b0155440 2024-09-15T21:48:02.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.810+0000 7fe7a67fc640 1 --2- 172.21.15.100:0/1044801215 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe79406c2b0 0x7fe79406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:02.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.811+0000 7fe7b5c65640 1 --2- 172.21.15.100:0/1044801215 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe79406c2b0 0x7fe79406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:02.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.811+0000 7fe7a67fc640 1 -- 172.21.15.100:0/1044801215 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6060+0+0 (secure 0 0 0) 0x7fe7a80ac020 con 0x7fe7b0155440 2024-09-15T21:48:02.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.811+0000 7fe7b5c65640 1 --2- 172.21.15.100:0/1044801215 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe79406c2b0 0x7fe79406e770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac000f60 tx=0x7fe7ac007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:02.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:02.815+0000 7fe7a67fc640 1 -- 172.21.15.100:0/1044801215 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe7a807a070 con 0x7fe7b0155440 2024-09-15T21:48:02.826 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:48:02.861 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:48:02.859+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:48:02.961 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:48:02.959+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:48:02.975 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:02.975 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:48:02.975 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:48:02.975 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:48:02.975 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:02.984 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:48:03.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.004+0000 7fe7b7ec9640 1 -- 172.21.15.100:0/1044801215 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd set", "key": "noscrub"} v 0) -- 0x7fe7b00caed0 con 0x7fe7b0155440 2024-09-15T21:48:03.062 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:48:03.061+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:48:03.114 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:03.114 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:48:03.114 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:48:03.115 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:48:03.115 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.115 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:03.115 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:48:03.115 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 94.698345054000001, 2024-09-15T21:48:03.115 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047486860000000002, 2024-09-15T21:48:03.115 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.115 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.115 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:48:03.115 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.115 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.115 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.115 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:48:03.116 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.116 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.116 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.116 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.116 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:48:03.116 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.116 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.116 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.116 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.116 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413816+0000", 2024-09-15T21:48:03.116 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4810000000000003e-06 2024-09-15T21:48:03.116 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.116 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.116 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.117 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413827+0000", 2024-09-15T21:48:03.117 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.187e-05 2024-09-15T21:48:03.117 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.117 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.117 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.117 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413862+0000", 2024-09-15T21:48:03.117 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4041000000000002e-05 2024-09-15T21:48:03.117 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.117 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.117 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.117 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413871+0000", 2024-09-15T21:48:03.117 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8919999999999995e-06 2024-09-15T21:48:03.117 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.117 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.118 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.118 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413997+0000", 2024-09-15T21:48:03.118 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012601299999999999 2024-09-15T21:48:03.118 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.118 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.118 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:03.118 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414100+0000", 2024-09-15T21:48:03.118 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102555 2024-09-15T21:48:03.118 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.118 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.118 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.118 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418558+0000", 2024-09-15T21:48:03.118 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044578339999999999 2024-09-15T21:48:03.118 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.118 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.119 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.119 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.119 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.119 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-15T21:48:03.119 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:48:03.119 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 94.694700240000003, 2024-09-15T21:48:03.119 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045423590000000002, 2024-09-15T21:48:03.119 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.119 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.119 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.119 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.119 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.119 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.119 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:48:03.120 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.120 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.120 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.120 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.120 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:48:03.120 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.120 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.120 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.120 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.120 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417455+0000", 2024-09-15T21:48:03.120 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9100000000000003e-07 2024-09-15T21:48:03.120 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.120 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.120 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.121 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417463+0000", 2024-09-15T21:48:03.121 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3750000000000003e-06 2024-09-15T21:48:03.121 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.121 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.121 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.121 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417475+0000", 2024-09-15T21:48:03.121 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2473000000000001e-05 2024-09-15T21:48:03.121 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.121 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.121 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.121 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417482+0000", 2024-09-15T21:48:03.121 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4010000000000002e-06 2024-09-15T21:48:03.121 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.121 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.122 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.122 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417524+0000", 2024-09-15T21:48:03.122 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1870999999999999e-05 2024-09-15T21:48:03.122 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.122 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.122 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:03.122 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417542+0000", 2024-09-15T21:48:03.122 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8224e-05 2024-09-15T21:48:03.122 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.123 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.123 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:03.123 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421955+0000", 2024-09-15T21:48:03.123 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044130740000000003 2024-09-15T21:48:03.123 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.123 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.123 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.123 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421996+0000", 2024-09-15T21:48:03.123 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.125e-05 2024-09-15T21:48:03.123 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.123 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.123 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.123 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.123 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.124 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 7 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:48:03.124 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:48:03.124 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 94.694439435000007, 2024-09-15T21:48:03.124 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043086519999999996, 2024-09-15T21:48:03.124 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.124 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.124 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:48:03.124 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.124 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.124 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.124 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:48:03.124 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.124 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.125 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.125 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.125 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:48:03.125 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.125 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.125 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.125 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.125 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:48:03.125 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.58e-07 2024-09-15T21:48:03.125 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.125 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.125 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.125 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417722+0000", 2024-09-15T21:48:03.125 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4729999999999997e-06 2024-09-15T21:48:03.125 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.126 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.126 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.126 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417737+0000", 2024-09-15T21:48:03.126 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5155e-05 2024-09-15T21:48:03.126 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.126 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.126 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.126 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417752+0000", 2024-09-15T21:48:03.126 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4863e-05 2024-09-15T21:48:03.126 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.126 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.126 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.126 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417819+0000", 2024-09-15T21:48:03.126 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7680000000000003e-05 2024-09-15T21:48:03.127 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.127 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.127 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.127 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422023+0000", 2024-09-15T21:48:03.127 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042040230000000003 2024-09-15T21:48:03.127 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.127 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.127 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.127 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.127 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.127 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:25 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:03.127 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:48:03.127 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 88.654950404999994, 2024-09-15T21:48:03.127 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070572830000000001, 2024-09-15T21:48:03.128 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.128 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.128 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.128 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.128 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.128 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.128 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:48:03.128 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.128 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.128 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.128 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.128 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:48:03.128 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.129 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.129 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.129 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.129 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457205+0000", 2024-09-15T21:48:03.129 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6899999999999999e-06 2024-09-15T21:48:03.129 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.129 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.129 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.129 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457211+0000", 2024-09-15T21:48:03.129 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.626e-06 2024-09-15T21:48:03.129 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.129 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.129 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.130 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457244+0000", 2024-09-15T21:48:03.130 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2691999999999998e-05 2024-09-15T21:48:03.130 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.130 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.130 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.130 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457251+0000", 2024-09-15T21:48:03.130 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3839999999999999e-06 2024-09-15T21:48:03.130 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.130 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.130 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.130 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457309+0000", 2024-09-15T21:48:03.130 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8023999999999999e-05 2024-09-15T21:48:03.130 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.130 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.131 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:03.131 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457413+0000", 2024-09-15T21:48:03.131 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010331999999999999 2024-09-15T21:48:03.131 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.131 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.131 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:03.131 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:48:03.131 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118439 2024-09-15T21:48:03.131 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.131 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.131 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:03.131 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463246+0000", 2024-09-15T21:48:03.131 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057148219999999996 2024-09-15T21:48:03.131 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.132 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.132 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:03.132 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463428+0000", 2024-09-15T21:48:03.132 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018218299999999999 2024-09-15T21:48:03.132 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.132 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.132 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:03.132 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464212+0000", 2024-09-15T21:48:03.133 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00078402799999999998 2024-09-15T21:48:03.133 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.133 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.133 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.133 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464261+0000", 2024-09-15T21:48:03.133 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9075e-05 2024-09-15T21:48:03.133 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.133 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.133 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.133 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.133 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.133 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:28 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:03.134 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:48:03.134 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 88.654940378000006, 2024-09-15T21:48:03.134 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0072141449999999999, 2024-09-15T21:48:03.134 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.134 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.134 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:48:03.134 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.134 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.134 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.134 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:48:03.134 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.134 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.134 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.134 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.134 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:48:03.135 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.135 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.135 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.135 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.135 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:48:03.135 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:48:03.135 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.135 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.135 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.135 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457217+0000", 2024-09-15T21:48:03.135 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7259999999999998e-06 2024-09-15T21:48:03.135 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.135 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.135 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.136 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457842+0000", 2024-09-15T21:48:03.136 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00062482599999999998 2024-09-15T21:48:03.136 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.136 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.136 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.136 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457848+0000", 2024-09-15T21:48:03.136 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7089999999999996e-06 2024-09-15T21:48:03.136 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.136 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.136 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.136 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457895+0000", 2024-09-15T21:48:03.136 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6317999999999999e-05 2024-09-15T21:48:03.136 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.136 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.137 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:03.137 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458013+0000", 2024-09-15T21:48:03.137 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011848100000000001 2024-09-15T21:48:03.137 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.137 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.137 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.137 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464428+0000", 2024-09-15T21:48:03.137 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0064146990000000003 2024-09-15T21:48:03.137 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.137 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.137 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.137 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.137 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.137 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:29 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:03.137 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:48:03.138 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 88.654935934999997, 2024-09-15T21:48:03.138 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073658980000000001, 2024-09-15T21:48:03.138 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.138 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.138 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:48:03.138 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.138 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.138 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.138 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:48:03.138 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.138 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.138 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.138 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.138 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:48:03.139 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.139 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.139 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.139 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.139 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457219+0000", 2024-09-15T21:48:03.139 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2000000000000001e-07 2024-09-15T21:48:03.139 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.139 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.139 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.139 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457226+0000", 2024-09-15T21:48:03.139 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.956e-06 2024-09-15T21:48:03.139 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.139 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.139 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.139 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458266+0000", 2024-09-15T21:48:03.140 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001040476 2024-09-15T21:48:03.140 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.140 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.140 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.140 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458273+0000", 2024-09-15T21:48:03.140 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0690000000000004e-06 2024-09-15T21:48:03.140 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.140 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.140 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.140 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458319+0000", 2024-09-15T21:48:03.140 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5668999999999999e-05 2024-09-15T21:48:03.140 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.140 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.140 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:03.141 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458415+0000", 2024-09-15T21:48:03.141 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6701999999999997e-05 2024-09-15T21:48:03.141 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.141 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.141 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.141 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464584+0000", 2024-09-15T21:48:03.141 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061687060000000004 2024-09-15T21:48:03.141 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.141 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.141 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.141 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.141 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.141 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.1 epoch 22/19 rep_tid 12 entries 22'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.10:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:48:03.141 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:48:03.142 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 88.653505929000005, 2024-09-15T21:48:03.142 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0039881550000000002, 2024-09-15T21:48:03.142 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.142 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.142 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:48:03.142 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.142 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.142 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.142 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:48:03.142 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.142 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.142 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.143 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.143 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:48:03.143 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.143 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.143 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.143 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.143 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458649+0000", 2024-09-15T21:48:03.143 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9599999999999999e-07 2024-09-15T21:48:03.143 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.143 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.144 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.144 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458656+0000", 2024-09-15T21:48:03.144 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2629999999999997e-06 2024-09-15T21:48:03.144 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.144 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.144 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.144 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458670+0000", 2024-09-15T21:48:03.144 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4166e-05 2024-09-15T21:48:03.144 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.144 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.144 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.144 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458682+0000", 2024-09-15T21:48:03.144 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1865000000000001e-05 2024-09-15T21:48:03.144 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.145 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.145 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.145 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458749+0000", 2024-09-15T21:48:03.145 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6507000000000005e-05 2024-09-15T21:48:03.145 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.145 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.145 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.145 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462636+0000", 2024-09-15T21:48:03.145 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038877579999999998 2024-09-15T21:48:03.145 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.145 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.145 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.145 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.145 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.145 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.1.10:12 2.12 e22/19 2:48cd7578:::60e.00000000:head v 22'2, mlcod=0'0)", 2024-09-15T21:48:03.146 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:48:03.146 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 88.653191597000003, 2024-09-15T21:48:03.146 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042479500000000003, 2024-09-15T21:48:03.146 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.146 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.146 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.146 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.146 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.146 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.146 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:48:03.146 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.146 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.146 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.146 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.147 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:48:03.147 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.147 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.147 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.147 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.147 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:48:03.147 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2900000000000004e-07 2024-09-15T21:48:03.147 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.147 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.147 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.147 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458968+0000", 2024-09-15T21:48:03.147 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2039999999999996e-06 2024-09-15T21:48:03.147 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.147 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.148 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.148 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458977+0000", 2024-09-15T21:48:03.148 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.6179999999999995e-06 2024-09-15T21:48:03.148 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.148 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.148 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.148 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458983+0000", 2024-09-15T21:48:03.148 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9510000000000003e-06 2024-09-15T21:48:03.148 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.148 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.148 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.148 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459479+0000", 2024-09-15T21:48:03.148 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000496274 2024-09-15T21:48:03.148 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.149 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.149 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:03.149 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459496+0000", 2024-09-15T21:48:03.149 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6538e-05 2024-09-15T21:48:03.149 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.149 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.149 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:03.149 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463169+0000", 2024-09-15T21:48:03.149 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003672832 2024-09-15T21:48:03.149 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.149 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.149 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.149 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463211+0000", 2024-09-15T21:48:03.149 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2004e-05 2024-09-15T21:48:03.149 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.150 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.150 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.150 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.150 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.150 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.12 epoch 22/19 rep_tid 2 entries 22'3 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.10:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:48:03.150 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:48:03.150 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 88.652512043000002, 2024-09-15T21:48:03.150 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035859609999999999, 2024-09-15T21:48:03.150 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.150 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.150 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:48:03.150 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.150 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.150 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.151 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:48:03.151 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.151 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.151 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.151 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.151 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:48:03.151 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.151 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.151 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.151 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.151 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459643+0000", 2024-09-15T21:48:03.151 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2200000000000004e-07 2024-09-15T21:48:03.151 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.151 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.151 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.152 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459650+0000", 2024-09-15T21:48:03.152 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4479999999999997e-06 2024-09-15T21:48:03.152 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.152 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.152 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.152 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459665+0000", 2024-09-15T21:48:03.152 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4586999999999999e-05 2024-09-15T21:48:03.152 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.152 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.152 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.152 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459676+0000", 2024-09-15T21:48:03.152 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1673e-05 2024-09-15T21:48:03.152 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.152 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.153 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.153 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459732+0000", 2024-09-15T21:48:03.153 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5612999999999998e-05 2024-09-15T21:48:03.153 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.153 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.153 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.153 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463228+0000", 2024-09-15T21:48:03.153 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0034961179999999999 2024-09-15T21:48:03.153 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.154 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.154 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.154 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.154 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.154 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:34 2.1b 2:db457a8a:::201.00000001:head [write 0~588] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:03.154 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:48:03.154 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 88.647300407000003, 2024-09-15T21:48:03.154 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070946129999999996, 2024-09-15T21:48:03.154 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.154 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.154 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.154 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.154 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.154 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.155 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:48:03.155 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.155 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.155 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.155 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.155 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:48:03.155 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.155 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.155 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.155 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.155 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464855+0000", 2024-09-15T21:48:03.155 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5880000000000001e-06 2024-09-15T21:48:03.155 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.155 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.155 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.156 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464863+0000", 2024-09-15T21:48:03.156 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0879999999999995e-06 2024-09-15T21:48:03.156 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.156 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.156 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.156 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464889+0000", 2024-09-15T21:48:03.156 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5729e-05 2024-09-15T21:48:03.156 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.156 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.156 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.156 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464898+0000", 2024-09-15T21:48:03.156 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0389999999999993e-06 2024-09-15T21:48:03.156 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.157 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.157 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.157 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464962+0000", 2024-09-15T21:48:03.157 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3567999999999999e-05 2024-09-15T21:48:03.157 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.157 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.157 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:03.157 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.465041+0000", 2024-09-15T21:48:03.157 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8889999999999999e-05 2024-09-15T21:48:03.157 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.157 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.157 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:03.157 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.465173+0000", 2024-09-15T21:48:03.157 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013226899999999999 2024-09-15T21:48:03.158 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.158 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.158 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:03.158 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.466319+0000", 2024-09-15T21:48:03.158 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011463039999999999 2024-09-15T21:48:03.158 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.158 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.158 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:03.158 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471825+0000", 2024-09-15T21:48:03.158 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055062160000000004 2024-09-15T21:48:03.158 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.158 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.158 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:03.158 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471886+0000", 2024-09-15T21:48:03.158 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0503e-05 2024-09-15T21:48:03.159 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.159 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.159 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.159 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471948+0000", 2024-09-15T21:48:03.159 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2419e-05 2024-09-15T21:48:03.159 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.159 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.159 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.159 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.159 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.159 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:03.159 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:48:03.159 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 82.636642242999997, 2024-09-15T21:48:03.159 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047360500000000003, 2024-09-15T21:48:03.160 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.160 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.160 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.160 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.160 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.160 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.160 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:48:03.160 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.160 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.160 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.160 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.160 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:48:03.160 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.160 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.160 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.161 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.161 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475514+0000", 2024-09-15T21:48:03.161 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0990000000000001e-06 2024-09-15T21:48:03.161 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.161 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.161 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.161 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475524+0000", 2024-09-15T21:48:03.161 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5850000000000006e-06 2024-09-15T21:48:03.161 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.161 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.161 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.161 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475557+0000", 2024-09-15T21:48:03.161 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3090000000000003e-05 2024-09-15T21:48:03.161 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.162 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.162 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.162 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475568+0000", 2024-09-15T21:48:03.162 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1116e-05 2024-09-15T21:48:03.162 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.162 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.162 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.162 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475649+0000", 2024-09-15T21:48:03.162 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0815999999999993e-05 2024-09-15T21:48:03.162 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.162 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.162 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:03.162 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475805+0000", 2024-09-15T21:48:03.162 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015667200000000001 2024-09-15T21:48:03.162 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.163 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.163 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:48:03.163 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475944+0000", 2024-09-15T21:48:03.163 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000138651 2024-09-15T21:48:03.163 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.163 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.163 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:03.163 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476900+0000", 2024-09-15T21:48:03.163 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00095588699999999999 2024-09-15T21:48:03.163 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.164 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.164 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:03.164 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480153+0000", 2024-09-15T21:48:03.164 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0032535200000000002 2024-09-15T21:48:03.164 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.164 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.164 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:03.164 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480203+0000", 2024-09-15T21:48:03.164 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9932999999999998e-05 2024-09-15T21:48:03.164 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.164 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.164 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.164 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480248+0000", 2024-09-15T21:48:03.164 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4681000000000001e-05 2024-09-15T21:48:03.165 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.165 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.165 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.165 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.165 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.165 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:03.165 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:48:03.165 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 82.636626301999996, 2024-09-15T21:48:03.165 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049206709999999997, 2024-09-15T21:48:03.165 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.165 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.165 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.165 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.165 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.166 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.166 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:48:03.166 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.166 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.166 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.166 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.166 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:48:03.166 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.166 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.166 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.166 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.166 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:48:03.166 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1699999999999999e-07 2024-09-15T21:48:03.166 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.167 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.167 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.167 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475531+0000", 2024-09-15T21:48:03.167 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.05e-06 2024-09-15T21:48:03.167 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.167 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.167 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.167 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475743+0000", 2024-09-15T21:48:03.167 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021138099999999999 2024-09-15T21:48:03.167 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.167 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.167 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.167 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475751+0000", 2024-09-15T21:48:03.167 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7770000000000002e-06 2024-09-15T21:48:03.167 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.168 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.168 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.168 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476272+0000", 2024-09-15T21:48:03.168 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052158399999999996 2024-09-15T21:48:03.168 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.168 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.168 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:03.168 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476348+0000", 2024-09-15T21:48:03.168 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5963000000000004e-05 2024-09-15T21:48:03.168 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.168 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.168 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:48:03.168 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476474+0000", 2024-09-15T21:48:03.168 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012638800000000001 2024-09-15T21:48:03.169 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.169 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.169 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:03.169 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478194+0000", 2024-09-15T21:48:03.169 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001719768 2024-09-15T21:48:03.169 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.169 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.169 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:03.169 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480358+0000", 2024-09-15T21:48:03.169 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021637340000000001 2024-09-15T21:48:03.169 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.169 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.169 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:03.169 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480411+0000", 2024-09-15T21:48:03.170 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2720999999999999e-05 2024-09-15T21:48:03.170 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.170 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.170 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.170 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480449+0000", 2024-09-15T21:48:03.170 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7888e-05 2024-09-15T21:48:03.170 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.170 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.170 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.170 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.170 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.170 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:03.170 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:48:03.170 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 82.636621512999994, 2024-09-15T21:48:03.171 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005215478, 2024-09-15T21:48:03.171 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.171 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.171 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:48:03.171 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.171 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.171 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.171 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:48:03.171 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.171 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.171 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.171 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.171 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:48:03.171 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.172 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.172 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.172 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.172 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:48:03.172 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5199999999999998e-07 2024-09-15T21:48:03.172 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.172 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.172 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.172 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475535+0000", 2024-09-15T21:48:03.172 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9520000000000001e-06 2024-09-15T21:48:03.172 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.172 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.172 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.172 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475785+0000", 2024-09-15T21:48:03.173 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024966099999999998 2024-09-15T21:48:03.173 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.173 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.173 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.173 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475799+0000", 2024-09-15T21:48:03.173 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4649e-05 2024-09-15T21:48:03.173 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.173 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.173 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.173 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475856+0000", 2024-09-15T21:48:03.173 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6956000000000002e-05 2024-09-15T21:48:03.174 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.174 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.174 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:03.174 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476016+0000", 2024-09-15T21:48:03.174 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000159457 2024-09-15T21:48:03.174 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.174 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.174 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.174 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480748+0000", 2024-09-15T21:48:03.174 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047324510000000004 2024-09-15T21:48:03.174 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.174 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.175 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.175 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.175 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.175 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.2.12:1 2.1 e22/19 2:85bbe569:::202.00000000:head v 22'4, mlcod=0'0)", 2024-09-15T21:48:03.175 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:48:03.175 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 82.633819041999999, 2024-09-15T21:48:03.175 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045423649999999996, 2024-09-15T21:48:03.175 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.175 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.175 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.175 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.175 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.175 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.175 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:48:03.176 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.176 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.176 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.176 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.176 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:48:03.176 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.176 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.176 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.176 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.176 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478336+0000", 2024-09-15T21:48:03.176 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4499999999999996e-07 2024-09-15T21:48:03.176 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.176 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.176 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.177 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478342+0000", 2024-09-15T21:48:03.177 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5579999999999997e-06 2024-09-15T21:48:03.177 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.177 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.177 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.177 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478356+0000", 2024-09-15T21:48:03.177 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3567000000000001e-05 2024-09-15T21:48:03.177 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.177 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.177 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.177 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478364+0000", 2024-09-15T21:48:03.177 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7889999999999995e-06 2024-09-15T21:48:03.177 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.177 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.178 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.178 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478416+0000", 2024-09-15T21:48:03.178 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2188000000000002e-05 2024-09-15T21:48:03.178 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.178 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.178 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:03.178 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478433+0000", 2024-09-15T21:48:03.178 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6674e-05 2024-09-15T21:48:03.178 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.178 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.178 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:03.178 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482832+0000", 2024-09-15T21:48:03.178 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043996110000000003 2024-09-15T21:48:03.178 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.179 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.179 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.179 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482878+0000", 2024-09-15T21:48:03.179 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5232999999999999e-05 2024-09-15T21:48:03.179 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.179 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.179 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.179 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.179 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.179 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:25 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:03.179 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:48:03.179 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 71.546959002999998, 2024-09-15T21:48:03.179 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058459619999999997, 2024-09-15T21:48:03.180 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.180 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.180 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.180 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.180 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.180 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.180 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:48:03.180 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.180 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.180 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.180 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.180 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:48:03.180 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.180 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.181 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.181 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.181 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565197+0000", 2024-09-15T21:48:03.181 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7889999999999999e-06 2024-09-15T21:48:03.181 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.181 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.181 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.181 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565208+0000", 2024-09-15T21:48:03.181 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1358999999999999e-05 2024-09-15T21:48:03.181 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.181 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.181 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.181 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565237+0000", 2024-09-15T21:48:03.181 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8626e-05 2024-09-15T21:48:03.182 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.182 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.182 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.182 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565249+0000", 2024-09-15T21:48:03.182 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1897e-05 2024-09-15T21:48:03.182 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.182 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.182 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.182 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565332+0000", 2024-09-15T21:48:03.182 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2943999999999997e-05 2024-09-15T21:48:03.182 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.182 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.182 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:03.182 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565452+0000", 2024-09-15T21:48:03.183 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119992 2024-09-15T21:48:03.183 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.183 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.183 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:48:03.183 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565610+0000", 2024-09-15T21:48:03.183 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015845099999999999 2024-09-15T21:48:03.183 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.183 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.183 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:03.183 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.568999+0000", 2024-09-15T21:48:03.183 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0033887190000000001 2024-09-15T21:48:03.183 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.183 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.183 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:03.184 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.570979+0000", 2024-09-15T21:48:03.184 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019804919999999999 2024-09-15T21:48:03.184 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.184 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.184 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:03.184 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571014+0000", 2024-09-15T21:48:03.184 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4331e-05 2024-09-15T21:48:03.184 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.184 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.184 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.184 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571041+0000", 2024-09-15T21:48:03.184 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7362e-05 2024-09-15T21:48:03.184 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.184 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.185 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.185 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.185 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.185 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:5 2.e e22/19 2:75ee5fd8:::629.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:48:03.185 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:48:03.185 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 71.546283356999993, 2024-09-15T21:48:03.185 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057947119999999996, 2024-09-15T21:48:03.185 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.185 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.185 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.185 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.185 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.186 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.186 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:48:03.186 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.186 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.186 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.186 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.186 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:48:03.186 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.186 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.186 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.186 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.186 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565872+0000", 2024-09-15T21:48:03.187 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9800000000000002e-07 2024-09-15T21:48:03.187 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.187 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.187 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.187 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565880+0000", 2024-09-15T21:48:03.187 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7619999999999995e-06 2024-09-15T21:48:03.187 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.187 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.187 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.187 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565890+0000", 2024-09-15T21:48:03.188 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0648999999999999e-05 2024-09-15T21:48:03.188 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.188 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.188 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.188 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565897+0000", 2024-09-15T21:48:03.188 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.153e-06 2024-09-15T21:48:03.188 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.188 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.188 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.188 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565966+0000", 2024-09-15T21:48:03.188 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9046000000000003e-05 2024-09-15T21:48:03.188 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.189 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.189 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:03.189 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565985+0000", 2024-09-15T21:48:03.189 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8695e-05 2024-09-15T21:48:03.189 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.189 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.189 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:03.189 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571625+0000", 2024-09-15T21:48:03.189 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056397139999999997 2024-09-15T21:48:03.189 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.189 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.189 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.189 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571666+0000", 2024-09-15T21:48:03.189 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0695000000000003e-05 2024-09-15T21:48:03.190 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.190 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.190 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.190 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.190 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.190 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:6 2.e e22/19 2:75ee5fd8:::629.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:48:03.190 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:48:03.190 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 71.546221109000001, 2024-09-15T21:48:03.190 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057607429999999996, 2024-09-15T21:48:03.190 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.190 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.190 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.190 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.190 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.190 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.191 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:48:03.191 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.191 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.191 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.191 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.191 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:48:03.191 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.191 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.191 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.191 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.191 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565934+0000", 2024-09-15T21:48:03.191 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6199999999999998e-07 2024-09-15T21:48:03.191 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.191 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.192 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.192 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565938+0000", 2024-09-15T21:48:03.192 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4970000000000003e-06 2024-09-15T21:48:03.192 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.192 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.192 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.192 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565945+0000", 2024-09-15T21:48:03.192 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5069999999999997e-06 2024-09-15T21:48:03.192 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.192 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.192 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.192 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565948+0000", 2024-09-15T21:48:03.192 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:48:03.192 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.193 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.193 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.193 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566200+0000", 2024-09-15T21:48:03.193 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025170200000000002 2024-09-15T21:48:03.193 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.193 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.193 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:03.193 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566218+0000", 2024-09-15T21:48:03.193 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7543000000000001e-05 2024-09-15T21:48:03.193 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.193 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.193 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:03.193 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571668+0000", 2024-09-15T21:48:03.193 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054502999999999999 2024-09-15T21:48:03.194 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.194 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.194 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.194 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571694+0000", 2024-09-15T21:48:03.194 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5979e-05 2024-09-15T21:48:03.194 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.194 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.194 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.194 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.194 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.194 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:2 2.1 e22/19 2:82971f97:::404.00000000:head v 22'5, mlcod=22'3)", 2024-09-15T21:48:03.194 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:48:03.194 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 71.546013173000006, 2024-09-15T21:48:03.194 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056016019999999998, 2024-09-15T21:48:03.195 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.195 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.195 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.195 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.195 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.195 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.195 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:48:03.195 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.195 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.195 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.195 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.195 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:48:03.195 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.195 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.195 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.196 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.196 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566142+0000", 2024-09-15T21:48:03.196 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:48:03.196 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.196 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.196 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.196 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566149+0000", 2024-09-15T21:48:03.196 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7510000000000008e-06 2024-09-15T21:48:03.196 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.196 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.196 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.196 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566161+0000", 2024-09-15T21:48:03.196 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1259e-05 2024-09-15T21:48:03.196 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.197 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.197 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.197 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566168+0000", 2024-09-15T21:48:03.197 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.255e-06 2024-09-15T21:48:03.197 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.197 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.197 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.197 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566227+0000", 2024-09-15T21:48:03.197 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8961e-05 2024-09-15T21:48:03.197 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.197 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.197 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:03.197 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566247+0000", 2024-09-15T21:48:03.197 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0536000000000001e-05 2024-09-15T21:48:03.198 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.198 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.198 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:03.198 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571694+0000", 2024-09-15T21:48:03.198 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005446762 2024-09-15T21:48:03.198 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.198 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.198 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.198 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571743+0000", 2024-09-15T21:48:03.198 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8396999999999998e-05 2024-09-15T21:48:03.198 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.198 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.198 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.198 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.199 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.199 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:17 2.14 e22/19 2:2eebc8f4:::62f.00000000:head v 22'9, mlcod=22'7)", 2024-09-15T21:48:03.199 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:48:03.199 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 71.545997076000006, 2024-09-15T21:48:03.199 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056532860000000004, 2024-09-15T21:48:03.199 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.199 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.199 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.199 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.199 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.199 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.199 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:48:03.199 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.199 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.199 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.200 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.200 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:48:03.200 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.200 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.200 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.200 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.200 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566158+0000", 2024-09-15T21:48:03.200 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.92e-07 2024-09-15T21:48:03.200 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.200 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.200 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.200 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566172+0000", 2024-09-15T21:48:03.200 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3976e-05 2024-09-15T21:48:03.201 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.201 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.201 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.201 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566407+0000", 2024-09-15T21:48:03.201 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023483900000000001 2024-09-15T21:48:03.201 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.201 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.201 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.201 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566416+0000", 2024-09-15T21:48:03.201 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0780000000000002e-06 2024-09-15T21:48:03.201 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.201 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.201 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.201 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566476+0000", 2024-09-15T21:48:03.201 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9908999999999998e-05 2024-09-15T21:48:03.202 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.202 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.202 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:03.202 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566493+0000", 2024-09-15T21:48:03.202 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7567e-05 2024-09-15T21:48:03.202 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.202 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.202 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:03.202 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571762+0000", 2024-09-15T21:48:03.202 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052682959999999996 2024-09-15T21:48:03.202 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.202 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.202 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.202 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571810+0000", 2024-09-15T21:48:03.203 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8829000000000003e-05 2024-09-15T21:48:03.203 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.203 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.203 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.203 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.203 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.203 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:18 2.14 e22/19 2:2eebc8f4:::62f.00000000:head v 22'10, mlcod=22'7)", 2024-09-15T21:48:03.203 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:48:03.203 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 71.545708000999994, 2024-09-15T21:48:03.203 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053992579999999997, 2024-09-15T21:48:03.203 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:03.203 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:03.203 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.203 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:03.203 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.204 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:03.204 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:48:03.204 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.204 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.204 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.204 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:03.204 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:48:03.204 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:03.204 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.204 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.204 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:03.204 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566447+0000", 2024-09-15T21:48:03.204 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2e-07 2024-09-15T21:48:03.204 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.205 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.205 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:03.205 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566452+0000", 2024-09-15T21:48:03.205 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0390000000000004e-06 2024-09-15T21:48:03.205 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.205 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.205 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:03.205 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566460+0000", 2024-09-15T21:48:03.205 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1389999999999995e-06 2024-09-15T21:48:03.205 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.205 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.205 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.205 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566473+0000", 2024-09-15T21:48:03.205 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3093e-05 2024-09-15T21:48:03.206 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.206 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.206 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:03.206 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566688+0000", 2024-09-15T21:48:03.206 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021537100000000001 2024-09-15T21:48:03.206 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.206 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.206 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:03.206 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566706+0000", 2024-09-15T21:48:03.206 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7802999999999999e-05 2024-09-15T21:48:03.206 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.206 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.206 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:03.206 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571813+0000", 2024-09-15T21:48:03.207 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051065140000000004 2024-09-15T21:48:03.207 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:03.207 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:03.207 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:03.207 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571845+0000", 2024-09-15T21:48:03.207 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2879000000000001e-05 2024-09-15T21:48:03.207 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.207 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.207 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.207 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:03.207 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:03.207 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:03.209 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:48:03.209 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:48:03.207+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:48:03.310 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:48:03.308+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:48:03.322 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:48:03.322 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:48:03.322 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:48:03.322 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:48:03.331 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:48:03.411 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:48:03.409+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:48:03.435 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:48:03.435 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:48:03.435 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:48:03.435 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:48:03.435 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:48:03.445 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:48:03.512 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:48:03.511+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:48:03.514 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:48:03.550 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:48:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:48:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:48:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:48:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:03.550 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:48:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 95.134668594999994, 2024-09-15T21:48:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061533810000000003, 2024-09-15T21:48:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:48:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:48:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413085+0000", 2024-09-15T21:48:03.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8390000000000003e-06 2024-09-15T21:48:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413092+0000", 2024-09-15T21:48:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3479999999999998e-06 2024-09-15T21:48:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413109+0000", 2024-09-15T21:48:03.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7192000000000001e-05 2024-09-15T21:48:03.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413118+0000", 2024-09-15T21:48:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.935e-06 2024-09-15T21:48:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413219+0000", 2024-09-15T21:48:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010121 2024-09-15T21:48:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413381+0000", 2024-09-15T21:48:03.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016148399999999999 2024-09-15T21:48:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413525+0000", 2024-09-15T21:48:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014428100000000001 2024-09-15T21:48:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415866+0000", 2024-09-15T21:48:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023409300000000002 2024-09-15T21:48:03.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419156+0000", 2024-09-15T21:48:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032903580000000002 2024-09-15T21:48:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419199+0000", 2024-09-15T21:48:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2171e-05 2024-09-15T21:48:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.557 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419232+0000", 2024-09-15T21:48:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3633000000000001e-05 2024-09-15T21:48:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.4:28 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-15T21:48:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:48:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 95.133405085999996, 2024-09-15T21:48:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002614594, 2024-09-15T21:48:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.558 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:48:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:48:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414344+0000", 2024-09-15T21:48:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6589999999999999e-06 2024-09-15T21:48:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414355+0000", 2024-09-15T21:48:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0604999999999999e-05 2024-09-15T21:48:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414370+0000", 2024-09-15T21:48:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-15T21:48:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414379+0000", 2024-09-15T21:48:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2449999999999998e-06 2024-09-15T21:48:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414477+0000", 2024-09-15T21:48:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8968000000000005e-05 2024-09-15T21:48:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414504+0000", 2024-09-15T21:48:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6899000000000001e-05 2024-09-15T21:48:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416837+0000", 2024-09-15T21:48:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023327130000000001 2024-09-15T21:48:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416957+0000", 2024-09-15T21:48:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000119796 2024-09-15T21:48:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.1.10:8 2.b e22/19 2:d70a01c6:::60c.00000000:head v 22'4, mlcod=22'1)", 2024-09-15T21:48:03.563 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:48:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 89.089825633999993, 2024-09-15T21:48:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027633890000000002, 2024-09-15T21:48:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:48:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:48:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457923+0000", 2024-09-15T21:48:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.136e-06 2024-09-15T21:48:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457931+0000", 2024-09-15T21:48:03.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1529999999999997e-06 2024-09-15T21:48:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457946+0000", 2024-09-15T21:48:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4482e-05 2024-09-15T21:48:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457956+0000", 2024-09-15T21:48:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0774999999999999e-05 2024-09-15T21:48:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.566 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458016+0000", 2024-09-15T21:48:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9247999999999999e-05 2024-09-15T21:48:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458036+0000", 2024-09-15T21:48:03.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0920999999999998e-05 2024-09-15T21:48:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460636+0000", 2024-09-15T21:48:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025992089999999999 2024-09-15T21:48:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460685+0000", 2024-09-15T21:48:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9465000000000002e-05 2024-09-15T21:48:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.18 epoch 22/19 rep_tid 5 entries 22'1 (0'0) error 2:1d00e16c:::201.00000005:head by mds.1.10:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:48:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:48:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 89.088133502999995, 2024-09-15T21:48:03.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003509599, 2024-09-15T21:48:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:48:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:48:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:48:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459615+0000", 2024-09-15T21:48:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4200000000000005e-07 2024-09-15T21:48:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459626+0000", 2024-09-15T21:48:03.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1456e-05 2024-09-15T21:48:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459650+0000", 2024-09-15T21:48:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.429e-05 2024-09-15T21:48:03.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459664+0000", 2024-09-15T21:48:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3805000000000001e-05 2024-09-15T21:48:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459784+0000", 2024-09-15T21:48:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000119614 2024-09-15T21:48:03.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463123+0000", 2024-09-15T21:48:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033395920000000002 2024-09-15T21:48:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.575 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:7 2.13 e22/19 2:cdf3ac81:::616.00000000:head v 22'5, mlcod=22'3)", 2024-09-15T21:48:03.575 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:48:03.575 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 83.070352725000006, 2024-09-15T21:48:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002138615, 2024-09-15T21:48:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:48:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:48:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477396+0000", 2024-09-15T21:48:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4190000000000001e-06 2024-09-15T21:48:03.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477405+0000", 2024-09-15T21:48:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2320000000000007e-06 2024-09-15T21:48:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477419+0000", 2024-09-15T21:48:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3589e-05 2024-09-15T21:48:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477428+0000", 2024-09-15T21:48:03.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4779999999999997e-06 2024-09-15T21:48:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477493+0000", 2024-09-15T21:48:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4546000000000002e-05 2024-09-15T21:48:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477511+0000", 2024-09-15T21:48:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7793000000000001e-05 2024-09-15T21:48:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:03.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479481+0000", 2024-09-15T21:48:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019701660000000002 2024-09-15T21:48:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479533+0000", 2024-09-15T21:48:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2392000000000003e-05 2024-09-15T21:48:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:26 2.11 e22/19 2:893cd530:::mds2_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-15T21:48:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:48:03.580 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 83.070205798999993, 2024-09-15T21:48:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020450450000000001, 2024-09-15T21:48:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:48:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:48:03.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:48:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9699999999999995e-07 2024-09-15T21:48:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477555+0000", 2024-09-15T21:48:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3144e-05 2024-09-15T21:48:03.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477565+0000", 2024-09-15T21:48:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8579999999999995e-06 2024-09-15T21:48:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477573+0000", 2024-09-15T21:48:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4869999999999996e-06 2024-09-15T21:48:03.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477632+0000", 2024-09-15T21:48:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9178999999999997e-05 2024-09-15T21:48:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477654+0000", 2024-09-15T21:48:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2234000000000001e-05 2024-09-15T21:48:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479526+0000", 2024-09-15T21:48:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018717600000000001 2024-09-15T21:48:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479587+0000", 2024-09-15T21:48:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0686000000000001e-05 2024-09-15T21:48:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:22 2.1a e22/19 2:5fd68f60:::61d.00000000:head v 22'3, mlcod=0'0)", 2024-09-15T21:48:03.585 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:48:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 83.069845744000006, 2024-09-15T21:48:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023694380000000002, 2024-09-15T21:48:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:48:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:48:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:48:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4899999999999995e-07 2024-09-15T21:48:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477910+0000", 2024-09-15T21:48:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:48:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477920+0000", 2024-09-15T21:48:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9250000000000007e-06 2024-09-15T21:48:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477927+0000", 2024-09-15T21:48:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0940000000000002e-06 2024-09-15T21:48:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477985+0000", 2024-09-15T21:48:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8451e-05 2024-09-15T21:48:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478005+0000", 2024-09-15T21:48:03.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0064e-05 2024-09-15T21:48:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480223+0000", 2024-09-15T21:48:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022177910000000002 2024-09-15T21:48:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480271+0000", 2024-09-15T21:48:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8164999999999997e-05 2024-09-15T21:48:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.590 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:8 2.13 e22/19 2:cdf3ac81:::616.00000000:head v 22'6, mlcod=22'3)", 2024-09-15T21:48:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:48:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 83.069646785000003, 2024-09-15T21:48:03.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021760249999999998, 2024-09-15T21:48:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:48:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:48:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478102+0000", 2024-09-15T21:48:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3099999999999996e-07 2024-09-15T21:48:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478124+0000", 2024-09-15T21:48:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2790999999999999e-05 2024-09-15T21:48:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478136+0000", 2024-09-15T21:48:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1922999999999999e-05 2024-09-15T21:48:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478144+0000", 2024-09-15T21:48:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9300000000000003e-06 2024-09-15T21:48:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478192+0000", 2024-09-15T21:48:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8155999999999998e-05 2024-09-15T21:48:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478209+0000", 2024-09-15T21:48:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6816e-05 2024-09-15T21:48:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480243+0000", 2024-09-15T21:48:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020336199999999999 2024-09-15T21:48:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480277+0000", 2024-09-15T21:48:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3858e-05 2024-09-15T21:48:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.11 epoch 22/19 rep_tid 19 entries 22'2 (0'0) error 2:8975f766:::202.00000001:head by mds.2.12:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:48:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:48:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 83.069520847000007, 2024-09-15T21:48:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023882299999999999, 2024-09-15T21:48:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.596 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:48:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:48:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:48:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:48:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2399999999999998e-07 2024-09-15T21:48:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478236+0000", 2024-09-15T21:48:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5299999999999996e-06 2024-09-15T21:48:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478248+0000", 2024-09-15T21:48:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2806000000000001e-05 2024-09-15T21:48:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478264+0000", 2024-09-15T21:48:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5468000000000002e-05 2024-09-15T21:48:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478339+0000", 2024-09-15T21:48:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4851000000000006e-05 2024-09-15T21:48:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480615+0000", 2024-09-15T21:48:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022760509999999999 2024-09-15T21:48:03.599 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:3 2.1a e22/19 2:5fdc322d:::61e.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:48:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:48:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 78.001336245000005, 2024-09-15T21:48:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023961220000000001, 2024-09-15T21:48:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:48:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:48:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546413+0000", 2024-09-15T21:48:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.812e-06 2024-09-15T21:48:03.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546424+0000", 2024-09-15T21:48:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1313e-05 2024-09-15T21:48:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546444+0000", 2024-09-15T21:48:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9706000000000001e-05 2024-09-15T21:48:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546453+0000", 2024-09-15T21:48:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:48:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546519+0000", 2024-09-15T21:48:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5710999999999995e-05 2024-09-15T21:48:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546539+0000", 2024-09-15T21:48:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0275000000000002e-05 2024-09-15T21:48:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548763+0000", 2024-09-15T21:48:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002223226 2024-09-15T21:48:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548807+0000", 2024-09-15T21:48:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4598000000000002e-05 2024-09-15T21:48:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:19 2.11 e22/19 2:890f50f7:::626.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:48:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:48:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 78.001039085000002, 2024-09-15T21:48:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022671280000000002, 2024-09-15T21:48:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.607 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:48:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:48:03.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546709+0000", 2024-09-15T21:48:03.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4499999999999997e-07 2024-09-15T21:48:03.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546723+0000", 2024-09-15T21:48:03.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4623999999999999e-05 2024-09-15T21:48:03.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546734+0000", 2024-09-15T21:48:03.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0331e-05 2024-09-15T21:48:03.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546741+0000", 2024-09-15T21:48:03.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.075e-06 2024-09-15T21:48:03.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546954+0000", 2024-09-15T21:48:03.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021275600000000001 2024-09-15T21:48:03.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:03.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546971+0000", 2024-09-15T21:48:03.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7503000000000002e-05 2024-09-15T21:48:03.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:03.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548926+0000", 2024-09-15T21:48:03.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019550129999999998 2024-09-15T21:48:03.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548975+0000", 2024-09-15T21:48:03.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9280999999999998e-05 2024-09-15T21:48:03.617 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.617 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.d epoch 22/19 rep_tid 18 entries 22'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.14:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:48:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:48:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 78.001032734999995, 2024-09-15T21:48:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021738489999999998, 2024-09-15T21:48:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:48:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:48:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:48:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546716+0000", 2024-09-15T21:48:03.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.615e-06 2024-09-15T21:48:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546730+0000", 2024-09-15T21:48:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3519999999999999e-05 2024-09-15T21:48:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546753+0000", 2024-09-15T21:48:03.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3643999999999999e-05 2024-09-15T21:48:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546770+0000", 2024-09-15T21:48:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6518999999999999e-05 2024-09-15T21:48:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546900+0000", 2024-09-15T21:48:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013049099999999999 2024-09-15T21:48:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548888+0000", 2024-09-15T21:48:03.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019880599999999998 2024-09-15T21:48:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:4 2.1a e22/19 2:5fdc322d:::61e.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:48:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:48:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 78.000924087000001, 2024-09-15T21:48:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002968769, 2024-09-15T21:48:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:48:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:48:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546824+0000", 2024-09-15T21:48:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6499999999999999e-07 2024-09-15T21:48:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546831+0000", 2024-09-15T21:48:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9940000000000003e-06 2024-09-15T21:48:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546840+0000", 2024-09-15T21:48:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3689999999999996e-06 2024-09-15T21:48:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546847+0000", 2024-09-15T21:48:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0589999999999997e-06 2024-09-15T21:48:03.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547709+0000", 2024-09-15T21:48:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00086144699999999999 2024-09-15T21:48:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547730+0000", 2024-09-15T21:48:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1542000000000001e-05 2024-09-15T21:48:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549750+0000", 2024-09-15T21:48:03.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002020165 2024-09-15T21:48:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549792+0000", 2024-09-15T21:48:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1628000000000002e-05 2024-09-15T21:48:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:20 2.11 e22/19 2:890f50f7:::626.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:48:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:48:03.626 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.999899479000007, 2024-09-15T21:48:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017734579999999999, 2024-09-15T21:48:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:48:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:48:03.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:48:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:48:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547856+0000", 2024-09-15T21:48:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7999999999999999e-06 2024-09-15T21:48:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547867+0000", 2024-09-15T21:48:03.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0971999999999999e-05 2024-09-15T21:48:03.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547881+0000", 2024-09-15T21:48:03.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4168999999999999e-05 2024-09-15T21:48:03.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.629 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547935+0000", 2024-09-15T21:48:03.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3721999999999998e-05 2024-09-15T21:48:03.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:03.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547976+0000", 2024-09-15T21:48:03.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1248999999999998e-05 2024-09-15T21:48:03.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:03.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549559+0000", 2024-09-15T21:48:03.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001582815 2024-09-15T21:48:03.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549621+0000", 2024-09-15T21:48:03.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2156999999999995e-05 2024-09-15T21:48:03.630 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.631 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.631 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.631 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:13 2.1a e22/19 2:594e21d0:::623.00000000:head v 22'6, mlcod=22'2)", 2024-09-15T21:48:03.631 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:48:03.631 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.999834483000001, 2024-09-15T21:48:03.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019085300000000001, 2024-09-15T21:48:03.631 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.631 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.631 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.631 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:48:03.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:48:03.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547914+0000", 2024-09-15T21:48:03.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1500000000000004e-07 2024-09-15T21:48:03.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547920+0000", 2024-09-15T21:48:03.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5239999999999997e-06 2024-09-15T21:48:03.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.633 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.633 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.633 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547930+0000", 2024-09-15T21:48:03.633 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5109999999999992e-06 2024-09-15T21:48:03.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547937+0000", 2024-09-15T21:48:03.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5410000000000003e-06 2024-09-15T21:48:03.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548518+0000", 2024-09-15T21:48:03.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00058074299999999999 2024-09-15T21:48:03.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.634 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.634 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:03.634 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548538+0000", 2024-09-15T21:48:03.634 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9959e-05 2024-09-15T21:48:03.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:03.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549795+0000", 2024-09-15T21:48:03.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012570120000000001 2024-09-15T21:48:03.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.635 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.635 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549821+0000", 2024-09-15T21:48:03.635 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6625e-05 2024-09-15T21:48:03.635 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.635 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.635 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.635 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.636 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:23 2.18 e22/19 2:1c1db442:::103.00000000.inode:head v 22'2, mlcod=0'0)", 2024-09-15T21:48:03.636 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:48:03.636 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.999773064999999, 2024-09-15T21:48:03.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002713198, 2024-09-15T21:48:03.636 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.636 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.636 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.636 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.636 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.636 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:48:03.636 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.636 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:48:03.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.637 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547975+0000", 2024-09-15T21:48:03.637 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3900000000000004e-07 2024-09-15T21:48:03.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.637 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.637 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547983+0000", 2024-09-15T21:48:03.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6690000000000006e-06 2024-09-15T21:48:03.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547994+0000", 2024-09-15T21:48:03.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1503e-05 2024-09-15T21:48:03.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.638 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.638 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548002+0000", 2024-09-15T21:48:03.638 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1340000000000004e-06 2024-09-15T21:48:03.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.638 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548098+0000", 2024-09-15T21:48:03.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5791000000000005e-05 2024-09-15T21:48:03.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:03.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548118+0000", 2024-09-15T21:48:03.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0129e-05 2024-09-15T21:48:03.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.639 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.639 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:03.639 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550643+0000", 2024-09-15T21:48:03.639 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002524728 2024-09-15T21:48:03.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.640 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.640 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550688+0000", 2024-09-15T21:48:03.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4404999999999999e-05 2024-09-15T21:48:03.640 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.640 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.640 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.640 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.640 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:14 2.1a e22/19 2:594e21d0:::623.00000000:head v 22'7, mlcod=22'2)", 2024-09-15T21:48:03.640 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:48:03.640 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.999758104999998, 2024-09-15T21:48:03.640 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024358579999999999, 2024-09-15T21:48:03.640 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.641 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.641 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.641 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:48:03.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.641 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.641 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.641 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:48:03.641 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547990+0000", 2024-09-15T21:48:03.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5799999999999999e-07 2024-09-15T21:48:03.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.642 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.642 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548011+0000", 2024-09-15T21:48:03.642 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0942999999999999e-05 2024-09-15T21:48:03.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.642 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548020+0000", 2024-09-15T21:48:03.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2380000000000003e-06 2024-09-15T21:48:03.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548027+0000", 2024-09-15T21:48:03.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6429999999999998e-06 2024-09-15T21:48:03.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.643 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.643 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.643 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548848+0000", 2024-09-15T21:48:03.643 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082167999999999998 2024-09-15T21:48:03.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:03.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548868+0000", 2024-09-15T21:48:03.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9791000000000001e-05 2024-09-15T21:48:03.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:03.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550387+0000", 2024-09-15T21:48:03.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001519322 2024-09-15T21:48:03.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.644 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.644 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.644 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550425+0000", 2024-09-15T21:48:03.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7682999999999998e-05 2024-09-15T21:48:03.645 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.645 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.645 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.645 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:35 2.f 2:f426ad4f:::mds2_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:03.645 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:48:03.645 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 77.809646420000007, 2024-09-15T21:48:03.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017328949999999999, 2024-09-15T21:48:03.645 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.645 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.645 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.645 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:48:03.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:48:03.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738103+0000", 2024-09-15T21:48:03.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0020000000000001e-06 2024-09-15T21:48:03.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738112+0000", 2024-09-15T21:48:03.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2220000000000008e-06 2024-09-15T21:48:03.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738127+0000", 2024-09-15T21:48:03.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5016e-05 2024-09-15T21:48:03.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738137+0000", 2024-09-15T21:48:03.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0301e-05 2024-09-15T21:48:03.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738216+0000", 2024-09-15T21:48:03.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8356000000000007e-05 2024-09-15T21:48:03.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:03.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738340+0000", 2024-09-15T21:48:03.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012372800000000001 2024-09-15T21:48:03.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:03.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738453+0000", 2024-09-15T21:48:03.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000113785 2024-09-15T21:48:03.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:03.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739115+0000", 2024-09-15T21:48:03.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00066178500000000002 2024-09-15T21:48:03.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:03.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739745+0000", 2024-09-15T21:48:03.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00063017900000000003 2024-09-15T21:48:03.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:03.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739783+0000", 2024-09-15T21:48:03.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7941000000000001e-05 2024-09-15T21:48:03.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739834+0000", 2024-09-15T21:48:03.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.058e-05 2024-09-15T21:48:03.650 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.650 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.650 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.651 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.4.16:13 2.13 e22/19 2:c81eaac8:::62d.00000000:head v 22'8, mlcod=22'6)", 2024-09-15T21:48:03.651 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:48:03.651 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.982241361000007, 2024-09-15T21:48:03.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002151778, 2024-09-15T21:48:03.651 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.651 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.651 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:03.651 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:48:03.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:48:03.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565507+0000", 2024-09-15T21:48:03.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3340000000000001e-06 2024-09-15T21:48:03.652 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.652 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.652 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565518+0000", 2024-09-15T21:48:03.652 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0339999999999999e-05 2024-09-15T21:48:03.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565536+0000", 2024-09-15T21:48:03.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8647e-05 2024-09-15T21:48:03.653 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.653 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.653 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.653 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565545+0000", 2024-09-15T21:48:03.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1430000000000004e-06 2024-09-15T21:48:03.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565611+0000", 2024-09-15T21:48:03.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5129000000000006e-05 2024-09-15T21:48:03.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:03.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565634+0000", 2024-09-15T21:48:03.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3173e-05 2024-09-15T21:48:03.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:03.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567561+0000", 2024-09-15T21:48:03.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019268180000000001 2024-09-15T21:48:03.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567658+0000", 2024-09-15T21:48:03.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7194000000000005e-05 2024-09-15T21:48:03.655 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.655 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.655 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.656 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.d epoch 22/19 rep_tid 24 entries 22'3 (0'0) error 2:b2057508:::204.00000002:head by mds.4.16:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 22'1)", 2024-09-15T21:48:03.656 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:48:03.656 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 71.982080499000006, 2024-09-15T21:48:03.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026915049999999999, 2024-09-15T21:48:03.656 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:03.656 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:03.656 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:48:03.656 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:03.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:03.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:48:03.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:03.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:48:03.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:03.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:03.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565668+0000", 2024-09-15T21:48:03.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.113e-06 2024-09-15T21:48:03.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:03.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565684+0000", 2024-09-15T21:48:03.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5943999999999998e-05 2024-09-15T21:48:03.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:03.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565705+0000", 2024-09-15T21:48:03.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0761999999999999e-05 2024-09-15T21:48:03.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:03.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565724+0000", 2024-09-15T21:48:03.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9208e-05 2024-09-15T21:48:03.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:03.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565817+0000", 2024-09-15T21:48:03.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3218000000000001e-05 2024-09-15T21:48:03.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:03.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:03.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:03.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568358+0000", 2024-09-15T21:48:03.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00254126 2024-09-15T21:48:03.659 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.659 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.659 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.659 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:03.659 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:03.659 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:48:03.661 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:48:03.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.568+0000 7fe7a67fc640 1 -- 172.21.15.100:0/1044801215 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "noscrub"}]=0 noscrub is set v52) ==== 87+0+0 (secure 0 0 0) 0x7fe7a807e370 con 0x7fe7b0155440 2024-09-15T21:48:03.663 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is set 2024-09-15T21:48:03.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.572+0000 7fe7b7ec9640 1 -- 172.21.15.100:0/1044801215 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe79406c2b0 msgr2=0x7fe79406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:03.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.572+0000 7fe7b7ec9640 1 --2- 172.21.15.100:0/1044801215 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe79406c2b0 0x7fe79406e770 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac000f60 tx=0x7fe7ac007d50 comp rx=0 tx=0).stop 2024-09-15T21:48:03.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.572+0000 7fe7b7ec9640 1 -- 172.21.15.100:0/1044801215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7b0155440 msgr2=0x7fe7b014a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:03.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.572+0000 7fe7b7ec9640 1 --2- 172.21.15.100:0/1044801215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7b0155440 0x7fe7b014a1c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8009d90 tx=0x7fe7a8004a40 comp rx=0 tx=0).stop 2024-09-15T21:48:03.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.572+0000 7fe7b7ec9640 1 -- 172.21.15.100:0/1044801215 shutdown_connections 2024-09-15T21:48:03.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.572+0000 7fe7b7ec9640 1 --2- 172.21.15.100:0/1044801215 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe79406c2b0 0x7fe79406e770 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:03.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.572+0000 7fe7b7ec9640 1 --2- 172.21.15.100:0/1044801215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7b00cb7d0 0x7fe7b0149c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:03.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.572+0000 7fe7b7ec9640 1 --2- 172.21.15.100:0/1044801215 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe7b0155440 0x7fe7b014a1c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:03.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.572+0000 7fe7b7ec9640 1 --2- 172.21.15.100:0/1044801215 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe7b006d0e0 0x7fe7b0141290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:03.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.572+0000 7fe7b7ec9640 1 -- 172.21.15.100:0/1044801215 >> 172.21.15.100:0/1044801215 conn(0x7fe7b00680d0 msgr2=0x7fe7b0159aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:03.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.572+0000 7fe7b7ec9640 1 -- 172.21.15.100:0/1044801215 shutdown_connections 2024-09-15T21:48:03.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.572+0000 7fe7b7ec9640 1 -- 172.21.15.100:0/1044801215 wait complete. 2024-09-15T21:48:03.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.612+0000 7fd3a2cdd640 1 Processor -- start 2024-09-15T21:48:03.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.612+0000 7fd3a2cdd640 1 -- start start 2024-09-15T21:48:03.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.613+0000 7fd3a2cdd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd39c155670 0x7fd39c15e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:03.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.613+0000 7fd3a2cdd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd39c15bb60 0x7fd39c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:03.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.613+0000 7fd3a2cdd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd39c15edb0 con 0x7fd39c155290 2024-09-15T21:48:03.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.613+0000 7fd3a2cdd640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fd39c15f090 con 0x7fd39c155670 2024-09-15T21:48:03.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.613+0000 7fd3a2cdd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd39c15f370 con 0x7fd39c15bb60 2024-09-15T21:48:03.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.613+0000 7fd3a0a79640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd39c155290 0x7fd39c0c65b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57442/0 (socket says 172.21.15.100:57442) 2024-09-15T21:48:03.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.613+0000 7fd3a0a79640 1 -- 172.21.15.100:0/273725633 learned_addr learned my addr 172.21.15.100:0/273725633 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:03.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd39b7fe640 1 -- 172.21.15.100:0/273725633 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1260650524 0 0) 0x7fd39c15edb0 con 0x7fd39c155290 2024-09-15T21:48:03.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd39bfff640 1 --2- 172.21.15.100:0/273725633 >> v2:172.21.15.122:3300/0 conn(0x7fd39c155670 0x7fd39c15e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:03.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd3a127a640 1 --2- 172.21.15.100:0/273725633 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd39c15bb60 0x7fd39c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:03.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd39b7fe640 1 -- 172.21.15.100:0/273725633 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd38c003520 con 0x7fd39c155290 2024-09-15T21:48:03.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd39b7fe640 1 -- 172.21.15.100:0/273725633 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3479450709 0 0) 0x7fd38c003520 con 0x7fd39c155290 2024-09-15T21:48:03.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd39b7fe640 1 -- 172.21.15.100:0/273725633 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd39c15bb60 msgr2=0x7fd39c06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:03.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd39b7fe640 1 --2- 172.21.15.100:0/273725633 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd39c15bb60 0x7fd39c06d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:03.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd39b7fe640 1 -- 172.21.15.100:0/273725633 >> v2:172.21.15.122:3300/0 conn(0x7fd39c155670 msgr2=0x7fd39c15e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:03.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd39b7fe640 1 --2- 172.21.15.100:0/273725633 >> v2:172.21.15.122:3300/0 conn(0x7fd39c155670 0x7fd39c15e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:03.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd39b7fe640 1 -- 172.21.15.100:0/273725633 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39c0c5db0 con 0x7fd39c155290 2024-09-15T21:48:03.671 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd39b7fe640 1 -- 172.21.15.100:0/273725633 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7fd390001670 con 0x7fd39c155290 2024-09-15T21:48:03.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd39b7fe640 1 -- 172.21.15.100:0/273725633 >> v1:172.21.15.100:6789/0 conn(0x7fd39c155290 legacy=0x7fd39c0c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:03.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd39b7fe640 1 --2- 172.21.15.100:0/273725633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd38c0039b0 0x7fd38c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:03.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd39b7fe640 1 --2- 172.21.15.100:0/273725633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38c004800 0x7fd38c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:03.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd39b7fe640 1 --2- 172.21.15.100:0/273725633 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd39c15bb60 0x7fd38c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:03.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd39b7fe640 1 -- 172.21.15.100:0/273725633 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c15f090 con 0x7fd38c004800 2024-09-15T21:48:03.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd39b7fe640 1 -- 172.21.15.100:0/273725633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c15f370 con 0x7fd38c0039b0 2024-09-15T21:48:03.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd39b7fe640 1 -- 172.21.15.100:0/273725633 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd38c00b9c0 con 0x7fd39c15bb60 2024-09-15T21:48:03.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd3a0a79640 1 --2- 172.21.15.100:0/273725633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd38c0039b0 0x7fd38c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:03.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd39bfff640 1 --2- 172.21.15.100:0/273725633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38c004800 0x7fd38c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:03.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd3a127a640 1 --2- 172.21.15.100:0/273725633 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd39c15bb60 0x7fd38c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:03.672 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd3a0a79640 1 -- 172.21.15.100:0/273725633 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd39c15bb60 msgr2=0x7fd38c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:03.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd3a0a79640 1 --2- 172.21.15.100:0/273725633 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd39c15bb60 0x7fd38c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:03.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd3a0a79640 1 -- 172.21.15.100:0/273725633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38c004800 msgr2=0x7fd38c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:03.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd3a0a79640 1 --2- 172.21.15.100:0/273725633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38c004800 0x7fd38c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:03.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd3a0a79640 1 -- 172.21.15.100:0/273725633 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38c003520 con 0x7fd38c0039b0 2024-09-15T21:48:03.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd39bfff640 1 --2- 172.21.15.100:0/273725633 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd38c004800 0x7fd38c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:48:03.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.614+0000 7fd3a127a640 1 --2- 172.21.15.100:0/273725633 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd39c15bb60 0x7fd38c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:48:03.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.615+0000 7fd3a0a79640 1 --2- 172.21.15.100:0/273725633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd38c0039b0 0x7fd38c003dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd390007fd0 tx=0x7fd38c005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=316d862fd5540fe0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:03.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.616+0000 7fd39b7fe640 1 -- 172.21.15.100:0/273725633 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd390007d50 con 0x7fd38c0039b0 2024-09-15T21:48:03.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.616+0000 7fd39b7fe640 1 -- 172.21.15.100:0/273725633 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd390005ad0 con 0x7fd38c0039b0 2024-09-15T21:48:03.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.616+0000 7fd39b7fe640 1 -- 172.21.15.100:0/273725633 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd390007070 con 0x7fd38c0039b0 2024-09-15T21:48:03.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.616+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/273725633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd38c0039b0 msgr2=0x7fd38c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:03.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.616+0000 7fd3a2cdd640 1 --2- 172.21.15.100:0/273725633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd38c0039b0 0x7fd38c003dd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd390007fd0 tx=0x7fd38c005830 comp rx=0 tx=0).stop 2024-09-15T21:48:03.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.616+0000 7fd3a127a640 1 -- 172.21.15.100:0/273725633 reap_dead start 2024-09-15T21:48:03.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.616+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/273725633 shutdown_connections 2024-09-15T21:48:03.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.616+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/273725633 >> 172.21.15.100:0/273725633 conn(0x7fd39c073310 msgr2=0x7fd39c073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:03.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.616+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/273725633 shutdown_connections 2024-09-15T21:48:03.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.616+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/273725633 wait complete. 2024-09-15T21:48:03.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.617+0000 7fd3a2cdd640 1 Processor -- start 2024-09-15T21:48:03.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.618+0000 7fd3a2cdd640 1 -- start start 2024-09-15T21:48:03.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.618+0000 7fd3a2cdd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd38c0039b0 0x7fd39c1f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:03.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.618+0000 7fd3a2cdd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd39c1f4640 0x7fd39c159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:03.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.618+0000 7fd3a2cdd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39c15a070 0x7fd39c155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:03.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.618+0000 7fd3a2cdd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c15bf90 con 0x7fd39c1f4640 2024-09-15T21:48:03.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.618+0000 7fd3a2cdd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd39c0b39e0 con 0x7fd39c15a070 2024-09-15T21:48:03.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.618+0000 7fd3a2cdd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd39c15c590 con 0x7fd38c0039b0 2024-09-15T21:48:03.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.618+0000 7fd39bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd39c1f4640 0x7fd39c159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:03.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.618+0000 7fd39bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd39c1f4640 0x7fd39c159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57776/0 (socket says 172.21.15.100:57776) 2024-09-15T21:48:03.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.618+0000 7fd39bfff640 1 -- 172.21.15.100:0/3971101975 learned_addr learned my addr 172.21.15.100:0/3971101975 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:03.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.619+0000 7fd3a0a79640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd38c0039b0 0x7fd39c1f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:03.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.619+0000 7fd3a127a640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39c15a070 0x7fd39c155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:03.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.619+0000 7fd39bfff640 1 -- 172.21.15.100:0/3971101975 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd38c0039b0 msgr2=0x7fd39c1f4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:03.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.619+0000 7fd39bfff640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd38c0039b0 0x7fd39c1f4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:03.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.619+0000 7fd39bfff640 1 -- 172.21.15.100:0/3971101975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39c15a070 msgr2=0x7fd39c155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:03.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.619+0000 7fd39bfff640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd39c15a070 0x7fd39c155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:03.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.619+0000 7fd39bfff640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd39c1f4b70 con 0x7fd39c1f4640 2024-09-15T21:48:03.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.619+0000 7fd3a0a79640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd38c0039b0 0x7fd39c1f4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:48:03.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.619+0000 7fd39bfff640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd39c1f4640 0x7fd39c159b30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd388011a70 tx=0x7fd388011f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:03.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.619+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38800fbb0 con 0x7fd39c1f4640 2024-09-15T21:48:03.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.619+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd38800fd50 con 0x7fd39c1f4640 2024-09-15T21:48:03.675 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.619+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd39c14f6b0 con 0x7fd39c1f4640 2024-09-15T21:48:03.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.619+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd388014640 con 0x7fd39c1f4640 2024-09-15T21:48:03.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.619+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd39c155a50 con 0x7fd39c1f4640 2024-09-15T21:48:03.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.620+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fd39c15e690 con 0x7fd39c1f4640 2024-09-15T21:48:03.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.621+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd388012040 con 0x7fd39c1f4640 2024-09-15T21:48:03.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.621+0000 7fd3997fa640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd38006c2b0 0x7fd38006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:03.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.621+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6060+0+0 (secure 0 0 0) 0x7fd38808bc50 con 0x7fd39c1f4640 2024-09-15T21:48:03.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.621+0000 7fd3a0a79640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd38006c2b0 0x7fd38006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:03.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.621+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v52) ==== 54+0+16 (secure 0 0 0) 0x7fd388019030 con 0x7fd39c1f4640 2024-09-15T21:48:03.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.622+0000 7fd3a2cdd640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fd39c156360 0x7fd39c14d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:03.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.622+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd39c15a520 con 0x7fd39c156360 2024-09-15T21:48:03.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.622+0000 7fd3a127a640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fd39c156360 0x7fd39c14d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:03.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.622+0000 7fd3a0a79640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd38006c2b0 0x7fd38006e770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd390033000 tx=0x7fd390009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:03.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.622+0000 7fd3a127a640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fd39c156360 0x7fd39c14d600 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:03.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.623+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd39c15a520 con 0x7fd39c156360 2024-09-15T21:48:03.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.637+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd39c159750 con 0x7fd39c156360 2024-09-15T21:48:03.677 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.638+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd39c159750 con 0x7fd39c156360 2024-09-15T21:48:03.677 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:48:03.678 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:48:03.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.638+0000 7fd3a2cdd640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fd39c156740 0x7fd39c14e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:03.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.638+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fd39c14fbd0 con 0x7fd39c156740 2024-09-15T21:48:03.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.639+0000 7fd3a0a79640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fd39c156740 0x7fd39c14e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:03.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.639+0000 7fd3a0a79640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fd39c156740 0x7fd39c14e050 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:03.678 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.639+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd39c14fbd0 con 0x7fd39c156740 2024-09-15T21:48:03.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.654+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd39c1f0b70 con 0x7fd39c156740 2024-09-15T21:48:03.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.655+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd39c1f0b70 con 0x7fd39c156740 2024-09-15T21:48:03.679 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:48:03.680 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:48:03.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.655+0000 7fd3a2cdd640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fd39c062630 0x7fd39c062a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:03.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.655+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fd39c0d0d60 con 0x7fd39c062630 2024-09-15T21:48:03.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.655+0000 7fd39bfff640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fd39c062630 0x7fd39c062a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:03.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.656+0000 7fd39bfff640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fd39c062630 0x7fd39c062a70 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:03.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.656+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd39c0d0d60 con 0x7fd39c062630 2024-09-15T21:48:03.680 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:48:03.659+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:48:03.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.670+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd39c062fb0 con 0x7fd39c062630 2024-09-15T21:48:03.685 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.671+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd39c062fb0 con 0x7fd39c062630 2024-09-15T21:48:03.685 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:48:03.686 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:48:03.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.672+0000 7fd3a2cdd640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fd39c0cb8f0 0x7fd39c0cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:03.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.672+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fd39c063580 con 0x7fd39c0cb8f0 2024-09-15T21:48:03.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.672+0000 7fd3a127a640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fd39c0cb8f0 0x7fd39c0cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:03.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.672+0000 7fd3a127a640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fd39c0cb8f0 0x7fd39c0cbd10 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:03.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.673+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd39c063580 con 0x7fd39c0cb8f0 2024-09-15T21:48:03.687 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.686+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd39c02c640 con 0x7fd39c0cb8f0 2024-09-15T21:48:03.689 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.688+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd39c02c640 con 0x7fd39c0cb8f0 2024-09-15T21:48:03.689 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:48:03.690 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:48:03.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.688+0000 7fd3a2cdd640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fd39c02cd60 0x7fd39c02d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:03.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.688+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fd39c02d700 con 0x7fd39c02cd60 2024-09-15T21:48:03.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.688+0000 7fd3a0a79640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fd39c02cd60 0x7fd39c02d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:03.690 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.689+0000 7fd3a0a79640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fd39c02cd60 0x7fd39c02d1c0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:03.691 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.689+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd39c02d700 con 0x7fd39c02cd60 2024-09-15T21:48:03.704 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.702+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd39c02d700 con 0x7fd39c02cd60 2024-09-15T21:48:03.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.704+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd39c02d700 con 0x7fd39c02cd60 2024-09-15T21:48:03.706 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:48:03.707 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:48:03.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.704+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=53}) -- 0x7fd39c02d700 con 0x7fd39c1f4640 2024-09-15T21:48:03.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.704+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd39c02da90 con 0x7fd39c1f4640 2024-09-15T21:48:03.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.704+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fd388071050 con 0x7fd39c1f4640 2024-09-15T21:48:03.707 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:48:03.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.705+0000 7fd3a2cdd640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fd39c205cf0 0x7fd39c2080e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:03.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.705+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fd39c20aeb0 con 0x7fd39c205cf0 2024-09-15T21:48:03.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.705+0000 7fd39bfff640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fd39c205cf0 0x7fd39c2080e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:03.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.705+0000 7fd39bfff640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fd39c205cf0 0x7fd39c2080e0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:03.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.706+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd39c20aeb0 con 0x7fd39c205cf0 2024-09-15T21:48:03.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.719+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd39c208620 con 0x7fd39c205cf0 2024-09-15T21:48:03.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.721+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd39c208620 con 0x7fd39c205cf0 2024-09-15T21:48:03.723 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:48:03.724 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:48:03.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.721+0000 7fd3a2cdd640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fd39c208d40 0x7fd39c2091a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:03.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.721+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fd39c20e2a0 con 0x7fd39c208d40 2024-09-15T21:48:03.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.722+0000 7fd3a127a640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fd39c208d40 0x7fd39c2091a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:03.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.722+0000 7fd3a127a640 1 --2- 172.21.15.100:0/3971101975 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fd39c208d40 0x7fd39c2091a0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:03.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.723+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd39c20e2a0 con 0x7fd39c208d40 2024-09-15T21:48:03.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.736+0000 7fd3a2cdd640 1 -- 172.21.15.100:0/3971101975 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7fd39c2097f0 con 0x7fd39c208d40 2024-09-15T21:48:03.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:03.738+0000 7fd3997fa640 1 -- 172.21.15.100:0/3971101975 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fd39c2097f0 con 0x7fd39c208d40 2024-09-15T21:48:03.740 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:48:03.740 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:48:03.743 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:48:03.744 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:48:03.772 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:03.772 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:03.781 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:48:03.784 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:48:03.783+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:48:03.862 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:48:03.860+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:48:03.884 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:03.885 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:03.893 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:48:03.947 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6] out_osds: [7] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-09-15T21:48:03.947 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:48:03.947 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-15T21:48:03.947 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-15T21:48:03.947 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-15T21:48:03.948 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-09-15T21:48:03.963 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:48:03.961+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:48:03.997 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:03.997 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:04.006 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:48:04.055 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:04.055 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-15T21:48:04.055 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:04.090 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:48:04.089+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:48:04.150 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:04.151 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:04.159 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:48:04.165 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:48:04.163+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:48:04.251 INFO:tasks.workunit.client.0.smithi100.stderr:Updating files: 97% (12518/12889) Updating files: 98% (12632/12889) Updating files: 99% (12761/12889) Updating files: 100% (12889/12889) Updating files: 100% (12889/12889), done. 2024-09-15T21:48:04.265 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:48:04.263+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:48:04.304 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:04.304 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:04.313 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:48:04.366 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:48:04.364+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:48:04.456 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:04.456 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:04.465 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:48:04.469 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:48:04.467+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:48:04.569 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:48:04.568+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:48:04.573 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:04.573 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:04.582 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:48:04.673 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:48:04.672+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:48:04.688 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:04.688 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:04.698 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:48:04.774 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:48:04.773+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:48:04.803 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:04.803 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:04.812 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:48:04.830 INFO:tasks.workunit.client.0.smithi100.stderr:Note: switching to 'ad35db06e8924912f01e32b35576297c0f1352b4'. 2024-09-15T21:48:04.830 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-09-15T21:48:04.830 INFO:tasks.workunit.client.0.smithi100.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-15T21:48:04.830 INFO:tasks.workunit.client.0.smithi100.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-15T21:48:04.830 INFO:tasks.workunit.client.0.smithi100.stderr:state without impacting any branches by switching back to a branch. 2024-09-15T21:48:04.830 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-09-15T21:48:04.830 INFO:tasks.workunit.client.0.smithi100.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-15T21:48:04.830 INFO:tasks.workunit.client.0.smithi100.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-15T21:48:04.830 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-09-15T21:48:04.830 INFO:tasks.workunit.client.0.smithi100.stderr: git switch -c 2024-09-15T21:48:04.831 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-09-15T21:48:04.831 INFO:tasks.workunit.client.0.smithi100.stderr:Or undo this operation with: 2024-09-15T21:48:04.831 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-09-15T21:48:04.831 INFO:tasks.workunit.client.0.smithi100.stderr: git switch - 2024-09-15T21:48:04.831 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-09-15T21:48:04.831 INFO:tasks.workunit.client.0.smithi100.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-15T21:48:04.831 INFO:tasks.workunit.client.0.smithi100.stderr: 2024-09-15T21:48:04.831 INFO:tasks.workunit.client.0.smithi100.stderr:HEAD is now at ad35db06e89 Merge pull request #59794 from zdover23/wip-doc-2024-09-15-backport-59613-to-squid 2024-09-15T21:48:04.836 DEBUG:teuthology.orchestra.run.smithi100:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-15T21:48:04.878 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:48:04.876+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:48:04.903 INFO:tasks.workunit.client.0.smithi100.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-15T21:48:04.905 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-15T21:48:04.905 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-15T21:48:04.920 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:04.920 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:04.929 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:48:05.013 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:48:05.012+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:48:05.073 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:05.074 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:05.081 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:48:05.079+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:48:05.082 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:48:05.182 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:48:05.180+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:48:05.231 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:05.231 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:05.240 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:48:05.285 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:48:05.283+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:48:05.385 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:05.385 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:05.386 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:48:05.385+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:48:05.394 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:48:05.488 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:48:05.485+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:48:05.498 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:05.498 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:05.507 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:48:05.587 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:48:05.586+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:48:05.621 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-15T21:48:05.656 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:05.656 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:05.664 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:48:05.666 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:48:05.694 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-15T21:48:05.695 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:48:05.689+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:48:05.732 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-15T21:48:05.733 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-15T21:48:05.733 INFO:tasks.workunit.client.0.smithi100.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-15T21:48:05.744 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:48:05.774 INFO:tasks.workunit.client.0.smithi100.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-15T21:48:05.777 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:48:05.777 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-15T21:48:05.791 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:48:05.790+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:48:05.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.794+0000 7f8369903640 1 Processor -- start 2024-09-15T21:48:05.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.794+0000 7f8369903640 1 -- start start 2024-09-15T21:48:05.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.795+0000 7f8369903640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8364155440 0x7f83640cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.795+0000 7f8369903640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f83640cb7d0 0x7f83640cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.795+0000 7f8369903640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f836415e7a0 con 0x7f836406d0e0 2024-09-15T21:48:05.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.795+0000 7f8369903640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f836415b470 con 0x7f83640cb7d0 2024-09-15T21:48:05.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.795+0000 7f8369903640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8364155820 con 0x7f8364155440 2024-09-15T21:48:05.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.795+0000 7f83627fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f83640cb7d0 0x7f83640cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.795+0000 7f83637fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f836406d0e0 0x7f83640cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:57456/0 (socket says 172.21.15.100:57456) 2024-09-15T21:48:05.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.795+0000 7f83637fe640 1 -- 172.21.15.100:0/3520623 learned_addr learned my addr 172.21.15.100:0/3520623 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:05.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.795+0000 7f8362ffd640 1 --2- 172.21.15.100:0/3520623 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8364155440 0x7f83640cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.795+0000 7f83627fc640 1 -- 172.21.15.100:0/3520623 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8364155440 msgr2=0x7f83640cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:05.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.795+0000 7f83627fc640 1 --2- 172.21.15.100:0/3520623 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8364155440 0x7f83640cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:05.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.795+0000 7f83627fc640 1 -- 172.21.15.100:0/3520623 >> v1:172.21.15.100:6789/0 conn(0x7f836406d0e0 legacy=0x7f83640cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:05.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.795+0000 7f83627fc640 1 -- 172.21.15.100:0/3520623 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f836415ea40 con 0x7f83640cb7d0 2024-09-15T21:48:05.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.795+0000 7f8361ffb640 1 -- 172.21.15.100:0/3520623 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3580132057 0 0) 0x7f836415e7a0 con 0x7f836406d0e0 2024-09-15T21:48:05.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.796+0000 7f83627fc640 1 --2- 172.21.15.100:0/3520623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83640cb7d0 0x7f83640cd960 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f83580097a0 tx=0x7f835802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df225f17169bf8aa server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:05.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.796+0000 7f8361ffb640 1 -- 172.21.15.100:0/3520623 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358037070 con 0x7f83640cb7d0 2024-09-15T21:48:05.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.796+0000 7f8361ffb640 1 -- 172.21.15.100:0/3520623 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8358032440 con 0x7f83640cb7d0 2024-09-15T21:48:05.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.796+0000 7f8361ffb640 1 -- 172.21.15.100:0/3520623 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8358032810 con 0x7f83640cb7d0 2024-09-15T21:48:05.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.797+0000 7f8369903640 1 -- 172.21.15.100:0/3520623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83640cb7d0 msgr2=0x7f83640cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:05.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.797+0000 7f8369903640 1 --2- 172.21.15.100:0/3520623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83640cb7d0 0x7f83640cd960 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f83580097a0 tx=0x7f835802a180 comp rx=0 tx=0).stop 2024-09-15T21:48:05.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.797+0000 7f8369903640 1 -- 172.21.15.100:0/3520623 shutdown_connections 2024-09-15T21:48:05.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.797+0000 7f8369903640 1 --2- 172.21.15.100:0/3520623 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f83640cb7d0 0x7f83640cd960 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:05.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.797+0000 7f8369903640 1 --2- 172.21.15.100:0/3520623 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8364155440 0x7f83640cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:05.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.797+0000 7f8369903640 1 -- 172.21.15.100:0/3520623 >> 172.21.15.100:0/3520623 conn(0x7f83640680d0 msgr2=0x7f836406d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:05.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.797+0000 7f8369903640 1 -- 172.21.15.100:0/3520623 shutdown_connections 2024-09-15T21:48:05.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.797+0000 7f8369903640 1 -- 172.21.15.100:0/3520623 wait complete. 2024-09-15T21:48:05.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.799+0000 7f8369903640 1 Processor -- start 2024-09-15T21:48:05.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.799+0000 7f8369903640 1 -- start start 2024-09-15T21:48:05.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.801+0000 7f8369903640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f836406d0e0 0x7f8364158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.801+0000 7f8369903640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83640cb7d0 0x7f83640c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.801+0000 7f8369903640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8364155440 0x7f83640c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.801+0000 7f8369903640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f836415f2e0 con 0x7f8364155440 2024-09-15T21:48:05.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.801+0000 7f8369903640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f836415f000 con 0x7f83640cb7d0 2024-09-15T21:48:05.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.801+0000 7f8369903640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f836415b070 con 0x7f836406d0e0 2024-09-15T21:48:05.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.801+0000 7f83637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8364155440 0x7f83640c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.801+0000 7f83637fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8364155440 0x7f83640c9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57788/0 (socket says 172.21.15.100:57788) 2024-09-15T21:48:05.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.801+0000 7f8362ffd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f836406d0e0 0x7f8364158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.801+0000 7f83637fe640 1 -- 172.21.15.100:0/1927941529 learned_addr learned my addr 172.21.15.100:0/1927941529 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:05.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.801+0000 7f83627fc640 1 --2- 172.21.15.100:0/1927941529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83640cb7d0 0x7f83640c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.805 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:05.805 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:05.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.801+0000 7f83637fe640 1 -- 172.21.15.100:0/1927941529 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f836406d0e0 msgr2=0x7f8364158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:05.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.801+0000 7f83637fe640 1 --2- 172.21.15.100:0/1927941529 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f836406d0e0 0x7f8364158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:05.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.801+0000 7f83637fe640 1 -- 172.21.15.100:0/1927941529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83640cb7d0 msgr2=0x7f83640c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:05.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.801+0000 7f83637fe640 1 --2- 172.21.15.100:0/1927941529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83640cb7d0 0x7f83640c8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:05.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.801+0000 7f83637fe640 1 -- 172.21.15.100:0/1927941529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83640ca100 con 0x7f8364155440 2024-09-15T21:48:05.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.801+0000 7f83637fe640 1 --2- 172.21.15.100:0/1927941529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8364155440 0x7f83640c9bc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f835c00c700 tx=0x7f835c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:05.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.802+0000 7f834bfff640 1 -- 172.21.15.100:0/1927941529 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c018070 con 0x7f8364155440 2024-09-15T21:48:05.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.802+0000 7f834bfff640 1 -- 172.21.15.100:0/1927941529 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f835c00ce10 con 0x7f8364155440 2024-09-15T21:48:05.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.802+0000 7f8369903640 1 -- 172.21.15.100:0/1927941529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83640c9420 con 0x7f8364155440 2024-09-15T21:48:05.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.802+0000 7f834bfff640 1 -- 172.21.15.100:0/1927941529 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f835c00a860 con 0x7f8364155440 2024-09-15T21:48:05.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.802+0000 7f8369903640 1 -- 172.21.15.100:0/1927941529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f836415c420 con 0x7f8364155440 2024-09-15T21:48:05.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.803+0000 7f8369903640 1 -- 172.21.15.100:0/1927941529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83640b3940 con 0x7f8364155440 2024-09-15T21:48:05.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.803+0000 7f834bfff640 1 -- 172.21.15.100:0/1927941529 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f835c01b070 con 0x7f8364155440 2024-09-15T21:48:05.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.804+0000 7f834bfff640 1 --2- 172.21.15.100:0/1927941529 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f835006c2e0 0x7f835006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.804+0000 7f834bfff640 1 -- 172.21.15.100:0/1927941529 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6060+0+0 (secure 0 0 0) 0x7f835c08aa10 con 0x7f8364155440 2024-09-15T21:48:05.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.804+0000 7f8362ffd640 1 --2- 172.21.15.100:0/1927941529 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f835006c2e0 0x7f835006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.808 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.805+0000 7f8362ffd640 1 --2- 172.21.15.100:0/1927941529 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f835006c2e0 0x7f835006e7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8354006fd0 tx=0x7f8354008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:05.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.809+0000 7f834bfff640 1 -- 172.21.15.100:0/1927941529 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f835c0589d0 con 0x7f8364155440 2024-09-15T21:48:05.813 INFO:tasks.workunit:Running workunits matching fs/misc/trivial_sync.sh on client.0... 2024-09-15T21:48:05.814 INFO:tasks.workunit:Running workunit fs/misc/trivial_sync.sh... 2024-09-15T21:48:05.814 DEBUG:teuthology.orchestra.run.smithi100:workunit test fs/misc/trivial_sync.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/trivial_sync.sh 2024-09-15T21:48:05.817 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:48:05.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.849+0000 7fcfdb8ba640 1 Processor -- start 2024-09-15T21:48:05.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.849+0000 7fcfdb8ba640 1 -- start start 2024-09-15T21:48:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.850+0000 7fcfdb8ba640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fcfd406c6e0 0x7fcfd40c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.850+0000 7fcfdb8ba640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcfd40c9ea0 0x7fcfd4158100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.850+0000 7fcfdb8ba640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcfd406cac0 0x7fcfd4072210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.850+0000 7fcfdb8ba640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fcfd4158640 con 0x7fcfd406cac0 2024-09-15T21:48:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.850+0000 7fcfdb8ba640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fcfd40c37c0 con 0x7fcfd406c6e0 2024-09-15T21:48:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.850+0000 7fcfdb8ba640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fcfd4071130 con 0x7fcfd40c9ea0 2024-09-15T21:48:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.850+0000 7fcfd9e57640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcfd406cac0 0x7fcfd4072210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.850+0000 7fcfd9e57640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcfd406cac0 0x7fcfd4072210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57790/0 (socket says 172.21.15.100:57790) 2024-09-15T21:48:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.850+0000 7fcfd9e57640 1 -- 172.21.15.100:0/2092071845 learned_addr learned my addr 172.21.15.100:0/2092071845 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.850+0000 7fcfd9656640 1 --2- 172.21.15.100:0/2092071845 >> v2:172.21.15.122:3300/0 conn(0x7fcfd406c6e0 0x7fcfd40c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.850+0000 7fcfd8e55640 1 --2- 172.21.15.100:0/2092071845 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcfd40c9ea0 0x7fcfd4158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.850+0000 7fcfd9e57640 1 -- 172.21.15.100:0/2092071845 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcfd40c9ea0 msgr2=0x7fcfd4158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:05.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.850+0000 7fcfd9e57640 1 --2- 172.21.15.100:0/2092071845 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcfd40c9ea0 0x7fcfd4158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.850+0000 7fcfd9e57640 1 -- 172.21.15.100:0/2092071845 >> v2:172.21.15.122:3300/0 conn(0x7fcfd406c6e0 msgr2=0x7fcfd40c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.850+0000 7fcfd9e57640 1 --2- 172.21.15.100:0/2092071845 >> v2:172.21.15.122:3300/0 conn(0x7fcfd406c6e0 0x7fcfd40c31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:05.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.850+0000 7fcfd9e57640 1 -- 172.21.15.100:0/2092071845 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd40c3a60 con 0x7fcfd406cac0 2024-09-15T21:48:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.851+0000 7fcfd9e57640 1 --2- 172.21.15.100:0/2092071845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd406cac0 0x7fcfd4072210 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40742a0 tx=0x7fcfcc02c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=836b9f8f8fc4e054 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.851+0000 7fcfcbfff640 1 -- 172.21.15.100:0/2092071845 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc009600 con 0x7fcfd406cac0 2024-09-15T21:48:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.851+0000 7fcfcbfff640 1 -- 172.21.15.100:0/2092071845 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfcc037070 con 0x7fcfd406cac0 2024-09-15T21:48:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.851+0000 7fcfcbfff640 1 -- 172.21.15.100:0/2092071845 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0099c0 con 0x7fcfd406cac0 2024-09-15T21:48:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.852+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/2092071845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd406cac0 msgr2=0x7fcfd4072210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.852+0000 7fcfdb8ba640 1 --2- 172.21.15.100:0/2092071845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd406cac0 0x7fcfd4072210 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fcfd40742a0 tx=0x7fcfcc02c960 comp rx=0 tx=0).stop 2024-09-15T21:48:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.852+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/2092071845 shutdown_connections 2024-09-15T21:48:05.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.852+0000 7fcfdb8ba640 1 --2- 172.21.15.100:0/2092071845 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcfd40c9ea0 0x7fcfd4158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.852+0000 7fcfdb8ba640 1 --2- 172.21.15.100:0/2092071845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd406cac0 0x7fcfd4072210 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.852+0000 7fcfdb8ba640 1 --2- 172.21.15.100:0/2092071845 >> v2:172.21.15.122:3300/0 conn(0x7fcfd406c6e0 0x7fcfd40c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.852+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/2092071845 >> 172.21.15.100:0/2092071845 conn(0x7fcfd40c9380 msgr2=0x7fcfd40c9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.852+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/2092071845 shutdown_connections 2024-09-15T21:48:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.852+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/2092071845 wait complete. 2024-09-15T21:48:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.854+0000 7fcfdb8ba640 1 Processor -- start 2024-09-15T21:48:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.854+0000 7fcfdb8ba640 1 -- start start 2024-09-15T21:48:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.854+0000 7fcfdb8ba640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd406c6e0 0x7fcfd41ed120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.854+0000 7fcfdb8ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfd406cac0 0x7fcfd4154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.855+0000 7fcfdb8ba640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcfd40c9ea0 0x7fcfd4155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.855+0000 7fcfdb8ba640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd4071e90 con 0x7fcfd406c6e0 2024-09-15T21:48:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.855+0000 7fcfdb8ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd4071690 con 0x7fcfd406cac0 2024-09-15T21:48:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.855+0000 7fcfdb8ba640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fcfd4150af0 con 0x7fcfd40c9ea0 2024-09-15T21:48:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.855+0000 7fcfd8e55640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfd406cac0 0x7fcfd4154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.855+0000 7fcfd9656640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd406c6e0 0x7fcfd41ed120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.855+0000 7fcfd9e57640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcfd40c9ea0 0x7fcfd4155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.855+0000 7fcfd9656640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd406c6e0 0x7fcfd41ed120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57794/0 (socket says 172.21.15.100:57794) 2024-09-15T21:48:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.855+0000 7fcfd9656640 1 -- 172.21.15.100:0/396691928 learned_addr learned my addr 172.21.15.100:0/396691928 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.855+0000 7fcfd8e55640 1 -- 172.21.15.100:0/396691928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcfd40c9ea0 msgr2=0x7fcfd4155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.855+0000 7fcfd8e55640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcfd40c9ea0 0x7fcfd4155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.855+0000 7fcfd8e55640 1 -- 172.21.15.100:0/396691928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd406c6e0 msgr2=0x7fcfd41ed120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.855+0000 7fcfd8e55640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcfd406c6e0 0x7fcfd41ed120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.855+0000 7fcfd8e55640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd40c5a60 con 0x7fcfd406cac0 2024-09-15T21:48:05.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.855+0000 7fcfd9e57640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcfd40c9ea0 0x7fcfd4155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:48:05.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.855+0000 7fcfd8e55640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcfd406cac0 0x7fcfd4154d80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcfc400e8a0 tx=0x7fcfc400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:05.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.856+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4017070 con 0x7fcfd406cac0 2024-09-15T21:48:05.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.856+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfc400c940 con 0x7fcfd406cac0 2024-09-15T21:48:05.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.856+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4009e60 con 0x7fcfd406cac0 2024-09-15T21:48:05.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.856+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd40c27e0 con 0x7fcfd406cac0 2024-09-15T21:48:05.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.856+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd40c2da0 con 0x7fcfd406cac0 2024-09-15T21:48:05.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.857+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7fcfd4069420 con 0x7fcfd406cac0 2024-09-15T21:48:05.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.858+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcfc401a070 con 0x7fcfd406cac0 2024-09-15T21:48:05.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.858+0000 7fcfc9ffb640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fcfb806c2e0 0x7fcfb806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.858+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6060+0+0 (secure 0 0 0) 0x7fcfc408b2b0 con 0x7fcfd406cac0 2024-09-15T21:48:05.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.859+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v52) ==== 54+0+16 (secure 0 0 0) 0x7fcfc408d850 con 0x7fcfd406cac0 2024-09-15T21:48:05.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.859+0000 7fcfd9656640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fcfb806c2e0 0x7fcfb806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.859+0000 7fcfdb8ba640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fcfd41552c0 0x7fcfd4156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.859+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcfd40c31a0 con 0x7fcfd41552c0 2024-09-15T21:48:05.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.859+0000 7fcfd9e57640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fcfd41552c0 0x7fcfd4156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.859+0000 7fcfd9656640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fcfb806c2e0 0x7fcfb806e7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009b80 tx=0x7fcfd0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:05.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.860+0000 7fcfd9e57640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7fcfd41552c0 0x7fcfd4156380 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:05.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.860+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcfd40c31a0 con 0x7fcfd41552c0 2024-09-15T21:48:05.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.877+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcfd40c6650 con 0x7fcfd41552c0 2024-09-15T21:48:05.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.879+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcfd40c6650 con 0x7fcfd41552c0 2024-09-15T21:48:05.882 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:48:05.882 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:48:05.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.880+0000 7fcfdb8ba640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fcfd414e160 0x7fcfd414e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.880+0000 7fcfd9656640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fcfd414e160 0x7fcfd414e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.880+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7fcfd4151bb0 con 0x7fcfd414e160 2024-09-15T21:48:05.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.881+0000 7fcfd9656640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fcfd414e160 0x7fcfd414e540 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:05.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.882+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcfd4151bb0 con 0x7fcfd414e160 2024-09-15T21:48:05.891 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:48:05.890+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:48:05.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.898+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcfd41518d0 con 0x7fcfd414e160 2024-09-15T21:48:05.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.900+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcfd41518d0 con 0x7fcfd414e160 2024-09-15T21:48:05.902 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:48:05.903 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:48:05.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.900+0000 7fcfdb8ba640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fcfd41f0120 0x7fcfd41f0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.900+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7fcfd41f0a60 con 0x7fcfd41f0120 2024-09-15T21:48:05.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.900+0000 7fcfd8e55640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fcfd41f0120 0x7fcfd41f0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.901+0000 7fcfd8e55640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fcfd41f0120 0x7fcfd41f0520 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:05.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.902+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcfc4059260 con 0x7fcfd41f0120 2024-09-15T21:48:05.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.915+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcfd4062880 con 0x7fcfd41f0120 2024-09-15T21:48:05.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.918+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcfd4062880 con 0x7fcfd41f0120 2024-09-15T21:48:05.920 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:48:05.921 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:48:05.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.918+0000 7fcfdb8ba640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fcfd4063360 0x7fcfd415a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.918+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7fcfd4062880 con 0x7fcfd4063360 2024-09-15T21:48:05.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.918+0000 7fcfd9e57640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fcfd4063360 0x7fcfd415a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.919+0000 7fcfd9e57640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7fcfd4063360 0x7fcfd415a880 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:05.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.919+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcfd4062880 con 0x7fcfd4063360 2024-09-15T21:48:05.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.933+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcfd4062880 con 0x7fcfd4063360 2024-09-15T21:48:05.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.934+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcfd4062880 con 0x7fcfd4063360 2024-09-15T21:48:05.936 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:48:05.937 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:48:05.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.935+0000 7fcfdb8ba640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fcfd415b580 0x7fcfd402c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.935+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7fcfd4206ad0 con 0x7fcfd415b580 2024-09-15T21:48:05.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.935+0000 7fcfd9656640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fcfd415b580 0x7fcfd402c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.936+0000 7fcfd9656640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7fcfd415b580 0x7fcfd402c640 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:05.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.936+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcfd4206ad0 con 0x7fcfd415b580 2024-09-15T21:48:05.952 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.950+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcfd402cb80 con 0x7fcfd415b580 2024-09-15T21:48:05.954 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.952+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcfd402cb80 con 0x7fcfd415b580 2024-09-15T21:48:05.955 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:48:05.956 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:48:05.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.952+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=53}) -- 0x7fcfd402cd90 con 0x7fcfd406cac0 2024-09-15T21:48:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.952+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcfd402cb80 con 0x7fcfd406cac0 2024-09-15T21:48:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.953+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fcfc4021040 con 0x7fcfd406cac0 2024-09-15T21:48:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:48:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.953+0000 7fcfdb8ba640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fcfd42053c0 0x7fcfd42077b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.953+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7fcfd420a580 con 0x7fcfd42053c0 2024-09-15T21:48:05.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.953+0000 7fcfd8e55640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fcfd42053c0 0x7fcfd42077b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.954+0000 7fcfd8e55640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fcfd42053c0 0x7fcfd42077b0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:05.958 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.954+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcfd420a580 con 0x7fcfd42053c0 2024-09-15T21:48:05.962 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:05.962 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:48:05.962 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:48:05.962 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:48:05.962 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:05.969 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.968+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcfd4207cf0 con 0x7fcfd42053c0 2024-09-15T21:48:05.971 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:48:05.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.970+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcfd4207cf0 con 0x7fcfd42053c0 2024-09-15T21:48:05.972 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:48:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:48:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.970+0000 7fcfdb8ba640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fcfd4209af0 0x7fcfd4209f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.970+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7fcfd420d940 con 0x7fcfd4209af0 2024-09-15T21:48:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.970+0000 7fcfd9e57640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fcfd4209af0 0x7fcfd4209f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:05.973 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.971+0000 7fcfd9e57640 1 --2- 172.21.15.100:0/396691928 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fcfd4209af0 0x7fcfd4209f30 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:05.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.971+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcfd420d940 con 0x7fcfd4209af0 2024-09-15T21:48:05.987 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.985+0000 7fcfdb8ba640 1 -- 172.21.15.100:0/396691928 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=true"]}) -- 0x7fcfd4207e80 con 0x7fcfd4209af0 2024-09-15T21:48:05.989 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:05.987+0000 7fcfc9ffb640 1 -- 172.21.15.100:0/396691928 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7fcfd4207e80 con 0x7fcfd4209af0 2024-09-15T21:48:05.991 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:48:05.991 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:48:05.992 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:48:05.991+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:48:05.993 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:48:05.994 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:48:06.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:06.010+0000 7f8369903640 1 -- 172.21.15.100:0/1927941529 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd set", "key": "nodeep-scrub"} v 0) -- 0x7f83640c8aa0 con 0x7f8364155440 2024-09-15T21:48:06.078 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:06.078 INFO:teuthology.orchestra.run.smithi100.stdout: "size": 20, 2024-09-15T21:48:06.079 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 600, 2024-09-15T21:48:06.079 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [ 2024-09-15T21:48:06.079 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.079 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.0.4:33 2.1b 2:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:06.079 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:48:06.079 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 97.662483418999997, 2024-09-15T21:48:06.079 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047486860000000002, 2024-09-15T21:48:06.079 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.080 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.080 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:48:06.080 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.080 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.080 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.080 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:48:06.080 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.080 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.080 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.080 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.080 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413809+0000", 2024-09-15T21:48:06.080 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.080 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.080 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.081 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.081 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413816+0000", 2024-09-15T21:48:06.081 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4810000000000003e-06 2024-09-15T21:48:06.081 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.081 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.081 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.081 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413827+0000", 2024-09-15T21:48:06.081 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.187e-05 2024-09-15T21:48:06.081 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.081 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.081 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.081 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413862+0000", 2024-09-15T21:48:06.081 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4041000000000002e-05 2024-09-15T21:48:06.082 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.082 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.082 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.082 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413871+0000", 2024-09-15T21:48:06.082 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.8919999999999995e-06 2024-09-15T21:48:06.082 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.082 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.082 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.082 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.413997+0000", 2024-09-15T21:48:06.082 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012601299999999999 2024-09-15T21:48:06.082 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.082 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.082 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:06.083 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.414100+0000", 2024-09-15T21:48:06.083 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000102555 2024-09-15T21:48:06.083 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.083 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.083 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.083 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.418558+0000", 2024-09-15T21:48:06.083 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044578339999999999 2024-09-15T21:48:06.083 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.083 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.083 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.083 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.084 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.084 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.0.4:3 2.14 e22/19 2:287034c4:::1.00000000.inode:head v 22'4, mlcod=0'0)", 2024-09-15T21:48:06.084 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:48:06.084 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 97.658838605, 2024-09-15T21:48:06.084 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045423590000000002, 2024-09-15T21:48:06.084 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.085 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.085 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.085 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.085 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.085 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.085 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:48:06.085 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.085 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.085 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.085 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417454+0000", 2024-09-15T21:48:06.085 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.085 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.085 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.086 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.086 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417455+0000", 2024-09-15T21:48:06.086 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9100000000000003e-07 2024-09-15T21:48:06.086 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.086 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.086 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.086 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417463+0000", 2024-09-15T21:48:06.086 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.3750000000000003e-06 2024-09-15T21:48:06.086 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.086 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.086 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.086 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417475+0000", 2024-09-15T21:48:06.086 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.2473000000000001e-05 2024-09-15T21:48:06.086 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.087 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.087 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.087 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417482+0000", 2024-09-15T21:48:06.087 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4010000000000002e-06 2024-09-15T21:48:06.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.087 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.087 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.087 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417524+0000", 2024-09-15T21:48:06.087 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1870999999999999e-05 2024-09-15T21:48:06.087 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.087 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.087 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:06.087 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417542+0000", 2024-09-15T21:48:06.087 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8224e-05 2024-09-15T21:48:06.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.088 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.088 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:06.088 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421955+0000", 2024-09-15T21:48:06.088 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0044130740000000003 2024-09-15T21:48:06.088 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.088 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.088 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.088 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.421996+0000", 2024-09-15T21:48:06.089 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.125e-05 2024-09-15T21:48:06.089 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.089 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.089 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.089 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.089 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.089 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.14 epoch 22/19 rep_tid 7 entries 22'5 (0'0) error 2:2e2fa760:::200.00000001:head by mds.0.4:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:48:06.089 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:48:06.089 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 97.658577800000003, 2024-09-15T21:48:06.089 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043086519999999996, 2024-09-15T21:48:06.090 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.090 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.090 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:48:06.090 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.090 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.090 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.090 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:48:06.090 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.090 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.090 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.091 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.091 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:48:06.091 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.091 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.091 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.091 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417715+0000", 2024-09-15T21:48:06.091 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.58e-07 2024-09-15T21:48:06.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.091 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.091 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.091 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417722+0000", 2024-09-15T21:48:06.091 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.4729999999999997e-06 2024-09-15T21:48:06.091 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.092 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.092 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.092 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417737+0000", 2024-09-15T21:48:06.092 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5155e-05 2024-09-15T21:48:06.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.092 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.092 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.092 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417752+0000", 2024-09-15T21:48:06.092 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4863e-05 2024-09-15T21:48:06.092 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.092 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.092 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.092 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.417819+0000", 2024-09-15T21:48:06.093 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7680000000000003e-05 2024-09-15T21:48:06.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.093 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.093 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.093 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:28.422023+0000", 2024-09-15T21:48:06.093 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042040230000000003 2024-09-15T21:48:06.093 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.093 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.093 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.093 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.093 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.093 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:25 2.1b 2:dbae76b5:::mds1_inotable:head [writefull 0~34] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:06.093 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:48:06.093 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 91.619088770000005, 2024-09-15T21:48:06.093 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070572830000000001, 2024-09-15T21:48:06.094 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.094 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.094 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.094 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.094 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.094 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.094 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:48:06.094 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.094 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.094 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.094 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457204+0000", 2024-09-15T21:48:06.094 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.094 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.095 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.095 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.095 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457205+0000", 2024-09-15T21:48:06.095 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6899999999999999e-06 2024-09-15T21:48:06.095 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.095 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.095 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.095 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457211+0000", 2024-09-15T21:48:06.095 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.626e-06 2024-09-15T21:48:06.095 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.095 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.095 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.095 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457244+0000", 2024-09-15T21:48:06.095 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2691999999999998e-05 2024-09-15T21:48:06.096 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.096 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.096 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.096 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457251+0000", 2024-09-15T21:48:06.096 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.3839999999999999e-06 2024-09-15T21:48:06.096 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.096 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.096 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.096 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457309+0000", 2024-09-15T21:48:06.096 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8023999999999999e-05 2024-09-15T21:48:06.096 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.096 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.096 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:06.097 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457413+0000", 2024-09-15T21:48:06.097 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00010331999999999999 2024-09-15T21:48:06.097 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.097 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.097 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:06.097 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:48:06.097 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000118439 2024-09-15T21:48:06.097 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.097 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.097 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:06.097 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463246+0000", 2024-09-15T21:48:06.097 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057148219999999996 2024-09-15T21:48:06.097 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.097 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.097 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:06.098 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463428+0000", 2024-09-15T21:48:06.098 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00018218299999999999 2024-09-15T21:48:06.098 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.098 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.098 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:06.098 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464212+0000", 2024-09-15T21:48:06.098 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00078402799999999998 2024-09-15T21:48:06.098 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.098 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.098 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.098 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464261+0000", 2024-09-15T21:48:06.098 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9075e-05 2024-09-15T21:48:06.098 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.098 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.099 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.099 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.099 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.099 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:28 2.1b 2:db457a8a:::201.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:06.099 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:48:06.099 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 91.619078743000003, 2024-09-15T21:48:06.099 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0072141449999999999, 2024-09-15T21:48:06.099 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.099 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.099 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:48:06.099 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.099 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.099 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.099 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:48:06.100 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.100 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.100 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.100 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.100 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:48:06.100 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.100 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.100 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.100 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.100 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457214+0000", 2024-09-15T21:48:06.101 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:48:06.101 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.105 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.105 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.105 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457217+0000", 2024-09-15T21:48:06.105 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7259999999999998e-06 2024-09-15T21:48:06.105 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.105 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.105 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.105 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457842+0000", 2024-09-15T21:48:06.105 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00062482599999999998 2024-09-15T21:48:06.105 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.105 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.105 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.105 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457848+0000", 2024-09-15T21:48:06.106 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.7089999999999996e-06 2024-09-15T21:48:06.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.106 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.106 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.106 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457895+0000", 2024-09-15T21:48:06.106 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6317999999999999e-05 2024-09-15T21:48:06.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.106 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.106 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:06.106 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458013+0000", 2024-09-15T21:48:06.106 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00011848100000000001 2024-09-15T21:48:06.106 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.106 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.106 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.107 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464428+0000", 2024-09-15T21:48:06.107 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0064146990000000003 2024-09-15T21:48:06.107 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.107 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.107 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.107 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.107 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.107 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:29 2.1b 2:d80f8093:::201.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:06.107 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:48:06.107 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 91.619074299999994, 2024-09-15T21:48:06.107 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0073658980000000001, 2024-09-15T21:48:06.107 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.107 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.107 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:48:06.108 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.108 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.108 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.108 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:48:06.108 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.108 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.108 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.108 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457218+0000", 2024-09-15T21:48:06.108 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.108 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.108 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.108 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.108 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457219+0000", 2024-09-15T21:48:06.109 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2000000000000001e-07 2024-09-15T21:48:06.109 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.109 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.109 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.109 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.457226+0000", 2024-09-15T21:48:06.109 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.956e-06 2024-09-15T21:48:06.109 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.109 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.109 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.109 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458266+0000", 2024-09-15T21:48:06.109 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001040476 2024-09-15T21:48:06.109 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.109 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.109 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.109 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458273+0000", 2024-09-15T21:48:06.110 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.0690000000000004e-06 2024-09-15T21:48:06.110 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.110 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.110 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.110 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458319+0000", 2024-09-15T21:48:06.110 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5668999999999999e-05 2024-09-15T21:48:06.110 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.110 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.110 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:06.110 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458415+0000", 2024-09-15T21:48:06.110 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.6701999999999997e-05 2024-09-15T21:48:06.110 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.110 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.110 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.111 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464584+0000", 2024-09-15T21:48:06.111 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0061687060000000004 2024-09-15T21:48:06.111 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.111 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.111 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.111 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.111 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.111 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.1 epoch 22/19 rep_tid 12 entries 22'1 (0'0) error 2:810cebc8:::201.00000004:head by mds.1.10:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:48:06.111 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:48:06.111 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 91.617644294000002, 2024-09-15T21:48:06.112 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0039881550000000002, 2024-09-15T21:48:06.112 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.112 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.112 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:48:06.112 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.112 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.112 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.112 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:48:06.112 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.112 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.112 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.112 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.112 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458648+0000", 2024-09-15T21:48:06.112 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.113 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.113 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.113 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.113 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458649+0000", 2024-09-15T21:48:06.113 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9599999999999999e-07 2024-09-15T21:48:06.113 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.113 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.113 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.113 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458656+0000", 2024-09-15T21:48:06.113 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.2629999999999997e-06 2024-09-15T21:48:06.113 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.113 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.113 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.114 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458670+0000", 2024-09-15T21:48:06.114 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4166e-05 2024-09-15T21:48:06.114 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.114 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.114 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.114 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458682+0000", 2024-09-15T21:48:06.114 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1865000000000001e-05 2024-09-15T21:48:06.114 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.114 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.114 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.115 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458749+0000", 2024-09-15T21:48:06.115 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.6507000000000005e-05 2024-09-15T21:48:06.115 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.115 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.115 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.115 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.462636+0000", 2024-09-15T21:48:06.115 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0038877579999999998 2024-09-15T21:48:06.115 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.115 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.115 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.115 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.115 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.115 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.1.10:12 2.12 e22/19 2:48cd7578:::60e.00000000:head v 22'2, mlcod=0'0)", 2024-09-15T21:48:06.115 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:48:06.115 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 91.617329961999999, 2024-09-15T21:48:06.116 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0042479500000000003, 2024-09-15T21:48:06.116 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.116 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.116 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.116 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.116 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.116 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.116 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:48:06.116 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.116 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.116 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.116 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.116 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:48:06.117 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.117 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.117 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.117 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.117 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458963+0000", 2024-09-15T21:48:06.117 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2900000000000004e-07 2024-09-15T21:48:06.117 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.117 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.117 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.117 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458968+0000", 2024-09-15T21:48:06.117 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2039999999999996e-06 2024-09-15T21:48:06.117 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.117 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.117 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.117 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458977+0000", 2024-09-15T21:48:06.118 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.6179999999999995e-06 2024-09-15T21:48:06.118 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.118 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.118 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.118 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.458983+0000", 2024-09-15T21:48:06.118 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9510000000000003e-06 2024-09-15T21:48:06.118 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.118 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.118 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.118 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459479+0000", 2024-09-15T21:48:06.118 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000496274 2024-09-15T21:48:06.118 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.118 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.118 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:06.119 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459496+0000", 2024-09-15T21:48:06.119 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6538e-05 2024-09-15T21:48:06.119 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.119 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.119 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:06.119 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463169+0000", 2024-09-15T21:48:06.119 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.003672832 2024-09-15T21:48:06.119 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.119 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.119 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.119 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463211+0000", 2024-09-15T21:48:06.119 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2004e-05 2024-09-15T21:48:06.119 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.119 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.120 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.120 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.120 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.120 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "pg_update_log_missing(2.12 epoch 22/19 rep_tid 2 entries 22'3 (0'0) error 2:4bb29eec:::201.00000003:head by mds.1.10:30 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:48:06.120 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:48:06.120 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 91.616650407999998, 2024-09-15T21:48:06.120 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0035859609999999999, 2024-09-15T21:48:06.120 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.120 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.120 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "reached pg", 2024-09-15T21:48:06.120 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.120 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.120 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.120 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:48:06.121 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.121 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.121 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.121 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.121 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459642+0000", 2024-09-15T21:48:06.121 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.121 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.121 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.121 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.121 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459643+0000", 2024-09-15T21:48:06.122 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2200000000000004e-07 2024-09-15T21:48:06.122 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.122 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.122 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.122 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459650+0000", 2024-09-15T21:48:06.122 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4479999999999997e-06 2024-09-15T21:48:06.122 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.122 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.122 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.122 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459665+0000", 2024-09-15T21:48:06.122 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4586999999999999e-05 2024-09-15T21:48:06.122 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.122 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.122 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.123 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459676+0000", 2024-09-15T21:48:06.123 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1673e-05 2024-09-15T21:48:06.123 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.123 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.123 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.123 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.459732+0000", 2024-09-15T21:48:06.123 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.5612999999999998e-05 2024-09-15T21:48:06.123 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.123 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.123 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.123 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.463228+0000", 2024-09-15T21:48:06.123 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0034961179999999999 2024-09-15T21:48:06.123 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.123 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.123 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.124 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.124 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.124 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.1.10:34 2.1b 2:db457a8a:::201.00000001:head [write 0~588] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:06.124 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:48:06.124 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 91.611438772, 2024-09-15T21:48:06.124 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0070946129999999996, 2024-09-15T21:48:06.124 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.124 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.124 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.124 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.124 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.124 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.124 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:48:06.125 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.125 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.125 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.125 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.125 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464854+0000", 2024-09-15T21:48:06.125 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.125 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.125 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.125 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.125 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464855+0000", 2024-09-15T21:48:06.125 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.5880000000000001e-06 2024-09-15T21:48:06.125 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.125 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.125 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.125 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464863+0000", 2024-09-15T21:48:06.126 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0879999999999995e-06 2024-09-15T21:48:06.126 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.126 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.126 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.126 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464889+0000", 2024-09-15T21:48:06.126 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5729e-05 2024-09-15T21:48:06.126 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.126 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.126 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.126 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464898+0000", 2024-09-15T21:48:06.126 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0389999999999993e-06 2024-09-15T21:48:06.127 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.127 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.127 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.127 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.464962+0000", 2024-09-15T21:48:06.127 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.3567999999999999e-05 2024-09-15T21:48:06.127 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.127 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.127 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:06.127 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.465041+0000", 2024-09-15T21:48:06.127 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.8889999999999999e-05 2024-09-15T21:48:06.127 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.127 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.127 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:06.128 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.465173+0000", 2024-09-15T21:48:06.128 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00013226899999999999 2024-09-15T21:48:06.128 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.128 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.128 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:06.128 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.466319+0000", 2024-09-15T21:48:06.128 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0011463039999999999 2024-09-15T21:48:06.128 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.128 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.128 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:06.128 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471825+0000", 2024-09-15T21:48:06.128 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0055062160000000004 2024-09-15T21:48:06.128 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.128 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.129 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:06.129 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471886+0000", 2024-09-15T21:48:06.129 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.0503e-05 2024-09-15T21:48:06.129 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.129 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.129 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.129 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:34.471948+0000", 2024-09-15T21:48:06.129 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.2419e-05 2024-09-15T21:48:06.129 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.129 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.129 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.129 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.129 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.129 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:15 2.19 2:9ec52373:::61a.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:06.130 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:48:06.130 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 85.600780607999994, 2024-09-15T21:48:06.130 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047360500000000003, 2024-09-15T21:48:06.130 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.130 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.130 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.130 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.130 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.130 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.130 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:48:06.130 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.130 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.130 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.130 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.130 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475512+0000", 2024-09-15T21:48:06.131 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.131 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.131 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.131 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.131 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475514+0000", 2024-09-15T21:48:06.131 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0990000000000001e-06 2024-09-15T21:48:06.131 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.131 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.131 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.131 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475524+0000", 2024-09-15T21:48:06.132 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.5850000000000006e-06 2024-09-15T21:48:06.132 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.132 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.132 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.132 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475557+0000", 2024-09-15T21:48:06.132 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.3090000000000003e-05 2024-09-15T21:48:06.132 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.132 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.132 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.132 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475568+0000", 2024-09-15T21:48:06.132 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1116e-05 2024-09-15T21:48:06.132 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.132 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.133 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.133 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475649+0000", 2024-09-15T21:48:06.133 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.0815999999999993e-05 2024-09-15T21:48:06.133 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.133 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.133 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:06.133 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475805+0000", 2024-09-15T21:48:06.133 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015667200000000001 2024-09-15T21:48:06.133 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.133 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.133 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:48:06.133 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475944+0000", 2024-09-15T21:48:06.133 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000138651 2024-09-15T21:48:06.133 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.133 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.134 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:06.134 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476900+0000", 2024-09-15T21:48:06.134 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00095588699999999999 2024-09-15T21:48:06.134 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.134 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.134 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:06.134 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480153+0000", 2024-09-15T21:48:06.134 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0032535200000000002 2024-09-15T21:48:06.134 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.134 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.134 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:06.134 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480203+0000", 2024-09-15T21:48:06.134 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.9932999999999998e-05 2024-09-15T21:48:06.134 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.134 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.135 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.135 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480248+0000", 2024-09-15T21:48:06.135 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4681000000000001e-05 2024-09-15T21:48:06.135 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.135 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.135 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.135 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.135 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.135 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:16 2.19 2:9ec52373:::61a.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:06.135 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:48:06.135 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 85.600764666999993, 2024-09-15T21:48:06.135 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0049206709999999997, 2024-09-15T21:48:06.135 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.135 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.136 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.136 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.136 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.136 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.136 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:48:06.136 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.136 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.136 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.136 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.136 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:48:06.136 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.136 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.136 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.136 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.137 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475528+0000", 2024-09-15T21:48:06.137 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.1699999999999999e-07 2024-09-15T21:48:06.137 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.137 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.137 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.137 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475531+0000", 2024-09-15T21:48:06.137 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.05e-06 2024-09-15T21:48:06.137 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.137 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.137 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.137 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475743+0000", 2024-09-15T21:48:06.137 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021138099999999999 2024-09-15T21:48:06.137 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.137 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.137 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.138 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475751+0000", 2024-09-15T21:48:06.138 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7770000000000002e-06 2024-09-15T21:48:06.138 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.138 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.138 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.138 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476272+0000", 2024-09-15T21:48:06.138 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00052158399999999996 2024-09-15T21:48:06.138 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.138 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.138 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:06.138 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476348+0000", 2024-09-15T21:48:06.138 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.5963000000000004e-05 2024-09-15T21:48:06.138 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.138 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.138 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:48:06.139 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476474+0000", 2024-09-15T21:48:06.139 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00012638800000000001 2024-09-15T21:48:06.139 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.139 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.139 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:06.139 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478194+0000", 2024-09-15T21:48:06.139 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.001719768 2024-09-15T21:48:06.139 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.139 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.139 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:06.139 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480358+0000", 2024-09-15T21:48:06.139 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0021637340000000001 2024-09-15T21:48:06.139 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.139 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.140 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:06.140 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480411+0000", 2024-09-15T21:48:06.140 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2720999999999999e-05 2024-09-15T21:48:06.140 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.140 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.140 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.140 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480449+0000", 2024-09-15T21:48:06.140 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7888e-05 2024-09-15T21:48:06.140 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.140 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.140 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.140 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.140 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.140 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.2.12:29 2.1e 2:7d92db23:::202.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:06.141 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:48:06.141 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 85.600759878000005, 2024-09-15T21:48:06.141 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005215478, 2024-09-15T21:48:06.141 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.141 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.141 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "started", 2024-09-15T21:48:06.141 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.141 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.141 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.141 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:48:06.141 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.141 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.141 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.141 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.142 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:48:06.142 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.142 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.142 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.142 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.142 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475533+0000", 2024-09-15T21:48:06.142 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.5199999999999998e-07 2024-09-15T21:48:06.142 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.142 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.142 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.142 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475535+0000", 2024-09-15T21:48:06.142 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.9520000000000001e-06 2024-09-15T21:48:06.142 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.142 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.143 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.143 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475785+0000", 2024-09-15T21:48:06.143 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00024966099999999998 2024-09-15T21:48:06.143 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.143 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.143 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.143 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475799+0000", 2024-09-15T21:48:06.143 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.4649e-05 2024-09-15T21:48:06.143 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.143 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.143 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.143 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.475856+0000", 2024-09-15T21:48:06.143 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.6956000000000002e-05 2024-09-15T21:48:06.143 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.144 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.144 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:06.144 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.476016+0000", 2024-09-15T21:48:06.144 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000159457 2024-09-15T21:48:06.144 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.144 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.144 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.144 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.480748+0000", 2024-09-15T21:48:06.144 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0047324510000000004 2024-09-15T21:48:06.144 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.144 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.144 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.144 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.144 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.2.12:1 2.1 e22/19 2:85bbe569:::202.00000000:head v 22'4, mlcod=0'0)", 2024-09-15T21:48:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:48:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 85.597957406999996, 2024-09-15T21:48:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0045423649999999996, 2024-09-15T21:48:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:48:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.145 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478335+0000", 2024-09-15T21:48:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478336+0000", 2024-09-15T21:48:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.4499999999999996e-07 2024-09-15T21:48:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478342+0000", 2024-09-15T21:48:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5579999999999997e-06 2024-09-15T21:48:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.146 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.147 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478356+0000", 2024-09-15T21:48:06.147 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3567000000000001e-05 2024-09-15T21:48:06.147 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.147 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.147 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.147 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478364+0000", 2024-09-15T21:48:06.147 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7889999999999995e-06 2024-09-15T21:48:06.147 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.147 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.147 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.147 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478416+0000", 2024-09-15T21:48:06.147 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.2188000000000002e-05 2024-09-15T21:48:06.147 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.147 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.148 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:06.148 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.478433+0000", 2024-09-15T21:48:06.148 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.6674e-05 2024-09-15T21:48:06.148 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.148 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.148 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:06.148 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482832+0000", 2024-09-15T21:48:06.148 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0043996110000000003 2024-09-15T21:48:06.148 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.148 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.148 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.148 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:40.482878+0000", 2024-09-15T21:48:06.148 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.5232999999999999e-05 2024-09-15T21:48:06.148 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.149 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.149 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.149 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.149 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.149 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_op(mds.4.16:25 2.19 2:9e6a133d:::104.00000000:head [omap-set-header,omap-set-vals] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:06.149 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:48:06.149 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 74.511097367999994, 2024-09-15T21:48:06.149 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0058459619999999997, 2024-09-15T21:48:06.149 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.149 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.149 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.149 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.149 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.149 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.150 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:48:06.150 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.150 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.150 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.150 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.150 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565195+0000", 2024-09-15T21:48:06.150 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.150 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.150 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.150 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.150 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565197+0000", 2024-09-15T21:48:06.150 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7889999999999999e-06 2024-09-15T21:48:06.150 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.150 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.151 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.151 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565208+0000", 2024-09-15T21:48:06.151 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1358999999999999e-05 2024-09-15T21:48:06.151 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.151 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.151 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.151 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565237+0000", 2024-09-15T21:48:06.151 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.8626e-05 2024-09-15T21:48:06.151 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.151 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.151 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.151 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565249+0000", 2024-09-15T21:48:06.151 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1897e-05 2024-09-15T21:48:06.151 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.151 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.152 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.152 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565332+0000", 2024-09-15T21:48:06.152 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.2943999999999997e-05 2024-09-15T21:48:06.152 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.152 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.152 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:06.152 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565452+0000", 2024-09-15T21:48:06.152 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.000119992 2024-09-15T21:48:06.152 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.152 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.152 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "waiting for subops from 6", 2024-09-15T21:48:06.152 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565610+0000", 2024-09-15T21:48:06.152 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00015845099999999999 2024-09-15T21:48:06.152 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.153 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.153 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "op_commit", 2024-09-15T21:48:06.153 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.568999+0000", 2024-09-15T21:48:06.153 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0033887190000000001 2024-09-15T21:48:06.153 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.153 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.153 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:06.153 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.570979+0000", 2024-09-15T21:48:06.153 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0019804919999999999 2024-09-15T21:48:06.153 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.153 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.153 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:06.153 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571014+0000", 2024-09-15T21:48:06.153 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.4331e-05 2024-09-15T21:48:06.154 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.154 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.154 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.154 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571041+0000", 2024-09-15T21:48:06.154 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.7362e-05 2024-09-15T21:48:06.154 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.154 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.154 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.154 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.154 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.154 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:5 2.e e22/19 2:75ee5fd8:::629.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:48:06.154 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:48:06.154 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 74.510421722000004, 2024-09-15T21:48:06.154 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057947119999999996, 2024-09-15T21:48:06.155 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.155 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.155 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.155 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.155 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.155 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.155 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:48:06.155 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.155 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.155 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.155 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.155 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565871+0000", 2024-09-15T21:48:06.155 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.155 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.156 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.156 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.156 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565872+0000", 2024-09-15T21:48:06.156 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.9800000000000002e-07 2024-09-15T21:48:06.156 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.156 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.156 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.156 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565880+0000", 2024-09-15T21:48:06.156 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7619999999999995e-06 2024-09-15T21:48:06.156 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.156 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.156 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.156 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565890+0000", 2024-09-15T21:48:06.156 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.0648999999999999e-05 2024-09-15T21:48:06.157 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.157 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.157 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.157 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565897+0000", 2024-09-15T21:48:06.157 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.153e-06 2024-09-15T21:48:06.157 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.157 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.157 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.157 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565966+0000", 2024-09-15T21:48:06.157 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.9046000000000003e-05 2024-09-15T21:48:06.157 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.157 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.157 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:06.157 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565985+0000", 2024-09-15T21:48:06.157 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.8695e-05 2024-09-15T21:48:06.158 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.158 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.158 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:06.158 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571625+0000", 2024-09-15T21:48:06.158 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056397139999999997 2024-09-15T21:48:06.158 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.158 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.158 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.158 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571666+0000", 2024-09-15T21:48:06.158 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.0695000000000003e-05 2024-09-15T21:48:06.158 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.158 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.158 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.159 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.159 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.159 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:6 2.e e22/19 2:75ee5fd8:::629.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:48:06.159 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:48:06.159 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 74.510359473999998, 2024-09-15T21:48:06.159 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0057607429999999996, 2024-09-15T21:48:06.159 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.159 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.159 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.159 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.159 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.159 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.159 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:48:06.159 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.160 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.160 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.160 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.160 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565933+0000", 2024-09-15T21:48:06.160 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.160 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.160 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.160 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.160 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565934+0000", 2024-09-15T21:48:06.160 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.6199999999999998e-07 2024-09-15T21:48:06.160 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.160 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.160 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.160 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565938+0000", 2024-09-15T21:48:06.161 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.4970000000000003e-06 2024-09-15T21:48:06.161 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.161 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.161 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.161 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565945+0000", 2024-09-15T21:48:06.161 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.5069999999999997e-06 2024-09-15T21:48:06.161 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.161 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.161 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.161 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.565948+0000", 2024-09-15T21:48:06.161 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:48:06.161 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.161 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.161 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.162 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566200+0000", 2024-09-15T21:48:06.162 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00025170200000000002 2024-09-15T21:48:06.162 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.162 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.162 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:06.162 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566218+0000", 2024-09-15T21:48:06.162 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7543000000000001e-05 2024-09-15T21:48:06.162 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.162 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.162 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:06.162 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571668+0000", 2024-09-15T21:48:06.162 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0054502999999999999 2024-09-15T21:48:06.162 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.162 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.162 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.163 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571694+0000", 2024-09-15T21:48:06.163 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.5979e-05 2024-09-15T21:48:06.163 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.163 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.163 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.163 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.163 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.163 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:2 2.1 e22/19 2:82971f97:::404.00000000:head v 22'5, mlcod=22'3)", 2024-09-15T21:48:06.163 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:48:06.163 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 74.510151538000002, 2024-09-15T21:48:06.163 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056016019999999998, 2024-09-15T21:48:06.163 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.163 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.163 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.164 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.164 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.164 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.164 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:48:06.164 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.164 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.164 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.164 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.164 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566141+0000", 2024-09-15T21:48:06.164 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.164 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.164 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.164 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.164 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566142+0000", 2024-09-15T21:48:06.165 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:48:06.165 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.165 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.165 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.165 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566149+0000", 2024-09-15T21:48:06.165 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.7510000000000008e-06 2024-09-15T21:48:06.165 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.165 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.165 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.165 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566161+0000", 2024-09-15T21:48:06.165 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.1259e-05 2024-09-15T21:48:06.165 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.165 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.165 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.165 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566168+0000", 2024-09-15T21:48:06.166 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.255e-06 2024-09-15T21:48:06.166 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.166 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.166 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.166 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566227+0000", 2024-09-15T21:48:06.166 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.8961e-05 2024-09-15T21:48:06.166 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.166 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.166 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:06.166 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566247+0000", 2024-09-15T21:48:06.166 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 2.0536000000000001e-05 2024-09-15T21:48:06.166 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.166 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.166 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:06.167 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571694+0000", 2024-09-15T21:48:06.167 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.005446762 2024-09-15T21:48:06.167 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.167 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.167 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.167 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571743+0000", 2024-09-15T21:48:06.167 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8396999999999998e-05 2024-09-15T21:48:06.167 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.167 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.167 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.167 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.167 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.167 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:17 2.14 e22/19 2:2eebc8f4:::62f.00000000:head v 22'9, mlcod=22'7)", 2024-09-15T21:48:06.167 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:48:06.167 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 74.510135441000003, 2024-09-15T21:48:06.168 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0056532860000000004, 2024-09-15T21:48:06.168 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.168 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.168 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.168 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.168 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.168 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.168 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:48:06.168 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.168 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.168 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.168 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.168 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566157+0000", 2024-09-15T21:48:06.169 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.169 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.169 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.169 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.169 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566158+0000", 2024-09-15T21:48:06.169 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 7.92e-07 2024-09-15T21:48:06.169 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.169 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.169 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.169 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566172+0000", 2024-09-15T21:48:06.169 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3976e-05 2024-09-15T21:48:06.169 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.169 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.169 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.169 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566407+0000", 2024-09-15T21:48:06.170 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00023483900000000001 2024-09-15T21:48:06.170 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.170 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.170 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.170 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566416+0000", 2024-09-15T21:48:06.170 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 9.0780000000000002e-06 2024-09-15T21:48:06.170 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.170 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.170 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.170 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566476+0000", 2024-09-15T21:48:06.170 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.9908999999999998e-05 2024-09-15T21:48:06.170 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.170 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.170 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:06.171 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566493+0000", 2024-09-15T21:48:06.171 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7567e-05 2024-09-15T21:48:06.171 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.171 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.171 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:06.171 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571762+0000", 2024-09-15T21:48:06.171 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0052682959999999996 2024-09-15T21:48:06.171 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.171 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.171 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.171 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571810+0000", 2024-09-15T21:48:06.171 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.8829000000000003e-05 2024-09-15T21:48:06.171 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.171 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.171 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.172 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.172 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.172 INFO:teuthology.orchestra.run.smithi100.stdout: "description": "osd_repop(mds.4.16:18 2.14 e22/19 2:2eebc8f4:::62f.00000000:head v 22'10, mlcod=22'7)", 2024-09-15T21:48:06.172 INFO:teuthology.orchestra.run.smithi100.stdout: "initiated_at": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:48:06.172 INFO:teuthology.orchestra.run.smithi100.stdout: "age": 74.509846366000005, 2024-09-15T21:48:06.172 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0053992579999999997, 2024-09-15T21:48:06.172 INFO:teuthology.orchestra.run.smithi100.stdout: "continuous": false, 2024-09-15T21:48:06.172 INFO:teuthology.orchestra.run.smithi100.stdout: "type_data": { 2024-09-15T21:48:06.172 INFO:teuthology.orchestra.run.smithi100.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.172 INFO:teuthology.orchestra.run.smithi100.stdout: "events": [ 2024-09-15T21:48:06.172 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.172 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "initiated", 2024-09-15T21:48:06.172 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:48:06.172 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.173 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.173 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.173 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "header_read", 2024-09-15T21:48:06.173 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566446+0000", 2024-09-15T21:48:06.173 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0 2024-09-15T21:48:06.173 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.173 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.173 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "throttled", 2024-09-15T21:48:06.173 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566447+0000", 2024-09-15T21:48:06.173 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 4.2e-07 2024-09-15T21:48:06.173 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.173 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.173 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "all_read", 2024-09-15T21:48:06.173 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566452+0000", 2024-09-15T21:48:06.174 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 5.0390000000000004e-06 2024-09-15T21:48:06.174 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.174 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.174 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "dispatched", 2024-09-15T21:48:06.174 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566460+0000", 2024-09-15T21:48:06.174 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 8.1389999999999995e-06 2024-09-15T21:48:06.174 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.174 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.174 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.174 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566473+0000", 2024-09-15T21:48:06.174 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.3093e-05 2024-09-15T21:48:06.174 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.174 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.174 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "reached_pg", 2024-09-15T21:48:06.174 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566688+0000", 2024-09-15T21:48:06.175 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.00021537100000000001 2024-09-15T21:48:06.175 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.175 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.175 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "started", 2024-09-15T21:48:06.175 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.566706+0000", 2024-09-15T21:48:06.175 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 1.7802999999999999e-05 2024-09-15T21:48:06.175 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.175 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.175 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "commit_sent", 2024-09-15T21:48:06.175 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571813+0000", 2024-09-15T21:48:06.175 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 0.0051065140000000004 2024-09-15T21:48:06.175 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-09-15T21:48:06.175 INFO:teuthology.orchestra.run.smithi100.stdout: { 2024-09-15T21:48:06.175 INFO:teuthology.orchestra.run.smithi100.stdout: "event": "done", 2024-09-15T21:48:06.175 INFO:teuthology.orchestra.run.smithi100.stdout: "time": "2024-09-15T21:46:51.571845+0000", 2024-09-15T21:48:06.176 INFO:teuthology.orchestra.run.smithi100.stdout: "duration": 3.2879000000000001e-05 2024-09-15T21:48:06.176 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.176 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.176 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.176 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-09-15T21:48:06.176 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-09-15T21:48:06.176 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:06.178 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:48:06.178 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:48:06.176+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:48:06.278 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:48:06.277+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:48:06.291 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:48:06.291 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:48:06.291 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:48:06.291 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:48:06.300 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:48:06.379 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:48:06.377+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:48:06.403 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:48:06.403 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:48:06.403 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:48:06.403 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:48:06.403 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:48:06.412 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:48:06.480 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:48:06.478+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:48:06.517 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:48:06.517 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:48:06.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:48:06.517 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:48:06.517 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.517 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:2 2.f 2:f1f69726:::400.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:06.517 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:48:06.517 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.101479753999996, 2024-09-15T21:48:06.517 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061533810000000003, 2024-09-15T21:48:06.517 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.517 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.517 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.518 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:48:06.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413079+0000", 2024-09-15T21:48:06.518 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.518 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.518 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.518 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.518 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413085+0000", 2024-09-15T21:48:06.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8390000000000003e-06 2024-09-15T21:48:06.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413092+0000", 2024-09-15T21:48:06.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3479999999999998e-06 2024-09-15T21:48:06.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.519 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413109+0000", 2024-09-15T21:48:06.519 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7192000000000001e-05 2024-09-15T21:48:06.519 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.519 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.519 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413118+0000", 2024-09-15T21:48:06.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.935e-06 2024-09-15T21:48:06.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413219+0000", 2024-09-15T21:48:06.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010121 2024-09-15T21:48:06.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.520 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:06.520 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413381+0000", 2024-09-15T21:48:06.520 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016148399999999999 2024-09-15T21:48:06.520 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.520 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:06.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413525+0000", 2024-09-15T21:48:06.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014428100000000001 2024-09-15T21:48:06.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:06.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415866+0000", 2024-09-15T21:48:06.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023409300000000002 2024-09-15T21:48:06.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.521 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.521 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:06.521 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419156+0000", 2024-09-15T21:48:06.521 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0032903580000000002 2024-09-15T21:48:06.521 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:06.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419199+0000", 2024-09-15T21:48:06.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2171e-05 2024-09-15T21:48:06.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.522 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.522 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419232+0000", 2024-09-15T21:48:06.522 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3633000000000001e-05 2024-09-15T21:48:06.522 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.522 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.522 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.522 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.522 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.522 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.0.4:28 2.b e22/19 2:d0630e4c:::mds0_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-15T21:48:06.523 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:48:06.523 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.100216244999999, 2024-09-15T21:48:06.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002614594, 2024-09-15T21:48:06.523 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.523 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.523 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.523 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.523 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:48:06.523 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.523 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.523 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.523 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414342+0000", 2024-09-15T21:48:06.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414344+0000", 2024-09-15T21:48:06.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6589999999999999e-06 2024-09-15T21:48:06.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.524 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.524 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414355+0000", 2024-09-15T21:48:06.524 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0604999999999999e-05 2024-09-15T21:48:06.524 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.524 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414370+0000", 2024-09-15T21:48:06.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5709000000000002e-05 2024-09-15T21:48:06.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414379+0000", 2024-09-15T21:48:06.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2449999999999998e-06 2024-09-15T21:48:06.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.525 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.525 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.525 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414477+0000", 2024-09-15T21:48:06.525 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8968000000000005e-05 2024-09-15T21:48:06.525 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:06.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414504+0000", 2024-09-15T21:48:06.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6899000000000001e-05 2024-09-15T21:48:06.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:06.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416837+0000", 2024-09-15T21:48:06.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023327130000000001 2024-09-15T21:48:06.526 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.526 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.526 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.526 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416957+0000", 2024-09-15T21:48:06.526 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000119796 2024-09-15T21:48:06.526 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.527 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.527 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.527 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.527 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.1.10:8 2.b e22/19 2:d70a01c6:::60c.00000000:head v 22'4, mlcod=22'1)", 2024-09-15T21:48:06.527 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:48:06.527 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 92.056636792999996, 2024-09-15T21:48:06.527 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027633890000000002, 2024-09-15T21:48:06.527 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.527 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.527 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.527 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.527 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.527 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:48:06.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457922+0000", 2024-09-15T21:48:06.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.528 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.528 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457923+0000", 2024-09-15T21:48:06.528 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.136e-06 2024-09-15T21:48:06.528 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.528 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457931+0000", 2024-09-15T21:48:06.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1529999999999997e-06 2024-09-15T21:48:06.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457946+0000", 2024-09-15T21:48:06.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4482e-05 2024-09-15T21:48:06.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.529 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.529 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.529 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457956+0000", 2024-09-15T21:48:06.529 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0774999999999999e-05 2024-09-15T21:48:06.529 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458016+0000", 2024-09-15T21:48:06.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9247999999999999e-05 2024-09-15T21:48:06.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:06.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458036+0000", 2024-09-15T21:48:06.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0920999999999998e-05 2024-09-15T21:48:06.530 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.530 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.530 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:06.530 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460636+0000", 2024-09-15T21:48:06.530 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025992089999999999 2024-09-15T21:48:06.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.531 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.531 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460685+0000", 2024-09-15T21:48:06.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9465000000000002e-05 2024-09-15T21:48:06.531 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.531 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.531 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.531 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.531 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.531 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.18 epoch 22/19 rep_tid 5 entries 22'1 (0'0) error 2:1d00e16c:::201.00000005:head by mds.1.10:32 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:48:06.531 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:48:06.531 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 92.054944661999997, 2024-09-15T21:48:06.531 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003509599, 2024-09-15T21:48:06.532 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.532 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.532 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:48:06.532 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:48:06.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.532 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.532 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.532 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459614+0000", 2024-09-15T21:48:06.532 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.532 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459615+0000", 2024-09-15T21:48:06.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4200000000000005e-07 2024-09-15T21:48:06.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459626+0000", 2024-09-15T21:48:06.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1456e-05 2024-09-15T21:48:06.533 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.533 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.533 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.533 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459650+0000", 2024-09-15T21:48:06.533 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.429e-05 2024-09-15T21:48:06.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459664+0000", 2024-09-15T21:48:06.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3805000000000001e-05 2024-09-15T21:48:06.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.459784+0000", 2024-09-15T21:48:06.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000119614 2024-09-15T21:48:06.534 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.534 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.534 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.534 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463123+0000", 2024-09-15T21:48:06.534 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0033395920000000002 2024-09-15T21:48:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:7 2.13 e22/19 2:cdf3ac81:::616.00000000:head v 22'5, mlcod=22'3)", 2024-09-15T21:48:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:48:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 86.037163883999995, 2024-09-15T21:48:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002138615, 2024-09-15T21:48:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.535 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:48:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477395+0000", 2024-09-15T21:48:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477396+0000", 2024-09-15T21:48:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4190000000000001e-06 2024-09-15T21:48:06.536 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477405+0000", 2024-09-15T21:48:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2320000000000007e-06 2024-09-15T21:48:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477419+0000", 2024-09-15T21:48:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3589e-05 2024-09-15T21:48:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477428+0000", 2024-09-15T21:48:06.537 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4779999999999997e-06 2024-09-15T21:48:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477493+0000", 2024-09-15T21:48:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4546000000000002e-05 2024-09-15T21:48:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477511+0000", 2024-09-15T21:48:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7793000000000001e-05 2024-09-15T21:48:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:06.538 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479481+0000", 2024-09-15T21:48:06.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019701660000000002 2024-09-15T21:48:06.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.539 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.539 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479533+0000", 2024-09-15T21:48:06.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2392000000000003e-05 2024-09-15T21:48:06.539 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.539 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.539 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.539 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.539 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.539 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:26 2.11 e22/19 2:893cd530:::mds2_sessionmap:head v 22'1, mlcod=0'0)", 2024-09-15T21:48:06.539 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:48:06.539 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 86.037016957999995, 2024-09-15T21:48:06.539 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020450450000000001, 2024-09-15T21:48:06.540 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.540 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.540 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.540 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:48:06.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.540 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.540 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.540 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.540 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:48:06.540 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477542+0000", 2024-09-15T21:48:06.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9699999999999995e-07 2024-09-15T21:48:06.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477555+0000", 2024-09-15T21:48:06.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3144e-05 2024-09-15T21:48:06.541 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.541 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.541 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.541 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477565+0000", 2024-09-15T21:48:06.541 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8579999999999995e-06 2024-09-15T21:48:06.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477573+0000", 2024-09-15T21:48:06.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4869999999999996e-06 2024-09-15T21:48:06.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477632+0000", 2024-09-15T21:48:06.542 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.9178999999999997e-05 2024-09-15T21:48:06.542 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.542 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.542 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:06.542 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477654+0000", 2024-09-15T21:48:06.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2234000000000001e-05 2024-09-15T21:48:06.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:06.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479526+0000", 2024-09-15T21:48:06.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018717600000000001 2024-09-15T21:48:06.543 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.543 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.543 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.543 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479587+0000", 2024-09-15T21:48:06.543 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0686000000000001e-05 2024-09-15T21:48:06.543 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.543 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.543 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.544 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.544 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.544 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:22 2.1a e22/19 2:5fd68f60:::61d.00000000:head v 22'3, mlcod=0'0)", 2024-09-15T21:48:06.545 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:48:06.545 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 86.036656902999994, 2024-09-15T21:48:06.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023694380000000002, 2024-09-15T21:48:06.546 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.546 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.546 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.546 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.546 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.546 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.546 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:48:06.546 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.546 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:48:06.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.547 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.547 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.547 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477902+0000", 2024-09-15T21:48:06.547 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4899999999999995e-07 2024-09-15T21:48:06.547 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477910+0000", 2024-09-15T21:48:06.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:48:06.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.548 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477920+0000", 2024-09-15T21:48:06.548 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9250000000000007e-06 2024-09-15T21:48:06.548 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.548 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.548 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477927+0000", 2024-09-15T21:48:06.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0940000000000002e-06 2024-09-15T21:48:06.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.549 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.549 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.549 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.477985+0000", 2024-09-15T21:48:06.549 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8451e-05 2024-09-15T21:48:06.549 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:06.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478005+0000", 2024-09-15T21:48:06.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0064e-05 2024-09-15T21:48:06.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.550 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:06.550 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480223+0000", 2024-09-15T21:48:06.550 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022177910000000002 2024-09-15T21:48:06.550 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.550 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.551 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.551 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480271+0000", 2024-09-15T21:48:06.551 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8164999999999997e-05 2024-09-15T21:48:06.551 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.551 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.551 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.551 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.551 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.552 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.2.12:8 2.13 e22/19 2:cdf3ac81:::616.00000000:head v 22'6, mlcod=22'3)", 2024-09-15T21:48:06.552 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:48:06.552 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 86.036457944000006, 2024-09-15T21:48:06.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021760249999999998, 2024-09-15T21:48:06.552 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.552 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.552 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.552 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.552 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.552 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.552 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:48:06.552 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.552 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478101+0000", 2024-09-15T21:48:06.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478102+0000", 2024-09-15T21:48:06.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3099999999999996e-07 2024-09-15T21:48:06.553 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.553 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.553 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.553 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478124+0000", 2024-09-15T21:48:06.553 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2790999999999999e-05 2024-09-15T21:48:06.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478136+0000", 2024-09-15T21:48:06.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1922999999999999e-05 2024-09-15T21:48:06.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.554 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.554 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478144+0000", 2024-09-15T21:48:06.554 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9300000000000003e-06 2024-09-15T21:48:06.554 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.554 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478192+0000", 2024-09-15T21:48:06.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8155999999999998e-05 2024-09-15T21:48:06.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:06.555 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478209+0000", 2024-09-15T21:48:06.555 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6816e-05 2024-09-15T21:48:06.555 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.555 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.555 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:06.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480243+0000", 2024-09-15T21:48:06.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020336199999999999 2024-09-15T21:48:06.556 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.556 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.556 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.556 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480277+0000", 2024-09-15T21:48:06.556 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3858e-05 2024-09-15T21:48:06.556 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.556 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.556 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.557 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.557 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.11 epoch 22/19 rep_tid 19 entries 22'2 (0'0) error 2:8975f766:::202.00000001:head by mds.2.12:28 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:48:06.557 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:48:06.557 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 86.036332005999995, 2024-09-15T21:48:06.557 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023882299999999999, 2024-09-15T21:48:06.557 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.557 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.557 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:48:06.557 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.557 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.557 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:48:06.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:48:06.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.558 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.558 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478227+0000", 2024-09-15T21:48:06.558 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2399999999999998e-07 2024-09-15T21:48:06.558 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.558 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478236+0000", 2024-09-15T21:48:06.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5299999999999996e-06 2024-09-15T21:48:06.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478248+0000", 2024-09-15T21:48:06.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2806000000000001e-05 2024-09-15T21:48:06.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.559 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.559 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.559 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478264+0000", 2024-09-15T21:48:06.559 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5468000000000002e-05 2024-09-15T21:48:06.559 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.478339+0000", 2024-09-15T21:48:06.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4851000000000006e-05 2024-09-15T21:48:06.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.560 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.560 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.560 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480615+0000", 2024-09-15T21:48:06.560 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022760509999999999 2024-09-15T21:48:06.560 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.560 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.560 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.560 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.561 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:3 2.1a e22/19 2:5fdc322d:::61e.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:48:06.561 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:48:06.561 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 80.968147404000007, 2024-09-15T21:48:06.561 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0023961220000000001, 2024-09-15T21:48:06.561 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.561 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.561 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.561 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.561 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:48:06.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.562 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546411+0000", 2024-09-15T21:48:06.562 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.562 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.562 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.562 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546413+0000", 2024-09-15T21:48:06.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.812e-06 2024-09-15T21:48:06.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.563 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546424+0000", 2024-09-15T21:48:06.563 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1313e-05 2024-09-15T21:48:06.563 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.563 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.563 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546444+0000", 2024-09-15T21:48:06.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9706000000000001e-05 2024-09-15T21:48:06.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546453+0000", 2024-09-15T21:48:06.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:48:06.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.564 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.564 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.564 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546519+0000", 2024-09-15T21:48:06.564 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5710999999999995e-05 2024-09-15T21:48:06.564 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:06.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546539+0000", 2024-09-15T21:48:06.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0275000000000002e-05 2024-09-15T21:48:06.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:06.565 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548763+0000", 2024-09-15T21:48:06.565 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002223226 2024-09-15T21:48:06.565 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.565 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.565 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.566 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548807+0000", 2024-09-15T21:48:06.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4598000000000002e-05 2024-09-15T21:48:06.566 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.566 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.566 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.566 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.566 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.566 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:19 2.11 e22/19 2:890f50f7:::626.00000000:head v 22'4, mlcod=22'2)", 2024-09-15T21:48:06.566 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:48:06.566 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 80.967850244000005, 2024-09-15T21:48:06.566 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022671280000000002, 2024-09-15T21:48:06.566 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.567 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.567 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.567 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:48:06.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.567 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.567 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546708+0000", 2024-09-15T21:48:06.567 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.567 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.567 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546709+0000", 2024-09-15T21:48:06.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4499999999999997e-07 2024-09-15T21:48:06.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546723+0000", 2024-09-15T21:48:06.568 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4623999999999999e-05 2024-09-15T21:48:06.568 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.568 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.568 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.568 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546734+0000", 2024-09-15T21:48:06.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0331e-05 2024-09-15T21:48:06.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546741+0000", 2024-09-15T21:48:06.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.075e-06 2024-09-15T21:48:06.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.569 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546954+0000", 2024-09-15T21:48:06.569 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021275600000000001 2024-09-15T21:48:06.569 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.569 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.569 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:06.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546971+0000", 2024-09-15T21:48:06.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7503000000000002e-05 2024-09-15T21:48:06.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:06.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548926+0000", 2024-09-15T21:48:06.570 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019550129999999998 2024-09-15T21:48:06.570 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.570 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.570 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.570 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548975+0000", 2024-09-15T21:48:06.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9280999999999998e-05 2024-09-15T21:48:06.571 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.571 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.571 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.571 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.571 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.571 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.d epoch 22/19 rep_tid 18 entries 22'2 (0'0) error 2:b73edb83:::203.00000004:head by mds.3.14:31 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 0'0)", 2024-09-15T21:48:06.571 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:48:06.571 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 80.967843893999998, 2024-09-15T21:48:06.571 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021738489999999998, 2024-09-15T21:48:06.571 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.571 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.571 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:48:06.571 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:48:06.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546715+0000", 2024-09-15T21:48:06.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.572 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.572 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.572 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.572 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546716+0000", 2024-09-15T21:48:06.572 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.615e-06 2024-09-15T21:48:06.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546730+0000", 2024-09-15T21:48:06.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3519999999999999e-05 2024-09-15T21:48:06.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.573 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.573 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546753+0000", 2024-09-15T21:48:06.573 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3643999999999999e-05 2024-09-15T21:48:06.573 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.573 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546770+0000", 2024-09-15T21:48:06.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6518999999999999e-05 2024-09-15T21:48:06.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546900+0000", 2024-09-15T21:48:06.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013049099999999999 2024-09-15T21:48:06.574 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.574 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.574 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.574 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548888+0000", 2024-09-15T21:48:06.574 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019880599999999998 2024-09-15T21:48:06.574 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.575 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.575 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.575 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.575 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:4 2.1a e22/19 2:5fdc322d:::61e.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:48:06.575 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:48:06.575 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 80.967735246000004, 2024-09-15T21:48:06.575 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002968769, 2024-09-15T21:48:06.575 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.575 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.575 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.575 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.575 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:48:06.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.576 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.576 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546823+0000", 2024-09-15T21:48:06.576 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.576 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.576 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546824+0000", 2024-09-15T21:48:06.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6499999999999999e-07 2024-09-15T21:48:06.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546831+0000", 2024-09-15T21:48:06.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9940000000000003e-06 2024-09-15T21:48:06.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.577 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.577 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.577 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546840+0000", 2024-09-15T21:48:06.577 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3689999999999996e-06 2024-09-15T21:48:06.577 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546847+0000", 2024-09-15T21:48:06.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0589999999999997e-06 2024-09-15T21:48:06.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547709+0000", 2024-09-15T21:48:06.578 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00086144699999999999 2024-09-15T21:48:06.578 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.578 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.578 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:06.578 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547730+0000", 2024-09-15T21:48:06.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1542000000000001e-05 2024-09-15T21:48:06.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:06.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549750+0000", 2024-09-15T21:48:06.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002020165 2024-09-15T21:48:06.579 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.579 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.579 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.579 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549792+0000", 2024-09-15T21:48:06.579 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1628000000000002e-05 2024-09-15T21:48:06.580 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.580 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.580 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.580 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.580 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.580 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:20 2.11 e22/19 2:890f50f7:::626.00000000:head v 22'5, mlcod=22'2)", 2024-09-15T21:48:06.580 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:48:06.580 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 80.966710637999995, 2024-09-15T21:48:06.580 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017734579999999999, 2024-09-15T21:48:06.580 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.580 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.581 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.581 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:48:06.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.581 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.581 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:48:06.581 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.581 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.581 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547848+0000", 2024-09-15T21:48:06.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:48:06.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547856+0000", 2024-09-15T21:48:06.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7999999999999999e-06 2024-09-15T21:48:06.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.582 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.582 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.582 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547867+0000", 2024-09-15T21:48:06.582 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0971999999999999e-05 2024-09-15T21:48:06.582 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547881+0000", 2024-09-15T21:48:06.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4168999999999999e-05 2024-09-15T21:48:06.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547935+0000", 2024-09-15T21:48:06.583 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3721999999999998e-05 2024-09-15T21:48:06.583 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.583 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.583 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:06.583 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547976+0000", 2024-09-15T21:48:06.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.1248999999999998e-05 2024-09-15T21:48:06.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:06.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549559+0000", 2024-09-15T21:48:06.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001582815 2024-09-15T21:48:06.584 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.584 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.584 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.584 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549621+0000", 2024-09-15T21:48:06.584 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2156999999999995e-05 2024-09-15T21:48:06.584 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.584 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.585 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.585 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.585 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:13 2.1a e22/19 2:594e21d0:::623.00000000:head v 22'6, mlcod=22'2)", 2024-09-15T21:48:06.585 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:48:06.585 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 80.966645642000003, 2024-09-15T21:48:06.585 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019085300000000001, 2024-09-15T21:48:06.585 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.585 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.585 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.585 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.585 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:48:06.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547913+0000", 2024-09-15T21:48:06.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.586 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.586 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.586 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.586 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547914+0000", 2024-09-15T21:48:06.586 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.1500000000000004e-07 2024-09-15T21:48:06.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547920+0000", 2024-09-15T21:48:06.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5239999999999997e-06 2024-09-15T21:48:06.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.587 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547930+0000", 2024-09-15T21:48:06.587 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5109999999999992e-06 2024-09-15T21:48:06.587 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.587 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.587 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547937+0000", 2024-09-15T21:48:06.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5410000000000003e-06 2024-09-15T21:48:06.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548518+0000", 2024-09-15T21:48:06.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00058074299999999999 2024-09-15T21:48:06.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.588 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:06.588 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548538+0000", 2024-09-15T21:48:06.588 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9959e-05 2024-09-15T21:48:06.588 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.588 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:06.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549795+0000", 2024-09-15T21:48:06.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012570120000000001 2024-09-15T21:48:06.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.589 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.589 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549821+0000", 2024-09-15T21:48:06.589 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6625e-05 2024-09-15T21:48:06.589 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.589 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.589 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.589 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.590 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:23 2.18 e22/19 2:1c1db442:::103.00000000.inode:head v 22'2, mlcod=0'0)", 2024-09-15T21:48:06.590 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:48:06.590 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 80.966584224000002, 2024-09-15T21:48:06.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002713198, 2024-09-15T21:48:06.590 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.590 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.590 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.590 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:48:06.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547974+0000", 2024-09-15T21:48:06.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547975+0000", 2024-09-15T21:48:06.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3900000000000004e-07 2024-09-15T21:48:06.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547983+0000", 2024-09-15T21:48:06.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6690000000000006e-06 2024-09-15T21:48:06.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547994+0000", 2024-09-15T21:48:06.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1503e-05 2024-09-15T21:48:06.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548002+0000", 2024-09-15T21:48:06.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1340000000000004e-06 2024-09-15T21:48:06.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548098+0000", 2024-09-15T21:48:06.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.5791000000000005e-05 2024-09-15T21:48:06.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:06.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548118+0000", 2024-09-15T21:48:06.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0129e-05 2024-09-15T21:48:06.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:06.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550643+0000", 2024-09-15T21:48:06.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002524728 2024-09-15T21:48:06.594 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550688+0000", 2024-09-15T21:48:06.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4404999999999999e-05 2024-09-15T21:48:06.595 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.595 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.595 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.595 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.3.14:14 2.1a e22/19 2:594e21d0:::623.00000000:head v 22'7, mlcod=22'2)", 2024-09-15T21:48:06.595 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:48:06.595 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 80.966569264, 2024-09-15T21:48:06.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024358579999999999, 2024-09-15T21:48:06.595 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.595 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.596 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.596 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:48:06.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547989+0000", 2024-09-15T21:48:06.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.547990+0000", 2024-09-15T21:48:06.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5799999999999999e-07 2024-09-15T21:48:06.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548011+0000", 2024-09-15T21:48:06.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0942999999999999e-05 2024-09-15T21:48:06.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548020+0000", 2024-09-15T21:48:06.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2380000000000003e-06 2024-09-15T21:48:06.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548027+0000", 2024-09-15T21:48:06.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.6429999999999998e-06 2024-09-15T21:48:06.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548848+0000", 2024-09-15T21:48:06.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082167999999999998 2024-09-15T21:48:06.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:06.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548868+0000", 2024-09-15T21:48:06.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9791000000000001e-05 2024-09-15T21:48:06.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:06.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550387+0000", 2024-09-15T21:48:06.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001519322 2024-09-15T21:48:06.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550425+0000", 2024-09-15T21:48:06.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7682999999999998e-05 2024-09-15T21:48:06.600 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.600 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.600 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.600 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:35 2.f 2:f426ad4f:::mds2_openfiles.0:head [omap-clear,omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:06.601 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:48:06.601 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 80.776457578999995, 2024-09-15T21:48:06.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017328949999999999, 2024-09-15T21:48:06.601 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.601 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.601 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.601 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:48:06.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738101+0000", 2024-09-15T21:48:06.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.602 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738103+0000", 2024-09-15T21:48:06.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0020000000000001e-06 2024-09-15T21:48:06.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738112+0000", 2024-09-15T21:48:06.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.2220000000000008e-06 2024-09-15T21:48:06.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738127+0000", 2024-09-15T21:48:06.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5016e-05 2024-09-15T21:48:06.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738137+0000", 2024-09-15T21:48:06.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0301e-05 2024-09-15T21:48:06.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738216+0000", 2024-09-15T21:48:06.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8356000000000007e-05 2024-09-15T21:48:06.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:06.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738340+0000", 2024-09-15T21:48:06.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012372800000000001 2024-09-15T21:48:06.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:06.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.738453+0000", 2024-09-15T21:48:06.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000113785 2024-09-15T21:48:06.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:06.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739115+0000", 2024-09-15T21:48:06.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00066178500000000002 2024-09-15T21:48:06.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:06.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739745+0000", 2024-09-15T21:48:06.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00063017900000000003 2024-09-15T21:48:06.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.606 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:06.606 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739783+0000", 2024-09-15T21:48:06.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7941000000000001e-05 2024-09-15T21:48:06.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.739834+0000", 2024-09-15T21:48:06.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.058e-05 2024-09-15T21:48:06.607 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.607 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.607 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.607 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(mds.4.16:13 2.13 e22/19 2:c81eaac8:::62d.00000000:head v 22'8, mlcod=22'6)", 2024-09-15T21:48:06.607 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:48:06.607 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 74.949052519999995, 2024-09-15T21:48:06.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002151778, 2024-09-15T21:48:06.608 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.608 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.608 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:06.608 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:48:06.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565506+0000", 2024-09-15T21:48:06.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565507+0000", 2024-09-15T21:48:06.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3340000000000001e-06 2024-09-15T21:48:06.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565518+0000", 2024-09-15T21:48:06.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0339999999999999e-05 2024-09-15T21:48:06.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565536+0000", 2024-09-15T21:48:06.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8647e-05 2024-09-15T21:48:06.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565545+0000", 2024-09-15T21:48:06.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1430000000000004e-06 2024-09-15T21:48:06.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565611+0000", 2024-09-15T21:48:06.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5129000000000006e-05 2024-09-15T21:48:06.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:06.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565634+0000", 2024-09-15T21:48:06.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3173e-05 2024-09-15T21:48:06.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:06.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567561+0000", 2024-09-15T21:48:06.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019268180000000001 2024-09-15T21:48:06.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567658+0000", 2024-09-15T21:48:06.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7194000000000005e-05 2024-09-15T21:48:06.612 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.612 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.612 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.612 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "pg_update_log_missing(2.d epoch 22/19 rep_tid 24 entries 22'3 (0'0) error 2:b2057508:::204.00000002:head by mds.4.16:29 0.000000 -2 [r=-2+0b] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 22'1)", 2024-09-15T21:48:06.612 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:48:06.612 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 74.948891657999994, 2024-09-15T21:48:06.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026915049999999999, 2024-09-15T21:48:06.612 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:06.612 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:06.612 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "reached pg", 2024-09-15T21:48:06.613 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:06.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:06.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:48:06.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:06.613 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565667+0000", 2024-09-15T21:48:06.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:06.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:06.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565668+0000", 2024-09-15T21:48:06.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.113e-06 2024-09-15T21:48:06.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:06.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565684+0000", 2024-09-15T21:48:06.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5943999999999998e-05 2024-09-15T21:48:06.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:06.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565705+0000", 2024-09-15T21:48:06.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0761999999999999e-05 2024-09-15T21:48:06.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:06.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565724+0000", 2024-09-15T21:48:06.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9208e-05 2024-09-15T21:48:06.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:06.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565817+0000", 2024-09-15T21:48:06.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3218000000000001e-05 2024-09-15T21:48:06.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:06.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:06.616 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:06.616 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568358+0000", 2024-09-15T21:48:06.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00254126 2024-09-15T21:48:06.616 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.616 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.616 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.616 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:06.616 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:06.616 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:48:06.618 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:48:06.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:06.574+0000 7f834bfff640 1 -- 172.21.15.100:0/1927941529 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "nodeep-scrub"}]=0 nodeep-scrub is set v53) ==== 97+0+0 (secure 0 0 0) 0x7f835c05ccd0 con 0x7f8364155440 2024-09-15T21:48:06.619 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is set 2024-09-15T21:48:06.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:06.578+0000 7f8369903640 1 -- 172.21.15.100:0/1927941529 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f835006c2e0 msgr2=0x7f835006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:06.578+0000 7f8369903640 1 --2- 172.21.15.100:0/1927941529 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f835006c2e0 0x7f835006e7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8354006fd0 tx=0x7f8354008040 comp rx=0 tx=0).stop 2024-09-15T21:48:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:06.578+0000 7f8369903640 1 -- 172.21.15.100:0/1927941529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8364155440 msgr2=0x7f83640c9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:06.578+0000 7f8369903640 1 --2- 172.21.15.100:0/1927941529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8364155440 0x7f83640c9bc0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f835c00c700 tx=0x7f835c00cbc0 comp rx=0 tx=0).stop 2024-09-15T21:48:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:06.578+0000 7f8369903640 1 -- 172.21.15.100:0/1927941529 shutdown_connections 2024-09-15T21:48:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:06.578+0000 7f8369903640 1 --2- 172.21.15.100:0/1927941529 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f835006c2e0 0x7f835006e7a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:06.578+0000 7f8369903640 1 --2- 172.21.15.100:0/1927941529 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f83640cb7d0 0x7f83640c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:06.578+0000 7f8369903640 1 --2- 172.21.15.100:0/1927941529 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8364155440 0x7f83640c9bc0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:06.578+0000 7f8369903640 1 --2- 172.21.15.100:0/1927941529 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f836406d0e0 0x7f8364158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:06.578+0000 7f8369903640 1 -- 172.21.15.100:0/1927941529 >> 172.21.15.100:0/1927941529 conn(0x7f83640680d0 msgr2=0x7f83640cc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:06.578+0000 7f8369903640 1 -- 172.21.15.100:0/1927941529 shutdown_connections 2024-09-15T21:48:06.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:06.578+0000 7f8369903640 1 -- 172.21.15.100:0/1927941529 wait complete. 2024-09-15T21:48:06.621 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:48:06.616+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:48:06.718 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:48:06.717+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:48:06.763 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:48:06.763 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:48:06.763 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:48:06.763 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:48:06.772 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:48:06.858 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:48:06.856+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:48:06.917 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:48:06.917 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:48:06.917 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:48:06.917 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:48:06.917 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:48:06.920 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:48:06.919+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:48:06.925 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:48:07.022 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:48:07.020+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:48:07.070 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:48:07.070 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:48:07.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:48:07.070 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:48:07.070 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.070 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:24 2.7 2:e0b41d06:::609.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:07.070 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:48:07.070 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.653981795999997, 2024-09-15T21:48:07.070 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0050344960000000003, 2024-09-15T21:48:07.071 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.071 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.071 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.071 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:48:07.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.071 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.071 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.071 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.071 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:48:07.071 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413468+0000", 2024-09-15T21:48:07.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0199999999999998e-07 2024-09-15T21:48:07.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.072 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413470+0000", 2024-09-15T21:48:07.072 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4329999999999998e-06 2024-09-15T21:48:07.072 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.072 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.072 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413579+0000", 2024-09-15T21:48:07.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000108593 2024-09-15T21:48:07.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413587+0000", 2024-09-15T21:48:07.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3100000000000001e-06 2024-09-15T21:48:07.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.073 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.073 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.073 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413691+0000", 2024-09-15T21:48:07.073 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00010357 2024-09-15T21:48:07.073 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413887+0000", 2024-09-15T21:48:07.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019666799999999999 2024-09-15T21:48:07.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:07.074 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414027+0000", 2024-09-15T21:48:07.074 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013996700000000001 2024-09-15T21:48:07.074 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.074 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.074 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416360+0000", 2024-09-15T21:48:07.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002332204 2024-09-15T21:48:07.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418437+0000", 2024-09-15T21:48:07.075 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002077279 2024-09-15T21:48:07.075 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.075 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.075 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.075 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418469+0000", 2024-09-15T21:48:07.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2359e-05 2024-09-15T21:48:07.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.076 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.076 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418502+0000", 2024-09-15T21:48:07.076 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2811000000000001e-05 2024-09-15T21:48:07.076 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.076 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.076 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.076 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.076 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.076 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:25 2.7 2:e0b41d06:::609.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:07.076 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:48:07.077 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.653977756000003, 2024-09-15T21:48:07.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0053538780000000003, 2024-09-15T21:48:07.077 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.077 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.077 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.077 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.077 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:48:07.077 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.077 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.077 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.077 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:48:07.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413472+0000", 2024-09-15T21:48:07.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2000000000000001e-07 2024-09-15T21:48:07.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.078 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.078 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.078 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413474+0000", 2024-09-15T21:48:07.078 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5450000000000002e-06 2024-09-15T21:48:07.078 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413612+0000", 2024-09-15T21:48:07.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000137727 2024-09-15T21:48:07.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413618+0000", 2024-09-15T21:48:07.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.241e-06 2024-09-15T21:48:07.079 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.079 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.079 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.079 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414443+0000", 2024-09-15T21:48:07.079 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082447 2024-09-15T21:48:07.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414523+0000", 2024-09-15T21:48:07.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0059e-05 2024-09-15T21:48:07.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:07.080 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414653+0000", 2024-09-15T21:48:07.080 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013031699999999999 2024-09-15T21:48:07.080 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.080 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.080 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416366+0000", 2024-09-15T21:48:07.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001712917 2024-09-15T21:48:07.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418769+0000", 2024-09-15T21:48:07.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024031529999999999 2024-09-15T21:48:07.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.081 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.081 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418802+0000", 2024-09-15T21:48:07.081 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2925000000000001e-05 2024-09-15T21:48:07.081 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.081 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.082 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.082 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.418825+0000", 2024-09-15T21:48:07.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3204e-05 2024-09-15T21:48:07.082 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.082 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.082 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.082 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.082 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.082 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:1 2.14 2:292cf221:::200.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:07.082 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:48:07.082 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.653969735000004, 2024-09-15T21:48:07.082 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0059708640000000002, 2024-09-15T21:48:07.082 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.082 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.084 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.084 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:48:07.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.084 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.084 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.084 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.084 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:48:07.084 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.085 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.085 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413480+0000", 2024-09-15T21:48:07.085 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7300000000000002e-07 2024-09-15T21:48:07.085 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.085 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413482+0000", 2024-09-15T21:48:07.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2359999999999999e-06 2024-09-15T21:48:07.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414323+0000", 2024-09-15T21:48:07.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00084095399999999999 2024-09-15T21:48:07.086 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.086 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.086 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.086 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414332+0000", 2024-09-15T21:48:07.086 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0880000000000001e-06 2024-09-15T21:48:07.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415439+0000", 2024-09-15T21:48:07.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011071869999999999 2024-09-15T21:48:07.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.087 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415554+0000", 2024-09-15T21:48:07.087 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000114499 2024-09-15T21:48:07.087 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.087 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.087 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:48:07.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.415682+0000", 2024-09-15T21:48:07.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012802500000000001 2024-09-15T21:48:07.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.417957+0000", 2024-09-15T21:48:07.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022746720000000002 2024-09-15T21:48:07.088 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.088 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.088 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.088 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419385+0000", 2024-09-15T21:48:07.088 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001428826 2024-09-15T21:48:07.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419420+0000", 2024-09-15T21:48:07.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4875000000000001e-05 2024-09-15T21:48:07.089 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.089 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.089 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.089 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.419450+0000", 2024-09-15T21:48:07.089 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0128999999999999e-05 2024-09-15T21:48:07.090 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.090 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.090 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.090 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.090 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:3 2.14 2:287034c4:::1.00000000.inode:head [writefull 0~582] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:07.090 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:48:07.090 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.653965916999994, 2024-09-15T21:48:07.090 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0087186220000000005, 2024-09-15T21:48:07.090 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.090 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.090 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.090 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.090 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:48:07.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413483+0000", 2024-09-15T21:48:07.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.091 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.091 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413484+0000", 2024-09-15T21:48:07.091 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.72e-07 2024-09-15T21:48:07.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413487+0000", 2024-09-15T21:48:07.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.534e-06 2024-09-15T21:48:07.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.092 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414353+0000", 2024-09-15T21:48:07.092 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00086616999999999998 2024-09-15T21:48:07.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.092 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.092 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414359+0000", 2024-09-15T21:48:07.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3279999999999996e-06 2024-09-15T21:48:07.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416041+0000", 2024-09-15T21:48:07.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001682467 2024-09-15T21:48:07.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.093 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.093 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416158+0000", 2024-09-15T21:48:07.093 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117242 2024-09-15T21:48:07.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:48:07.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416284+0000", 2024-09-15T21:48:07.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012509000000000001 2024-09-15T21:48:07.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.417992+0000", 2024-09-15T21:48:07.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017087059999999999 2024-09-15T21:48:07.094 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.094 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.094 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422142+0000", 2024-09-15T21:48:07.094 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041501180000000004 2024-09-15T21:48:07.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422176+0000", 2024-09-15T21:48:07.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4109999999999997e-05 2024-09-15T21:48:07.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.095 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.095 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.095 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422202+0000", 2024-09-15T21:48:07.095 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5585e-05 2024-09-15T21:48:07.095 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.095 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.095 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.095 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.096 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.0.4:32 2.14 2:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e21)", 2024-09-15T21:48:07.096 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:48:07.096 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 98.653960491999996, 2024-09-15T21:48:07.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0088000370000000001, 2024-09-15T21:48:07.096 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.096 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.096 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:48:07.096 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.096 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.096 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.096 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:48:07.096 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:48:07.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413489+0000", 2024-09-15T21:48:07.097 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:48:07.097 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.097 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.097 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.097 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.413491+0000", 2024-09-15T21:48:07.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.773e-06 2024-09-15T21:48:07.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414372+0000", 2024-09-15T21:48:07.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00088122000000000005 2024-09-15T21:48:07.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.098 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.414378+0000", 2024-09-15T21:48:07.098 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4380000000000003e-06 2024-09-15T21:48:07.098 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.098 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.098 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416631+0000", 2024-09-15T21:48:07.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022530319999999999 2024-09-15T21:48:07.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.416765+0000", 2024-09-15T21:48:07.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013423499999999999 2024-09-15T21:48:07.099 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.099 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.099 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.099 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:28.422289+0000", 2024-09-15T21:48:07.099 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055240530000000001 2024-09-15T21:48:07.099 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.099 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.100 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.100 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.100 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:6 2.7 2:e455cf45:::60b.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:07.100 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:48:07.100 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 92.612392173000003, 2024-09-15T21:48:07.100 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056900609999999997, 2024-09-15T21:48:07.100 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.100 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.100 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.100 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.100 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.100 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.100 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:48:07.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455057+0000", 2024-09-15T21:48:07.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.101 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.101 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455058+0000", 2024-09-15T21:48:07.101 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.51e-07 2024-09-15T21:48:07.101 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.101 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.102 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455061+0000", 2024-09-15T21:48:07.102 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.145e-06 2024-09-15T21:48:07.102 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.102 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.102 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455158+0000", 2024-09-15T21:48:07.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.7101000000000004e-05 2024-09-15T21:48:07.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455165+0000", 2024-09-15T21:48:07.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1019999999999999e-06 2024-09-15T21:48:07.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.103 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.103 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456433+0000", 2024-09-15T21:48:07.103 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012678259999999999 2024-09-15T21:48:07.103 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.103 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456517+0000", 2024-09-15T21:48:07.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.4373e-05 2024-09-15T21:48:07.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.104 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:07.104 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456677+0000", 2024-09-15T21:48:07.104 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015963300000000001 2024-09-15T21:48:07.104 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.104 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458636+0000", 2024-09-15T21:48:07.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019594030000000002 2024-09-15T21:48:07.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460665+0000", 2024-09-15T21:48:07.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020283549999999999 2024-09-15T21:48:07.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.105 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.105 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.105 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460707+0000", 2024-09-15T21:48:07.105 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2398000000000003e-05 2024-09-15T21:48:07.105 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.106 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.106 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460747+0000", 2024-09-15T21:48:07.106 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0274000000000001e-05 2024-09-15T21:48:07.106 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.106 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.106 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.106 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.106 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.106 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:9 2.17 2:e8b9846e:::60d.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:07.106 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:48:07.107 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 92.612384978999998, 2024-09-15T21:48:07.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0086420030000000005, 2024-09-15T21:48:07.107 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.107 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.107 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.107 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.107 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:48:07.107 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.107 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.107 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.107 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455064+0000", 2024-09-15T21:48:07.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.108 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.108 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455065+0000", 2024-09-15T21:48:07.108 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3700000000000001e-07 2024-09-15T21:48:07.108 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.108 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.109 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455068+0000", 2024-09-15T21:48:07.109 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.915e-06 2024-09-15T21:48:07.109 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.109 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.109 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455366+0000", 2024-09-15T21:48:07.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00029825400000000002 2024-09-15T21:48:07.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455374+0000", 2024-09-15T21:48:07.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6240000000000008e-06 2024-09-15T21:48:07.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.110 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.110 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456199+0000", 2024-09-15T21:48:07.110 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00082448500000000002 2024-09-15T21:48:07.110 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.110 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457034+0000", 2024-09-15T21:48:07.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00083553399999999997 2024-09-15T21:48:07.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:48:07.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457171+0000", 2024-09-15T21:48:07.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013653600000000001 2024-09-15T21:48:07.111 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.111 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.111 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.111 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462663+0000", 2024-09-15T21:48:07.111 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054924420000000002 2024-09-15T21:48:07.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463613+0000", 2024-09-15T21:48:07.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094971499999999996 2024-09-15T21:48:07.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463663+0000", 2024-09-15T21:48:07.112 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9636999999999999e-05 2024-09-15T21:48:07.112 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.112 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.112 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.112 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.463706+0000", 2024-09-15T21:48:07.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3523999999999999e-05 2024-09-15T21:48:07.113 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.113 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.113 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.113 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.113 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.113 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:10 2.17 2:e8b9846e:::60d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:07.113 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:48:07.113 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 92.612380451000007, 2024-09-15T21:48:07.113 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090408420000000003, 2024-09-15T21:48:07.113 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.113 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.113 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.113 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:48:07.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:48:07.114 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.114 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.114 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.114 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.114 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455069+0000", 2024-09-15T21:48:07.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.79e-07 2024-09-15T21:48:07.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455072+0000", 2024-09-15T21:48:07.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6189999999999998e-06 2024-09-15T21:48:07.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.115 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455403+0000", 2024-09-15T21:48:07.115 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00033167299999999999 2024-09-15T21:48:07.115 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.115 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.115 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455420+0000", 2024-09-15T21:48:07.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6518999999999999e-05 2024-09-15T21:48:07.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457448+0000", 2024-09-15T21:48:07.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020275710000000001 2024-09-15T21:48:07.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.116 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.116 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.116 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:48:07.116 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3844999999999994e-05 2024-09-15T21:48:07.116 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:48:07.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457684+0000", 2024-09-15T21:48:07.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00015239 2024-09-15T21:48:07.117 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.117 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.117 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.117 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462678+0000", 2024-09-15T21:48:07.117 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0049939279999999999 2024-09-15T21:48:07.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.120 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.120 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.120 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464017+0000", 2024-09-15T21:48:07.120 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013397140000000001 2024-09-15T21:48:07.120 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464071+0000", 2024-09-15T21:48:07.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3735e-05 2024-09-15T21:48:07.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.121 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.121 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.121 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464110+0000", 2024-09-15T21:48:07.121 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8569000000000003e-05 2024-09-15T21:48:07.121 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.121 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.121 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.121 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.122 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:1 2.17 2:ea85266a:::201.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:07.122 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:48:07.122 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 92.612368837999995, 2024-09-15T21:48:07.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091802889999999995, 2024-09-15T21:48:07.122 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.122 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.122 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.122 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.122 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.122 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:48:07.122 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455080+0000", 2024-09-15T21:48:07.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455081+0000", 2024-09-15T21:48:07.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8599999999999999e-07 2024-09-15T21:48:07.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.123 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.123 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.123 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455084+0000", 2024-09-15T21:48:07.123 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7810000000000001e-06 2024-09-15T21:48:07.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455536+0000", 2024-09-15T21:48:07.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00045219100000000001 2024-09-15T21:48:07.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455583+0000", 2024-09-15T21:48:07.124 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7308999999999997e-05 2024-09-15T21:48:07.124 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.124 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.124 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457998+0000", 2024-09-15T21:48:07.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0024154290000000002 2024-09-15T21:48:07.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458147+0000", 2024-09-15T21:48:07.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014834599999999999 2024-09-15T21:48:07.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 5", 2024-09-15T21:48:07.125 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458265+0000", 2024-09-15T21:48:07.125 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000118458 2024-09-15T21:48:07.125 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.125 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.125 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464134+0000", 2024-09-15T21:48:07.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.005869038 2024-09-15T21:48:07.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464187+0000", 2024-09-15T21:48:07.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2343999999999999e-05 2024-09-15T21:48:07.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.126 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.126 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464222+0000", 2024-09-15T21:48:07.126 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5447000000000001e-05 2024-09-15T21:48:07.126 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.126 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.127 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.127 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.464261+0000", 2024-09-15T21:48:07.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8659999999999999e-05 2024-09-15T21:48:07.127 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.127 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.127 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.127 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.127 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.127 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:22 2.1a 2:5fd68f60:::61d.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:07.127 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:48:07.127 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 86.591746459000007, 2024-09-15T21:48:07.127 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0048468290000000004, 2024-09-15T21:48:07.127 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.127 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.128 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.128 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:48:07.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.128 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.128 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:48:07.128 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.128 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.128 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475703+0000", 2024-09-15T21:48:07.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.35e-07 2024-09-15T21:48:07.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475706+0000", 2024-09-15T21:48:07.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.994e-06 2024-09-15T21:48:07.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.129 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.129 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.129 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475770+0000", 2024-09-15T21:48:07.129 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3904000000000003e-05 2024-09-15T21:48:07.129 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475775+0000", 2024-09-15T21:48:07.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2490000000000001e-06 2024-09-15T21:48:07.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476502+0000", 2024-09-15T21:48:07.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00072697300000000001 2024-09-15T21:48:07.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.130 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.130 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.130 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476611+0000", 2024-09-15T21:48:07.130 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0001091 2024-09-15T21:48:07.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:48:07.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476807+0000", 2024-09-15T21:48:07.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000195892 2024-09-15T21:48:07.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.479678+0000", 2024-09-15T21:48:07.131 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028704709999999999 2024-09-15T21:48:07.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.131 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.131 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.131 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480444+0000", 2024-09-15T21:48:07.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00076602099999999998 2024-09-15T21:48:07.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480502+0000", 2024-09-15T21:48:07.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8313999999999998e-05 2024-09-15T21:48:07.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.132 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.132 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.132 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480550+0000", 2024-09-15T21:48:07.132 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7676000000000002e-05 2024-09-15T21:48:07.132 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.132 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.133 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.133 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.2.12:28 2.11 2:8975f766:::202.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:07.133 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:48:07.133 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 86.591739283999999, 2024-09-15T21:48:07.133 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051250929999999998, 2024-09-15T21:48:07.133 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.133 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.133 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-15T21:48:07.133 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.133 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.133 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.133 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:48:07.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:48:07.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.134 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.134 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475710+0000", 2024-09-15T21:48:07.134 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1e-07 2024-09-15T21:48:07.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.134 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475723+0000", 2024-09-15T21:48:07.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2758000000000001e-05 2024-09-15T21:48:07.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475972+0000", 2024-09-15T21:48:07.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024886299999999999 2024-09-15T21:48:07.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.135 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.135 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.135 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.475979+0000", 2024-09-15T21:48:07.135 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0230000000000004e-06 2024-09-15T21:48:07.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476560+0000", 2024-09-15T21:48:07.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00058097599999999998 2024-09-15T21:48:07.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.476689+0000", 2024-09-15T21:48:07.136 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000129486 2024-09-15T21:48:07.136 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.136 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.136 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.136 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:40.480835+0000", 2024-09-15T21:48:07.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041457769999999998 2024-09-15T21:48:07.137 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.137 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.137 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.137 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.137 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.137 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:4 2.1a 2:5fdc322d:::61e.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:07.137 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:48:07.137 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 81.522824059000001, 2024-09-15T21:48:07.137 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054420570000000001, 2024-09-15T21:48:07.137 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.137 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.137 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.138 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:48:07.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544625+0000", 2024-09-15T21:48:07.138 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.138 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.138 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.138 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.138 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544626+0000", 2024-09-15T21:48:07.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8100000000000003e-07 2024-09-15T21:48:07.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544629+0000", 2024-09-15T21:48:07.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9869999999999999e-06 2024-09-15T21:48:07.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.139 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.139 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544775+0000", 2024-09-15T21:48:07.139 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014610599999999999 2024-09-15T21:48:07.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.139 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544782+0000", 2024-09-15T21:48:07.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4109999999999999e-06 2024-09-15T21:48:07.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545617+0000", 2024-09-15T21:48:07.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00083468300000000002 2024-09-15T21:48:07.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.140 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.140 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.140 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545690+0000", 2024-09-15T21:48:07.140 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3545000000000001e-05 2024-09-15T21:48:07.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:48:07.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545803+0000", 2024-09-15T21:48:07.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000112083 2024-09-15T21:48:07.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548353+0000", 2024-09-15T21:48:07.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0025507220000000001 2024-09-15T21:48:07.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.141 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.141 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.141 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549964+0000", 2024-09-15T21:48:07.141 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001610382 2024-09-15T21:48:07.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550021+0000", 2024-09-15T21:48:07.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6898999999999999e-05 2024-09-15T21:48:07.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.142 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.142 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.142 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550067+0000", 2024-09-15T21:48:07.142 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6758000000000002e-05 2024-09-15T21:48:07.142 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.142 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.142 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.142 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.143 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:13 2.1a 2:594e21d0:::623.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:07.143 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:48:07.143 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 81.522818780999998, 2024-09-15T21:48:07.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0056323399999999996, 2024-09-15T21:48:07.143 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.143 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.143 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.143 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.143 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.143 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.143 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:48:07.143 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:48:07.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544631+0000", 2024-09-15T21:48:07.144 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0600000000000001e-07 2024-09-15T21:48:07.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.144 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.144 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.144 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544634+0000", 2024-09-15T21:48:07.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.58e-06 2024-09-15T21:48:07.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544800+0000", 2024-09-15T21:48:07.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016576899999999999 2024-09-15T21:48:07.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.145 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544808+0000", 2024-09-15T21:48:07.145 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5179999999999998e-06 2024-09-15T21:48:07.145 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.145 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.145 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546075+0000", 2024-09-15T21:48:07.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012676969999999999 2024-09-15T21:48:07.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546179+0000", 2024-09-15T21:48:07.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000103935 2024-09-15T21:48:07.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.146 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:48:07.146 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546286+0000", 2024-09-15T21:48:07.146 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106758 2024-09-15T21:48:07.146 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.146 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548528+0000", 2024-09-15T21:48:07.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022420510000000001 2024-09-15T21:48:07.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550119+0000", 2024-09-15T21:48:07.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0015907079999999999 2024-09-15T21:48:07.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.147 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.147 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.147 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550202+0000", 2024-09-15T21:48:07.147 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3374000000000005e-05 2024-09-15T21:48:07.147 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.148 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.148 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550263+0000", 2024-09-15T21:48:07.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0643999999999997e-05 2024-09-15T21:48:07.148 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.148 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.148 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.148 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.148 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.148 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:14 2.1a 2:594e21d0:::623.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:07.148 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:48:07.148 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 81.522813810000002, 2024-09-15T21:48:07.148 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0061905470000000002, 2024-09-15T21:48:07.149 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.149 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.149 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.149 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:48:07.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.149 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.149 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.149 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:48:07.149 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.149 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544636+0000", 2024-09-15T21:48:07.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6899999999999999e-07 2024-09-15T21:48:07.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544638+0000", 2024-09-15T21:48:07.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3920000000000001e-06 2024-09-15T21:48:07.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.150 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.150 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.150 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544819+0000", 2024-09-15T21:48:07.150 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018065099999999999 2024-09-15T21:48:07.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544825+0000", 2024-09-15T21:48:07.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7239999999999997e-06 2024-09-15T21:48:07.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546583+0000", 2024-09-15T21:48:07.151 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017588219999999999 2024-09-15T21:48:07.151 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.151 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.151 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.151 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546665+0000", 2024-09-15T21:48:07.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1693000000000005e-05 2024-09-15T21:48:07.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:48:07.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546808+0000", 2024-09-15T21:48:07.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142993 2024-09-15T21:48:07.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.152 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548533+0000", 2024-09-15T21:48:07.152 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0017246 2024-09-15T21:48:07.152 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.152 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.152 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550606+0000", 2024-09-15T21:48:07.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020733969999999998 2024-09-15T21:48:07.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550783+0000", 2024-09-15T21:48:07.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000176565 2024-09-15T21:48:07.153 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.153 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.153 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.153 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.550826+0000", 2024-09-15T21:48:07.153 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3441e-05 2024-09-15T21:48:07.153 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.154 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.154 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.154 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.154 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.154 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:20 2.11 2:890f50f7:::626.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:07.154 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:48:07.154 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 81.522806088999999, 2024-09-15T21:48:07.154 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052813010000000004, 2024-09-15T21:48:07.154 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.154 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.154 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.154 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:48:07.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:48:07.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.155 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.155 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.155 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.155 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544643+0000", 2024-09-15T21:48:07.155 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3799999999999999e-07 2024-09-15T21:48:07.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544647+0000", 2024-09-15T21:48:07.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6569999999999999e-06 2024-09-15T21:48:07.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.156 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544858+0000", 2024-09-15T21:48:07.156 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021069900000000001 2024-09-15T21:48:07.156 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.156 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.156 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544862+0000", 2024-09-15T21:48:07.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4900000000000002e-06 2024-09-15T21:48:07.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545610+0000", 2024-09-15T21:48:07.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00074786999999999998 2024-09-15T21:48:07.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.157 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545706+0000", 2024-09-15T21:48:07.157 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6271e-05 2024-09-15T21:48:07.157 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.157 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.157 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 4", 2024-09-15T21:48:07.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545834+0000", 2024-09-15T21:48:07.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012759499999999999 2024-09-15T21:48:07.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548554+0000", 2024-09-15T21:48:07.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0027199339999999998 2024-09-15T21:48:07.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.158 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.158 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549818+0000", 2024-09-15T21:48:07.158 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012642199999999999 2024-09-15T21:48:07.158 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.158 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549884+0000", 2024-09-15T21:48:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5843e-05 2024-09-15T21:48:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549925+0000", 2024-09-15T21:48:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0484000000000001e-05 2024-09-15T21:48:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.159 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.3.14:2 2.7 2:e4812e9a:::403.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:48:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 81.522797412000003, 2024-09-15T21:48:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051106110000000001, 2024-09-15T21:48:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:48:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.160 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:48:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544652+0000", 2024-09-15T21:48:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2700000000000001e-07 2024-09-15T21:48:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.544667+0000", 2024-09-15T21:48:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4746000000000001e-05 2024-09-15T21:48:07.161 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545021+0000", 2024-09-15T21:48:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00035412900000000001 2024-09-15T21:48:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.545052+0000", 2024-09-15T21:48:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1158000000000002e-05 2024-09-15T21:48:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546050+0000", 2024-09-15T21:48:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00099782800000000008 2024-09-15T21:48:07.162 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546291+0000", 2024-09-15T21:48:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00024093500000000001 2024-09-15T21:48:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.546434+0000", 2024-09-15T21:48:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014308600000000001 2024-09-15T21:48:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.548583+0000", 2024-09-15T21:48:07.163 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021491029999999999 2024-09-15T21:48:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549695+0000", 2024-09-15T21:48:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011116419999999999 2024-09-15T21:48:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549726+0000", 2024-09-15T21:48:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1195000000000002e-05 2024-09-15T21:48:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.164 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:45.549763+0000", 2024-09-15T21:48:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6562e-05 2024-09-15T21:48:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:17 2.14 2:2eebc8f4:::62f.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:48:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 75.502367800000002, 2024-09-15T21:48:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071004800000000002, 2024-09-15T21:48:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.165 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:48:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565082+0000", 2024-09-15T21:48:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.166 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565083+0000", 2024-09-15T21:48:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.689e-06 2024-09-15T21:48:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565090+0000", 2024-09-15T21:48:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.5930000000000002e-06 2024-09-15T21:48:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565119+0000", 2024-09-15T21:48:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8785999999999998e-05 2024-09-15T21:48:07.167 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565126+0000", 2024-09-15T21:48:07.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.0110000000000003e-06 2024-09-15T21:48:07.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565237+0000", 2024-09-15T21:48:07.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011161 2024-09-15T21:48:07.168 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.168 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.168 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.168 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565383+0000", 2024-09-15T21:48:07.168 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014626900000000001 2024-09-15T21:48:07.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:48:07.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565520+0000", 2024-09-15T21:48:07.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013676399999999999 2024-09-15T21:48:07.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569909+0000", 2024-09-15T21:48:07.169 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043890149999999996 2024-09-15T21:48:07.169 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.169 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.169 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.169 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572096+0000", 2024-09-15T21:48:07.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021866559999999999 2024-09-15T21:48:07.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572144+0000", 2024-09-15T21:48:07.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7725000000000001e-05 2024-09-15T21:48:07.170 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.170 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.170 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.170 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572182+0000", 2024-09-15T21:48:07.170 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8362000000000003e-05 2024-09-15T21:48:07.170 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.170 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.170 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.171 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.171 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:18 2.14 2:2eebc8f4:::62f.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:07.171 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:48:07.171 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 75.502356702, 2024-09-15T21:48:07.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072110280000000004, 2024-09-15T21:48:07.171 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.171 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.171 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.171 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.171 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.171 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.171 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:48:07.171 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:48:07.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565093+0000", 2024-09-15T21:48:07.172 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4299999999999999e-07 2024-09-15T21:48:07.172 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.172 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.172 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.172 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565096+0000", 2024-09-15T21:48:07.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4319999999999999e-06 2024-09-15T21:48:07.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565152+0000", 2024-09-15T21:48:07.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5686999999999997e-05 2024-09-15T21:48:07.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.173 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565157+0000", 2024-09-15T21:48:07.173 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2700000000000004e-06 2024-09-15T21:48:07.173 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.173 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.173 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565847+0000", 2024-09-15T21:48:07.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00068937299999999996 2024-09-15T21:48:07.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565940+0000", 2024-09-15T21:48:07.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.3573999999999996e-05 2024-09-15T21:48:07.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.174 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 3", 2024-09-15T21:48:07.174 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566085+0000", 2024-09-15T21:48:07.174 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000144902 2024-09-15T21:48:07.174 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.174 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570595+0000", 2024-09-15T21:48:07.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0045103469999999996 2024-09-15T21:48:07.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572224+0000", 2024-09-15T21:48:07.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016282930000000001 2024-09-15T21:48:07.175 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.175 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.175 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.175 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572260+0000", 2024-09-15T21:48:07.175 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6103999999999999e-05 2024-09-15T21:48:07.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.176 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.176 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572304+0000", 2024-09-15T21:48:07.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3803000000000001e-05 2024-09-15T21:48:07.176 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.176 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.176 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.176 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.176 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:19 2.7 2:e196673b:::630.00000000:head [omap-set-header] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:07.176 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:48:07.176 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 75.502352236999997, 2024-09-15T21:48:07.176 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0055995389999999997, 2024-09-15T21:48:07.177 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.177 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.177 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.177 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:48:07.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.177 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.177 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:48:07.177 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565097+0000", 2024-09-15T21:48:07.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9999999999999999e-07 2024-09-15T21:48:07.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:48:07.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2399999999999999e-06 2024-09-15T21:48:07.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.178 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.178 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565280+0000", 2024-09-15T21:48:07.178 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017929400000000001 2024-09-15T21:48:07.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565292+0000", 2024-09-15T21:48:07.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2193e-05 2024-09-15T21:48:07.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565357+0000", 2024-09-15T21:48:07.179 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.4684000000000006e-05 2024-09-15T21:48:07.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.179 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.179 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565499+0000", 2024-09-15T21:48:07.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000142032 2024-09-15T21:48:07.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:07.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565634+0000", 2024-09-15T21:48:07.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000135613 2024-09-15T21:48:07.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.180 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569941+0000", 2024-09-15T21:48:07.180 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0043066789999999999 2024-09-15T21:48:07.180 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.180 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.180 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570585+0000", 2024-09-15T21:48:07.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00064364700000000001 2024-09-15T21:48:07.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570647+0000", 2024-09-15T21:48:07.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2173000000000003e-05 2024-09-15T21:48:07.181 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.181 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.181 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.181 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570697+0000", 2024-09-15T21:48:07.181 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.9784000000000003e-05 2024-09-15T21:48:07.181 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.181 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.182 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.182 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.182 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:20 2.7 2:e196673b:::630.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:07.182 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:48:07.182 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 75.502348006999995, 2024-09-15T21:48:07.182 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066548939999999997, 2024-09-15T21:48:07.182 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.182 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.182 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.182 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.182 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.182 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.182 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:48:07.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565101+0000", 2024-09-15T21:48:07.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.183 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565102+0000", 2024-09-15T21:48:07.183 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0800000000000001e-07 2024-09-15T21:48:07.183 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.183 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.183 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565103+0000", 2024-09-15T21:48:07.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.6700000000000001e-06 2024-09-15T21:48:07.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565608+0000", 2024-09-15T21:48:07.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00050518299999999998 2024-09-15T21:48:07.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.184 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.184 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565616+0000", 2024-09-15T21:48:07.184 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8199999999999997e-06 2024-09-15T21:48:07.184 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.184 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566598+0000", 2024-09-15T21:48:07.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00098176300000000008 2024-09-15T21:48:07.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.566871+0000", 2024-09-15T21:48:07.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027312799999999998 2024-09-15T21:48:07.185 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.185 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.185 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 7", 2024-09-15T21:48:07.185 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.567015+0000", 2024-09-15T21:48:07.185 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014428499999999999 2024-09-15T21:48:07.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.570729+0000", 2024-09-15T21:48:07.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0037132459999999999 2024-09-15T21:48:07.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571667+0000", 2024-09-15T21:48:07.186 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00093862299999999997 2024-09-15T21:48:07.186 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.186 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.186 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.186 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571718+0000", 2024-09-15T21:48:07.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1146999999999997e-05 2024-09-15T21:48:07.187 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.187 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.187 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.187 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571756+0000", 2024-09-15T21:48:07.187 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.7821000000000003e-05 2024-09-15T21:48:07.187 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.187 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.187 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.187 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.187 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.187 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:48:07.189 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:48:07.190 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:48:07.187+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:48:07.289 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:48:07.288+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:48:07.334 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:48:07.334 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:48:07.334 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:48:07.334 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:48:07.343 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:48:07.427 INFO:tasks.ceph.osd.6.smithi122.stderr:2024-09-15T21:48:07.426+0000 7fa8fb9e6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 37853) UID: 0 2024-09-15T21:48:07.487 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:48:07.487 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:48:07.487 INFO:teuthology.orchestra.run.smithi122.stdout: "complaint_time": 180, 2024-09-15T21:48:07.488 INFO:teuthology.orchestra.run.smithi122.stdout: "num_blocked_ops": 0 2024-09-15T21:48:07.488 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:48:07.491 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:48:07.489+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:48:07.496 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:48:07.591 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:48:07.590+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:48:07.644 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:48:07.644 INFO:teuthology.orchestra.run.smithi122.stdout: "size": 20, 2024-09-15T21:48:07.644 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 600, 2024-09-15T21:48:07.644 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-15T21:48:07.645 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.645 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:48:07.645 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:48:07.645 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 114.40709569000001, 2024-09-15T21:48:07.645 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0944283340000001, 2024-09-15T21:48:07.645 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.645 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.645 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.645 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:48:07.645 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:48:07.645 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:48:07.645 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 1 2024-09-15T21:48:07.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.646 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:48:07.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234359+0000", 2024-09-15T21:48:07.646 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.646 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.646 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.646 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.646 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234360+0000", 2024-09-15T21:48:07.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3710000000000001e-06 2024-09-15T21:48:07.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234374+0000", 2024-09-15T21:48:07.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.4003e-05 2024-09-15T21:48:07.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.647 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234392+0000", 2024-09-15T21:48:07.647 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7294000000000001e-05 2024-09-15T21:48:07.647 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.647 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.647 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234405+0000", 2024-09-15T21:48:07.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2961e-05 2024-09-15T21:48:07.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.648 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.648 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234483+0000", 2024-09-15T21:48:07.648 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8850000000000006e-05 2024-09-15T21:48:07.648 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.648 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for peered", 2024-09-15T21:48:07.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:13.234491+0000", 2024-09-15T21:48:07.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.3529999999999997e-06 2024-09-15T21:48:07.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327253+0000", 2024-09-15T21:48:07.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.092761962 2024-09-15T21:48:07.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.649 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.649 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.649 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327456+0000", 2024-09-15T21:48:07.649 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000202893 2024-09-15T21:48:07.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.327652+0000", 2024-09-15T21:48:07.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00019613000000000001 2024-09-15T21:48:07.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328368+0000", 2024-09-15T21:48:07.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000716582 2024-09-15T21:48:07.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.650 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.650 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.650 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328682+0000", 2024-09-15T21:48:07.650 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000313825 2024-09-15T21:48:07.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328743+0000", 2024-09-15T21:48:07.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0571e-05 2024-09-15T21:48:07.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.651 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.651 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.651 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.328787+0000", 2024-09-15T21:48:07.651 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4539000000000002e-05 2024-09-15T21:48:07.651 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.651 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.651 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.651 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.652 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.653 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:48:07.653 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:48:07.653 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 113.285930288, 2024-09-15T21:48:07.653 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0089551270000000002, 2024-09-15T21:48:07.653 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.653 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.653 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.653 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:48:07.654 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:48:07.654 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:48:07.654 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 35 2024-09-15T21:48:07.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.654 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:48:07.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.654 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.654 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.654 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.654 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355524+0000", 2024-09-15T21:48:07.654 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.655 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.655 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.655 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355525+0000", 2024-09-15T21:48:07.655 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:48:07.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355606+0000", 2024-09-15T21:48:07.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1723000000000003e-05 2024-09-15T21:48:07.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.656 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.656 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.656 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355624+0000", 2024-09-15T21:48:07.656 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7645000000000001e-05 2024-09-15T21:48:07.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355629+0000", 2024-09-15T21:48:07.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7319999999999998e-06 2024-09-15T21:48:07.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.657 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.657 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.657 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358315+0000", 2024-09-15T21:48:07.657 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026858659999999999 2024-09-15T21:48:07.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358373+0000", 2024-09-15T21:48:07.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8593e-05 2024-09-15T21:48:07.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.658 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.658 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.658 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.358521+0000", 2024-09-15T21:48:07.658 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00014817500000000001 2024-09-15T21:48:07.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360353+0000", 2024-09-15T21:48:07.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0018317100000000001 2024-09-15T21:48:07.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364424+0000", 2024-09-15T21:48:07.659 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040707970000000001 2024-09-15T21:48:07.659 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.659 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.659 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.659 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364452+0000", 2024-09-15T21:48:07.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7937999999999999e-05 2024-09-15T21:48:07.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.660 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.660 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.660 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364479+0000", 2024-09-15T21:48:07.660 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7744999999999999e-05 2024-09-15T21:48:07.660 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.661 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.661 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.661 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.661 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:36 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:48:07.661 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:48:07.661 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 113.28584632499999, 2024-09-15T21:48:07.661 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0090267869999999997, 2024-09-15T21:48:07.661 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.661 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.661 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.661 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:48:07.662 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:48:07.662 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:48:07.662 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 36 2024-09-15T21:48:07.662 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.662 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.662 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.662 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.662 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:48:07.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355608+0000", 2024-09-15T21:48:07.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.663 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.663 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.663 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355609+0000", 2024-09-15T21:48:07.663 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.3099999999999998e-07 2024-09-15T21:48:07.663 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355612+0000", 2024-09-15T21:48:07.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1489999999999998e-06 2024-09-15T21:48:07.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.664 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355637+0000", 2024-09-15T21:48:07.664 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.508e-05 2024-09-15T21:48:07.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.664 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.664 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355640+0000", 2024-09-15T21:48:07.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7930000000000002e-06 2024-09-15T21:48:07.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.665 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.665 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359195+0000", 2024-09-15T21:48:07.665 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0035555529999999999 2024-09-15T21:48:07.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.666 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.666 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359283+0000", 2024-09-15T21:48:07.666 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.8042999999999997e-05 2024-09-15T21:48:07.666 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359419+0000", 2024-09-15T21:48:07.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013576699999999999 2024-09-15T21:48:07.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360361+0000", 2024-09-15T21:48:07.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00094171000000000001 2024-09-15T21:48:07.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.667 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.667 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.667 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364536+0000", 2024-09-15T21:48:07.667 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00417484 2024-09-15T21:48:07.667 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.668 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.668 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.668 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364590+0000", 2024-09-15T21:48:07.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4355999999999999e-05 2024-09-15T21:48:07.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.669 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.669 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.669 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364635+0000", 2024-09-15T21:48:07.669 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5065000000000003e-05 2024-09-15T21:48:07.669 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.669 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.669 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.669 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.670 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.670 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:48:07.670 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:48:07.670 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 113.28584171999999, 2024-09-15T21:48:07.670 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0091162750000000001, 2024-09-15T21:48:07.670 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.670 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.670 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.670 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:48:07.670 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:48:07.670 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:48:07.670 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 37 2024-09-15T21:48:07.670 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.670 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:48:07.671 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.671 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.671 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.671 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.671 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:48:07.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355613+0000", 2024-09-15T21:48:07.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.03e-07 2024-09-15T21:48:07.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.672 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.672 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.672 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355615+0000", 2024-09-15T21:48:07.672 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0729999999999999e-06 2024-09-15T21:48:07.672 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355646+0000", 2024-09-15T21:48:07.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0391000000000001e-05 2024-09-15T21:48:07.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.355648+0000", 2024-09-15T21:48:07.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6309999999999999e-06 2024-09-15T21:48:07.673 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.673 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.673 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.673 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359770+0000", 2024-09-15T21:48:07.673 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041222899999999998 2024-09-15T21:48:07.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.674 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.674 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359854+0000", 2024-09-15T21:48:07.674 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3399000000000006e-05 2024-09-15T21:48:07.674 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.674 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.359988+0000", 2024-09-15T21:48:07.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013442200000000001 2024-09-15T21:48:07.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.360524+0000", 2024-09-15T21:48:07.675 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00053582900000000004 2024-09-15T21:48:07.675 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.675 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.675 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.675 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364673+0000", 2024-09-15T21:48:07.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0041493199999999997 2024-09-15T21:48:07.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.676 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.676 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364704+0000", 2024-09-15T21:48:07.676 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0380999999999999e-05 2024-09-15T21:48:07.676 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.676 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.677 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.677 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.364729+0000", 2024-09-15T21:48:07.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5335999999999999e-05 2024-09-15T21:48:07.677 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.677 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.677 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.677 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.677 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.677 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:88 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:48:07.677 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:48:07.677 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 113.217124482, 2024-09-15T21:48:07.677 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0026270020000001, 2024-09-15T21:48:07.677 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.678 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.678 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.678 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:48:07.678 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:48:07.678 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:48:07.678 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 88 2024-09-15T21:48:07.678 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.678 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.678 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:48:07.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424330+0000", 2024-09-15T21:48:07.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.679 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.679 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.679 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424331+0000", 2024-09-15T21:48:07.679 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4299999999999998e-07 2024-09-15T21:48:07.679 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:14.424333+0000", 2024-09-15T21:48:07.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2359999999999999e-06 2024-09-15T21:48:07.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422895+0000", 2024-09-15T21:48:07.680 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.99856184199999998 2024-09-15T21:48:07.680 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.680 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.680 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.680 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.422902+0000", 2024-09-15T21:48:07.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4440000000000002e-06 2024-09-15T21:48:07.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423053+0000", 2024-09-15T21:48:07.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000150958 2024-09-15T21:48:07.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.681 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423160+0000", 2024-09-15T21:48:07.681 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000106774 2024-09-15T21:48:07.681 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.681 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.681 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.423344+0000", 2024-09-15T21:48:07.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00018424199999999999 2024-09-15T21:48:07.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.424730+0000", 2024-09-15T21:48:07.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001386155 2024-09-15T21:48:07.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.682 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.682 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426756+0000", 2024-09-15T21:48:07.682 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020261150000000002 2024-09-15T21:48:07.682 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.682 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426906+0000", 2024-09-15T21:48:07.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000149176 2024-09-15T21:48:07.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.683 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.683 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.426957+0000", 2024-09-15T21:48:07.683 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1616999999999997e-05 2024-09-15T21:48:07.683 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.683 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.683 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.683 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.683 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.683 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:48:07.684 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:48:07.684 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.087051284, 2024-09-15T21:48:07.684 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0095037790000000004, 2024-09-15T21:48:07.684 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.684 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.684 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.684 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:48:07.684 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:48:07.684 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:48:07.684 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 114 2024-09-15T21:48:07.684 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.684 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.684 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:48:07.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554403+0000", 2024-09-15T21:48:07.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.685 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.685 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.685 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.685 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554404+0000", 2024-09-15T21:48:07.685 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2399999999999999e-07 2024-09-15T21:48:07.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554407+0000", 2024-09-15T21:48:07.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0299999999999998e-06 2024-09-15T21:48:07.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.686 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554632+0000", 2024-09-15T21:48:07.686 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022568999999999999 2024-09-15T21:48:07.686 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.686 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.686 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554640+0000", 2024-09-15T21:48:07.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7970000000000001e-06 2024-09-15T21:48:07.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.687 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.687 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561325+0000", 2024-09-15T21:48:07.687 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0066848020000000001 2024-09-15T21:48:07.687 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.687 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561402+0000", 2024-09-15T21:48:07.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.6851e-05 2024-09-15T21:48:07.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.688 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561530+0000", 2024-09-15T21:48:07.688 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012863999999999999 2024-09-15T21:48:07.688 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.688 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.688 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563144+0000", 2024-09-15T21:48:07.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016132849999999999 2024-09-15T21:48:07.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563800+0000", 2024-09-15T21:48:07.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00065586099999999999 2024-09-15T21:48:07.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.689 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.689 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.689 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563857+0000", 2024-09-15T21:48:07.689 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.702e-05 2024-09-15T21:48:07.689 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.690 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.690 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563907+0000", 2024-09-15T21:48:07.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0479000000000002e-05 2024-09-15T21:48:07.690 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.690 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.690 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.690 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.690 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.690 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:48:07.690 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:48:07.690 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.08704628700001, 2024-09-15T21:48:07.690 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.011813423999999999, 2024-09-15T21:48:07.690 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.691 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.691 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.691 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:48:07.691 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:48:07.691 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:48:07.691 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 115 2024-09-15T21:48:07.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.691 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.691 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.691 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:48:07.691 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.691 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.691 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554408+0000", 2024-09-15T21:48:07.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554409+0000", 2024-09-15T21:48:07.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1100000000000002e-07 2024-09-15T21:48:07.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.692 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.692 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.692 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554498+0000", 2024-09-15T21:48:07.692 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.9642999999999995e-05 2024-09-15T21:48:07.692 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554656+0000", 2024-09-15T21:48:07.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000158114 2024-09-15T21:48:07.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554661+0000", 2024-09-15T21:48:07.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0730000000000004e-06 2024-09-15T21:48:07.693 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.693 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.693 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.693 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561862+0000", 2024-09-15T21:48:07.693 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072002999999999998 2024-09-15T21:48:07.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.561939+0000", 2024-09-15T21:48:07.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7265e-05 2024-09-15T21:48:07.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562073+0000", 2024-09-15T21:48:07.694 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00013371600000000001 2024-09-15T21:48:07.694 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.694 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.694 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.694 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563196+0000", 2024-09-15T21:48:07.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011231679999999999 2024-09-15T21:48:07.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566124+0000", 2024-09-15T21:48:07.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002928403 2024-09-15T21:48:07.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.695 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566183+0000", 2024-09-15T21:48:07.695 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8328000000000001e-05 2024-09-15T21:48:07.695 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.695 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.695 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.696 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566222+0000", 2024-09-15T21:48:07.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.9103000000000002e-05 2024-09-15T21:48:07.696 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.696 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.696 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.696 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.696 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.696 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:48:07.696 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:48:07.696 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.08695284700001, 2024-09-15T21:48:07.696 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.01181426, 2024-09-15T21:48:07.696 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.696 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.696 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.697 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:48:07.697 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:48:07.697 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:48:07.697 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 116 2024-09-15T21:48:07.697 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.697 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.697 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.697 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:48:07.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554502+0000", 2024-09-15T21:48:07.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.698 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.698 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554503+0000", 2024-09-15T21:48:07.698 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8000000000000005e-07 2024-09-15T21:48:07.698 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.698 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554506+0000", 2024-09-15T21:48:07.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.5899999999999999e-06 2024-09-15T21:48:07.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554674+0000", 2024-09-15T21:48:07.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016763600000000001 2024-09-15T21:48:07.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.699 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.699 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.699 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554678+0000", 2024-09-15T21:48:07.699 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.5310000000000003e-06 2024-09-15T21:48:07.699 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562714+0000", 2024-09-15T21:48:07.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.008035258 2024-09-15T21:48:07.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562765+0000", 2024-09-15T21:48:07.700 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.1093e-05 2024-09-15T21:48:07.700 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.700 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.700 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.700 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.562845+0000", 2024-09-15T21:48:07.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.0427999999999996e-05 2024-09-15T21:48:07.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563943+0000", 2024-09-15T21:48:07.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097647 2024-09-15T21:48:07.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.701 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566232+0000", 2024-09-15T21:48:07.701 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022890850000000002 2024-09-15T21:48:07.701 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.701 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.701 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566279+0000", 2024-09-15T21:48:07.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6692e-05 2024-09-15T21:48:07.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.702 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.702 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566316+0000", 2024-09-15T21:48:07.702 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.752e-05 2024-09-15T21:48:07.702 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.702 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.702 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.702 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.702 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.702 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:48:07.702 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:48:07.702 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 112.086946748, 2024-09-15T21:48:07.703 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.012232125, 2024-09-15T21:48:07.703 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.703 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.703 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.703 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:48:07.703 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:48:07.703 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:48:07.703 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 117 2024-09-15T21:48:07.703 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.703 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.703 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.703 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.703 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:48:07.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:48:07.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.704 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554508+0000", 2024-09-15T21:48:07.704 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7700000000000001e-07 2024-09-15T21:48:07.704 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.704 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.704 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554511+0000", 2024-09-15T21:48:07.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8430000000000001e-06 2024-09-15T21:48:07.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554690+0000", 2024-09-15T21:48:07.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00017856299999999999 2024-09-15T21:48:07.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.705 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.705 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.554698+0000", 2024-09-15T21:48:07.705 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6200000000000005e-06 2024-09-15T21:48:07.705 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.705 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563224+0000", 2024-09-15T21:48:07.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0085262640000000004 2024-09-15T21:48:07.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563342+0000", 2024-09-15T21:48:07.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000117687 2024-09-15T21:48:07.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.706 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.706 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.706 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.563472+0000", 2024-09-15T21:48:07.706 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012957999999999999 2024-09-15T21:48:07.706 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566367+0000", 2024-09-15T21:48:07.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028952280000000001 2024-09-15T21:48:07.707 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.707 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.707 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.707 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566680+0000", 2024-09-15T21:48:07.707 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00031345100000000002 2024-09-15T21:48:07.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566714+0000", 2024-09-15T21:48:07.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3701000000000001e-05 2024-09-15T21:48:07.708 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.708 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.708 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.708 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.566740+0000", 2024-09-15T21:48:07.708 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5911e-05 2024-09-15T21:48:07.708 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.708 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.708 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.709 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.709 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.712 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:134 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:48:07.712 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:48:07.712 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 111.744054666, 2024-09-15T21:48:07.712 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0069806590000000002, 2024-09-15T21:48:07.712 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.712 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.713 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.713 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:48:07.713 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:48:07.713 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:48:07.713 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 134 2024-09-15T21:48:07.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.713 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.713 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:48:07.713 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.713 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.713 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.713 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897400+0000", 2024-09-15T21:48:07.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897401+0000", 2024-09-15T21:48:07.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0500000000000003e-07 2024-09-15T21:48:07.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.714 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.714 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897492+0000", 2024-09-15T21:48:07.714 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.1770999999999999e-05 2024-09-15T21:48:07.714 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.714 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897750+0000", 2024-09-15T21:48:07.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00025778299999999998 2024-09-15T21:48:07.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897756+0000", 2024-09-15T21:48:07.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2140000000000001e-06 2024-09-15T21:48:07.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.715 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.715 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.715 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899312+0000", 2024-09-15T21:48:07.715 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001556144 2024-09-15T21:48:07.715 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899372+0000", 2024-09-15T21:48:07.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.0015000000000001e-05 2024-09-15T21:48:07.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899444+0000", 2024-09-15T21:48:07.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1923999999999998e-05 2024-09-15T21:48:07.716 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.716 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.716 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.716 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901354+0000", 2024-09-15T21:48:07.716 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0019092530000000001 2024-09-15T21:48:07.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904327+0000", 2024-09-15T21:48:07.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0029734520000000001 2024-09-15T21:48:07.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904360+0000", 2024-09-15T21:48:07.717 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3216000000000002e-05 2024-09-15T21:48:07.717 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.717 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.717 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.717 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904381+0000", 2024-09-15T21:48:07.718 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0282e-05 2024-09-15T21:48:07.718 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.718 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.718 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.718 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.718 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.718 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:48:07.718 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:48:07.719 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 111.74395880900001, 2024-09-15T21:48:07.719 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070121059999999997, 2024-09-15T21:48:07.719 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.719 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.719 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.719 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:48:07.719 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:48:07.719 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:48:07.719 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 135 2024-09-15T21:48:07.719 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.719 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.719 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.719 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:48:07.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897496+0000", 2024-09-15T21:48:07.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.720 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.720 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897497+0000", 2024-09-15T21:48:07.720 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.4799999999999997e-07 2024-09-15T21:48:07.720 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.720 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897500+0000", 2024-09-15T21:48:07.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.2949999999999998e-06 2024-09-15T21:48:07.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897781+0000", 2024-09-15T21:48:07.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028064300000000002 2024-09-15T21:48:07.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.721 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.721 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.721 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897786+0000", 2024-09-15T21:48:07.721 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5169999999999999e-06 2024-09-15T21:48:07.721 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899857+0000", 2024-09-15T21:48:07.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020705340000000002 2024-09-15T21:48:07.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.899932+0000", 2024-09-15T21:48:07.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5840999999999994e-05 2024-09-15T21:48:07.722 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.722 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.722 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.722 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900005+0000", 2024-09-15T21:48:07.722 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2716e-05 2024-09-15T21:48:07.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902296+0000", 2024-09-15T21:48:07.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022912010000000001 2024-09-15T21:48:07.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904424+0000", 2024-09-15T21:48:07.723 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0021278600000000001 2024-09-15T21:48:07.723 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.723 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.723 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.723 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904472+0000", 2024-09-15T21:48:07.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7586000000000001e-05 2024-09-15T21:48:07.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.724 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.724 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904508+0000", 2024-09-15T21:48:07.724 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6164999999999997e-05 2024-09-15T21:48:07.724 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.724 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.724 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.724 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.724 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.724 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:48:07.724 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:48:07.725 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 111.743934318, 2024-09-15T21:48:07.725 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0070611429999999998, 2024-09-15T21:48:07.725 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.725 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.725 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.725 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:48:07.725 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:48:07.725 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:48:07.725 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 136 2024-09-15T21:48:07.725 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.725 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.725 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.725 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.725 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:48:07.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897520+0000", 2024-09-15T21:48:07.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.726 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.726 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.726 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897521+0000", 2024-09-15T21:48:07.726 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2900000000000004e-07 2024-09-15T21:48:07.726 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897527+0000", 2024-09-15T21:48:07.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8490000000000002e-06 2024-09-15T21:48:07.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.727 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897799+0000", 2024-09-15T21:48:07.727 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00027219499999999998 2024-09-15T21:48:07.727 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.727 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.727 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897804+0000", 2024-09-15T21:48:07.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4219999999999999e-06 2024-09-15T21:48:07.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900248+0000", 2024-09-15T21:48:07.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002443457 2024-09-15T21:48:07.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.728 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.728 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.728 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900306+0000", 2024-09-15T21:48:07.728 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8263000000000001e-05 2024-09-15T21:48:07.728 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900385+0000", 2024-09-15T21:48:07.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.9161999999999999e-05 2024-09-15T21:48:07.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902300+0000", 2024-09-15T21:48:07.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001914947 2024-09-15T21:48:07.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.729 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.729 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.729 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904536+0000", 2024-09-15T21:48:07.729 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022354419999999998 2024-09-15T21:48:07.729 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904562+0000", 2024-09-15T21:48:07.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6378999999999999e-05 2024-09-15T21:48:07.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.730 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.730 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.730 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904581+0000", 2024-09-15T21:48:07.730 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9497999999999999e-05 2024-09-15T21:48:07.730 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.730 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.730 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.730 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.731 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.731 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:48:07.731 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:48:07.731 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 111.74392575100001, 2024-09-15T21:48:07.731 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007180306, 2024-09-15T21:48:07.731 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.731 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.731 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.731 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:48:07.731 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:48:07.731 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:48:07.731 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 137 2024-09-15T21:48:07.731 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.731 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:48:07.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.732 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.732 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:48:07.732 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.732 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.732 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897529+0000", 2024-09-15T21:48:07.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.1399999999999998e-07 2024-09-15T21:48:07.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897612+0000", 2024-09-15T21:48:07.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2607000000000004e-05 2024-09-15T21:48:07.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.733 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.733 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897815+0000", 2024-09-15T21:48:07.733 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00020352400000000001 2024-09-15T21:48:07.733 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.733 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897821+0000", 2024-09-15T21:48:07.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4670000000000003e-06 2024-09-15T21:48:07.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900665+0000", 2024-09-15T21:48:07.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028443689999999998 2024-09-15T21:48:07.734 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.734 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.734 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.734 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900714+0000", 2024-09-15T21:48:07.734 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8812e-05 2024-09-15T21:48:07.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.900799+0000", 2024-09-15T21:48:07.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.5129000000000004e-05 2024-09-15T21:48:07.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903413+0000", 2024-09-15T21:48:07.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026142890000000001 2024-09-15T21:48:07.735 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.735 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.735 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.735 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904625+0000", 2024-09-15T21:48:07.735 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012121149999999999 2024-09-15T21:48:07.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904673+0000", 2024-09-15T21:48:07.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7540999999999998e-05 2024-09-15T21:48:07.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.736 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.736 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.736 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904709+0000", 2024-09-15T21:48:07.736 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6139000000000001e-05 2024-09-15T21:48:07.736 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.736 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.736 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.736 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.737 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.737 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:48:07.737 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:48:07.737 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 111.743839796, 2024-09-15T21:48:07.737 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071727450000000003, 2024-09-15T21:48:07.737 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.737 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.737 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.737 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:48:07.737 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:48:07.737 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:48:07.737 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 138 2024-09-15T21:48:07.737 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.737 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:48:07.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:48:07.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.738 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.738 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.738 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.738 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897615+0000", 2024-09-15T21:48:07.738 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:48:07.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897619+0000", 2024-09-15T21:48:07.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3869999999999999e-06 2024-09-15T21:48:07.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897832+0000", 2024-09-15T21:48:07.739 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00021329500000000001 2024-09-15T21:48:07.739 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.739 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.739 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.739 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897837+0000", 2024-09-15T21:48:07.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3730000000000001e-06 2024-09-15T21:48:07.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901391+0000", 2024-09-15T21:48:07.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.003553601 2024-09-15T21:48:07.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.740 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901466+0000", 2024-09-15T21:48:07.740 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.5130999999999996e-05 2024-09-15T21:48:07.740 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.740 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.740 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901601+0000", 2024-09-15T21:48:07.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134797 2024-09-15T21:48:07.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903417+0000", 2024-09-15T21:48:07.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001816156 2024-09-15T21:48:07.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.741 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.741 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904741+0000", 2024-09-15T21:48:07.741 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0013241049999999999 2024-09-15T21:48:07.741 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.741 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904767+0000", 2024-09-15T21:48:07.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5876000000000001e-05 2024-09-15T21:48:07.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.742 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.742 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904788+0000", 2024-09-15T21:48:07.742 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0449999999999999e-05 2024-09-15T21:48:07.742 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.742 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.742 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.742 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.742 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.742 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:48:07.743 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:48:07.743 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 111.743834143, 2024-09-15T21:48:07.743 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0072970250000000004, 2024-09-15T21:48:07.743 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.743 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.743 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.743 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:48:07.743 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:48:07.743 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:48:07.743 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 139 2024-09-15T21:48:07.743 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.743 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.743 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.743 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.743 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:48:07.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897620+0000", 2024-09-15T21:48:07.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.744 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897621+0000", 2024-09-15T21:48:07.744 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.3099999999999999e-07 2024-09-15T21:48:07.744 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.744 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.744 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897625+0000", 2024-09-15T21:48:07.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.698e-06 2024-09-15T21:48:07.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897848+0000", 2024-09-15T21:48:07.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00022375699999999999 2024-09-15T21:48:07.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.745 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.745 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897853+0000", 2024-09-15T21:48:07.745 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.7709999999999997e-06 2024-09-15T21:48:07.745 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.745 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901929+0000", 2024-09-15T21:48:07.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0040757780000000004 2024-09-15T21:48:07.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.901976+0000", 2024-09-15T21:48:07.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6751e-05 2024-09-15T21:48:07.746 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.746 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.746 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.746 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902054+0000", 2024-09-15T21:48:07.746 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8856999999999994e-05 2024-09-15T21:48:07.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903421+0000", 2024-09-15T21:48:07.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00136619 2024-09-15T21:48:07.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.747 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.747 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.747 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904827+0000", 2024-09-15T21:48:07.747 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0014067369999999999 2024-09-15T21:48:07.747 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904881+0000", 2024-09-15T21:48:07.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3520999999999998e-05 2024-09-15T21:48:07.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.748 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.748 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904918+0000", 2024-09-15T21:48:07.748 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6634000000000002e-05 2024-09-15T21:48:07.748 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.748 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.748 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.748 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.748 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.749 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:48:07.749 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:48:07.749 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 111.743828314, 2024-09-15T21:48:07.749 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097221730000000006, 2024-09-15T21:48:07.749 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.749 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.749 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.749 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:48:07.749 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:48:07.749 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:48:07.749 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 140 2024-09-15T21:48:07.749 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.749 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.749 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:48:07.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897626+0000", 2024-09-15T21:48:07.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.750 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.750 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.750 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897627+0000", 2024-09-15T21:48:07.750 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.41e-07 2024-09-15T21:48:07.750 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897698+0000", 2024-09-15T21:48:07.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.1722000000000002e-05 2024-09-15T21:48:07.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897863+0000", 2024-09-15T21:48:07.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00016479099999999999 2024-09-15T21:48:07.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.751 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.751 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.751 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897868+0000", 2024-09-15T21:48:07.751 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.4950000000000002e-06 2024-09-15T21:48:07.751 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902335+0000", 2024-09-15T21:48:07.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.004467632 2024-09-15T21:48:07.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902414+0000", 2024-09-15T21:48:07.752 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.8688000000000003e-05 2024-09-15T21:48:07.752 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.752 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.752 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.752 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.902535+0000", 2024-09-15T21:48:07.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00012066900000000001 2024-09-15T21:48:07.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903714+0000", 2024-09-15T21:48:07.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0011794520000000001 2024-09-15T21:48:07.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.753 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.753 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907115+0000", 2024-09-15T21:48:07.753 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0034008879999999999 2024-09-15T21:48:07.753 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.753 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907250+0000", 2024-09-15T21:48:07.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000134657 2024-09-15T21:48:07.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.754 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.754 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907348+0000", 2024-09-15T21:48:07.754 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8838000000000005e-05 2024-09-15T21:48:07.754 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.754 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.754 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.754 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.754 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.755 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(client.4181.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e18)", 2024-09-15T21:48:07.755 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:48:07.755 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 111.74375323300001, 2024-09-15T21:48:07.755 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0097618050000000001, 2024-09-15T21:48:07.755 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.755 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.755 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.755 INFO:teuthology.orchestra.run.smithi122.stdout: "client_info": { 2024-09-15T21:48:07.755 INFO:teuthology.orchestra.run.smithi122.stdout: "client": "client.4181", 2024-09-15T21:48:07.755 INFO:teuthology.orchestra.run.smithi122.stdout: "client_addr": "172.21.15.100:0/1752340873", 2024-09-15T21:48:07.756 INFO:teuthology.orchestra.run.smithi122.stdout: "tid": 141 2024-09-15T21:48:07.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.756 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:48:07.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.756 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.756 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.756 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897701+0000", 2024-09-15T21:48:07.756 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.756 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897702+0000", 2024-09-15T21:48:07.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.6199999999999998e-07 2024-09-15T21:48:07.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897705+0000", 2024-09-15T21:48:07.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.0180000000000002e-06 2024-09-15T21:48:07.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.757 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.757 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.757 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897889+0000", 2024-09-15T21:48:07.757 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000183678 2024-09-15T21:48:07.757 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.897894+0000", 2024-09-15T21:48:07.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.8930000000000004e-06 2024-09-15T21:48:07.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903099+0000", 2024-09-15T21:48:07.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0052056280000000003 2024-09-15T21:48:07.758 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.758 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.758 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.758 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903151+0000", 2024-09-15T21:48:07.758 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.2163000000000002e-05 2024-09-15T21:48:07.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.903220+0000", 2024-09-15T21:48:07.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.8483000000000001e-05 2024-09-15T21:48:07.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.904794+0000", 2024-09-15T21:48:07.759 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001574521 2024-09-15T21:48:07.759 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.759 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.759 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.759 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907405+0000", 2024-09-15T21:48:07.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0026110339999999999 2024-09-15T21:48:07.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.760 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907440+0000", 2024-09-15T21:48:07.760 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4564999999999999e-05 2024-09-15T21:48:07.760 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.760 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.760 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.761 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:15.907463+0000", 2024-09-15T21:48:07.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3260000000000001e-05 2024-09-15T21:48:07.761 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.761 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.761 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.761 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.761 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.761 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:22 2.8 2:1470b2f8:::613.00000000:head [create,setxattr parent (62) in=6b,setxattr layout (30) in=6b] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:07.761 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:48:07.761 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.186312905999998, 2024-09-15T21:48:07.761 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.007454937, 2024-09-15T21:48:07.761 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.761 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.761 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.762 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:48:07.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.762 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:48:07.762 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.762 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.762 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.762 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455142+0000", 2024-09-15T21:48:07.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.6800000000000002e-07 2024-09-15T21:48:07.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455145+0000", 2024-09-15T21:48:07.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.4470000000000002e-06 2024-09-15T21:48:07.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.763 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456425+0000", 2024-09-15T21:48:07.763 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012798200000000001 2024-09-15T21:48:07.763 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.763 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.763 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456433+0000", 2024-09-15T21:48:07.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.7389999999999999e-06 2024-09-15T21:48:07.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457531+0000", 2024-09-15T21:48:07.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001097865 2024-09-15T21:48:07.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.764 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.764 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457614+0000", 2024-09-15T21:48:07.764 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3560000000000006e-05 2024-09-15T21:48:07.764 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.764 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.457998+0000", 2024-09-15T21:48:07.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00038372699999999999 2024-09-15T21:48:07.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.460488+0000", 2024-09-15T21:48:07.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002489855 2024-09-15T21:48:07.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.765 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.765 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.765 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462512+0000", 2024-09-15T21:48:07.765 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0020235269999999998 2024-09-15T21:48:07.765 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462554+0000", 2024-09-15T21:48:07.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2472000000000003e-05 2024-09-15T21:48:07.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.766 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.766 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.766 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462597+0000", 2024-09-15T21:48:07.766 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.2657000000000002e-05 2024-09-15T21:48:07.766 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.766 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.766 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.766 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.767 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.1.10:2 2.5 2:a6e36b3f:::401.00000000:head [writefull 0~22] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:07.767 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:48:07.767 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 93.186307936999995, 2024-09-15T21:48:07.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0074484590000000002, 2024-09-15T21:48:07.767 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.767 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.767 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.767 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.767 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.767 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.767 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:48:07.767 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.767 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:48:07.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455147+0000", 2024-09-15T21:48:07.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.9299999999999999e-07 2024-09-15T21:48:07.768 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.768 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.768 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.768 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.455150+0000", 2024-09-15T21:48:07.768 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.7319999999999998e-06 2024-09-15T21:48:07.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456449+0000", 2024-09-15T21:48:07.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0012991490000000001 2024-09-15T21:48:07.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.769 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456454+0000", 2024-09-15T21:48:07.769 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.0390000000000004e-06 2024-09-15T21:48:07.769 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.769 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.769 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456513+0000", 2024-09-15T21:48:07.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.8768999999999999e-05 2024-09-15T21:48:07.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456628+0000", 2024-09-15T21:48:07.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115641 2024-09-15T21:48:07.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.770 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.770 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.456741+0000", 2024-09-15T21:48:07.770 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000113094 2024-09-15T21:48:07.770 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.770 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.458520+0000", 2024-09-15T21:48:07.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001778993 2024-09-15T21:48:07.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462494+0000", 2024-09-15T21:48:07.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0039737009999999996 2024-09-15T21:48:07.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.771 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.771 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462549+0000", 2024-09-15T21:48:07.771 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.4537000000000003e-05 2024-09-15T21:48:07.771 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.771 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.772 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.772 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:34.462595+0000", 2024-09-15T21:48:07.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.6511000000000002e-05 2024-09-15T21:48:07.773 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.773 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.773 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.773 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.773 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.773 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_op(mds.4.16:27 2.5 2:a5fa685f:::504.00000000:head [writefull 0~90] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e22)", 2024-09-15T21:48:07.773 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:48:07.773 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 76.076615552999996, 2024-09-15T21:48:07.773 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0071804599999999996, 2024-09-15T21:48:07.773 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-15T21:48:07.774 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-15T21:48:07.774 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:48:07.774 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-15T21:48:07.774 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.774 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-15T21:48:07.774 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:48:07.774 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-15T21:48:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:48:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-15T21:48:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-15T21:48:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564839+0000", 2024-09-15T21:48:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.0100000000000001e-07 2024-09-15T21:48:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-15T21:48:07.775 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.564841+0000", 2024-09-15T21:48:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.1550000000000001e-06 2024-09-15T21:48:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-15T21:48:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565126+0000", 2024-09-15T21:48:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00028469900000000002 2024-09-15T21:48:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-15T21:48:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.565133+0000", 2024-09-15T21:48:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.9480000000000002e-06 2024-09-15T21:48:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.776 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-15T21:48:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568006+0000", 2024-09-15T21:48:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0028730330000000001 2024-09-15T21:48:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-15T21:48:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568119+0000", 2024-09-15T21:48:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.00011325899999999999 2024-09-15T21:48:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "waiting for subops from 0", 2024-09-15T21:48:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.568235+0000", 2024-09-15T21:48:07.777 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000115868 2024-09-15T21:48:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "op_commit", 2024-09-15T21:48:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.569668+0000", 2024-09-15T21:48:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001433099 2024-09-15T21:48:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:48:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571926+0000", 2024-09-15T21:48:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0022572410000000001 2024-09-15T21:48:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.778 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "commit_sent", 2024-09-15T21:48:07.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.571981+0000", 2024-09-15T21:48:07.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.5383000000000002e-05 2024-09-15T21:48:07.779 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-15T21:48:07.779 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-15T21:48:07.779 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "done", 2024-09-15T21:48:07.779 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-15T21:46:51.572020+0000", 2024-09-15T21:48:07.779 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.8574e-05 2024-09-15T21:48:07.779 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.779 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.779 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.780 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-15T21:48:07.780 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-15T21:48:07.780 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:48:07.782 INFO:tasks.ceph.osd.7.smithi122.stderr:2024-09-15T21:48:07.780+0000 7f7c621ee640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 37851) UID: 0 2024-09-15T21:48:07.782 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:48:07.883 INFO:tasks.ceph.osd.3.smithi100.stderr:2024-09-15T21:48:07.881+0000 7f98504e9640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 36481) UID: 0 2024-09-15T21:48:07.898 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:07.898 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:48:07.898 INFO:teuthology.orchestra.run.smithi100.stdout: "num_ops": 0 2024-09-15T21:48:07.898 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:07.907 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:48:07.993 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:48:07.991+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:48:07.994 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:48:08.055 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:08.055 INFO:teuthology.orchestra.run.smithi100.stdout: "ops": [], 2024-09-15T21:48:08.055 INFO:teuthology.orchestra.run.smithi100.stdout: "complaint_time": 180, 2024-09-15T21:48:08.055 INFO:teuthology.orchestra.run.smithi100.stdout: "num_blocked_ops": 0 2024-09-15T21:48:08.055 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:08.065 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:48:08.084 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:48:08.083+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:48:08.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.091+0000 7f7e88b18640 1 Processor -- start 2024-09-15T21:48:08.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.091+0000 7f7e88b18640 1 -- start start 2024-09-15T21:48:08.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.092+0000 7f7e88b18640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7e84152040 0x7f7e8415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.092+0000 7f7e88b18640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e84157d20 0x7f7e84072080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.092+0000 7f7e88b18640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e840c9f20 con 0x7f7e840c9320 2024-09-15T21:48:08.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.092+0000 7f7e88b18640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f7e840c7e90 con 0x7f7e84152040 2024-09-15T21:48:08.094 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.092+0000 7f7e88b18640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7e840c9720 con 0x7f7e84157d20 2024-09-15T21:48:08.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.092+0000 7f7e8259c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7e84152040 0x7f7e8415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.092+0000 7f7e82d9d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e84157d20 0x7f7e84072080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.092+0000 7f7e81d9b640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f7e840c9320 0x7f7e840c8af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.100:43682/0 (socket says 172.21.15.100:43682) 2024-09-15T21:48:08.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.092+0000 7f7e82d9d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e84157d20 0x7f7e84072080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:46710/0 (socket says 172.21.15.100:46710) 2024-09-15T21:48:08.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.092+0000 7f7e8259c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7e84152040 0x7f7e8415bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:49452/0 (socket says 172.21.15.100:49452) 2024-09-15T21:48:08.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.092+0000 7f7e81d9b640 1 -- 172.21.15.100:0/841382514 learned_addr learned my addr 172.21.15.100:0/841382514 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:08.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.092+0000 7f7e8259c640 1 -- 172.21.15.100:0/841382514 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e84157d20 msgr2=0x7f7e84072080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:08.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.092+0000 7f7e8259c640 1 --2- 172.21.15.100:0/841382514 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e84157d20 0x7f7e84072080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:08.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.092+0000 7f7e8259c640 1 -- 172.21.15.100:0/841382514 >> v1:172.21.15.122:6789/0 conn(0x7f7e840c9320 legacy=0x7f7e840c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:08.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.092+0000 7f7e8259c640 1 -- 172.21.15.100:0/841382514 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8415b440 con 0x7f7e84152040 2024-09-15T21:48:08.095 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.092+0000 7f7e82d9d640 1 --2- 172.21.15.100:0/841382514 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e84157d20 0x7f7e84072080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:48:08.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.093+0000 7f7e8259c640 1 --2- 172.21.15.100:0/841382514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e84152040 0x7f7e8415bee0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7e740097a0 tx=0x7f7e7402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57d89c623ca4fcae server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:08.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.093+0000 7f7e8159a640 1 -- 172.21.15.100:0/841382514 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74037070 con 0x7f7e84152040 2024-09-15T21:48:08.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.093+0000 7f7e8159a640 1 -- 172.21.15.100:0/841382514 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e74032440 con 0x7f7e84152040 2024-09-15T21:48:08.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.093+0000 7f7e8159a640 1 -- 172.21.15.100:0/841382514 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e74032810 con 0x7f7e84152040 2024-09-15T21:48:08.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.094+0000 7f7e88b18640 1 -- 172.21.15.100:0/841382514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e84152040 msgr2=0x7f7e8415bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:08.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.094+0000 7f7e88b18640 1 --2- 172.21.15.100:0/841382514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e84152040 0x7f7e8415bee0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7e740097a0 tx=0x7f7e7402a210 comp rx=0 tx=0).stop 2024-09-15T21:48:08.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.094+0000 7f7e88b18640 1 -- 172.21.15.100:0/841382514 shutdown_connections 2024-09-15T21:48:08.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.094+0000 7f7e88b18640 1 --2- 172.21.15.100:0/841382514 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e84157d20 0x7f7e84072080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:08.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.094+0000 7f7e88b18640 1 --2- 172.21.15.100:0/841382514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e84152040 0x7f7e8415bee0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:08.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.094+0000 7f7e88b18640 1 -- 172.21.15.100:0/841382514 >> 172.21.15.100:0/841382514 conn(0x7f7e840cd9a0 msgr2=0x7f7e840cdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:08.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.094+0000 7f7e88b18640 1 -- 172.21.15.100:0/841382514 shutdown_connections 2024-09-15T21:48:08.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.094+0000 7f7e88b18640 1 -- 172.21.15.100:0/841382514 wait complete. 2024-09-15T21:48:08.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.095+0000 7f7e88b18640 1 Processor -- start 2024-09-15T21:48:08.098 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.095+0000 7f7e88b18640 1 -- start start 2024-09-15T21:48:08.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.096+0000 7f7e88b18640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e840c9320 0x7f7e8406c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.096+0000 7f7e88b18640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e84152040 0x7f7e84148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.096+0000 7f7e88b18640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e84157d20 0x7f7e8406cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.096+0000 7f7e88b18640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7e841581b0 con 0x7f7e84157d20 2024-09-15T21:48:08.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.096+0000 7f7e88b18640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8415ba50 con 0x7f7e84152040 2024-09-15T21:48:08.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.096+0000 7f7e88b18640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7e840c9c40 con 0x7f7e840c9320 2024-09-15T21:48:08.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.096+0000 7f7e82d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e84157d20 0x7f7e8406cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.096+0000 7f7e82d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e84157d20 0x7f7e8406cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49758/0 (socket says 172.21.15.100:49758) 2024-09-15T21:48:08.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.096+0000 7f7e82d9d640 1 -- 172.21.15.100:0/2820550315 learned_addr learned my addr 172.21.15.100:0/2820550315 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:08.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.096+0000 7f7e8259c640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e840c9320 0x7f7e8406c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.096+0000 7f7e81d9b640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e84152040 0x7f7e84148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.096+0000 7f7e82d9d640 1 -- 172.21.15.100:0/2820550315 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e840c9320 msgr2=0x7f7e8406c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:08.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.096+0000 7f7e82d9d640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7e840c9320 0x7f7e8406c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:08.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.096+0000 7f7e82d9d640 1 -- 172.21.15.100:0/2820550315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e84152040 msgr2=0x7f7e84148f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:08.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.096+0000 7f7e82d9d640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e84152040 0x7f7e84148f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:08.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.096+0000 7f7e82d9d640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8406c820 con 0x7f7e84157d20 2024-09-15T21:48:08.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.096+0000 7f7e81d9b640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e84152040 0x7f7e84148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:48:08.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.097+0000 7f7e82d9d640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7e84157d20 0x7f7e8406cd60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00e0f0 tx=0x7f7e7c00e5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:08.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.097+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c00b0d0 con 0x7f7e84157d20 2024-09-15T21:48:08.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.097+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e7c00b270 con 0x7f7e84157d20 2024-09-15T21:48:08.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.097+0000 7f7e88b18640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e840cb840 con 0x7f7e84157d20 2024-09-15T21:48:08.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.097+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c003b90 con 0x7f7e84157d20 2024-09-15T21:48:08.101 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.097+0000 7f7e88b18640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e840cbe00 con 0x7f7e84157d20 2024-09-15T21:48:08.102 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.098+0000 7f7e88b18640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd ls"} v 0) -- 0x7f7e8415b6e0 con 0x7f7e84157d20 2024-09-15T21:48:08.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.098+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7e7c01d030 con 0x7f7e84157d20 2024-09-15T21:48:08.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.099+0000 7f7e72ffd640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7e6406c2e0 0x7f7e6406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.099+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6060+0+0 (secure 0 0 0) 0x7f7e7c08d8b0 con 0x7f7e84157d20 2024-09-15T21:48:08.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.099+0000 7f7e8259c640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7e6406c2e0 0x7f7e6406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.099+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "osd ls"}]=0 v53) ==== 54+0+16 (secure 0 0 0) 0x7f7e7c024020 con 0x7f7e84157d20 2024-09-15T21:48:08.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.099+0000 7f7e88b18640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f7e840cbfd0 0x7f7e8406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.099+0000 7f7e88b18640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7e840cc450 con 0x7f7e840cbfd0 2024-09-15T21:48:08.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.099+0000 7f7e8259c640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7e6406c2e0 0x7f7e6406e7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7e74034000 tx=0x7f7e7402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:08.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.100+0000 7f7e81d9b640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f7e840cbfd0 0x7f7e8406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.100+0000 7f7e81d9b640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f7e840cbfd0 0x7f7e8406d2a0 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:08.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.100+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7e840cc450 con 0x7f7e840cbfd0 2024-09-15T21:48:08.114 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-15T21:48:08.114 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-15T21:48:08.128 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.127+0000 7f7e88b18640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7e840c42d0 con 0x7f7e840cbfd0 2024-09-15T21:48:08.131 INFO:teuthology.orchestra.run.smithi100.stdout:osd.0: {} 2024-09-15T21:48:08.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.128+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7e840c42d0 con 0x7f7e840cbfd0 2024-09-15T21:48:08.131 INFO:teuthology.orchestra.run.smithi100.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:48:08.131 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.128+0000 7f7e88b18640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f7e84147860 0x7f7e84147c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.128+0000 7f7e88b18640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 3: {"prefix": "get_command_descriptions"}) -- 0x7f7e840739b0 con 0x7f7e84147860 2024-09-15T21:48:08.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.129+0000 7f7e8259c640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f7e84147860 0x7f7e84147c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.129+0000 7f7e8259c640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7f7e84147860 0x7f7e84147c80 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:08.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.130+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 3: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7e840739b0 con 0x7f7e84147860 2024-09-15T21:48:08.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.145+0000 7f7e88b18640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 4: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7e841481c0 con 0x7f7e84147860 2024-09-15T21:48:08.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.147+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 4: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7e841481c0 con 0x7f7e84147860 2024-09-15T21:48:08.148 INFO:teuthology.orchestra.run.smithi100.stdout:osd.1: {} 2024-09-15T21:48:08.149 INFO:teuthology.orchestra.run.smithi100.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:48:08.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.147+0000 7f7e88b18640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f7e841f03f0 0x7f7e841f07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.147+0000 7f7e88b18640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 5: {"prefix": "get_command_descriptions"}) -- 0x7f7e840c6120 con 0x7f7e841f03f0 2024-09-15T21:48:08.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.147+0000 7f7e81d9b640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f7e841f03f0 0x7f7e841f07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.147+0000 7f7e81d9b640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7f7e841f03f0 0x7f7e841f07f0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:08.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.148+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 5: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7e840c6120 con 0x7f7e841f03f0 2024-09-15T21:48:08.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.162+0000 7f7e88b18640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 6: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7e840c6630 con 0x7f7e841f03f0 2024-09-15T21:48:08.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.164+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 6: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7e840c6630 con 0x7f7e841f03f0 2024-09-15T21:48:08.168 INFO:teuthology.orchestra.run.smithi100.stdout:osd.2: {} 2024-09-15T21:48:08.168 INFO:teuthology.orchestra.run.smithi100.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:48:08.169 INFO:tasks.workunit:Stopping ['fs/misc/trivial_sync.sh'] on client.0... 2024-09-15T21:48:08.169 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-15T21:48:08.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.165+0000 7f7e88b18640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f7e84062630 0x7f7e84062a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.165+0000 7f7e88b18640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 7: {"prefix": "get_command_descriptions"}) -- 0x7f7e84062f90 con 0x7f7e84062630 2024-09-15T21:48:08.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.165+0000 7f7e82d9d640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f7e84062630 0x7f7e84062a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.165+0000 7f7e82d9d640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f7e84062630 0x7f7e84062a50 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:08.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.166+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 7: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7e84062f90 con 0x7f7e84062630 2024-09-15T21:48:08.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.179+0000 7f7e88b18640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 8: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7e840634a0 con 0x7f7e84062630 2024-09-15T21:48:08.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.181+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 8: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7e840634a0 con 0x7f7e84062630 2024-09-15T21:48:08.183 INFO:teuthology.orchestra.run.smithi100.stdout:osd.3: {} 2024-09-15T21:48:08.184 INFO:teuthology.orchestra.run.smithi100.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:48:08.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.181+0000 7f7e88b18640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f7e8402cb50 0x7f7e8402cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.181+0000 7f7e88b18640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 9: {"prefix": "get_command_descriptions"}) -- 0x7f7e84062f90 con 0x7f7e8402cb50 2024-09-15T21:48:08.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.181+0000 7f7e8259c640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f7e8402cb50 0x7f7e8402cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.182+0000 7f7e8259c640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f7e8402cb50 0x7f7e8402cf70 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:08.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.183+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 9: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7e84062f90 con 0x7f7e8402cb50 2024-09-15T21:48:08.185 INFO:tasks.ceph.osd.4.smithi122.stderr:2024-09-15T21:48:08.184+0000 7f44de052640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 37845) UID: 0 2024-09-15T21:48:08.185 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:08.186 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:08.195 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:48:08.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.196+0000 7f7e88b18640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 10: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7e840634a0 con 0x7f7e8402cb50 2024-09-15T21:48:08.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.198+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 10: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7e840634a0 con 0x7f7e8402cb50 2024-09-15T21:48:08.200 INFO:teuthology.orchestra.run.smithi100.stdout:osd.4: {} 2024-09-15T21:48:08.202 INFO:teuthology.orchestra.run.smithi100.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:48:08.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.199+0000 7f7e88b18640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=54}) -- 0x7f7e84062f90 con 0x7f7e84157d20 2024-09-15T21:48:08.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.199+0000 7f7e88b18640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7e840634a0 con 0x7f7e84157d20 2024-09-15T21:48:08.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.199+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f7e7c05b390 con 0x7f7e84157d20 2024-09-15T21:48:08.202 INFO:teuthology.orchestra.run.smithi100.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-09-15T21:48:08.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.199+0000 7f7e88b18640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f7e8402d870 0x7f7e84203610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.199+0000 7f7e88b18640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 12: {"prefix": "get_command_descriptions"}) -- 0x7f7e84206460 con 0x7f7e8402d870 2024-09-15T21:48:08.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.199+0000 7f7e81d9b640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f7e8402d870 0x7f7e84203610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.202 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.200+0000 7f7e81d9b640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f7e8402d870 0x7f7e84203610 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:08.203 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.200+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 12: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7e84206460 con 0x7f7e8402d870 2024-09-15T21:48:08.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.214+0000 7f7e88b18640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 13: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7e84203b50 con 0x7f7e8402d870 2024-09-15T21:48:08.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.216+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 13: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7e84203b50 con 0x7f7e8402d870 2024-09-15T21:48:08.218 INFO:teuthology.orchestra.run.smithi100.stdout:osd.6: {} 2024-09-15T21:48:08.219 INFO:teuthology.orchestra.run.smithi100.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:48:08.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.216+0000 7f7e88b18640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f7e84204270 0x7f7e842046b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.216+0000 7f7e88b18640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 14: {"prefix": "get_command_descriptions"}) -- 0x7f7e84209820 con 0x7f7e84204270 2024-09-15T21:48:08.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.217+0000 7f7e82d9d640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f7e84204270 0x7f7e842046b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.217+0000 7f7e82d9d640 1 --2- 172.21.15.100:0/2820550315 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7f7e84204270 0x7f7e842046b0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:08.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.218+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 14: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7e84209820 con 0x7f7e84204270 2024-09-15T21:48:08.233 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.231+0000 7f7e88b18640 1 -- 172.21.15.100:0/2820550315 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 15: {"prefix": "injectargs", "injected_args": ["--osd_enable_op_tracker=false"]}) -- 0x7f7e84204d00 con 0x7f7e84204270 2024-09-15T21:48:08.235 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.233+0000 7f7e72ffd640 1 -- 172.21.15.100:0/2820550315 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 15: 0 osd_enable_op_tracker = '' ) ==== 35+0+3 (crc 0 0 0) 0x7f7e84204d00 con 0x7f7e84204270 2024-09-15T21:48:08.235 INFO:teuthology.orchestra.run.smithi100.stdout:osd.7: {} 2024-09-15T21:48:08.235 INFO:teuthology.orchestra.run.smithi100.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:48:08.239 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-15T21:48:08.239 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-15T21:48:08.286 INFO:tasks.ceph.osd.2.smithi100.stderr:2024-09-15T21:48:08.284+0000 7f1315cc2640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 36472) UID: 0 2024-09-15T21:48:08.303 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:08.303 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:08.313 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:48:08.396 INFO:tasks.ceph.osd.0.smithi100.stderr:2024-09-15T21:48:08.395+0000 7f5512493640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 36468) UID: 0 2024-09-15T21:48:08.458 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:08.459 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:08.468 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:48:08.483 DEBUG:teuthology.parallel:result is None 2024-09-15T21:48:08.484 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T21:48:08.526 INFO:tasks.ceph.osd.1.smithi100.stderr:2024-09-15T21:48:08.524+0000 7fa4cc654640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 36463) UID: 0 2024-09-15T21:48:08.557 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-15T21:48:08.558 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-15T21:48:08.567 INFO:tasks.thrashosds:joining thrashosds 2024-09-15T21:48:08.622 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:48:08.630 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:08.630 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:08.640 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:48:08.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.722+0000 7f5989731640 1 Processor -- start 2024-09-15T21:48:08.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.722+0000 7f5989731640 1 -- start start 2024-09-15T21:48:08.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.723+0000 7f5989731640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5984153550 0x7f59840c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.723+0000 7f5989731640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59840cfb90 con 0x7f59840c5c90 2024-09-15T21:48:08.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.723+0000 7f5989731640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59841593f0 con 0x7f598406fbd0 2024-09-15T21:48:08.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.723+0000 7f5989731640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f59841596d0 con 0x7f5984153550 2024-09-15T21:48:08.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.723+0000 7f597a7fc640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f59840c5c90 0x7f598406f4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:38476/0 (socket says 172.21.15.100:38476) 2024-09-15T21:48:08.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.723+0000 7f597a7fc640 1 -- 172.21.15.100:0/1993443151 learned_addr learned my addr 172.21.15.100:0/1993443151 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:08.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.723+0000 7f5982ffd640 1 --2- 172.21.15.100:0/1993443151 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5984153550 0x7f59840c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.723+0000 7f59827fc640 1 -- 172.21.15.100:0/1993443151 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2683976233 0 0) 0x7f596c001110 con 0x7f59840c5c90 2024-09-15T21:48:08.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.723+0000 7f59827fc640 1 -- 172.21.15.100:0/1993443151 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5970003520 con 0x7f59840c5c90 2024-09-15T21:48:08.727 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.723+0000 7f59827fc640 1 -- 172.21.15.100:0/1993443151 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 747210242 0 0) 0x7f5970003520 con 0x7f59840c5c90 2024-09-15T21:48:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.723+0000 7f59827fc640 1 -- 172.21.15.100:0/1993443151 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5984153550 msgr2=0x7f59840c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.723+0000 7f59827fc640 1 --2- 172.21.15.100:0/1993443151 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5984153550 0x7f59840c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.723+0000 7f59827fc640 1 -- 172.21.15.100:0/1993443151 >> v1:172.21.15.122:6789/0 conn(0x7f598406fbd0 legacy=0x7f5984163980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.723+0000 7f59827fc640 1 -- 172.21.15.100:0/1993443151 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59841599b0 con 0x7f59840c5c90 2024-09-15T21:48:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.723+0000 7f59827fc640 1 -- 172.21.15.100:0/1993443151 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7f596c001840 con 0x7f59840c5c90 2024-09-15T21:48:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.723+0000 7f59827fc640 1 -- 172.21.15.100:0/1993443151 >> v1:172.21.15.100:6789/0 conn(0x7f59840c5c90 legacy=0x7f598406f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.723+0000 7f59827fc640 1 --2- 172.21.15.100:0/1993443151 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5984153550 0x7f59700039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.723+0000 7f59827fc640 1 --2- 172.21.15.100:0/1993443151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5970003f20 0x7f5970004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.723+0000 7f59827fc640 1 --2- 172.21.15.100:0/1993443151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5970004d40 0x7f59700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.724+0000 7f59827fc640 1 -- 172.21.15.100:0/1993443151 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f59841596d0 con 0x7f5970003f20 2024-09-15T21:48:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.724+0000 7f59827fc640 1 -- 172.21.15.100:0/1993443151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f597000ba20 con 0x7f5970004d40 2024-09-15T21:48:08.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.724+0000 7f59827fc640 1 -- 172.21.15.100:0/1993443151 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f597000bc20 con 0x7f5984153550 2024-09-15T21:48:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.724+0000 7f597a7fc640 1 --2- 172.21.15.100:0/1993443151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5970003f20 0x7f5970004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.724+0000 7f5982ffd640 1 --2- 172.21.15.100:0/1993443151 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5984153550 0x7f59700039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.724+0000 7f59837fe640 1 --2- 172.21.15.100:0/1993443151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5970004d40 0x7f59700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.724+0000 7f597a7fc640 1 -- 172.21.15.100:0/1993443151 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5984153550 msgr2=0x7f59700039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.724+0000 7f597a7fc640 1 --2- 172.21.15.100:0/1993443151 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5984153550 0x7f59700039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.724+0000 7f597a7fc640 1 -- 172.21.15.100:0/1993443151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5970004d40 msgr2=0x7f59700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.724+0000 7f597a7fc640 1 --2- 172.21.15.100:0/1993443151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5970004d40 0x7f59700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.724+0000 7f597a7fc640 1 -- 172.21.15.100:0/1993443151 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5970003520 con 0x7f5970003f20 2024-09-15T21:48:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.724+0000 7f5982ffd640 1 --2- 172.21.15.100:0/1993443151 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5984153550 0x7f59700039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:48:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.724+0000 7f59837fe640 1 --2- 172.21.15.100:0/1993443151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5970004d40 0x7f59700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:48:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.725+0000 7f597a7fc640 1 --2- 172.21.15.100:0/1993443151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5970003f20 0x7f5970004800 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f59840692d0 tx=0x7f5970005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c627bbc5a01e2a server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.725+0000 7f59827fc640 1 -- 172.21.15.100:0/1993443151 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c036070 con 0x7f5970003f20 2024-09-15T21:48:08.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.725+0000 7f59827fc640 1 -- 172.21.15.100:0/1993443151 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f596c005ad0 con 0x7f5970003f20 2024-09-15T21:48:08.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.725+0000 7f59827fc640 1 -- 172.21.15.100:0/1993443151 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596c00c0e0 con 0x7f5970003f20 2024-09-15T21:48:08.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.726+0000 7f5989731640 1 -- 172.21.15.100:0/1993443151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5970003f20 msgr2=0x7f5970004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:08.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.726+0000 7f5989731640 1 --2- 172.21.15.100:0/1993443151 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5970003f20 0x7f5970004800 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f59840692d0 tx=0x7f5970005830 comp rx=0 tx=0).stop 2024-09-15T21:48:08.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.726+0000 7f59837fe640 1 -- 172.21.15.100:0/1993443151 reap_dead start 2024-09-15T21:48:08.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.726+0000 7f5989731640 1 -- 172.21.15.100:0/1993443151 shutdown_connections 2024-09-15T21:48:08.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.726+0000 7f5989731640 1 -- 172.21.15.100:0/1993443151 >> 172.21.15.100:0/1993443151 conn(0x7f59840cec80 msgr2=0x7f59840cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:08.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.726+0000 7f5989731640 1 -- 172.21.15.100:0/1993443151 shutdown_connections 2024-09-15T21:48:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.726+0000 7f5989731640 1 -- 172.21.15.100:0/1993443151 wait complete. 2024-09-15T21:48:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.728+0000 7f5989731640 1 Processor -- start 2024-09-15T21:48:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.728+0000 7f5989731640 1 -- start start 2024-09-15T21:48:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.729+0000 7f5989731640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5970003f20 0x7f59841574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.729+0000 7f5989731640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5984156650 0x7f59841ef220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.729+0000 7f5989731640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59841ef760 0x7f59840c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.729+0000 7f5989731640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f598406efa0 con 0x7f59841ef760 2024-09-15T21:48:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.729+0000 7f5989731640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5984159f70 con 0x7f5984156650 2024-09-15T21:48:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.729+0000 7f5989731640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f59841590e0 con 0x7f5970003f20 2024-09-15T21:48:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.729+0000 7f59837fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59841ef760 0x7f59840c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.729+0000 7f59837fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59841ef760 0x7f59840c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49778/0 (socket says 172.21.15.100:49778) 2024-09-15T21:48:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.729+0000 7f59837fe640 1 -- 172.21.15.100:0/1884688839 learned_addr learned my addr 172.21.15.100:0/1884688839 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.729+0000 7f597a7fc640 1 --2- 172.21.15.100:0/1884688839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5984156650 0x7f59841ef220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.729+0000 7f5982ffd640 1 --2- 172.21.15.100:0/1884688839 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5970003f20 0x7f59841574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.730+0000 7f59837fe640 1 -- 172.21.15.100:0/1884688839 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5970003f20 msgr2=0x7f59841574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:08.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.730+0000 7f59837fe640 1 --2- 172.21.15.100:0/1884688839 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5970003f20 0x7f59841574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:08.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.730+0000 7f59837fe640 1 -- 172.21.15.100:0/1884688839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5984156650 msgr2=0x7f59841ef220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:08.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.730+0000 7f59837fe640 1 --2- 172.21.15.100:0/1884688839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5984156650 0x7f59841ef220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:08.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.730+0000 7f59837fe640 1 -- 172.21.15.100:0/1884688839 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5984153d90 con 0x7f59841ef760 2024-09-15T21:48:08.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.730+0000 7f597a7fc640 1 --2- 172.21.15.100:0/1884688839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5984156650 0x7f59841ef220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:48:08.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.730+0000 7f59837fe640 1 --2- 172.21.15.100:0/1884688839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59841ef760 0x7f59840c70d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f59840cb750 tx=0x7f597c00feb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:08.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.730+0000 7f597bfff640 1 -- 172.21.15.100:0/1884688839 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f597c001b30 con 0x7f59841ef760 2024-09-15T21:48:08.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.730+0000 7f597bfff640 1 -- 172.21.15.100:0/1884688839 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f597c018070 con 0x7f59841ef760 2024-09-15T21:48:08.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.730+0000 7f597bfff640 1 -- 172.21.15.100:0/1884688839 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5974009690 con 0x7f59841ef760 2024-09-15T21:48:08.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.730+0000 7f5989731640 1 -- 172.21.15.100:0/1884688839 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59841e4cd0 con 0x7f59841ef760 2024-09-15T21:48:08.734 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.730+0000 7f5989731640 1 -- 172.21.15.100:0/1884688839 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59841f2ca0 con 0x7f59841ef760 2024-09-15T21:48:08.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.731+0000 7f5989731640 1 -- 172.21.15.100:0/1884688839 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5984070260 con 0x7f59841ef760 2024-09-15T21:48:08.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.732+0000 7f597bfff640 1 -- 172.21.15.100:0/1884688839 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f597c00d7a0 con 0x7f59841ef760 2024-09-15T21:48:08.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.732+0000 7f597bfff640 1 --2- 172.21.15.100:0/1884688839 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f596406c2e0 0x7f596406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:08.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.732+0000 7f597bfff640 1 -- 172.21.15.100:0/1884688839 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6060+0+0 (secure 0 0 0) 0x7f597c08b260 con 0x7f59841ef760 2024-09-15T21:48:08.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.732+0000 7f5982ffd640 1 --2- 172.21.15.100:0/1884688839 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f596406c2e0 0x7f596406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:08.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.733+0000 7f5982ffd640 1 --2- 172.21.15.100:0/1884688839 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f596406c2e0 0x7f596406e7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5974009d60 tx=0x7f5974009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:08.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.737+0000 7f597bfff640 1 -- 172.21.15.100:0/1884688839 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f597c059870 con 0x7f59841ef760 2024-09-15T21:48:08.789 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:08.789 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:08.798 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:48:08.903 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:08.903 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:08.912 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:48:08.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:08.919+0000 7f5989731640 1 -- 172.21.15.100:0/1884688839 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd unset", "key": "noscrub"} v 0) -- 0x7f59840c4670 con 0x7f59841ef760 2024-09-15T21:48:09.015 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:09.015 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:09.024 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:48:09.065 INFO:tasks.thrashosds.thrasher:reviving osd 2024-09-15T21:48:09.065 INFO:tasks.thrashosds.thrasher:Reviving osd 5 2024-09-15T21:48:09.065 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-15T21:48:09.065 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-15T21:48:09.068 INFO:tasks.ceph.osd.5:Started 2024-09-15T21:48:09.068 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:48:09.169 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:09.169 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:09.176 INFO:teuthology.orchestra.run.smithi122.stderr:no valid command found; 10 closest matches: 2024-09-15T21:48:09.176 INFO:teuthology.orchestra.run.smithi122.stderr:0 2024-09-15T21:48:09.176 INFO:teuthology.orchestra.run.smithi122.stderr:1 2024-09-15T21:48:09.176 INFO:teuthology.orchestra.run.smithi122.stderr:2 2024-09-15T21:48:09.176 INFO:teuthology.orchestra.run.smithi122.stderr:abort 2024-09-15T21:48:09.176 INFO:teuthology.orchestra.run.smithi122.stderr:assert 2024-09-15T21:48:09.176 INFO:teuthology.orchestra.run.smithi122.stderr:config diff 2024-09-15T21:48:09.176 INFO:teuthology.orchestra.run.smithi122.stderr:config diff get 2024-09-15T21:48:09.176 INFO:teuthology.orchestra.run.smithi122.stderr:config get 2024-09-15T21:48:09.176 INFO:teuthology.orchestra.run.smithi122.stderr:config help [] 2024-09-15T21:48:09.176 INFO:teuthology.orchestra.run.smithi122.stderr:config set ... 2024-09-15T21:48:09.177 INFO:teuthology.orchestra.run.smithi122.stderr:admin_socket: invalid command 2024-09-15T21:48:09.178 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:48:09.178 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T21:48:09.179 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-5, ['dump_ops_in_flight'] 2024-09-15T21:48:09.281 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:09.281 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:09.290 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:48:09.393 INFO:teuthology.orchestra.run.smithi100.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:09.393 INFO:teuthology.orchestra.run.smithi100.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:09.402 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:48:09.507 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:09.507 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:09.516 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:48:09.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.580+0000 7f597bfff640 1 -- 172.21.15.100:0/1884688839 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "noscrub"}]=0 noscrub is unset v54) ==== 91+0+0 (secure 0 0 0) 0x7f597c059170 con 0x7f59841ef760 2024-09-15T21:48:09.582 INFO:teuthology.orchestra.run.smithi100.stderr:noscrub is unset 2024-09-15T21:48:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.583+0000 7f5989731640 1 -- 172.21.15.100:0/1884688839 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f596406c2e0 msgr2=0x7f596406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.583+0000 7f5989731640 1 --2- 172.21.15.100:0/1884688839 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f596406c2e0 0x7f596406e7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5974009d60 tx=0x7f5974009190 comp rx=0 tx=0).stop 2024-09-15T21:48:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.583+0000 7f5989731640 1 -- 172.21.15.100:0/1884688839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59841ef760 msgr2=0x7f59840c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.583+0000 7f5989731640 1 --2- 172.21.15.100:0/1884688839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59841ef760 0x7f59840c70d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f59840cb750 tx=0x7f597c00feb0 comp rx=0 tx=0).stop 2024-09-15T21:48:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.583+0000 7f5989731640 1 -- 172.21.15.100:0/1884688839 shutdown_connections 2024-09-15T21:48:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.583+0000 7f5989731640 1 --2- 172.21.15.100:0/1884688839 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f596406c2e0 0x7f596406e7a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.583+0000 7f5989731640 1 --2- 172.21.15.100:0/1884688839 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5984156650 0x7f59841ef220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.583+0000 7f5989731640 1 --2- 172.21.15.100:0/1884688839 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f59841ef760 0x7f59840c70d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:09.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.583+0000 7f5989731640 1 --2- 172.21.15.100:0/1884688839 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5970003f20 0x7f59841574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:09.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.583+0000 7f5989731640 1 -- 172.21.15.100:0/1884688839 >> 172.21.15.100:0/1884688839 conn(0x7f59840cec80 msgr2=0x7f59840ca1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:09.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.583+0000 7f5989731640 1 -- 172.21.15.100:0/1884688839 shutdown_connections 2024-09-15T21:48:09.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.583+0000 7f5989731640 1 -- 172.21.15.100:0/1884688839 wait complete. 2024-09-15T21:48:09.596 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:48:09.621 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:09.621 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:09.630 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:48:09.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.693+0000 7f62446af640 1 Processor -- start 2024-09-15T21:48:09.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.693+0000 7f62446af640 1 -- start start 2024-09-15T21:48:09.696 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.694+0000 7f62446af640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f623c0c9b40 0x7f623c164830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:09.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.694+0000 7f62446af640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f623c15c530 con 0x7f623c152fe0 2024-09-15T21:48:09.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.694+0000 7f62446af640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f623c15d3d0 con 0x7f623c0c9710 2024-09-15T21:48:09.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.694+0000 7f62446af640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f623c159ec0 con 0x7f623c0c9b40 2024-09-15T21:48:09.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.694+0000 7f624244b640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f623c152fe0 0x7f623c15ccc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:38480/0 (socket says 172.21.15.100:38480) 2024-09-15T21:48:09.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.694+0000 7f624244b640 1 -- 172.21.15.100:0/853200424 learned_addr learned my addr 172.21.15.100:0/853200424 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:09.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.694+0000 7f6242c4c640 1 --2- 172.21.15.100:0/853200424 >> v2:172.21.15.122:3300/0 conn(0x7f623c0c9b40 0x7f623c164830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:09.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.694+0000 7f6241449640 1 -- 172.21.15.100:0/853200424 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2405422749 0 0) 0x7f623c15c530 con 0x7f623c152fe0 2024-09-15T21:48:09.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.694+0000 7f6241449640 1 -- 172.21.15.100:0/853200424 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6228003520 con 0x7f623c152fe0 2024-09-15T21:48:09.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.695+0000 7f6241449640 1 -- 172.21.15.100:0/853200424 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3613386974 0 0) 0x7f623c15d3d0 con 0x7f623c0c9710 2024-09-15T21:48:09.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.695+0000 7f6241449640 1 -- 172.21.15.100:0/853200424 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f623c15c530 con 0x7f623c0c9710 2024-09-15T21:48:09.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.695+0000 7f6241449640 1 -- 172.21.15.100:0/853200424 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 133808315 0 0) 0x7f6228003520 con 0x7f623c152fe0 2024-09-15T21:48:09.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.695+0000 7f6242c4c640 1 -- 172.21.15.100:0/853200424 >> v1:172.21.15.122:6789/0 conn(0x7f623c0c9710 legacy=0x7f623c0730f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:09.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.695+0000 7f6242c4c640 1 -- 172.21.15.100:0/853200424 >> v1:172.21.15.100:6789/0 conn(0x7f623c152fe0 legacy=0x7f623c15ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:09.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.695+0000 7f6242c4c640 1 -- 172.21.15.100:0/853200424 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623c0c3f30 con 0x7f623c0c9b40 2024-09-15T21:48:09.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.695+0000 7f6242c4c640 1 --2- 172.21.15.100:0/853200424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f623c0c9b40 0x7f623c164830 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6234009d90 tx=0x7f623402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bd10403ff7ba087 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:09.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.696+0000 7f6241449640 1 -- 172.21.15.100:0/853200424 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6238001670 con 0x7f623c0c9b40 2024-09-15T21:48:09.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.696+0000 7f6241449640 1 -- 172.21.15.100:0/853200424 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6234007540 con 0x7f623c0c9b40 2024-09-15T21:48:09.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.696+0000 7f6241449640 1 -- 172.21.15.100:0/853200424 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62340078f0 con 0x7f623c0c9b40 2024-09-15T21:48:09.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.696+0000 7f62446af640 1 -- 172.21.15.100:0/853200424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f623c0c9b40 msgr2=0x7f623c164830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:09.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.696+0000 7f62446af640 1 --2- 172.21.15.100:0/853200424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f623c0c9b40 0x7f623c164830 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6234009d90 tx=0x7f623402a710 comp rx=0 tx=0).stop 2024-09-15T21:48:09.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.696+0000 7f62446af640 1 -- 172.21.15.100:0/853200424 shutdown_connections 2024-09-15T21:48:09.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.696+0000 7f62446af640 1 --2- 172.21.15.100:0/853200424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f623c0c9b40 0x7f623c164830 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:09.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.696+0000 7f62446af640 1 -- 172.21.15.100:0/853200424 >> 172.21.15.100:0/853200424 conn(0x7f623c067dd0 msgr2=0x7f623c06bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:09.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.696+0000 7f62446af640 1 -- 172.21.15.100:0/853200424 shutdown_connections 2024-09-15T21:48:09.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.696+0000 7f62446af640 1 -- 172.21.15.100:0/853200424 wait complete. 2024-09-15T21:48:09.707 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.697+0000 7f62446af640 1 Processor -- start 2024-09-15T21:48:09.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.697+0000 7f62446af640 1 -- start start 2024-09-15T21:48:09.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.698+0000 7f62446af640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f623c0c9710 0x7f623c1f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:09.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.698+0000 7f62446af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623c0c9b40 0x7f623c154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:09.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.698+0000 7f62446af640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f623c152fe0 0x7f623c154100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:09.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.698+0000 7f62446af640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f623c0c44f0 con 0x7f623c0c9b40 2024-09-15T21:48:09.708 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.698+0000 7f62446af640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f623c15c830 con 0x7f623c0c9710 2024-09-15T21:48:09.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.698+0000 7f62446af640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f623c158ea0 con 0x7f623c152fe0 2024-09-15T21:48:09.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.698+0000 7f6241c4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623c0c9b40 0x7f623c154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:09.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.698+0000 7f6241c4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623c0c9b40 0x7f623c154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:49788/0 (socket says 172.21.15.100:49788) 2024-09-15T21:48:09.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.698+0000 7f6241c4a640 1 -- 172.21.15.100:0/3784399448 learned_addr learned my addr 172.21.15.100:0/3784399448 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:09.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.698+0000 7f624244b640 1 --2- 172.21.15.100:0/3784399448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f623c0c9710 0x7f623c1f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:09.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.698+0000 7f6242c4c640 1 --2- 172.21.15.100:0/3784399448 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f623c152fe0 0x7f623c154100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:09.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.698+0000 7f6241c4a640 1 -- 172.21.15.100:0/3784399448 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f623c152fe0 msgr2=0x7f623c154100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:09.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.698+0000 7f6241c4a640 1 --2- 172.21.15.100:0/3784399448 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f623c152fe0 0x7f623c154100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:09.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.698+0000 7f6241c4a640 1 -- 172.21.15.100:0/3784399448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f623c0c9710 msgr2=0x7f623c1f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:09.710 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.698+0000 7f6241c4a640 1 --2- 172.21.15.100:0/3784399448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f623c0c9710 0x7f623c1f25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:09.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.698+0000 7f6241c4a640 1 -- 172.21.15.100:0/3784399448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f623c0cc110 con 0x7f623c0c9b40 2024-09-15T21:48:09.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.699+0000 7f6241c4a640 1 --2- 172.21.15.100:0/3784399448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623c0c9b40 0x7f623c154fa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f622c00c490 tx=0x7f622c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:09.711 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.699+0000 7f6232ffd640 1 -- 172.21.15.100:0/3784399448 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c0052d0 con 0x7f623c0c9b40 2024-09-15T21:48:09.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.699+0000 7f6232ffd640 1 -- 172.21.15.100:0/3784399448 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f622c005470 con 0x7f623c0c9b40 2024-09-15T21:48:09.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.699+0000 7f6232ffd640 1 -- 172.21.15.100:0/3784399448 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c012870 con 0x7f623c0c9b40 2024-09-15T21:48:09.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.699+0000 7f62446af640 1 -- 172.21.15.100:0/3784399448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f623c156d60 con 0x7f623c0c9b40 2024-09-15T21:48:09.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.699+0000 7f62446af640 1 -- 172.21.15.100:0/3784399448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f623c153900 con 0x7f623c0c9b40 2024-09-15T21:48:09.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.700+0000 7f62446af640 1 -- 172.21.15.100:0/3784399448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f623c1574b0 con 0x7f623c0c9b40 2024-09-15T21:48:09.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.700+0000 7f6232ffd640 1 -- 172.21.15.100:0/3784399448 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f622c012a10 con 0x7f623c0c9b40 2024-09-15T21:48:09.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.701+0000 7f6232ffd640 1 --2- 172.21.15.100:0/3784399448 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f622006c2b0 0x7f622006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:09.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.701+0000 7f624244b640 1 --2- 172.21.15.100:0/3784399448 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f622006c2b0 0x7f622006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:09.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.701+0000 7f6232ffd640 1 -- 172.21.15.100:0/3784399448 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6060+0+0 (secure 0 0 0) 0x7f622c08b0d0 con 0x7f623c0c9b40 2024-09-15T21:48:09.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.701+0000 7f624244b640 1 --2- 172.21.15.100:0/3784399448 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f622006c2b0 0x7f622006e770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f62380017e0 tx=0x7f6238001610 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:09.714 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.705+0000 7f6232ffd640 1 -- 172.21.15.100:0/3784399448 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f623c1574b0 con 0x7f623c0c9b40 2024-09-15T21:48:09.734 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:09.735 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:09.743 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:48:09.848 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:09.848 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:09.856 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:48:09.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:09.882+0000 7f62446af640 1 -- 172.21.15.100:0/3784399448 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd unset", "key": "nodeep-scrub"} v 0) -- 0x7f623c0c47d0 con 0x7f623c0c9b40 2024-09-15T21:48:09.959 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:09.960 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:09.969 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:48:10.072 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:10.072 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:10.081 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:48:10.183 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:10.184 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:10.193 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:48:10.296 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:10.296 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:10.305 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:48:10.408 INFO:teuthology.orchestra.run.smithi122.stdout:ERROR: (22) Invalid argument 2024-09-15T21:48:10.409 INFO:teuthology.orchestra.run.smithi122.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:48:10.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:10.588+0000 7f6232ffd640 1 -- 172.21.15.100:0/3784399448 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "nodeep-scrub"}]=0 nodeep-scrub is unset v55) ==== 101+0+0 (secure 0 0 0) 0x7f622c0590a0 con 0x7f623c0c9b40 2024-09-15T21:48:10.590 INFO:teuthology.orchestra.run.smithi100.stderr:nodeep-scrub is unset 2024-09-15T21:48:10.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:10.591+0000 7f62446af640 1 -- 172.21.15.100:0/3784399448 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f622006c2b0 msgr2=0x7f622006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:10.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:10.591+0000 7f62446af640 1 --2- 172.21.15.100:0/3784399448 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f622006c2b0 0x7f622006e770 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f62380017e0 tx=0x7f6238001610 comp rx=0 tx=0).stop 2024-09-15T21:48:10.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:10.591+0000 7f62446af640 1 -- 172.21.15.100:0/3784399448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623c0c9b40 msgr2=0x7f623c154fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:10.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:10.591+0000 7f62446af640 1 --2- 172.21.15.100:0/3784399448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623c0c9b40 0x7f623c154fa0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f622c00c490 tx=0x7f622c00c950 comp rx=0 tx=0).stop 2024-09-15T21:48:10.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:10.591+0000 7f62446af640 1 -- 172.21.15.100:0/3784399448 shutdown_connections 2024-09-15T21:48:10.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:10.591+0000 7f62446af640 1 --2- 172.21.15.100:0/3784399448 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f622006c2b0 0x7f622006e770 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:10.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:10.591+0000 7f62446af640 1 --2- 172.21.15.100:0/3784399448 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f623c152fe0 0x7f623c154100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:10.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:10.591+0000 7f62446af640 1 --2- 172.21.15.100:0/3784399448 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f623c0c9b40 0x7f623c154fa0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:10.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:10.591+0000 7f62446af640 1 --2- 172.21.15.100:0/3784399448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f623c0c9710 0x7f623c1f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:10.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:10.591+0000 7f62446af640 1 -- 172.21.15.100:0/3784399448 >> 172.21.15.100:0/3784399448 conn(0x7f623c067dd0 msgr2=0x7f623c0730f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:10.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:10.592+0000 7f62446af640 1 -- 172.21.15.100:0/3784399448 shutdown_connections 2024-09-15T21:48:10.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:10.592+0000 7f62446af640 1 -- 172.21.15.100:0/3784399448 wait complete. 2024-09-15T21:48:11.005 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:48:11.001+0000 7f2141341640 -1 Falling back to public interface 2024-09-15T21:48:14.180 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:48:14.279 INFO:teuthology.orchestra.run.smithi122.stderr:no valid command found; 10 closest matches: 2024-09-15T21:48:14.279 INFO:teuthology.orchestra.run.smithi122.stderr:0 2024-09-15T21:48:14.279 INFO:teuthology.orchestra.run.smithi122.stderr:1 2024-09-15T21:48:14.279 INFO:teuthology.orchestra.run.smithi122.stderr:2 2024-09-15T21:48:14.279 INFO:teuthology.orchestra.run.smithi122.stderr:abort 2024-09-15T21:48:14.279 INFO:teuthology.orchestra.run.smithi122.stderr:assert 2024-09-15T21:48:14.279 INFO:teuthology.orchestra.run.smithi122.stderr:config diff 2024-09-15T21:48:14.279 INFO:teuthology.orchestra.run.smithi122.stderr:config diff get 2024-09-15T21:48:14.279 INFO:teuthology.orchestra.run.smithi122.stderr:config get 2024-09-15T21:48:14.279 INFO:teuthology.orchestra.run.smithi122.stderr:config help [] 2024-09-15T21:48:14.280 INFO:teuthology.orchestra.run.smithi122.stderr:config set ... 2024-09-15T21:48:14.280 INFO:teuthology.orchestra.run.smithi122.stderr:admin_socket: invalid command 2024-09-15T21:48:14.281 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T21:48:14.281 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-5, ['dump_ops_in_flight'] 2024-09-15T21:48:15.014 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:48:15.011+0000 7f2141341640 -1 osd.5 29 log_to_monitors true 2024-09-15T21:48:15.642 INFO:tasks.ceph.osd.5.smithi122.stderr:2024-09-15T21:48:15.640+0000 7f21350df640 -1 osd.5 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:48:16.956 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T21:48:16.959 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T21:48:19.282 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:48:19.400 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-15T21:48:19.400 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-15T21:48:19.400 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-15T21:48:19.400 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-15T21:48:19.409 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-15T21:48:19.410 INFO:tasks.thrashosds.thrasher:Adding osd 7 2024-09-15T21:48:19.410 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 7 2024-09-15T21:48:19.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.520+0000 7f15f0df6640 1 Processor -- start 2024-09-15T21:48:19.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.520+0000 7f15f0df6640 1 -- start start 2024-09-15T21:48:19.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.521+0000 7f15f0df6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f15ec073b80 0x7f15ec152d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:19.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.521+0000 7f15f0df6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f15ec0c9a50 0x7f15ec1532c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:19.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.521+0000 7f15f0df6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15ec0c8510 con 0x7f15ec0c78c0 2024-09-15T21:48:19.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.521+0000 7f15f0df6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f15ec0c82c0 con 0x7f15ec0c9a50 2024-09-15T21:48:19.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.521+0000 7f15f0df6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f15ec06c890 con 0x7f15ec073b80 2024-09-15T21:48:19.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.521+0000 7f15e1d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f15ec0c9a50 0x7f15ec1532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:19.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.521+0000 7f15ead9d640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f15ec0c78c0 0x7f15ec15c570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:49298/0 (socket says 172.21.15.100:49298) 2024-09-15T21:48:19.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.521+0000 7f15e1d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f15ec0c9a50 0x7f15ec1532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57254/0 (socket says 172.21.15.100:57254) 2024-09-15T21:48:19.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.521+0000 7f15ead9d640 1 -- 172.21.15.100:0/1205092261 learned_addr learned my addr 172.21.15.100:0/1205092261 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:19.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.521+0000 7f15ea59c640 1 --2- 172.21.15.100:0/1205092261 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f15ec073b80 0x7f15ec152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:19.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.521+0000 7f15e1d9b640 1 -- 172.21.15.100:0/1205092261 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f15ec073b80 msgr2=0x7f15ec152d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:19.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.521+0000 7f15e1d9b640 1 --2- 172.21.15.100:0/1205092261 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f15ec073b80 0x7f15ec152d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:19.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.521+0000 7f15e1d9b640 1 -- 172.21.15.100:0/1205092261 >> v1:172.21.15.100:6789/0 conn(0x7f15ec0c78c0 legacy=0x7f15ec15c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:19.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.521+0000 7f15e1d9b640 1 -- 172.21.15.100:0/1205092261 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ec159cb0 con 0x7f15ec0c9a50 2024-09-15T21:48:19.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.521+0000 7f15e9d9b640 1 -- 172.21.15.100:0/1205092261 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3453862170 0 0) 0x7f15ec0c8510 con 0x7f15ec0c78c0 2024-09-15T21:48:19.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.522+0000 7f15e1d9b640 1 --2- 172.21.15.100:0/1205092261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec0c9a50 0x7f15ec1532c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f15d40097d0 tx=0x7f15d402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e9c5bacbc24092 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:19.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.522+0000 7f15e9d9b640 1 -- 172.21.15.100:0/1205092261 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4037070 con 0x7f15ec0c9a50 2024-09-15T21:48:19.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.522+0000 7f15e9d9b640 1 -- 172.21.15.100:0/1205092261 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15d4032440 con 0x7f15ec0c9a50 2024-09-15T21:48:19.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.522+0000 7f15e9d9b640 1 -- 172.21.15.100:0/1205092261 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4032810 con 0x7f15ec0c9a50 2024-09-15T21:48:19.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.523+0000 7f15f0df6640 1 -- 172.21.15.100:0/1205092261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec0c9a50 msgr2=0x7f15ec1532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:19.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.523+0000 7f15f0df6640 1 --2- 172.21.15.100:0/1205092261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec0c9a50 0x7f15ec1532c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f15d40097d0 tx=0x7f15d402a250 comp rx=0 tx=0).stop 2024-09-15T21:48:19.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.523+0000 7f15f0df6640 1 -- 172.21.15.100:0/1205092261 shutdown_connections 2024-09-15T21:48:19.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.523+0000 7f15f0df6640 1 --2- 172.21.15.100:0/1205092261 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec0c9a50 0x7f15ec1532c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:19.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.523+0000 7f15f0df6640 1 --2- 172.21.15.100:0/1205092261 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f15ec073b80 0x7f15ec152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:19.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.523+0000 7f15f0df6640 1 -- 172.21.15.100:0/1205092261 >> 172.21.15.100:0/1205092261 conn(0x7f15ec06d890 msgr2=0x7f15ec06bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:19.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.523+0000 7f15f0df6640 1 -- 172.21.15.100:0/1205092261 shutdown_connections 2024-09-15T21:48:19.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.523+0000 7f15f0df6640 1 -- 172.21.15.100:0/1205092261 wait complete. 2024-09-15T21:48:19.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.524+0000 7f15f0df6640 1 Processor -- start 2024-09-15T21:48:19.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.524+0000 7f15f0df6640 1 -- start start 2024-09-15T21:48:19.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.525+0000 7f15f0df6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15ec073b80 0x7f15ec0cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:19.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.525+0000 7f15f0df6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec0c78c0 0x7f15ec1e5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:19.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.525+0000 7f15f0df6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f15ec0c9a50 0x7f15ec071700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:19.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.525+0000 7f15f0df6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec073f60 con 0x7f15ec0c78c0 2024-09-15T21:48:19.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.525+0000 7f15f0df6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec159980 con 0x7f15ec073b80 2024-09-15T21:48:19.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.525+0000 7f15f0df6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f15ec0740e0 con 0x7f15ec0c9a50 2024-09-15T21:48:19.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.525+0000 7f15e1d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec0c78c0 0x7f15ec1e5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:19.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.525+0000 7f15e1d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec0c78c0 0x7f15ec1e5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57264/0 (socket says 172.21.15.100:57264) 2024-09-15T21:48:19.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.525+0000 7f15e1d9b640 1 -- 172.21.15.100:0/1123459019 learned_addr learned my addr 172.21.15.100:0/1123459019 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:19.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.525+0000 7f15ea59c640 1 --2- 172.21.15.100:0/1123459019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15ec073b80 0x7f15ec0cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:19.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.525+0000 7f15ead9d640 1 --2- 172.21.15.100:0/1123459019 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f15ec0c9a50 0x7f15ec071700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:19.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.525+0000 7f15e1d9b640 1 -- 172.21.15.100:0/1123459019 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f15ec0c9a50 msgr2=0x7f15ec071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:19.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.525+0000 7f15e1d9b640 1 --2- 172.21.15.100:0/1123459019 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f15ec0c9a50 0x7f15ec071700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:19.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.525+0000 7f15e1d9b640 1 -- 172.21.15.100:0/1123459019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15ec073b80 msgr2=0x7f15ec0cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:19.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.525+0000 7f15e1d9b640 1 --2- 172.21.15.100:0/1123459019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15ec073b80 0x7f15ec0cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:19.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.525+0000 7f15e1d9b640 1 -- 172.21.15.100:0/1123459019 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ec14b940 con 0x7f15ec0c78c0 2024-09-15T21:48:19.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.525+0000 7f15e1d9b640 1 --2- 172.21.15.100:0/1123459019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec0c78c0 0x7f15ec1e5900 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f15d4035040 tx=0x7f15d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:19.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.526+0000 7f15e37fe640 1 -- 172.21.15.100:0/1123459019 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4037050 con 0x7f15ec0c78c0 2024-09-15T21:48:19.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.526+0000 7f15e37fe640 1 -- 172.21.15.100:0/1123459019 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15d4038040 con 0x7f15ec0c78c0 2024-09-15T21:48:19.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.526+0000 7f15e37fe640 1 -- 172.21.15.100:0/1123459019 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4004470 con 0x7f15ec0c78c0 2024-09-15T21:48:19.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.526+0000 7f15f0df6640 1 -- 172.21.15.100:0/1123459019 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15ec14bc20 con 0x7f15ec0c78c0 2024-09-15T21:48:19.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.526+0000 7f15f0df6640 1 -- 172.21.15.100:0/1123459019 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15ec14c1e0 con 0x7f15ec0c78c0 2024-09-15T21:48:19.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.527+0000 7f15f0df6640 1 -- 172.21.15.100:0/1123459019 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15ec0c7d50 con 0x7f15ec0c78c0 2024-09-15T21:48:19.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.527+0000 7f15e37fe640 1 -- 172.21.15.100:0/1123459019 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f15d403c6d0 con 0x7f15ec0c78c0 2024-09-15T21:48:19.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.528+0000 7f15e37fe640 1 --2- 172.21.15.100:0/1123459019 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f15cc06c2e0 0x7f15cc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:19.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.528+0000 7f15e37fe640 1 -- 172.21.15.100:0/1123459019 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5857+0+0 (secure 0 0 0) 0x7f15d40abe50 con 0x7f15ec0c78c0 2024-09-15T21:48:19.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.528+0000 7f15ea59c640 1 --2- 172.21.15.100:0/1123459019 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f15cc06c2e0 0x7f15cc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:19.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.528+0000 7f15ea59c640 1 --2- 172.21.15.100:0/1123459019 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f15cc06c2e0 0x7f15cc06e7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009ad0 tx=0x7f15dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:19.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.532+0000 7f15e37fe640 1 -- 172.21.15.100:0/1123459019 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f15d407a520 con 0x7f15ec0c78c0 2024-09-15T21:48:19.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:19.707+0000 7f15f0df6640 1 -- 172.21.15.100:0/1123459019 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd in", "ids": ["7"]} v 0) -- 0x7f15ec157820 con 0x7f15ec0c78c0 2024-09-15T21:48:19.709 INFO:tasks.ceph.mon.a.smithi100.stderr:2024-09-15T21:48:19.707+0000 7f81c6dea640 -1 mon.a@0(leader).osd e59 definitely_dead 0 2024-09-15T21:48:20.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.617+0000 7f15e37fe640 1 -- 172.21.15.100:0/1123459019 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd in", "ids": ["7"]}]=0 marked in osd.7. v60) ==== 85+0+0 (secure 0 0 0) 0x7f15d4079e20 con 0x7f15ec0c78c0 2024-09-15T21:48:20.619 INFO:teuthology.orchestra.run.smithi100.stderr:marked in osd.7. 2024-09-15T21:48:20.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.621+0000 7f15f0df6640 1 -- 172.21.15.100:0/1123459019 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f15cc06c2e0 msgr2=0x7f15cc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:20.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.621+0000 7f15f0df6640 1 --2- 172.21.15.100:0/1123459019 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f15cc06c2e0 0x7f15cc06e7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009ad0 tx=0x7f15dc008040 comp rx=0 tx=0).stop 2024-09-15T21:48:20.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.621+0000 7f15f0df6640 1 -- 172.21.15.100:0/1123459019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec0c78c0 msgr2=0x7f15ec1e5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:20.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.621+0000 7f15f0df6640 1 --2- 172.21.15.100:0/1123459019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec0c78c0 0x7f15ec1e5900 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f15d4035040 tx=0x7f15d40046c0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.621+0000 7f15f0df6640 1 -- 172.21.15.100:0/1123459019 shutdown_connections 2024-09-15T21:48:20.623 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.621+0000 7f15f0df6640 1 --2- 172.21.15.100:0/1123459019 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f15cc06c2e0 0x7f15cc06e7a0 secure :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009ad0 tx=0x7f15dc008040 comp rx=0 tx=0).stop 2024-09-15T21:48:20.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.621+0000 7f15f0df6640 1 --2- 172.21.15.100:0/1123459019 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f15ec0c9a50 0x7f15ec071700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.621+0000 7f15f0df6640 1 --2- 172.21.15.100:0/1123459019 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f15ec0c78c0 0x7f15ec1e5900 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.621+0000 7f15f0df6640 1 --2- 172.21.15.100:0/1123459019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15ec073b80 0x7f15ec0cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.621+0000 7f15f0df6640 1 -- 172.21.15.100:0/1123459019 >> 172.21.15.100:0/1123459019 conn(0x7f15ec06d890 msgr2=0x7f15ec157d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:20.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.621+0000 7f15f0df6640 1 -- 172.21.15.100:0/1123459019 shutdown_connections 2024-09-15T21:48:20.624 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.622+0000 7f15f0df6640 1 -- 172.21.15.100:0/1123459019 wait complete. 2024-09-15T21:48:20.643 INFO:tasks.thrashosds.thrasher:Added osd 7 2024-09-15T21:48:20.643 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-15T21:48:20.744 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.742+0000 7f9adb38d640 1 Processor -- start 2024-09-15T21:48:20.745 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.742+0000 7f9adb38d640 1 -- start start 2024-09-15T21:48:20.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.743+0000 7f9adb38d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad41551b0 0x7f9ad406bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:20.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.743+0000 7f9adb38d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9ad415ba70 0x7f9ad406c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:20.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.743+0000 7f9adb38d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9ad4070260 0x7f9ad415c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:20.748 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.743+0000 7f9adb38d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f9ad4072c30 con 0x7f9ad4070260 2024-09-15T21:48:20.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.743+0000 7f9adb38d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f9ad406f650 con 0x7f9ad415ba70 2024-09-15T21:48:20.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.743+0000 7f9adb38d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad415bf40 con 0x7f9ad41551b0 2024-09-15T21:48:20.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.743+0000 7f9ad992a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9ad4070260 0x7f9ad415c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:20.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.743+0000 7f9ad992a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f9ad4070260 0x7f9ad415c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57270/0 (socket says 172.21.15.100:57270) 2024-09-15T21:48:20.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.743+0000 7f9ad992a640 1 -- 172.21.15.100:0/137918987 learned_addr learned my addr 172.21.15.100:0/137918987 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:20.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.744+0000 7f9ad9129640 1 --2- 172.21.15.100:0/137918987 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad41551b0 0x7f9ad406bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:20.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.744+0000 7f9ad8928640 1 --2- 172.21.15.100:0/137918987 >> v2:172.21.15.122:3300/0 conn(0x7f9ad415ba70 0x7f9ad406c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:20.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.744+0000 7f9ad992a640 1 -- 172.21.15.100:0/137918987 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad41551b0 msgr2=0x7f9ad406bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:20.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.744+0000 7f9ad992a640 1 --2- 172.21.15.100:0/137918987 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad41551b0 0x7f9ad406bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.744+0000 7f9ad992a640 1 -- 172.21.15.100:0/137918987 >> v2:172.21.15.122:3300/0 conn(0x7f9ad415ba70 msgr2=0x7f9ad406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:20.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.744+0000 7f9ad992a640 1 --2- 172.21.15.100:0/137918987 >> v2:172.21.15.122:3300/0 conn(0x7f9ad415ba70 0x7f9ad406c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.744+0000 7f9ad992a640 1 -- 172.21.15.100:0/137918987 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad4072020 con 0x7f9ad4070260 2024-09-15T21:48:20.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.744+0000 7f9ad9129640 1 --2- 172.21.15.100:0/137918987 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad41551b0 0x7f9ad406bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:48:20.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.744+0000 7f9ad8928640 1 --2- 172.21.15.100:0/137918987 >> v2:172.21.15.122:3300/0 conn(0x7f9ad415ba70 0x7f9ad406c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:48:20.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.744+0000 7f9ad992a640 1 --2- 172.21.15.100:0/137918987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad4070260 0x7f9ad415c4e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4073600 tx=0x7f9ad002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffe56eea603399f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:20.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.745+0000 7f9acbfff640 1 -- 172.21.15.100:0/137918987 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0037070 con 0x7f9ad4070260 2024-09-15T21:48:20.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.745+0000 7f9acbfff640 1 -- 172.21.15.100:0/137918987 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad0032440 con 0x7f9ad4070260 2024-09-15T21:48:20.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.745+0000 7f9acbfff640 1 -- 172.21.15.100:0/137918987 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0032810 con 0x7f9ad4070260 2024-09-15T21:48:20.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.745+0000 7f9adb38d640 1 -- 172.21.15.100:0/137918987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad4070260 msgr2=0x7f9ad415c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:20.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.745+0000 7f9adb38d640 1 --2- 172.21.15.100:0/137918987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad4070260 0x7f9ad415c4e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4073600 tx=0x7f9ad002a270 comp rx=0 tx=0).stop 2024-09-15T21:48:20.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.745+0000 7f9adb38d640 1 -- 172.21.15.100:0/137918987 shutdown_connections 2024-09-15T21:48:20.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.745+0000 7f9adb38d640 1 --2- 172.21.15.100:0/137918987 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad4070260 0x7f9ad415c4e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.745+0000 7f9adb38d640 1 --2- 172.21.15.100:0/137918987 >> v2:172.21.15.122:3300/0 conn(0x7f9ad415ba70 0x7f9ad406c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.745+0000 7f9adb38d640 1 --2- 172.21.15.100:0/137918987 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad41551b0 0x7f9ad406bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.745+0000 7f9adb38d640 1 -- 172.21.15.100:0/137918987 >> 172.21.15.100:0/137918987 conn(0x7f9ad406ccf0 msgr2=0x7f9ad406d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:20.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.745+0000 7f9adb38d640 1 -- 172.21.15.100:0/137918987 shutdown_connections 2024-09-15T21:48:20.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.745+0000 7f9adb38d640 1 -- 172.21.15.100:0/137918987 wait complete. 2024-09-15T21:48:20.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.746+0000 7f9adb38d640 1 Processor -- start 2024-09-15T21:48:20.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.746+0000 7f9adb38d640 1 -- start start 2024-09-15T21:48:20.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.747+0000 7f9adb38d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad4070260 0x7f9ad4151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:20.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.747+0000 7f9adb38d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad41551b0 0x7f9ad41f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:20.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.747+0000 7f9adb38d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad415ba70 0x7f9ad414fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:20.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.747+0000 7f9adb38d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad40728c0 con 0x7f9ad415ba70 2024-09-15T21:48:20.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.747+0000 7f9adb38d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad40725e0 con 0x7f9ad41551b0 2024-09-15T21:48:20.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.747+0000 7f9adb38d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad40c8160 con 0x7f9ad4070260 2024-09-15T21:48:20.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.747+0000 7f9ad992a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad415ba70 0x7f9ad414fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:20.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.747+0000 7f9ad992a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad415ba70 0x7f9ad414fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57280/0 (socket says 172.21.15.100:57280) 2024-09-15T21:48:20.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.747+0000 7f9ad992a640 1 -- 172.21.15.100:0/779541947 learned_addr learned my addr 172.21.15.100:0/779541947 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:20.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.747+0000 7f9ad8928640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad41551b0 0x7f9ad41f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:20.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.747+0000 7f9ad9129640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad4070260 0x7f9ad4151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:20.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.747+0000 7f9ad992a640 1 -- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad4070260 msgr2=0x7f9ad4151ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:20.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.748+0000 7f9ad992a640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad4070260 0x7f9ad4151ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.748+0000 7f9ad992a640 1 -- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad41551b0 msgr2=0x7f9ad41f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:20.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.748+0000 7f9ad992a640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad41551b0 0x7f9ad41f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.748+0000 7f9ad992a640 1 -- 172.21.15.100:0/779541947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ad41582e0 con 0x7f9ad415ba70 2024-09-15T21:48:20.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.748+0000 7f9ad992a640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad415ba70 0x7f9ad414fb90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00090b0 tx=0x7f9ad00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:20.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.748+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0037040 con 0x7f9ad415ba70 2024-09-15T21:48:20.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.748+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad0042070 con 0x7f9ad415ba70 2024-09-15T21:48:20.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.748+0000 7f9adb38d640 1 -- 172.21.15.100:0/779541947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ad41f4350 con 0x7f9ad415ba70 2024-09-15T21:48:20.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.748+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0030bd0 con 0x7f9ad415ba70 2024-09-15T21:48:20.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.748+0000 7f9adb38d640 1 -- 172.21.15.100:0/779541947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ad4158880 con 0x7f9ad415ba70 2024-09-15T21:48:20.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.749+0000 7f9adb38d640 1 -- 172.21.15.100:0/779541947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f9ad406bbe0 con 0x7f9ad415ba70 2024-09-15T21:48:20.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.750+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9ad0047460 con 0x7f9ad415ba70 2024-09-15T21:48:20.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.750+0000 7f9ac9ffb640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9ab806c2b0 0x7f9ab806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:20.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.750+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5857+0+0 (secure 0 0 0) 0x7f9ad00abf80 con 0x7f9ad415ba70 2024-09-15T21:48:20.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.750+0000 7f9ad9129640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9ab806c2b0 0x7f9ab806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:20.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.750+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+1289 (secure 0 0 0) 0x7f9ad00ae660 con 0x7f9ad415ba70 2024-09-15T21:48:20.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.750+0000 7f9adb38d640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad41570a0 0x7f9ad41f4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:20.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.750+0000 7f9adb38d640 1 -- 172.21.15.100:0/779541947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad40c8b90 con 0x7f9ad41570a0 2024-09-15T21:48:20.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.750+0000 7f9ad9129640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9ab806c2b0 0x7f9ab806e770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9acc007130 tx=0x7f9acc0077c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:20.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.751+0000 7f9ad8928640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad41570a0 0x7f9ad41f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:20.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.751+0000 7f9ad8928640 1 -- 172.21.15.100:0/779541947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- command(tid 2: {"prefix": "get_command_descriptions"}) -- 0x7f9ad4070890 con 0x7f9ad41570a0 2024-09-15T21:48:20.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.751+0000 7f9ad8928640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad41570a0 0x7f9ad41f4570 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9ad415a8f0 tx=0x7f9ac400fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:20.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.751+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400dbf0 con 0x7f9ad41570a0 2024-09-15T21:48:20.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.751+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 <== mon.0 v2:172.21.15.100:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9ac400dbf0 con 0x7f9ad41570a0 2024-09-15T21:48:20.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.751+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad41570a0 msgr2=0x7f9ad41f4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:20.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.751+0000 7f9ac9ffb640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad41570a0 0x7f9ad41f4570 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9ad415a8f0 tx=0x7f9ac400fc70 comp rx=0 tx=0).stop 2024-09-15T21:48:20.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.751+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad41570a0 msgr2=0x7f9ad41f4570 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:48:20.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.751+0000 7f9ac9ffb640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad41570a0 0x7f9ad41f4570 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.757+0000 7f9adb38d640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad4159ed0 0x7f9ad414a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:20.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.757+0000 7f9adb38d640 1 -- 172.21.15.100:0/779541947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad4158080 con 0x7f9ad4159ed0 2024-09-15T21:48:20.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.757+0000 7f9ad8928640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad4159ed0 0x7f9ad414a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:20.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.758+0000 7f9ad8928640 1 -- 172.21.15.100:0/779541947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- command(tid 3: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) -- 0x7f9ad4152770 con 0x7f9ad4159ed0 2024-09-15T21:48:20.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.758+0000 7f9ad8928640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad4159ed0 0x7f9ad414a040 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9ac401a000 tx=0x7f9ac40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:20.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.758+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac40065c0 con 0x7f9ad4159ed0 2024-09-15T21:48:20.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.759+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 <== mon.0 v2:172.21.15.100:3300/0 2 ==== command_reply(tid 3: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f9ac4006680 con 0x7f9ad4159ed0 2024-09-15T21:48:20.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.759+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad4159ed0 msgr2=0x7f9ad414a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:20.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.759+0000 7f9ac9ffb640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad4159ed0 0x7f9ad414a040 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9ac401a000 tx=0x7f9ac40046c0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.759+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad4159ed0 msgr2=0x7f9ad414a040 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:48:20.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.759+0000 7f9ac9ffb640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad4159ed0 0x7f9ad414a040 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.762 INFO:teuthology.orchestra.run.smithi100.stdout:mon.a: {} 2024-09-15T21:48:20.764 INFO:teuthology.orchestra.run.smithi100.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T21:48:20.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.759+0000 7f9adb38d640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad415a2b0 0x7f9ad414a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:20.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.759+0000 7f9adb38d640 1 -- 172.21.15.100:0/779541947 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad41548b0 con 0x7f9ad415a2b0 2024-09-15T21:48:20.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.760+0000 7f9ad8928640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad415a2b0 0x7f9ad414a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:20.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.760+0000 7f9ad8928640 1 -- 172.21.15.100:0/779541947 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- command(tid 4: {"prefix": "get_command_descriptions"}) -- 0x7f9ad414aac0 con 0x7f9ad415a2b0 2024-09-15T21:48:20.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.760+0000 7f9ad8928640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad415a2b0 0x7f9ad414a580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4158250 tx=0x7f9ac400fc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:20.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.760+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4036070 con 0x7f9ad415a2b0 2024-09-15T21:48:20.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.761+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 <== mon.1 v2:172.21.15.122:3300/0 2 ==== command_reply(tid 4: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9ac403b450 con 0x7f9ad415a2b0 2024-09-15T21:48:20.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.761+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad415a2b0 msgr2=0x7f9ad414a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:20.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.761+0000 7f9ac9ffb640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad415a2b0 0x7f9ad414a580 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4158250 tx=0x7f9ac400fc70 comp rx=0 tx=0).stop 2024-09-15T21:48:20.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.761+0000 7f9ad992a640 1 -- 172.21.15.100:0/779541947 reap_dead start 2024-09-15T21:48:20.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.761+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad415a2b0 msgr2=0x7f9ad414a580 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:48:20.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.761+0000 7f9ac9ffb640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad415a2b0 0x7f9ad414a580 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.767+0000 7f9adb38d640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad41f4e10 0x7f9ad4151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:20.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.767+0000 7f9adb38d640 1 -- 172.21.15.100:0/779541947 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ad40008d0 con 0x7f9ad41f4e10 2024-09-15T21:48:20.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.767+0000 7f9ad8928640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad41f4e10 0x7f9ad4151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:20.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.767+0000 7f9ad8928640 1 -- 172.21.15.100:0/779541947 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- command(tid 5: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) -- 0x7f9ad415af80 con 0x7f9ad41f4e10 2024-09-15T21:48:20.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.767+0000 7f9ad8928640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad41f4e10 0x7f9ad4151ac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4000c00 tx=0x7f9ac402a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:20.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.767+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac404c0a0 con 0x7f9ad41f4e10 2024-09-15T21:48:20.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.769+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 <== mon.1 v2:172.21.15.122:3300/0 2 ==== command_reply(tid 5: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f9ac4020570 con 0x7f9ad41f4e10 2024-09-15T21:48:20.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.769+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad41f4e10 msgr2=0x7f9ad4151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:20.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.769+0000 7f9ac9ffb640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad41f4e10 0x7f9ad4151ac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4000c00 tx=0x7f9ac402a000 comp rx=0 tx=0).stop 2024-09-15T21:48:20.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.769+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad41f4e10 msgr2=0x7f9ad4151ac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:48:20.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.769+0000 7f9ac9ffb640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ad41f4e10 0x7f9ad4151ac0 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4000c00 tx=0x7f9ac402a000 comp rx=0 tx=0).stop 2024-09-15T21:48:20.772 INFO:teuthology.orchestra.run.smithi100.stdout:mon.b: {} 2024-09-15T21:48:20.774 INFO:teuthology.orchestra.run.smithi100.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T21:48:20.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.769+0000 7f9adb38d640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad41558e0 0x7f9ad4155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:20.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.769+0000 7f9adb38d640 1 -- 172.21.15.100:0/779541947 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad4156200 con 0x7f9ad41558e0 2024-09-15T21:48:20.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.769+0000 7f9ad8928640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad41558e0 0x7f9ad4155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:20.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.770+0000 7f9ad8928640 1 -- 172.21.15.100:0/779541947 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- command(tid 6: {"prefix": "get_command_descriptions"}) -- 0x7f9ad41563b0 con 0x7f9ad41558e0 2024-09-15T21:48:20.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.770+0000 7f9ad8928640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad41558e0 0x7f9ad4155cc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400ac30 tx=0x7f9ac400fc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:20.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.770+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac403ce60 con 0x7f9ad41558e0 2024-09-15T21:48:20.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.770+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 <== mon.2 v2:172.21.15.122:3301/0 2 ==== command_reply(tid 6: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9ac4053470 con 0x7f9ad41558e0 2024-09-15T21:48:20.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.770+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad41558e0 msgr2=0x7f9ad4155cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:20.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.770+0000 7f9ac9ffb640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad41558e0 0x7f9ad4155cc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9ac400ac30 tx=0x7f9ac400fc70 comp rx=0 tx=0).stop 2024-09-15T21:48:20.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.770+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad41558e0 msgr2=0x7f9ad4155cc0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:48:20.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.770+0000 7f9ac9ffb640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad41558e0 0x7f9ad4155cc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.776+0000 7f9adb38d640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad415ccb0 0x7f9ad415d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:20.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.776+0000 7f9adb38d640 1 -- 172.21.15.100:0/779541947 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9ad41565c0 con 0x7f9ad415ccb0 2024-09-15T21:48:20.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.776+0000 7f9ad8928640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad415ccb0 0x7f9ad415d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:20.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.777+0000 7f9ad8928640 1 -- 172.21.15.100:0/779541947 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- command(tid 7: {"prefix": "injectargs", "injected_args": ["--mon_osd_down_out_interval=600"]}) -- 0x7f9ad415d5d0 con 0x7f9ad415ccb0 2024-09-15T21:48:20.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.777+0000 7f9ad8928640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad415ccb0 0x7f9ad415d090 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4000c00 tx=0x7f9ac404a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:20.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.777+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ac4053af0 con 0x7f9ad415ccb0 2024-09-15T21:48:20.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.778+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 <== mon.2 v2:172.21.15.122:3301/0 2 ==== command_reply(tid 7: 0 mon_osd_down_out_interval = '' (not observed, change may require restart) ) ==== 82+0+3 (secure 0 0 0) 0x7f9ac40539b0 con 0x7f9ad415ccb0 2024-09-15T21:48:20.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.778+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad415ccb0 msgr2=0x7f9ad415d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:20.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.778+0000 7f9ac9ffb640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad415ccb0 0x7f9ad415d090 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9ac4000c00 tx=0x7f9ac404a000 comp rx=0 tx=0).stop 2024-09-15T21:48:20.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.778+0000 7f9ac9ffb640 1 -- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad415ccb0 msgr2=0x7f9ad415d090 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-15T21:48:20.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.778+0000 7f9ac9ffb640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9ad415ccb0 0x7f9ad415d090 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.781 INFO:teuthology.orchestra.run.smithi100.stdout:mon.c: {} 2024-09-15T21:48:20.782 INFO:teuthology.orchestra.run.smithi100.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T21:48:20.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.779+0000 7f9adb38d640 1 -- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9ab806c2b0 msgr2=0x7f9ab806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:20.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.779+0000 7f9adb38d640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f9ab806c2b0 0x7f9ab806e770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9acc007130 tx=0x7f9acc0077c0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.779+0000 7f9adb38d640 1 -- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad415ba70 msgr2=0x7f9ad414fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:20.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.779+0000 7f9adb38d640 1 --2- 172.21.15.100:0/779541947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9ad415ba70 0x7f9ad414fb90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9ad00090b0 tx=0x7f9ad00046c0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.779+0000 7f9ad992a640 1 -- 172.21.15.100:0/779541947 reap_dead start 2024-09-15T21:48:20.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.779+0000 7f9adb38d640 1 -- 172.21.15.100:0/779541947 shutdown_connections 2024-09-15T21:48:20.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.779+0000 7f9adb38d640 1 -- 172.21.15.100:0/779541947 >> 172.21.15.100:0/779541947 conn(0x7f9ad406ccf0 msgr2=0x7f9ad4153de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:20.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.779+0000 7f9adb38d640 1 -- 172.21.15.100:0/779541947 shutdown_connections 2024-09-15T21:48:20.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.779+0000 7f9adb38d640 1 -- 172.21.15.100:0/779541947 wait complete. 2024-09-15T21:48:20.791 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-15T21:48:20.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.890+0000 7f0fe0be8640 1 Processor -- start 2024-09-15T21:48:20.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.890+0000 7f0fe0be8640 1 -- start start 2024-09-15T21:48:20.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.891+0000 7f0fe0be8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0fdc0c58a0 0x7f0fdc152a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:20.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.891+0000 7f0fe0be8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fdc0c83e0 con 0x7f0fdc0ceb40 2024-09-15T21:48:20.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.891+0000 7f0fe0be8640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fdc0c86c0 con 0x7f0fdc0cef20 2024-09-15T21:48:20.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.891+0000 7f0fe0be8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f0fdc0c60e0 con 0x7f0fdc0c58a0 2024-09-15T21:48:20.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.891+0000 7f0fdad9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0fdc0c58a0 0x7f0fdc152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:20.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.891+0000 7f0fda59c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f0fdc0ceb40 0x7f0fdc0c6570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:49304/0 (socket says 172.21.15.100:49304) 2024-09-15T21:48:20.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.891+0000 7f0fda59c640 1 -- 172.21.15.100:0/220496560 learned_addr learned my addr 172.21.15.100:0/220496560 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:20.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.892+0000 7f0fdad9d640 1 -- 172.21.15.100:0/220496560 >> v1:172.21.15.122:6789/0 conn(0x7f0fdc0cef20 legacy=0x7f0fdc0c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:20.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.892+0000 7f0fdad9d640 1 -- 172.21.15.100:0/220496560 >> v1:172.21.15.100:6789/0 conn(0x7f0fdc0ceb40 legacy=0x7f0fdc0c6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:20.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.892+0000 7f0fdad9d640 1 -- 172.21.15.100:0/220496560 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fdc0c89a0 con 0x7f0fdc0c58a0 2024-09-15T21:48:20.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.892+0000 7f0fd959a640 1 -- 172.21.15.100:0/220496560 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 904550354 0 0) 0x7f0fdc0c83e0 con 0x7f0fdc0ceb40 2024-09-15T21:48:20.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.892+0000 7f0fdad9d640 1 --2- 172.21.15.100:0/220496560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fdc0c58a0 0x7f0fdc152a50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0fd40097a0 tx=0x7f0fd402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9971e46b22d17284 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:20.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.892+0000 7f0fd959a640 1 -- 172.21.15.100:0/220496560 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4037070 con 0x7f0fdc0c58a0 2024-09-15T21:48:20.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.892+0000 7f0fd959a640 1 -- 172.21.15.100:0/220496560 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd4032440 con 0x7f0fdc0c58a0 2024-09-15T21:48:20.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.892+0000 7f0fd959a640 1 -- 172.21.15.100:0/220496560 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd4032810 con 0x7f0fdc0c58a0 2024-09-15T21:48:20.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.893+0000 7f0fe0be8640 1 -- 172.21.15.100:0/220496560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fdc0c58a0 msgr2=0x7f0fdc152a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:20.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.893+0000 7f0fe0be8640 1 --2- 172.21.15.100:0/220496560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fdc0c58a0 0x7f0fdc152a50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0fd40097a0 tx=0x7f0fd402a270 comp rx=0 tx=0).stop 2024-09-15T21:48:20.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.893+0000 7f0fe0be8640 1 -- 172.21.15.100:0/220496560 shutdown_connections 2024-09-15T21:48:20.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.893+0000 7f0fe0be8640 1 --2- 172.21.15.100:0/220496560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fdc0c58a0 0x7f0fdc152a50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.893+0000 7f0fe0be8640 1 -- 172.21.15.100:0/220496560 >> 172.21.15.100:0/220496560 conn(0x7f0fdc067980 msgr2=0x7f0fdc067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:20.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.893+0000 7f0fe0be8640 1 -- 172.21.15.100:0/220496560 shutdown_connections 2024-09-15T21:48:20.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.893+0000 7f0fe0be8640 1 -- 172.21.15.100:0/220496560 wait complete. 2024-09-15T21:48:20.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.894+0000 7f0fe0be8640 1 Processor -- start 2024-09-15T21:48:20.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.894+0000 7f0fe0be8640 1 -- start start 2024-09-15T21:48:20.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.895+0000 7f0fe0be8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0fdc0c58a0 0x7f0fdc1f1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:20.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.895+0000 7f0fe0be8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fdc0ceb40 0x7f0fdc0cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:20.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.895+0000 7f0fe0be8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fdc0cef20 0x7f0fdc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:20.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.895+0000 7f0fe0be8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0fdc0c9d00 con 0x7f0fdc0ceb40 2024-09-15T21:48:20.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.895+0000 7f0fe0be8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0fdc0c6c80 con 0x7f0fdc0cef20 2024-09-15T21:48:20.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.895+0000 7f0fe0be8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0fdc158d40 con 0x7f0fdc0c58a0 2024-09-15T21:48:20.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.895+0000 7f0fd9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fdc0ceb40 0x7f0fdc0cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:20.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.895+0000 7f0fd9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fdc0ceb40 0x7f0fdc0cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57324/0 (socket says 172.21.15.100:57324) 2024-09-15T21:48:20.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.895+0000 7f0fd9d9b640 1 -- 172.21.15.100:0/1611656060 learned_addr learned my addr 172.21.15.100:0/1611656060 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:20.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.895+0000 7f0fdad9d640 1 --2- 172.21.15.100:0/1611656060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fdc0cef20 0x7f0fdc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:20.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.895+0000 7f0fda59c640 1 --2- 172.21.15.100:0/1611656060 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0fdc0c58a0 0x7f0fdc1f1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:20.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.895+0000 7f0fd9d9b640 1 -- 172.21.15.100:0/1611656060 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0fdc0c58a0 msgr2=0x7f0fdc1f1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:20.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.895+0000 7f0fd9d9b640 1 --2- 172.21.15.100:0/1611656060 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0fdc0c58a0 0x7f0fdc1f1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.895+0000 7f0fd9d9b640 1 -- 172.21.15.100:0/1611656060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fdc0cef20 msgr2=0x7f0fdc06d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:20.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.895+0000 7f0fd9d9b640 1 --2- 172.21.15.100:0/1611656060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fdc0cef20 0x7f0fdc06d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:20.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.895+0000 7f0fd9d9b640 1 -- 172.21.15.100:0/1611656060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fdc06ce90 con 0x7f0fdc0ceb40 2024-09-15T21:48:20.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.896+0000 7f0fd9d9b640 1 --2- 172.21.15.100:0/1611656060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fdc0ceb40 0x7f0fdc0cb850 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000b3f0 tx=0x7f0fd000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:20.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.896+0000 7f0fcaffd640 1 -- 172.21.15.100:0/1611656060 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0015020 con 0x7f0fdc0ceb40 2024-09-15T21:48:20.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.896+0000 7f0fcaffd640 1 -- 172.21.15.100:0/1611656060 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fd0011830 con 0x7f0fdc0ceb40 2024-09-15T21:48:20.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.896+0000 7f0fe0be8640 1 -- 172.21.15.100:0/1611656060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fdc158050 con 0x7f0fdc0ceb40 2024-09-15T21:48:20.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.896+0000 7f0fcaffd640 1 -- 172.21.15.100:0/1611656060 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0011c20 con 0x7f0fdc0ceb40 2024-09-15T21:48:20.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.896+0000 7f0fe0be8640 1 -- 172.21.15.100:0/1611656060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fdc06d130 con 0x7f0fdc0ceb40 2024-09-15T21:48:20.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.897+0000 7f0fe0be8640 1 -- 172.21.15.100:0/1611656060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fdc1512d0 con 0x7f0fdc0ceb40 2024-09-15T21:48:20.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.897+0000 7f0fcaffd640 1 -- 172.21.15.100:0/1611656060 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0fd0018020 con 0x7f0fdc0ceb40 2024-09-15T21:48:20.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.898+0000 7f0fcaffd640 1 --2- 172.21.15.100:0/1611656060 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0fbc06c2b0 0x7f0fbc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:20.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.898+0000 7f0fcaffd640 1 -- 172.21.15.100:0/1611656060 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5857+0+0 (secure 0 0 0) 0x7f0fd008af80 con 0x7f0fdc0ceb40 2024-09-15T21:48:20.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.898+0000 7f0fda59c640 1 --2- 172.21.15.100:0/1611656060 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0fbc06c2b0 0x7f0fbc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:20.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.898+0000 7f0fda59c640 1 --2- 172.21.15.100:0/1611656060 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0fbc06c2b0 0x7f0fbc06e770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc0012e0 tx=0x7f0fcc001700 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:20.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:20.902+0000 7f0fcaffd640 1 -- 172.21.15.100:0/1611656060 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0fd001f030 con 0x7f0fdc0ceb40 2024-09-15T21:48:21.078 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.075+0000 7f0fe0be8640 1 -- 172.21.15.100:0/1611656060 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd reweight", "id": 0, "weight": 1.0} v 0) -- 0x7f0fdc152d10 con 0x7f0fdc0ceb40 2024-09-15T21:48:21.625 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.623+0000 7f0fcaffd640 1 -- 172.21.15.100:0/1611656060 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 0, "weight": 1.0}]=0 reweighted osd.0 to 1 (10000) v61) ==== 113+0+0 (secure 0 0 0) 0x7f0fd0059000 con 0x7f0fdc0ceb40 2024-09-15T21:48:21.625 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.0 to 1 (10000) 2024-09-15T21:48:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.625+0000 7f0fe0be8640 1 -- 172.21.15.100:0/1611656060 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0fbc06c2b0 msgr2=0x7f0fbc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.625+0000 7f0fe0be8640 1 --2- 172.21.15.100:0/1611656060 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0fbc06c2b0 0x7f0fbc06e770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0fcc0012e0 tx=0x7f0fcc001700 comp rx=0 tx=0).stop 2024-09-15T21:48:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.625+0000 7f0fe0be8640 1 -- 172.21.15.100:0/1611656060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fdc0ceb40 msgr2=0x7f0fdc0cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.625+0000 7f0fe0be8640 1 --2- 172.21.15.100:0/1611656060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fdc0ceb40 0x7f0fdc0cb850 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000b3f0 tx=0x7f0fd000b8b0 comp rx=0 tx=0).stop 2024-09-15T21:48:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.626+0000 7f0fe0be8640 1 -- 172.21.15.100:0/1611656060 shutdown_connections 2024-09-15T21:48:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.626+0000 7f0fe0be8640 1 --2- 172.21.15.100:0/1611656060 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0fbc06c2b0 0x7f0fbc06e770 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.626+0000 7f0fe0be8640 1 --2- 172.21.15.100:0/1611656060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fdc0cef20 0x7f0fdc06d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.626+0000 7f0fe0be8640 1 --2- 172.21.15.100:0/1611656060 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0fdc0ceb40 0x7f0fdc0cb850 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.626+0000 7f0fe0be8640 1 --2- 172.21.15.100:0/1611656060 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0fdc0c58a0 0x7f0fdc1f1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:21.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.626+0000 7f0fe0be8640 1 -- 172.21.15.100:0/1611656060 >> 172.21.15.100:0/1611656060 conn(0x7f0fdc067980 msgr2=0x7f0fdc151ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:21.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.626+0000 7f0fe0be8640 1 -- 172.21.15.100:0/1611656060 shutdown_connections 2024-09-15T21:48:21.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.626+0000 7f0fe0be8640 1 -- 172.21.15.100:0/1611656060 wait complete. 2024-09-15T21:48:21.645 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-15T21:48:21.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.747+0000 7f71b5e9b640 1 Processor -- start 2024-09-15T21:48:21.749 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.747+0000 7f71b5e9b640 1 -- start start 2024-09-15T21:48:21.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.748+0000 7f71b5e9b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f71b01551f0 0x7f71b00cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:21.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.748+0000 7f71b5e9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f71b00c4950 0x7f71b00cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:21.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.748+0000 7f71b5e9b640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71b015b250 con 0x7f71b015ea80 2024-09-15T21:48:21.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.748+0000 7f71b5e9b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f71b01555d0 con 0x7f71b00c4950 2024-09-15T21:48:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.748+0000 7f71b5e9b640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f71b015b530 con 0x7f71b01551f0 2024-09-15T21:48:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.748+0000 7f71aeffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f71b00c4950 0x7f71b00cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.748+0000 7f71aeffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f71b00c4950 0x7f71b00cbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57338/0 (socket says 172.21.15.100:57338) 2024-09-15T21:48:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.748+0000 7f71aeffd640 1 -- 172.21.15.100:0/3637127412 learned_addr learned my addr 172.21.15.100:0/3637127412 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.748+0000 7f71af7fe640 1 --2- 172.21.15.100:0/3637127412 >> v2:172.21.15.122:3300/0 conn(0x7f71b01551f0 0x7f71b00cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.748+0000 7f71aeffd640 1 -- 172.21.15.100:0/3637127412 >> v2:172.21.15.122:3300/0 conn(0x7f71b01551f0 msgr2=0x7f71b00cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.748+0000 7f71aeffd640 1 --2- 172.21.15.100:0/3637127412 >> v2:172.21.15.122:3300/0 conn(0x7f71b01551f0 0x7f71b00cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.748+0000 7f71aeffd640 1 -- 172.21.15.100:0/3637127412 >> v1:172.21.15.122:6789/0 conn(0x7f71b015ea80 legacy=0x7f71b015ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:21.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.748+0000 7f71aeffd640 1 -- 172.21.15.100:0/3637127412 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b015b810 con 0x7f71b00c4950 2024-09-15T21:48:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.749+0000 7f71aeffd640 1 --2- 172.21.15.100:0/3637127412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b00c4950 0x7f71b00cbb20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f71b0155970 tx=0x7f71a402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18dd64847aa824f server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:21.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.749+0000 7f71ae7fc640 1 -- 172.21.15.100:0/3637127412 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4004230 con 0x7f71b00c4950 2024-09-15T21:48:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.749+0000 7f71ae7fc640 1 -- 172.21.15.100:0/3637127412 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71a402ae50 con 0x7f71b00c4950 2024-09-15T21:48:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.749+0000 7f71ae7fc640 1 -- 172.21.15.100:0/3637127412 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4003610 con 0x7f71b00c4950 2024-09-15T21:48:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.750+0000 7f71b5e9b640 1 -- 172.21.15.100:0/3637127412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b00c4950 msgr2=0x7f71b00cbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.750+0000 7f71b5e9b640 1 --2- 172.21.15.100:0/3637127412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b00c4950 0x7f71b00cbb20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f71b0155970 tx=0x7f71a402a300 comp rx=0 tx=0).stop 2024-09-15T21:48:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.750+0000 7f71b5e9b640 1 -- 172.21.15.100:0/3637127412 shutdown_connections 2024-09-15T21:48:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.750+0000 7f71b5e9b640 1 --2- 172.21.15.100:0/3637127412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b00c4950 0x7f71b00cbb20 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.750+0000 7f71b5e9b640 1 --2- 172.21.15.100:0/3637127412 >> v2:172.21.15.122:3300/0 conn(0x7f71b01551f0 0x7f71b00cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.750+0000 7f71b5e9b640 1 -- 172.21.15.100:0/3637127412 >> 172.21.15.100:0/3637127412 conn(0x7f71b0067980 msgr2=0x7f71b0067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.750+0000 7f71b5e9b640 1 -- 172.21.15.100:0/3637127412 shutdown_connections 2024-09-15T21:48:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.750+0000 7f71b5e9b640 1 -- 172.21.15.100:0/3637127412 wait complete. 2024-09-15T21:48:21.753 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.751+0000 7f71b5e9b640 1 Processor -- start 2024-09-15T21:48:21.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.751+0000 7f71b5e9b640 1 -- start start 2024-09-15T21:48:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.752+0000 7f71b5e9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71b00c4950 0x7f71b00ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.752+0000 7f71b5e9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b01551f0 0x7f71b01f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.752+0000 7f71b5e9b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71b015ea80 0x7f71b01f13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.752+0000 7f71b5e9b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f71b0071070 con 0x7f71b01551f0 2024-09-15T21:48:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.752+0000 7f71b5e9b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f71b015bdd0 con 0x7f71b00c4950 2024-09-15T21:48:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.752+0000 7f71b5e9b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f71b00711f0 con 0x7f71b015ea80 2024-09-15T21:48:21.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.752+0000 7f71aeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b01551f0 0x7f71b01f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:21.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.752+0000 7f71aeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b01551f0 0x7f71b01f0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57352/0 (socket says 172.21.15.100:57352) 2024-09-15T21:48:21.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.752+0000 7f71aeffd640 1 -- 172.21.15.100:0/2459243977 learned_addr learned my addr 172.21.15.100:0/2459243977 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:21.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.752+0000 7f71af7fe640 1 --2- 172.21.15.100:0/2459243977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71b00c4950 0x7f71b00ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:21.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.752+0000 7f71affff640 1 --2- 172.21.15.100:0/2459243977 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71b015ea80 0x7f71b01f13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:21.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.752+0000 7f71aeffd640 1 -- 172.21.15.100:0/2459243977 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71b015ea80 msgr2=0x7f71b01f13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:21.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.752+0000 7f71aeffd640 1 --2- 172.21.15.100:0/2459243977 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71b015ea80 0x7f71b01f13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:21.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.752+0000 7f71aeffd640 1 -- 172.21.15.100:0/2459243977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71b00c4950 msgr2=0x7f71b00ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:21.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.752+0000 7f71aeffd640 1 --2- 172.21.15.100:0/2459243977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71b00c4950 0x7f71b00ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:21.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.752+0000 7f71aeffd640 1 -- 172.21.15.100:0/2459243977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71b015c760 con 0x7f71b01551f0 2024-09-15T21:48:21.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.752+0000 7f71af7fe640 1 --2- 172.21.15.100:0/2459243977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71b00c4950 0x7f71b00ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:48:21.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.753+0000 7f71aeffd640 1 --2- 172.21.15.100:0/2459243977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b01551f0 0x7f71b01f0e60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f71a402a340 tx=0x7f71a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:21.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.753+0000 7f7197fff640 1 -- 172.21.15.100:0/2459243977 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4042020 con 0x7f71b01551f0 2024-09-15T21:48:21.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.753+0000 7f7197fff640 1 -- 172.21.15.100:0/2459243977 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71a4003a20 con 0x7f71b01551f0 2024-09-15T21:48:21.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.753+0000 7f71b5e9b640 1 -- 172.21.15.100:0/2459243977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71b00c5f50 con 0x7f71b01551f0 2024-09-15T21:48:21.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.753+0000 7f7197fff640 1 -- 172.21.15.100:0/2459243977 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4003e10 con 0x7f71b01551f0 2024-09-15T21:48:21.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.753+0000 7f71b5e9b640 1 -- 172.21.15.100:0/2459243977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71b00c6510 con 0x7f71b01551f0 2024-09-15T21:48:21.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.754+0000 7f71b5e9b640 1 -- 172.21.15.100:0/2459243977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71b015ef10 con 0x7f71b01551f0 2024-09-15T21:48:21.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.754+0000 7f7197fff640 1 -- 172.21.15.100:0/2459243977 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f71a4045020 con 0x7f71b01551f0 2024-09-15T21:48:21.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.755+0000 7f7197fff640 1 --2- 172.21.15.100:0/2459243977 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f719c06c2e0 0x7f719c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:21.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.755+0000 7f7197fff640 1 -- 172.21.15.100:0/2459243977 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5882+0+0 (secure 0 0 0) 0x7f71a40ac000 con 0x7f71b01551f0 2024-09-15T21:48:21.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.755+0000 7f71af7fe640 1 --2- 172.21.15.100:0/2459243977 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f719c06c2e0 0x7f719c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:21.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.755+0000 7f71af7fe640 1 --2- 172.21.15.100:0/2459243977 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f719c06c2e0 0x7f719c06e7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f71a0006fd0 tx=0x7f71a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:21.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.759+0000 7f7197fff640 1 -- 172.21.15.100:0/2459243977 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f71a404c030 con 0x7f71b01551f0 2024-09-15T21:48:21.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:21.940+0000 7f71b5e9b640 1 -- 172.21.15.100:0/2459243977 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 0, "weight": 1.0} v 0) -- 0x7f71b0153910 con 0x7f71b01551f0 2024-09-15T21:48:22.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.628+0000 7f7197fff640 1 -- 172.21.15.100:0/2459243977 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]=0 set osd.0 primary-affinity to 1 (8655362) v62) ==== 133+0+0 (secure 0 0 0) 0x7f71a407a190 con 0x7f71b01551f0 2024-09-15T21:48:22.630 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-15T21:48:22.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.631+0000 7f71b5e9b640 1 -- 172.21.15.100:0/2459243977 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f719c06c2e0 msgr2=0x7f719c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:22.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.631+0000 7f71b5e9b640 1 --2- 172.21.15.100:0/2459243977 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f719c06c2e0 0x7f719c06e7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f71a0006fd0 tx=0x7f71a0008040 comp rx=0 tx=0).stop 2024-09-15T21:48:22.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.631+0000 7f71b5e9b640 1 -- 172.21.15.100:0/2459243977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b01551f0 msgr2=0x7f71b01f0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:22.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.631+0000 7f71b5e9b640 1 --2- 172.21.15.100:0/2459243977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b01551f0 0x7f71b01f0e60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f71a402a340 tx=0x7f71a40046c0 comp rx=0 tx=0).stop 2024-09-15T21:48:22.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.632+0000 7f71b5e9b640 1 -- 172.21.15.100:0/2459243977 shutdown_connections 2024-09-15T21:48:22.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.632+0000 7f71b5e9b640 1 --2- 172.21.15.100:0/2459243977 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f719c06c2e0 0x7f719c06e7a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:22.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.632+0000 7f71b5e9b640 1 --2- 172.21.15.100:0/2459243977 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f71b015ea80 0x7f71b01f13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:22.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.632+0000 7f71b5e9b640 1 --2- 172.21.15.100:0/2459243977 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71b01551f0 0x7f71b01f0e60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:22.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.632+0000 7f71b5e9b640 1 --2- 172.21.15.100:0/2459243977 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71b00c4950 0x7f71b00ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:22.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.632+0000 7f71b5e9b640 1 -- 172.21.15.100:0/2459243977 >> 172.21.15.100:0/2459243977 conn(0x7f71b0067980 msgr2=0x7f71b00c9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:22.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.632+0000 7f71b5e9b640 1 -- 172.21.15.100:0/2459243977 shutdown_connections 2024-09-15T21:48:22.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.632+0000 7f71b5e9b640 1 -- 172.21.15.100:0/2459243977 wait complete. 2024-09-15T21:48:22.649 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-15T21:48:22.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.753+0000 7f94e2c65640 1 Processor -- start 2024-09-15T21:48:22.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.753+0000 7f94e2c65640 1 -- start start 2024-09-15T21:48:22.757 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.754+0000 7f94e2c65640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f94dc0c9550 0x7f94dc0c9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.754+0000 7f94e2c65640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94dc1535b0 con 0x7f94dc1531d0 2024-09-15T21:48:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.754+0000 7f94e2c65640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94dc15c540 con 0x7f94dc0cb5e0 2024-09-15T21:48:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.754+0000 7f94e2c65640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f94dc15d430 con 0x7f94dc0c9550 2024-09-15T21:48:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.754+0000 7f94e0a01640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f94dc1531d0 0x7f94dc15cd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:49306/0 (socket says 172.21.15.100:49306) 2024-09-15T21:48:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.754+0000 7f94e0a01640 1 -- 172.21.15.100:0/619581357 learned_addr learned my addr 172.21.15.100:0/619581357 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.755+0000 7f94dbfff640 1 --2- 172.21.15.100:0/619581357 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f94dc0c9550 0x7f94dc0c9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.755+0000 7f94db7fe640 1 -- 172.21.15.100:0/619581357 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3310935212 0 0) 0x7f94dc1535b0 con 0x7f94dc1531d0 2024-09-15T21:48:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.755+0000 7f94db7fe640 1 -- 172.21.15.100:0/619581357 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94c4003520 con 0x7f94dc1531d0 2024-09-15T21:48:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.755+0000 7f94db7fe640 1 -- 172.21.15.100:0/619581357 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 14997990 0 0) 0x7f94dc15c540 con 0x7f94dc0cb5e0 2024-09-15T21:48:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.755+0000 7f94db7fe640 1 -- 172.21.15.100:0/619581357 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f94dc1535b0 con 0x7f94dc0cb5e0 2024-09-15T21:48:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.755+0000 7f94dbfff640 1 -- 172.21.15.100:0/619581357 >> v1:172.21.15.122:6789/0 conn(0x7f94dc0cb5e0 legacy=0x7f94dc163830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.755+0000 7f94dbfff640 1 -- 172.21.15.100:0/619581357 >> v1:172.21.15.100:6789/0 conn(0x7f94dc1531d0 legacy=0x7f94dc15cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:22.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.755+0000 7f94dbfff640 1 -- 172.21.15.100:0/619581357 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94dc0c41d0 con 0x7f94dc0c9550 2024-09-15T21:48:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.756+0000 7f94dbfff640 1 --2- 172.21.15.100:0/619581357 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f94dc0c9550 0x7f94dc0c9970 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009e10 tx=0x7f94d002a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9440025d9be4836 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.756+0000 7f94db7fe640 1 -- 172.21.15.100:0/619581357 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d0007740 con 0x7f94dc0c9550 2024-09-15T21:48:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.756+0000 7f94db7fe640 1 -- 172.21.15.100:0/619581357 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94d00078e0 con 0x7f94dc0c9550 2024-09-15T21:48:22.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.756+0000 7f94db7fe640 1 -- 172.21.15.100:0/619581357 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d0007c90 con 0x7f94dc0c9550 2024-09-15T21:48:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.756+0000 7f94e2c65640 1 -- 172.21.15.100:0/619581357 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f94dc0c9550 msgr2=0x7f94dc0c9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.756+0000 7f94e2c65640 1 --2- 172.21.15.100:0/619581357 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f94dc0c9550 0x7f94dc0c9970 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009e10 tx=0x7f94d002a730 comp rx=0 tx=0).stop 2024-09-15T21:48:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.757+0000 7f94e2c65640 1 -- 172.21.15.100:0/619581357 shutdown_connections 2024-09-15T21:48:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.757+0000 7f94e2c65640 1 --2- 172.21.15.100:0/619581357 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f94dc0c9550 0x7f94dc0c9970 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.757+0000 7f94e2c65640 1 -- 172.21.15.100:0/619581357 >> 172.21.15.100:0/619581357 conn(0x7f94dc067980 msgr2=0x7f94dc067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.757+0000 7f94e2c65640 1 -- 172.21.15.100:0/619581357 shutdown_connections 2024-09-15T21:48:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.757+0000 7f94e2c65640 1 -- 172.21.15.100:0/619581357 wait complete. 2024-09-15T21:48:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.758+0000 7f94e2c65640 1 Processor -- start 2024-09-15T21:48:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.758+0000 7f94e2c65640 1 -- start start 2024-09-15T21:48:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94e2c65640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94dc0c9550 0x7f94dc0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94e2c65640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f94dc0cb5e0 0x7f94dc151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:22.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94e2c65640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94dc1531d0 0x7f94dc1f0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:22.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94e2c65640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc0c3eb0 con 0x7f94dc1531d0 2024-09-15T21:48:22.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94e2c65640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f94dc0c4790 con 0x7f94dc0c9550 2024-09-15T21:48:22.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94e2c65640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f94dc0b4ee0 con 0x7f94dc0cb5e0 2024-09-15T21:48:22.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94e1202640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94dc1531d0 0x7f94dc1f0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:22.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94e1202640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94dc1531d0 0x7f94dc1f0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57360/0 (socket says 172.21.15.100:57360) 2024-09-15T21:48:22.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94e1202640 1 -- 172.21.15.100:0/796308985 learned_addr learned my addr 172.21.15.100:0/796308985 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:22.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94e0a01640 1 --2- 172.21.15.100:0/796308985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94dc0c9550 0x7f94dc0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:22.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94dbfff640 1 --2- 172.21.15.100:0/796308985 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f94dc0cb5e0 0x7f94dc151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:22.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94e1202640 1 -- 172.21.15.100:0/796308985 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f94dc0cb5e0 msgr2=0x7f94dc151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:22.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94e1202640 1 --2- 172.21.15.100:0/796308985 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f94dc0cb5e0 0x7f94dc151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:22.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94e1202640 1 -- 172.21.15.100:0/796308985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94dc0c9550 msgr2=0x7f94dc0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:22.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94e1202640 1 --2- 172.21.15.100:0/796308985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94dc0c9550 0x7f94dc0cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:22.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94e1202640 1 -- 172.21.15.100:0/796308985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94dc06e5e0 con 0x7f94dc1531d0 2024-09-15T21:48:22.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94e1202640 1 --2- 172.21.15.100:0/796308985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94dc1531d0 0x7f94dc1f0660 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f94d400c400 tx=0x7f94d400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:22.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94d97fa640 1 -- 172.21.15.100:0/796308985 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d4017070 con 0x7f94dc1531d0 2024-09-15T21:48:22.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94d97fa640 1 -- 172.21.15.100:0/796308985 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94d40052d0 con 0x7f94dc1531d0 2024-09-15T21:48:22.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94e2c65640 1 -- 172.21.15.100:0/796308985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94dc1570d0 con 0x7f94dc1531d0 2024-09-15T21:48:22.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94d97fa640 1 -- 172.21.15.100:0/796308985 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d40037f0 con 0x7f94dc1531d0 2024-09-15T21:48:22.765 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.759+0000 7f94e2c65640 1 -- 172.21.15.100:0/796308985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94dc157690 con 0x7f94dc1531d0 2024-09-15T21:48:22.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.760+0000 7f94e2c65640 1 -- 172.21.15.100:0/796308985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94dc0c4a70 con 0x7f94dc1531d0 2024-09-15T21:48:22.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.761+0000 7f94d97fa640 1 -- 172.21.15.100:0/796308985 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f94d401a070 con 0x7f94dc1531d0 2024-09-15T21:48:22.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.761+0000 7f94d97fa640 1 --2- 172.21.15.100:0/796308985 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f94c806c2b0 0x7f94c806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:22.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.761+0000 7f94d97fa640 1 -- 172.21.15.100:0/796308985 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5882+0+0 (secure 0 0 0) 0x7f94d408a890 con 0x7f94dc1531d0 2024-09-15T21:48:22.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.761+0000 7f94e0a01640 1 --2- 172.21.15.100:0/796308985 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f94c806c2b0 0x7f94c806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:22.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.762+0000 7f94e0a01640 1 --2- 172.21.15.100:0/796308985 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f94c806c2b0 0x7f94c806e770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0011e0 tx=0x7f94cc007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:22.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.766+0000 7f94d97fa640 1 -- 172.21.15.100:0/796308985 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f94d4058880 con 0x7f94dc1531d0 2024-09-15T21:48:22.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:22.947+0000 7f94e2c65640 1 -- 172.21.15.100:0/796308985 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd reweight", "id": 1, "weight": 1.0} v 0) -- 0x7f94dc15cf30 con 0x7f94dc1531d0 2024-09-15T21:48:23.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.633+0000 7f94d97fa640 1 -- 172.21.15.100:0/796308985 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 1, "weight": 1.0}]=0 reweighted osd.1 to 1 (10000) v63) ==== 113+0+0 (secure 0 0 0) 0x7f94d405cb80 con 0x7f94dc1531d0 2024-09-15T21:48:23.635 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.1 to 1 (10000) 2024-09-15T21:48:23.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.636+0000 7f94e2c65640 1 -- 172.21.15.100:0/796308985 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f94c806c2b0 msgr2=0x7f94c806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:23.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.636+0000 7f94e2c65640 1 --2- 172.21.15.100:0/796308985 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f94c806c2b0 0x7f94c806e770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f94cc0011e0 tx=0x7f94cc007d30 comp rx=0 tx=0).stop 2024-09-15T21:48:23.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.636+0000 7f94e2c65640 1 -- 172.21.15.100:0/796308985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94dc1531d0 msgr2=0x7f94dc1f0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:23.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.636+0000 7f94e2c65640 1 --2- 172.21.15.100:0/796308985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94dc1531d0 0x7f94dc1f0660 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f94d400c400 tx=0x7f94d400c8c0 comp rx=0 tx=0).stop 2024-09-15T21:48:23.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.636+0000 7f94e2c65640 1 -- 172.21.15.100:0/796308985 shutdown_connections 2024-09-15T21:48:23.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.636+0000 7f94e2c65640 1 --2- 172.21.15.100:0/796308985 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f94c806c2b0 0x7f94c806e770 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:23.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.636+0000 7f94e2c65640 1 --2- 172.21.15.100:0/796308985 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f94dc1531d0 0x7f94dc1f0660 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:23.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.636+0000 7f94e2c65640 1 --2- 172.21.15.100:0/796308985 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f94dc0cb5e0 0x7f94dc151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:23.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.636+0000 7f94e2c65640 1 --2- 172.21.15.100:0/796308985 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94dc0c9550 0x7f94dc0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:23.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.636+0000 7f94e2c65640 1 -- 172.21.15.100:0/796308985 >> 172.21.15.100:0/796308985 conn(0x7f94dc067980 msgr2=0x7f94dc15a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:23.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.636+0000 7f94e2c65640 1 -- 172.21.15.100:0/796308985 shutdown_connections 2024-09-15T21:48:23.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.636+0000 7f94e2c65640 1 -- 172.21.15.100:0/796308985 wait complete. 2024-09-15T21:48:23.652 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-15T21:48:23.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.756+0000 7ff3f8067640 1 Processor -- start 2024-09-15T21:48:23.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.756+0000 7ff3f8067640 1 -- start start 2024-09-15T21:48:23.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.757+0000 7ff3f8067640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff3f0151a10 0x7ff3f0157660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:23.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.757+0000 7ff3f8067640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3f0071220 0x7ff3f006c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:23.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.757+0000 7ff3f8067640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff3f00c91d0 0x7ff3f006c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:23.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.757+0000 7ff3f8067640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff3f0072c30 con 0x7ff3f00c91d0 2024-09-15T21:48:23.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.757+0000 7ff3f8067640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7ff3f0157cf0 con 0x7ff3f0151a10 2024-09-15T21:48:23.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.757+0000 7ff3f8067640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f0157f90 con 0x7ff3f0071220 2024-09-15T21:48:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.757+0000 7ff3f5602640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3f0071220 0x7ff3f006c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.757+0000 7ff3f6604640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff3f00c91d0 0x7ff3f006c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.757+0000 7ff3f6604640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff3f00c91d0 0x7ff3f006c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57374/0 (socket says 172.21.15.100:57374) 2024-09-15T21:48:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.757+0000 7ff3f6604640 1 -- 172.21.15.100:0/4014315571 learned_addr learned my addr 172.21.15.100:0/4014315571 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.757+0000 7ff3f5e03640 1 --2- 172.21.15.100:0/4014315571 >> v2:172.21.15.122:3300/0 conn(0x7ff3f0151a10 0x7ff3f0157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.757+0000 7ff3f6604640 1 -- 172.21.15.100:0/4014315571 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3f0071220 msgr2=0x7ff3f006c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.757+0000 7ff3f6604640 1 --2- 172.21.15.100:0/4014315571 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3f0071220 0x7ff3f006c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.757+0000 7ff3f6604640 1 -- 172.21.15.100:0/4014315571 >> v2:172.21.15.122:3300/0 conn(0x7ff3f0151a10 msgr2=0x7ff3f0157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.757+0000 7ff3f6604640 1 --2- 172.21.15.100:0/4014315571 >> v2:172.21.15.122:3300/0 conn(0x7ff3f0151a10 0x7ff3f0157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:23.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.757+0000 7ff3f6604640 1 -- 172.21.15.100:0/4014315571 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f0158270 con 0x7ff3f00c91d0 2024-09-15T21:48:23.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.758+0000 7ff3f6604640 1 --2- 172.21.15.100:0/4014315571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00c91d0 0x7ff3f006c9b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0050630 tx=0x7ff3e802b620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39480278681873e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.758+0000 7ff3f4e01640 1 -- 172.21.15.100:0/4014315571 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e800b040 con 0x7ff3f00c91d0 2024-09-15T21:48:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.758+0000 7ff3f4e01640 1 -- 172.21.15.100:0/4014315571 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e8030710 con 0x7ff3f00c91d0 2024-09-15T21:48:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.759+0000 7ff3f4e01640 1 -- 172.21.15.100:0/4014315571 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8030ae0 con 0x7ff3f00c91d0 2024-09-15T21:48:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.759+0000 7ff3f8067640 1 -- 172.21.15.100:0/4014315571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00c91d0 msgr2=0x7ff3f006c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.759+0000 7ff3f8067640 1 --2- 172.21.15.100:0/4014315571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00c91d0 0x7ff3f006c9b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff3f0050630 tx=0x7ff3e802b620 comp rx=0 tx=0).stop 2024-09-15T21:48:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.759+0000 7ff3f8067640 1 -- 172.21.15.100:0/4014315571 shutdown_connections 2024-09-15T21:48:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.759+0000 7ff3f8067640 1 --2- 172.21.15.100:0/4014315571 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3f0071220 0x7ff3f006c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.759+0000 7ff3f8067640 1 --2- 172.21.15.100:0/4014315571 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00c91d0 0x7ff3f006c9b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.759+0000 7ff3f8067640 1 --2- 172.21.15.100:0/4014315571 >> v2:172.21.15.122:3300/0 conn(0x7ff3f0151a10 0x7ff3f0157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:23.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.759+0000 7ff3f8067640 1 -- 172.21.15.100:0/4014315571 >> 172.21.15.100:0/4014315571 conn(0x7ff3f00ccbd0 msgr2=0x7ff3f00ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:23.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.759+0000 7ff3f8067640 1 -- 172.21.15.100:0/4014315571 shutdown_connections 2024-09-15T21:48:23.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.759+0000 7ff3f8067640 1 -- 172.21.15.100:0/4014315571 wait complete. 2024-09-15T21:48:23.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.761+0000 7ff3f8067640 1 Processor -- start 2024-09-15T21:48:23.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.761+0000 7ff3f8067640 1 -- start start 2024-09-15T21:48:23.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.761+0000 7ff3f8067640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3f0071220 0x7ff3f0148b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:23.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.761+0000 7ff3f8067640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00c91d0 0x7ff3f006c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:23.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.761+0000 7ff3f8067640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3f0151a10 0x7ff3f0149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:23.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.761+0000 7ff3f8067640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f006bbe0 con 0x7ff3f00c91d0 2024-09-15T21:48:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.761+0000 7ff3f8067640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f0158550 con 0x7ff3f0071220 2024-09-15T21:48:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.761+0000 7ff3f8067640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff3f006bd60 con 0x7ff3f0151a10 2024-09-15T21:48:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.761+0000 7ff3f5602640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00c91d0 0x7ff3f006c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.761+0000 7ff3f5602640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00c91d0 0x7ff3f006c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57378/0 (socket says 172.21.15.100:57378) 2024-09-15T21:48:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.761+0000 7ff3f5602640 1 -- 172.21.15.100:0/801336112 learned_addr learned my addr 172.21.15.100:0/801336112 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.761+0000 7ff3f6604640 1 --2- 172.21.15.100:0/801336112 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3f0151a10 0x7ff3f0149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.761+0000 7ff3f5e03640 1 --2- 172.21.15.100:0/801336112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3f0071220 0x7ff3f0148b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.762+0000 7ff3f5602640 1 -- 172.21.15.100:0/801336112 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3f0151a10 msgr2=0x7ff3f0149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.762+0000 7ff3f5602640 1 --2- 172.21.15.100:0/801336112 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3f0151a10 0x7ff3f0149e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.762+0000 7ff3f5602640 1 -- 172.21.15.100:0/801336112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3f0071220 msgr2=0x7ff3f0148b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.762+0000 7ff3f5602640 1 --2- 172.21.15.100:0/801336112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3f0071220 0x7ff3f0148b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.762+0000 7ff3f5602640 1 -- 172.21.15.100:0/801336112 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f0149670 con 0x7ff3f00c91d0 2024-09-15T21:48:23.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.762+0000 7ff3f5602640 1 --2- 172.21.15.100:0/801336112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00c91d0 0x7ff3f006c3b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000e890 tx=0x7ff3e000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.762+0000 7ff3e67fc640 1 -- 172.21.15.100:0/801336112 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e0017070 con 0x7ff3f00c91d0 2024-09-15T21:48:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.762+0000 7ff3e67fc640 1 -- 172.21.15.100:0/801336112 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3e000c940 con 0x7ff3f00c91d0 2024-09-15T21:48:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.762+0000 7ff3e67fc640 1 -- 172.21.15.100:0/801336112 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e000cd10 con 0x7ff3f00c91d0 2024-09-15T21:48:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.762+0000 7ff3f8067640 1 -- 172.21.15.100:0/801336112 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f01504f0 con 0x7ff3f00c91d0 2024-09-15T21:48:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.762+0000 7ff3f8067640 1 -- 172.21.15.100:0/801336112 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f014fc10 con 0x7ff3f00c91d0 2024-09-15T21:48:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.763+0000 7ff3f8067640 1 -- 172.21.15.100:0/801336112 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f0157660 con 0x7ff3f00c91d0 2024-09-15T21:48:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.764+0000 7ff3e67fc640 1 -- 172.21.15.100:0/801336112 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff3e001a070 con 0x7ff3f00c91d0 2024-09-15T21:48:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.764+0000 7ff3e67fc640 1 --2- 172.21.15.100:0/801336112 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff3d406c2e0 0x7ff3d406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.764+0000 7ff3e67fc640 1 -- 172.21.15.100:0/801336112 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5857+0+0 (secure 0 0 0) 0x7ff3e008b370 con 0x7ff3f00c91d0 2024-09-15T21:48:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.764+0000 7ff3f5e03640 1 --2- 172.21.15.100:0/801336112 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff3d406c2e0 0x7ff3d406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:23.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.764+0000 7ff3f5e03640 1 --2- 172.21.15.100:0/801336112 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff3d406c2e0 0x7ff3d406e7a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec011340 tx=0x7ff3ec00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:23.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.768+0000 7ff3e67fc640 1 -- 172.21.15.100:0/801336112 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff3e0059480 con 0x7ff3f00c91d0 2024-09-15T21:48:23.949 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:23.947+0000 7ff3f8067640 1 -- 172.21.15.100:0/801336112 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 1, "weight": 1.0} v 0) -- 0x7ff3f0155080 con 0x7ff3f00c91d0 2024-09-15T21:48:24.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.637+0000 7ff3e67fc640 1 -- 172.21.15.100:0/801336112 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]=0 set osd.1 primary-affinity to 1 (8655362) v64) ==== 133+0+0 (secure 0 0 0) 0x7ff3e005d780 con 0x7ff3f00c91d0 2024-09-15T21:48:24.640 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-15T21:48:24.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.640+0000 7ff3f8067640 1 -- 172.21.15.100:0/801336112 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff3d406c2e0 msgr2=0x7ff3d406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:24.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.640+0000 7ff3f8067640 1 --2- 172.21.15.100:0/801336112 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff3d406c2e0 0x7ff3d406e7a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec011340 tx=0x7ff3ec00b040 comp rx=0 tx=0).stop 2024-09-15T21:48:24.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.640+0000 7ff3f8067640 1 -- 172.21.15.100:0/801336112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00c91d0 msgr2=0x7ff3f006c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:24.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.640+0000 7ff3f8067640 1 --2- 172.21.15.100:0/801336112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00c91d0 0x7ff3f006c3b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff3e000e890 tx=0x7ff3e000ed50 comp rx=0 tx=0).stop 2024-09-15T21:48:24.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.640+0000 7ff3f8067640 1 -- 172.21.15.100:0/801336112 shutdown_connections 2024-09-15T21:48:24.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.640+0000 7ff3f8067640 1 --2- 172.21.15.100:0/801336112 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff3d406c2e0 0x7ff3d406e7a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:24.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.640+0000 7ff3f8067640 1 --2- 172.21.15.100:0/801336112 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff3f0151a10 0x7ff3f0149e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:24.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.640+0000 7ff3f8067640 1 --2- 172.21.15.100:0/801336112 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff3f00c91d0 0x7ff3f006c3b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:24.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.640+0000 7ff3f8067640 1 --2- 172.21.15.100:0/801336112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3f0071220 0x7ff3f0148b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:24.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.641+0000 7ff3f8067640 1 -- 172.21.15.100:0/801336112 >> 172.21.15.100:0/801336112 conn(0x7ff3f00ccbd0 msgr2=0x7ff3f00730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:24.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.641+0000 7ff3f8067640 1 -- 172.21.15.100:0/801336112 shutdown_connections 2024-09-15T21:48:24.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.641+0000 7ff3f8067640 1 -- 172.21.15.100:0/801336112 wait complete. 2024-09-15T21:48:24.656 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-15T21:48:24.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.757+0000 7fc2606d0640 1 Processor -- start 2024-09-15T21:48:24.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.757+0000 7fc2606d0640 1 -- start start 2024-09-15T21:48:24.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.758+0000 7fc2606d0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc258155570 0x7fc2580ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:24.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.758+0000 7fc2606d0640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc258073a60 0x7fc2580c8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:24.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.758+0000 7fc2606d0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc2580c87d0 0x7fc25815e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:24.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.758+0000 7fc2606d0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc258073430 con 0x7fc258155570 2024-09-15T21:48:24.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.758+0000 7fc2606d0640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fc25815f800 con 0x7fc258073a60 2024-09-15T21:48:24.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.758+0000 7fc2606d0640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc258155950 con 0x7fc2580c87d0 2024-09-15T21:48:24.760 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.758+0000 7fc25e46c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc258155570 0x7fc2580ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.758+0000 7fc25e46c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc258155570 0x7fc2580ce3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57384/0 (socket says 172.21.15.100:57384) 2024-09-15T21:48:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.758+0000 7fc25e46c640 1 -- 172.21.15.100:0/1591198265 learned_addr learned my addr 172.21.15.100:0/1591198265 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.758+0000 7fc25dc6b640 1 --2- 172.21.15.100:0/1591198265 >> v2:172.21.15.122:3300/0 conn(0x7fc258073a60 0x7fc2580c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.758+0000 7fc25ec6d640 1 --2- 172.21.15.100:0/1591198265 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc2580c87d0 0x7fc25815e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.758+0000 7fc25e46c640 1 -- 172.21.15.100:0/1591198265 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc2580c87d0 msgr2=0x7fc25815e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.758+0000 7fc25e46c640 1 --2- 172.21.15.100:0/1591198265 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc2580c87d0 0x7fc25815e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.758+0000 7fc25e46c640 1 -- 172.21.15.100:0/1591198265 >> v2:172.21.15.122:3300/0 conn(0x7fc258073a60 msgr2=0x7fc2580c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.758+0000 7fc25e46c640 1 --2- 172.21.15.100:0/1591198265 >> v2:172.21.15.122:3300/0 conn(0x7fc258073a60 0x7fc2580c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.758+0000 7fc25e46c640 1 -- 172.21.15.100:0/1591198265 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25806cf00 con 0x7fc258155570 2024-09-15T21:48:24.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.758+0000 7fc25ec6d640 1 --2- 172.21.15.100:0/1591198265 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc2580c87d0 0x7fc25815e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:48:24.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.759+0000 7fc25e46c640 1 --2- 172.21.15.100:0/1591198265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc258155570 0x7fc2580ce3c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc2580c5120 tx=0x7fc25402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8252233b33c7b146 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:24.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.759+0000 7fc25d46a640 1 -- 172.21.15.100:0/1591198265 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254037070 con 0x7fc258155570 2024-09-15T21:48:24.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.759+0000 7fc25d46a640 1 -- 172.21.15.100:0/1591198265 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc254032440 con 0x7fc258155570 2024-09-15T21:48:24.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.759+0000 7fc25d46a640 1 -- 172.21.15.100:0/1591198265 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254032810 con 0x7fc258155570 2024-09-15T21:48:24.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.760+0000 7fc2606d0640 1 -- 172.21.15.100:0/1591198265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc258155570 msgr2=0x7fc2580ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:24.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.760+0000 7fc2606d0640 1 --2- 172.21.15.100:0/1591198265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc258155570 0x7fc2580ce3c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc2580c5120 tx=0x7fc25402a210 comp rx=0 tx=0).stop 2024-09-15T21:48:24.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.760+0000 7fc2606d0640 1 -- 172.21.15.100:0/1591198265 shutdown_connections 2024-09-15T21:48:24.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.760+0000 7fc2606d0640 1 --2- 172.21.15.100:0/1591198265 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc2580c87d0 0x7fc25815e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:24.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.760+0000 7fc2606d0640 1 --2- 172.21.15.100:0/1591198265 >> v2:172.21.15.122:3300/0 conn(0x7fc258073a60 0x7fc2580c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:24.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.760+0000 7fc2606d0640 1 --2- 172.21.15.100:0/1591198265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc258155570 0x7fc2580ce3c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:24.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.760+0000 7fc2606d0640 1 -- 172.21.15.100:0/1591198265 >> 172.21.15.100:0/1591198265 conn(0x7fc2580cd890 msgr2=0x7fc2580cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:24.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.760+0000 7fc2606d0640 1 -- 172.21.15.100:0/1591198265 shutdown_connections 2024-09-15T21:48:24.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.760+0000 7fc2606d0640 1 -- 172.21.15.100:0/1591198265 wait complete. 2024-09-15T21:48:24.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.761+0000 7fc2606d0640 1 Processor -- start 2024-09-15T21:48:24.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.761+0000 7fc2606d0640 1 -- start start 2024-09-15T21:48:24.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.761+0000 7fc2606d0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc258073a60 0x7fc2581e7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:24.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.761+0000 7fc2606d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2580c87d0 0x7fc2581e80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:24.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.761+0000 7fc2606d0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc258155570 0x7fc2580c0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:24.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.761+0000 7fc2606d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc25806c700 con 0x7fc2580c87d0 2024-09-15T21:48:24.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.761+0000 7fc2606d0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc25815f520 con 0x7fc258073a60 2024-09-15T21:48:24.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.761+0000 7fc2606d0640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc25806c9e0 con 0x7fc258155570 2024-09-15T21:48:24.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.762+0000 7fc25dc6b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2580c87d0 0x7fc2581e80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:24.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.762+0000 7fc25dc6b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2580c87d0 0x7fc2581e80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57392/0 (socket says 172.21.15.100:57392) 2024-09-15T21:48:24.766 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.762+0000 7fc25dc6b640 1 -- 172.21.15.100:0/2031299502 learned_addr learned my addr 172.21.15.100:0/2031299502 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:24.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.762+0000 7fc25e46c640 1 --2- 172.21.15.100:0/2031299502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc258073a60 0x7fc2581e7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:24.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.762+0000 7fc25ec6d640 1 --2- 172.21.15.100:0/2031299502 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc258155570 0x7fc2580c0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:24.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.762+0000 7fc25dc6b640 1 -- 172.21.15.100:0/2031299502 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc258155570 msgr2=0x7fc2580c0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:24.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.762+0000 7fc25dc6b640 1 --2- 172.21.15.100:0/2031299502 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc258155570 0x7fc2580c0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:24.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.762+0000 7fc25dc6b640 1 -- 172.21.15.100:0/2031299502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc258073a60 msgr2=0x7fc2581e7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:24.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.762+0000 7fc25dc6b640 1 --2- 172.21.15.100:0/2031299502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc258073a60 0x7fc2581e7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:24.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.762+0000 7fc25dc6b640 1 -- 172.21.15.100:0/2031299502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2580bf5c0 con 0x7fc2580c87d0 2024-09-15T21:48:24.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.762+0000 7fc25dc6b640 1 --2- 172.21.15.100:0/2031299502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2580c87d0 0x7fc2581e80b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc24800e870 tx=0x7fc24800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:24.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.762+0000 7fc24effd640 1 -- 172.21.15.100:0/2031299502 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248017070 con 0x7fc2580c87d0 2024-09-15T21:48:24.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.762+0000 7fc24effd640 1 -- 172.21.15.100:0/2031299502 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc24800ee60 con 0x7fc2580c87d0 2024-09-15T21:48:24.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.762+0000 7fc2606d0640 1 -- 172.21.15.100:0/2031299502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2580bf8a0 con 0x7fc2580c87d0 2024-09-15T21:48:24.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.762+0000 7fc24effd640 1 -- 172.21.15.100:0/2031299502 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24800cb80 con 0x7fc2580c87d0 2024-09-15T21:48:24.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.762+0000 7fc2606d0640 1 -- 172.21.15.100:0/2031299502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2580bfe60 con 0x7fc2580c87d0 2024-09-15T21:48:24.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.763+0000 7fc2606d0640 1 -- 172.21.15.100:0/2031299502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc258072080 con 0x7fc2580c87d0 2024-09-15T21:48:24.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.764+0000 7fc24effd640 1 -- 172.21.15.100:0/2031299502 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc24801a070 con 0x7fc2580c87d0 2024-09-15T21:48:24.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.764+0000 7fc24effd640 1 --2- 172.21.15.100:0/2031299502 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc23c06c2e0 0x7fc23c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:24.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.764+0000 7fc24effd640 1 -- 172.21.15.100:0/2031299502 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5857+0+0 (secure 0 0 0) 0x7fc24808b5f0 con 0x7fc2580c87d0 2024-09-15T21:48:24.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.764+0000 7fc25e46c640 1 --2- 172.21.15.100:0/2031299502 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc23c06c2e0 0x7fc23c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:24.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.765+0000 7fc25e46c640 1 --2- 172.21.15.100:0/2031299502 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc23c06c2e0 0x7fc23c06e7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc254034000 tx=0x7fc25402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:24.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.768+0000 7fc24effd640 1 -- 172.21.15.100:0/2031299502 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc248059670 con 0x7fc2580c87d0 2024-09-15T21:48:24.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:24.948+0000 7fc2606d0640 1 -- 172.21.15.100:0/2031299502 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd reweight", "id": 2, "weight": 1.0} v 0) -- 0x7fc2580726d0 con 0x7fc2580c87d0 2024-09-15T21:48:25.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.639+0000 7fc24effd640 1 -- 172.21.15.100:0/2031299502 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 2, "weight": 1.0}]=0 reweighted osd.2 to 1 (10000) v65) ==== 113+0+0 (secure 0 0 0) 0x7fc24805d970 con 0x7fc2580c87d0 2024-09-15T21:48:25.641 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.2 to 1 (10000) 2024-09-15T21:48:25.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.642+0000 7fc2606d0640 1 -- 172.21.15.100:0/2031299502 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc23c06c2e0 msgr2=0x7fc23c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:25.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.642+0000 7fc2606d0640 1 --2- 172.21.15.100:0/2031299502 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc23c06c2e0 0x7fc23c06e7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc254034000 tx=0x7fc25402e000 comp rx=0 tx=0).stop 2024-09-15T21:48:25.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.642+0000 7fc2606d0640 1 -- 172.21.15.100:0/2031299502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2580c87d0 msgr2=0x7fc2581e80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:25.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.642+0000 7fc2606d0640 1 --2- 172.21.15.100:0/2031299502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2580c87d0 0x7fc2581e80b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc24800e870 tx=0x7fc24800ed30 comp rx=0 tx=0).stop 2024-09-15T21:48:25.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.642+0000 7fc2606d0640 1 -- 172.21.15.100:0/2031299502 shutdown_connections 2024-09-15T21:48:25.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.642+0000 7fc2606d0640 1 --2- 172.21.15.100:0/2031299502 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc23c06c2e0 0x7fc23c06e7a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:25.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.642+0000 7fc2606d0640 1 --2- 172.21.15.100:0/2031299502 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc258155570 0x7fc2580c0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:25.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.642+0000 7fc2606d0640 1 --2- 172.21.15.100:0/2031299502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc2580c87d0 0x7fc2581e80b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:25.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.642+0000 7fc2606d0640 1 --2- 172.21.15.100:0/2031299502 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc258073a60 0x7fc2581e7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:25.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.642+0000 7fc2606d0640 1 -- 172.21.15.100:0/2031299502 >> 172.21.15.100:0/2031299502 conn(0x7fc2580cd890 msgr2=0x7fc2580cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:25.644 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.642+0000 7fc2606d0640 1 -- 172.21.15.100:0/2031299502 shutdown_connections 2024-09-15T21:48:25.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.642+0000 7fc2606d0640 1 -- 172.21.15.100:0/2031299502 wait complete. 2024-09-15T21:48:25.656 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-15T21:48:25.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.757+0000 7ff7d7549640 1 Processor -- start 2024-09-15T21:48:25.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.757+0000 7ff7d7549640 1 -- start start 2024-09-15T21:48:25.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.758+0000 7ff7d7549640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff7d0152130 0x7ff7d0157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:25.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.758+0000 7ff7d7549640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff7d0071ff0 0x7ff7d0072420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:25.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.758+0000 7ff7d7549640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7d00cd790 0x7ff7d00cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:25.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.758+0000 7ff7d7549640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff7d00c7c30 con 0x7ff7d0152130 2024-09-15T21:48:25.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.758+0000 7ff7d7549640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7ff7d01584e0 con 0x7ff7d0071ff0 2024-09-15T21:48:25.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.758+0000 7ff7d7549640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d01587c0 con 0x7ff7d00cd790 2024-09-15T21:48:25.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.758+0000 7ff7d52e5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff7d0152130 0x7ff7d0157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:25.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.758+0000 7ff7d52e5640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff7d0152130 0x7ff7d0157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57394/0 (socket says 172.21.15.100:57394) 2024-09-15T21:48:25.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.759+0000 7ff7d52e5640 1 -- 172.21.15.100:0/1949929141 learned_addr learned my addr 172.21.15.100:0/1949929141 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:25.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.759+0000 7ff7d4ae4640 1 --2- 172.21.15.100:0/1949929141 >> v2:172.21.15.122:3300/0 conn(0x7ff7d0071ff0 0x7ff7d0072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:25.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.759+0000 7ff7d5ae6640 1 --2- 172.21.15.100:0/1949929141 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7d00cd790 0x7ff7d00cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:25.761 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.759+0000 7ff7d52e5640 1 -- 172.21.15.100:0/1949929141 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7d00cd790 msgr2=0x7ff7d00cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:25.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.759+0000 7ff7d52e5640 1 --2- 172.21.15.100:0/1949929141 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7d00cd790 0x7ff7d00cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:25.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.759+0000 7ff7d52e5640 1 -- 172.21.15.100:0/1949929141 >> v2:172.21.15.122:3300/0 conn(0x7ff7d0071ff0 msgr2=0x7ff7d0072420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:25.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.759+0000 7ff7d52e5640 1 --2- 172.21.15.100:0/1949929141 >> v2:172.21.15.122:3300/0 conn(0x7ff7d0071ff0 0x7ff7d0072420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:25.762 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.759+0000 7ff7d52e5640 1 -- 172.21.15.100:0/1949929141 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d0158aa0 con 0x7ff7d0152130 2024-09-15T21:48:25.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.759+0000 7ff7d52e5640 1 --2- 172.21.15.100:0/1949929141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d0152130 0x7ff7d0157ea0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff7d00cb240 tx=0x7ff7cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f4d38c2d087fd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:25.763 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.760+0000 7ff7c7fff640 1 -- 172.21.15.100:0/1949929141 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc037070 con 0x7ff7d0152130 2024-09-15T21:48:25.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.760+0000 7ff7c7fff640 1 -- 172.21.15.100:0/1949929141 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7cc032440 con 0x7ff7d0152130 2024-09-15T21:48:25.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.760+0000 7ff7c7fff640 1 -- 172.21.15.100:0/1949929141 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc032810 con 0x7ff7d0152130 2024-09-15T21:48:25.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.760+0000 7ff7d7549640 1 -- 172.21.15.100:0/1949929141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d0152130 msgr2=0x7ff7d0157ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:25.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.760+0000 7ff7d7549640 1 --2- 172.21.15.100:0/1949929141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d0152130 0x7ff7d0157ea0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7ff7d00cb240 tx=0x7ff7cc02a270 comp rx=0 tx=0).stop 2024-09-15T21:48:25.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.760+0000 7ff7d7549640 1 -- 172.21.15.100:0/1949929141 shutdown_connections 2024-09-15T21:48:25.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.760+0000 7ff7d7549640 1 --2- 172.21.15.100:0/1949929141 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7d00cd790 0x7ff7d00cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:25.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.760+0000 7ff7d7549640 1 --2- 172.21.15.100:0/1949929141 >> v2:172.21.15.122:3300/0 conn(0x7ff7d0071ff0 0x7ff7d0072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:25.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.760+0000 7ff7d7549640 1 --2- 172.21.15.100:0/1949929141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d0152130 0x7ff7d0157ea0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:25.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.760+0000 7ff7d7549640 1 -- 172.21.15.100:0/1949929141 >> 172.21.15.100:0/1949929141 conn(0x7ff7d0067fb0 msgr2=0x7ff7d006fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:25.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.760+0000 7ff7d7549640 1 -- 172.21.15.100:0/1949929141 shutdown_connections 2024-09-15T21:48:25.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.760+0000 7ff7d7549640 1 -- 172.21.15.100:0/1949929141 wait complete. 2024-09-15T21:48:25.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.761+0000 7ff7d7549640 1 Processor -- start 2024-09-15T21:48:25.764 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.761+0000 7ff7d7549640 1 -- start start 2024-09-15T21:48:25.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.762+0000 7ff7d7549640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7d0071ff0 0x7ff7d01f17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:25.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.762+0000 7ff7d7549640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7d00cd790 0x7ff7d0157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:25.767 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.762+0000 7ff7d7549640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d0152130 0x7ff7d01560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.762+0000 7ff7d7549640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d0072d70 con 0x7ff7d0152130 2024-09-15T21:48:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.762+0000 7ff7d7549640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff7d0072a90 con 0x7ff7d00cd790 2024-09-15T21:48:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.762+0000 7ff7d7549640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff7d0152510 con 0x7ff7d0071ff0 2024-09-15T21:48:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.762+0000 7ff7d5ae6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d0152130 0x7ff7d01560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.762+0000 7ff7d5ae6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d0152130 0x7ff7d01560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57406/0 (socket says 172.21.15.100:57406) 2024-09-15T21:48:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.762+0000 7ff7d5ae6640 1 -- 172.21.15.100:0/3595059661 learned_addr learned my addr 172.21.15.100:0/3595059661 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.762+0000 7ff7d4ae4640 1 --2- 172.21.15.100:0/3595059661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7d00cd790 0x7ff7d0157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.762+0000 7ff7d52e5640 1 --2- 172.21.15.100:0/3595059661 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7d0071ff0 0x7ff7d01f17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.762+0000 7ff7d5ae6640 1 -- 172.21.15.100:0/3595059661 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7d0071ff0 msgr2=0x7ff7d01f17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.762+0000 7ff7d5ae6640 1 --2- 172.21.15.100:0/3595059661 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7d0071ff0 0x7ff7d01f17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:25.768 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.763+0000 7ff7d5ae6640 1 -- 172.21.15.100:0/3595059661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7d00cd790 msgr2=0x7ff7d0157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.763+0000 7ff7d5ae6640 1 --2- 172.21.15.100:0/3595059661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7d00cd790 0x7ff7d0157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.763+0000 7ff7d5ae6640 1 -- 172.21.15.100:0/3595059661 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7d0156fd0 con 0x7ff7d0152130 2024-09-15T21:48:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.763+0000 7ff7d5ae6640 1 --2- 172.21.15.100:0/3595059661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d0152130 0x7ff7d01560f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8007e70 tx=0x7ff7c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.763+0000 7ff7c5ffb640 1 -- 172.21.15.100:0/3595059661 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8014620 con 0x7ff7d0152130 2024-09-15T21:48:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.763+0000 7ff7c5ffb640 1 -- 172.21.15.100:0/3595059661 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7c8019070 con 0x7ff7d0152130 2024-09-15T21:48:25.769 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.763+0000 7ff7d7549640 1 -- 172.21.15.100:0/3595059661 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7d00bffd0 con 0x7ff7d0152130 2024-09-15T21:48:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.763+0000 7ff7c5ffb640 1 -- 172.21.15.100:0/3595059661 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8014a20 con 0x7ff7d0152130 2024-09-15T21:48:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.763+0000 7ff7d7549640 1 -- 172.21.15.100:0/3595059661 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7d00bf730 con 0x7ff7d0152130 2024-09-15T21:48:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.764+0000 7ff7d7549640 1 -- 172.21.15.100:0/3595059661 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7d00ce710 con 0x7ff7d0152130 2024-09-15T21:48:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.764+0000 7ff7c5ffb640 1 -- 172.21.15.100:0/3595059661 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff7c8014bc0 con 0x7ff7d0152130 2024-09-15T21:48:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.765+0000 7ff7c5ffb640 1 --2- 172.21.15.100:0/3595059661 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff7b406c2e0 0x7ff7b406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.765+0000 7ff7c5ffb640 1 -- 172.21.15.100:0/3595059661 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 5857+0+0 (secure 0 0 0) 0x7ff7c808bd30 con 0x7ff7d0152130 2024-09-15T21:48:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.765+0000 7ff7d52e5640 1 --2- 172.21.15.100:0/3595059661 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff7b406c2e0 0x7ff7b406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:25.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.765+0000 7ff7d52e5640 1 --2- 172.21.15.100:0/3595059661 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff7b406c2e0 0x7ff7b406e7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc034000 tx=0x7ff7cc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:25.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.769+0000 7ff7c5ffb640 1 -- 172.21.15.100:0/3595059661 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff7d00ce710 con 0x7ff7d0152130 2024-09-15T21:48:25.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:25.945+0000 7ff7d7549640 1 -- 172.21.15.100:0/3595059661 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 2, "weight": 1.0} v 0) -- 0x7ff7d0072420 con 0x7ff7d0152130 2024-09-15T21:48:26.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.647+0000 7ff7c5ffb640 1 -- 172.21.15.100:0/3595059661 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]=0 set osd.2 primary-affinity to 1 (8655362) v66) ==== 133+0+0 (secure 0 0 0) 0x7ff7c805a400 con 0x7ff7d0152130 2024-09-15T21:48:26.649 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-15T21:48:26.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.651+0000 7ff7d7549640 1 -- 172.21.15.100:0/3595059661 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff7b406c2e0 msgr2=0x7ff7b406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:26.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.651+0000 7ff7d7549640 1 --2- 172.21.15.100:0/3595059661 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff7b406c2e0 0x7ff7b406e7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc034000 tx=0x7ff7cc02e000 comp rx=0 tx=0).stop 2024-09-15T21:48:26.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.651+0000 7ff7d7549640 1 -- 172.21.15.100:0/3595059661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d0152130 msgr2=0x7ff7d01560f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:26.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.651+0000 7ff7d7549640 1 --2- 172.21.15.100:0/3595059661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d0152130 0x7ff7d01560f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8007e70 tx=0x7ff7c800e4b0 comp rx=0 tx=0).stop 2024-09-15T21:48:26.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.651+0000 7ff7d7549640 1 -- 172.21.15.100:0/3595059661 shutdown_connections 2024-09-15T21:48:26.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.651+0000 7ff7d7549640 1 --2- 172.21.15.100:0/3595059661 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff7b406c2e0 0x7ff7b406e7a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:26.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.651+0000 7ff7d7549640 1 --2- 172.21.15.100:0/3595059661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7d00cd790 0x7ff7d0157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:26.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.651+0000 7ff7d7549640 1 --2- 172.21.15.100:0/3595059661 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7d0152130 0x7ff7d01560f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:26.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.651+0000 7ff7d7549640 1 --2- 172.21.15.100:0/3595059661 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7d0071ff0 0x7ff7d01f17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:26.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.651+0000 7ff7d7549640 1 -- 172.21.15.100:0/3595059661 >> 172.21.15.100:0/3595059661 conn(0x7ff7d0067fb0 msgr2=0x7ff7d0073170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:26.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.651+0000 7ff7d7549640 1 -- 172.21.15.100:0/3595059661 shutdown_connections 2024-09-15T21:48:26.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.651+0000 7ff7d7549640 1 -- 172.21.15.100:0/3595059661 wait complete. 2024-09-15T21:48:26.668 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-15T21:48:26.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.772+0000 7fd1f0f66640 1 Processor -- start 2024-09-15T21:48:26.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.772+0000 7fd1f0f66640 1 -- start start 2024-09-15T21:48:26.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.773+0000 7fd1f0f66640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd1ec15c050 0x7fd1ec067b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:26.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.773+0000 7fd1f0f66640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd1ec0720c0 0x7fd1ec072550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:26.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.773+0000 7fd1f0f66640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1ec071780 con 0x7fd1ec0c6ff0 2024-09-15T21:48:26.776 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.773+0000 7fd1f0f66640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fd1ec15b440 con 0x7fd1ec15c050 2024-09-15T21:48:26.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.773+0000 7fd1f0f66640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fd1ec15f560 con 0x7fd1ec0720c0 2024-09-15T21:48:26.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.773+0000 7fd1e9d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd1ec15c050 0x7fd1ec067b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:26.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.773+0000 7fd1e9d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd1ec15c050 0x7fd1ec067b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53734/0 (socket says 172.21.15.100:53734) 2024-09-15T21:48:26.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.773+0000 7fd1e9d9b640 1 -- 172.21.15.100:0/578444586 learned_addr learned my addr 172.21.15.100:0/578444586 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:26.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.774+0000 7fd1ead9d640 1 --2- 172.21.15.100:0/578444586 >> v2:172.21.15.122:3300/0 conn(0x7fd1ec0720c0 0x7fd1ec072550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:26.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.774+0000 7fd1e9d9b640 1 -- 172.21.15.100:0/578444586 >> v2:172.21.15.122:3300/0 conn(0x7fd1ec0720c0 msgr2=0x7fd1ec072550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:26.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.774+0000 7fd1e9d9b640 1 --2- 172.21.15.100:0/578444586 >> v2:172.21.15.122:3300/0 conn(0x7fd1ec0720c0 0x7fd1ec072550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:26.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.774+0000 7fd1e9d9b640 1 -- 172.21.15.100:0/578444586 >> v1:172.21.15.122:6789/0 conn(0x7fd1ec0c6ff0 legacy=0x7fd1ec15b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:26.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.774+0000 7fd1e9d9b640 1 -- 172.21.15.100:0/578444586 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ec071a20 con 0x7fd1ec15c050 2024-09-15T21:48:26.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.774+0000 7fd1e9d9b640 1 --2- 172.21.15.100:0/578444586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ec15c050 0x7fd1ec067b10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec159050 tx=0x7fd1e002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b81f465b3a29e6b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:26.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.775+0000 7fd1e959a640 1 -- 172.21.15.100:0/578444586 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0037070 con 0x7fd1ec15c050 2024-09-15T21:48:26.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.775+0000 7fd1e959a640 1 -- 172.21.15.100:0/578444586 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e0032440 con 0x7fd1ec15c050 2024-09-15T21:48:26.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.775+0000 7fd1e959a640 1 -- 172.21.15.100:0/578444586 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0032810 con 0x7fd1ec15c050 2024-09-15T21:48:26.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.775+0000 7fd1f0f66640 1 -- 172.21.15.100:0/578444586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ec15c050 msgr2=0x7fd1ec067b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:26.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.775+0000 7fd1f0f66640 1 --2- 172.21.15.100:0/578444586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ec15c050 0x7fd1ec067b10 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec159050 tx=0x7fd1e002a2c0 comp rx=0 tx=0).stop 2024-09-15T21:48:26.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.775+0000 7fd1f0f66640 1 -- 172.21.15.100:0/578444586 shutdown_connections 2024-09-15T21:48:26.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.775+0000 7fd1f0f66640 1 --2- 172.21.15.100:0/578444586 >> v2:172.21.15.122:3300/0 conn(0x7fd1ec0720c0 0x7fd1ec072550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:26.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.775+0000 7fd1f0f66640 1 --2- 172.21.15.100:0/578444586 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ec15c050 0x7fd1ec067b10 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:26.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.775+0000 7fd1f0f66640 1 -- 172.21.15.100:0/578444586 >> 172.21.15.100:0/578444586 conn(0x7fd1ec06bbe0 msgr2=0x7fd1ec06bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:26.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.775+0000 7fd1f0f66640 1 -- 172.21.15.100:0/578444586 shutdown_connections 2024-09-15T21:48:26.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.775+0000 7fd1f0f66640 1 -- 172.21.15.100:0/578444586 wait complete. 2024-09-15T21:48:26.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.776+0000 7fd1f0f66640 1 Processor -- start 2024-09-15T21:48:26.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.777+0000 7fd1f0f66640 1 -- start start 2024-09-15T21:48:26.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.777+0000 7fd1f0f66640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1ec0720c0 0x7fd1ec1e56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:26.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.777+0000 7fd1f0f66640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ec0c6ff0 0x7fd1ec1e5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:26.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.777+0000 7fd1f0f66640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd1ec15c050 0x7fd1ec158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:26.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.777+0000 7fd1f0f66640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec06eac0 con 0x7fd1ec0c6ff0 2024-09-15T21:48:26.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.777+0000 7fd1f0f66640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd1ec15f840 con 0x7fd1ec0720c0 2024-09-15T21:48:26.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.777+0000 7fd1f0f66640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd1ec06ec40 con 0x7fd1ec15c050 2024-09-15T21:48:26.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.777+0000 7fd1e9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ec0c6ff0 0x7fd1ec1e5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:26.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.777+0000 7fd1e9d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ec0c6ff0 0x7fd1ec1e5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53744/0 (socket says 172.21.15.100:53744) 2024-09-15T21:48:26.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.777+0000 7fd1e9d9b640 1 -- 172.21.15.100:0/4291694534 learned_addr learned my addr 172.21.15.100:0/4291694534 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:26.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.777+0000 7fd1ea59c640 1 --2- 172.21.15.100:0/4291694534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1ec0720c0 0x7fd1ec1e56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:26.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.777+0000 7fd1ead9d640 1 --2- 172.21.15.100:0/4291694534 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd1ec15c050 0x7fd1ec158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:26.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.778+0000 7fd1e9d9b640 1 -- 172.21.15.100:0/4291694534 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd1ec15c050 msgr2=0x7fd1ec158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:26.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.778+0000 7fd1e9d9b640 1 --2- 172.21.15.100:0/4291694534 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd1ec15c050 0x7fd1ec158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:26.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.778+0000 7fd1e9d9b640 1 -- 172.21.15.100:0/4291694534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1ec0720c0 msgr2=0x7fd1ec1e56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:26.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.778+0000 7fd1e9d9b640 1 --2- 172.21.15.100:0/4291694534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1ec0720c0 0x7fd1ec1e56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:26.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.778+0000 7fd1e9d9b640 1 -- 172.21.15.100:0/4291694534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1ec1e6730 con 0x7fd1ec0c6ff0 2024-09-15T21:48:26.784 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.778+0000 7fd1ea59c640 1 --2- 172.21.15.100:0/4291694534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1ec0720c0 0x7fd1ec1e56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:48:26.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.778+0000 7fd1ead9d640 1 --2- 172.21.15.100:0/4291694534 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd1ec15c050 0x7fd1ec158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:48:26.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.778+0000 7fd1e9d9b640 1 --2- 172.21.15.100:0/4291694534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ec0c6ff0 0x7fd1ec1e5be0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd1e002a300 tx=0x7fd1e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:26.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.778+0000 7fd1daffd640 1 -- 172.21.15.100:0/4291694534 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0037040 con 0x7fd1ec0c6ff0 2024-09-15T21:48:26.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.778+0000 7fd1daffd640 1 -- 172.21.15.100:0/4291694534 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1e0042070 con 0x7fd1ec0c6ff0 2024-09-15T21:48:26.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.778+0000 7fd1f0f66640 1 -- 172.21.15.100:0/4291694534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1ec1e6ff0 con 0x7fd1ec0c6ff0 2024-09-15T21:48:26.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.778+0000 7fd1daffd640 1 -- 172.21.15.100:0/4291694534 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e0030bd0 con 0x7fd1ec0c6ff0 2024-09-15T21:48:26.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.778+0000 7fd1f0f66640 1 -- 172.21.15.100:0/4291694534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1ec1e5180 con 0x7fd1ec0c6ff0 2024-09-15T21:48:26.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.779+0000 7fd1f0f66640 1 -- 172.21.15.100:0/4291694534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1ec0c73d0 con 0x7fd1ec0c6ff0 2024-09-15T21:48:26.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.779+0000 7fd1daffd640 1 -- 172.21.15.100:0/4291694534 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd1e0047460 con 0x7fd1ec0c6ff0 2024-09-15T21:48:26.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.780+0000 7fd1daffd640 1 --2- 172.21.15.100:0/4291694534 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd1cc06c2b0 0x7fd1cc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:26.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.780+0000 7fd1daffd640 1 -- 172.21.15.100:0/4291694534 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5857+0+0 (secure 0 0 0) 0x7fd1e00ac140 con 0x7fd1ec0c6ff0 2024-09-15T21:48:26.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.780+0000 7fd1ea59c640 1 --2- 172.21.15.100:0/4291694534 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd1cc06c2b0 0x7fd1cc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:26.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.780+0000 7fd1ea59c640 1 --2- 172.21.15.100:0/4291694534 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd1cc06c2b0 0x7fd1cc06e770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc001750 tx=0x7fd1dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:26.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.784+0000 7fd1daffd640 1 -- 172.21.15.100:0/4291694534 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd1e007a8a0 con 0x7fd1ec0c6ff0 2024-09-15T21:48:26.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:26.959+0000 7fd1f0f66640 1 -- 172.21.15.100:0/4291694534 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd reweight", "id": 3, "weight": 1.0} v 0) -- 0x7fd1ec0c0070 con 0x7fd1ec0c6ff0 2024-09-15T21:48:27.651 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.649+0000 7fd1daffd640 1 -- 172.21.15.100:0/4291694534 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 3, "weight": 1.0}]=0 reweighted osd.3 to 1 (10000) v67) ==== 113+0+0 (secure 0 0 0) 0x7fd1e007a1a0 con 0x7fd1ec0c6ff0 2024-09-15T21:48:27.651 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.3 to 1 (10000) 2024-09-15T21:48:27.653 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.651+0000 7fd1f0f66640 1 -- 172.21.15.100:0/4291694534 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd1cc06c2b0 msgr2=0x7fd1cc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:27.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.651+0000 7fd1f0f66640 1 --2- 172.21.15.100:0/4291694534 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd1cc06c2b0 0x7fd1cc06e770 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc001750 tx=0x7fd1dc009040 comp rx=0 tx=0).stop 2024-09-15T21:48:27.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.651+0000 7fd1f0f66640 1 -- 172.21.15.100:0/4291694534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ec0c6ff0 msgr2=0x7fd1ec1e5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:27.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.651+0000 7fd1f0f66640 1 --2- 172.21.15.100:0/4291694534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ec0c6ff0 0x7fd1ec1e5be0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd1e002a300 tx=0x7fd1e00046c0 comp rx=0 tx=0).stop 2024-09-15T21:48:27.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.651+0000 7fd1f0f66640 1 -- 172.21.15.100:0/4291694534 shutdown_connections 2024-09-15T21:48:27.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.651+0000 7fd1f0f66640 1 --2- 172.21.15.100:0/4291694534 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd1cc06c2b0 0x7fd1cc06e770 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:27.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.651+0000 7fd1f0f66640 1 --2- 172.21.15.100:0/4291694534 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd1ec15c050 0x7fd1ec158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:27.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.651+0000 7fd1f0f66640 1 --2- 172.21.15.100:0/4291694534 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1ec0c6ff0 0x7fd1ec1e5be0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:27.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.651+0000 7fd1f0f66640 1 --2- 172.21.15.100:0/4291694534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1ec0720c0 0x7fd1ec1e56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:27.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.651+0000 7fd1f0f66640 1 -- 172.21.15.100:0/4291694534 >> 172.21.15.100:0/4291694534 conn(0x7fd1ec06bbe0 msgr2=0x7fd1ec0c7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:27.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.651+0000 7fd1f0f66640 1 -- 172.21.15.100:0/4291694534 shutdown_connections 2024-09-15T21:48:27.654 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.651+0000 7fd1f0f66640 1 -- 172.21.15.100:0/4291694534 wait complete. 2024-09-15T21:48:27.668 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-15T21:48:27.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.768+0000 7f0dcef66640 1 Processor -- start 2024-09-15T21:48:27.770 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.768+0000 7f0dcef66640 1 -- start start 2024-09-15T21:48:27.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.769+0000 7f0dcef66640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0dc8071220 0x7f0dc8151a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:27.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.769+0000 7f0dcef66640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0dc8157660 0x7f0dc806bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:27.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.769+0000 7f0dcef66640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dc8157cf0 con 0x7f0dc80c91d0 2024-09-15T21:48:27.771 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.769+0000 7f0dcef66640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f0dc8072c30 con 0x7f0dc8157660 2024-09-15T21:48:27.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.769+0000 7f0dcef66640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc8157f90 con 0x7f0dc8071220 2024-09-15T21:48:27.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.769+0000 7f0dcd503640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0dc8157660 0x7f0dc806bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:27.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.769+0000 7f0dc7fff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f0dc80c91d0 0x7f0dc80730c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48080/0 (socket says 172.21.15.100:48080) 2024-09-15T21:48:27.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.769+0000 7f0dc7fff640 1 -- 172.21.15.100:0/2621119527 learned_addr learned my addr 172.21.15.100:0/2621119527 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:27.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.769+0000 7f0dccd02640 1 --2- 172.21.15.100:0/2621119527 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0dc8071220 0x7f0dc8151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:27.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.769+0000 7f0dcd503640 1 -- 172.21.15.100:0/2621119527 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0dc8071220 msgr2=0x7f0dc8151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:27.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.769+0000 7f0dcd503640 1 --2- 172.21.15.100:0/2621119527 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0dc8071220 0x7f0dc8151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:27.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.769+0000 7f0dcd503640 1 -- 172.21.15.100:0/2621119527 >> v1:172.21.15.100:6789/0 conn(0x7f0dc80c91d0 legacy=0x7f0dc80730c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:27.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.769+0000 7f0dc77fe640 1 -- 172.21.15.100:0/2621119527 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2064177864 0 0) 0x7f0dc8157cf0 con 0x7f0dc80c91d0 2024-09-15T21:48:27.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.769+0000 7f0dcd503640 1 -- 172.21.15.100:0/2621119527 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc8158270 con 0x7f0dc8157660 2024-09-15T21:48:27.772 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.769+0000 7f0dccd02640 1 --2- 172.21.15.100:0/2621119527 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0dc8071220 0x7f0dc8151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:48:27.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.770+0000 7f0dcd503640 1 --2- 172.21.15.100:0/2621119527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8157660 0x7f0dc806bbe0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009d90 tx=0x7f0dc002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a381c54eab964c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:27.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.770+0000 7f0dc77fe640 1 -- 172.21.15.100:0/2621119527 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0007740 con 0x7f0dc8157660 2024-09-15T21:48:27.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.770+0000 7f0dc77fe640 1 -- 172.21.15.100:0/2621119527 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dc00078e0 con 0x7f0dc8157660 2024-09-15T21:48:27.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.770+0000 7f0dc77fe640 1 -- 172.21.15.100:0/2621119527 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc0007cb0 con 0x7f0dc8157660 2024-09-15T21:48:27.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.771+0000 7f0dcef66640 1 -- 172.21.15.100:0/2621119527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8157660 msgr2=0x7f0dc806bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:27.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.771+0000 7f0dcef66640 1 --2- 172.21.15.100:0/2621119527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8157660 0x7f0dc806bbe0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009d90 tx=0x7f0dc002a6f0 comp rx=0 tx=0).stop 2024-09-15T21:48:27.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.771+0000 7f0dcef66640 1 -- 172.21.15.100:0/2621119527 shutdown_connections 2024-09-15T21:48:27.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.771+0000 7f0dcef66640 1 --2- 172.21.15.100:0/2621119527 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8157660 0x7f0dc806bbe0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:27.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.771+0000 7f0dcef66640 1 --2- 172.21.15.100:0/2621119527 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0dc8071220 0x7f0dc8151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:27.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.771+0000 7f0dcef66640 1 -- 172.21.15.100:0/2621119527 >> 172.21.15.100:0/2621119527 conn(0x7f0dc80ccbd0 msgr2=0x7f0dc80ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:27.774 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.771+0000 7f0dcef66640 1 -- 172.21.15.100:0/2621119527 shutdown_connections 2024-09-15T21:48:27.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.771+0000 7f0dcef66640 1 -- 172.21.15.100:0/2621119527 wait complete. 2024-09-15T21:48:27.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.772+0000 7f0dcef66640 1 Processor -- start 2024-09-15T21:48:27.775 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.772+0000 7f0dcef66640 1 -- start start 2024-09-15T21:48:27.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dcef66640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8071220 0x7f0dc8146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:27.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dcef66640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc80c91d0 0x7f0dc814e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:27.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dcef66640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0dc8157660 0x7f0dc813e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:27.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dcef66640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc80737d0 con 0x7f0dc8071220 2024-09-15T21:48:27.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dcef66640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0dc8158550 con 0x7f0dc80c91d0 2024-09-15T21:48:27.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dcef66640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0dc8073950 con 0x7f0dc8157660 2024-09-15T21:48:27.777 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dccd02640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8071220 0x7f0dc8146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:27.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dccd02640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8071220 0x7f0dc8146710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53774/0 (socket says 172.21.15.100:53774) 2024-09-15T21:48:27.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dccd02640 1 -- 172.21.15.100:0/1477698119 learned_addr learned my addr 172.21.15.100:0/1477698119 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:27.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dcd503640 1 --2- 172.21.15.100:0/1477698119 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0dc8157660 0x7f0dc813e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:27.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dc7fff640 1 --2- 172.21.15.100:0/1477698119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc80c91d0 0x7f0dc814e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:27.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dccd02640 1 -- 172.21.15.100:0/1477698119 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0dc8157660 msgr2=0x7f0dc813e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:27.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dccd02640 1 --2- 172.21.15.100:0/1477698119 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0dc8157660 0x7f0dc813e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:27.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dccd02640 1 -- 172.21.15.100:0/1477698119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc80c91d0 msgr2=0x7f0dc814e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:27.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dccd02640 1 --2- 172.21.15.100:0/1477698119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc80c91d0 0x7f0dc814e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:27.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dccd02640 1 -- 172.21.15.100:0/1477698119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dc81f2000 con 0x7f0dc8071220 2024-09-15T21:48:27.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dc7fff640 1 --2- 172.21.15.100:0/1477698119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc80c91d0 0x7f0dc814e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:48:27.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dccd02640 1 --2- 172.21.15.100:0/1477698119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8071220 0x7f0dc8146710 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0db800c820 tx=0x7f0db800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:27.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dc57fa640 1 -- 172.21.15.100:0/1477698119 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8015070 con 0x7f0dc8071220 2024-09-15T21:48:27.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.773+0000 7f0dc57fa640 1 -- 172.21.15.100:0/1477698119 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0db8007a30 con 0x7f0dc8071220 2024-09-15T21:48:27.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.774+0000 7f0dc57fa640 1 -- 172.21.15.100:0/1477698119 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db8007de0 con 0x7f0dc8071220 2024-09-15T21:48:27.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.774+0000 7f0dcef66640 1 -- 172.21.15.100:0/1477698119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dc81f22e0 con 0x7f0dc8071220 2024-09-15T21:48:27.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.774+0000 7f0dcef66640 1 -- 172.21.15.100:0/1477698119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dc814df00 con 0x7f0dc8071220 2024-09-15T21:48:27.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.774+0000 7f0dcef66640 1 -- 172.21.15.100:0/1477698119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dc80769a0 con 0x7f0dc8071220 2024-09-15T21:48:27.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.775+0000 7f0dc57fa640 1 -- 172.21.15.100:0/1477698119 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0db8018070 con 0x7f0dc8071220 2024-09-15T21:48:27.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.775+0000 7f0dc57fa640 1 --2- 172.21.15.100:0/1477698119 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0db406c2b0 0x7f0db406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:27.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.775+0000 7f0dc57fa640 1 -- 172.21.15.100:0/1477698119 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5857+0+0 (secure 0 0 0) 0x7f0db808b360 con 0x7f0dc8071220 2024-09-15T21:48:27.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.775+0000 7f0dc7fff640 1 --2- 172.21.15.100:0/1477698119 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0db406c2b0 0x7f0db406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:27.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.776+0000 7f0dc7fff640 1 --2- 172.21.15.100:0/1477698119 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0db406c2b0 0x7f0db406e770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0dc81f5b10 tx=0x7f0dbc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:27.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.780+0000 7f0dc57fa640 1 -- 172.21.15.100:0/1477698119 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0dc80769a0 con 0x7f0dc8071220 2024-09-15T21:48:27.961 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:27.959+0000 7f0dcef66640 1 -- 172.21.15.100:0/1477698119 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 3, "weight": 1.0} v 0) -- 0x7f0dc80c8a00 con 0x7f0dc8071220 2024-09-15T21:48:28.662 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.660+0000 7f0dc57fa640 1 -- 172.21.15.100:0/1477698119 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]=0 set osd.3 primary-affinity to 1 (8655362) v68) ==== 133+0+0 (secure 0 0 0) 0x7f0dc80c8a00 con 0x7f0dc8071220 2024-09-15T21:48:28.662 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-15T21:48:28.665 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.663+0000 7f0dcef66640 1 -- 172.21.15.100:0/1477698119 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0db406c2b0 msgr2=0x7f0db406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.663+0000 7f0dcef66640 1 --2- 172.21.15.100:0/1477698119 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0db406c2b0 0x7f0db406e770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0dc81f5b10 tx=0x7f0dbc009040 comp rx=0 tx=0).stop 2024-09-15T21:48:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.663+0000 7f0dcef66640 1 -- 172.21.15.100:0/1477698119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8071220 msgr2=0x7f0dc8146710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.663+0000 7f0dcef66640 1 --2- 172.21.15.100:0/1477698119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8071220 0x7f0dc8146710 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0db800c820 tx=0x7f0db800cce0 comp rx=0 tx=0).stop 2024-09-15T21:48:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.663+0000 7f0dcef66640 1 -- 172.21.15.100:0/1477698119 shutdown_connections 2024-09-15T21:48:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.663+0000 7f0dcef66640 1 --2- 172.21.15.100:0/1477698119 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0db406c2b0 0x7f0db406e770 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.663+0000 7f0dcef66640 1 --2- 172.21.15.100:0/1477698119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0dc80c91d0 0x7f0dc814e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.663+0000 7f0dcef66640 1 --2- 172.21.15.100:0/1477698119 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0dc8157660 0x7f0dc813e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.663+0000 7f0dcef66640 1 --2- 172.21.15.100:0/1477698119 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0dc8071220 0x7f0dc8146710 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.663+0000 7f0dcef66640 1 -- 172.21.15.100:0/1477698119 >> 172.21.15.100:0/1477698119 conn(0x7f0dc80ccbd0 msgr2=0x7f0dc80753b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.663+0000 7f0dcef66640 1 -- 172.21.15.100:0/1477698119 shutdown_connections 2024-09-15T21:48:28.666 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.663+0000 7f0dcef66640 1 -- 172.21.15.100:0/1477698119 wait complete. 2024-09-15T21:48:28.679 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-15T21:48:28.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.783+0000 7f1a3fdb7640 1 Processor -- start 2024-09-15T21:48:28.785 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.784+0000 7f1a3fdb7640 1 -- start start 2024-09-15T21:48:28.786 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.784+0000 7f1a3fdb7640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1a38155680 0x7f1a3815b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:28.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.784+0000 7f1a3fdb7640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1a380cb910 0x7f1a380cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:28.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.784+0000 7f1a3fdb7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a3815b7f0 con 0x7f1a381552a0 2024-09-15T21:48:28.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.784+0000 7f1a3fdb7640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f1a3815bab0 con 0x7f1a380cb910 2024-09-15T21:48:28.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.784+0000 7f1a3fdb7640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3815bd90 con 0x7f1a38155680 2024-09-15T21:48:28.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.784+0000 7f1a3db53640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1a381552a0 0x7f1a380c5ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48092/0 (socket says 172.21.15.100:48092) 2024-09-15T21:48:28.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.784+0000 7f1a3e354640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1a380cb910 0x7f1a380cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:28.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.785+0000 7f1a3db53640 1 -- 172.21.15.100:0/622039198 learned_addr learned my addr 172.21.15.100:0/622039198 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:28.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.785+0000 7f1a3d352640 1 --2- 172.21.15.100:0/622039198 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1a38155680 0x7f1a3815b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:28.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.785+0000 7f1a3e354640 1 -- 172.21.15.100:0/622039198 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1a38155680 msgr2=0x7f1a3815b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:28.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.785+0000 7f1a3e354640 1 --2- 172.21.15.100:0/622039198 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1a38155680 0x7f1a3815b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:28.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.785+0000 7f1a3e354640 1 -- 172.21.15.100:0/622039198 >> v1:172.21.15.100:6789/0 conn(0x7f1a381552a0 legacy=0x7f1a380c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:28.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.785+0000 7f1a3cb51640 1 -- 172.21.15.100:0/622039198 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1997860866 0 0) 0x7f1a3815b7f0 con 0x7f1a381552a0 2024-09-15T21:48:28.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.785+0000 7f1a3e354640 1 -- 172.21.15.100:0/622039198 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3815c070 con 0x7f1a380cb910 2024-09-15T21:48:28.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.785+0000 7f1a3e354640 1 --2- 172.21.15.100:0/622039198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a380cb910 0x7f1a380cbd30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1a300097a0 tx=0x7f1a3002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1df33d802383f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:28.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.786+0000 7f1a3cb51640 1 -- 172.21.15.100:0/622039198 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30037070 con 0x7f1a380cb910 2024-09-15T21:48:28.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.786+0000 7f1a3cb51640 1 -- 172.21.15.100:0/622039198 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a30032440 con 0x7f1a380cb910 2024-09-15T21:48:28.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.786+0000 7f1a3cb51640 1 -- 172.21.15.100:0/622039198 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30032810 con 0x7f1a380cb910 2024-09-15T21:48:28.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.786+0000 7f1a3fdb7640 1 -- 172.21.15.100:0/622039198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a380cb910 msgr2=0x7f1a380cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:28.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.786+0000 7f1a3fdb7640 1 --2- 172.21.15.100:0/622039198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a380cb910 0x7f1a380cbd30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1a300097a0 tx=0x7f1a3002a210 comp rx=0 tx=0).stop 2024-09-15T21:48:28.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.786+0000 7f1a3fdb7640 1 -- 172.21.15.100:0/622039198 shutdown_connections 2024-09-15T21:48:28.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.786+0000 7f1a3fdb7640 1 --2- 172.21.15.100:0/622039198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a380cb910 0x7f1a380cbd30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:28.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.786+0000 7f1a3fdb7640 1 --2- 172.21.15.100:0/622039198 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1a38155680 0x7f1a3815b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:28.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.786+0000 7f1a3fdb7640 1 -- 172.21.15.100:0/622039198 >> 172.21.15.100:0/622039198 conn(0x7f1a3806d9a0 msgr2=0x7f1a3806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:28.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.786+0000 7f1a3fdb7640 1 -- 172.21.15.100:0/622039198 shutdown_connections 2024-09-15T21:48:28.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.786+0000 7f1a3fdb7640 1 -- 172.21.15.100:0/622039198 wait complete. 2024-09-15T21:48:28.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.787+0000 7f1a3fdb7640 1 Processor -- start 2024-09-15T21:48:28.790 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.788+0000 7f1a3fdb7640 1 -- start start 2024-09-15T21:48:28.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.788+0000 7f1a3fdb7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a380cb910 0x7f1a381503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:28.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.788+0000 7f1a3fdb7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a381552a0 0x7f1a381f4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:28.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.788+0000 7f1a3fdb7640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1a38155680 0x7f1a381e5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:28.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.788+0000 7f1a3fdb7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1a380c61f0 con 0x7f1a380cb910 2024-09-15T21:48:28.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.788+0000 7f1a3fdb7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1a380c6820 con 0x7f1a381552a0 2024-09-15T21:48:28.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.788+0000 7f1a3fdb7640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1a380c6d40 con 0x7f1a38155680 2024-09-15T21:48:28.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.788+0000 7f1a3db53640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a380cb910 0x7f1a381503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:28.792 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.788+0000 7f1a3db53640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a380cb910 0x7f1a381503a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53778/0 (socket says 172.21.15.100:53778) 2024-09-15T21:48:28.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.788+0000 7f1a3db53640 1 -- 172.21.15.100:0/2901616940 learned_addr learned my addr 172.21.15.100:0/2901616940 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:28.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.788+0000 7f1a3d352640 1 --2- 172.21.15.100:0/2901616940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a381552a0 0x7f1a381f4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:28.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.788+0000 7f1a3e354640 1 --2- 172.21.15.100:0/2901616940 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1a38155680 0x7f1a381e5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:28.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.789+0000 7f1a3db53640 1 -- 172.21.15.100:0/2901616940 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1a38155680 msgr2=0x7f1a381e5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:28.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.789+0000 7f1a3db53640 1 --2- 172.21.15.100:0/2901616940 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1a38155680 0x7f1a381e5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:28.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.789+0000 7f1a3db53640 1 -- 172.21.15.100:0/2901616940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a381552a0 msgr2=0x7f1a381f4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:28.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.789+0000 7f1a3db53640 1 --2- 172.21.15.100:0/2901616940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a381552a0 0x7f1a381f4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:28.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.789+0000 7f1a3db53640 1 -- 172.21.15.100:0/2901616940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a381f5590 con 0x7f1a380cb910 2024-09-15T21:48:28.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.789+0000 7f1a3e354640 1 --2- 172.21.15.100:0/2901616940 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1a38155680 0x7f1a381e5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:48:28.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.789+0000 7f1a3db53640 1 --2- 172.21.15.100:0/2901616940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a380cb910 0x7f1a381503a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800c8c0 tx=0x7f1a2800cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:28.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.789+0000 7f1a367fc640 1 -- 172.21.15.100:0/2901616940 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a28018070 con 0x7f1a380cb910 2024-09-15T21:48:28.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.789+0000 7f1a367fc640 1 -- 172.21.15.100:0/2901616940 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2800a640 con 0x7f1a380cb910 2024-09-15T21:48:28.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.789+0000 7f1a3fdb7640 1 -- 172.21.15.100:0/2901616940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a381f5850 con 0x7f1a380cb910 2024-09-15T21:48:28.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.789+0000 7f1a367fc640 1 -- 172.21.15.100:0/2901616940 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2800aa30 con 0x7f1a380cb910 2024-09-15T21:48:28.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.789+0000 7f1a3fdb7640 1 -- 172.21.15.100:0/2901616940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a381500c0 con 0x7f1a380cb910 2024-09-15T21:48:28.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.790+0000 7f1a3fdb7640 1 -- 172.21.15.100:0/2901616940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a3815a3a0 con 0x7f1a380cb910 2024-09-15T21:48:28.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.790+0000 7f1a367fc640 1 -- 172.21.15.100:0/2901616940 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1a2801b070 con 0x7f1a380cb910 2024-09-15T21:48:28.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.791+0000 7f1a367fc640 1 --2- 172.21.15.100:0/2901616940 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1a2406c2b0 0x7f1a2406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:28.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.791+0000 7f1a367fc640 1 -- 172.21.15.100:0/2901616940 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5857+0+0 (secure 0 0 0) 0x7f1a2808abc0 con 0x7f1a380cb910 2024-09-15T21:48:28.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.791+0000 7f1a3d352640 1 --2- 172.21.15.100:0/2901616940 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1a2406c2b0 0x7f1a2406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:28.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.791+0000 7f1a3d352640 1 --2- 172.21.15.100:0/2901616940 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1a2406c2b0 0x7f1a2406e770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1a38150b20 tx=0x7f1a2c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:28.797 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.795+0000 7f1a367fc640 1 -- 172.21.15.100:0/2901616940 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1a3815a3a0 con 0x7f1a380cb910 2024-09-15T21:48:28.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:28.972+0000 7f1a3fdb7640 1 -- 172.21.15.100:0/2901616940 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd reweight", "id": 4, "weight": 1.0} v 0) -- 0x7f1a3815a0c0 con 0x7f1a380cb910 2024-09-15T21:48:29.670 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.668+0000 7f1a367fc640 1 -- 172.21.15.100:0/2901616940 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 4, "weight": 1.0}]=0 reweighted osd.4 to 1 (10000) v69) ==== 113+0+0 (secure 0 0 0) 0x7f1a3815a0c0 con 0x7f1a380cb910 2024-09-15T21:48:29.670 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.4 to 1 (10000) 2024-09-15T21:48:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.671+0000 7f1a3fdb7640 1 -- 172.21.15.100:0/2901616940 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1a2406c2b0 msgr2=0x7f1a2406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.671+0000 7f1a3fdb7640 1 --2- 172.21.15.100:0/2901616940 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1a2406c2b0 0x7f1a2406e770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1a38150b20 tx=0x7f1a2c009190 comp rx=0 tx=0).stop 2024-09-15T21:48:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.671+0000 7f1a3fdb7640 1 -- 172.21.15.100:0/2901616940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a380cb910 msgr2=0x7f1a381503a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.671+0000 7f1a3fdb7640 1 --2- 172.21.15.100:0/2901616940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a380cb910 0x7f1a381503a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1a2800c8c0 tx=0x7f1a2800cd80 comp rx=0 tx=0).stop 2024-09-15T21:48:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.671+0000 7f1a3fdb7640 1 -- 172.21.15.100:0/2901616940 shutdown_connections 2024-09-15T21:48:29.673 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.671+0000 7f1a3fdb7640 1 --2- 172.21.15.100:0/2901616940 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1a2406c2b0 0x7f1a2406e770 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:29.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.671+0000 7f1a3fdb7640 1 --2- 172.21.15.100:0/2901616940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a381552a0 0x7f1a381f4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:29.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.671+0000 7f1a3fdb7640 1 --2- 172.21.15.100:0/2901616940 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1a38155680 0x7f1a381e5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:29.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.671+0000 7f1a3fdb7640 1 --2- 172.21.15.100:0/2901616940 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1a380cb910 0x7f1a381503a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:29.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.671+0000 7f1a3fdb7640 1 -- 172.21.15.100:0/2901616940 >> 172.21.15.100:0/2901616940 conn(0x7f1a3806d9a0 msgr2=0x7f1a38153b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:29.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.671+0000 7f1a3fdb7640 1 -- 172.21.15.100:0/2901616940 shutdown_connections 2024-09-15T21:48:29.674 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.671+0000 7f1a3fdb7640 1 -- 172.21.15.100:0/2901616940 wait complete. 2024-09-15T21:48:29.686 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-15T21:48:29.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.787+0000 7feb923a8640 1 Processor -- start 2024-09-15T21:48:29.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.787+0000 7feb923a8640 1 -- start start 2024-09-15T21:48:29.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.788+0000 7feb923a8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7feb8c152060 0x7feb8c06c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:29.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.788+0000 7feb923a8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb8c0c9290 0x7feb8c0c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:29.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.788+0000 7feb923a8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb8c0c50b0 con 0x7feb8c158900 2024-09-15T21:48:29.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.788+0000 7feb923a8640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7feb8c06bff0 con 0x7feb8c152060 2024-09-15T21:48:29.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.788+0000 7feb923a8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7feb8c071800 con 0x7feb8c0c9290 2024-09-15T21:48:29.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.788+0000 7feb90945640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7feb8c158900 0x7feb8c15b470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48094/0 (socket says 172.21.15.100:48094) 2024-09-15T21:48:29.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.788+0000 7feb90945640 1 -- 172.21.15.100:0/2542098454 learned_addr learned my addr 172.21.15.100:0/2542098454 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:29.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8affd640 1 -- 172.21.15.100:0/2542098454 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1229674553 0 0) 0x7feb8c0c50b0 con 0x7feb8c158900 2024-09-15T21:48:29.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8affd640 1 -- 172.21.15.100:0/2542098454 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb74003520 con 0x7feb8c158900 2024-09-15T21:48:29.793 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8bfff640 1 --2- 172.21.15.100:0/2542098454 >> v2:172.21.15.122:3300/0 conn(0x7feb8c152060 0x7feb8c06c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:29.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8b7fe640 1 --2- 172.21.15.100:0/2542098454 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb8c0c9290 0x7feb8c0c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:29.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8affd640 1 -- 172.21.15.100:0/2542098454 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2245668773 0 0) 0x7feb74003520 con 0x7feb8c158900 2024-09-15T21:48:29.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8affd640 1 -- 172.21.15.100:0/2542098454 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb8c0c9290 msgr2=0x7feb8c0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:29.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8affd640 1 --2- 172.21.15.100:0/2542098454 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb8c0c9290 0x7feb8c0c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:29.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8affd640 1 -- 172.21.15.100:0/2542098454 >> v2:172.21.15.122:3300/0 conn(0x7feb8c152060 msgr2=0x7feb8c06c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:29.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8affd640 1 --2- 172.21.15.100:0/2542098454 >> v2:172.21.15.122:3300/0 conn(0x7feb8c152060 0x7feb8c06c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:29.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8affd640 1 -- 172.21.15.100:0/2542098454 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb8c071ae0 con 0x7feb8c158900 2024-09-15T21:48:29.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8affd640 1 -- 172.21.15.100:0/2542098454 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7feb840019f0 con 0x7feb8c158900 2024-09-15T21:48:29.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8affd640 1 -- 172.21.15.100:0/2542098454 >> v1:172.21.15.100:6789/0 conn(0x7feb8c158900 legacy=0x7feb8c15b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:29.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8affd640 1 --2- 172.21.15.100:0/2542098454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb740039b0 0x7feb74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:29.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8affd640 1 --2- 172.21.15.100:0/2542098454 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb8c0c9290 0x7feb74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:29.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8affd640 1 --2- 172.21.15.100:0/2542098454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb74004d40 0x7feb740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:29.794 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8affd640 1 -- 172.21.15.100:0/2542098454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c06bff0 con 0x7feb740039b0 2024-09-15T21:48:29.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8affd640 1 -- 172.21.15.100:0/2542098454 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c071800 con 0x7feb74004d40 2024-09-15T21:48:29.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8affd640 1 -- 172.21.15.100:0/2542098454 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7feb7400b9c0 con 0x7feb8c0c9290 2024-09-15T21:48:29.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8bfff640 1 --2- 172.21.15.100:0/2542098454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb740039b0 0x7feb74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:29.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8b7fe640 1 --2- 172.21.15.100:0/2542098454 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb8c0c9290 0x7feb74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:29.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb90945640 1 --2- 172.21.15.100:0/2542098454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb74004d40 0x7feb740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:29.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8bfff640 1 -- 172.21.15.100:0/2542098454 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb8c0c9290 msgr2=0x7feb74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:29.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8bfff640 1 --2- 172.21.15.100:0/2542098454 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb8c0c9290 0x7feb74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:29.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8bfff640 1 -- 172.21.15.100:0/2542098454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb74004d40 msgr2=0x7feb740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:29.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8bfff640 1 --2- 172.21.15.100:0/2542098454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb74004d40 0x7feb740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:29.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb8bfff640 1 -- 172.21.15.100:0/2542098454 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb74003520 con 0x7feb740039b0 2024-09-15T21:48:29.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.789+0000 7feb90945640 1 --2- 172.21.15.100:0/2542098454 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb74004d40 0x7feb740051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:48:29.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.790+0000 7feb8bfff640 1 --2- 172.21.15.100:0/2542098454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb740039b0 0x7feb74003dd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009ac0 tx=0x7feb74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a8794078c06c9b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:29.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.790+0000 7feb8affd640 1 -- 172.21.15.100:0/2542098454 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c036070 con 0x7feb740039b0 2024-09-15T21:48:29.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.790+0000 7feb8affd640 1 -- 172.21.15.100:0/2542098454 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb7c031400 con 0x7feb740039b0 2024-09-15T21:48:29.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.790+0000 7feb8affd640 1 -- 172.21.15.100:0/2542098454 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c009690 con 0x7feb740039b0 2024-09-15T21:48:29.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.791+0000 7feb923a8640 1 -- 172.21.15.100:0/2542098454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb740039b0 msgr2=0x7feb74003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:29.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.791+0000 7feb923a8640 1 --2- 172.21.15.100:0/2542098454 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb740039b0 0x7feb74003dd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009ac0 tx=0x7feb74005830 comp rx=0 tx=0).stop 2024-09-15T21:48:29.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.791+0000 7feb90945640 1 -- 172.21.15.100:0/2542098454 reap_dead start 2024-09-15T21:48:29.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.791+0000 7feb923a8640 1 -- 172.21.15.100:0/2542098454 shutdown_connections 2024-09-15T21:48:29.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.791+0000 7feb923a8640 1 -- 172.21.15.100:0/2542098454 >> 172.21.15.100:0/2542098454 conn(0x7feb8c0ccbd0 msgr2=0x7feb8c0ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:29.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.791+0000 7feb923a8640 1 -- 172.21.15.100:0/2542098454 shutdown_connections 2024-09-15T21:48:29.796 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.791+0000 7feb923a8640 1 -- 172.21.15.100:0/2542098454 wait complete. 2024-09-15T21:48:29.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.792+0000 7feb923a8640 1 Processor -- start 2024-09-15T21:48:29.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.792+0000 7feb923a8640 1 -- start start 2024-09-15T21:48:29.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.793+0000 7feb923a8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb740039b0 0x7feb8c148250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:29.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.793+0000 7feb923a8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb8c0caf50 0x7feb8c146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:29.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.793+0000 7feb923a8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb8c0cb390 0x7feb8c147040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:29.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.793+0000 7feb923a8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c072b00 con 0x7feb8c0caf50 2024-09-15T21:48:29.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.793+0000 7feb923a8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c071dc0 con 0x7feb740039b0 2024-09-15T21:48:29.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.793+0000 7feb923a8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7feb8c072c80 con 0x7feb8c0cb390 2024-09-15T21:48:29.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.793+0000 7feb8b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb8c0caf50 0x7feb8c146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:29.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.793+0000 7feb8b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb8c0caf50 0x7feb8c146b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53804/0 (socket says 172.21.15.100:53804) 2024-09-15T21:48:29.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.793+0000 7feb8b7fe640 1 -- 172.21.15.100:0/2259337538 learned_addr learned my addr 172.21.15.100:0/2259337538 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:29.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.793+0000 7feb90945640 1 --2- 172.21.15.100:0/2259337538 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb8c0cb390 0x7feb8c147040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:29.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.793+0000 7feb8bfff640 1 --2- 172.21.15.100:0/2259337538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb740039b0 0x7feb8c148250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:29.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.793+0000 7feb8b7fe640 1 -- 172.21.15.100:0/2259337538 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb8c0cb390 msgr2=0x7feb8c147040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:29.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.793+0000 7feb8b7fe640 1 --2- 172.21.15.100:0/2259337538 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb8c0cb390 0x7feb8c147040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:29.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.793+0000 7feb8b7fe640 1 -- 172.21.15.100:0/2259337538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb740039b0 msgr2=0x7feb8c148250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:29.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.793+0000 7feb8b7fe640 1 --2- 172.21.15.100:0/2259337538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb740039b0 0x7feb8c148250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:29.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.793+0000 7feb8b7fe640 1 -- 172.21.15.100:0/2259337538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb8c149f10 con 0x7feb8c0caf50 2024-09-15T21:48:29.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.793+0000 7feb90945640 1 --2- 172.21.15.100:0/2259337538 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb8c0cb390 0x7feb8c147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:48:29.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.793+0000 7feb8bfff640 1 --2- 172.21.15.100:0/2259337538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb740039b0 0x7feb8c148250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:48:29.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.793+0000 7feb8b7fe640 1 --2- 172.21.15.100:0/2259337538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb8c0caf50 0x7feb8c146b00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7feb800119a0 tx=0x7feb80011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:29.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.794+0000 7feb88ff9640 1 -- 172.21.15.100:0/2259337538 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8000faf0 con 0x7feb8c0caf50 2024-09-15T21:48:29.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.794+0000 7feb88ff9640 1 -- 172.21.15.100:0/2259337538 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb8000fc90 con 0x7feb8c0caf50 2024-09-15T21:48:29.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.794+0000 7feb923a8640 1 -- 172.21.15.100:0/2259337538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb8c14ba30 con 0x7feb8c0caf50 2024-09-15T21:48:29.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.794+0000 7feb88ff9640 1 -- 172.21.15.100:0/2259337538 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb80014440 con 0x7feb8c0caf50 2024-09-15T21:48:29.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.794+0000 7feb923a8640 1 -- 172.21.15.100:0/2259337538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb8c14a360 con 0x7feb8c0caf50 2024-09-15T21:48:29.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.795+0000 7feb923a8640 1 -- 172.21.15.100:0/2259337538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb8c0c7dc0 con 0x7feb8c0caf50 2024-09-15T21:48:29.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.795+0000 7feb88ff9640 1 -- 172.21.15.100:0/2259337538 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7feb80012040 con 0x7feb8c0caf50 2024-09-15T21:48:29.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.795+0000 7feb88ff9640 1 --2- 172.21.15.100:0/2259337538 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7feb7806c2b0 0x7feb7806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:29.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.795+0000 7feb88ff9640 1 -- 172.21.15.100:0/2259337538 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5857+0+0 (secure 0 0 0) 0x7feb8008bb30 con 0x7feb8c0caf50 2024-09-15T21:48:29.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.795+0000 7feb8bfff640 1 --2- 172.21.15.100:0/2259337538 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7feb7806c2b0 0x7feb7806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:29.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.796+0000 7feb8bfff640 1 --2- 172.21.15.100:0/2259337538 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7feb7806c2b0 0x7feb7806e770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7feb7c033000 tx=0x7feb7c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:29.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.800+0000 7feb88ff9640 1 -- 172.21.15.100:0/2259337538 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7feb8c0c7dc0 con 0x7feb8c0caf50 2024-09-15T21:48:29.980 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:29.978+0000 7feb923a8640 1 -- 172.21.15.100:0/2259337538 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 4, "weight": 1.0} v 0) -- 0x7feb8c06c1a0 con 0x7feb8c0caf50 2024-09-15T21:48:30.676 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.674+0000 7feb88ff9640 1 -- 172.21.15.100:0/2259337538 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]=0 set osd.4 primary-affinity to 1 (8655362) v70) ==== 133+0+0 (secure 0 0 0) 0x7feb8c06c1a0 con 0x7feb8c0caf50 2024-09-15T21:48:30.676 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-15T21:48:30.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.677+0000 7feb923a8640 1 -- 172.21.15.100:0/2259337538 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7feb7806c2b0 msgr2=0x7feb7806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:30.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.677+0000 7feb923a8640 1 --2- 172.21.15.100:0/2259337538 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7feb7806c2b0 0x7feb7806e770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7feb7c033000 tx=0x7feb7c008030 comp rx=0 tx=0).stop 2024-09-15T21:48:30.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.677+0000 7feb923a8640 1 -- 172.21.15.100:0/2259337538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb8c0caf50 msgr2=0x7feb8c146b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:30.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.677+0000 7feb923a8640 1 --2- 172.21.15.100:0/2259337538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb8c0caf50 0x7feb8c146b00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7feb800119a0 tx=0x7feb80011e60 comp rx=0 tx=0).stop 2024-09-15T21:48:30.679 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.677+0000 7feb923a8640 1 -- 172.21.15.100:0/2259337538 shutdown_connections 2024-09-15T21:48:30.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.677+0000 7feb923a8640 1 --2- 172.21.15.100:0/2259337538 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7feb7806c2b0 0x7feb7806e770 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:30.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.677+0000 7feb923a8640 1 --2- 172.21.15.100:0/2259337538 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feb8c0cb390 0x7feb8c147040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:30.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.677+0000 7feb923a8640 1 --2- 172.21.15.100:0/2259337538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feb8c0caf50 0x7feb8c146b00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:30.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.677+0000 7feb923a8640 1 --2- 172.21.15.100:0/2259337538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feb740039b0 0x7feb8c148250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:30.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.677+0000 7feb923a8640 1 -- 172.21.15.100:0/2259337538 >> 172.21.15.100:0/2259337538 conn(0x7feb8c0ccbd0 msgr2=0x7feb8c06eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:30.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.677+0000 7feb923a8640 1 -- 172.21.15.100:0/2259337538 shutdown_connections 2024-09-15T21:48:30.680 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.677+0000 7feb923a8640 1 -- 172.21.15.100:0/2259337538 wait complete. 2024-09-15T21:48:30.691 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-15T21:48:30.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.793+0000 7fe4c02d0640 1 Processor -- start 2024-09-15T21:48:30.795 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.793+0000 7fe4c02d0640 1 -- start start 2024-09-15T21:48:30.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4c02d0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe4b8151fc0 0x7fe4b815bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:30.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4c02d0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4b815c280 con 0x7fe4b806d160 2024-09-15T21:48:30.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4c02d0640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4b80c5c00 con 0x7fe4b80739b0 2024-09-15T21:48:30.798 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4c02d0640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b80741f0 con 0x7fe4b8151fc0 2024-09-15T21:48:30.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4be86d640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fe4b806d160 0x7fe4b806d5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48108/0 (socket says 172.21.15.100:48108) 2024-09-15T21:48:30.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4be86d640 1 -- 172.21.15.100:0/1476511108 learned_addr learned my addr 172.21.15.100:0/1476511108 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:30.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4be06c640 1 --2- 172.21.15.100:0/1476511108 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe4b8151fc0 0x7fe4b815bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:30.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4bd06a640 1 -- 172.21.15.100:0/1476511108 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 744570645 0 0) 0x7fe4b0001490 con 0x7fe4b806d160 2024-09-15T21:48:30.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4bd06a640 1 -- 172.21.15.100:0/1476511108 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe4a4003520 con 0x7fe4b806d160 2024-09-15T21:48:30.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4bd06a640 1 -- 172.21.15.100:0/1476511108 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 814320828 0 0) 0x7fe4a4003520 con 0x7fe4b806d160 2024-09-15T21:48:30.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4bd06a640 1 -- 172.21.15.100:0/1476511108 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe4b8151fc0 msgr2=0x7fe4b815bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:30.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4bd06a640 1 --2- 172.21.15.100:0/1476511108 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe4b8151fc0 0x7fe4b815bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:30.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4bd06a640 1 -- 172.21.15.100:0/1476511108 >> v1:172.21.15.122:6789/0 conn(0x7fe4b80739b0 legacy=0x7fe4b80ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:30.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4bd06a640 1 -- 172.21.15.100:0/1476511108 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b815e9e0 con 0x7fe4b806d160 2024-09-15T21:48:30.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4bd06a640 1 -- 172.21.15.100:0/1476511108 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7fe4b0001bc0 con 0x7fe4b806d160 2024-09-15T21:48:30.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4bd06a640 1 -- 172.21.15.100:0/1476511108 >> v1:172.21.15.100:6789/0 conn(0x7fe4b806d160 legacy=0x7fe4b806d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:30.799 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4bd06a640 1 --2- 172.21.15.100:0/1476511108 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe4b8151fc0 0x7fe4a40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:30.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4bd06a640 1 --2- 172.21.15.100:0/1476511108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4a4003f60 0x7fe4a4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:30.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4bd06a640 1 --2- 172.21.15.100:0/1476511108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4a4004d40 0x7fe4a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:30.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4bd06a640 1 -- 172.21.15.100:0/1476511108 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b80741f0 con 0x7fe4a4003f60 2024-09-15T21:48:30.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4bd06a640 1 -- 172.21.15.100:0/1476511108 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a400bb50 con 0x7fe4a4004d40 2024-09-15T21:48:30.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.794+0000 7fe4bd06a640 1 -- 172.21.15.100:0/1476511108 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a400bd50 con 0x7fe4b8151fc0 2024-09-15T21:48:30.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.795+0000 7fe4bd86b640 1 --2- 172.21.15.100:0/1476511108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4a4003f60 0x7fe4a4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:30.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.795+0000 7fe4be06c640 1 --2- 172.21.15.100:0/1476511108 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe4b8151fc0 0x7fe4a40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:30.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.795+0000 7fe4be86d640 1 --2- 172.21.15.100:0/1476511108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4a4004d40 0x7fe4a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:30.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.795+0000 7fe4bd86b640 1 -- 172.21.15.100:0/1476511108 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe4b8151fc0 msgr2=0x7fe4a40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:30.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.795+0000 7fe4bd86b640 1 --2- 172.21.15.100:0/1476511108 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe4b8151fc0 0x7fe4a40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:30.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.795+0000 7fe4bd86b640 1 -- 172.21.15.100:0/1476511108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4a4004d40 msgr2=0x7fe4a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:30.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.795+0000 7fe4bd86b640 1 --2- 172.21.15.100:0/1476511108 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4a4004d40 0x7fe4a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:30.800 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.795+0000 7fe4bd86b640 1 -- 172.21.15.100:0/1476511108 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a4003520 con 0x7fe4a4003f60 2024-09-15T21:48:30.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.795+0000 7fe4bd86b640 1 --2- 172.21.15.100:0/1476511108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4a4003f60 0x7fe4a4004800 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8001720 tx=0x7fe4a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2bc097f233a245c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:30.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.796+0000 7fe4bd06a640 1 -- 172.21.15.100:0/1476511108 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8001370 con 0x7fe4a4003f60 2024-09-15T21:48:30.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.796+0000 7fe4bd06a640 1 -- 172.21.15.100:0/1476511108 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a80052d0 con 0x7fe4a4003f60 2024-09-15T21:48:30.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.796+0000 7fe4bd06a640 1 -- 172.21.15.100:0/1476511108 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a802abc0 con 0x7fe4a4003f60 2024-09-15T21:48:30.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.796+0000 7fe4c02d0640 1 -- 172.21.15.100:0/1476511108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4a4003f60 msgr2=0x7fe4a4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:30.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.796+0000 7fe4c02d0640 1 --2- 172.21.15.100:0/1476511108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4a4003f60 0x7fe4a4004800 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8001720 tx=0x7fe4a4005830 comp rx=0 tx=0).stop 2024-09-15T21:48:30.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.796+0000 7fe4be86d640 1 -- 172.21.15.100:0/1476511108 reap_dead start 2024-09-15T21:48:30.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.796+0000 7fe4c02d0640 1 -- 172.21.15.100:0/1476511108 shutdown_connections 2024-09-15T21:48:30.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.796+0000 7fe4c02d0640 1 -- 172.21.15.100:0/1476511108 >> 172.21.15.100:0/1476511108 conn(0x7fe4b806f990 msgr2=0x7fe4b8070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:30.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.796+0000 7fe4c02d0640 1 -- 172.21.15.100:0/1476511108 shutdown_connections 2024-09-15T21:48:30.801 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.796+0000 7fe4c02d0640 1 -- 172.21.15.100:0/1476511108 wait complete. 2024-09-15T21:48:30.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.797+0000 7fe4c02d0640 1 Processor -- start 2024-09-15T21:48:30.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.797+0000 7fe4c02d0640 1 -- start start 2024-09-15T21:48:30.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.798+0000 7fe4c02d0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe4a4003f60 0x7fe4b80cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:30.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.798+0000 7fe4c02d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4b815aa80 0x7fe4b81f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:30.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.798+0000 7fe4c02d0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4b80cc950 0x7fe4b80ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:30.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.798+0000 7fe4c02d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b815f280 con 0x7fe4b815aa80 2024-09-15T21:48:30.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.798+0000 7fe4c02d0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b815efa0 con 0x7fe4b80cc950 2024-09-15T21:48:30.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.798+0000 7fe4c02d0640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b81523a0 con 0x7fe4a4003f60 2024-09-15T21:48:30.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.798+0000 7fe4bd86b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4b815aa80 0x7fe4b81f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:30.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.798+0000 7fe4bd86b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4b815aa80 0x7fe4b81f1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53824/0 (socket says 172.21.15.100:53824) 2024-09-15T21:48:30.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.798+0000 7fe4bd86b640 1 -- 172.21.15.100:0/2090950842 learned_addr learned my addr 172.21.15.100:0/2090950842 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:30.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.798+0000 7fe4be06c640 1 --2- 172.21.15.100:0/2090950842 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe4a4003f60 0x7fe4b80cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:30.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.798+0000 7fe4be86d640 1 --2- 172.21.15.100:0/2090950842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4b80cc950 0x7fe4b80ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:30.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.798+0000 7fe4bd86b640 1 -- 172.21.15.100:0/2090950842 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe4a4003f60 msgr2=0x7fe4b80cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:30.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.798+0000 7fe4bd86b640 1 --2- 172.21.15.100:0/2090950842 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe4a4003f60 0x7fe4b80cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:30.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.798+0000 7fe4bd86b640 1 -- 172.21.15.100:0/2090950842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4b80cc950 msgr2=0x7fe4b80ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:30.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.798+0000 7fe4bd86b640 1 --2- 172.21.15.100:0/2090950842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4b80cc950 0x7fe4b80ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:30.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.798+0000 7fe4bd86b640 1 -- 172.21.15.100:0/2090950842 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b81f2350 con 0x7fe4b815aa80 2024-09-15T21:48:30.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.799+0000 7fe4bd86b640 1 --2- 172.21.15.100:0/2090950842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4b815aa80 0x7fe4b81f1e10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe4a802a570 tx=0x7fe4a8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:30.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.799+0000 7fe4aeffd640 1 -- 172.21.15.100:0/2090950842 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8040020 con 0x7fe4b815aa80 2024-09-15T21:48:30.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.799+0000 7fe4aeffd640 1 -- 172.21.15.100:0/2090950842 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a802be50 con 0x7fe4b815aa80 2024-09-15T21:48:30.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.799+0000 7fe4c02d0640 1 -- 172.21.15.100:0/2090950842 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b815c520 con 0x7fe4b815aa80 2024-09-15T21:48:30.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.799+0000 7fe4aeffd640 1 -- 172.21.15.100:0/2090950842 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8007a30 con 0x7fe4b815aa80 2024-09-15T21:48:30.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.799+0000 7fe4c02d0640 1 -- 172.21.15.100:0/2090950842 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b815cae0 con 0x7fe4b815aa80 2024-09-15T21:48:30.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.800+0000 7fe4c02d0640 1 -- 172.21.15.100:0/2090950842 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b8073db0 con 0x7fe4b815aa80 2024-09-15T21:48:30.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.800+0000 7fe4aeffd640 1 -- 172.21.15.100:0/2090950842 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fe4a8043020 con 0x7fe4b815aa80 2024-09-15T21:48:30.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.801+0000 7fe4aeffd640 1 --2- 172.21.15.100:0/2090950842 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe49c06c2e0 0x7fe49c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:30.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.801+0000 7fe4aeffd640 1 -- 172.21.15.100:0/2090950842 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5857+0+0 (secure 0 0 0) 0x7fe4a80ab360 con 0x7fe4b815aa80 2024-09-15T21:48:30.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.801+0000 7fe4be06c640 1 --2- 172.21.15.100:0/2090950842 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe49c06c2e0 0x7fe49c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:30.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.801+0000 7fe4be06c640 1 --2- 172.21.15.100:0/2090950842 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe49c06c2e0 0x7fe49c06e7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4009bc0 tx=0x7fe4b4009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:30.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.805+0000 7fe4aeffd640 1 -- 172.21.15.100:0/2090950842 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fe4a804a030 con 0x7fe4b815aa80 2024-09-15T21:48:30.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:30.980+0000 7fe4c02d0640 1 -- 172.21.15.100:0/2090950842 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd reweight", "id": 6, "weight": 1.0} v 0) -- 0x7fe4b8158c70 con 0x7fe4b815aa80 2024-09-15T21:48:31.684 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.680+0000 7fe4aeffd640 1 -- 172.21.15.100:0/2090950842 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 6, "weight": 1.0}]=0 reweighted osd.6 to 1 (10000) v71) ==== 113+0+0 (secure 0 0 0) 0x7fe4a8079a60 con 0x7fe4b815aa80 2024-09-15T21:48:31.684 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.6 to 1 (10000) 2024-09-15T21:48:31.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.684+0000 7fe4c02d0640 1 -- 172.21.15.100:0/2090950842 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe49c06c2e0 msgr2=0x7fe49c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:31.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.684+0000 7fe4c02d0640 1 --2- 172.21.15.100:0/2090950842 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe49c06c2e0 0x7fe49c06e7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4009bc0 tx=0x7fe4b4009190 comp rx=0 tx=0).stop 2024-09-15T21:48:31.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.684+0000 7fe4c02d0640 1 -- 172.21.15.100:0/2090950842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4b815aa80 msgr2=0x7fe4b81f1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:31.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.684+0000 7fe4c02d0640 1 --2- 172.21.15.100:0/2090950842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4b815aa80 0x7fe4b81f1e10 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe4a802a570 tx=0x7fe4a8001120 comp rx=0 tx=0).stop 2024-09-15T21:48:31.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.684+0000 7fe4c02d0640 1 -- 172.21.15.100:0/2090950842 shutdown_connections 2024-09-15T21:48:31.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.684+0000 7fe4c02d0640 1 --2- 172.21.15.100:0/2090950842 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fe49c06c2e0 0x7fe49c06e7a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:31.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.684+0000 7fe4c02d0640 1 --2- 172.21.15.100:0/2090950842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4b80cc950 0x7fe4b80ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:31.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.684+0000 7fe4c02d0640 1 --2- 172.21.15.100:0/2090950842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4b815aa80 0x7fe4b81f1e10 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:31.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.684+0000 7fe4c02d0640 1 --2- 172.21.15.100:0/2090950842 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fe4a4003f60 0x7fe4b80cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:31.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.684+0000 7fe4c02d0640 1 -- 172.21.15.100:0/2090950842 >> 172.21.15.100:0/2090950842 conn(0x7fe4b806f990 msgr2=0x7fe4b80ceae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:31.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.684+0000 7fe4c02d0640 1 -- 172.21.15.100:0/2090950842 shutdown_connections 2024-09-15T21:48:31.686 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.684+0000 7fe4c02d0640 1 -- 172.21.15.100:0/2090950842 wait complete. 2024-09-15T21:48:31.699 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-15T21:48:31.802 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.801+0000 7efe26c8d640 1 Processor -- start 2024-09-15T21:48:31.803 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.801+0000 7efe26c8d640 1 -- start start 2024-09-15T21:48:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.801+0000 7efe26c8d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efe201552a0 0x7efe2015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.801+0000 7efe26c8d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efe20155680 0x7efe200c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.801+0000 7efe26c8d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7efe200c6020 0x7efe200cb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.801+0000 7efe26c8d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7efe2015b7f0 con 0x7efe20155680 2024-09-15T21:48:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.801+0000 7efe26c8d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7efe2015bab0 con 0x7efe200c6020 2024-09-15T21:48:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.801+0000 7efe26c8d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7efe2015bd90 con 0x7efe201552a0 2024-09-15T21:48:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.801+0000 7efe1ffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efe20155680 0x7efe200c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:31.805 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.801+0000 7efe1ffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7efe20155680 0x7efe200c5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53840/0 (socket says 172.21.15.100:53840) 2024-09-15T21:48:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.802+0000 7efe1ffff640 1 -- 172.21.15.100:0/2089980195 learned_addr learned my addr 172.21.15.100:0/2089980195 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.802+0000 7efe24a29640 1 --2- 172.21.15.100:0/2089980195 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efe201552a0 0x7efe2015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.802+0000 7efe2522a640 1 --2- 172.21.15.100:0/2089980195 >> v2:172.21.15.122:3300/0 conn(0x7efe200c6020 0x7efe200cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.802+0000 7efe1ffff640 1 -- 172.21.15.100:0/2089980195 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efe201552a0 msgr2=0x7efe2015b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.802+0000 7efe1ffff640 1 --2- 172.21.15.100:0/2089980195 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efe201552a0 0x7efe2015b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.802+0000 7efe1ffff640 1 -- 172.21.15.100:0/2089980195 >> v2:172.21.15.122:3300/0 conn(0x7efe200c6020 msgr2=0x7efe200cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.802+0000 7efe1ffff640 1 --2- 172.21.15.100:0/2089980195 >> v2:172.21.15.122:3300/0 conn(0x7efe200c6020 0x7efe200cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.802+0000 7efe1ffff640 1 -- 172.21.15.100:0/2089980195 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2015c070 con 0x7efe20155680 2024-09-15T21:48:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.802+0000 7efe1ffff640 1 --2- 172.21.15.100:0/2089980195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe20155680 0x7efe200c5ae0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efe2006e550 tx=0x7efe0c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d09d70a4e16e64 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.803+0000 7efe1f7fe640 1 -- 172.21.15.100:0/2089980195 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c037070 con 0x7efe20155680 2024-09-15T21:48:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.803+0000 7efe1f7fe640 1 -- 172.21.15.100:0/2089980195 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe0c032440 con 0x7efe20155680 2024-09-15T21:48:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.803+0000 7efe1f7fe640 1 -- 172.21.15.100:0/2089980195 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c032810 con 0x7efe20155680 2024-09-15T21:48:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.803+0000 7efe26c8d640 1 -- 172.21.15.100:0/2089980195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe20155680 msgr2=0x7efe200c5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:31.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.803+0000 7efe26c8d640 1 --2- 172.21.15.100:0/2089980195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe20155680 0x7efe200c5ae0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efe2006e550 tx=0x7efe0c02a270 comp rx=0 tx=0).stop 2024-09-15T21:48:31.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.803+0000 7efe26c8d640 1 -- 172.21.15.100:0/2089980195 shutdown_connections 2024-09-15T21:48:31.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.803+0000 7efe26c8d640 1 --2- 172.21.15.100:0/2089980195 >> v2:172.21.15.122:3300/0 conn(0x7efe200c6020 0x7efe200cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:31.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.803+0000 7efe26c8d640 1 --2- 172.21.15.100:0/2089980195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe20155680 0x7efe200c5ae0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:31.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.803+0000 7efe26c8d640 1 --2- 172.21.15.100:0/2089980195 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efe201552a0 0x7efe2015b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:31.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.803+0000 7efe26c8d640 1 -- 172.21.15.100:0/2089980195 >> 172.21.15.100:0/2089980195 conn(0x7efe2006d9a0 msgr2=0x7efe2006ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:31.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.803+0000 7efe26c8d640 1 -- 172.21.15.100:0/2089980195 shutdown_connections 2024-09-15T21:48:31.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.803+0000 7efe26c8d640 1 -- 172.21.15.100:0/2089980195 wait complete. 2024-09-15T21:48:31.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.804+0000 7efe26c8d640 1 Processor -- start 2024-09-15T21:48:31.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.804+0000 7efe26c8d640 1 -- start start 2024-09-15T21:48:31.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.805+0000 7efe26c8d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efe200c6020 0x7efe200b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:31.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.805+0000 7efe26c8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe201552a0 0x7efe201f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:31.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.805+0000 7efe26c8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe20155680 0x7efe2014e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:31.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.805+0000 7efe26c8d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7efe200cbe50 con 0x7efe20155680 2024-09-15T21:48:31.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.805+0000 7efe26c8d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efe200c6820 con 0x7efe201552a0 2024-09-15T21:48:31.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.805+0000 7efe26c8d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7efe200cbfd0 con 0x7efe200c6020 2024-09-15T21:48:31.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.805+0000 7efe2522a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe20155680 0x7efe2014e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:31.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.805+0000 7efe2522a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe20155680 0x7efe2014e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53842/0 (socket says 172.21.15.100:53842) 2024-09-15T21:48:31.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.805+0000 7efe2522a640 1 -- 172.21.15.100:0/719355747 learned_addr learned my addr 172.21.15.100:0/719355747 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:31.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.805+0000 7efe24a29640 1 --2- 172.21.15.100:0/719355747 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efe200c6020 0x7efe200b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:31.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.805+0000 7efe1ffff640 1 --2- 172.21.15.100:0/719355747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe201552a0 0x7efe201f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:31.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.806+0000 7efe2522a640 1 -- 172.21.15.100:0/719355747 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efe200c6020 msgr2=0x7efe200b5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:31.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.806+0000 7efe2522a640 1 --2- 172.21.15.100:0/719355747 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efe200c6020 0x7efe200b5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:31.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.806+0000 7efe2522a640 1 -- 172.21.15.100:0/719355747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe201552a0 msgr2=0x7efe201f56d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:31.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.806+0000 7efe2522a640 1 --2- 172.21.15.100:0/719355747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe201552a0 0x7efe201f56d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:31.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.806+0000 7efe2522a640 1 -- 172.21.15.100:0/719355747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe20159340 con 0x7efe20155680 2024-09-15T21:48:31.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.806+0000 7efe2522a640 1 --2- 172.21.15.100:0/719355747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe20155680 0x7efe2014e370 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efe18007e90 tx=0x7efe1800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:31.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.806+0000 7efe1d7fa640 1 -- 172.21.15.100:0/719355747 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe18014420 con 0x7efe20155680 2024-09-15T21:48:31.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.806+0000 7efe1d7fa640 1 -- 172.21.15.100:0/719355747 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe18019070 con 0x7efe20155680 2024-09-15T21:48:31.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.806+0000 7efe26c8d640 1 -- 172.21.15.100:0/719355747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe20159600 con 0x7efe20155680 2024-09-15T21:48:31.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.806+0000 7efe1d7fa640 1 -- 172.21.15.100:0/719355747 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe18014820 con 0x7efe20155680 2024-09-15T21:48:31.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.806+0000 7efe26c8d640 1 -- 172.21.15.100:0/719355747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe20159bc0 con 0x7efe20155680 2024-09-15T21:48:31.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.807+0000 7efe26c8d640 1 -- 172.21.15.100:0/719355747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe200cb910 con 0x7efe20155680 2024-09-15T21:48:31.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.807+0000 7efe1d7fa640 1 -- 172.21.15.100:0/719355747 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7efe180149c0 con 0x7efe20155680 2024-09-15T21:48:31.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.808+0000 7efe1d7fa640 1 --2- 172.21.15.100:0/719355747 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7efe0406c2b0 0x7efe0406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:31.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.808+0000 7efe1d7fa640 1 -- 172.21.15.100:0/719355747 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5857+0+0 (secure 0 0 0) 0x7efe1808bc50 con 0x7efe20155680 2024-09-15T21:48:31.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.808+0000 7efe24a29640 1 --2- 172.21.15.100:0/719355747 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7efe0406c2b0 0x7efe0406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:31.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.808+0000 7efe24a29640 1 --2- 172.21.15.100:0/719355747 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7efe0406c2b0 0x7efe0406e770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efe14006fd0 tx=0x7efe14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:31.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.812+0000 7efe1d7fa640 1 -- 172.21.15.100:0/719355747 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7efe18091020 con 0x7efe20155680 2024-09-15T21:48:31.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:31.989+0000 7efe26c8d640 1 -- 172.21.15.100:0/719355747 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 6, "weight": 1.0} v 0) -- 0x7efe200b39e0 con 0x7efe20155680 2024-09-15T21:48:32.695 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.693+0000 7efe1d7fa640 1 -- 172.21.15.100:0/719355747 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]=0 set osd.6 primary-affinity to 1 (8655362) v72) ==== 133+0+0 (secure 0 0 0) 0x7efe18059c40 con 0x7efe20155680 2024-09-15T21:48:32.695 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-15T21:48:32.698 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.696+0000 7efe26c8d640 1 -- 172.21.15.100:0/719355747 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7efe0406c2b0 msgr2=0x7efe0406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.696+0000 7efe26c8d640 1 --2- 172.21.15.100:0/719355747 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7efe0406c2b0 0x7efe0406e770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efe14006fd0 tx=0x7efe14008040 comp rx=0 tx=0).stop 2024-09-15T21:48:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.696+0000 7efe26c8d640 1 -- 172.21.15.100:0/719355747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe20155680 msgr2=0x7efe2014e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.696+0000 7efe26c8d640 1 --2- 172.21.15.100:0/719355747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe20155680 0x7efe2014e370 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efe18007e90 tx=0x7efe1800e4b0 comp rx=0 tx=0).stop 2024-09-15T21:48:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.696+0000 7efe26c8d640 1 -- 172.21.15.100:0/719355747 shutdown_connections 2024-09-15T21:48:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.696+0000 7efe26c8d640 1 --2- 172.21.15.100:0/719355747 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7efe0406c2b0 0x7efe0406e770 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.696+0000 7efe26c8d640 1 --2- 172.21.15.100:0/719355747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe201552a0 0x7efe201f56d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.696+0000 7efe26c8d640 1 --2- 172.21.15.100:0/719355747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7efe20155680 0x7efe2014e370 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.696+0000 7efe26c8d640 1 --2- 172.21.15.100:0/719355747 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7efe200c6020 0x7efe200b5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.696+0000 7efe26c8d640 1 -- 172.21.15.100:0/719355747 >> 172.21.15.100:0/719355747 conn(0x7efe2006d9a0 msgr2=0x7efe2014fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.696+0000 7efe26c8d640 1 -- 172.21.15.100:0/719355747 shutdown_connections 2024-09-15T21:48:32.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.696+0000 7efe26c8d640 1 -- 172.21.15.100:0/719355747 wait complete. 2024-09-15T21:48:32.711 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-15T21:48:32.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.813+0000 7fb6f2e9b640 1 Processor -- start 2024-09-15T21:48:32.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.813+0000 7fb6f2e9b640 1 -- start start 2024-09-15T21:48:32.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.814+0000 7fb6f2e9b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb6ec1552b0 0x7fb6ec06c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:32.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.814+0000 7fb6f2e9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb6ec0c39c0 0x7fb6ec06c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:32.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.814+0000 7fb6f2e9b640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6ec0c7fe0 con 0x7fb6ec155690 2024-09-15T21:48:32.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.814+0000 7fb6f2e9b640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fb6ec0c8540 con 0x7fb6ec0c39c0 2024-09-15T21:48:32.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.814+0000 7fb6f2e9b640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fb6ec0c8280 con 0x7fb6ec1552b0 2024-09-15T21:48:32.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.814+0000 7fb6f1438640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb6ec0c39c0 0x7fb6ec06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:32.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.814+0000 7fb6f1438640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fb6ec0c39c0 0x7fb6ec06c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53846/0 (socket says 172.21.15.100:53846) 2024-09-15T21:48:32.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.814+0000 7fb6f1438640 1 -- 172.21.15.100:0/685343068 learned_addr learned my addr 172.21.15.100:0/685343068 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:32.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.814+0000 7fb6f0c37640 1 --2- 172.21.15.100:0/685343068 >> v2:172.21.15.122:3300/0 conn(0x7fb6ec1552b0 0x7fb6ec06c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:32.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.814+0000 7fb6f1438640 1 -- 172.21.15.100:0/685343068 >> v2:172.21.15.122:3300/0 conn(0x7fb6ec1552b0 msgr2=0x7fb6ec06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:32.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.814+0000 7fb6f1438640 1 --2- 172.21.15.100:0/685343068 >> v2:172.21.15.122:3300/0 conn(0x7fb6ec1552b0 0x7fb6ec06c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:32.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.814+0000 7fb6f1438640 1 -- 172.21.15.100:0/685343068 >> v1:172.21.15.122:6789/0 conn(0x7fb6ec155690 legacy=0x7fb6ec15b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:32.817 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.814+0000 7fb6f1438640 1 -- 172.21.15.100:0/685343068 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ec0c8a70 con 0x7fb6ec0c39c0 2024-09-15T21:48:32.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.815+0000 7fb6f1438640 1 --2- 172.21.15.100:0/685343068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ec0c39c0 0x7fb6ec06c920 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec06e500 tx=0x7fb6e402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d357be60fdcc1450 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:32.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.815+0000 7fb6e37fe640 1 -- 172.21.15.100:0/685343068 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4036070 con 0x7fb6ec0c39c0 2024-09-15T21:48:32.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.815+0000 7fb6e37fe640 1 -- 172.21.15.100:0/685343068 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e4031600 con 0x7fb6ec0c39c0 2024-09-15T21:48:32.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.815+0000 7fb6e37fe640 1 -- 172.21.15.100:0/685343068 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e40319d0 con 0x7fb6ec0c39c0 2024-09-15T21:48:32.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.815+0000 7fb6f2e9b640 1 -- 172.21.15.100:0/685343068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ec0c39c0 msgr2=0x7fb6ec06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:32.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.815+0000 7fb6f2e9b640 1 --2- 172.21.15.100:0/685343068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ec0c39c0 0x7fb6ec06c920 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec06e500 tx=0x7fb6e402a6a0 comp rx=0 tx=0).stop 2024-09-15T21:48:32.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.816+0000 7fb6f2e9b640 1 -- 172.21.15.100:0/685343068 shutdown_connections 2024-09-15T21:48:32.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.816+0000 7fb6f2e9b640 1 --2- 172.21.15.100:0/685343068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ec0c39c0 0x7fb6ec06c920 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:32.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.816+0000 7fb6f2e9b640 1 --2- 172.21.15.100:0/685343068 >> v2:172.21.15.122:3300/0 conn(0x7fb6ec1552b0 0x7fb6ec06c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:32.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.816+0000 7fb6f2e9b640 1 -- 172.21.15.100:0/685343068 >> 172.21.15.100:0/685343068 conn(0x7fb6ec0ccbd0 msgr2=0x7fb6ec0ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:32.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.816+0000 7fb6f2e9b640 1 -- 172.21.15.100:0/685343068 shutdown_connections 2024-09-15T21:48:32.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.816+0000 7fb6f2e9b640 1 -- 172.21.15.100:0/685343068 wait complete. 2024-09-15T21:48:32.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.817+0000 7fb6f2e9b640 1 Processor -- start 2024-09-15T21:48:32.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.817+0000 7fb6f2e9b640 1 -- start start 2024-09-15T21:48:32.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6f2e9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ec0c39c0 0x7fb6ec1f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:32.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6f2e9b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb6ec1552b0 0x7fb6ec151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:32.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6f2e9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6ec155690 0x7fb6ec14cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:32.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6f2e9b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec06bec0 con 0x7fb6ec0c39c0 2024-09-15T21:48:32.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6f2e9b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb6ec06bbe0 con 0x7fb6ec155690 2024-09-15T21:48:32.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6f2e9b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fb6ec0c9460 con 0x7fb6ec1552b0 2024-09-15T21:48:32.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6f0c37640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ec0c39c0 0x7fb6ec1f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:32.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6f0c37640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ec0c39c0 0x7fb6ec1f1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53854/0 (socket says 172.21.15.100:53854) 2024-09-15T21:48:32.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6f0c37640 1 -- 172.21.15.100:0/2001982947 learned_addr learned my addr 172.21.15.100:0/2001982947 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:32.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6e3fff640 1 --2- 172.21.15.100:0/2001982947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb6ec1552b0 0x7fb6ec151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:32.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6f1438640 1 --2- 172.21.15.100:0/2001982947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6ec155690 0x7fb6ec14cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:32.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6f0c37640 1 -- 172.21.15.100:0/2001982947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb6ec1552b0 msgr2=0x7fb6ec151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:32.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6f0c37640 1 --2- 172.21.15.100:0/2001982947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb6ec1552b0 0x7fb6ec151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:32.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6f0c37640 1 -- 172.21.15.100:0/2001982947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6ec155690 msgr2=0x7fb6ec14cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:32.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6f0c37640 1 --2- 172.21.15.100:0/2001982947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6ec155690 0x7fb6ec14cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:32.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6f0c37640 1 -- 172.21.15.100:0/2001982947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6ec159150 con 0x7fb6ec0c39c0 2024-09-15T21:48:32.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6f0c37640 1 --2- 172.21.15.100:0/2001982947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ec0c39c0 0x7fb6ec1f1d10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800e9e0 tx=0x7fb6e800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:32.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6e17fa640 1 -- 172.21.15.100:0/2001982947 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e8017070 con 0x7fb6ec0c39c0 2024-09-15T21:48:32.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6e17fa640 1 -- 172.21.15.100:0/2001982947 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6e800c9d0 con 0x7fb6ec0c39c0 2024-09-15T21:48:32.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6e17fa640 1 -- 172.21.15.100:0/2001982947 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800cdc0 con 0x7fb6ec0c39c0 2024-09-15T21:48:32.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6f2e9b640 1 -- 172.21.15.100:0/2001982947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6ec151a90 con 0x7fb6ec0c39c0 2024-09-15T21:48:32.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.818+0000 7fb6f2e9b640 1 -- 172.21.15.100:0/2001982947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6ec1596d0 con 0x7fb6ec0c39c0 2024-09-15T21:48:32.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.819+0000 7fb6f2e9b640 1 -- 172.21.15.100:0/2001982947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6ec158df0 con 0x7fb6ec0c39c0 2024-09-15T21:48:32.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.820+0000 7fb6e17fa640 1 -- 172.21.15.100:0/2001982947 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fb6e801a070 con 0x7fb6ec0c39c0 2024-09-15T21:48:32.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.820+0000 7fb6e17fa640 1 --2- 172.21.15.100:0/2001982947 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb6d806c2b0 0x7fb6d806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:32.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.820+0000 7fb6e3fff640 1 --2- 172.21.15.100:0/2001982947 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb6d806c2b0 0x7fb6d806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:32.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.820+0000 7fb6e17fa640 1 -- 172.21.15.100:0/2001982947 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5857+0+0 (secure 0 0 0) 0x7fb6e808b7e0 con 0x7fb6ec0c39c0 2024-09-15T21:48:32.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.821+0000 7fb6e3fff640 1 --2- 172.21.15.100:0/2001982947 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb6d806c2b0 0x7fb6d806e770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec14dcd0 tx=0x7fb6dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:32.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.825+0000 7fb6e17fa640 1 -- 172.21.15.100:0/2001982947 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fb6ec158df0 con 0x7fb6ec0c39c0 2024-09-15T21:48:33.001 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:32.999+0000 7fb6f2e9b640 1 -- 172.21.15.100:0/2001982947 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd reweight", "id": 7, "weight": 1.0} v 0) -- 0x7fb6ec0c5630 con 0x7fb6ec0c39c0 2024-09-15T21:48:33.697 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.695+0000 7fb6e17fa640 1 -- 172.21.15.100:0/2001982947 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 7, "weight": 1.0}]=0 reweighted osd.7 to 1 (10000) v73) ==== 113+0+0 (secure 0 0 0) 0x7fb6ec0c5630 con 0x7fb6ec0c39c0 2024-09-15T21:48:33.697 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.7 to 1 (10000) 2024-09-15T21:48:33.699 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.697+0000 7fb6f2e9b640 1 -- 172.21.15.100:0/2001982947 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb6d806c2b0 msgr2=0x7fb6d806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:33.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.697+0000 7fb6f2e9b640 1 --2- 172.21.15.100:0/2001982947 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb6d806c2b0 0x7fb6d806e770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec14dcd0 tx=0x7fb6dc009040 comp rx=0 tx=0).stop 2024-09-15T21:48:33.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.697+0000 7fb6f2e9b640 1 -- 172.21.15.100:0/2001982947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ec0c39c0 msgr2=0x7fb6ec1f1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:33.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.697+0000 7fb6f2e9b640 1 --2- 172.21.15.100:0/2001982947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ec0c39c0 0x7fb6ec1f1d10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800e9e0 tx=0x7fb6e800eea0 comp rx=0 tx=0).stop 2024-09-15T21:48:33.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.697+0000 7fb6f2e9b640 1 -- 172.21.15.100:0/2001982947 shutdown_connections 2024-09-15T21:48:33.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.697+0000 7fb6f2e9b640 1 --2- 172.21.15.100:0/2001982947 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fb6d806c2b0 0x7fb6d806e770 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:33.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.698+0000 7fb6f2e9b640 1 --2- 172.21.15.100:0/2001982947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6ec155690 0x7fb6ec14cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:33.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.698+0000 7fb6f2e9b640 1 --2- 172.21.15.100:0/2001982947 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fb6ec1552b0 0x7fb6ec151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:33.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.698+0000 7fb6f2e9b640 1 --2- 172.21.15.100:0/2001982947 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6ec0c39c0 0x7fb6ec1f1d10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:33.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.698+0000 7fb6f2e9b640 1 -- 172.21.15.100:0/2001982947 >> 172.21.15.100:0/2001982947 conn(0x7fb6ec0ccbd0 msgr2=0x7fb6ec075220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:33.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.698+0000 7fb6f2e9b640 1 -- 172.21.15.100:0/2001982947 shutdown_connections 2024-09-15T21:48:33.700 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.698+0000 7fb6f2e9b640 1 -- 172.21.15.100:0/2001982947 wait complete. 2024-09-15T21:48:33.712 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-15T21:48:33.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.812+0000 7f2495b33640 1 Processor -- start 2024-09-15T21:48:33.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.812+0000 7f2495b33640 1 -- start start 2024-09-15T21:48:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.812+0000 7f2495b33640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f24900cef20 0x7f24900c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.812+0000 7f2495b33640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f24900c9b30 0x7f2490152a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.812+0000 7f2495b33640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24900c83e0 con 0x7f24900ceb40 2024-09-15T21:48:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.812+0000 7f2495b33640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f24900c60e0 con 0x7f24900cef20 2024-09-15T21:48:33.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.812+0000 7f2495b33640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f24900c86c0 con 0x7f24900c9b30 2024-09-15T21:48:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.813+0000 7f2487fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f24900cef20 0x7f24900c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.813+0000 7f248ffff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f24900c9b30 0x7f2490152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.813+0000 7f248ffff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f24900c9b30 0x7f2490152a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:53444/0 (socket says 172.21.15.100:53444) 2024-09-15T21:48:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.813+0000 7f2487fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f24900cef20 0x7f24900c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:60556/0 (socket says 172.21.15.100:60556) 2024-09-15T21:48:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.813+0000 7f248ffff640 1 -- 172.21.15.100:0/387705191 learned_addr learned my addr 172.21.15.100:0/387705191 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.813+0000 7f248ffff640 1 -- 172.21.15.100:0/387705191 >> v2:172.21.15.122:3300/0 conn(0x7f24900cef20 msgr2=0x7f24900c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.813+0000 7f248ffff640 1 --2- 172.21.15.100:0/387705191 >> v2:172.21.15.122:3300/0 conn(0x7f24900cef20 0x7f24900c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.813+0000 7f248ffff640 1 -- 172.21.15.100:0/387705191 >> v1:172.21.15.122:6789/0 conn(0x7f24900ceb40 legacy=0x7f24900c6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.813+0000 7f248ffff640 1 -- 172.21.15.100:0/387705191 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24900c89a0 con 0x7f24900c9b30 2024-09-15T21:48:33.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.813+0000 7f2487fff640 1 --2- 172.21.15.100:0/387705191 >> v2:172.21.15.122:3300/0 conn(0x7f24900cef20 0x7f24900c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:48:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.814+0000 7f248ffff640 1 --2- 172.21.15.100:0/387705191 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f24900c9b30 0x7f2490152a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2488009e50 tx=0x7f248802a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cc98dd3f5d8e7a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.814+0000 7f248effd640 1 -- 172.21.15.100:0/387705191 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2488036070 con 0x7f24900c9b30 2024-09-15T21:48:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.814+0000 7f248effd640 1 -- 172.21.15.100:0/387705191 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2488031600 con 0x7f24900c9b30 2024-09-15T21:48:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.814+0000 7f248effd640 1 -- 172.21.15.100:0/387705191 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24880319d0 con 0x7f24900c9b30 2024-09-15T21:48:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.815+0000 7f2495b33640 1 -- 172.21.15.100:0/387705191 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f24900c9b30 msgr2=0x7f2490152a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.815+0000 7f2495b33640 1 --2- 172.21.15.100:0/387705191 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f24900c9b30 0x7f2490152a50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2488009e50 tx=0x7f248802a7a0 comp rx=0 tx=0).stop 2024-09-15T21:48:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.815+0000 7f2495b33640 1 -- 172.21.15.100:0/387705191 shutdown_connections 2024-09-15T21:48:33.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.815+0000 7f2495b33640 1 --2- 172.21.15.100:0/387705191 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f24900c9b30 0x7f2490152a50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.815+0000 7f2495b33640 1 --2- 172.21.15.100:0/387705191 >> v2:172.21.15.122:3300/0 conn(0x7f24900cef20 0x7f24900c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.815+0000 7f2495b33640 1 -- 172.21.15.100:0/387705191 >> 172.21.15.100:0/387705191 conn(0x7f2490067980 msgr2=0x7f2490067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.815+0000 7f2495b33640 1 -- 172.21.15.100:0/387705191 shutdown_connections 2024-09-15T21:48:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.815+0000 7f2495b33640 1 -- 172.21.15.100:0/387705191 wait complete. 2024-09-15T21:48:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.816+0000 7f2495b33640 1 Processor -- start 2024-09-15T21:48:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.816+0000 7f2495b33640 1 -- start start 2024-09-15T21:48:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f2495b33640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24900c9b30 0x7f249015a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f2495b33640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24900ceb40 0x7f24901f42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f2495b33640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f24900cef20 0x7f24900cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f2495b33640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2490158d40 con 0x7f24900c9b30 2024-09-15T21:48:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f2495b33640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f24900c6c80 con 0x7f24900ceb40 2024-09-15T21:48:33.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f2495b33640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2490158ec0 con 0x7f24900cef20 2024-09-15T21:48:33.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f248f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24900c9b30 0x7f249015a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:33.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f248f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24900c9b30 0x7f249015a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53870/0 (socket says 172.21.15.100:53870) 2024-09-15T21:48:33.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f248f7fe640 1 -- 172.21.15.100:0/2017287542 learned_addr learned my addr 172.21.15.100:0/2017287542 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:33.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f248ffff640 1 --2- 172.21.15.100:0/2017287542 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f24900cef20 0x7f24900cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:33.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f2487fff640 1 --2- 172.21.15.100:0/2017287542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24900ceb40 0x7f24901f42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:33.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f248f7fe640 1 -- 172.21.15.100:0/2017287542 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f24900cef20 msgr2=0x7f24900cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:33.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f248f7fe640 1 --2- 172.21.15.100:0/2017287542 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f24900cef20 0x7f24900cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:33.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f248f7fe640 1 -- 172.21.15.100:0/2017287542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24900ceb40 msgr2=0x7f24901f42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:33.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f248f7fe640 1 --2- 172.21.15.100:0/2017287542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24900ceb40 0x7f24901f42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:33.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f248f7fe640 1 -- 172.21.15.100:0/2017287542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24900ce700 con 0x7f24900c9b30 2024-09-15T21:48:33.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f248f7fe640 1 --2- 172.21.15.100:0/2017287542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24900c9b30 0x7f249015a8f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f248000c430 tx=0x7f248000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:33.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f248cff9640 1 -- 172.21.15.100:0/2017287542 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480017070 con 0x7f24900c9b30 2024-09-15T21:48:33.822 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f248cff9640 1 -- 172.21.15.100:0/2017287542 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24800052d0 con 0x7f24900c9b30 2024-09-15T21:48:33.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f2495b33640 1 -- 172.21.15.100:0/2017287542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24900ce140 con 0x7f24900c9b30 2024-09-15T21:48:33.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f248cff9640 1 -- 172.21.15.100:0/2017287542 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2480003850 con 0x7f24900c9b30 2024-09-15T21:48:33.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.817+0000 7f2495b33640 1 -- 172.21.15.100:0/2017287542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f249015c310 con 0x7f24900c9b30 2024-09-15T21:48:33.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.818+0000 7f2495b33640 1 -- 172.21.15.100:0/2017287542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f249015c6b0 con 0x7f24900c9b30 2024-09-15T21:48:33.823 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.819+0000 7f248cff9640 1 -- 172.21.15.100:0/2017287542 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f248001a070 con 0x7f24900c9b30 2024-09-15T21:48:33.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.819+0000 7f248cff9640 1 --2- 172.21.15.100:0/2017287542 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f247006c2b0 0x7f247006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:33.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.819+0000 7f248cff9640 1 -- 172.21.15.100:0/2017287542 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5857+0+0 (secure 0 0 0) 0x7f248008a760 con 0x7f24900c9b30 2024-09-15T21:48:33.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.819+0000 7f2487fff640 1 --2- 172.21.15.100:0/2017287542 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f247006c2b0 0x7f247006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:33.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.820+0000 7f2487fff640 1 --2- 172.21.15.100:0/2017287542 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f247006c2b0 0x7f247006e770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2478009560 tx=0x7f2478009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:33.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:33.824+0000 7f248cff9640 1 -- 172.21.15.100:0/2017287542 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2480058870 con 0x7f24900c9b30 2024-09-15T21:48:34.004 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.002+0000 7f2495b33640 1 -- 172.21.15.100:0/2017287542 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 7, "weight": 1.0} v 0) -- 0x7f24901532c0 con 0x7f24900c9b30 2024-09-15T21:48:34.702 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.701+0000 7f248cff9640 1 -- 172.21.15.100:0/2017287542 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]=0 set osd.7 primary-affinity to 1 (8655362) v74) ==== 133+0+0 (secure 0 0 0) 0x7f248005cb70 con 0x7f24900c9b30 2024-09-15T21:48:34.703 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-15T21:48:34.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.703+0000 7f2495b33640 1 -- 172.21.15.100:0/2017287542 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f247006c2b0 msgr2=0x7f247006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:34.705 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.703+0000 7f2495b33640 1 --2- 172.21.15.100:0/2017287542 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f247006c2b0 0x7f247006e770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2478009560 tx=0x7f2478009190 comp rx=0 tx=0).stop 2024-09-15T21:48:34.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.703+0000 7f2495b33640 1 -- 172.21.15.100:0/2017287542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24900c9b30 msgr2=0x7f249015a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:34.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.703+0000 7f2495b33640 1 --2- 172.21.15.100:0/2017287542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24900c9b30 0x7f249015a8f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f248000c430 tx=0x7f248000c8f0 comp rx=0 tx=0).stop 2024-09-15T21:48:34.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.704+0000 7f2495b33640 1 -- 172.21.15.100:0/2017287542 shutdown_connections 2024-09-15T21:48:34.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.704+0000 7f2495b33640 1 --2- 172.21.15.100:0/2017287542 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f247006c2b0 0x7f247006e770 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:34.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.704+0000 7f2495b33640 1 --2- 172.21.15.100:0/2017287542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24900ceb40 0x7f24901f42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:34.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.704+0000 7f2495b33640 1 --2- 172.21.15.100:0/2017287542 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f24900cef20 0x7f24900cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:34.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.704+0000 7f2495b33640 1 --2- 172.21.15.100:0/2017287542 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24900c9b30 0x7f249015a8f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:34.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.704+0000 7f2495b33640 1 -- 172.21.15.100:0/2017287542 >> 172.21.15.100:0/2017287542 conn(0x7f2490067980 msgr2=0x7f249014c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:34.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.704+0000 7f2495b33640 1 -- 172.21.15.100:0/2017287542 shutdown_connections 2024-09-15T21:48:34.706 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.704+0000 7f2495b33640 1 -- 172.21.15.100:0/2017287542 wait complete. 2024-09-15T21:48:34.719 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-15T21:48:34.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.822+0000 7f8082166640 1 Processor -- start 2024-09-15T21:48:34.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.822+0000 7f8082166640 1 -- start start 2024-09-15T21:48:34.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.823+0000 7f8082166640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f807c0cf110 0x7f807c15be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:34.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.823+0000 7f8082166640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f807c15eb40 con 0x7f807c144250 2024-09-15T21:48:34.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.823+0000 7f8082166640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f807c15ee20 con 0x7f807c0ced30 2024-09-15T21:48:34.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.823+0000 7f8082166640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f807c15c340 con 0x7f807c0cf110 2024-09-15T21:48:34.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.823+0000 7f807affd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f807c0cf110 0x7f807c15be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:34.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.823+0000 7f807bfff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f807c144250 0x7f807c144690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48118/0 (socket says 172.21.15.100:48118) 2024-09-15T21:48:34.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.823+0000 7f807bfff640 1 -- 172.21.15.100:0/4253444132 learned_addr learned my addr 172.21.15.100:0/4253444132 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:34.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.823+0000 7f807affd640 1 -- 172.21.15.100:0/4253444132 >> v1:172.21.15.122:6789/0 conn(0x7f807c0ced30 legacy=0x7f807c0739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:34.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.823+0000 7f807affd640 1 -- 172.21.15.100:0/4253444132 >> v1:172.21.15.100:6789/0 conn(0x7f807c144250 legacy=0x7f807c144690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:34.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.823+0000 7f807affd640 1 -- 172.21.15.100:0/4253444132 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807c15f100 con 0x7f807c0cf110 2024-09-15T21:48:34.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.823+0000 7f807a7fc640 1 -- 172.21.15.100:0/4253444132 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3746271372 0 0) 0x7f807c15eb40 con 0x7f807c144250 2024-09-15T21:48:34.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.824+0000 7f807affd640 1 --2- 172.21.15.100:0/4253444132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c0cf110 0x7f807c15be00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f80700097a0 tx=0x7f807002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36eb4bbe47ee587 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:34.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.824+0000 7f807a7fc640 1 -- 172.21.15.100:0/4253444132 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070037070 con 0x7f807c0cf110 2024-09-15T21:48:34.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.824+0000 7f807a7fc640 1 -- 172.21.15.100:0/4253444132 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8070032640 con 0x7f807c0cf110 2024-09-15T21:48:34.828 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.824+0000 7f807a7fc640 1 -- 172.21.15.100:0/4253444132 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8070032a10 con 0x7f807c0cf110 2024-09-15T21:48:34.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.825+0000 7f8082166640 1 -- 172.21.15.100:0/4253444132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c0cf110 msgr2=0x7f807c15be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:34.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.825+0000 7f8082166640 1 --2- 172.21.15.100:0/4253444132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c0cf110 0x7f807c15be00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f80700097a0 tx=0x7f807002a2c0 comp rx=0 tx=0).stop 2024-09-15T21:48:34.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.825+0000 7f8082166640 1 -- 172.21.15.100:0/4253444132 shutdown_connections 2024-09-15T21:48:34.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.825+0000 7f8082166640 1 --2- 172.21.15.100:0/4253444132 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c0cf110 0x7f807c15be00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:34.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.825+0000 7f8082166640 1 -- 172.21.15.100:0/4253444132 >> 172.21.15.100:0/4253444132 conn(0x7f807c06f990 msgr2=0x7f807c070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:34.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.825+0000 7f8082166640 1 -- 172.21.15.100:0/4253444132 shutdown_connections 2024-09-15T21:48:34.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.825+0000 7f8082166640 1 -- 172.21.15.100:0/4253444132 wait complete. 2024-09-15T21:48:34.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.826+0000 7f8082166640 1 Processor -- start 2024-09-15T21:48:34.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.826+0000 7f8082166640 1 -- start start 2024-09-15T21:48:34.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.827+0000 7f8082166640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c0ced30 0x7f807c0c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:34.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.827+0000 7f8082166640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f807c0cf110 0x7f807c06deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.827+0000 7f8082166640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f807c144250 0x7f807c1433d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.827+0000 7f8082166640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f807c15f750 con 0x7f807c0ced30 2024-09-15T21:48:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.827+0000 7f8082166640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f807c15e8f0 con 0x7f807c0cf110 2024-09-15T21:48:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.827+0000 7f8082166640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f807c0740c0 con 0x7f807c144250 2024-09-15T21:48:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.827+0000 7f807b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c0ced30 0x7f807c0c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.827+0000 7f807b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c0ced30 0x7f807c0c5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53884/0 (socket says 172.21.15.100:53884) 2024-09-15T21:48:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.827+0000 7f807b7fe640 1 -- 172.21.15.100:0/1093490486 learned_addr learned my addr 172.21.15.100:0/1093490486 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:34.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.827+0000 7f807bfff640 1 --2- 172.21.15.100:0/1093490486 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f807c144250 0x7f807c1433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.827+0000 7f807affd640 1 --2- 172.21.15.100:0/1093490486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f807c0cf110 0x7f807c06deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.827+0000 7f807b7fe640 1 -- 172.21.15.100:0/1093490486 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f807c144250 msgr2=0x7f807c1433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.827+0000 7f807b7fe640 1 --2- 172.21.15.100:0/1093490486 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f807c144250 0x7f807c1433d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.827+0000 7f807b7fe640 1 -- 172.21.15.100:0/1093490486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f807c0cf110 msgr2=0x7f807c06deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.827+0000 7f807b7fe640 1 --2- 172.21.15.100:0/1093490486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f807c0cf110 0x7f807c06deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.827+0000 7f807b7fe640 1 -- 172.21.15.100:0/1093490486 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f807c1f0b10 con 0x7f807c0ced30 2024-09-15T21:48:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.828+0000 7f807b7fe640 1 --2- 172.21.15.100:0/1093490486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c0ced30 0x7f807c0c5e90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f806c00c8a0 tx=0x7f806c00cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.828+0000 7f8063fff640 1 -- 172.21.15.100:0/1093490486 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c003b70 con 0x7f807c0ced30 2024-09-15T21:48:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.828+0000 7f8063fff640 1 -- 172.21.15.100:0/1093490486 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f806c003d30 con 0x7f807c0ced30 2024-09-15T21:48:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.828+0000 7f8063fff640 1 -- 172.21.15.100:0/1093490486 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806c00aa90 con 0x7f807c0ced30 2024-09-15T21:48:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.828+0000 7f8082166640 1 -- 172.21.15.100:0/1093490486 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f807c1f0df0 con 0x7f807c0ced30 2024-09-15T21:48:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.828+0000 7f8082166640 1 -- 172.21.15.100:0/1093490486 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f807c1f13b0 con 0x7f807c0ced30 2024-09-15T21:48:34.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.829+0000 7f8082166640 1 -- 172.21.15.100:0/1093490486 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f807c15be00 con 0x7f807c0ced30 2024-09-15T21:48:34.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.829+0000 7f8063fff640 1 -- 172.21.15.100:0/1093490486 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f806c00ac30 con 0x7f807c0ced30 2024-09-15T21:48:34.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.830+0000 7f8063fff640 1 --2- 172.21.15.100:0/1093490486 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f806806c2b0 0x7f806806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:34.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.830+0000 7f8063fff640 1 -- 172.21.15.100:0/1093490486 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5857+0+0 (secure 0 0 0) 0x7f806c08b360 con 0x7f807c0ced30 2024-09-15T21:48:34.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.830+0000 7f807affd640 1 --2- 172.21.15.100:0/1093490486 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f806806c2b0 0x7f806806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:34.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.830+0000 7f807affd640 1 --2- 172.21.15.100:0/1093490486 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f806806c2b0 0x7f806806e770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8070034000 tx=0x7f807002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:34.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:34.834+0000 7f8063fff640 1 -- 172.21.15.100:0/1093490486 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f806c059470 con 0x7f807c0ced30 2024-09-15T21:48:35.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.010+0000 7f8082166640 1 -- 172.21.15.100:0/1093490486 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd reweight", "id": 5, "weight": 1.0} v 0) -- 0x7f807c0c9700 con 0x7f807c0ced30 2024-09-15T21:48:35.709 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.707+0000 7f8063fff640 1 -- 172.21.15.100:0/1093490486 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd reweight", "id": 5, "weight": 1.0}]=0 reweighted osd.5 to 1 (10000) v75) ==== 113+0+0 (secure 0 0 0) 0x7f806c05d770 con 0x7f807c0ced30 2024-09-15T21:48:35.709 INFO:teuthology.orchestra.run.smithi100.stderr:reweighted osd.5 to 1 (10000) 2024-09-15T21:48:35.712 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.710+0000 7f8082166640 1 -- 172.21.15.100:0/1093490486 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f806806c2b0 msgr2=0x7f806806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:35.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.710+0000 7f8082166640 1 --2- 172.21.15.100:0/1093490486 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f806806c2b0 0x7f806806e770 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8070034000 tx=0x7f807002e000 comp rx=0 tx=0).stop 2024-09-15T21:48:35.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.710+0000 7f8082166640 1 -- 172.21.15.100:0/1093490486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c0ced30 msgr2=0x7f807c0c5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:35.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.710+0000 7f8082166640 1 --2- 172.21.15.100:0/1093490486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c0ced30 0x7f807c0c5e90 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f806c00c8a0 tx=0x7f806c00cd60 comp rx=0 tx=0).stop 2024-09-15T21:48:35.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.710+0000 7f8082166640 1 -- 172.21.15.100:0/1093490486 shutdown_connections 2024-09-15T21:48:35.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.710+0000 7f8082166640 1 --2- 172.21.15.100:0/1093490486 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f806806c2b0 0x7f806806e770 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:35.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.710+0000 7f8082166640 1 --2- 172.21.15.100:0/1093490486 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f807c0cf110 0x7f807c06deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:35.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.710+0000 7f8082166640 1 --2- 172.21.15.100:0/1093490486 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f807c144250 0x7f807c1433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:35.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.710+0000 7f8082166640 1 --2- 172.21.15.100:0/1093490486 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f807c0ced30 0x7f807c0c5e90 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:35.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.710+0000 7f8082166640 1 -- 172.21.15.100:0/1093490486 >> 172.21.15.100:0/1093490486 conn(0x7f807c06f990 msgr2=0x7f807c159da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:35.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.710+0000 7f8082166640 1 -- 172.21.15.100:0/1093490486 shutdown_connections 2024-09-15T21:48:35.713 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.710+0000 7f8082166640 1 -- 172.21.15.100:0/1093490486 wait complete. 2024-09-15T21:48:35.726 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-15T21:48:35.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.830+0000 7f2d558dd640 1 Processor -- start 2024-09-15T21:48:35.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.831+0000 7f2d558dd640 1 -- start start 2024-09-15T21:48:35.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.831+0000 7f2d558dd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f2d50155570 0x7f2d500ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:35.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.831+0000 7f2d558dd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2d50073a60 0x7f2d500c8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:35.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.831+0000 7f2d558dd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d5015f800 con 0x7f2d500c87d0 2024-09-15T21:48:35.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.831+0000 7f2d558dd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f2d50073430 con 0x7f2d50073a60 2024-09-15T21:48:35.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.831+0000 7f2d558dd640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f2d50155950 con 0x7f2d50155570 2024-09-15T21:48:35.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.832+0000 7f2d4e7fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2d50073a60 0x7f2d500c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:35.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.832+0000 7f2d4f7fe640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f2d500c87d0 0x7f2d5015e880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48124/0 (socket says 172.21.15.100:48124) 2024-09-15T21:48:35.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.832+0000 7f2d4f7fe640 1 -- 172.21.15.100:0/2070448622 learned_addr learned my addr 172.21.15.100:0/2070448622 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:35.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.832+0000 7f2d4effd640 1 --2- 172.21.15.100:0/2070448622 >> v2:172.21.15.122:3300/0 conn(0x7f2d50155570 0x7f2d500ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:35.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.832+0000 7f2d4e7fc640 1 -- 172.21.15.100:0/2070448622 >> v2:172.21.15.122:3300/0 conn(0x7f2d50155570 msgr2=0x7f2d500ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:35.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.832+0000 7f2d4e7fc640 1 --2- 172.21.15.100:0/2070448622 >> v2:172.21.15.122:3300/0 conn(0x7f2d50155570 0x7f2d500ce3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:35.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.832+0000 7f2d4e7fc640 1 -- 172.21.15.100:0/2070448622 >> v1:172.21.15.100:6789/0 conn(0x7f2d500c87d0 legacy=0x7f2d5015e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:35.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.832+0000 7f2d4e7fc640 1 -- 172.21.15.100:0/2070448622 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d5006cf00 con 0x7f2d50073a60 2024-09-15T21:48:35.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.832+0000 7f2d4dffb640 1 -- 172.21.15.100:0/2070448622 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4203450024 0 0) 0x7f2d5015f800 con 0x7f2d500c87d0 2024-09-15T21:48:35.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.832+0000 7f2d4e7fc640 1 --2- 172.21.15.100:0/2070448622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d50073a60 0x7f2d500c8290 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2d50077680 tx=0x7f2d4402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1fb09d6c8a0694 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:35.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.833+0000 7f2d4dffb640 1 -- 172.21.15.100:0/2070448622 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44037070 con 0x7f2d50073a60 2024-09-15T21:48:35.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.833+0000 7f2d4dffb640 1 -- 172.21.15.100:0/2070448622 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d44032440 con 0x7f2d50073a60 2024-09-15T21:48:35.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.833+0000 7f2d4dffb640 1 -- 172.21.15.100:0/2070448622 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44032810 con 0x7f2d50073a60 2024-09-15T21:48:35.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.833+0000 7f2d558dd640 1 -- 172.21.15.100:0/2070448622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d50073a60 msgr2=0x7f2d500c8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:35.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.833+0000 7f2d558dd640 1 --2- 172.21.15.100:0/2070448622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d50073a60 0x7f2d500c8290 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2d50077680 tx=0x7f2d4402a1a0 comp rx=0 tx=0).stop 2024-09-15T21:48:35.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.833+0000 7f2d558dd640 1 -- 172.21.15.100:0/2070448622 shutdown_connections 2024-09-15T21:48:35.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.833+0000 7f2d558dd640 1 --2- 172.21.15.100:0/2070448622 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d50073a60 0x7f2d500c8290 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:35.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.833+0000 7f2d558dd640 1 --2- 172.21.15.100:0/2070448622 >> v2:172.21.15.122:3300/0 conn(0x7f2d50155570 0x7f2d500ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:35.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.833+0000 7f2d558dd640 1 -- 172.21.15.100:0/2070448622 >> 172.21.15.100:0/2070448622 conn(0x7f2d500cd890 msgr2=0x7f2d500cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:35.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.833+0000 7f2d558dd640 1 -- 172.21.15.100:0/2070448622 shutdown_connections 2024-09-15T21:48:35.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.833+0000 7f2d558dd640 1 -- 172.21.15.100:0/2070448622 wait complete. 2024-09-15T21:48:35.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.834+0000 7f2d558dd640 1 Processor -- start 2024-09-15T21:48:35.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.835+0000 7f2d558dd640 1 -- start start 2024-09-15T21:48:35.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.835+0000 7f2d558dd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d50073a60 0x7f2d5014a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:35.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.835+0000 7f2d558dd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d500c87d0 0x7f2d501e6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:35.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.835+0000 7f2d558dd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2d50155570 0x7f2d5014f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:35.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.835+0000 7f2d558dd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5006c700 con 0x7f2d50073a60 2024-09-15T21:48:35.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.835+0000 7f2d558dd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2d5015f520 con 0x7f2d500c87d0 2024-09-15T21:48:35.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.835+0000 7f2d558dd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2d5006c9e0 con 0x7f2d50155570 2024-09-15T21:48:35.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.836+0000 7f2d4effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d50073a60 0x7f2d5014a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:35.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.836+0000 7f2d4e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d500c87d0 0x7f2d501e6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:35.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.836+0000 7f2d4effd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d50073a60 0x7f2d5014a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53904/0 (socket says 172.21.15.100:53904) 2024-09-15T21:48:35.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.836+0000 7f2d4e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d500c87d0 0x7f2d501e6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:60586/0 (socket says 172.21.15.100:60586) 2024-09-15T21:48:35.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.836+0000 7f2d4f7fe640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2d50155570 0x7f2d5014f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:35.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.836+0000 7f2d4effd640 1 -- 172.21.15.100:0/333240137 learned_addr learned my addr 172.21.15.100:0/333240137 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:35.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.836+0000 7f2d4effd640 1 -- 172.21.15.100:0/333240137 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2d50155570 msgr2=0x7f2d5014f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:35.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.836+0000 7f2d4effd640 1 --2- 172.21.15.100:0/333240137 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2d50155570 0x7f2d5014f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:35.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.836+0000 7f2d4effd640 1 -- 172.21.15.100:0/333240137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d500c87d0 msgr2=0x7f2d501e6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:35.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.836+0000 7f2d4effd640 1 --2- 172.21.15.100:0/333240137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d500c87d0 0x7f2d501e6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:35.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.836+0000 7f2d4effd640 1 -- 172.21.15.100:0/333240137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d501e7470 con 0x7f2d50073a60 2024-09-15T21:48:35.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.836+0000 7f2d4effd640 1 --2- 172.21.15.100:0/333240137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d50073a60 0x7f2d5014a2b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000c7f0 tx=0x7f2d4000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:35.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.836+0000 7f2d37fff640 1 -- 172.21.15.100:0/333240137 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40015070 con 0x7f2d50073a60 2024-09-15T21:48:35.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.836+0000 7f2d37fff640 1 -- 172.21.15.100:0/333240137 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d40004240 con 0x7f2d50073a60 2024-09-15T21:48:35.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.836+0000 7f2d37fff640 1 -- 172.21.15.100:0/333240137 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40007620 con 0x7f2d50073a60 2024-09-15T21:48:35.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.836+0000 7f2d558dd640 1 -- 172.21.15.100:0/333240137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d50151d60 con 0x7f2d50073a60 2024-09-15T21:48:35.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.836+0000 7f2d558dd640 1 -- 172.21.15.100:0/333240137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d501522e0 con 0x7f2d50073a60 2024-09-15T21:48:35.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.837+0000 7f2d558dd640 1 -- 172.21.15.100:0/333240137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d50072080 con 0x7f2d50073a60 2024-09-15T21:48:35.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.838+0000 7f2d37fff640 1 -- 172.21.15.100:0/333240137 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2d400079d0 con 0x7f2d50073a60 2024-09-15T21:48:35.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.838+0000 7f2d37fff640 1 --2- 172.21.15.100:0/333240137 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2d3c06c2b0 0x7f2d3c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:35.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.838+0000 7f2d37fff640 1 -- 172.21.15.100:0/333240137 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5857+0+0 (secure 0 0 0) 0x7f2d4008b830 con 0x7f2d50073a60 2024-09-15T21:48:35.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.838+0000 7f2d4e7fc640 1 --2- 172.21.15.100:0/333240137 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2d3c06c2b0 0x7f2d3c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:35.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.839+0000 7f2d4e7fc640 1 --2- 172.21.15.100:0/333240137 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2d3c06c2b0 0x7f2d3c06e770 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2d44034000 tx=0x7f2d4402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:35.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:35.842+0000 7f2d37fff640 1 -- 172.21.15.100:0/333240137 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2d40059940 con 0x7f2d50073a60 2024-09-15T21:48:36.020 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.018+0000 7f2d558dd640 1 -- 172.21.15.100:0/333240137 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd primary-affinity", "id": 5, "weight": 1.0} v 0) -- 0x7f2d500726d0 con 0x7f2d50073a60 2024-09-15T21:48:36.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.720+0000 7f2d37fff640 1 -- 172.21.15.100:0/333240137 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]=0 set osd.5 primary-affinity to 1 (8655362) v76) ==== 133+0+0 (secure 0 0 0) 0x7f2d4005dc40 con 0x7f2d50073a60 2024-09-15T21:48:36.722 INFO:teuthology.orchestra.run.smithi100.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-15T21:48:36.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.722+0000 7f2d558dd640 1 -- 172.21.15.100:0/333240137 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2d3c06c2b0 msgr2=0x7f2d3c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:36.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.722+0000 7f2d558dd640 1 --2- 172.21.15.100:0/333240137 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2d3c06c2b0 0x7f2d3c06e770 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2d44034000 tx=0x7f2d4402e000 comp rx=0 tx=0).stop 2024-09-15T21:48:36.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.722+0000 7f2d558dd640 1 -- 172.21.15.100:0/333240137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d50073a60 msgr2=0x7f2d5014a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:36.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.722+0000 7f2d558dd640 1 --2- 172.21.15.100:0/333240137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d50073a60 0x7f2d5014a2b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000c7f0 tx=0x7f2d4000ccb0 comp rx=0 tx=0).stop 2024-09-15T21:48:36.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.722+0000 7f2d558dd640 1 -- 172.21.15.100:0/333240137 shutdown_connections 2024-09-15T21:48:36.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.722+0000 7f2d558dd640 1 --2- 172.21.15.100:0/333240137 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2d3c06c2b0 0x7f2d3c06e770 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:36.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.722+0000 7f2d558dd640 1 --2- 172.21.15.100:0/333240137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2d500c87d0 0x7f2d501e6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:36.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.722+0000 7f2d558dd640 1 --2- 172.21.15.100:0/333240137 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2d50155570 0x7f2d5014f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:36.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.722+0000 7f2d558dd640 1 --2- 172.21.15.100:0/333240137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2d50073a60 0x7f2d5014a2b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:36.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.722+0000 7f2d558dd640 1 -- 172.21.15.100:0/333240137 >> 172.21.15.100:0/333240137 conn(0x7f2d500cd890 msgr2=0x7f2d500c7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:36.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.723+0000 7f2d558dd640 1 -- 172.21.15.100:0/333240137 shutdown_connections 2024-09-15T21:48:36.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.723+0000 7f2d558dd640 1 -- 172.21.15.100:0/333240137 wait complete. 2024-09-15T21:48:36.739 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-15T21:48:36.739 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-15T21:48:36.739 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-15T21:48:36.739 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-15T21:48:36.739 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T21:48:36.740 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:48:36.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.842+0000 7f1e0dc23640 1 Processor -- start 2024-09-15T21:48:36.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.842+0000 7f1e0dc23640 1 -- start start 2024-09-15T21:48:36.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.843+0000 7f1e0dc23640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1e08153550 0x7f1e080c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:36.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.843+0000 7f1e0dc23640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1e080c5c90 0x7f1e080c4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:36.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.843+0000 7f1e0dc23640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1e08073240 0x7f1e0806f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:36.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.843+0000 7f1e0dc23640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f1e080cfb90 con 0x7f1e08073240 2024-09-15T21:48:36.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.843+0000 7f1e0dc23640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f1e081593f0 con 0x7f1e080c5c90 2024-09-15T21:48:36.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.843+0000 7f1e0dc23640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1e081596d0 con 0x7f1e08153550 2024-09-15T21:48:36.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.843+0000 7f1e07fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1e08073240 0x7f1e0806f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:36.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.843+0000 7f1e07fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1e08073240 0x7f1e0806f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53970/0 (socket says 172.21.15.100:53970) 2024-09-15T21:48:36.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.843+0000 7f1e07fff640 1 -- 172.21.15.100:0/3467662406 learned_addr learned my addr 172.21.15.100:0/3467662406 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:36.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.844+0000 7f1dfffff640 1 --2- 172.21.15.100:0/3467662406 >> v2:172.21.15.122:3300/0 conn(0x7f1e080c5c90 0x7f1e080c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:36.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.844+0000 7f1e077fe640 1 --2- 172.21.15.100:0/3467662406 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1e08153550 0x7f1e080c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:36.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.844+0000 7f1e07fff640 1 -- 172.21.15.100:0/3467662406 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1e08153550 msgr2=0x7f1e080c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:36.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.844+0000 7f1e07fff640 1 --2- 172.21.15.100:0/3467662406 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1e08153550 0x7f1e080c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:36.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.844+0000 7f1e07fff640 1 -- 172.21.15.100:0/3467662406 >> v2:172.21.15.122:3300/0 conn(0x7f1e080c5c90 msgr2=0x7f1e080c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:36.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.844+0000 7f1e07fff640 1 --2- 172.21.15.100:0/3467662406 >> v2:172.21.15.122:3300/0 conn(0x7f1e080c5c90 0x7f1e080c4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:36.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.844+0000 7f1e07fff640 1 -- 172.21.15.100:0/3467662406 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e081599b0 con 0x7f1e08073240 2024-09-15T21:48:36.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.844+0000 7f1e07fff640 1 --2- 172.21.15.100:0/3467662406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e08073240 0x7f1e0806f4c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1e08070650 tx=0x7f1e0002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e45825ae9cc6ec4d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:36.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.844+0000 7f1e06ffd640 1 -- 172.21.15.100:0/3467662406 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e00036070 con 0x7f1e08073240 2024-09-15T21:48:36.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.845+0000 7f1e06ffd640 1 -- 172.21.15.100:0/3467662406 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e00031600 con 0x7f1e08073240 2024-09-15T21:48:36.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.845+0000 7f1e06ffd640 1 -- 172.21.15.100:0/3467662406 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e000319d0 con 0x7f1e08073240 2024-09-15T21:48:36.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.845+0000 7f1e0dc23640 1 -- 172.21.15.100:0/3467662406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e08073240 msgr2=0x7f1e0806f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:36.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.845+0000 7f1e0dc23640 1 --2- 172.21.15.100:0/3467662406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e08073240 0x7f1e0806f4c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1e08070650 tx=0x7f1e0002a730 comp rx=0 tx=0).stop 2024-09-15T21:48:36.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.845+0000 7f1e0dc23640 1 -- 172.21.15.100:0/3467662406 shutdown_connections 2024-09-15T21:48:36.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.845+0000 7f1e0dc23640 1 --2- 172.21.15.100:0/3467662406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e08073240 0x7f1e0806f4c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:36.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.845+0000 7f1e0dc23640 1 --2- 172.21.15.100:0/3467662406 >> v2:172.21.15.122:3300/0 conn(0x7f1e080c5c90 0x7f1e080c4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:36.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.845+0000 7f1e0dc23640 1 --2- 172.21.15.100:0/3467662406 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1e08153550 0x7f1e080c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:36.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.845+0000 7f1e0dc23640 1 -- 172.21.15.100:0/3467662406 >> 172.21.15.100:0/3467662406 conn(0x7f1e080cec80 msgr2=0x7f1e080cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:36.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.845+0000 7f1e0dc23640 1 -- 172.21.15.100:0/3467662406 shutdown_connections 2024-09-15T21:48:36.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.845+0000 7f1e0dc23640 1 -- 172.21.15.100:0/3467662406 wait complete. 2024-09-15T21:48:36.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.846+0000 7f1e0dc23640 1 Processor -- start 2024-09-15T21:48:36.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.846+0000 7f1e0dc23640 1 -- start start 2024-09-15T21:48:36.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.847+0000 7f1e0dc23640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e08073240 0x7f1e080c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:36.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.847+0000 7f1e0dc23640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1e08153550 0x7f1e081558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:36.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.847+0000 7f1e0dc23640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e080c79e0 0x7f1e0814ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:36.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.847+0000 7f1e0dc23640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0806efa0 con 0x7f1e08073240 2024-09-15T21:48:36.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.847+0000 7f1e0dc23640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1e08159f70 con 0x7f1e080c79e0 2024-09-15T21:48:36.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.847+0000 7f1e0dc23640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1e081590e0 con 0x7f1e08153550 2024-09-15T21:48:36.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.847+0000 7f1e077fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e08073240 0x7f1e080c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.847+0000 7f1e077fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e08073240 0x7f1e080c6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53982/0 (socket says 172.21.15.100:53982) 2024-09-15T21:48:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.847+0000 7f1e077fe640 1 -- 172.21.15.100:0/3294628179 learned_addr learned my addr 172.21.15.100:0/3294628179 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.847+0000 7f1dfffff640 1 --2- 172.21.15.100:0/3294628179 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1e08153550 0x7f1e081558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.847+0000 7f1e07fff640 1 --2- 172.21.15.100:0/3294628179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e080c79e0 0x7f1e0814ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.848+0000 7f1e077fe640 1 -- 172.21.15.100:0/3294628179 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1e08153550 msgr2=0x7f1e081558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.848+0000 7f1e077fe640 1 --2- 172.21.15.100:0/3294628179 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1e08153550 0x7f1e081558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.848+0000 7f1e077fe640 1 -- 172.21.15.100:0/3294628179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e080c79e0 msgr2=0x7f1e0814ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.848+0000 7f1e077fe640 1 --2- 172.21.15.100:0/3294628179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e080c79e0 0x7f1e0814ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.848+0000 7f1e077fe640 1 -- 172.21.15.100:0/3294628179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e081ef330 con 0x7f1e08073240 2024-09-15T21:48:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.848+0000 7f1e077fe640 1 --2- 172.21.15.100:0/3294628179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e08073240 0x7f1e080c6c90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1df800d5f0 tx=0x7f1df800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.848+0000 7f1e04ff9640 1 -- 172.21.15.100:0/3294628179 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df80125f0 con 0x7f1e08073240 2024-09-15T21:48:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.848+0000 7f1e04ff9640 1 -- 172.21.15.100:0/3294628179 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1df8019070 con 0x7f1e08073240 2024-09-15T21:48:36.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.848+0000 7f1e0dc23640 1 -- 172.21.15.100:0/3294628179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e080c61b0 con 0x7f1e08073240 2024-09-15T21:48:36.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.848+0000 7f1e04ff9640 1 -- 172.21.15.100:0/3294628179 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df80129d0 con 0x7f1e08073240 2024-09-15T21:48:36.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.848+0000 7f1e0dc23640 1 -- 172.21.15.100:0/3294628179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e080c6770 con 0x7f1e08073240 2024-09-15T21:48:36.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.850+0000 7f1e0dc23640 1 -- 172.21.15.100:0/3294628179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e08068f40 con 0x7f1e08073240 2024-09-15T21:48:36.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.850+0000 7f1e04ff9640 1 -- 172.21.15.100:0/3294628179 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1df8012b70 con 0x7f1e08073240 2024-09-15T21:48:36.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.850+0000 7f1e04ff9640 1 --2- 172.21.15.100:0/3294628179 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1de806c2b0 0x7f1de806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:36.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.851+0000 7f1dfffff640 1 --2- 172.21.15.100:0/3294628179 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1de806c2b0 0x7f1de806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:36.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.851+0000 7f1e04ff9640 1 -- 172.21.15.100:0/3294628179 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f1df808b770 con 0x7f1e08073240 2024-09-15T21:48:36.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.851+0000 7f1dfffff640 1 --2- 172.21.15.100:0/3294628179 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1de806c2b0 0x7f1de806e770 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1e08155770 tx=0x7f1df0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:36.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:36.855+0000 7f1e04ff9640 1 -- 172.21.15.100:0/3294628179 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1e08068f40 con 0x7f1e08073240 2024-09-15T21:48:37.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.023+0000 7f1e0dc23640 1 -- 172.21.15.100:0/3294628179 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e08070260 con 0x7f1e08073240 2024-09-15T21:48:37.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.024+0000 7f1e04ff9640 1 -- 172.21.15.100:0/3294628179 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+16959 (secure 0 0 0) 0x7f1df80597f0 con 0x7f1e08073240 2024-09-15T21:48:37.026 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:48:37.026 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":76,"fsid":"30e2fff4-2171-4531-8806-0e03b63f494f","created":"2024-09-15T21:45:59.746897+0000","modified":"2024-09-15T21:48:36.714481+0000","last_up_change":"2024-09-15T21:48:16.603329+0000","last_in_change":"2024-09-15T21:48:20.610986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:46:11.837603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T21:46:20.577982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T21:46:21.729231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5887900b-522d-476f-9704-cc2c5ee87a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6819","nonce":3920435546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6821","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6822","nonce":3920435546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6829","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6831","nonce":3920435546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6825","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6827","nonce":3920435546}]},"public_addr":"172.21.15.100:6819/3920435546","cluster_addr":"172.21.15.100:6822/3920435546","heartbeat_back_addr":"172.21.15.100:6831/3920435546","heartbeat_front_addr":"172.21.15.100:6827/3920435546","state":["exists","up"]},{"osd":1,"uuid":"adc17993-5c50-4251-85e1-4c7ac62e25e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6801","nonce":3802679972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6803","nonce":3802679972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6807","nonce":3802679972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6805","nonce":3802679972}]},"public_addr":"172.21.15.100:6801/3802679972","cluster_addr":"172.21.15.100:6803/3802679972","heartbeat_back_addr":"172.21.15.100:6807/3802679972","heartbeat_front_addr":"172.21.15.100:6805/3802679972","state":["exists","up"]},{"osd":2,"uuid":"8de3b7d5-1980-4a3b-a934-264f867c7c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6823","nonce":2316787974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6826","nonce":2316787974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6833","nonce":2316787974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6830","nonce":2316787974}]},"public_addr":"172.21.15.100:6823/2316787974","cluster_addr":"172.21.15.100:6826/2316787974","heartbeat_back_addr":"172.21.15.100:6833/2316787974","heartbeat_front_addr":"172.21.15.100:6830/2316787974","state":["exists","up"]},{"osd":3,"uuid":"93dbcd1c-6a6f-49cf-bc94-e64b883de5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6811","nonce":1287562741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6813","nonce":1287562741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6817","nonce":1287562741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6815","nonce":1287562741}]},"public_addr":"172.21.15.100:6811/1287562741","cluster_addr":"172.21.15.100:6813/1287562741","heartbeat_back_addr":"172.21.15.100:6817/1287562741","heartbeat_front_addr":"172.21.15.100:6815/1287562741","state":["exists","up"]},{"osd":4,"uuid":"7caee2a0-bab4-45a9-8109-724948c863c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6801","nonce":4045445512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6803","nonce":4045445512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6807","nonce":4045445512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6805","nonce":4045445512}]},"public_addr":"172.21.15.122:6801/4045445512","cluster_addr":"172.21.15.122:6803/4045445512","heartbeat_back_addr":"172.21.15.122:6807/4045445512","heartbeat_front_addr":"172.21.15.122:6805/4045445512","state":["exists","up"]},{"osd":5,"uuid":"6e64070a-671a-444b-a061-b3051a29b121","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":29,"up_from":57,"up_thru":60,"down_at":30,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":822572374},{"type":"v1","addr":"172.21.15.122:6825","nonce":822572374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":822572374},{"type":"v1","addr":"172.21.15.122:6827","nonce":822572374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":822572374},{"type":"v1","addr":"172.21.15.122:6831","nonce":822572374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":822572374},{"type":"v1","addr":"172.21.15.122:6829","nonce":822572374}]},"public_addr":"172.21.15.122:6825/822572374","cluster_addr":"172.21.15.122:6827/822572374","heartbeat_back_addr":"172.21.15.122:6831/822572374","heartbeat_front_addr":"172.21.15.122:6829/822572374","state":["exists","up"]},{"osd":6,"uuid":"47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6809","nonce":1437167311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6811","nonce":1437167311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6815","nonce":1437167311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6813","nonce":1437167311}]},"public_addr":"172.21.15.122:6809/1437167311","cluster_addr":"172.21.15.122:6811/1437167311","heartbeat_back_addr":"172.21.15.122:6815/1437167311","heartbeat_front_addr":"172.21.15.122:6813/1437167311","state":["exists","up"]},{"osd":7,"uuid":"9a916b6d-af82-486f-af40-4a7291c900fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6817","nonce":2431377162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6819","nonce":2431377162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6823","nonce":2431377162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6821","nonce":2431377162}]},"public_addr":"172.21.15.122:6817/2431377162","cluster_addr":"172.21.15.122:6819/2431377162","heartbeat_back_addr":"172.21.15.122:6823/2431377162","heartbeat_front_addr":"172.21.15.122:6821/2431377162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.168771+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.886614+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.154129+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.570531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.926773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.146320+0000","dead_epoch":30},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.409137+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.373359+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.8","osds":[0,5]}],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:48:37.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.026+0000 7f1e0dc23640 1 -- 172.21.15.100:0/3294628179 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1de806c2b0 msgr2=0x7f1de806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.026+0000 7f1e0dc23640 1 --2- 172.21.15.100:0/3294628179 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1de806c2b0 0x7f1de806e770 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1e08155770 tx=0x7f1df0008040 comp rx=0 tx=0).stop 2024-09-15T21:48:37.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.026+0000 7f1e0dc23640 1 -- 172.21.15.100:0/3294628179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e08073240 msgr2=0x7f1e080c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.026+0000 7f1e0dc23640 1 --2- 172.21.15.100:0/3294628179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e08073240 0x7f1e080c6c90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1df800d5f0 tx=0x7f1df800dab0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.026+0000 7f1e0dc23640 1 -- 172.21.15.100:0/3294628179 shutdown_connections 2024-09-15T21:48:37.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.026+0000 7f1e0dc23640 1 --2- 172.21.15.100:0/3294628179 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1de806c2b0 0x7f1de806e770 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.026+0000 7f1e0dc23640 1 --2- 172.21.15.100:0/3294628179 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e080c79e0 0x7f1e0814ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.026+0000 7f1e0dc23640 1 --2- 172.21.15.100:0/3294628179 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1e08153550 0x7f1e081558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.026+0000 7f1e0dc23640 1 --2- 172.21.15.100:0/3294628179 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1e08073240 0x7f1e080c6c90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.026+0000 7f1e0dc23640 1 -- 172.21.15.100:0/3294628179 >> 172.21.15.100:0/3294628179 conn(0x7f1e080cec80 msgr2=0x7f1e08073620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.026+0000 7f1e0dc23640 1 -- 172.21.15.100:0/3294628179 shutdown_connections 2024-09-15T21:48:37.029 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.026+0000 7f1e0dc23640 1 -- 172.21.15.100:0/3294628179 wait complete. 2024-09-15T21:48:37.039 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T21:48:37.040 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:48:37.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.136+0000 7fdbfcef1640 1 Processor -- start 2024-09-15T21:48:37.138 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.136+0000 7fdbfcef1640 1 -- start start 2024-09-15T21:48:37.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.137+0000 7fdbfcef1640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdbf8073320 0x7fdbf8158e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.139 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.137+0000 7fdbfcef1640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdbf80c8a10 0x7fdbf80c6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.137+0000 7fdbfcef1640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbf80c91a0 con 0x7fdbf8073700 2024-09-15T21:48:37.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.137+0000 7fdbfcef1640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fdbf80c9440 con 0x7fdbf80c8a10 2024-09-15T21:48:37.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.137+0000 7fdbfcef1640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf80c9cc0 con 0x7fdbf8073320 2024-09-15T21:48:37.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.137+0000 7fdbf6d9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdbf80c8a10 0x7fdbf80c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.137+0000 7fdbf6d9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fdbf80c8a10 0x7fdbf80c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:53998/0 (socket says 172.21.15.100:53998) 2024-09-15T21:48:37.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.137+0000 7fdbf6d9d640 1 -- 172.21.15.100:0/216630612 learned_addr learned my addr 172.21.15.100:0/216630612 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.137+0000 7fdbf659c640 1 --2- 172.21.15.100:0/216630612 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdbf8073320 0x7fdbf8158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.137+0000 7fdbf6d9d640 1 -- 172.21.15.100:0/216630612 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdbf8073320 msgr2=0x7fdbf8158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:37.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.137+0000 7fdbf6d9d640 1 --2- 172.21.15.100:0/216630612 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdbf8073320 0x7fdbf8158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.137+0000 7fdbf6d9d640 1 -- 172.21.15.100:0/216630612 >> v1:172.21.15.122:6789/0 conn(0x7fdbf8073700 legacy=0x7fdbf8152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:37.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.137+0000 7fdbf6d9d640 1 -- 172.21.15.100:0/216630612 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf80c9700 con 0x7fdbf80c8a10 2024-09-15T21:48:37.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.138+0000 7fdbf6d9d640 1 --2- 172.21.15.100:0/216630612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf80c8a10 0x7fdbf80c6540 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8071c10 tx=0x7fdbe802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7542f51e28caf532 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.140 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.138+0000 7fdbf559a640 1 -- 172.21.15.100:0/216630612 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8037070 con 0x7fdbf80c8a10 2024-09-15T21:48:37.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.138+0000 7fdbf559a640 1 -- 172.21.15.100:0/216630612 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbe8032440 con 0x7fdbf80c8a10 2024-09-15T21:48:37.141 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.138+0000 7fdbf559a640 1 -- 172.21.15.100:0/216630612 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8032810 con 0x7fdbf80c8a10 2024-09-15T21:48:37.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.138+0000 7fdbfcef1640 1 -- 172.21.15.100:0/216630612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf80c8a10 msgr2=0x7fdbf80c6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.138+0000 7fdbfcef1640 1 --2- 172.21.15.100:0/216630612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf80c8a10 0x7fdbf80c6540 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8071c10 tx=0x7fdbe802a1e0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.138+0000 7fdbfcef1640 1 -- 172.21.15.100:0/216630612 shutdown_connections 2024-09-15T21:48:37.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.138+0000 7fdbfcef1640 1 --2- 172.21.15.100:0/216630612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf80c8a10 0x7fdbf80c6540 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.138+0000 7fdbfcef1640 1 --2- 172.21.15.100:0/216630612 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdbf8073320 0x7fdbf8158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.138+0000 7fdbfcef1640 1 -- 172.21.15.100:0/216630612 >> 172.21.15.100:0/216630612 conn(0x7fdbf8070ca0 msgr2=0x7fdbf8071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.139+0000 7fdbfcef1640 1 -- 172.21.15.100:0/216630612 shutdown_connections 2024-09-15T21:48:37.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.139+0000 7fdbfcef1640 1 -- 172.21.15.100:0/216630612 wait complete. 2024-09-15T21:48:37.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.140+0000 7fdbfcef1640 1 Processor -- start 2024-09-15T21:48:37.142 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.140+0000 7fdbfcef1640 1 -- start start 2024-09-15T21:48:37.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.140+0000 7fdbfcef1640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdbf8073320 0x7fdbf814c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.140+0000 7fdbfcef1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf8073700 0x7fdbf81f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.140+0000 7fdbfcef1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbf80c8a10 0x7fdbf81f4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.141+0000 7fdbfcef1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf80c6020 con 0x7fdbf8073700 2024-09-15T21:48:37.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.141+0000 7fdbfcef1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf80c5d40 con 0x7fdbf80c8a10 2024-09-15T21:48:37.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.141+0000 7fdbfcef1640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fdbf80c78c0 con 0x7fdbf8073320 2024-09-15T21:48:37.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.141+0000 7fdbf5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf8073700 0x7fdbf81f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.141+0000 7fdbf5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf8073700 0x7fdbf81f41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54012/0 (socket says 172.21.15.100:54012) 2024-09-15T21:48:37.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.141+0000 7fdbf5d9b640 1 -- 172.21.15.100:0/2240872135 learned_addr learned my addr 172.21.15.100:0/2240872135 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.145 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.141+0000 7fdbf659c640 1 --2- 172.21.15.100:0/2240872135 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdbf8073320 0x7fdbf814c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.141+0000 7fdbf6d9d640 1 --2- 172.21.15.100:0/2240872135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbf80c8a10 0x7fdbf81f4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.141+0000 7fdbf5d9b640 1 -- 172.21.15.100:0/2240872135 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdbf8073320 msgr2=0x7fdbf814c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.141+0000 7fdbf5d9b640 1 --2- 172.21.15.100:0/2240872135 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdbf8073320 0x7fdbf814c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.141+0000 7fdbf5d9b640 1 -- 172.21.15.100:0/2240872135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbf80c8a10 msgr2=0x7fdbf81f4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.141+0000 7fdbf5d9b640 1 --2- 172.21.15.100:0/2240872135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbf80c8a10 0x7fdbf81f4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.141+0000 7fdbf5d9b640 1 -- 172.21.15.100:0/2240872135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf81e38b0 con 0x7fdbf8073700 2024-09-15T21:48:37.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.141+0000 7fdbf5d9b640 1 --2- 172.21.15.100:0/2240872135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf8073700 0x7fdbf81f41d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000c450 tx=0x7fdbe000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.141+0000 7fdbeeffd640 1 -- 172.21.15.100:0/2240872135 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0016020 con 0x7fdbf8073700 2024-09-15T21:48:37.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.141+0000 7fdbeeffd640 1 -- 172.21.15.100:0/2240872135 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbe0012830 con 0x7fdbf8073700 2024-09-15T21:48:37.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.141+0000 7fdbfcef1640 1 -- 172.21.15.100:0/2240872135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf81e2030 con 0x7fdbf8073700 2024-09-15T21:48:37.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.141+0000 7fdbeeffd640 1 -- 172.21.15.100:0/2240872135 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0012c20 con 0x7fdbf8073700 2024-09-15T21:48:37.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.141+0000 7fdbfcef1640 1 -- 172.21.15.100:0/2240872135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf80cdbb0 con 0x7fdbf8073700 2024-09-15T21:48:37.146 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.142+0000 7fdbfcef1640 1 -- 172.21.15.100:0/2240872135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbf80ced90 con 0x7fdbf8073700 2024-09-15T21:48:37.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.143+0000 7fdbeeffd640 1 -- 172.21.15.100:0/2240872135 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fdbe0019020 con 0x7fdbf8073700 2024-09-15T21:48:37.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.143+0000 7fdbeeffd640 1 --2- 172.21.15.100:0/2240872135 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fdbd806c2e0 0x7fdbd806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.143+0000 7fdbeeffd640 1 -- 172.21.15.100:0/2240872135 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7fdbe008b210 con 0x7fdbf8073700 2024-09-15T21:48:37.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.143+0000 7fdbf659c640 1 --2- 172.21.15.100:0/2240872135 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fdbd806c2e0 0x7fdbd806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.147 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.144+0000 7fdbf659c640 1 --2- 172.21.15.100:0/2240872135 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fdbd806c2e0 0x7fdbd806e7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000f510 tx=0x7fdbf00077b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.148+0000 7fdbeeffd640 1 -- 172.21.15.100:0/2240872135 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fdbe0059200 con 0x7fdbf8073700 2024-09-15T21:48:37.314 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.312+0000 7fdbfcef1640 1 -- 172.21.15.100:0/2240872135 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdbf8156890 con 0x7fdbf8073700 2024-09-15T21:48:37.315 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.314+0000 7fdbeeffd640 1 -- 172.21.15.100:0/2240872135 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+16959 (secure 0 0 0) 0x7fdbe005d500 con 0x7fdbf8073700 2024-09-15T21:48:37.316 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:48:37.316 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":76,"fsid":"30e2fff4-2171-4531-8806-0e03b63f494f","created":"2024-09-15T21:45:59.746897+0000","modified":"2024-09-15T21:48:36.714481+0000","last_up_change":"2024-09-15T21:48:16.603329+0000","last_in_change":"2024-09-15T21:48:20.610986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:46:11.837603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T21:46:20.577982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T21:46:21.729231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5887900b-522d-476f-9704-cc2c5ee87a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6819","nonce":3920435546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6821","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6822","nonce":3920435546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6829","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6831","nonce":3920435546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6825","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6827","nonce":3920435546}]},"public_addr":"172.21.15.100:6819/3920435546","cluster_addr":"172.21.15.100:6822/3920435546","heartbeat_back_addr":"172.21.15.100:6831/3920435546","heartbeat_front_addr":"172.21.15.100:6827/3920435546","state":["exists","up"]},{"osd":1,"uuid":"adc17993-5c50-4251-85e1-4c7ac62e25e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6801","nonce":3802679972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6803","nonce":3802679972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6807","nonce":3802679972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6805","nonce":3802679972}]},"public_addr":"172.21.15.100:6801/3802679972","cluster_addr":"172.21.15.100:6803/3802679972","heartbeat_back_addr":"172.21.15.100:6807/3802679972","heartbeat_front_addr":"172.21.15.100:6805/3802679972","state":["exists","up"]},{"osd":2,"uuid":"8de3b7d5-1980-4a3b-a934-264f867c7c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6823","nonce":2316787974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6826","nonce":2316787974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6833","nonce":2316787974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6830","nonce":2316787974}]},"public_addr":"172.21.15.100:6823/2316787974","cluster_addr":"172.21.15.100:6826/2316787974","heartbeat_back_addr":"172.21.15.100:6833/2316787974","heartbeat_front_addr":"172.21.15.100:6830/2316787974","state":["exists","up"]},{"osd":3,"uuid":"93dbcd1c-6a6f-49cf-bc94-e64b883de5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6811","nonce":1287562741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6813","nonce":1287562741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6817","nonce":1287562741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6815","nonce":1287562741}]},"public_addr":"172.21.15.100:6811/1287562741","cluster_addr":"172.21.15.100:6813/1287562741","heartbeat_back_addr":"172.21.15.100:6817/1287562741","heartbeat_front_addr":"172.21.15.100:6815/1287562741","state":["exists","up"]},{"osd":4,"uuid":"7caee2a0-bab4-45a9-8109-724948c863c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6801","nonce":4045445512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6803","nonce":4045445512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6807","nonce":4045445512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6805","nonce":4045445512}]},"public_addr":"172.21.15.122:6801/4045445512","cluster_addr":"172.21.15.122:6803/4045445512","heartbeat_back_addr":"172.21.15.122:6807/4045445512","heartbeat_front_addr":"172.21.15.122:6805/4045445512","state":["exists","up"]},{"osd":5,"uuid":"6e64070a-671a-444b-a061-b3051a29b121","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":29,"up_from":57,"up_thru":60,"down_at":30,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":822572374},{"type":"v1","addr":"172.21.15.122:6825","nonce":822572374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":822572374},{"type":"v1","addr":"172.21.15.122:6827","nonce":822572374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":822572374},{"type":"v1","addr":"172.21.15.122:6831","nonce":822572374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":822572374},{"type":"v1","addr":"172.21.15.122:6829","nonce":822572374}]},"public_addr":"172.21.15.122:6825/822572374","cluster_addr":"172.21.15.122:6827/822572374","heartbeat_back_addr":"172.21.15.122:6831/822572374","heartbeat_front_addr":"172.21.15.122:6829/822572374","state":["exists","up"]},{"osd":6,"uuid":"47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6809","nonce":1437167311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6811","nonce":1437167311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6815","nonce":1437167311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6813","nonce":1437167311}]},"public_addr":"172.21.15.122:6809/1437167311","cluster_addr":"172.21.15.122:6811/1437167311","heartbeat_back_addr":"172.21.15.122:6815/1437167311","heartbeat_front_addr":"172.21.15.122:6813/1437167311","state":["exists","up"]},{"osd":7,"uuid":"9a916b6d-af82-486f-af40-4a7291c900fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6817","nonce":2431377162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6819","nonce":2431377162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6823","nonce":2431377162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6821","nonce":2431377162}]},"public_addr":"172.21.15.122:6817/2431377162","cluster_addr":"172.21.15.122:6819/2431377162","heartbeat_back_addr":"172.21.15.122:6823/2431377162","heartbeat_front_addr":"172.21.15.122:6821/2431377162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.168771+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.886614+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.154129+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.570531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.926773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.146320+0000","dead_epoch":30},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.409137+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.373359+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.8","osds":[0,5]}],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:48:37.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.316+0000 7fdbfcef1640 1 -- 172.21.15.100:0/2240872135 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fdbd806c2e0 msgr2=0x7fdbd806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.316+0000 7fdbfcef1640 1 --2- 172.21.15.100:0/2240872135 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fdbd806c2e0 0x7fdbd806e7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000f510 tx=0x7fdbf00077b0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.316+0000 7fdbfcef1640 1 -- 172.21.15.100:0/2240872135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf8073700 msgr2=0x7fdbf81f41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.316+0000 7fdbfcef1640 1 --2- 172.21.15.100:0/2240872135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf8073700 0x7fdbf81f41d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000c450 tx=0x7fdbe000c910 comp rx=0 tx=0).stop 2024-09-15T21:48:37.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.316+0000 7fdbfcef1640 1 -- 172.21.15.100:0/2240872135 shutdown_connections 2024-09-15T21:48:37.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.316+0000 7fdbfcef1640 1 --2- 172.21.15.100:0/2240872135 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fdbd806c2e0 0x7fdbd806e7a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.316+0000 7fdbfcef1640 1 --2- 172.21.15.100:0/2240872135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdbf80c8a10 0x7fdbf81f4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.316+0000 7fdbfcef1640 1 --2- 172.21.15.100:0/2240872135 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdbf8073700 0x7fdbf81f41d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.316+0000 7fdbfcef1640 1 --2- 172.21.15.100:0/2240872135 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fdbf8073320 0x7fdbf814c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.318 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.316+0000 7fdbfcef1640 1 -- 172.21.15.100:0/2240872135 >> 172.21.15.100:0/2240872135 conn(0x7fdbf8070ca0 msgr2=0x7fdbf8151f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.316+0000 7fdbfcef1640 1 -- 172.21.15.100:0/2240872135 shutdown_connections 2024-09-15T21:48:37.319 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.316+0000 7fdbfcef1640 1 -- 172.21.15.100:0/2240872135 wait complete. 2024-09-15T21:48:37.329 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:48:37.329 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T21:48:37.330 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T21:48:37.330 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T21:48:37.331 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-15T21:48:37.331 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-15T21:48:37.332 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-15T21:48:37.332 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-15T21:48:37.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.494+0000 7fa7da242640 1 Processor -- start 2024-09-15T21:48:37.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.494+0000 7fa7da242640 1 -- start start 2024-09-15T21:48:37.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.497+0000 7fad7aa82640 1 Processor -- start 2024-09-15T21:48:37.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.495+0000 7fa7da242640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fa7d4159780 0x7fa7d40c4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.495+0000 7fa7da242640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa7d40c5600 0x7fa7d406e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.495+0000 7fa7da242640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7d415f680 con 0x7fa7d406f1f0 2024-09-15T21:48:37.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.495+0000 7fa7da242640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa7d415f960 con 0x7fa7d40c5600 2024-09-15T21:48:37.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.495+0000 7fa7da242640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fa7d415fc40 con 0x7fa7d4159780 2024-09-15T21:48:37.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.496+0000 7fa7d9a41640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa7d40c5600 0x7fa7d406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.496+0000 7fa7d9a41640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa7d40c5600 0x7fa7d406e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54014/0 (socket says 172.21.15.100:54014) 2024-09-15T21:48:37.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.496+0000 7fa7d9a41640 1 -- 172.21.15.100:0/2238614197 learned_addr learned my addr 172.21.15.100:0/2238614197 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.496+0000 7fa7d9240640 1 --2- 172.21.15.100:0/2238614197 >> v2:172.21.15.122:3300/0 conn(0x7fa7d4159780 0x7fa7d40c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.496+0000 7fa7d9a41640 1 -- 172.21.15.100:0/2238614197 >> v2:172.21.15.122:3300/0 conn(0x7fa7d4159780 msgr2=0x7fa7d40c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:37.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.496+0000 7fa7d9a41640 1 --2- 172.21.15.100:0/2238614197 >> v2:172.21.15.122:3300/0 conn(0x7fa7d4159780 0x7fa7d40c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.496+0000 7fa7d9a41640 1 -- 172.21.15.100:0/2238614197 >> v1:172.21.15.122:6789/0 conn(0x7fa7d406f1f0 legacy=0x7fa7d40c4ef0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:48:37.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.496+0000 7fa7d9a41640 1 -- 172.21.15.100:0/2238614197 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d415ff20 con 0x7fa7d40c5600 2024-09-15T21:48:37.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.497+0000 7fa7d9a41640 1 --2- 172.21.15.100:0/2238614197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d40c5600 0x7fa7d406e350 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40704e0 tx=0x7fa7d002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87e7b73cdbf7a17 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.496+0000 7f4f495bc640 1 Processor -- start 2024-09-15T21:48:37.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.496+0000 7f4f495bc640 1 -- start start 2024-09-15T21:48:37.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.498+0000 7fa7cbfff640 1 -- 172.21.15.100:0/2238614197 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0037070 con 0x7fa7d40c5600 2024-09-15T21:48:37.501 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.498+0000 7fa7cbfff640 1 -- 172.21.15.100:0/2238614197 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7d0032440 con 0x7fa7d40c5600 2024-09-15T21:48:37.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.499+0000 7fa7da242640 1 -- 172.21.15.100:0/2238614197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d40c5600 msgr2=0x7fa7d406e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.499+0000 7fa7da242640 1 --2- 172.21.15.100:0/2238614197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d40c5600 0x7fa7d406e350 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40704e0 tx=0x7fa7d002a1a0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.498+0000 7f4f495bc640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f4f44168a80 0x7f4f440c34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.498+0000 7f4f495bc640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f4406f290 con 0x7f4f4408d600 2024-09-15T21:48:37.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.498+0000 7f4f495bc640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f4406f570 con 0x7f4f440c30d0 2024-09-15T21:48:37.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.498+0000 7f4f495bc640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f4f4406f850 con 0x7f4f44168a80 2024-09-15T21:48:37.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.498+0000 7f4f437fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f4f44168a80 0x7f4f440c34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.498+0000 7f4f43fff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f4f440c30d0 0x7f4f440c4af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.100:59342/0 (socket says 172.21.15.100:59342) 2024-09-15T21:48:37.502 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.498+0000 7f4f43fff640 1 -- 172.21.15.100:0/4284785006 learned_addr learned my addr 172.21.15.100:0/4284785006 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:37.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.499+0000 7f4f437fe640 1 -- 172.21.15.100:0/4284785006 >> v1:172.21.15.122:6789/0 conn(0x7f4f440c30d0 legacy=0x7f4f440c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.499+0000 7f4f437fe640 1 -- 172.21.15.100:0/4284785006 >> v1:172.21.15.100:6789/0 conn(0x7f4f4408d600 legacy=0x7f4f4406e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:37.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.499+0000 7f9586af1640 1 Processor -- start 2024-09-15T21:48:37.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.497+0000 7fad7aa82640 1 -- start start 2024-09-15T21:48:37.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.499+0000 7fad7aa82640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fad6c0059f0 0x7fad6c0da530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.499+0000 7fad7aa82640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fad6c04ae60 0x7fad6c04b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.499+0000 7fad7aa82640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad6c0d49c0 con 0x7fad6c0d45e0 2024-09-15T21:48:37.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.499+0000 7fad7aa82640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fad6c032c00 con 0x7fad6c0059f0 2024-09-15T21:48:37.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.499+0000 7fad7aa82640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fad6c005fc0 con 0x7fad6c04ae60 2024-09-15T21:48:37.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.499+0000 7fad79a80640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fad6c0d45e0 0x7fad6c034240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42062/0 (socket says 172.21.15.100:42062) 2024-09-15T21:48:37.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.499+0000 7fad79a80640 1 -- 172.21.15.100:0/1063308760 learned_addr learned my addr 172.21.15.100:0/1063308760 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:37.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.499+0000 7fad7927f640 1 --2- 172.21.15.100:0/1063308760 >> v2:172.21.15.100:3300/0 conn(0x7fad6c0059f0 0x7fad6c0da530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.499+0000 7fad7a281640 1 --2- 172.21.15.100:0/1063308760 >> v2:172.21.15.122:3300/0 conn(0x7fad6c04ae60 0x7fad6c04b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.500+0000 7fad7927f640 1 -- 172.21.15.100:0/1063308760 >> v2:172.21.15.122:3300/0 conn(0x7fad6c04ae60 msgr2=0x7fad6c04b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:37.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.500+0000 7fad7927f640 1 --2- 172.21.15.100:0/1063308760 >> v2:172.21.15.122:3300/0 conn(0x7fad6c04ae60 0x7fad6c04b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.500+0000 7fad7927f640 1 -- 172.21.15.100:0/1063308760 >> v1:172.21.15.100:6789/0 conn(0x7fad6c0d45e0 legacy=0x7fad6c034240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.500+0000 7fad7927f640 1 -- 172.21.15.100:0/1063308760 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c0062a0 con 0x7fad6c0059f0 2024-09-15T21:48:37.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.499+0000 7f4f437fe640 1 -- 172.21.15.100:0/4284785006 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f44167fa0 con 0x7f4f44168a80 2024-09-15T21:48:37.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.499+0000 7f9586af1640 1 -- start start 2024-09-15T21:48:37.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.500+0000 7fa7da242640 1 -- 172.21.15.100:0/2238614197 shutdown_connections 2024-09-15T21:48:37.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.500+0000 7fa7da242640 1 --2- 172.21.15.100:0/2238614197 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d40c5600 0x7fa7d406e350 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.500+0000 7fa7da242640 1 --2- 172.21.15.100:0/2238614197 >> v2:172.21.15.122:3300/0 conn(0x7fa7d4159780 0x7fa7d40c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.500+0000 7fa7da242640 1 -- 172.21.15.100:0/2238614197 >> 172.21.15.100:0/2238614197 conn(0x7fa7d40c9110 msgr2=0x7fa7d40c9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.501+0000 7fa9e69ee640 1 Processor -- start 2024-09-15T21:48:37.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.501+0000 7f1fc50d0640 1 Processor -- start 2024-09-15T21:48:37.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.501+0000 7f6302d5b640 1 Processor -- start 2024-09-15T21:48:37.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.501+0000 7f1fc50d0640 1 -- start start 2024-09-15T21:48:37.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.501+0000 7f4f437fe640 1 --2- 172.21.15.100:0/4284785006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f44168a80 0x7f4f440c34b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009d90 tx=0x7f4f3c02a580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=937a2edba74e40c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.501+0000 7fa7da242640 1 -- 172.21.15.100:0/2238614197 shutdown_connections 2024-09-15T21:48:37.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.501+0000 7fa7da242640 1 -- 172.21.15.100:0/2238614197 wait complete. 2024-09-15T21:48:37.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.501+0000 7f9586af1640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95800c9a40 0x7f9580157040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.501+0000 7f9586af1640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f95800c6d20 0x7f9580151170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.501+0000 7fa9e69ee640 1 -- start start 2024-09-15T21:48:37.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.501+0000 7f9586af1640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95800c7210 con 0x7f95800c3fc0 2024-09-15T21:48:37.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.501+0000 7f9586af1640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f95800c8590 con 0x7f95800c6d20 2024-09-15T21:48:37.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.501+0000 7f9586af1640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f95800c74f0 con 0x7f95800c9a40 2024-09-15T21:48:37.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.501+0000 7f958488d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95800c9a40 0x7f9580157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.501+0000 7f957ffff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f95800c6d20 0x7f9580151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.501+0000 7f957ffff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f95800c6d20 0x7f9580151170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:42792/0 (socket says 172.21.15.100:42792) 2024-09-15T21:48:37.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.501+0000 7f957ffff640 1 -- 172.21.15.100:0/2314377990 learned_addr learned my addr 172.21.15.100:0/2314377990 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.501+0000 7f6302d5b640 1 -- start start 2024-09-15T21:48:37.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7f957ffff640 1 -- 172.21.15.100:0/2314377990 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95800c9a40 msgr2=0x7f9580157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:37.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7f957ffff640 1 --2- 172.21.15.100:0/2314377990 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95800c9a40 0x7f9580157040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7f957ffff640 1 -- 172.21.15.100:0/2314377990 >> v1:172.21.15.100:6789/0 conn(0x7f95800c3fc0 legacy=0x7f958006dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7f957ffff640 1 -- 172.21.15.100:0/2314377990 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95800c77d0 con 0x7f95800c6d20 2024-09-15T21:48:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7f1fc50d0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1fc0157bb0 0x7f1fc0158490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7f1fc50d0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f1fc01589d0 0x7f1fc00cc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7f1fc50d0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fc0151650 con 0x7f1fc0151270 2024-09-15T21:48:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7f1fc50d0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f1fc00c8420 con 0x7f1fc01589d0 2024-09-15T21:48:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7f1fc50d0640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc00c9200 con 0x7f1fc0157bb0 2024-09-15T21:48:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7f1fbed9d640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f1fc0151270 0x7f1fc00c77d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42066/0 (socket says 172.21.15.100:42066) 2024-09-15T21:48:37.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7f1fbed9d640 1 -- 172.21.15.100:0/3556285758 learned_addr learned my addr 172.21.15.100:0/3556285758 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:37.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7fa9e69ee640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fa9e00c62a0 0x7fa9e0073a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7fa9e69ee640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9e00c6680 0x7fa9e006c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7fa9e69ee640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa9e006c950 0x7fa9e015c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7fa9e69ee640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fa9e0071310 con 0x7fa9e006c950 2024-09-15T21:48:37.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7f1fbf59e640 1 --2- 172.21.15.100:0/3556285758 >> v2:172.21.15.100:3300/0 conn(0x7f1fc01589d0 0x7f1fc00cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7fa9e69ee640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fa9e00715f0 con 0x7fa9e00c62a0 2024-09-15T21:48:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7fa9e69ee640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e00718d0 con 0x7fa9e00c6680 2024-09-15T21:48:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7fa9e59ec640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fa9e00c62a0 0x7fa9e0073a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7f1fbdd9b640 1 -- 172.21.15.100:0/3556285758 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2411427790 0 0) 0x7f1fc0151650 con 0x7f1fc0151270 2024-09-15T21:48:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7f1fbdd9b640 1 -- 172.21.15.100:0/3556285758 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1fa8003520 con 0x7f1fc0151270 2024-09-15T21:48:37.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7fa9e51eb640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9e00c6680 0x7fa9e006c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7f6302d5b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f62f40091b0 0x7f62f404e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7f6302d5b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62f404df00 con 0x7f62f40dccc0 2024-09-15T21:48:37.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7f6302d5b640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62f40e60b0 con 0x7f62f40e6b50 2024-09-15T21:48:37.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7f6302d5b640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f62f40e6fa0 con 0x7f62f40091b0 2024-09-15T21:48:37.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7fd171401640 1 Processor -- start 2024-09-15T21:48:37.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7fa9e61ed640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa9e006c950 0x7fa9e015c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7fa9e61ed640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fa9e006c950 0x7fa9e015c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54046/0 (socket says 172.21.15.100:54046) 2024-09-15T21:48:37.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7fa9e61ed640 1 -- 172.21.15.100:0/3783411576 learned_addr learned my addr 172.21.15.100:0/3783411576 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7fd171401640 1 -- start start 2024-09-15T21:48:37.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7fa7da242640 1 Processor -- start 2024-09-15T21:48:37.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7fa7da242640 1 -- start start 2024-09-15T21:48:37.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7f6301d59640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f62f40dccc0 0x7f62f404cc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42076/0 (socket says 172.21.15.100:42076) 2024-09-15T21:48:37.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7f6301d59640 1 -- 172.21.15.100:0/3351753391 learned_addr learned my addr 172.21.15.100:0/3351753391 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7fad7927f640 1 --2- 172.21.15.100:0/1063308760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad6c0059f0 0x7fad6c0da530 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fad74000b60 tx=0x7fad740a9550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c328221ff3135446 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7fad78a7e640 1 -- 172.21.15.100:0/1063308760 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad740b0600 con 0x7fad6c0059f0 2024-09-15T21:48:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7f6301558640 1 --2- 172.21.15.100:0/3351753391 >> v2:172.21.15.122:3300/0 conn(0x7f62f40091b0 0x7f62f404e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7f6300d57640 1 -- 172.21.15.100:0/3351753391 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1463038458 0 0) 0x7f62f40e60b0 con 0x7f62f40e6b50 2024-09-15T21:48:37.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7f6300d57640 1 -- 172.21.15.100:0/3351753391 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f62e8003520 con 0x7f62f40e6b50 2024-09-15T21:48:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7f4f42ffd640 1 -- 172.21.15.100:0/4284785006 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c007740 con 0x7f4f44168a80 2024-09-15T21:48:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7f4f42ffd640 1 -- 172.21.15.100:0/4284785006 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f3c0078e0 con 0x7f4f44168a80 2024-09-15T21:48:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.502+0000 7f4f42ffd640 1 -- 172.21.15.100:0/4284785006 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c007cb0 con 0x7f4f44168a80 2024-09-15T21:48:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7f4f495bc640 1 -- 172.21.15.100:0/4284785006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f44168a80 msgr2=0x7f4f440c34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7f4f495bc640 1 --2- 172.21.15.100:0/4284785006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f44168a80 0x7f4f440c34b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009d90 tx=0x7f4f3c02a580 comp rx=0 tx=0).stop 2024-09-15T21:48:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7fad78a7e640 1 -- 172.21.15.100:0/1063308760 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad740b07a0 con 0x7fad6c0059f0 2024-09-15T21:48:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7fad78a7e640 1 -- 172.21.15.100:0/1063308760 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad740b0bd0 con 0x7fad6c0059f0 2024-09-15T21:48:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7f4f495bc640 1 -- 172.21.15.100:0/4284785006 shutdown_connections 2024-09-15T21:48:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7f4f495bc640 1 --2- 172.21.15.100:0/4284785006 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f44168a80 0x7f4f440c34b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7f4f495bc640 1 -- 172.21.15.100:0/4284785006 >> 172.21.15.100:0/4284785006 conn(0x7f4f440c9110 msgr2=0x7f4f440c9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7f957ffff640 1 --2- 172.21.15.100:0/2314377990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95800c6d20 0x7f9580151170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9578009e50 tx=0x7f957802a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1a43c73646b574b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7f957f7fe640 1 -- 172.21.15.100:0/2314377990 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9578036070 con 0x7f95800c6d20 2024-09-15T21:48:37.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7f957f7fe640 1 -- 172.21.15.100:0/2314377990 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9578031600 con 0x7f95800c6d20 2024-09-15T21:48:37.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7f957f7fe640 1 -- 172.21.15.100:0/2314377990 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9578031a10 con 0x7f95800c6d20 2024-09-15T21:48:37.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7f4f495bc640 1 -- 172.21.15.100:0/4284785006 shutdown_connections 2024-09-15T21:48:37.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7f6301558640 1 -- 172.21.15.100:0/3351753391 >> v1:172.21.15.122:6789/0 conn(0x7f62f40e6b50 legacy=0x7f62f4051df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7f6301558640 1 -- 172.21.15.100:0/3351753391 >> v1:172.21.15.100:6789/0 conn(0x7f62f40dccc0 legacy=0x7f62f404cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:37.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7f6301558640 1 -- 172.21.15.100:0/3351753391 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f4046e50 con 0x7f62f40091b0 2024-09-15T21:48:37.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fa7da242640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d406f1f0 0x7fa7d4156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fa7da242640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa7d40c5600 0x7fa7d415c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fd171401640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd16c0c7f40 0x7fd16c072850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7f1fbf59e640 1 -- 172.21.15.100:0/3556285758 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1fc0157bb0 msgr2=0x7f1fc0158490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T21:48:37.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7f1fbf59e640 1 --2- 172.21.15.100:0/3556285758 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1fc0157bb0 0x7f1fc0158490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7f1fbf59e640 1 -- 172.21.15.100:0/3556285758 >> v1:172.21.15.100:6789/0 conn(0x7f1fc0151270 legacy=0x7f1fc00c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.503+0000 7f1fbf59e640 1 -- 172.21.15.100:0/3556285758 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc00c94e0 con 0x7f1fc01589d0 2024-09-15T21:48:37.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fd171401640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd16c0c88c0 0x7fd16c0cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fd171401640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd16c152370 con 0x7fd16c151f90 2024-09-15T21:48:37.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fd171401640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fd16c1584d0 con 0x7fd16c0c88c0 2024-09-15T21:48:37.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fd171401640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd16c1587b0 con 0x7fd16c0c7f40 2024-09-15T21:48:37.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fd16b7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd16c0c88c0 0x7fd16c0cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fad7aa82640 1 -- 172.21.15.100:0/1063308760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad6c0059f0 msgr2=0x7fad6c0da530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fad7aa82640 1 --2- 172.21.15.100:0/1063308760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad6c0059f0 0x7fad6c0da530 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fad74000b60 tx=0x7fad740a9550 comp rx=0 tx=0).stop 2024-09-15T21:48:37.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fd16b7fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd16c0c88c0 0x7fd16c0cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54066/0 (socket says 172.21.15.100:54066) 2024-09-15T21:48:37.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fd16b7fe640 1 -- 172.21.15.100:0/2674319865 learned_addr learned my addr 172.21.15.100:0/2674319865 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7fad7aa82640 1 -- 172.21.15.100:0/1063308760 shutdown_connections 2024-09-15T21:48:37.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7fad7aa82640 1 --2- 172.21.15.100:0/1063308760 >> v2:172.21.15.122:3300/0 conn(0x7fad6c04ae60 0x7fad6c04b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7fad7aa82640 1 --2- 172.21.15.100:0/1063308760 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad6c0059f0 0x7fad6c0da530 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7fad7aa82640 1 -- 172.21.15.100:0/1063308760 >> 172.21.15.100:0/1063308760 conn(0x7fad6c04fc40 msgr2=0x7fad6c050050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fa9e59ec640 1 -- 172.21.15.100:0/3783411576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9e00c6680 msgr2=0x7fa9e006c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:37.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fa9e59ec640 1 --2- 172.21.15.100:0/3783411576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9e00c6680 0x7fa9e006c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fa9e59ec640 1 -- 172.21.15.100:0/3783411576 >> v2:172.21.15.100:3300/0 conn(0x7fa9e006c950 msgr2=0x7fa9e015c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:37.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fa9e59ec640 1 --2- 172.21.15.100:0/3783411576 >> v2:172.21.15.100:3300/0 conn(0x7fa9e006c950 0x7fa9e015c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fa9e59ec640 1 -- 172.21.15.100:0/3783411576 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e0071bb0 con 0x7fa9e00c62a0 2024-09-15T21:48:37.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fa7da242640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7d4159780 0x7fa7d4156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fa7da242640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d406e890 con 0x7fa7d406f1f0 2024-09-15T21:48:37.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fa7da242640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d4159b60 con 0x7fa7d4159780 2024-09-15T21:48:37.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7f9586af1640 1 -- 172.21.15.100:0/2314377990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95800c6d20 msgr2=0x7f9580151170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7f9586af1640 1 --2- 172.21.15.100:0/2314377990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95800c6d20 0x7f9580151170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9578009e50 tx=0x7f957802a730 comp rx=0 tx=0).stop 2024-09-15T21:48:37.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7f1fbf59e640 1 --2- 172.21.15.100:0/3556285758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc01589d0 0x7f1fc00cc620 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00087a0 tx=0x7f1fb002c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34e5ff8afc75091 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fa7da242640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa7d406ea10 con 0x7fa7d40c5600 2024-09-15T21:48:37.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fa7d9240640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d406f1f0 0x7fa7d4156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fa7d9240640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d406f1f0 0x7fa7d4156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54054/0 (socket says 172.21.15.100:54054) 2024-09-15T21:48:37.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fa7d9240640 1 -- 172.21.15.100:0/366678837 learned_addr learned my addr 172.21.15.100:0/366678837 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7fa7d9a41640 1 --2- 172.21.15.100:0/366678837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7d4159780 0x7fa7d4156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7f6301558640 1 --2- 172.21.15.100:0/3351753391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62f40091b0 0x7f62f404e330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0097a0 tx=0x7f62ec02a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b9c09e5742bfcdf server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.504+0000 7f4f495bc640 1 -- 172.21.15.100:0/4284785006 wait complete. 2024-09-15T21:48:37.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7fd16a7fc640 1 --2- 172.21.15.100:0/2674319865 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd16c0c7f40 0x7fd16c072850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7f1fbdd9b640 1 -- 172.21.15.100:0/3556285758 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb003a070 con 0x7f1fc01589d0 2024-09-15T21:48:37.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7fad7aa82640 1 -- 172.21.15.100:0/1063308760 shutdown_connections 2024-09-15T21:48:37.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7f1fbdd9b640 1 -- 172.21.15.100:0/3556285758 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb0032480 con 0x7f1fc01589d0 2024-09-15T21:48:37.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7f1fbdd9b640 1 -- 172.21.15.100:0/3556285758 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0032850 con 0x7f1fc01589d0 2024-09-15T21:48:37.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7fd16b7fe640 1 -- 172.21.15.100:0/2674319865 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd16c0c7f40 msgr2=0x7fd16c072850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:37.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7fd16b7fe640 1 --2- 172.21.15.100:0/2674319865 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd16c0c7f40 0x7fd16c072850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7fd169ffb640 1 -- 172.21.15.100:0/2674319865 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4266474062 0 0) 0x7fd16c152370 con 0x7fd16c151f90 2024-09-15T21:48:37.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7fa7d9240640 1 -- 172.21.15.100:0/366678837 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa7d40c5600 msgr2=0x7fa7d415c240 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T21:48:37.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7f6300d57640 1 -- 172.21.15.100:0/3351753391 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec037070 con 0x7f62f40091b0 2024-09-15T21:48:37.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7fd16b7fe640 1 -- 172.21.15.100:0/2674319865 >> v1:172.21.15.122:6789/0 conn(0x7fd16c151f90 legacy=0x7fd16c157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7fd16b7fe640 1 -- 172.21.15.100:0/2674319865 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16c158a90 con 0x7fd16c0c88c0 2024-09-15T21:48:37.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7f6300d57640 1 -- 172.21.15.100:0/3351753391 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62ec032440 con 0x7f62f40091b0 2024-09-15T21:48:37.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7f6300d57640 1 -- 172.21.15.100:0/3351753391 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ec032810 con 0x7f62f40091b0 2024-09-15T21:48:37.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7f4f495bc640 1 Processor -- start 2024-09-15T21:48:37.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7fad7aa82640 1 -- 172.21.15.100:0/1063308760 wait complete. 2024-09-15T21:48:37.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.506+0000 7fa9e59ec640 1 --2- 172.21.15.100:0/3783411576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9e00c62a0 0x7fa9e0073a80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00ca510 tx=0x7fa9dc02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d57cd64717bfb0ff server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.506+0000 7fa9e49ea640 1 -- 172.21.15.100:0/3783411576 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc037070 con 0x7fa9e00c62a0 2024-09-15T21:48:37.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7fa7d9240640 1 --2- 172.21.15.100:0/366678837 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa7d40c5600 0x7fa7d415c240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7fa7d9240640 1 -- 172.21.15.100:0/366678837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7d4159780 msgr2=0x7fa7d4156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7fa7d9240640 1 --2- 172.21.15.100:0/366678837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7d4159780 0x7fa7d4156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.505+0000 7fa7d9240640 1 -- 172.21.15.100:0/366678837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d4073900 con 0x7fa7d406f1f0 2024-09-15T21:48:37.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.506+0000 7fa7d9240640 1 --2- 172.21.15.100:0/366678837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d406f1f0 0x7fa7d4156570 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00eed0 tx=0x7fa7cc00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.506+0000 7fa7d9a41640 1 --2- 172.21.15.100:0/366678837 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7d4159780 0x7fa7d4156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:48:37.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.506+0000 7fa7c9ffb640 1 -- 172.21.15.100:0/366678837 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc018020 con 0x7fa7d406f1f0 2024-09-15T21:48:37.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.506+0000 7fa9e49ea640 1 -- 172.21.15.100:0/3783411576 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9dc032440 con 0x7fa9e00c62a0 2024-09-15T21:48:37.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.506+0000 7fa9e49ea640 1 -- 172.21.15.100:0/3783411576 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc032810 con 0x7fa9e00c62a0 2024-09-15T21:48:37.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.506+0000 7f1fc50d0640 1 -- 172.21.15.100:0/3556285758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc01589d0 msgr2=0x7f1fc00cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.506+0000 7f9586af1640 1 -- 172.21.15.100:0/2314377990 shutdown_connections 2024-09-15T21:48:37.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.506+0000 7f9586af1640 1 --2- 172.21.15.100:0/2314377990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95800c6d20 0x7f9580151170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.506+0000 7f9586af1640 1 --2- 172.21.15.100:0/2314377990 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95800c9a40 0x7f9580157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.506+0000 7f9586af1640 1 -- 172.21.15.100:0/2314377990 >> 172.21.15.100:0/2314377990 conn(0x7f95800c8f20 msgr2=0x7f95800c9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.506+0000 7f6302d5b640 1 -- 172.21.15.100:0/3351753391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62f40091b0 msgr2=0x7f62f404e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.506+0000 7f6302d5b640 1 --2- 172.21.15.100:0/3351753391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62f40091b0 0x7f62f404e330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0097a0 tx=0x7f62ec02a1e0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.506+0000 7f1fc50d0640 1 --2- 172.21.15.100:0/3556285758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc01589d0 0x7f1fc00cc620 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1fb00087a0 tx=0x7f1fb002c730 comp rx=0 tx=0).stop 2024-09-15T21:48:37.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f6302d5b640 1 -- 172.21.15.100:0/3351753391 shutdown_connections 2024-09-15T21:48:37.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f6302d5b640 1 --2- 172.21.15.100:0/3351753391 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62f40091b0 0x7f62f404e330 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f6302d5b640 1 -- 172.21.15.100:0/3351753391 >> 172.21.15.100:0/3351753391 conn(0x7f62f400db90 msgr2=0x7f62f400df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f9586af1640 1 -- 172.21.15.100:0/2314377990 shutdown_connections 2024-09-15T21:48:37.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.506+0000 7fa7da242640 1 -- 172.21.15.100:0/366678837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7d4073be0 con 0x7fa7d406f1f0 2024-09-15T21:48:37.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.506+0000 7fa7da242640 1 -- 172.21.15.100:0/366678837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7d41580c0 con 0x7fa7d406f1f0 2024-09-15T21:48:37.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7fa7c9ffb640 1 -- 172.21.15.100:0/366678837 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7cc00bda0 con 0x7fa7d406f1f0 2024-09-15T21:48:37.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7fa7c9ffb640 1 -- 172.21.15.100:0/366678837 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc014620 con 0x7fa7d406f1f0 2024-09-15T21:48:37.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f9586af1640 1 -- 172.21.15.100:0/2314377990 wait complete. 2024-09-15T21:48:37.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.506+0000 7f4f495bc640 1 -- start start 2024-09-15T21:48:37.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7fa9e69ee640 1 -- 172.21.15.100:0/3783411576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9e00c62a0 msgr2=0x7fa9e0073a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7fa9e69ee640 1 --2- 172.21.15.100:0/3783411576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9e00c62a0 0x7fa9e0073a80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00ca510 tx=0x7fa9dc02a1a0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f6302d5b640 1 -- 172.21.15.100:0/3351753391 shutdown_connections 2024-09-15T21:48:37.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7fad7aa82640 1 Processor -- start 2024-09-15T21:48:37.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f4f495bc640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4f4408d600 0x7f4f4420fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7fad7aa82640 1 -- start start 2024-09-15T21:48:37.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f6302d5b640 1 -- 172.21.15.100:0/3351753391 wait complete. 2024-09-15T21:48:37.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7fd16b7fe640 1 --2- 172.21.15.100:0/2674319865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd16c0c88c0 0x7fd16c0cd4b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd160009870 tx=0x7fd16002a2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2871f24c002ba9b server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f4f495bc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f440c30d0 0x7f4f44211d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f1fc50d0640 1 -- 172.21.15.100:0/3556285758 shutdown_connections 2024-09-15T21:48:37.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f1fc50d0640 1 --2- 172.21.15.100:0/3556285758 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc01589d0 0x7f1fc00cc620 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f1fc50d0640 1 --2- 172.21.15.100:0/3556285758 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1fc0157bb0 0x7f1fc0158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f1fc50d0640 1 -- 172.21.15.100:0/3556285758 >> 172.21.15.100:0/3556285758 conn(0x7f1fc0067dd0 msgr2=0x7f1fc006bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f4f495bc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f44168a80 0x7f4f44204cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f4f495bc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4f440c4520 con 0x7f4f440c30d0 2024-09-15T21:48:37.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f4f495bc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4f44168560 con 0x7f4f44168a80 2024-09-15T21:48:37.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f4f495bc640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4f440c46a0 con 0x7f4f4408d600 2024-09-15T21:48:37.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f4f48dbb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f44168a80 0x7f4f44204cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f4f48dbb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f44168a80 0x7f4f44204cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:42818/0 (socket says 172.21.15.100:42818) 2024-09-15T21:48:37.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.507+0000 7f4f48dbb640 1 -- 172.21.15.100:0/4236246704 learned_addr learned my addr 172.21.15.100:0/4236246704 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7fd169ffb640 1 -- 172.21.15.100:0/2674319865 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160037070 con 0x7fd16c0c88c0 2024-09-15T21:48:37.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7fd169ffb640 1 -- 172.21.15.100:0/2674319865 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd160032640 con 0x7fd16c0c88c0 2024-09-15T21:48:37.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7fd169ffb640 1 -- 172.21.15.100:0/2674319865 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160032a10 con 0x7fd16c0c88c0 2024-09-15T21:48:37.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7f4f437fe640 1 --2- 172.21.15.100:0/4236246704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f440c30d0 0x7f4f44211d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7fa7da242640 1 -- 172.21.15.100:0/366678837 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa7b4001030 con 0x7fa7d406f1f0 2024-09-15T21:48:37.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7fa9e69ee640 1 -- 172.21.15.100:0/3783411576 shutdown_connections 2024-09-15T21:48:37.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7fa9e69ee640 1 --2- 172.21.15.100:0/3783411576 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9e00c6680 0x7fa9e006c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7fa9e69ee640 1 --2- 172.21.15.100:0/3783411576 >> v2:172.21.15.100:3300/0 conn(0x7fa9e006c950 0x7fa9e015c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7fa9e69ee640 1 --2- 172.21.15.100:0/3783411576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9e00c62a0 0x7fa9e0073a80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7fa9e69ee640 1 -- 172.21.15.100:0/3783411576 >> 172.21.15.100:0/3783411576 conn(0x7fa9e006ecc0 msgr2=0x7fa9e006f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7f4f48dbb640 1 -- 172.21.15.100:0/4236246704 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4f4408d600 msgr2=0x7f4f4420fa80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T21:48:37.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7f4f48dbb640 1 --2- 172.21.15.100:0/4236246704 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4f4408d600 0x7f4f4420fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7f4f48dbb640 1 -- 172.21.15.100:0/4236246704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f440c30d0 msgr2=0x7f4f44211d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7f4f48dbb640 1 --2- 172.21.15.100:0/4236246704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f440c30d0 0x7f4f44211d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7f4f48dbb640 1 -- 172.21.15.100:0/4236246704 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4416bd70 con 0x7f4f44168a80 2024-09-15T21:48:37.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7fad7aa82640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fad6c0059f0 0x7fad6c04de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7f4f437fe640 1 --2- 172.21.15.100:0/4236246704 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4f440c30d0 0x7f4f44211d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:48:37.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7f9586af1640 1 Processor -- start 2024-09-15T21:48:37.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7fad7aa82640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad6c04ae60 0x7fad6c044c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7fad79a80640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fad6c0059f0 0x7fad6c04de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7fad7aa82640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad6c0d45e0 0x7fad6c045160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7fad79a80640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fad6c0059f0 0x7fad6c04de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:55508/0 (socket says 172.21.15.100:55508) 2024-09-15T21:48:37.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7fad7aa82640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c006b40 con 0x7fad6c04ae60 2024-09-15T21:48:37.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7fad79a80640 1 -- 172.21.15.100:0/1366976507 learned_addr learned my addr 172.21.15.100:0/1366976507 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7fad7aa82640 1 -- 172.21.15.100:0/1366976507 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fad6c006860 con 0x7fad6c0d45e0 2024-09-15T21:48:37.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7fad7aa82640 1 -- 172.21.15.100:0/1366976507 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fad6c006e20 con 0x7fad6c0059f0 2024-09-15T21:48:37.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7f4f48dbb640 1 --2- 172.21.15.100:0/4236246704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f44168a80 0x7f4f44204cb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800b410 tx=0x7f4f38009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7fad7a281640 1 --2- 172.21.15.100:0/1366976507 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad6c0d45e0 0x7fad6c045160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7fad7927f640 1 --2- 172.21.15.100:0/1366976507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad6c04ae60 0x7fad6c044c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.508+0000 7f9586af1640 1 -- start start 2024-09-15T21:48:37.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7f6302d5b640 1 Processor -- start 2024-09-15T21:48:37.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7fad7a281640 1 -- 172.21.15.100:0/1366976507 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fad6c0059f0 msgr2=0x7fad6c04de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7fad7a281640 1 --2- 172.21.15.100:0/1366976507 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fad6c0059f0 0x7fad6c04de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7fad7a281640 1 -- 172.21.15.100:0/1366976507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad6c04ae60 msgr2=0x7fad6c044c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7fad7a281640 1 --2- 172.21.15.100:0/1366976507 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad6c04ae60 0x7fad6c044c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7fad7a281640 1 -- 172.21.15.100:0/1366976507 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad6c168780 con 0x7fad6c0d45e0 2024-09-15T21:48:37.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7f6302d5b640 1 -- start start 2024-09-15T21:48:37.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7fd171401640 1 -- 172.21.15.100:0/2674319865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd16c0c88c0 msgr2=0x7fd16c0cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7f4f40ff9640 1 -- 172.21.15.100:0/4236246704 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38015020 con 0x7f4f44168a80 2024-09-15T21:48:37.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7fd171401640 1 --2- 172.21.15.100:0/2674319865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd16c0c88c0 0x7fd16c0cd4b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fd160009870 tx=0x7fd16002a2f0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7fa7c9ffb640 1 -- 172.21.15.100:0/366678837 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa7cc01b020 con 0x7fa7d406f1f0 2024-09-15T21:48:37.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7f4f40ff9640 1 -- 172.21.15.100:0/4236246704 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f38011830 con 0x7f4f44168a80 2024-09-15T21:48:37.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7fad7a281640 1 --2- 172.21.15.100:0/1366976507 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad6c0d45e0 0x7fad6c045160 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fad7000d590 tx=0x7fad7000da50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7fa9e69ee640 1 -- 172.21.15.100:0/3783411576 shutdown_connections 2024-09-15T21:48:37.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7fa9e69ee640 1 -- 172.21.15.100:0/3783411576 wait complete. 2024-09-15T21:48:37.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7fad6a7fc640 1 -- 172.21.15.100:0/1366976507 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad700125f0 con 0x7fad6c0d45e0 2024-09-15T21:48:37.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7fd171401640 1 -- 172.21.15.100:0/2674319865 shutdown_connections 2024-09-15T21:48:37.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.509+0000 7f1fc50d0640 1 -- 172.21.15.100:0/3556285758 shutdown_connections 2024-09-15T21:48:37.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7fad7aa82640 1 -- 172.21.15.100:0/1366976507 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad6c168a60 con 0x7fad6c0d45e0 2024-09-15T21:48:37.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f4f40ff9640 1 -- 172.21.15.100:0/4236246704 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38011c20 con 0x7f4f44168a80 2024-09-15T21:48:37.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7fa7c9ffb640 1 --2- 172.21.15.100:0/366678837 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa7b806c380 0x7fa7b806e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f4f495bc640 1 -- 172.21.15.100:0/4236246704 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f440cb810 con 0x7f4f44168a80 2024-09-15T21:48:37.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f4f495bc640 1 -- 172.21.15.100:0/4236246704 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f440c1300 con 0x7f4f44168a80 2024-09-15T21:48:37.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7fad7aa82640 1 -- 172.21.15.100:0/1366976507 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad6c169020 con 0x7fad6c0d45e0 2024-09-15T21:48:37.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f6302d5b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f62f40091b0 0x7f62f417c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f9586af1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95800c3fc0 0x7f9580157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f1fc50d0640 1 -- 172.21.15.100:0/3556285758 wait complete. 2024-09-15T21:48:37.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7fa7c9ffb640 1 -- 172.21.15.100:0/366678837 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7fa7cc08bdc0 con 0x7fa7d406f1f0 2024-09-15T21:48:37.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7fa7c9ffb640 1 --2- 172.21.15.100:0/366678837 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fa7b8074760 0x7fa7b8076c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7fa7c9ffb640 1 -- 172.21.15.100:0/366678837 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7b80772d0 con 0x7fa7b8074760 2024-09-15T21:48:37.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f6302d5b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f40dccc0 0x7f62f40dc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f6301558640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f40dccc0 0x7f62f40dc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f6301558640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f40dccc0 0x7f62f40dc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54122/0 (socket says 172.21.15.100:54122) 2024-09-15T21:48:37.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f9586af1640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95800c6d20 0x7f95801582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f9586af1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95800c9a40 0x7f95800c47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f9586af1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f95800c8070 con 0x7f95800c3fc0 2024-09-15T21:48:37.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f9586af1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f95800c7d90 con 0x7f95800c9a40 2024-09-15T21:48:37.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f9586af1640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f958006eae0 con 0x7f95800c6d20 2024-09-15T21:48:37.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f957ffff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95800c6d20 0x7f95801582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f957ffff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95800c6d20 0x7f95801582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:55514/0 (socket says 172.21.15.100:55514) 2024-09-15T21:48:37.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f957ffff640 1 -- 172.21.15.100:0/556177149 learned_addr learned my addr 172.21.15.100:0/556177149 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f958508e640 1 --2- 172.21.15.100:0/556177149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95800c9a40 0x7f95800c47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.511+0000 7f957ffff640 1 -- 172.21.15.100:0/556177149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95800c9a40 msgr2=0x7f95800c47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.511+0000 7f957ffff640 1 --2- 172.21.15.100:0/556177149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95800c9a40 0x7f95800c47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.511+0000 7f957ffff640 1 -- 172.21.15.100:0/556177149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95800c3fc0 msgr2=0x7f9580157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7fad6a7fc640 1 -- 172.21.15.100:0/1366976507 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad70019070 con 0x7fad6c0d45e0 2024-09-15T21:48:37.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7fad6a7fc640 1 -- 172.21.15.100:0/1366976507 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad700129b0 con 0x7fad6c0d45e0 2024-09-15T21:48:37.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f6302d5b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62f40e6b50 0x7f62f4051e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f6302d5b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f62f40e6630 con 0x7f62f40dccc0 2024-09-15T21:48:37.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f6302d5b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f62f40e6350 con 0x7f62f40e6b50 2024-09-15T21:48:37.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7f6302d5b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f62f40473f0 con 0x7f62f40091b0 2024-09-15T21:48:37.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.511+0000 7f6301d59640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f62f40091b0 0x7f62f417c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.511+0000 7f6301558640 1 -- 172.21.15.100:0/1889801944 learned_addr learned my addr 172.21.15.100:0/1889801944 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.511+0000 7f958488d640 1 --2- 172.21.15.100:0/556177149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95800c3fc0 0x7f9580157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.511+0000 7f957ffff640 1 --2- 172.21.15.100:0/556177149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95800c3fc0 0x7f9580157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.511+0000 7f957ffff640 1 -- 172.21.15.100:0/556177149 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9580153b40 con 0x7f95800c6d20 2024-09-15T21:48:37.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.511+0000 7f958488d640 1 --2- 172.21.15.100:0/556177149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f95800c3fc0 0x7f9580157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:48:37.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.511+0000 7f957ffff640 1 --2- 172.21.15.100:0/556177149 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95800c6d20 0x7f95801582e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9578000f80 tx=0x7f9578004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.511+0000 7f630255a640 1 --2- 172.21.15.100:0/1889801944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62f40e6b50 0x7f62f4051e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.511+0000 7f957d7fa640 1 -- 172.21.15.100:0/556177149 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9578041020 con 0x7f95800c6d20 2024-09-15T21:48:37.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7fd171401640 1 --2- 172.21.15.100:0/2674319865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd16c0c88c0 0x7fd16c0cd4b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7fd171401640 1 --2- 172.21.15.100:0/2674319865 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd16c0c7f40 0x7fd16c072850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.510+0000 7fd171401640 1 -- 172.21.15.100:0/2674319865 >> 172.21.15.100:0/2674319865 conn(0x7fd16c06ccf0 msgr2=0x7fd16c06d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.511+0000 7f6301d59640 1 -- 172.21.15.100:0/1889801944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62f40e6b50 msgr2=0x7f62f4051e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.512+0000 7f4f495bc640 1 -- 172.21.15.100:0/4236246704 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f440c34b0 con 0x7f4f44168a80 2024-09-15T21:48:37.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.511+0000 7f6301d59640 1 --2- 172.21.15.100:0/1889801944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62f40e6b50 0x7f62f4051e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.511+0000 7f6301d59640 1 -- 172.21.15.100:0/1889801944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f40dccc0 msgr2=0x7f62f40dc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.512+0000 7fad7aa82640 1 -- 172.21.15.100:0/1366976507 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fad6c1696e0 con 0x7fad6c0d45e0 2024-09-15T21:48:37.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.512+0000 7fa7d8a3f640 1 --2- 172.21.15.100:0/366678837 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa7b806c380 0x7fa7b806e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.511+0000 7f6301d59640 1 --2- 172.21.15.100:0/1889801944 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f62f40dccc0 0x7f62f40dc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.511+0000 7f6301d59640 1 -- 172.21.15.100:0/1889801944 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f404c000 con 0x7f62f40091b0 2024-09-15T21:48:37.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.512+0000 7f6301d59640 1 --2- 172.21.15.100:0/1889801944 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f62f40091b0 0x7f62f417c620 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f62f800c3e0 tx=0x7f62f800c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.511+0000 7fa9e69ee640 1 Processor -- start 2024-09-15T21:48:37.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.512+0000 7fa9e69ee640 1 -- start start 2024-09-15T21:48:37.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7fa9e69ee640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9e006c950 0x7fa9e00c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7fa9e69ee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e00c62a0 0x7fa9e015aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.511+0000 7f9586af1640 1 -- 172.21.15.100:0/556177149 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9580153de0 con 0x7f95800c6d20 2024-09-15T21:48:37.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.512+0000 7f9586af1640 1 -- 172.21.15.100:0/556177149 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9580154770 con 0x7f95800c6d20 2024-09-15T21:48:37.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.512+0000 7f957d7fa640 1 -- 172.21.15.100:0/556177149 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95780087b0 con 0x7f95800c6d20 2024-09-15T21:48:37.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7fa7d8a3f640 1 --2- 172.21.15.100:0/366678837 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa7b806c380 0x7fa7b806e840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa7d41eadc0 tx=0x7fa7c40079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7fa7d9a41640 1 --2- 172.21.15.100:0/366678837 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fa7b8074760 0x7fa7b8076c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7fa9e69ee640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9e00c6680 0x7fa9e01f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7fa9e59ec640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9e006c950 0x7fa9e00c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7fa9e51eb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e00c62a0 0x7fa9e015aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7fa9e69ee640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e0074140 con 0x7fa9e00c62a0 2024-09-15T21:48:37.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7fa9e51eb640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e00c62a0 0x7fa9e015aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54126/0 (socket says 172.21.15.100:54126) 2024-09-15T21:48:37.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7fa9e51eb640 1 -- 172.21.15.100:0/1316584600 learned_addr learned my addr 172.21.15.100:0/1316584600 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7fa9e59ec640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9e006c950 0x7fa9e00c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:55528/0 (socket says 172.21.15.100:55528) 2024-09-15T21:48:37.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7fa9e69ee640 1 -- 172.21.15.100:0/1316584600 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e0073fc0 con 0x7fa9e00c6680 2024-09-15T21:48:37.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.512+0000 7f62f27fc640 1 -- 172.21.15.100:0/1889801944 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f8017070 con 0x7f62f40091b0 2024-09-15T21:48:37.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.512+0000 7f62f27fc640 1 -- 172.21.15.100:0/1889801944 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62f8005ce0 con 0x7f62f40091b0 2024-09-15T21:48:37.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.512+0000 7f4f40ff9640 1 -- 172.21.15.100:0/4236246704 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4f38018020 con 0x7f4f44168a80 2024-09-15T21:48:37.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7f4f40ff9640 1 --2- 172.21.15.100:0/4236246704 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4f3006c380 0x7f4f3006e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7f4f40ff9640 1 -- 172.21.15.100:0/4236246704 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f4f3808b500 con 0x7f4f44168a80 2024-09-15T21:48:37.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7fa9e69ee640 1 -- 172.21.15.100:0/1316584600 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e00c3960 con 0x7fa9e006c950 2024-09-15T21:48:37.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7f62f27fc640 1 -- 172.21.15.100:0/1889801944 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f80072f0 con 0x7f62f40091b0 2024-09-15T21:48:37.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7f4f43fff640 1 --2- 172.21.15.100:0/4236246704 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4f3006c380 0x7f4f3006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7fad6a7fc640 1 -- 172.21.15.100:0/1366976507 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fad70012b50 con 0x7fad6c0d45e0 2024-09-15T21:48:37.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7fa9e61ed640 1 --2- 172.21.15.100:0/1316584600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9e00c6680 0x7fa9e01f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7f957d7fa640 1 -- 172.21.15.100:0/556177149 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9578036040 con 0x7f95800c6d20 2024-09-15T21:48:37.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7f4f40ff9640 1 --2- 172.21.15.100:0/4236246704 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f4f30074780 0x7f4f30076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7f4f40ff9640 1 -- 172.21.15.100:0/4236246704 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f3800ce50 con 0x7f4f30074780 2024-09-15T21:48:37.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7f4f40ff9640 1 -- 172.21.15.100:0/4236246704 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f4f38092050 con 0x7f4f44168a80 2024-09-15T21:48:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7f6302d5b640 1 -- 172.21.15.100:0/1889801944 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f404c2e0 con 0x7f62f40091b0 2024-09-15T21:48:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7f6302d5b640 1 -- 172.21.15.100:0/1889801944 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f404a580 con 0x7f62f40091b0 2024-09-15T21:48:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7f9566ffd640 1 -- 172.21.15.100:0/556177149 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9580155a90 con 0x7f95800c6d20 2024-09-15T21:48:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.512+0000 7f1fc50d0640 1 Processor -- start 2024-09-15T21:48:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.512+0000 7f1fc50d0640 1 -- start start 2024-09-15T21:48:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7f1fc50d0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fc0151270 0x7f1fc00ba170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7f1fc50d0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1fc0157bb0 0x7f1fc0151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7f1fc50d0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc01589d0 0x7f1fc01f8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7f1fc50d0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc00c7fe0 con 0x7f1fc01589d0 2024-09-15T21:48:37.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7f1fc50d0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc00c9aa0 con 0x7f1fc0151270 2024-09-15T21:48:37.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7f1fc50d0640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc00c8160 con 0x7f1fc0157bb0 2024-09-15T21:48:37.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.513+0000 7f4f437fe640 1 --2- 172.21.15.100:0/4236246704 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f4f30074780 0x7f4f30076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7fa9e59ec640 1 -- 172.21.15.100:0/1316584600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9e00c6680 msgr2=0x7fa9e01f41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7fa7c9ffb640 1 -- 172.21.15.100:0/366678837 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fa7cc0560a0 con 0x7fa7d406f1f0 2024-09-15T21:48:37.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7fa9e59ec640 1 --2- 172.21.15.100:0/1316584600 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9e00c6680 0x7fa9e01f41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7fa9e59ec640 1 -- 172.21.15.100:0/1316584600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e00c62a0 msgr2=0x7fa9e015aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7fa9e59ec640 1 --2- 172.21.15.100:0/1316584600 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9e00c62a0 0x7fa9e015aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7f1fbed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fc0151270 0x7f1fc00ba170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7f1fbed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fc0151270 0x7f1fc00ba170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:42858/0 (socket says 172.21.15.100:42858) 2024-09-15T21:48:37.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7fa9e59ec640 1 -- 172.21.15.100:0/1316584600 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e00c8b20 con 0x7fa9e006c950 2024-09-15T21:48:37.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7f1fbed9d640 1 -- 172.21.15.100:0/3606783447 learned_addr learned my addr 172.21.15.100:0/3606783447 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7f1fbf59e640 1 --2- 172.21.15.100:0/3606783447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc01589d0 0x7f1fc01f8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7fad6a7fc640 1 --2- 172.21.15.100:0/1366976507 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fad5806c3b0 0x7fad5806e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7f1fbf59e640 1 -- 172.21.15.100:0/3606783447 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1fc0157bb0 msgr2=0x7f1fc0151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7f1fbf59e640 1 --2- 172.21.15.100:0/3606783447 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f1fc0157bb0 0x7f1fc0151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7f1fbf59e640 1 -- 172.21.15.100:0/3606783447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fc0151270 msgr2=0x7f1fc00ba170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7f1fbf59e640 1 --2- 172.21.15.100:0/3606783447 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1fc0151270 0x7f1fc00ba170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7f1fbf59e640 1 -- 172.21.15.100:0/3606783447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc01f6930 con 0x7f1fc01589d0 2024-09-15T21:48:37.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7fad6a7fc640 1 -- 172.21.15.100:0/1366976507 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7fad7008b6e0 con 0x7fad6c0d45e0 2024-09-15T21:48:37.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7fd171401640 1 -- 172.21.15.100:0/2674319865 shutdown_connections 2024-09-15T21:48:37.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7fad79a80640 1 --2- 172.21.15.100:0/1366976507 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fad5806c3b0 0x7fad5806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.514+0000 7fa9e59ec640 1 --2- 172.21.15.100:0/1316584600 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9e006c950 0x7fa9e00c9590 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc031000 tx=0x7fa9dc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.515+0000 7fad6a7fc640 1 --2- 172.21.15.100:0/1366976507 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fad58074810 0x7fad58076cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.515+0000 7fad6a7fc640 1 -- 172.21.15.100:0/1366976507 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fad70001d30 con 0x7fad58074810 2024-09-15T21:48:37.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.515+0000 7fad6a7fc640 1 -- 172.21.15.100:0/1366976507 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fad70092050 con 0x7fad6c0d45e0 2024-09-15T21:48:37.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.515+0000 7f1fbf59e640 1 --2- 172.21.15.100:0/3606783447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc01589d0 0x7f1fc01f8f40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00c6cd0 tx=0x7f1fb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.515+0000 7f6302d5b640 1 -- 172.21.15.100:0/1889801944 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f62f404bdb0 con 0x7f62f40091b0 2024-09-15T21:48:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.515+0000 7fa9d67fc640 1 -- 172.21.15.100:0/1316584600 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc041030 con 0x7fa9e006c950 2024-09-15T21:48:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.515+0000 7fa9d67fc640 1 -- 172.21.15.100:0/1316584600 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9dc03d4e0 con 0x7fa9e006c950 2024-09-15T21:48:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.515+0000 7fa9d67fc640 1 -- 172.21.15.100:0/1316584600 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc041030 con 0x7fa9e006c950 2024-09-15T21:48:37.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.515+0000 7fad7927f640 1 --2- 172.21.15.100:0/1366976507 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fad58074810 0x7fad58076cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.515+0000 7fa9e69ee640 1 -- 172.21.15.100:0/1316584600 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e01538f0 con 0x7fa9e006c950 2024-09-15T21:48:37.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.515+0000 7fad79a80640 1 --2- 172.21.15.100:0/1366976507 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fad5806c3b0 0x7fad5806e870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fad64004e40 tx=0x7fad64007f10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.515+0000 7fd171401640 1 -- 172.21.15.100:0/2674319865 wait complete. 2024-09-15T21:48:37.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.515+0000 7f957d7fa640 1 -- 172.21.15.100:0/556177149 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9578044020 con 0x7f95800c6d20 2024-09-15T21:48:37.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.515+0000 7fa9e69ee640 1 -- 172.21.15.100:0/1316584600 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e00c8d90 con 0x7fa9e006c950 2024-09-15T21:48:37.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.516+0000 7f4f437fe640 1 --2- 172.21.15.100:0/4236246704 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f4f30074780 0x7f4f30076c20 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.580 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.515+0000 7fad7927f640 1 --2- 172.21.15.100:0/1366976507 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fad58074810 0x7fad58076cb0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.516+0000 7f957d7fa640 1 --2- 172.21.15.100:0/556177149 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f956c06c380 0x7f956c06e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.516+0000 7f957d7fa640 1 -- 172.21.15.100:0/556177149 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f95780abf80 con 0x7f95800c6d20 2024-09-15T21:48:37.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.516+0000 7f957d7fa640 1 --2- 172.21.15.100:0/556177149 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f956c0747e0 0x7f956c076c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.516+0000 7f957d7fa640 1 -- 172.21.15.100:0/556177149 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f95780087b0 con 0x7f956c0747e0 2024-09-15T21:48:37.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.516+0000 7f957d7fa640 1 -- 172.21.15.100:0/556177149 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f95780ae5e0 con 0x7f95800c6d20 2024-09-15T21:48:37.581 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.516+0000 7f1fa77fe640 1 -- 172.21.15.100:0/3606783447 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb003a040 con 0x7f1fc01589d0 2024-09-15T21:48:37.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.515+0000 7fa7d9a41640 1 --2- 172.21.15.100:0/366678837 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fa7b8074760 0x7fa7b8076c00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.516+0000 7fad6a7fc640 1 -- 172.21.15.100:0/1366976507 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fad70001d30 con 0x7fad58074810 2024-09-15T21:48:37.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.517+0000 7fa7c9ffb640 1 -- 172.21.15.100:0/366678837 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa7b80772d0 con 0x7fa7b8074760 2024-09-15T21:48:37.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.517+0000 7f958508e640 1 --2- 172.21.15.100:0/556177149 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f956c0747e0 0x7f956c076c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.517+0000 7f4f43fff640 1 --2- 172.21.15.100:0/4236246704 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4f3006c380 0x7f4f3006e840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4f34007fd0 tx=0x7f4f340079a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.516+0000 7f1fc50d0640 1 -- 172.21.15.100:0/3606783447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc014eae0 con 0x7f1fc01589d0 2024-09-15T21:48:37.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.516+0000 7f1fc50d0640 1 -- 172.21.15.100:0/3606783447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc01f5b70 con 0x7f1fc01589d0 2024-09-15T21:48:37.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.517+0000 7f1fa77fe640 1 -- 172.21.15.100:0/3606783447 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb0032070 con 0x7f1fc01589d0 2024-09-15T21:48:37.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.517+0000 7f1fa77fe640 1 -- 172.21.15.100:0/3606783447 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0032d90 con 0x7f1fc01589d0 2024-09-15T21:48:37.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.517+0000 7fd171401640 1 Processor -- start 2024-09-15T21:48:37.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.517+0000 7fd171401640 1 -- start start 2024-09-15T21:48:37.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.517+0000 7f62f27fc640 1 -- 172.21.15.100:0/1889801944 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f62f801a070 con 0x7f62f40091b0 2024-09-15T21:48:37.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.517+0000 7f1fc50d0640 1 -- 172.21.15.100:0/3606783447 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1fa0001030 con 0x7f1fc01589d0 2024-09-15T21:48:37.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.518+0000 7f4f40ff9640 1 -- 172.21.15.100:0/4236246704 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4f3800ce50 con 0x7f4f30074780 2024-09-15T21:48:37.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.518+0000 7f958488d640 1 --2- 172.21.15.100:0/556177149 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f956c06c380 0x7f956c06e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.517+0000 7fa9d67fc640 1 -- 172.21.15.100:0/1316584600 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa9dc03d690 con 0x7fa9e006c950 2024-09-15T21:48:37.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.518+0000 7fa9d67fc640 1 --2- 172.21.15.100:0/1316584600 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa9c406c6b0 0x7fa9c406eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.518+0000 7fa9d67fc640 1 -- 172.21.15.100:0/1316584600 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7fa9dc042070 con 0x7fa9e006c950 2024-09-15T21:48:37.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.518+0000 7fa9e69ee640 1 -- 172.21.15.100:0/1316584600 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa9e0157da0 con 0x7fa9e006c950 2024-09-15T21:48:37.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.519+0000 7fa9d67fc640 1 --2- 172.21.15.100:0/1316584600 >> [v2:172.21.15.122:6824/822572374,v1:172.21.15.122:6825/822572374] conn(0x7fa9c4074a90 0x7fa9c4076f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.519+0000 7fa9d67fc640 1 -- 172.21.15.100:0/1316584600 --> [v2:172.21.15.122:6824/822572374,v1:172.21.15.122:6825/822572374] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa9dc00af30 con 0x7fa9c4074a90 2024-09-15T21:48:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.519+0000 7fa9e51eb640 1 --2- 172.21.15.100:0/1316584600 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa9c406c6b0 0x7fa9c406eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.519+0000 7fa9e61ed640 1 --2- 172.21.15.100:0/1316584600 >> [v2:172.21.15.122:6824/822572374,v1:172.21.15.122:6825/822572374] conn(0x7fa9c4074a90 0x7fa9c4076f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.519+0000 7fa9d67fc640 1 -- 172.21.15.100:0/1316584600 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fa9dc075530 con 0x7fa9e006c950 2024-09-15T21:48:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.518+0000 7fd171401640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd16c0c7f40 0x7fd16c14db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.518+0000 7fd171401640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd16c0c88c0 0x7fd16c1f8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.518+0000 7fd171401640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd16c151f90 0x7fd16c06f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.518+0000 7fd171401640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c072330 con 0x7fd16c151f90 2024-09-15T21:48:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.518+0000 7fd171401640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd16c072050 con 0x7fd16c0c7f40 2024-09-15T21:48:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.518+0000 7fd171401640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd16c0c9070 con 0x7fd16c0c88c0 2024-09-15T21:48:37.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.519+0000 7fd16affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd16c0c7f40 0x7fd16c14db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.519+0000 7fd16affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd16c0c7f40 0x7fd16c14db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:42872/0 (socket says 172.21.15.100:42872) 2024-09-15T21:48:37.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.519+0000 7fd16affd640 1 -- 172.21.15.100:0/1529589970 learned_addr learned my addr 172.21.15.100:0/1529589970 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.518+0000 7f62f27fc640 1 --2- 172.21.15.100:0/1889801944 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f62e006c330 0x7f62e006e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.518+0000 7f62f27fc640 1 -- 172.21.15.100:0/1889801944 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f62f808a770 con 0x7f62f40091b0 2024-09-15T21:48:37.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.518+0000 7f62f27fc640 1 --2- 172.21.15.100:0/1889801944 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f62e0074710 0x7f62e0076bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.518+0000 7f62f27fc640 1 -- 172.21.15.100:0/1889801944 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f62f8011e50 con 0x7f62e0074710 2024-09-15T21:48:37.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.519+0000 7f62f27fc640 1 -- 172.21.15.100:0/1889801944 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f62f8054b70 con 0x7f62f40091b0 2024-09-15T21:48:37.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.519+0000 7f6301558640 1 --2- 172.21.15.100:0/1889801944 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f62e006c330 0x7f62e006e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.519+0000 7fd16affd640 1 -- 172.21.15.100:0/1529589970 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd16c0c88c0 msgr2=0x7fd16c1f8410 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T21:48:37.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.519+0000 7fd16affd640 1 --2- 172.21.15.100:0/1529589970 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd16c0c88c0 0x7fd16c1f8410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.519+0000 7fd16affd640 1 -- 172.21.15.100:0/1529589970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd16c151f90 msgr2=0x7fd16c06f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.519+0000 7fd16affd640 1 --2- 172.21.15.100:0/1529589970 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd16c151f90 0x7fd16c06f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.519+0000 7fd16affd640 1 -- 172.21.15.100:0/1529589970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16c0c7c30 con 0x7fd16c0c7f40 2024-09-15T21:48:37.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.519+0000 7fa9e51eb640 1 --2- 172.21.15.100:0/1316584600 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa9c406c6b0 0x7fa9c406eb70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa9e006f780 tx=0x7fa9d0009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.519+0000 7fd16affd640 1 --2- 172.21.15.100:0/1529589970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd16c0c7f40 0x7fd16c14db60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00c3c0 tx=0x7fd15c00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.518+0000 7f958508e640 1 --2- 172.21.15.100:0/556177149 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f956c0747e0 0x7f956c076c80 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.519+0000 7f958488d640 1 --2- 172.21.15.100:0/556177149 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f956c06c380 0x7f956c06e840 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f957000d440 tx=0x7f9570008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.519+0000 7f6301558640 1 --2- 172.21.15.100:0/1889801944 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f62e006c330 0x7f62e006e7f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f62f40e1950 tx=0x7f62ec02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.520+0000 7f957d7fa640 1 -- 172.21.15.100:0/556177149 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f95780087b0 con 0x7f956c0747e0 2024-09-15T21:48:37.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.520+0000 7fd1537fe640 1 -- 172.21.15.100:0/1529589970 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c017070 con 0x7fd16c0c7f40 2024-09-15T21:48:37.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.520+0000 7fd1537fe640 1 -- 172.21.15.100:0/1529589970 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd15c0035f0 con 0x7fd16c0c7f40 2024-09-15T21:48:37.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.520+0000 7fa9e61ed640 1 --2- 172.21.15.100:0/1316584600 >> [v2:172.21.15.122:6824/822572374,v1:172.21.15.122:6825/822572374] conn(0x7fa9c4074a90 0x7fa9c4076f30 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.520+0000 7fd1537fe640 1 -- 172.21.15.100:0/1529589970 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c0039e0 con 0x7fd16c0c7f40 2024-09-15T21:48:37.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.520+0000 7fd171401640 1 -- 172.21.15.100:0/1529589970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16c13c560 con 0x7fd16c0c7f40 2024-09-15T21:48:37.591 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.520+0000 7fd171401640 1 -- 172.21.15.100:0/1529589970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd16c0c7150 con 0x7fd16c0c7f40 2024-09-15T21:48:37.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.520+0000 7f1fa77fe640 1 -- 172.21.15.100:0/3606783447 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1fb0045020 con 0x7f1fc01589d0 2024-09-15T21:48:37.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.520+0000 7f630255a640 1 --2- 172.21.15.100:0/1889801944 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f62e0074710 0x7f62e0076bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.521+0000 7f1fa77fe640 1 --2- 172.21.15.100:0/3606783447 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1fac06c300 0x7f1fac06e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.521+0000 7f1fa77fe640 1 -- 172.21.15.100:0/3606783447 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f1fb00abde0 con 0x7f1fc01589d0 2024-09-15T21:48:37.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.521+0000 7f1fa77fe640 1 --2- 172.21.15.100:0/3606783447 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f1fac074700 0x7f1fac076ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.521+0000 7fa9d67fc640 1 -- 172.21.15.100:0/1316584600 <== osd.5 v2:172.21.15.122:6824/822572374 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa9dc00af30 con 0x7fa9c4074a90 2024-09-15T21:48:37.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.521+0000 7fd171401640 1 -- 172.21.15.100:0/1529589970 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd16c072850 con 0x7fd16c0c7f40 2024-09-15T21:48:37.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.521+0000 7f1fa77fe640 1 -- 172.21.15.100:0/3606783447 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1fb0007e60 con 0x7f1fac074700 2024-09-15T21:48:37.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.521+0000 7f1fa77fe640 1 -- 172.21.15.100:0/3606783447 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f1fb00ac1d0 con 0x7f1fc01589d0 2024-09-15T21:48:37.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.522+0000 7f1fbed9d640 1 --2- 172.21.15.100:0/3606783447 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1fac06c300 0x7f1fac06e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.522+0000 7f1fbe59c640 1 --2- 172.21.15.100:0/3606783447 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f1fac074700 0x7f1fac076ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.522+0000 7f1fbed9d640 1 --2- 172.21.15.100:0/3606783447 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1fac06c300 0x7f1fac06e7c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8000f80 tx=0x7f1fb8001700 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.522+0000 7fd1537fe640 1 -- 172.21.15.100:0/1529589970 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd15c01a070 con 0x7fd16c0c7f40 2024-09-15T21:48:37.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.522+0000 7f630255a640 1 --2- 172.21.15.100:0/1889801944 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f62e0074710 0x7f62e0076bb0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.523+0000 7f1fbe59c640 1 --2- 172.21.15.100:0/3606783447 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f1fac074700 0x7f1fac076ba0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.523+0000 7fd1537fe640 1 --2- 172.21.15.100:0/1529589970 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd15806c300 0x7fd15806e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.523+0000 7fd1537fe640 1 -- 172.21.15.100:0/1529589970 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7fd15c08a730 con 0x7fd16c0c7f40 2024-09-15T21:48:37.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.523+0000 7fd1537fe640 1 --2- 172.21.15.100:0/1529589970 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fd158074760 0x7fd158076c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.523+0000 7fd1537fe640 1 -- 172.21.15.100:0/1529589970 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd1580772d0 con 0x7fd158074760 2024-09-15T21:48:37.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.523+0000 7fd16a7fc640 1 --2- 172.21.15.100:0/1529589970 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd15806c300 0x7fd15806e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.523+0000 7fd1537fe640 1 -- 172.21.15.100:0/1529589970 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fd15c091050 con 0x7fd16c0c7f40 2024-09-15T21:48:37.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.523+0000 7f62f27fc640 1 -- 172.21.15.100:0/1889801944 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f62f8011e50 con 0x7f62e0074710 2024-09-15T21:48:37.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.523+0000 7fd16b7fe640 1 --2- 172.21.15.100:0/1529589970 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fd158074760 0x7fd158076c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.523+0000 7f1fa77fe640 1 -- 172.21.15.100:0/3606783447 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1fb0007e60 con 0x7f1fac074700 2024-09-15T21:48:37.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.524+0000 7fd16a7fc640 1 --2- 172.21.15.100:0/1529589970 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd15806c300 0x7fd15806e7c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0c54d0 tx=0x7fd164009830 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.524+0000 7fd16b7fe640 1 --2- 172.21.15.100:0/1529589970 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fd158074760 0x7fd158076c00 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.525+0000 7fd1537fe640 1 -- 172.21.15.100:0/1529589970 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1580772d0 con 0x7fd158074760 2024-09-15T21:48:37.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.548+0000 7fa7da242640 1 -- 172.21.15.100:0/366678837 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa7b4002dd0 con 0x7fa7b8074760 2024-09-15T21:48:37.597 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.548+0000 7fa7c9ffb640 1 -- 172.21.15.100:0/366678837 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa7b4002dd0 con 0x7fa7b8074760 2024-09-15T21:48:37.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.549+0000 7fad7aa82640 1 -- 172.21.15.100:0/1366976507 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fad6c033d40 con 0x7fad58074810 2024-09-15T21:48:37.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.549+0000 7fad6a7fc640 1 -- 172.21.15.100:0/1366976507 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fad6c033d40 con 0x7fad58074810 2024-09-15T21:48:37.598 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.549+0000 7f4f495bc640 1 -- 172.21.15.100:0/4236246704 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f440c0d80 con 0x7f4f30074780 2024-09-15T21:48:37.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.549+0000 7fa7da242640 1 -- 172.21.15.100:0/366678837 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fa7b8074760 msgr2=0x7fa7b8076c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.549+0000 7fa7da242640 1 --2- 172.21.15.100:0/366678837 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7fa7b8074760 0x7fa7b8076c00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7fad7aa82640 1 -- 172.21.15.100:0/1366976507 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fad58074810 msgr2=0x7fad58076cb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.599 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7fad7aa82640 1 --2- 172.21.15.100:0/1366976507 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7fad58074810 0x7fad58076cb0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7fa7da242640 1 -- 172.21.15.100:0/366678837 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa7b806c380 msgr2=0x7fa7b806e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7fa7da242640 1 --2- 172.21.15.100:0/366678837 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa7b806c380 0x7fa7b806e840 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa7d41eadc0 tx=0x7fa7c40079b0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7fa7da242640 1 -- 172.21.15.100:0/366678837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d406f1f0 msgr2=0x7fa7d4156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.600 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7fa7da242640 1 --2- 172.21.15.100:0/366678837 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7d406f1f0 0x7fa7d4156570 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00eed0 tx=0x7fa7cc00b480 comp rx=0 tx=0).stop 2024-09-15T21:48:37.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7fa7d9a41640 1 -- 172.21.15.100:0/366678837 reap_dead start 2024-09-15T21:48:37.601 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7f4f40ff9640 1 -- 172.21.15.100:0/4236246704 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4f440c0d80 con 0x7f4f30074780 2024-09-15T21:48:37.603 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7f9566ffd640 1 -- 172.21.15.100:0/556177149 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f958006f4f0 con 0x7f956c0747e0 2024-09-15T21:48:37.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7fa9e69ee640 1 -- 172.21.15.100:0/1316584600 --> [v2:172.21.15.122:6824/822572374,v1:172.21.15.122:6825/822572374] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa9e0158e60 con 0x7fa9c4074a90 2024-09-15T21:48:37.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7fad7aa82640 1 -- 172.21.15.100:0/1366976507 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fad5806c3b0 msgr2=0x7fad5806e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7fad7aa82640 1 --2- 172.21.15.100:0/1366976507 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fad5806c3b0 0x7fad5806e870 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fad64004e40 tx=0x7fad64007f10 comp rx=0 tx=0).stop 2024-09-15T21:48:37.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7fad7aa82640 1 -- 172.21.15.100:0/1366976507 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad6c0d45e0 msgr2=0x7fad6c045160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7fad7aa82640 1 --2- 172.21.15.100:0/1366976507 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fad6c0d45e0 0x7fad6c045160 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fad7000d590 tx=0x7fad7000da50 comp rx=0 tx=0).stop 2024-09-15T21:48:37.604 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7fad7a281640 1 -- 172.21.15.100:0/1366976507 reap_dead start 2024-09-15T21:48:37.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7fa7da242640 1 -- 172.21.15.100:0/366678837 shutdown_connections 2024-09-15T21:48:37.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7fad7aa82640 1 -- 172.21.15.100:0/1366976507 shutdown_connections 2024-09-15T21:48:37.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7fad7aa82640 1 -- 172.21.15.100:0/1366976507 >> 172.21.15.100:0/1366976507 conn(0x7fad6c04fc40 msgr2=0x7fad6c0d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7fa7da242640 1 -- 172.21.15.100:0/366678837 >> 172.21.15.100:0/366678837 conn(0x7fa7d40c9110 msgr2=0x7fa7d415d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.605 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7fa7da242640 1 -- 172.21.15.100:0/366678837 shutdown_connections 2024-09-15T21:48:37.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7f4f495bc640 1 -- 172.21.15.100:0/4236246704 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f4f30074780 msgr2=0x7f4f30076c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7f4f495bc640 1 --2- 172.21.15.100:0/4236246704 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7f4f30074780 0x7f4f30076c20 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.550+0000 7fad7aa82640 1 -- 172.21.15.100:0/1366976507 shutdown_connections 2024-09-15T21:48:37.606 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.551+0000 7fad7aa82640 1 -- 172.21.15.100:0/1366976507 wait complete. 2024-09-15T21:48:37.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.551+0000 7fa7da242640 1 -- 172.21.15.100:0/366678837 wait complete. 2024-09-15T21:48:37.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.551+0000 7f957d7fa640 1 -- 172.21.15.100:0/556177149 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f958006f4f0 con 0x7f956c0747e0 2024-09-15T21:48:37.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.551+0000 7f4f495bc640 1 -- 172.21.15.100:0/4236246704 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4f3006c380 msgr2=0x7f4f3006e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.551+0000 7f4f495bc640 1 --2- 172.21.15.100:0/4236246704 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4f3006c380 0x7f4f3006e840 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4f34007fd0 tx=0x7f4f340079a0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.551+0000 7f4f495bc640 1 -- 172.21.15.100:0/4236246704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f44168a80 msgr2=0x7f4f44204cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.551+0000 7f4f495bc640 1 --2- 172.21.15.100:0/4236246704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4f44168a80 0x7f4f44204cb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800b410 tx=0x7f4f38009f90 comp rx=0 tx=0).stop 2024-09-15T21:48:37.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.551+0000 7f4f48dbb640 1 -- 172.21.15.100:0/4236246704 reap_dead start 2024-09-15T21:48:37.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.551+0000 7f9566ffd640 1 -- 172.21.15.100:0/556177149 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f956c0747e0 msgr2=0x7f956c076c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.551+0000 7f9566ffd640 1 --2- 172.21.15.100:0/556177149 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f956c0747e0 0x7f956c076c80 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.551+0000 7f4f495bc640 1 -- 172.21.15.100:0/4236246704 shutdown_connections 2024-09-15T21:48:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.551+0000 7f4f495bc640 1 -- 172.21.15.100:0/4236246704 >> 172.21.15.100:0/4236246704 conn(0x7f4f440c9110 msgr2=0x7f4f44166660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.551+0000 7fa9d67fc640 1 -- 172.21.15.100:0/1316584600 <== osd.5 v2:172.21.15.122:6824/822572374 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa9e0158e60 con 0x7fa9c4074a90 2024-09-15T21:48:37.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.551+0000 7f4f495bc640 1 -- 172.21.15.100:0/4236246704 shutdown_connections 2024-09-15T21:48:37.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.551+0000 7f9566ffd640 1 -- 172.21.15.100:0/556177149 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f956c06c380 msgr2=0x7f956c06e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.551+0000 7f9566ffd640 1 --2- 172.21.15.100:0/556177149 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f956c06c380 0x7f956c06e840 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f957000d440 tx=0x7f9570008040 comp rx=0 tx=0).stop 2024-09-15T21:48:37.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.551+0000 7f9566ffd640 1 -- 172.21.15.100:0/556177149 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95800c6d20 msgr2=0x7f95801582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.551+0000 7f9566ffd640 1 --2- 172.21.15.100:0/556177149 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f95800c6d20 0x7f95801582e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9578000f80 tx=0x7f9578004a40 comp rx=0 tx=0).stop 2024-09-15T21:48:37.610 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.552+0000 7f958508e640 1 -- 172.21.15.100:0/556177149 reap_dead start 2024-09-15T21:48:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.551+0000 7f4f495bc640 1 -- 172.21.15.100:0/4236246704 wait complete. 2024-09-15T21:48:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.552+0000 7f9566ffd640 1 -- 172.21.15.100:0/556177149 shutdown_connections 2024-09-15T21:48:37.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.552+0000 7f9566ffd640 1 -- 172.21.15.100:0/556177149 >> 172.21.15.100:0/556177149 conn(0x7f95800c8f20 msgr2=0x7f95800c5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.552+0000 7f9566ffd640 1 -- 172.21.15.100:0/556177149 shutdown_connections 2024-09-15T21:48:37.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.552+0000 7f9566ffd640 1 -- 172.21.15.100:0/556177149 wait complete. 2024-09-15T21:48:37.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.552+0000 7fa9e69ee640 1 -- 172.21.15.100:0/1316584600 >> [v2:172.21.15.122:6824/822572374,v1:172.21.15.122:6825/822572374] conn(0x7fa9c4074a90 msgr2=0x7fa9c4076f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.552+0000 7fa9e69ee640 1 --2- 172.21.15.100:0/1316584600 >> [v2:172.21.15.122:6824/822572374,v1:172.21.15.122:6825/822572374] conn(0x7fa9c4074a90 0x7fa9c4076f30 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.553+0000 7fa9e69ee640 1 -- 172.21.15.100:0/1316584600 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa9c406c6b0 msgr2=0x7fa9c406eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.612 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.553+0000 7fa9e69ee640 1 --2- 172.21.15.100:0/1316584600 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa9c406c6b0 0x7fa9c406eb70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa9e006f780 tx=0x7fa9d0009190 comp rx=0 tx=0).stop 2024-09-15T21:48:37.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.553+0000 7fa9e69ee640 1 -- 172.21.15.100:0/1316584600 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9e006c950 msgr2=0x7fa9e00c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.553+0000 7fa9e69ee640 1 --2- 172.21.15.100:0/1316584600 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa9e006c950 0x7fa9e00c9590 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc031000 tx=0x7fa9dc0046c0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.553+0000 7fa9e61ed640 1 -- 172.21.15.100:0/1316584600 reap_dead start 2024-09-15T21:48:37.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.553+0000 7fa9e69ee640 1 -- 172.21.15.100:0/1316584600 shutdown_connections 2024-09-15T21:48:37.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.553+0000 7fa9e69ee640 1 -- 172.21.15.100:0/1316584600 >> 172.21.15.100:0/1316584600 conn(0x7fa9e006ecc0 msgr2=0x7fa9e0156a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.555+0000 7f1fc50d0640 1 -- 172.21.15.100:0/3606783447 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1fa0002dd0 con 0x7f1fac074700 2024-09-15T21:48:37.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.555+0000 7fa9e69ee640 1 -- 172.21.15.100:0/1316584600 shutdown_connections 2024-09-15T21:48:37.613 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.555+0000 7fa9e69ee640 1 -- 172.21.15.100:0/1316584600 wait complete. 2024-09-15T21:48:37.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.555+0000 7f6302d5b640 1 -- 172.21.15.100:0/1889801944 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f62f404ab20 con 0x7f62e0074710 2024-09-15T21:48:37.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.555+0000 7f1fa77fe640 1 -- 172.21.15.100:0/3606783447 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1fa0002dd0 con 0x7f1fac074700 2024-09-15T21:48:37.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.555+0000 7f62f27fc640 1 -- 172.21.15.100:0/1889801944 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f62fc0504d0 con 0x7f62e0074710 2024-09-15T21:48:37.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.556+0000 7f1fc50d0640 1 -- 172.21.15.100:0/3606783447 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f1fac074700 msgr2=0x7f1fac076ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.556+0000 7f1fc50d0640 1 --2- 172.21.15.100:0/3606783447 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f1fac074700 0x7f1fac076ba0 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.556+0000 7fd171401640 1 -- 172.21.15.100:0/1529589970 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd16c164e10 con 0x7fd158074760 2024-09-15T21:48:37.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.556+0000 7f1fc50d0640 1 -- 172.21.15.100:0/3606783447 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1fac06c300 msgr2=0x7f1fac06e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.556+0000 7f1fc50d0640 1 --2- 172.21.15.100:0/3606783447 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f1fac06c300 0x7f1fac06e7c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8000f80 tx=0x7f1fb8001700 comp rx=0 tx=0).stop 2024-09-15T21:48:37.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.556+0000 7f1fc50d0640 1 -- 172.21.15.100:0/3606783447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc01589d0 msgr2=0x7f1fc01f8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.556+0000 7f1fc50d0640 1 --2- 172.21.15.100:0/3606783447 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1fc01589d0 0x7f1fc01f8f40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00c6cd0 tx=0x7f1fb00046c0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.557+0000 7fd1537fe640 1 -- 172.21.15.100:0/1529589970 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd16c164e10 con 0x7fd158074760 2024-09-15T21:48:37.616 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.557+0000 7f1fbf59e640 1 -- 172.21.15.100:0/3606783447 reap_dead start 2024-09-15T21:48:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.557+0000 7f1fc50d0640 1 -- 172.21.15.100:0/3606783447 shutdown_connections 2024-09-15T21:48:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.557+0000 7f1fc50d0640 1 -- 172.21.15.100:0/3606783447 >> 172.21.15.100:0/3606783447 conn(0x7f1fc0067dd0 msgr2=0x7f1fc0159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.557+0000 7f1fc50d0640 1 -- 172.21.15.100:0/3606783447 shutdown_connections 2024-09-15T21:48:37.617 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.557+0000 7f1fc50d0640 1 -- 172.21.15.100:0/3606783447 wait complete. 2024-09-15T21:48:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.556+0000 7f6302d5b640 1 -- 172.21.15.100:0/1889801944 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f62e0074710 msgr2=0x7f62e0076bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.556+0000 7f6302d5b640 1 --2- 172.21.15.100:0/1889801944 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7f62e0074710 0x7f62e0076bb0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.556+0000 7f6302d5b640 1 -- 172.21.15.100:0/1889801944 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f62e006c330 msgr2=0x7f62e006e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.556+0000 7f6302d5b640 1 --2- 172.21.15.100:0/1889801944 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f62e006c330 0x7f62e006e7f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f62f40e1950 tx=0x7f62ec02e000 comp rx=0 tx=0).stop 2024-09-15T21:48:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.556+0000 7f6302d5b640 1 -- 172.21.15.100:0/1889801944 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f62f40091b0 msgr2=0x7f62f417c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.556+0000 7f6302d5b640 1 --2- 172.21.15.100:0/1889801944 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f62f40091b0 0x7f62f417c620 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f62f800c3e0 tx=0x7f62f800c8a0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.556+0000 7f630255a640 1 -- 172.21.15.100:0/1889801944 reap_dead start 2024-09-15T21:48:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.557+0000 7f6302d5b640 1 -- 172.21.15.100:0/1889801944 shutdown_connections 2024-09-15T21:48:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.557+0000 7f6302d5b640 1 -- 172.21.15.100:0/1889801944 >> 172.21.15.100:0/1889801944 conn(0x7f62f400db90 msgr2=0x7f62f40dd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.618 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.557+0000 7f6302d5b640 1 -- 172.21.15.100:0/1889801944 shutdown_connections 2024-09-15T21:48:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.557+0000 7fd171401640 1 -- 172.21.15.100:0/1529589970 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fd158074760 msgr2=0x7fd158076c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.557+0000 7fd171401640 1 --2- 172.21.15.100:0/1529589970 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fd158074760 0x7fd158076c00 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.557+0000 7f6302d5b640 1 -- 172.21.15.100:0/1889801944 wait complete. 2024-09-15T21:48:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.557+0000 7fd171401640 1 -- 172.21.15.100:0/1529589970 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd15806c300 msgr2=0x7fd15806e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.557+0000 7fd171401640 1 --2- 172.21.15.100:0/1529589970 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd15806c300 0x7fd15806e7c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0c54d0 tx=0x7fd164009830 comp rx=0 tx=0).stop 2024-09-15T21:48:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.557+0000 7fd171401640 1 -- 172.21.15.100:0/1529589970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd16c0c7f40 msgr2=0x7fd16c14db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.557+0000 7fd171401640 1 --2- 172.21.15.100:0/1529589970 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd16c0c7f40 0x7fd16c14db60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00c3c0 tx=0x7fd15c00c880 comp rx=0 tx=0).stop 2024-09-15T21:48:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.557+0000 7fd16b7fe640 1 -- 172.21.15.100:0/1529589970 reap_dead start 2024-09-15T21:48:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.558+0000 7fd171401640 1 -- 172.21.15.100:0/1529589970 shutdown_connections 2024-09-15T21:48:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.558+0000 7fd171401640 1 -- 172.21.15.100:0/1529589970 >> 172.21.15.100:0/1529589970 conn(0x7fd16c06ccf0 msgr2=0x7fd16c06d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.558+0000 7fd171401640 1 -- 172.21.15.100:0/1529589970 shutdown_connections 2024-09-15T21:48:37.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.558+0000 7fd171401640 1 -- 172.21.15.100:0/1529589970 wait complete. 2024-09-15T21:48:37.621 INFO:teuthology.orchestra.run.smithi100.stdout:60129542177 2024-09-15T21:48:37.621 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:48:37.622 INFO:teuthology.orchestra.run.smithi100.stdout:64424509472 2024-09-15T21:48:37.622 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:48:37.623 INFO:teuthology.orchestra.run.smithi100.stdout:68719476768 2024-09-15T21:48:37.623 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:48:37.624 INFO:teuthology.orchestra.run.smithi100.stdout:64424509473 2024-09-15T21:48:37.624 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T21:48:37.625 INFO:teuthology.orchestra.run.smithi100.stdout:55834574881 2024-09-15T21:48:37.626 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:48:37.627 INFO:teuthology.orchestra.run.smithi100.stdout:64424509472 2024-09-15T21:48:37.627 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T21:48:37.628 INFO:teuthology.orchestra.run.smithi100.stdout:244813135878 2024-09-15T21:48:37.628 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T21:48:37.629 INFO:teuthology.orchestra.run.smithi100.stdout:64424509474 2024-09-15T21:48:37.629 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T21:48:37.829 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.827+0000 7f84eb72a640 1 Processor -- start 2024-09-15T21:48:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.827+0000 7f39e8791640 1 Processor -- start 2024-09-15T21:48:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.827+0000 7f39e8791640 1 -- start start 2024-09-15T21:48:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.828+0000 7f39e8791640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f39e00b3710 0x7f39e00cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.828+0000 7f39e8791640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39e00cc9f0 0x7f39e00ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.829+0000 7f39e8791640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39e0163410 con 0x7f39e0159750 2024-09-15T21:48:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.829+0000 7f39e8791640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f39e015fbf0 con 0x7f39e00b3710 2024-09-15T21:48:37.830 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.829+0000 7f39e8791640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f39e00cbe50 con 0x7f39e00cc9f0 2024-09-15T21:48:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.829+0000 7f7dd253d640 1 Processor -- start 2024-09-15T21:48:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.829+0000 7f39e652d640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f39e0159750 0x7f39e015f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42078/0 (socket says 172.21.15.100:42078) 2024-09-15T21:48:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.829+0000 7f39e5d2c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f39e00b3710 0x7f39e00cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.829+0000 7f39e652d640 1 -- 172.21.15.100:0/3283745087 learned_addr learned my addr 172.21.15.100:0/3283745087 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:37.831 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.829+0000 7f39e6d2e640 1 --2- 172.21.15.100:0/3283745087 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39e00cc9f0 0x7f39e00ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.828+0000 7f84eb72a640 1 -- start start 2024-09-15T21:48:37.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.829+0000 7f84eb72a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f84e4151270 0x7f84e40c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.829+0000 7f84eb72a640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f84e40cc620 0x7f84e40ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.829+0000 7f84eb72a640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84e4151650 con 0x7f84e4157bb0 2024-09-15T21:48:37.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.829+0000 7f84eb72a640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f84e40c8420 con 0x7f84e40cc620 2024-09-15T21:48:37.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.829+0000 7f84eb72a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f84e40c9200 con 0x7f84e4151270 2024-09-15T21:48:37.832 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.829+0000 7fcabfae4640 1 Processor -- start 2024-09-15T21:48:37.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.830+0000 7f39e5d2c640 1 -- 172.21.15.100:0/3283745087 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39e00cc9f0 msgr2=0x7f39e00ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:37.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.830+0000 7f39e5d2c640 1 --2- 172.21.15.100:0/3283745087 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39e00cc9f0 0x7f39e00ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.830+0000 7f39e5d2c640 1 -- 172.21.15.100:0/3283745087 >> v1:172.21.15.100:6789/0 conn(0x7f39e0159750 legacy=0x7f39e015f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.830+0000 7f39e5d2c640 1 -- 172.21.15.100:0/3283745087 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e00cc080 con 0x7f39e00b3710 2024-09-15T21:48:37.833 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.829+0000 7fcabfae4640 1 -- start start 2024-09-15T21:48:37.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.830+0000 7f84e94c6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f84e4151270 0x7f84e40c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.830+0000 7f84e94c6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f84e4151270 0x7f84e40c77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:55572/0 (socket says 172.21.15.100:55572) 2024-09-15T21:48:37.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.830+0000 7f84e94c6640 1 -- 172.21.15.100:0/1146845430 learned_addr learned my addr 172.21.15.100:0/1146845430 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.834 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.830+0000 7f7dd253d640 1 -- start start 2024-09-15T21:48:37.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7fcabfae4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fcab8159720 0x7fcab80c7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7f7dd253d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f7dc40dcd90 0x7f7dc4045d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7f7dd253d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7dc404a680 0x7f7dc404aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.835 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7fcabfae4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcab80c7c90 0x7fcab8070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7fcabd07f640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fcab8159b00 0x7fcab815f570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42090/0 (socket says 172.21.15.100:42090) 2024-09-15T21:48:37.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7f39e5d2c640 1 --2- 172.21.15.100:0/3283745087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e00b3710 0x7f39e00cc430 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0097a0 tx=0x7f39dc02a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16ea4cf2854f826 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7fcabd880640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fcab8159720 0x7fcab80c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.836 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7fcabd880640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fcab8159720 0x7fcab80c7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:42886/0 (socket says 172.21.15.100:42886) 2024-09-15T21:48:37.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7fcabfae4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcab80c8a70 con 0x7fcab8159b00 2024-09-15T21:48:37.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7fcabfae4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fcab81634b0 con 0x7fcab80c7c90 2024-09-15T21:48:37.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7fcabe081640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcab80c7c90 0x7fcab8070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7fcabfae4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fcab8162b80 con 0x7fcab8159720 2024-09-15T21:48:37.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7fcabe081640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fcab80c7c90 0x7fcab8070ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54160/0 (socket says 172.21.15.100:54160) 2024-09-15T21:48:37.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7fcabe081640 1 -- 172.21.15.100:0/988845244 learned_addr learned my addr 172.21.15.100:0/988845244 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7f7dd1d3c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f7dc40e5f30 0x7f7dc40e63c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42092/0 (socket says 172.21.15.100:42092) 2024-09-15T21:48:37.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7f7dd1d3c640 1 -- 172.21.15.100:0/52424556 learned_addr learned my addr 172.21.15.100:0/52424556 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:37.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7f7dd153b640 1 --2- 172.21.15.100:0/52424556 >> v2:172.21.15.100:3300/0 conn(0x7f7dc40dcd90 0x7f7dc4045d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7f7dd253d640 1 -- 172.21.15.100:0/52424556 --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dc40e6d30 con 0x7f7dc40e5f30 2024-09-15T21:48:37.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.833+0000 7f7dd253d640 1 -- 172.21.15.100:0/52424556 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f7dc4044d40 con 0x7f7dc40dcd90 2024-09-15T21:48:37.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.833+0000 7f7dd253d640 1 -- 172.21.15.100:0/52424556 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7dc40dd170 con 0x7f7dc404a680 2024-09-15T21:48:37.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.833+0000 7f7dd0d3a640 1 --2- 172.21.15.100:0/52424556 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7dc404a680 0x7f7dc404aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.833+0000 7f7dc3fff640 1 -- 172.21.15.100:0/52424556 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3826301854 0 0) 0x7f7dc40e6d30 con 0x7f7dc40e5f30 2024-09-15T21:48:37.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7f39e552b640 1 -- 172.21.15.100:0/3283745087 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc037070 con 0x7f39e00b3710 2024-09-15T21:48:37.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7f39e552b640 1 -- 172.21.15.100:0/3283745087 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39dc02ae40 con 0x7f39e00b3710 2024-09-15T21:48:37.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7f39e552b640 1 -- 172.21.15.100:0/3283745087 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39dc032660 con 0x7f39e00b3710 2024-09-15T21:48:37.838 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7f84dbfff640 1 -- 172.21.15.100:0/1146845430 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2906507007 0 0) 0x7f84e4151650 con 0x7f84e4157bb0 2024-09-15T21:48:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7f84dbfff640 1 -- 172.21.15.100:0/1146845430 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f84d0003520 con 0x7f84e4157bb0 2024-09-15T21:48:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7f84dbfff640 1 -- 172.21.15.100:0/1146845430 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3358501699 0 0) 0x7f84d0003520 con 0x7f84e4157bb0 2024-09-15T21:48:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7f84dbfff640 1 -- 172.21.15.100:0/1146845430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f84e4151270 msgr2=0x7f84e40c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7f84dbfff640 1 --2- 172.21.15.100:0/1146845430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f84e4151270 0x7f84e40c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7f84dbfff640 1 -- 172.21.15.100:0/1146845430 >> v2:172.21.15.100:3300/0 conn(0x7f84e40cc620 msgr2=0x7f84e40ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7f84dbfff640 1 --2- 172.21.15.100:0/1146845430 >> v2:172.21.15.100:3300/0 conn(0x7f84e40cc620 0x7f84e40ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7f84dbfff640 1 -- 172.21.15.100:0/1146845430 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e40c94e0 con 0x7f84e4157bb0 2024-09-15T21:48:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7f84dbfff640 1 -- 172.21.15.100:0/1146845430 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7f84d4001670 con 0x7f84e4157bb0 2024-09-15T21:48:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.831+0000 7f84dbfff640 1 -- 172.21.15.100:0/1146845430 >> v1:172.21.15.100:6789/0 conn(0x7f84e4157bb0 legacy=0x7f84e4158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7f84dbfff640 1 --2- 172.21.15.100:0/1146845430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d00039d0 0x7f84d0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7f84dbfff640 1 --2- 172.21.15.100:0/1146845430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84d0004800 0x7f84d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7f84dbfff640 1 --2- 172.21.15.100:0/1146845430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f84e4151270 0x7f84d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.839 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7f84dbfff640 1 -- 172.21.15.100:0/1146845430 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f84e40c8420 con 0x7f84d00039d0 2024-09-15T21:48:37.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7f84dbfff640 1 -- 172.21.15.100:0/1146845430 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f84e40c9200 con 0x7f84d0004800 2024-09-15T21:48:37.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7f84dbfff640 1 -- 172.21.15.100:0/1146845430 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f84d0009b00 con 0x7f84e4151270 2024-09-15T21:48:37.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7f84e94c6640 1 --2- 172.21.15.100:0/1146845430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d00039d0 0x7f84d0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7f84e8cc5640 1 --2- 172.21.15.100:0/1146845430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84d0004800 0x7f84d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7f84e94c6640 1 -- 172.21.15.100:0/1146845430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f84e4151270 msgr2=0x7f84d00051c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:48:37.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7f84e94c6640 1 --2- 172.21.15.100:0/1146845430 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f84e4151270 0x7f84d00051c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7f84e94c6640 1 -- 172.21.15.100:0/1146845430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84d0004800 msgr2=0x7f84d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:37.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7f88002b7640 1 Processor -- start 2024-09-15T21:48:37.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7f069dc8d640 1 Processor -- start 2024-09-15T21:48:37.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.833+0000 7f7dc3fff640 1 -- 172.21.15.100:0/52424556 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7db8003520 con 0x7f7dc40e5f30 2024-09-15T21:48:37.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7f84e94c6640 1 --2- 172.21.15.100:0/1146845430 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84d0004800 0x7f84d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7f84e94c6640 1 -- 172.21.15.100:0/1146845430 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e40c94e0 con 0x7f84d00039d0 2024-09-15T21:48:37.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.833+0000 7f88002b7640 1 -- start start 2024-09-15T21:48:37.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7fcabd880640 1 -- 172.21.15.100:0/988845244 >> v2:172.21.15.100:3300/0 conn(0x7fcab80c7c90 msgr2=0x7fcab8070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:37.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7fcabd880640 1 --2- 172.21.15.100:0/988845244 >> v2:172.21.15.100:3300/0 conn(0x7fcab80c7c90 0x7fcab8070ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7fcabd880640 1 -- 172.21.15.100:0/988845244 >> v1:172.21.15.100:6789/0 conn(0x7fcab8159b00 legacy=0x7fcab815f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:37.842 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.832+0000 7fcabd880640 1 -- 172.21.15.100:0/988845244 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab8162e60 con 0x7fcab8159720 2024-09-15T21:48:37.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.833+0000 7f7dc3fff640 1 -- 172.21.15.100:0/52424556 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1905188352 0 0) 0x7f7db8003520 con 0x7f7dc40e5f30 2024-09-15T21:48:37.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.833+0000 7f7dd153b640 1 -- 172.21.15.100:0/52424556 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7dc404a680 msgr2=0x7f7dc404aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:37.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.833+0000 7f7dd153b640 1 --2- 172.21.15.100:0/52424556 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7dc404a680 0x7f7dc404aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.834+0000 7f7dd153b640 1 -- 172.21.15.100:0/52424556 >> v1:172.21.15.100:6789/0 conn(0x7f7dc40e5f30 legacy=0x7f7dc40e63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.834+0000 7f39e8791640 1 -- 172.21.15.100:0/3283745087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e00b3710 msgr2=0x7f39e00cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.834+0000 7f39e8791640 1 --2- 172.21.15.100:0/3283745087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e00b3710 0x7f39e00cc430 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f39dc0097a0 tx=0x7f39dc02a0f0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.843 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.834+0000 7f3aa7fff640 1 Processor -- start 2024-09-15T21:48:37.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.833+0000 7f069dc8d640 1 -- start start 2024-09-15T21:48:37.844 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.834+0000 7f7dd153b640 1 -- 172.21.15.100:0/52424556 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc4044f90 con 0x7f7dc40dcd90 2024-09-15T21:48:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.834+0000 7f39e8791640 1 -- 172.21.15.100:0/3283745087 shutdown_connections 2024-09-15T21:48:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.834+0000 7f39e8791640 1 --2- 172.21.15.100:0/3283745087 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39e00cc9f0 0x7f39e00ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.834+0000 7f39e8791640 1 --2- 172.21.15.100:0/3283745087 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e00b3710 0x7f39e00cc430 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.834+0000 7f39e8791640 1 -- 172.21.15.100:0/3283745087 >> 172.21.15.100:0/3283745087 conn(0x7f39e00680d0 msgr2=0x7f39e006d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.845 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.834+0000 7fcabd880640 1 --2- 172.21.15.100:0/988845244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcab8159720 0x7fcab80c7410 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcab80ca1b0 tx=0x7fcab402a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38727a91e5d0e167 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.834+0000 7f7dd0d3a640 1 --2- 172.21.15.100:0/52424556 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7dc404a680 0x7f7dc404aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:48:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.834+0000 7f3aa7fff640 1 -- start start 2024-09-15T21:48:37.846 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.834+0000 7fcabc87e640 1 -- 172.21.15.100:0/988845244 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4037070 con 0x7fcab8159720 2024-09-15T21:48:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.834+0000 7f39e8791640 1 -- 172.21.15.100:0/3283745087 shutdown_connections 2024-09-15T21:48:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.834+0000 7fcabc87e640 1 -- 172.21.15.100:0/988845244 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcab4032440 con 0x7fcab8159720 2024-09-15T21:48:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.834+0000 7fcabc87e640 1 -- 172.21.15.100:0/988845244 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab4032810 con 0x7fcab8159720 2024-09-15T21:48:37.847 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f88002b7640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87f80cc250 0x7f87f806db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f069dc8d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f06980c9f00 0x7f0698162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.834+0000 7f39e8791640 1 -- 172.21.15.100:0/3283745087 wait complete. 2024-09-15T21:48:37.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f88002b7640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87f815b3d0 con 0x7f87f80d0df0 2024-09-15T21:48:37.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f88002b7640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87f815b670 con 0x7f87f80ccef0 2024-09-15T21:48:37.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f88002b7640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f87f815b950 con 0x7f87f80cc250 2024-09-15T21:48:37.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f87fe854640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87f80cc250 0x7f87f806db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f069dc8d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f06980c7a00 0x7f069815ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f87fe053640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f87f80d0df0 0x7f87f8155060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42102/0 (socket says 172.21.15.100:42102) 2024-09-15T21:48:37.849 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f87fe053640 1 -- 172.21.15.100:0/2097332917 learned_addr learned my addr 172.21.15.100:0/2097332917 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f069dc8d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f069815f4e0 con 0x7f06980ca2e0 2024-09-15T21:48:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f069dc8d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f069815fb10 con 0x7f06980c9f00 2024-09-15T21:48:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f069dc8d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f06980c7f60 con 0x7f06980c7a00 2024-09-15T21:48:37.850 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7fcabfae4640 1 -- 172.21.15.100:0/988845244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcab8159720 msgr2=0x7fcab80c7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7fcabfae4640 1 --2- 172.21.15.100:0/988845244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcab8159720 0x7fcab80c7410 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcab80ca1b0 tx=0x7fcab402a1e0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7fcabfae4640 1 -- 172.21.15.100:0/988845244 shutdown_connections 2024-09-15T21:48:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7fcabfae4640 1 --2- 172.21.15.100:0/988845244 >> v2:172.21.15.100:3300/0 conn(0x7fcab80c7c90 0x7fcab8070ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7fcabfae4640 1 --2- 172.21.15.100:0/988845244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcab8159720 0x7fcab80c7410 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7fcabfae4640 1 -- 172.21.15.100:0/988845244 >> 172.21.15.100:0/988845244 conn(0x7fcab8067dd0 msgr2=0x7fcab806bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f0696ffd640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f06980ca2e0 0x7f0698148220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.100:59374/0 (socket says 172.21.15.100:59374) 2024-09-15T21:48:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f0696ffd640 1 -- 172.21.15.100:0/434873505 learned_addr learned my addr 172.21.15.100:0/434873505 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:37.851 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f3aa7fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3aa8152e80 0x7f3aa81596d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f0697fff640 1 --2- 172.21.15.100:0/434873505 >> v2:172.21.15.122:3300/0 conn(0x7f06980c7a00 0x7f069815ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f7dd153b640 1 --2- 172.21.15.100:0/52424556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dc40dcd90 0x7f7dc4045d50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0097a0 tx=0x7f7dbc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9f951d5972f8e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.852 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f06977fe640 1 --2- 172.21.15.100:0/434873505 >> v2:172.21.15.100:3300/0 conn(0x7f06980c9f00 0x7f0698162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f124670c640 1 Processor -- start 2024-09-15T21:48:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.835+0000 7f124670c640 1 -- start start 2024-09-15T21:48:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f3aa7fff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3aa8067d30 0x7f3aa806dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f87fd051640 1 -- 172.21.15.100:0/2097332917 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3600126938 0 0) 0x7f87f815b3d0 con 0x7f87f80d0df0 2024-09-15T21:48:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f87fd051640 1 -- 172.21.15.100:0/2097332917 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87e4003520 con 0x7f87f80d0df0 2024-09-15T21:48:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f3aa7fff640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aa8071f60 con 0x7f3aa806e1e0 2024-09-15T21:48:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f3aa7fff640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f3aa8159c10 con 0x7f3aa8152e80 2024-09-15T21:48:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f3aa7fff640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa80716c0 con 0x7f3aa8067d30 2024-09-15T21:48:37.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7fcabfae4640 1 -- 172.21.15.100:0/988845244 shutdown_connections 2024-09-15T21:48:37.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f3aa6ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3aa8152e80 0x7f3aa81596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f3aa6ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3aa8152e80 0x7f3aa81596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54186/0 (socket says 172.21.15.100:54186) 2024-09-15T21:48:37.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f3a9e7fc640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3aa8067d30 0x7f3aa806dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f3aa6ffd640 1 -- 172.21.15.100:0/3624955462 learned_addr learned my addr 172.21.15.100:0/3624955462 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f84e94c6640 1 --2- 172.21.15.100:0/1146845430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d00039d0 0x7f84d0003e10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f84e00097b0 tx=0x7f84d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6377aa98ff97e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f06967fc640 1 -- 172.21.15.100:0/434873505 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3000385904 0 0) 0x7f069815f4e0 con 0x7f06980ca2e0 2024-09-15T21:48:37.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f06967fc640 1 -- 172.21.15.100:0/434873505 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0680003520 con 0x7f06980ca2e0 2024-09-15T21:48:37.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f39e8791640 1 Processor -- start 2024-09-15T21:48:37.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f39e8791640 1 -- start start 2024-09-15T21:48:37.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f3a9e7fc640 1 -- 172.21.15.100:0/3624955462 >> v2:172.21.15.100:3300/0 conn(0x7f3aa8152e80 msgr2=0x7f3aa81596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:37.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f3a9e7fc640 1 --2- 172.21.15.100:0/3624955462 >> v2:172.21.15.100:3300/0 conn(0x7f3aa8152e80 0x7f3aa81596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f3a9e7fc640 1 -- 172.21.15.100:0/3624955462 >> v1:172.21.15.122:6789/0 conn(0x7f3aa806e1e0 legacy=0x7f3aa80734a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T21:48:37.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f3a9e7fc640 1 -- 172.21.15.100:0/3624955462 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa80719a0 con 0x7f3aa8067d30 2024-09-15T21:48:37.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f06977fe640 1 -- 172.21.15.100:0/434873505 >> v2:172.21.15.122:3300/0 conn(0x7f06980c7a00 msgr2=0x7f069815ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:37.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f06977fe640 1 --2- 172.21.15.100:0/434873505 >> v2:172.21.15.122:3300/0 conn(0x7f06980c7a00 0x7f069815ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f06977fe640 1 -- 172.21.15.100:0/434873505 >> v1:172.21.15.122:6789/0 conn(0x7f06980ca2e0 legacy=0x7f0698148220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7f06977fe640 1 -- 172.21.15.100:0/434873505 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06980c8240 con 0x7f06980c9f00 2024-09-15T21:48:37.858 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.836+0000 7fcabfae4640 1 -- 172.21.15.100:0/988845244 wait complete. 2024-09-15T21:48:37.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f124670c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f12400c8380 0x7f124015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f87fe854640 1 -- 172.21.15.100:0/2097332917 >> v1:172.21.15.122:6789/0 conn(0x7f87f80ccef0 legacy=0x7f87f80cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f87fe854640 1 -- 172.21.15.100:0/2097332917 >> v1:172.21.15.100:6789/0 conn(0x7f87f80d0df0 legacy=0x7f87f8155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f87fe854640 1 -- 172.21.15.100:0/2097332917 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f815bc30 con 0x7f87f80cc250 2024-09-15T21:48:37.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f124670c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f12400ca570 0x7f1240158150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f124670c640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12400c7460 con 0x7f12400c6fd0 2024-09-15T21:48:37.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f7dc3fff640 1 -- 172.21.15.100:0/52424556 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dbc037070 con 0x7f7dc40dcd90 2024-09-15T21:48:37.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f7dc3fff640 1 -- 172.21.15.100:0/52424556 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dbc02aba0 con 0x7f7dc40dcd90 2024-09-15T21:48:37.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f7dc3fff640 1 -- 172.21.15.100:0/52424556 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc063850 con 0x7f7dc40dcd90 2024-09-15T21:48:37.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f124670c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f1240068090 con 0x7f12400c8380 2024-09-15T21:48:37.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f124670c640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f12400c8870 con 0x7f12400ca570 2024-09-15T21:48:37.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f123ffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f12400c8380 0x7f124015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f123ffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f12400c8380 0x7f124015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54190/0 (socket says 172.21.15.100:54190) 2024-09-15T21:48:37.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f123ffff640 1 -- 172.21.15.100:0/1528713971 learned_addr learned my addr 172.21.15.100:0/1528713971 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f39e8791640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39e00b3710 0x7f39e01e8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f39e8791640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39e00cc9f0 0x7f39e01f90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f39e8791640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e0159750 0x7f39e01525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f123ffff640 1 -- 172.21.15.100:0/1528713971 >> v2:172.21.15.122:3300/0 conn(0x7f12400ca570 msgr2=0x7f1240158150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T21:48:37.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f123ffff640 1 --2- 172.21.15.100:0/1528713971 >> v2:172.21.15.122:3300/0 conn(0x7f12400ca570 0x7f1240158150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f39e6d2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e0159750 0x7f39e01525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f39e6d2e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e0159750 0x7f39e01525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54202/0 (socket says 172.21.15.100:54202) 2024-09-15T21:48:37.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f39e6d2e640 1 -- 172.21.15.100:0/959004194 learned_addr learned my addr 172.21.15.100:0/959004194 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f39e8791640 1 -- 172.21.15.100:0/959004194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f39e01636b0 con 0x7f39e0159750 2024-09-15T21:48:37.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f39e8791640 1 -- 172.21.15.100:0/959004194 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f39e0163990 con 0x7f39e00cc9f0 2024-09-15T21:48:37.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f39e8791640 1 -- 172.21.15.100:0/959004194 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f39e0159b30 con 0x7f39e00b3710 2024-09-15T21:48:37.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f39e652d640 1 --2- 172.21.15.100:0/959004194 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39e00b3710 0x7f39e01e8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f39e5d2c640 1 --2- 172.21.15.100:0/959004194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39e00cc9f0 0x7f39e01f90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f06977fe640 1 --2- 172.21.15.100:0/434873505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06980c9f00 0x7f0698162a70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f068c0097a0 tx=0x7f068c02a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b72d41b31bcc6da server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f123ffff640 1 -- 172.21.15.100:0/1528713971 >> v1:172.21.15.100:6789/0 conn(0x7f12400c6fd0 legacy=0x7f1240067980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:48:37.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.837+0000 7f123ffff640 1 -- 172.21.15.100:0/1528713971 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12400c8b50 con 0x7f12400c8380 2024-09-15T21:48:37.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f39e6d2e640 1 -- 172.21.15.100:0/959004194 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39e00b3710 msgr2=0x7f39e01e8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f39e6d2e640 1 --2- 172.21.15.100:0/959004194 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39e00b3710 0x7f39e01e8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f39e6d2e640 1 -- 172.21.15.100:0/959004194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39e00cc9f0 msgr2=0x7f39e01f90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f39e6d2e640 1 --2- 172.21.15.100:0/959004194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39e00cc9f0 0x7f39e01f90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f84dbfff640 1 -- 172.21.15.100:0/1146845430 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e005a070 con 0x7f84d00039d0 2024-09-15T21:48:37.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f39e6d2e640 1 -- 172.21.15.100:0/959004194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e01f95e0 con 0x7f39e0159750 2024-09-15T21:48:37.866 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f84dbfff640 1 -- 172.21.15.100:0/1146845430 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84e0055410 con 0x7f84d00039d0 2024-09-15T21:48:37.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f3a9e7fc640 1 --2- 172.21.15.100:0/3624955462 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3aa8067d30 0x7f3aa806dca0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3a980097a0 tx=0x7f3a9802a130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=592830e156a2e7bc server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f87fe854640 1 --2- 172.21.15.100:0/2097332917 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87f80cc250 0x7f87f806db90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009dc0 tx=0x7f87f002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28b07faa1459922d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7fcabfae4640 1 Processor -- start 2024-09-15T21:48:37.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f123ffff640 1 --2- 172.21.15.100:0/1528713971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12400c8380 0x7f124015abd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f12400c9d30 tx=0x7f123002a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1135ec08915ee743 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f39e6d2e640 1 --2- 172.21.15.100:0/959004194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e0159750 0x7f39e01525c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f39d800efc0 tx=0x7f39d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f7dd253d640 1 -- 172.21.15.100:0/52424556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dc40dcd90 msgr2=0x7f7dc4045d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f3aa67fc640 1 -- 172.21.15.100:0/3624955462 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98037070 con 0x7f3aa8067d30 2024-09-15T21:48:37.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f3aa67fc640 1 -- 172.21.15.100:0/3624955462 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a9802ae40 con 0x7f3aa8067d30 2024-09-15T21:48:37.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f7dd253d640 1 --2- 172.21.15.100:0/52424556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dc40dcd90 0x7f7dc4045d50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc0097a0 tx=0x7f7dbc02a110 comp rx=0 tx=0).stop 2024-09-15T21:48:37.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f7dd253d640 1 -- 172.21.15.100:0/52424556 shutdown_connections 2024-09-15T21:48:37.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f7dd253d640 1 --2- 172.21.15.100:0/52424556 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7dc404a680 0x7f7dc404aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f7dd253d640 1 --2- 172.21.15.100:0/52424556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dc40dcd90 0x7f7dc4045d50 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f7dd253d640 1 -- 172.21.15.100:0/52424556 >> 172.21.15.100:0/52424556 conn(0x7f7dc404fc50 msgr2=0x7f7dc4050060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f39d2ffd640 1 -- 172.21.15.100:0/959004194 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8018020 con 0x7f39e0159750 2024-09-15T21:48:37.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f3aa67fc640 1 -- 172.21.15.100:0/3624955462 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a98037070 con 0x7f3aa8067d30 2024-09-15T21:48:37.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f39e8791640 1 -- 172.21.15.100:0/959004194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e01f5380 con 0x7f39e0159750 2024-09-15T21:48:37.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f39e8791640 1 -- 172.21.15.100:0/959004194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e00ca710 con 0x7f39e0159750 2024-09-15T21:48:37.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f7dd253d640 1 -- 172.21.15.100:0/52424556 shutdown_connections 2024-09-15T21:48:37.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f39d2ffd640 1 -- 172.21.15.100:0/959004194 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39d8014480 con 0x7f39e0159750 2024-09-15T21:48:37.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f84dbfff640 1 -- 172.21.15.100:0/1146845430 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e00097f0 con 0x7f84d00039d0 2024-09-15T21:48:37.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f84eb72a640 1 -- 172.21.15.100:0/1146845430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d00039d0 msgr2=0x7f84d0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f84eb72a640 1 --2- 172.21.15.100:0/1146845430 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84d00039d0 0x7f84d0003e10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f84e00097b0 tx=0x7f84d0005830 comp rx=0 tx=0).stop 2024-09-15T21:48:37.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f06967fc640 1 -- 172.21.15.100:0/434873505 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c037070 con 0x7f06980c9f00 2024-09-15T21:48:37.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f06967fc640 1 -- 172.21.15.100:0/434873505 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f068c032440 con 0x7f06980c9f00 2024-09-15T21:48:37.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7f06967fc640 1 -- 172.21.15.100:0/434873505 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c032810 con 0x7f06980c9f00 2024-09-15T21:48:37.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f39d2ffd640 1 -- 172.21.15.100:0/959004194 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8014870 con 0x7f39e0159750 2024-09-15T21:48:37.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f84e9cc7640 1 -- 172.21.15.100:0/1146845430 reap_dead start 2024-09-15T21:48:37.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.838+0000 7fcabfae4640 1 -- start start 2024-09-15T21:48:37.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7fcabfae4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcab80c7c90 0x7fcab815dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f84eb72a640 1 -- 172.21.15.100:0/1146845430 shutdown_connections 2024-09-15T21:48:37.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f84eb72a640 1 -- 172.21.15.100:0/1146845430 >> 172.21.15.100:0/1146845430 conn(0x7f84e4067dd0 msgr2=0x7f84e406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f7dd253d640 1 -- 172.21.15.100:0/52424556 wait complete. 2024-09-15T21:48:37.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7fcabfae4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcab8159720 0x7fcab8159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7fcabfae4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcab8159b00 0x7fcab814adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f123effd640 1 -- 172.21.15.100:0/1528713971 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1230037070 con 0x7f12400c8380 2024-09-15T21:48:37.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f123effd640 1 -- 172.21.15.100:0/1528713971 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1230032440 con 0x7f12400c8380 2024-09-15T21:48:37.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f123effd640 1 -- 172.21.15.100:0/1528713971 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1230032810 con 0x7f12400c8380 2024-09-15T21:48:37.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7fcabfae4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fcab8163940 con 0x7fcab8159b00 2024-09-15T21:48:37.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7fcabfae4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcab815fc80 con 0x7fcab80c7c90 2024-09-15T21:48:37.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7fcabfae4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fcab80c7950 con 0x7fcab8159720 2024-09-15T21:48:37.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7fcabd07f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcab8159720 0x7fcab8159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7fcabe081640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcab8159b00 0x7fcab814adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7fcabd07f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcab8159720 0x7fcab8159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:55620/0 (socket says 172.21.15.100:55620) 2024-09-15T21:48:37.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7fcabe081640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcab8159b00 0x7fcab814adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54218/0 (socket says 172.21.15.100:54218) 2024-09-15T21:48:37.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7fcabd07f640 1 -- 172.21.15.100:0/1369220609 learned_addr learned my addr 172.21.15.100:0/1369220609 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7fcabd880640 1 --2- 172.21.15.100:0/1369220609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcab80c7c90 0x7fcab815dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7f124670c640 1 -- 172.21.15.100:0/1528713971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12400c8380 msgr2=0x7f124015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7f124670c640 1 --2- 172.21.15.100:0/1528713971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12400c8380 0x7f124015abd0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f12400c9d30 tx=0x7f123002a250 comp rx=0 tx=0).stop 2024-09-15T21:48:37.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f3aa7fff640 1 -- 172.21.15.100:0/3624955462 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3aa8067d30 msgr2=0x7f3aa806dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.839+0000 7f3aa7fff640 1 --2- 172.21.15.100:0/3624955462 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3aa8067d30 0x7f3aa806dca0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3a980097a0 tx=0x7f3a9802a130 comp rx=0 tx=0).stop 2024-09-15T21:48:37.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7f39e8791640 1 -- 172.21.15.100:0/959004194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e0158350 con 0x7f39e0159750 2024-09-15T21:48:37.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7f87fd051640 1 -- 172.21.15.100:0/2097332917 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0007740 con 0x7f87f80cc250 2024-09-15T21:48:37.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7f87fd051640 1 -- 172.21.15.100:0/2097332917 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f00078e0 con 0x7f87f80cc250 2024-09-15T21:48:37.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7f87fd051640 1 -- 172.21.15.100:0/2097332917 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0007cb0 con 0x7f87f80cc250 2024-09-15T21:48:37.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7f3aa7fff640 1 -- 172.21.15.100:0/3624955462 shutdown_connections 2024-09-15T21:48:37.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7f3aa7fff640 1 --2- 172.21.15.100:0/3624955462 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3aa8067d30 0x7f3aa806dca0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7f3aa7fff640 1 --2- 172.21.15.100:0/3624955462 >> v2:172.21.15.100:3300/0 conn(0x7f3aa8152e80 0x7f3aa81596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7f3aa7fff640 1 -- 172.21.15.100:0/3624955462 >> 172.21.15.100:0/3624955462 conn(0x7f3aa806f130 msgr2=0x7f3aa806f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7fcabe081640 1 -- 172.21.15.100:0/1369220609 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcab8159720 msgr2=0x7fcab8159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7fcabe081640 1 --2- 172.21.15.100:0/1369220609 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcab8159720 0x7fcab8159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7fcabe081640 1 -- 172.21.15.100:0/1369220609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcab80c7c90 msgr2=0x7fcab815dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7fcabe081640 1 --2- 172.21.15.100:0/1369220609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcab80c7c90 0x7fcab815dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7fcabe081640 1 -- 172.21.15.100:0/1369220609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab8149c80 con 0x7fcab8159b00 2024-09-15T21:48:37.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7fcabe081640 1 --2- 172.21.15.100:0/1369220609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcab8159b00 0x7fcab814adb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcab80c80e0 tx=0x7fcab000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7f069dc8d640 1 -- 172.21.15.100:0/434873505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06980c9f00 msgr2=0x7f0698162a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.840+0000 7f069dc8d640 1 --2- 172.21.15.100:0/434873505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06980c9f00 0x7f0698162a70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f068c0097a0 tx=0x7f068c02a1d0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f84eb72a640 1 -- 172.21.15.100:0/1146845430 shutdown_connections 2024-09-15T21:48:37.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f84eb72a640 1 -- 172.21.15.100:0/1146845430 wait complete. 2024-09-15T21:48:37.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f069dc8d640 1 -- 172.21.15.100:0/434873505 shutdown_connections 2024-09-15T21:48:37.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f069dc8d640 1 --2- 172.21.15.100:0/434873505 >> v2:172.21.15.122:3300/0 conn(0x7f06980c7a00 0x7f069815ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f069dc8d640 1 --2- 172.21.15.100:0/434873505 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06980c9f00 0x7f0698162a70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f069dc8d640 1 -- 172.21.15.100:0/434873505 >> 172.21.15.100:0/434873505 conn(0x7f0698073820 msgr2=0x7f0698073c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f069dc8d640 1 -- 172.21.15.100:0/434873505 shutdown_connections 2024-09-15T21:48:37.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f124670c640 1 -- 172.21.15.100:0/1528713971 shutdown_connections 2024-09-15T21:48:37.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f124670c640 1 --2- 172.21.15.100:0/1528713971 >> v2:172.21.15.122:3300/0 conn(0x7f12400ca570 0x7f1240158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f124670c640 1 --2- 172.21.15.100:0/1528713971 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12400c8380 0x7f124015abd0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f124670c640 1 -- 172.21.15.100:0/1528713971 >> 172.21.15.100:0/1528713971 conn(0x7f1240073820 msgr2=0x7f1240073c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f124670c640 1 -- 172.21.15.100:0/1528713971 shutdown_connections 2024-09-15T21:48:37.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f124670c640 1 -- 172.21.15.100:0/1528713971 wait complete. 2024-09-15T21:48:37.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f069dc8d640 1 -- 172.21.15.100:0/434873505 wait complete. 2024-09-15T21:48:37.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f88002b7640 1 -- 172.21.15.100:0/2097332917 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87f80cc250 msgr2=0x7f87f806db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f88002b7640 1 --2- 172.21.15.100:0/2097332917 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87f80cc250 0x7f87f806db90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009dc0 tx=0x7f87f002a6d0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f7dd253d640 1 Processor -- start 2024-09-15T21:48:37.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f88002b7640 1 -- 172.21.15.100:0/2097332917 shutdown_connections 2024-09-15T21:48:37.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f88002b7640 1 --2- 172.21.15.100:0/2097332917 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87f80cc250 0x7f87f806db90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f88002b7640 1 -- 172.21.15.100:0/2097332917 >> 172.21.15.100:0/2097332917 conn(0x7f87f8067980 msgr2=0x7f87f8067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:37.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f7dd253d640 1 -- start start 2024-09-15T21:48:37.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f88002b7640 1 -- 172.21.15.100:0/2097332917 shutdown_connections 2024-09-15T21:48:37.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.841+0000 7f88002b7640 1 -- 172.21.15.100:0/2097332917 wait complete. 2024-09-15T21:48:37.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.842+0000 7f84eb72a640 1 Processor -- start 2024-09-15T21:48:37.884 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.842+0000 7f124670c640 1 Processor -- start 2024-09-15T21:48:37.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.843+0000 7f7dd253d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dc404a680 0x7f7dc417c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.843+0000 7f7dd253d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7dc40e5f30 0x7f7dc416d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.843+0000 7f7dd253d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7dc416df20 0x7f7dc40d2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.843+0000 7f7dd253d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc4045550 con 0x7f7dc404a680 2024-09-15T21:48:37.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.843+0000 7f7dd253d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc40e6ad0 con 0x7f7dc40e5f30 2024-09-15T21:48:37.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.843+0000 7f7dd253d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7dc4045830 con 0x7f7dc416df20 2024-09-15T21:48:37.885 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.843+0000 7f124670c640 1 -- start start 2024-09-15T21:48:37.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.843+0000 7f069dc8d640 1 Processor -- start 2024-09-15T21:48:37.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.843+0000 7f88002b7640 1 Processor -- start 2024-09-15T21:48:37.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.843+0000 7f7dd1d3c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7dc416df20 0x7f7dc40d2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.843+0000 7f7dd1d3c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7dc416df20 0x7f7dc40d2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:55626/0 (socket says 172.21.15.100:55626) 2024-09-15T21:48:37.886 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.843+0000 7f7dd1d3c640 1 -- 172.21.15.100:0/1512778488 learned_addr learned my addr 172.21.15.100:0/1512778488 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.843+0000 7f7dd0d3a640 1 --2- 172.21.15.100:0/1512778488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7dc40e5f30 0x7f7dc416d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.843+0000 7f3aa7fff640 1 -- 172.21.15.100:0/3624955462 shutdown_connections 2024-09-15T21:48:37.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.843+0000 7f3aa7fff640 1 -- 172.21.15.100:0/3624955462 wait complete. 2024-09-15T21:48:37.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.843+0000 7f069dc8d640 1 -- start start 2024-09-15T21:48:37.887 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f7dd1d3c640 1 -- 172.21.15.100:0/1512778488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7dc40e5f30 msgr2=0x7f7dc416d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f7dd1d3c640 1 --2- 172.21.15.100:0/1512778488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7dc40e5f30 0x7f7dc416d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f7dd1d3c640 1 -- 172.21.15.100:0/1512778488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dc404a680 msgr2=0x7f7dc417c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f7dd1d3c640 1 --2- 172.21.15.100:0/1512778488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dc404a680 0x7f7dc417c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f7dd1d3c640 1 -- 172.21.15.100:0/1512778488 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc40d8d10 con 0x7f7dc416df20 2024-09-15T21:48:37.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.842+0000 7f84eb72a640 1 -- start start 2024-09-15T21:48:37.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.843+0000 7f84eb72a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84e414e200 0x7f84e4153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.888 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f84eb72a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e414e5e0 0x7f84e4071f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f84eb72a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f84e4154140 0x7f84e4072470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f84eb72a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f84e4158ba0 con 0x7f84e414e5e0 2024-09-15T21:48:37.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f84eb72a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f84e40c9aa0 con 0x7f84e414e200 2024-09-15T21:48:37.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f84eb72a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f84e4158d20 con 0x7f84e4154140 2024-09-15T21:48:37.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f84e8cc5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e414e5e0 0x7f84e4071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f39d2ffd640 1 -- 172.21.15.100:0/959004194 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f39d8005d10 con 0x7f39e0159750 2024-09-15T21:48:37.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f84e8cc5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e414e5e0 0x7f84e4071f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54228/0 (socket says 172.21.15.100:54228) 2024-09-15T21:48:37.889 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f84e8cc5640 1 -- 172.21.15.100:0/3908811596 learned_addr learned my addr 172.21.15.100:0/3908811596 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f7dd1d3c640 1 --2- 172.21.15.100:0/1512778488 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7dc416df20 0x7f7dc40d2ca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc065710 tx=0x7f7dcc063980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f84e8cc5640 1 -- 172.21.15.100:0/3908811596 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f84e4154140 msgr2=0x7f84e4072470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-15T21:48:37.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f84e8cc5640 1 --2- 172.21.15.100:0/3908811596 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f84e4154140 0x7f84e4072470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f84e8cc5640 1 -- 172.21.15.100:0/3908811596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84e414e200 msgr2=0x7f84e4153c00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:48:37.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f84e8cc5640 1 --2- 172.21.15.100:0/3908811596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84e414e200 0x7f84e4153c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.890 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f84e8cc5640 1 -- 172.21.15.100:0/3908811596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e414ed60 con 0x7f84e414e5e0 2024-09-15T21:48:37.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f84e8cc5640 1 --2- 172.21.15.100:0/3908811596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e414e5e0 0x7f84e4071f30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f84d40017e0 tx=0x7f84d400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f124670c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12400c6fd0 0x7f12400cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.845+0000 7f39d2ffd640 1 --2- 172.21.15.100:0/959004194 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f39c406c2e0 0x7f39c406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.891 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f7dc1ffb640 1 -- 172.21.15.100:0/1512778488 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc0741c0 con 0x7f7dc416df20 2024-09-15T21:48:37.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f7dc1ffb640 1 -- 172.21.15.100:0/1512778488 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dcc06d0d0 con 0x7f7dc416df20 2024-09-15T21:48:37.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7f7dc1ffb640 1 -- 172.21.15.100:0/1512778488 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc06d4c0 con 0x7f7dc416df20 2024-09-15T21:48:37.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.842+0000 7fcaae7fc640 1 -- 172.21.15.100:0/1369220609 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab000ccb0 con 0x7fcab8159b00 2024-09-15T21:48:37.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.842+0000 7fcabfae4640 1 -- 172.21.15.100:0/1369220609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab8149f20 con 0x7fcab8159b00 2024-09-15T21:48:37.892 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.842+0000 7fcabfae4640 1 -- 172.21.15.100:0/1369220609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab815d270 con 0x7fcab8159b00 2024-09-15T21:48:37.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.845+0000 7f39e652d640 1 --2- 172.21.15.100:0/959004194 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f39c406c2e0 0x7f39c406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.844+0000 7fcabfae4640 1 -- 172.21.15.100:0/1369220609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcab80c8d50 con 0x7fcab8159b00 2024-09-15T21:48:37.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.845+0000 7f069dc8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06980c7a00 0x7f06981f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.893 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.845+0000 7f7dd253d640 1 -- 172.21.15.100:0/1512778488 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc417cf10 con 0x7f7dc416df20 2024-09-15T21:48:37.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.845+0000 7f39d2ffd640 1 -- 172.21.15.100:0/959004194 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f39d808c680 con 0x7f39e0159750 2024-09-15T21:48:37.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.845+0000 7f124670c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f12400c8380 0x7f1240158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.845+0000 7f06977fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06980c7a00 0x7f06981f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.845+0000 7f7dd253d640 1 -- 172.21.15.100:0/1512778488 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc416c040 con 0x7f7dc416df20 2024-09-15T21:48:37.895 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.845+0000 7f06977fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06980c7a00 0x7f06981f91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54230/0 (socket says 172.21.15.100:54230) 2024-09-15T21:48:37.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.845+0000 7f06977fe640 1 -- 172.21.15.100:0/78896842 learned_addr learned my addr 172.21.15.100:0/78896842 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.845+0000 7f39e652d640 1 --2- 172.21.15.100:0/959004194 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f39c406c2e0 0x7f39c406e7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f39d4000f60 tx=0x7f39d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.845+0000 7f069dc8d640 1 --2- 172.21.15.100:0/78896842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06980c9f00 0x7f0698145fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.845+0000 7f069dc8d640 1 --2- 172.21.15.100:0/78896842 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f06980ca2e0 0x7f0698143ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.845+0000 7f123ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12400c6fd0 0x7f12400cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.845+0000 7f3aa7fff640 1 Processor -- start 2024-09-15T21:48:37.897 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f88002b7640 1 -- start start 2024-09-15T21:48:37.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f0696ffd640 1 --2- 172.21.15.100:0/78896842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06980c9f00 0x7f0698145fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f123ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12400c6fd0 0x7f12400cc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:42942/0 (socket says 172.21.15.100:42942) 2024-09-15T21:48:37.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f0697fff640 1 --2- 172.21.15.100:0/78896842 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f06980ca2e0 0x7f0698143ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f069dc8d640 1 -- 172.21.15.100:0/78896842 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0698148930 con 0x7f06980c7a00 2024-09-15T21:48:37.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f3aa7fff640 1 -- start start 2024-09-15T21:48:37.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f069dc8d640 1 -- 172.21.15.100:0/78896842 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f069815f7a0 con 0x7f06980c9f00 2024-09-15T21:48:37.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f069dc8d640 1 -- 172.21.15.100:0/78896842 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0698148ab0 con 0x7f06980ca2e0 2024-09-15T21:48:37.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f123f7fe640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f12400c8380 0x7f1240158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f123f7fe640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f12400c8380 0x7f1240158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:55652/0 (socket says 172.21.15.100:55652) 2024-09-15T21:48:37.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f123ffff640 1 -- 172.21.15.100:0/2196938417 learned_addr learned my addr 172.21.15.100:0/2196938417 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.900 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f124670c640 1 --2- 172.21.15.100:0/2196938417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12400ca570 0x7f1240158a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f0696ffd640 1 -- 172.21.15.100:0/78896842 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f06980ca2e0 msgr2=0x7f0698143ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f0696ffd640 1 --2- 172.21.15.100:0/78896842 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f06980ca2e0 0x7f0698143ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f0696ffd640 1 -- 172.21.15.100:0/78896842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06980c7a00 msgr2=0x7f06981f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f0696ffd640 1 --2- 172.21.15.100:0/78896842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06980c7a00 0x7f06981f91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f0696ffd640 1 -- 172.21.15.100:0/78896842 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069815cb90 con 0x7f06980c9f00 2024-09-15T21:48:37.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f88002b7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80cc250 0x7f87f81e3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f88002b7640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87f80ccef0 0x7f87f81e38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f0696ffd640 1 --2- 172.21.15.100:0/78896842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06980c9f00 0x7f0698145fc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f068800c390 tx=0x7f0688001e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.902 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f7dd253d640 1 -- 172.21.15.100:0/1512778488 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dc40d8690 con 0x7f7dc416df20 2024-09-15T21:48:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f88002b7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87f80d0df0 0x7f87f81e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f88002b7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f87f81558f0 con 0x7f87f80cc250 2024-09-15T21:48:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f88002b7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f87f8155770 con 0x7f87f80d0df0 2024-09-15T21:48:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f88002b7640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f87f80d11d0 con 0x7f87f80ccef0 2024-09-15T21:48:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f87fd852640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87f80ccef0 0x7f87f81e38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f87fe053640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80cc250 0x7f87f81e3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f87fd852640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87f80ccef0 0x7f87f81e38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:55656/0 (socket says 172.21.15.100:55656) 2024-09-15T21:48:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f87fd852640 1 -- 172.21.15.100:0/404532966 learned_addr learned my addr 172.21.15.100:0/404532966 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.903 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f87fe854640 1 --2- 172.21.15.100:0/404532966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87f80d0df0 0x7f87f81e5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f3aa7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3aa8067d30 0x7f3aa80c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f124670c640 1 -- 172.21.15.100:0/2196938417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f1240157c30 con 0x7f12400ca570 2024-09-15T21:48:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f124670c640 1 -- 172.21.15.100:0/2196938417 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1240157950 con 0x7f12400c6fd0 2024-09-15T21:48:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.846+0000 7f124670c640 1 -- 172.21.15.100:0/2196938417 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f1240157600 con 0x7f12400c8380 2024-09-15T21:48:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f1244ca9640 1 --2- 172.21.15.100:0/2196938417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12400ca570 0x7f1240158a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f1244ca9640 1 -- 172.21.15.100:0/2196938417 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f12400c8380 msgr2=0x7f1240158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f1244ca9640 1 --2- 172.21.15.100:0/2196938417 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f12400c8380 0x7f1240158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f1244ca9640 1 -- 172.21.15.100:0/2196938417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12400c6fd0 msgr2=0x7f12400cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f1244ca9640 1 --2- 172.21.15.100:0/2196938417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12400c6fd0 0x7f12400cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f1244ca9640 1 -- 172.21.15.100:0/2196938417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12401519b0 con 0x7f12400ca570 2024-09-15T21:48:37.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f123ffff640 1 --2- 172.21.15.100:0/2196938417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12400c6fd0 0x7f12400cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:48:37.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f87fe854640 1 -- 172.21.15.100:0/404532966 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87f80ccef0 msgr2=0x7f87f81e38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f87fe854640 1 --2- 172.21.15.100:0/404532966 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87f80ccef0 0x7f87f81e38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f87fe854640 1 -- 172.21.15.100:0/404532966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80cc250 msgr2=0x7f87f81e3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f87fe854640 1 --2- 172.21.15.100:0/404532966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80cc250 0x7f87f81e3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f87fe854640 1 -- 172.21.15.100:0/404532966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f80c8610 con 0x7f87f80d0df0 2024-09-15T21:48:37.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f067ffff640 1 -- 172.21.15.100:0/78896842 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0688017070 con 0x7f06980c9f00 2024-09-15T21:48:37.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f067ffff640 1 -- 172.21.15.100:0/78896842 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0688005ce0 con 0x7f06980c9f00 2024-09-15T21:48:37.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f067ffff640 1 -- 172.21.15.100:0/78896842 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06880072d0 con 0x7f06980c9f00 2024-09-15T21:48:37.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f069dc8d640 1 -- 172.21.15.100:0/78896842 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f069815ce70 con 0x7f06980c9f00 2024-09-15T21:48:37.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f069dc8d640 1 -- 172.21.15.100:0/78896842 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06981f89a0 con 0x7f06980c9f00 2024-09-15T21:48:37.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.847+0000 7f1244ca9640 1 --2- 172.21.15.100:0/2196938417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12400ca570 0x7f1240158a40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f123800b4d0 tx=0x7f123800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.906 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f87fe854640 1 --2- 172.21.15.100:0/404532966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87f80d0df0 0x7f87f81e5f30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f87f0031000 tx=0x7f87f0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f3aa7fff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3aa806e1e0 0x7f3aa814ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f3aa7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3aa8152e80 0x7f3aa814f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f3aa7fff640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa806d780 con 0x7f3aa8152e80 2024-09-15T21:48:37.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f3aa7fff640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa806d4a0 con 0x7f3aa8067d30 2024-09-15T21:48:37.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f3aa7fff640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa8153260 con 0x7f3aa806e1e0 2024-09-15T21:48:37.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f3a9e7fc640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3aa806e1e0 0x7f3aa814ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3aa8152e80 0x7f3aa814f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f3aa6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3aa8067d30 0x7f3aa80c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f3aa6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3aa8067d30 0x7f3aa80c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:42968/0 (socket says 172.21.15.100:42968) 2024-09-15T21:48:37.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f3aa6ffd640 1 -- 172.21.15.100:0/3736388898 learned_addr learned my addr 172.21.15.100:0/3736388898 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:37.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3aa8152e80 0x7f3aa814f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54256/0 (socket says 172.21.15.100:54256) 2024-09-15T21:48:37.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f87eeffd640 1 -- 172.21.15.100:0/404532966 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f003f070 con 0x7f87f80d0df0 2024-09-15T21:48:37.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f88002b7640 1 -- 172.21.15.100:0/404532966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f81e4490 con 0x7f87f80d0df0 2024-09-15T21:48:37.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f7dc1ffb640 1 -- 172.21.15.100:0/1512778488 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7dcc06d660 con 0x7f7dc416df20 2024-09-15T21:48:37.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f88002b7640 1 -- 172.21.15.100:0/404532966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f80c8330 con 0x7f87f80d0df0 2024-09-15T21:48:37.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f3aa6ffd640 1 -- 172.21.15.100:0/3736388898 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3aa806e1e0 msgr2=0x7f3aa814ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f3aa6ffd640 1 --2- 172.21.15.100:0/3736388898 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3aa806e1e0 0x7f3aa814ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f3aa6ffd640 1 -- 172.21.15.100:0/3736388898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3aa8152e80 msgr2=0x7f3aa814f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:37.909 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f3aa6ffd640 1 --2- 172.21.15.100:0/3736388898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3aa8152e80 0x7f3aa814f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:37.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f3aa6ffd640 1 -- 172.21.15.100:0/3736388898 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa80cc0e0 con 0x7f3aa8067d30 2024-09-15T21:48:37.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.848+0000 7f87eeffd640 1 -- 172.21.15.100:0/404532966 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f0031b40 con 0x7f87f80d0df0 2024-09-15T21:48:37.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.849+0000 7f87eeffd640 1 -- 172.21.15.100:0/404532966 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f0044460 con 0x7f87f80d0df0 2024-09-15T21:48:37.910 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.849+0000 7f069dc8d640 1 -- 172.21.15.100:0/78896842 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06980735c0 con 0x7f06980c9f00 2024-09-15T21:48:37.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.849+0000 7f3aa6ffd640 1 --2- 172.21.15.100:0/3736388898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3aa8067d30 0x7f3aa80c5b80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000e8b0 tx=0x7f3a9000ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.911 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.849+0000 7f7dc1ffb640 1 --2- 172.21.15.100:0/1512778488 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7db006c380 0x7f7db006e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.849+0000 7f3a9ffff640 1 -- 172.21.15.100:0/3736388898 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a90017070 con 0x7f3aa8067d30 2024-09-15T21:48:37.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.849+0000 7f3a9ffff640 1 -- 172.21.15.100:0/3736388898 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a9000c940 con 0x7f3aa8067d30 2024-09-15T21:48:37.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.849+0000 7f3a9ffff640 1 -- 172.21.15.100:0/3736388898 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a9000cd10 con 0x7f3aa8067d30 2024-09-15T21:48:37.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.849+0000 7f7dc1ffb640 1 -- 172.21.15.100:0/1512778488 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f7dcc107ab0 con 0x7f7dc416df20 2024-09-15T21:48:37.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.849+0000 7f7dd153b640 1 --2- 172.21.15.100:0/1512778488 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7db006c380 0x7f7db006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.849+0000 7f3aa7fff640 1 -- 172.21.15.100:0/3736388898 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa80cb6d0 con 0x7f3aa8067d30 2024-09-15T21:48:37.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.849+0000 7f3aa7fff640 1 -- 172.21.15.100:0/3736388898 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa80c4ff0 con 0x7f3aa8067d30 2024-09-15T21:48:37.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.850+0000 7f7dd153b640 1 --2- 172.21.15.100:0/1512778488 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7db006c380 0x7f7db006e840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc034000 tx=0x7f7dbc02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.850+0000 7f88002b7640 1 -- 172.21.15.100:0/404532966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e0005290 con 0x7f87f80d0df0 2024-09-15T21:48:37.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.850+0000 7f067ffff640 1 -- 172.21.15.100:0/78896842 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f068801a070 con 0x7f06980c9f00 2024-09-15T21:48:37.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.850+0000 7f87eeffd640 1 -- 172.21.15.100:0/404532966 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f87f0044600 con 0x7f87f80d0df0 2024-09-15T21:48:37.914 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.851+0000 7f067ffff640 1 --2- 172.21.15.100:0/78896842 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f068406c330 0x7f068406e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.851+0000 7f3aa7fff640 1 -- 172.21.15.100:0/3736388898 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa80cc890 con 0x7f3aa8067d30 2024-09-15T21:48:37.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.851+0000 7f067ffff640 1 -- 172.21.15.100:0/78896842 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f068808a580 con 0x7f06980c9f00 2024-09-15T21:48:37.915 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.851+0000 7f06977fe640 1 --2- 172.21.15.100:0/78896842 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f068406c330 0x7f068406e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.851+0000 7f87eeffd640 1 --2- 172.21.15.100:0/404532966 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f87dc06c2e0 0x7f87dc06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.851+0000 7f87fe053640 1 --2- 172.21.15.100:0/404532966 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f87dc06c2e0 0x7f87dc06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.851+0000 7f87eeffd640 1 -- 172.21.15.100:0/404532966 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f87f00acab0 con 0x7f87f80d0df0 2024-09-15T21:48:37.916 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.851+0000 7f39d2ffd640 1 -- 172.21.15.100:0/959004194 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f39e0158350 con 0x7f39e0159750 2024-09-15T21:48:37.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.851+0000 7f87fe053640 1 --2- 172.21.15.100:0/404532966 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f87dc06c2e0 0x7f87dc06e7a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f87e8000f60 tx=0x7f87e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.851+0000 7fcaae7fc640 1 -- 172.21.15.100:0/1369220609 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcab000ce50 con 0x7fcab8159b00 2024-09-15T21:48:37.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.851+0000 7fcaae7fc640 1 -- 172.21.15.100:0/1369220609 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcab0003b90 con 0x7fcab8159b00 2024-09-15T21:48:37.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.852+0000 7f84d9ffb640 1 -- 172.21.15.100:0/3908811596 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d400fe80 con 0x7f84e414e5e0 2024-09-15T21:48:37.917 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.852+0000 7f84eb72a640 1 -- 172.21.15.100:0/3908811596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e4154640 con 0x7f84e414e5e0 2024-09-15T21:48:37.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.852+0000 7f123cff9640 1 -- 172.21.15.100:0/2196938417 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238025ce0 con 0x7f12400ca570 2024-09-15T21:48:37.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.852+0000 7f06977fe640 1 --2- 172.21.15.100:0/78896842 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f068406c330 0x7f068406e7f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f068c009560 tx=0x7f068c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.852+0000 7f124670c640 1 -- 172.21.15.100:0/2196938417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1240141430 con 0x7f12400ca570 2024-09-15T21:48:37.918 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.852+0000 7f84eb72a640 1 -- 172.21.15.100:0/3908811596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e41519c0 con 0x7f84e414e5e0 2024-09-15T21:48:37.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.852+0000 7f124670c640 1 -- 172.21.15.100:0/2196938417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1240142150 con 0x7f12400ca570 2024-09-15T21:48:37.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.853+0000 7f7dc1ffb640 1 -- 172.21.15.100:0/1512778488 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7dcc0d61f0 con 0x7f7dc416df20 2024-09-15T21:48:37.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.853+0000 7f123cff9640 1 -- 172.21.15.100:0/2196938417 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1238013030 con 0x7f12400ca570 2024-09-15T21:48:37.919 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.853+0000 7f123cff9640 1 -- 172.21.15.100:0/2196938417 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1238013420 con 0x7f12400ca570 2024-09-15T21:48:37.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.853+0000 7f84eb72a640 1 -- 172.21.15.100:0/3908811596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84e40c7fe0 con 0x7f84e414e5e0 2024-09-15T21:48:37.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.854+0000 7f124670c640 1 -- 172.21.15.100:0/2196938417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1220005290 con 0x7f12400ca570 2024-09-15T21:48:37.920 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.854+0000 7f84d9ffb640 1 -- 172.21.15.100:0/3908811596 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84d4018070 con 0x7f84e414e5e0 2024-09-15T21:48:37.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.854+0000 7f067ffff640 1 -- 172.21.15.100:0/78896842 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f06980735c0 con 0x7f06980c9f00 2024-09-15T21:48:37.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.855+0000 7f84d9ffb640 1 -- 172.21.15.100:0/3908811596 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d400de30 con 0x7f84e414e5e0 2024-09-15T21:48:37.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.855+0000 7fcaae7fc640 1 -- 172.21.15.100:0/1369220609 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fcab0012050 con 0x7fcab8159b00 2024-09-15T21:48:37.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.855+0000 7f123cff9640 1 -- 172.21.15.100:0/2196938417 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f1238003a30 con 0x7f12400ca570 2024-09-15T21:48:37.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.856+0000 7f123cff9640 1 --2- 172.21.15.100:0/2196938417 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f122c06c2e0 0x7f122c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.856+0000 7f123ffff640 1 --2- 172.21.15.100:0/2196938417 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f122c06c2e0 0x7f122c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.855+0000 7fcaae7fc640 1 --2- 172.21.15.100:0/1369220609 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fca9c06c2b0 0x7fca9c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.855+0000 7fcaae7fc640 1 -- 172.21.15.100:0/1369220609 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7fcab008c3d0 con 0x7fcab8159b00 2024-09-15T21:48:37.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.856+0000 7fcabd880640 1 --2- 172.21.15.100:0/1369220609 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fca9c06c2b0 0x7fca9c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.856+0000 7fcabd880640 1 --2- 172.21.15.100:0/1369220609 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fca9c06c2b0 0x7fca9c06e770 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcab4034000 tx=0x7fcab402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.856+0000 7f3a9ffff640 1 -- 172.21.15.100:0/3736388898 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3a9001a070 con 0x7f3aa8067d30 2024-09-15T21:48:37.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.856+0000 7f3a9ffff640 1 --2- 172.21.15.100:0/3736388898 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3a8c06c500 0x7f3a8c06e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.856+0000 7f3a9e7fc640 1 --2- 172.21.15.100:0/3736388898 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3a8c06c500 0x7f3a8c06e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.857+0000 7f3a9e7fc640 1 --2- 172.21.15.100:0/3736388898 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3a8c06c500 0x7f3a8c06e9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3a98034000 tx=0x7f3a9802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.857+0000 7f123ffff640 1 --2- 172.21.15.100:0/2196938417 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f122c06c2e0 0x7f122c06e7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1230009630 tx=0x7f123002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.859+0000 7f3a9ffff640 1 -- 172.21.15.100:0/3736388898 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f3a9000caf0 con 0x7f3aa8067d30 2024-09-15T21:48:37.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.859+0000 7f87eeffd640 1 -- 172.21.15.100:0/404532966 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f87f007b180 con 0x7f87f80d0df0 2024-09-15T21:48:37.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.860+0000 7fcaae7fc640 1 -- 172.21.15.100:0/1369220609 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fcab0010040 con 0x7fcab8159b00 2024-09-15T21:48:37.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.860+0000 7f123cff9640 1 -- 172.21.15.100:0/2196938417 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f1238069a30 con 0x7f12400ca570 2024-09-15T21:48:37.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.862+0000 7f84d9ffb640 1 -- 172.21.15.100:0/3908811596 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f84d40050d0 con 0x7f84e414e5e0 2024-09-15T21:48:37.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.862+0000 7f84d9ffb640 1 --2- 172.21.15.100:0/3908811596 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f84c806c2b0 0x7f84c806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:37.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.863+0000 7f84e94c6640 1 --2- 172.21.15.100:0/3908811596 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f84c806c2b0 0x7f84c806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:37.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.863+0000 7f3a9ffff640 1 -- 172.21.15.100:0/3736388898 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3a90059a70 con 0x7f3aa8067d30 2024-09-15T21:48:37.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.863+0000 7f84e94c6640 1 --2- 172.21.15.100:0/3908811596 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f84c806c2b0 0x7f84c806e770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f84e0057000 tx=0x7f84e0052000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:37.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.871+0000 7f123cff9640 1 -- 172.21.15.100:0/2196938417 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f123800f050 con 0x7f12400ca570 2024-09-15T21:48:37.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.871+0000 7f84d9ffb640 1 -- 172.21.15.100:0/3908811596 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f84d4058210 con 0x7f84e414e5e0 2024-09-15T21:48:37.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:37.878+0000 7f84d9ffb640 1 -- 172.21.15.100:0/3908811596 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f84d405a870 con 0x7f84e414e5e0 2024-09-15T21:48:38.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.241+0000 7f7dd253d640 1 -- 172.21.15.100:0/1512778488 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7dc40e2c10 con 0x7f7dc416df20 2024-09-15T21:48:38.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.242+0000 7f7dc1ffb640 1 -- 172.21.15.100:0/1512778488 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7dc40e2c10 con 0x7f7dc416df20 2024-09-15T21:48:38.245 INFO:teuthology.orchestra.run.smithi100.stdout:60129542175 2024-09-15T21:48:38.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.245+0000 7f7daf7fe640 1 -- 172.21.15.100:0/1512778488 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7db006c380 msgr2=0x7f7db006e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:38.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.245+0000 7f7daf7fe640 1 --2- 172.21.15.100:0/1512778488 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7db006c380 0x7f7db006e840 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc034000 tx=0x7f7dbc02c000 comp rx=0 tx=0).stop 2024-09-15T21:48:38.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.246+0000 7f7daf7fe640 1 -- 172.21.15.100:0/1512778488 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7dc416df20 msgr2=0x7f7dc40d2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:38.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.246+0000 7f7daf7fe640 1 --2- 172.21.15.100:0/1512778488 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7dc416df20 0x7f7dc40d2ca0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc065710 tx=0x7f7dcc063980 comp rx=0 tx=0).stop 2024-09-15T21:48:38.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.246+0000 7f7daf7fe640 1 -- 172.21.15.100:0/1512778488 shutdown_connections 2024-09-15T21:48:38.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.246+0000 7f7daf7fe640 1 --2- 172.21.15.100:0/1512778488 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f7db006c380 0x7f7db006e840 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.246+0000 7f7daf7fe640 1 --2- 172.21.15.100:0/1512778488 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7dc40e5f30 0x7f7dc416d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.246+0000 7f7daf7fe640 1 --2- 172.21.15.100:0/1512778488 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f7dc416df20 0x7f7dc40d2ca0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.246+0000 7f7daf7fe640 1 --2- 172.21.15.100:0/1512778488 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7dc404a680 0x7f7dc417c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.246+0000 7f7daf7fe640 1 -- 172.21.15.100:0/1512778488 >> 172.21.15.100:0/1512778488 conn(0x7f7dc404fc50 msgr2=0x7f7dc40e63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:38.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.246+0000 7f7daf7fe640 1 -- 172.21.15.100:0/1512778488 shutdown_connections 2024-09-15T21:48:38.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.246+0000 7f7daf7fe640 1 -- 172.21.15.100:0/1512778488 wait complete. 2024-09-15T21:48:38.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.247+0000 7f39e8791640 1 -- 172.21.15.100:0/959004194 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f39e01f5170 con 0x7f39e0159750 2024-09-15T21:48:38.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.248+0000 7f39d2ffd640 1 -- 172.21.15.100:0/959004194 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f39d8022070 con 0x7f39e0159750 2024-09-15T21:48:38.251 INFO:teuthology.orchestra.run.smithi100.stdout:64424509471 2024-09-15T21:48:38.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.252+0000 7f39d0ff9640 1 -- 172.21.15.100:0/959004194 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f39c406c2e0 msgr2=0x7f39c406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:38.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.252+0000 7f39d0ff9640 1 --2- 172.21.15.100:0/959004194 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f39c406c2e0 0x7f39c406e7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f39d4000f60 tx=0x7f39d4009040 comp rx=0 tx=0).stop 2024-09-15T21:48:38.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.252+0000 7f39d0ff9640 1 -- 172.21.15.100:0/959004194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e0159750 msgr2=0x7f39e01525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:38.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.252+0000 7f39d0ff9640 1 --2- 172.21.15.100:0/959004194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e0159750 0x7f39e01525c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f39d800efc0 tx=0x7f39d800c490 comp rx=0 tx=0).stop 2024-09-15T21:48:38.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.253+0000 7f39d0ff9640 1 -- 172.21.15.100:0/959004194 shutdown_connections 2024-09-15T21:48:38.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.253+0000 7f39d0ff9640 1 --2- 172.21.15.100:0/959004194 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f39c406c2e0 0x7f39c406e7a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.253+0000 7f39d0ff9640 1 --2- 172.21.15.100:0/959004194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f39e00cc9f0 0x7f39e01f90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.253+0000 7f39d0ff9640 1 --2- 172.21.15.100:0/959004194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39e0159750 0x7f39e01525c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.256 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.253+0000 7f39d0ff9640 1 --2- 172.21.15.100:0/959004194 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f39e00b3710 0x7f39e01e8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.253+0000 7f39d0ff9640 1 -- 172.21.15.100:0/959004194 >> 172.21.15.100:0/959004194 conn(0x7f39e00680d0 msgr2=0x7f39e0156f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.253+0000 7f39d0ff9640 1 -- 172.21.15.100:0/959004194 shutdown_connections 2024-09-15T21:48:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.253+0000 7f39d0ff9640 1 -- 172.21.15.100:0/959004194 wait complete. 2024-09-15T21:48:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.254+0000 7f069dc8d640 1 -- 172.21.15.100:0/78896842 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0698161a90 con 0x7f06980c9f00 2024-09-15T21:48:38.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.255+0000 7f067ffff640 1 -- 172.21.15.100:0/78896842 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0698161a90 con 0x7f06980c9f00 2024-09-15T21:48:38.258 INFO:teuthology.orchestra.run.smithi100.stdout:244813135877 2024-09-15T21:48:38.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.259+0000 7f067dffb640 1 -- 172.21.15.100:0/78896842 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f068406c330 msgr2=0x7f068406e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:38.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.259+0000 7f067dffb640 1 --2- 172.21.15.100:0/78896842 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f068406c330 0x7f068406e7f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f068c009560 tx=0x7f068c02e000 comp rx=0 tx=0).stop 2024-09-15T21:48:38.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.259+0000 7f067dffb640 1 -- 172.21.15.100:0/78896842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06980c9f00 msgr2=0x7f0698145fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:38.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.259+0000 7f067dffb640 1 --2- 172.21.15.100:0/78896842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06980c9f00 0x7f0698145fc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f068800c390 tx=0x7f0688001e70 comp rx=0 tx=0).stop 2024-09-15T21:48:38.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.259+0000 7f067dffb640 1 -- 172.21.15.100:0/78896842 shutdown_connections 2024-09-15T21:48:38.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.259+0000 7f067dffb640 1 --2- 172.21.15.100:0/78896842 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f068406c330 0x7f068406e7f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.259+0000 7f067dffb640 1 --2- 172.21.15.100:0/78896842 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f06980c9f00 0x7f0698145fc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.259+0000 7f067dffb640 1 --2- 172.21.15.100:0/78896842 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f06980ca2e0 0x7f0698143ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.262 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.259+0000 7f067dffb640 1 --2- 172.21.15.100:0/78896842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06980c7a00 0x7f06981f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.259+0000 7f067dffb640 1 -- 172.21.15.100:0/78896842 >> 172.21.15.100:0/78896842 conn(0x7f0698073820 msgr2=0x7f0698161250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:38.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.259+0000 7f067dffb640 1 -- 172.21.15.100:0/78896842 shutdown_connections 2024-09-15T21:48:38.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.259+0000 7f067dffb640 1 -- 172.21.15.100:0/78896842 wait complete. 2024-09-15T21:48:38.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.260+0000 7fcabfae4640 1 -- 172.21.15.100:0/1369220609 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fcab814fcb0 con 0x7fcab8159b00 2024-09-15T21:48:38.263 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.260+0000 7fcaae7fc640 1 -- 172.21.15.100:0/1369220609 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcab005a430 con 0x7fcab8159b00 2024-09-15T21:48:38.264 INFO:teuthology.orchestra.run.smithi100.stdout:64424509471 2024-09-15T21:48:38.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.261+0000 7f88002b7640 1 -- 172.21.15.100:0/404532966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f87e0005850 con 0x7f87f80d0df0 2024-09-15T21:48:38.264 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.261+0000 7f87eeffd640 1 -- 172.21.15.100:0/404532966 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f87f007aa80 con 0x7f87f80d0df0 2024-09-15T21:48:38.264 INFO:teuthology.orchestra.run.smithi100.stdout:64424509472 2024-09-15T21:48:38.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.262+0000 7f3aa7fff640 1 -- 172.21.15.100:0/3736388898 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3aa806dca0 con 0x7f3aa8067d30 2024-09-15T21:48:38.265 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.262+0000 7f3a9ffff640 1 -- 172.21.15.100:0/3736388898 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3a9005dd70 con 0x7f3aa8067d30 2024-09-15T21:48:38.265 INFO:teuthology.orchestra.run.smithi100.stdout:55834574880 2024-09-15T21:48:38.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7f88002b7640 1 -- 172.21.15.100:0/404532966 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f87dc06c2e0 msgr2=0x7f87dc06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:38.268 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7f88002b7640 1 --2- 172.21.15.100:0/404532966 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f87dc06c2e0 0x7f87dc06e7a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f87e8000f60 tx=0x7f87e8009040 comp rx=0 tx=0).stop 2024-09-15T21:48:38.269 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542177 got 60129542175 for osd.1 2024-09-15T21:48:38.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7f88002b7640 1 -- 172.21.15.100:0/404532966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87f80d0df0 msgr2=0x7f87f81e5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:38.269 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7f88002b7640 1 --2- 172.21.15.100:0/404532966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87f80d0df0 0x7f87f81e5f30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f87f0031000 tx=0x7f87f0004a40 comp rx=0 tx=0).stop 2024-09-15T21:48:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7fcabfae4640 1 -- 172.21.15.100:0/1369220609 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fca9c06c2b0 msgr2=0x7fca9c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7fcabfae4640 1 --2- 172.21.15.100:0/1369220609 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fca9c06c2b0 0x7fca9c06e770 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcab4034000 tx=0x7fcab402e000 comp rx=0 tx=0).stop 2024-09-15T21:48:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7f88002b7640 1 -- 172.21.15.100:0/404532966 shutdown_connections 2024-09-15T21:48:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7f88002b7640 1 --2- 172.21.15.100:0/404532966 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f87dc06c2e0 0x7f87dc06e7a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7f88002b7640 1 --2- 172.21.15.100:0/404532966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f87f80d0df0 0x7f87f81e5f30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7f88002b7640 1 --2- 172.21.15.100:0/404532966 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f87f80ccef0 0x7f87f81e38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7f88002b7640 1 --2- 172.21.15.100:0/404532966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f87f80cc250 0x7f87f81e3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.270 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7f88002b7640 1 -- 172.21.15.100:0/404532966 >> 172.21.15.100:0/404532966 conn(0x7f87f8067980 msgr2=0x7f87f8151e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:38.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7fcabfae4640 1 -- 172.21.15.100:0/1369220609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcab8159b00 msgr2=0x7fcab814adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:38.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7fcabfae4640 1 --2- 172.21.15.100:0/1369220609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcab8159b00 0x7fcab814adb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fcab80c80e0 tx=0x7fcab000ef90 comp rx=0 tx=0).stop 2024-09-15T21:48:38.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7f88002b7640 1 -- 172.21.15.100:0/404532966 shutdown_connections 2024-09-15T21:48:38.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7f88002b7640 1 -- 172.21.15.100:0/404532966 wait complete. 2024-09-15T21:48:38.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7fcabfae4640 1 -- 172.21.15.100:0/1369220609 shutdown_connections 2024-09-15T21:48:38.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7fcabfae4640 1 --2- 172.21.15.100:0/1369220609 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fca9c06c2b0 0x7fca9c06e770 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7fcabfae4640 1 --2- 172.21.15.100:0/1369220609 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcab8159b00 0x7fcab814adb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7fcabfae4640 1 --2- 172.21.15.100:0/1369220609 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fcab8159720 0x7fcab8159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7fcabfae4640 1 --2- 172.21.15.100:0/1369220609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcab80c7c90 0x7fcab815dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.265+0000 7fcabfae4640 1 -- 172.21.15.100:0/1369220609 >> 172.21.15.100:0/1369220609 conn(0x7fcab8067dd0 msgr2=0x7fcab8157140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:38.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.266+0000 7f3aa7fff640 1 -- 172.21.15.100:0/3736388898 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3a8c06c500 msgr2=0x7f3a8c06e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:38.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.266+0000 7fcabfae4640 1 -- 172.21.15.100:0/1369220609 shutdown_connections 2024-09-15T21:48:38.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.266+0000 7fcabfae4640 1 -- 172.21.15.100:0/1369220609 wait complete. 2024-09-15T21:48:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.266+0000 7f3aa7fff640 1 --2- 172.21.15.100:0/3736388898 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3a8c06c500 0x7f3a8c06e9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3a98034000 tx=0x7f3a9802e000 comp rx=0 tx=0).stop 2024-09-15T21:48:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.266+0000 7f3aa7fff640 1 -- 172.21.15.100:0/3736388898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3aa8067d30 msgr2=0x7f3aa80c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.266+0000 7f3aa7fff640 1 --2- 172.21.15.100:0/3736388898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3aa8067d30 0x7f3aa80c5b80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3a9000e8b0 tx=0x7f3a9000ed70 comp rx=0 tx=0).stop 2024-09-15T21:48:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.266+0000 7f3aa7fff640 1 -- 172.21.15.100:0/3736388898 shutdown_connections 2024-09-15T21:48:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.266+0000 7f3aa7fff640 1 --2- 172.21.15.100:0/3736388898 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3a8c06c500 0x7f3a8c06e9c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.266+0000 7f3aa7fff640 1 --2- 172.21.15.100:0/3736388898 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3aa8152e80 0x7f3aa814f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.266+0000 7f3aa7fff640 1 --2- 172.21.15.100:0/3736388898 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3aa806e1e0 0x7f3aa814ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.266+0000 7f3aa7fff640 1 --2- 172.21.15.100:0/3736388898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3aa8067d30 0x7f3aa80c5b80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.266+0000 7f3aa7fff640 1 -- 172.21.15.100:0/3736388898 >> 172.21.15.100:0/3736388898 conn(0x7f3aa806f130 msgr2=0x7f3aa80c85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:38.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.266+0000 7f3aa7fff640 1 -- 172.21.15.100:0/3736388898 shutdown_connections 2024-09-15T21:48:38.276 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.267+0000 7f3aa7fff640 1 -- 172.21.15.100:0/3736388898 wait complete. 2024-09-15T21:48:38.277 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509472 got 64424509471 for osd.2 2024-09-15T21:48:38.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.274+0000 7f124670c640 1 -- 172.21.15.100:0/2196938417 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f1220005850 con 0x7f12400ca570 2024-09-15T21:48:38.279 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.275+0000 7f123cff9640 1 -- 172.21.15.100:0/2196938417 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1238070e20 con 0x7f12400ca570 2024-09-15T21:48:38.279 INFO:teuthology.orchestra.run.smithi100.stdout:64424509473 2024-09-15T21:48:38.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.279+0000 7f124670c640 1 -- 172.21.15.100:0/2196938417 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f122c06c2e0 msgr2=0x7f122c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:38.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.279+0000 7f124670c640 1 --2- 172.21.15.100:0/2196938417 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f122c06c2e0 0x7f122c06e7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1230009630 tx=0x7f123002e000 comp rx=0 tx=0).stop 2024-09-15T21:48:38.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.279+0000 7f124670c640 1 -- 172.21.15.100:0/2196938417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12400ca570 msgr2=0x7f1240158a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:38.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.279+0000 7f124670c640 1 --2- 172.21.15.100:0/2196938417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12400ca570 0x7f1240158a40 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f123800b4d0 tx=0x7f123800b990 comp rx=0 tx=0).stop 2024-09-15T21:48:38.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.279+0000 7f124670c640 1 -- 172.21.15.100:0/2196938417 shutdown_connections 2024-09-15T21:48:38.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.279+0000 7f124670c640 1 --2- 172.21.15.100:0/2196938417 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f122c06c2e0 0x7f122c06e7a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.279+0000 7f124670c640 1 --2- 172.21.15.100:0/2196938417 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12400ca570 0x7f1240158a40 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.279+0000 7f124670c640 1 --2- 172.21.15.100:0/2196938417 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f12400c8380 0x7f1240158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.279+0000 7f124670c640 1 --2- 172.21.15.100:0/2196938417 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12400c6fd0 0x7f12400cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.282 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.279+0000 7f124670c640 1 -- 172.21.15.100:0/2196938417 >> 172.21.15.100:0/2196938417 conn(0x7f1240073820 msgr2=0x7f12400c76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:38.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.279+0000 7f124670c640 1 -- 172.21.15.100:0/2196938417 shutdown_connections 2024-09-15T21:48:38.283 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.279+0000 7f124670c640 1 -- 172.21.15.100:0/2196938417 wait complete. 2024-09-15T21:48:38.283 INFO:tasks.ceph.ceph_manager.ceph:need seq 244813135878 got 244813135877 for osd.5 2024-09-15T21:48:38.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.281+0000 7f84eb72a640 1 -- 172.21.15.100:0/3908811596 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f84e4073e00 con 0x7f84e414e5e0 2024-09-15T21:48:38.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.282+0000 7f84d9ffb640 1 -- 172.21.15.100:0/3908811596 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f84d405a170 con 0x7f84e414e5e0 2024-09-15T21:48:38.284 INFO:teuthology.orchestra.run.smithi100.stdout:68719476767 2024-09-15T21:48:38.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.284+0000 7f84eb72a640 1 -- 172.21.15.100:0/3908811596 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f84c806c2b0 msgr2=0x7f84c806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:38.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.284+0000 7f84eb72a640 1 --2- 172.21.15.100:0/3908811596 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f84c806c2b0 0x7f84c806e770 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f84e0057000 tx=0x7f84e0052000 comp rx=0 tx=0).stop 2024-09-15T21:48:38.287 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.284+0000 7f84eb72a640 1 -- 172.21.15.100:0/3908811596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e414e5e0 msgr2=0x7f84e4071f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:38.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.284+0000 7f84eb72a640 1 --2- 172.21.15.100:0/3908811596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e414e5e0 0x7f84e4071f30 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f84d40017e0 tx=0x7f84d400d520 comp rx=0 tx=0).stop 2024-09-15T21:48:38.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.284+0000 7f84eb72a640 1 -- 172.21.15.100:0/3908811596 shutdown_connections 2024-09-15T21:48:38.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.284+0000 7f84eb72a640 1 --2- 172.21.15.100:0/3908811596 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f84c806c2b0 0x7f84c806e770 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.284+0000 7f84eb72a640 1 --2- 172.21.15.100:0/3908811596 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f84e4154140 0x7f84e4072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.284+0000 7f84eb72a640 1 --2- 172.21.15.100:0/3908811596 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84e414e5e0 0x7f84e4071f30 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.284+0000 7f84eb72a640 1 --2- 172.21.15.100:0/3908811596 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84e414e200 0x7f84e4153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:38.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.284+0000 7f84eb72a640 1 -- 172.21.15.100:0/3908811596 >> 172.21.15.100:0/3908811596 conn(0x7f84e4067dd0 msgr2=0x7f84e406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:38.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.284+0000 7f84eb72a640 1 -- 172.21.15.100:0/3908811596 shutdown_connections 2024-09-15T21:48:38.288 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:38.284+0000 7f84eb72a640 1 -- 172.21.15.100:0/3908811596 wait complete. 2024-09-15T21:48:38.289 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509472 got 64424509471 for osd.7 2024-09-15T21:48:38.290 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509473 got 64424509472 for osd.4 2024-09-15T21:48:38.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574881 got 55834574880 for osd.3 2024-09-15T21:48:38.296 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509474 got 64424509473 for osd.6 2024-09-15T21:48:38.300 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476768 got 68719476767 for osd.0 2024-09-15T21:48:39.270 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:48:39.278 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:48:39.284 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T21:48:39.290 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T21:48:39.290 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T21:48:39.292 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:48:39.297 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T21:48:39.300 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:48:39.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.434+0000 7f552a039640 1 Processor -- start 2024-09-15T21:48:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.435+0000 7f552a039640 1 -- start start 2024-09-15T21:48:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.436+0000 7f552a039640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f552406e5c0 0x7f5524154310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.436+0000 7f552a039640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f552406e9a0 0x7f5524154850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.436+0000 7f552a039640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f55240c3b10 0x7f55240c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.436+0000 7f552a039640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f55240c6f10 con 0x7f552406e9a0 2024-09-15T21:48:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.436+0000 7f552a039640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f5524157ca0 con 0x7f552406e5c0 2024-09-15T21:48:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.436+0000 7f552a039640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5524157f40 con 0x7f55240c3b10 2024-09-15T21:48:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.437+0000 7f5528836640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f552406e9a0 0x7f5524154850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.437+0000 7f5528836640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f552406e9a0 0x7f5524154850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54272/0 (socket says 172.21.15.100:54272) 2024-09-15T21:48:39.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.437+0000 7f5528836640 1 -- 172.21.15.100:0/1624487820 learned_addr learned my addr 172.21.15.100:0/1624487820 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:39.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.438+0000 7f5528836640 1 -- 172.21.15.100:0/1624487820 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f55240c3b10 msgr2=0x7f55240c7c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:48:39.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.438+0000 7f5528836640 1 --2- 172.21.15.100:0/1624487820 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f55240c3b10 0x7f55240c7c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.438+0000 7f5528836640 1 -- 172.21.15.100:0/1624487820 >> v2:172.21.15.122:3300/0 conn(0x7f552406e5c0 msgr2=0x7f5524154310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-15T21:48:39.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.438+0000 7f5528836640 1 --2- 172.21.15.100:0/1624487820 >> v2:172.21.15.122:3300/0 conn(0x7f552406e5c0 0x7f5524154310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.438+0000 7f5528836640 1 -- 172.21.15.100:0/1624487820 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55240c7160 con 0x7f552406e9a0 2024-09-15T21:48:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.439+0000 7f5528836640 1 --2- 172.21.15.100:0/1624487820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f552406e9a0 0x7f5524154850 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5520009ad0 tx=0x7f552002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3013e62d948a8644 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.440+0000 7f5517fff640 1 -- 172.21.15.100:0/1624487820 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520036070 con 0x7f552406e9a0 2024-09-15T21:48:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.440+0000 7f5517fff640 1 -- 172.21.15.100:0/1624487820 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5520031600 con 0x7f552406e9a0 2024-09-15T21:48:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.441+0000 7f552a039640 1 -- 172.21.15.100:0/1624487820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f552406e9a0 msgr2=0x7f5524154850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.441+0000 7f552a039640 1 --2- 172.21.15.100:0/1624487820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f552406e9a0 0x7f5524154850 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5520009ad0 tx=0x7f552002a420 comp rx=0 tx=0).stop 2024-09-15T21:48:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.442+0000 7f552a039640 1 -- 172.21.15.100:0/1624487820 shutdown_connections 2024-09-15T21:48:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.442+0000 7f552a039640 1 --2- 172.21.15.100:0/1624487820 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f55240c3b10 0x7f55240c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.442+0000 7f552a039640 1 --2- 172.21.15.100:0/1624487820 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f552406e9a0 0x7f5524154850 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.442+0000 7f552a039640 1 --2- 172.21.15.100:0/1624487820 >> v2:172.21.15.122:3300/0 conn(0x7f552406e5c0 0x7f5524154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.442+0000 7f552a039640 1 -- 172.21.15.100:0/1624487820 >> 172.21.15.100:0/1624487820 conn(0x7f55240c9110 msgr2=0x7f55240c9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.442+0000 7f552a039640 1 -- 172.21.15.100:0/1624487820 shutdown_connections 2024-09-15T21:48:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.442+0000 7f552a039640 1 -- 172.21.15.100:0/1624487820 wait complete. 2024-09-15T21:48:39.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.444+0000 7f552a039640 1 Processor -- start 2024-09-15T21:48:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.444+0000 7f552a039640 1 -- start start 2024-09-15T21:48:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.446+0000 7f552a039640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f552406e5c0 0x7f55240cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.446+0000 7f552a039640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f552406e9a0 0x7f55240c1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.446+0000 7f552a039640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55240c3b10 0x7f55240c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.446+0000 7f552a039640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f55241585f0 con 0x7f55240c3b10 2024-09-15T21:48:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.446+0000 7f552a039640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f55240c32f0 con 0x7f552406e5c0 2024-09-15T21:48:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.446+0000 7f552a039640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5524155820 con 0x7f552406e9a0 2024-09-15T21:48:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.446+0000 7f5529838640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55240c3b10 0x7f55240c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.447+0000 7f5529838640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55240c3b10 0x7f55240c1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54280/0 (socket says 172.21.15.100:54280) 2024-09-15T21:48:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.447+0000 7f5529838640 1 -- 172.21.15.100:0/806541077 learned_addr learned my addr 172.21.15.100:0/806541077 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.447+0000 7f5529838640 1 -- 172.21.15.100:0/806541077 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f552406e9a0 msgr2=0x7f55240c1800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:48:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.447+0000 7f5529838640 1 --2- 172.21.15.100:0/806541077 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f552406e9a0 0x7f55240c1800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.447+0000 7f5529838640 1 -- 172.21.15.100:0/806541077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f552406e5c0 msgr2=0x7f55240cb7d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-15T21:48:39.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.447+0000 7f5529838640 1 --2- 172.21.15.100:0/806541077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f552406e5c0 0x7f55240cb7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.447+0000 7f5529838640 1 -- 172.21.15.100:0/806541077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5524156720 con 0x7f55240c3b10 2024-09-15T21:48:39.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.448+0000 7f5529838640 1 --2- 172.21.15.100:0/806541077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55240c3b10 0x7f55240c1d40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f551c00bd60 tx=0x7f551c009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.448+0000 7f5515ffb640 1 -- 172.21.15.100:0/806541077 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c019070 con 0x7f55240c3b10 2024-09-15T21:48:39.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.449+0000 7f552a039640 1 -- 172.21.15.100:0/806541077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5524157180 con 0x7f55240c3b10 2024-09-15T21:48:39.454 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.449+0000 7f552a039640 1 -- 172.21.15.100:0/806541077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5524155260 con 0x7f55240c3b10 2024-09-15T21:48:39.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.453+0000 7f5515ffb640 1 -- 172.21.15.100:0/806541077 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f551c00cc20 con 0x7f55240c3b10 2024-09-15T21:48:39.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.454+0000 7f5515ffb640 1 -- 172.21.15.100:0/806541077 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f551c014440 con 0x7f55240c3b10 2024-09-15T21:48:39.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.454+0000 7f5515ffb640 1 -- 172.21.15.100:0/806541077 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f551c01c070 con 0x7f55240c3b10 2024-09-15T21:48:39.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.455+0000 7f5515ffb640 1 --2- 172.21.15.100:0/806541077 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f550806c3b0 0x7f550806e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.455+0000 7f5529037640 1 --2- 172.21.15.100:0/806541077 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f550806c3b0 0x7f550806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.455+0000 7f5515ffb640 1 -- 172.21.15.100:0/806541077 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f551c08b280 con 0x7f55240c3b10 2024-09-15T21:48:39.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.456+0000 7f552a039640 1 -- 172.21.15.100:0/806541077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5504005290 con 0x7f55240c3b10 2024-09-15T21:48:39.459 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.456+0000 7f5529037640 1 --2- 172.21.15.100:0/806541077 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f550806c3b0 0x7f550806e870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5518009670 tx=0x7f5518006bd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.468 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.465+0000 7f5515ffb640 1 -- 172.21.15.100:0/806541077 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f551c0592e0 con 0x7f55240c3b10 2024-09-15T21:48:39.491 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.488+0000 7f67f359e640 1 Processor -- start 2024-09-15T21:48:39.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.489+0000 7f67f359e640 1 -- start start 2024-09-15T21:48:39.492 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.489+0000 7f240b853640 1 Processor -- start 2024-09-15T21:48:39.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.490+0000 7f67f359e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67f40c9c30 0x7f67f406e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.490+0000 7f67f359e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f67f40c3b10 0x7f67f406e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.493 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.490+0000 7f67f359e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f67f40c32f0 0x7f67f4154310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.490+0000 7f67f359e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f67f40c6f10 con 0x7f67f40c9c30 2024-09-15T21:48:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.490+0000 7f67f359e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f67f4157f40 con 0x7f67f40c32f0 2024-09-15T21:48:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.490+0000 7f67f359e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f67f40c7160 con 0x7f67f40c3b10 2024-09-15T21:48:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f67f259c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67f40c9c30 0x7f67f406e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f67f259c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f67f40c9c30 0x7f67f406e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54292/0 (socket says 172.21.15.100:54292) 2024-09-15T21:48:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f67f259c640 1 -- 172.21.15.100:0/4248388025 learned_addr learned my addr 172.21.15.100:0/4248388025 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.489+0000 7f240b853640 1 -- start start 2024-09-15T21:48:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f240b853640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2404155130 0x7f24040cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.494 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f240b853640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f240415b830 0x7f24040d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f240b853640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f24040d13d0 0x7f24040cc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f67f2d9d640 1 --2- 172.21.15.100:0/4248388025 >> v2:172.21.15.122:3300/0 conn(0x7f67f40c32f0 0x7f67f4154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f2409df0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f24040d13d0 0x7f24040cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f2409df0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f24040d13d0 0x7f24040cc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54294/0 (socket says 172.21.15.100:54294) 2024-09-15T21:48:39.495 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f2409df0640 1 -- 172.21.15.100:0/2631502604 learned_addr learned my addr 172.21.15.100:0/2631502604 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f67f1d9b640 1 --2- 172.21.15.100:0/4248388025 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f67f40c3b10 0x7f67f406e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f67f259c640 1 -- 172.21.15.100:0/4248388025 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f67f40c3b10 msgr2=0x7f67f406e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f67f259c640 1 --2- 172.21.15.100:0/4248388025 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f67f40c3b10 0x7f67f406e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f67f259c640 1 -- 172.21.15.100:0/4248388025 >> v2:172.21.15.122:3300/0 conn(0x7f67f40c32f0 msgr2=0x7f67f4154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f67f259c640 1 --2- 172.21.15.100:0/4248388025 >> v2:172.21.15.122:3300/0 conn(0x7f67f40c32f0 0x7f67f4154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.496 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f240b853640 1 -- 172.21.15.100:0/2631502604 --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f2404155510 con 0x7f24040d13d0 2024-09-15T21:48:39.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f67f259c640 1 -- 172.21.15.100:0/4248388025 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f40c7440 con 0x7f67f40c9c30 2024-09-15T21:48:39.497 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f67f1d9b640 1 --2- 172.21.15.100:0/4248388025 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f67f40c3b10 0x7f67f406e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:48:39.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f240b853640 1 -- 172.21.15.100:0/2631502604 --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f240415f2a0 con 0x7f240415b830 2024-09-15T21:48:39.498 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f240b853640 1 -- 172.21.15.100:0/2631502604 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f240415e440 con 0x7f2404155130 2024-09-15T21:48:39.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f24095ef640 1 --2- 172.21.15.100:0/2631502604 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2404155130 0x7f24040cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7f2408dee640 1 --2- 172.21.15.100:0/2631502604 >> v2:172.21.15.122:3300/0 conn(0x7f240415b830 0x7f24040d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.492+0000 7f2409df0640 1 -- 172.21.15.100:0/2631502604 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2404155130 msgr2=0x7f24040cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:39.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.492+0000 7f2409df0640 1 --2- 172.21.15.100:0/2631502604 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2404155130 0x7f24040cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.492+0000 7f2409df0640 1 -- 172.21.15.100:0/2631502604 >> v2:172.21.15.122:3300/0 conn(0x7f240415b830 msgr2=0x7f24040d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:39.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.492+0000 7f2409df0640 1 --2- 172.21.15.100:0/2631502604 >> v2:172.21.15.122:3300/0 conn(0x7f240415b830 0x7f24040d0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.492+0000 7f2409df0640 1 -- 172.21.15.100:0/2631502604 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240415e720 con 0x7f24040d13d0 2024-09-15T21:48:39.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.493+0000 7f2409df0640 1 --2- 172.21.15.100:0/2631502604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24040d13d0 0x7f24040cc140 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f240406c000 tx=0x7f23fc02a930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32cd6cfa4e266752 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.499 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.491+0000 7fca17a1e640 1 Processor -- start 2024-09-15T21:48:39.500 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.493+0000 7f23fbfff640 1 -- 172.21.15.100:0/2631502604 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc030420 con 0x7f24040d13d0 2024-09-15T21:48:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.492+0000 7fca17a1e640 1 -- start start 2024-09-15T21:48:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.493+0000 7fca17a1e640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fca101597d0 0x7fca1006c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.493+0000 7fca17a1e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fca100c8770 0x7fca100c7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.493+0000 7fca17a1e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca100c6b30 con 0x7fca100c7550 2024-09-15T21:48:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.493+0000 7fca17a1e640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fca101604d0 con 0x7fca101597d0 2024-09-15T21:48:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.493+0000 7fca17a1e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fca100c8c40 con 0x7fca100c8770 2024-09-15T21:48:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.493+0000 7fca15fbb640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fca100c7550 0x7fca10070ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.100:59398/0 (socket says 172.21.15.100:59398) 2024-09-15T21:48:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.493+0000 7fca15fbb640 1 -- 172.21.15.100:0/1483161939 learned_addr learned my addr 172.21.15.100:0/1483161939 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:39.503 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.493+0000 7fca157ba640 1 --2- 172.21.15.100:0/1483161939 >> v2:172.21.15.100:3300/0 conn(0x7fca101597d0 0x7fca1006c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.493+0000 7f67f259c640 1 --2- 172.21.15.100:0/4248388025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f40c9c30 0x7f67f406e350 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f67f4074200 tx=0x7f67e402a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aca45780804acaec server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.494+0000 7f67f159a640 1 -- 172.21.15.100:0/4248388025 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4037070 con 0x7f67f40c9c30 2024-09-15T21:48:39.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.494+0000 7f67f159a640 1 -- 172.21.15.100:0/4248388025 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67e4032440 con 0x7f67f40c9c30 2024-09-15T21:48:39.504 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.493+0000 7f23fbfff640 1 -- 172.21.15.100:0/2631502604 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23fc035070 con 0x7f24040d13d0 2024-09-15T21:48:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.494+0000 7f67f159a640 1 -- 172.21.15.100:0/4248388025 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e4032810 con 0x7f67f40c9c30 2024-09-15T21:48:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.493+0000 7f23fbfff640 1 -- 172.21.15.100:0/2631502604 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc0307c0 con 0x7f24040d13d0 2024-09-15T21:48:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.494+0000 7fca14fb9640 1 --2- 172.21.15.100:0/1483161939 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fca100c8770 0x7fca100c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.494+0000 7fca07fff640 1 -- 172.21.15.100:0/1483161939 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 293788801 0 0) 0x7fca100c6b30 con 0x7fca100c7550 2024-09-15T21:48:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.494+0000 7fca07fff640 1 -- 172.21.15.100:0/1483161939 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9fc003520 con 0x7fca100c7550 2024-09-15T21:48:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.494+0000 7fca157ba640 1 -- 172.21.15.100:0/1483161939 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fca100c8770 msgr2=0x7fca100c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:39.505 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.494+0000 7fca157ba640 1 --2- 172.21.15.100:0/1483161939 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fca100c8770 0x7fca100c7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.494+0000 7fca157ba640 1 -- 172.21.15.100:0/1483161939 >> v1:172.21.15.122:6789/0 conn(0x7fca100c7550 legacy=0x7fca10070ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.494+0000 7fca157ba640 1 -- 172.21.15.100:0/1483161939 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca1015f710 con 0x7fca101597d0 2024-09-15T21:48:39.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.495+0000 7f67f359e640 1 -- 172.21.15.100:0/4248388025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f40c9c30 msgr2=0x7f67f406e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.506 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.495+0000 7f67f359e640 1 --2- 172.21.15.100:0/4248388025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f40c9c30 0x7f67f406e350 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f67f4074200 tx=0x7f67e402a1d0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.495+0000 7f240b853640 1 -- 172.21.15.100:0/2631502604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24040d13d0 msgr2=0x7f24040cc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.495+0000 7f240b853640 1 --2- 172.21.15.100:0/2631502604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24040d13d0 0x7f24040cc140 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f240406c000 tx=0x7f23fc02a930 comp rx=0 tx=0).stop 2024-09-15T21:48:39.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.495+0000 7f240b853640 1 -- 172.21.15.100:0/2631502604 shutdown_connections 2024-09-15T21:48:39.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.495+0000 7f67f359e640 1 -- 172.21.15.100:0/4248388025 shutdown_connections 2024-09-15T21:48:39.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.495+0000 7f67f359e640 1 --2- 172.21.15.100:0/4248388025 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f67f40c3b10 0x7f67f406e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.507 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.495+0000 7f67f359e640 1 --2- 172.21.15.100:0/4248388025 >> v2:172.21.15.122:3300/0 conn(0x7f67f40c32f0 0x7f67f4154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.495+0000 7f67f359e640 1 --2- 172.21.15.100:0/4248388025 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f40c9c30 0x7f67f406e350 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.495+0000 7f67f359e640 1 -- 172.21.15.100:0/4248388025 >> 172.21.15.100:0/4248388025 conn(0x7f67f40c9110 msgr2=0x7f67f40c9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:39.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.495+0000 7f67f359e640 1 -- 172.21.15.100:0/4248388025 shutdown_connections 2024-09-15T21:48:39.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.495+0000 7f67f359e640 1 -- 172.21.15.100:0/4248388025 wait complete. 2024-09-15T21:48:39.508 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.495+0000 7f240b853640 1 --2- 172.21.15.100:0/2631502604 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24040d13d0 0x7f24040cc140 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.495+0000 7f240b853640 1 --2- 172.21.15.100:0/2631502604 >> v2:172.21.15.122:3300/0 conn(0x7f240415b830 0x7f24040d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.495+0000 7f240b853640 1 --2- 172.21.15.100:0/2631502604 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2404155130 0x7f24040cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.495+0000 7f240b853640 1 -- 172.21.15.100:0/2631502604 >> 172.21.15.100:0/2631502604 conn(0x7f2404067dd0 msgr2=0x7f24040681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:39.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.496+0000 7f240b853640 1 -- 172.21.15.100:0/2631502604 shutdown_connections 2024-09-15T21:48:39.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.496+0000 7fca157ba640 1 --2- 172.21.15.100:0/1483161939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca101597d0 0x7fca1006c450 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0097a0 tx=0x7fca0c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2b45615d066710 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.509 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.496+0000 7f240b853640 1 -- 172.21.15.100:0/2631502604 wait complete. 2024-09-15T21:48:39.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.496+0000 7fca07fff640 1 -- 172.21.15.100:0/1483161939 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c037070 con 0x7fca101597d0 2024-09-15T21:48:39.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.496+0000 7fca07fff640 1 -- 172.21.15.100:0/1483161939 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca0c032440 con 0x7fca101597d0 2024-09-15T21:48:39.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.496+0000 7fca07fff640 1 -- 172.21.15.100:0/1483161939 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c032810 con 0x7fca101597d0 2024-09-15T21:48:39.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.496+0000 7f8e0f047640 1 Processor -- start 2024-09-15T21:48:39.510 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.497+0000 7f8e0f047640 1 -- start start 2024-09-15T21:48:39.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.497+0000 7f67f359e640 1 Processor -- start 2024-09-15T21:48:39.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.497+0000 7f67f359e640 1 -- start start 2024-09-15T21:48:39.511 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.497+0000 7f240b853640 1 Processor -- start 2024-09-15T21:48:39.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.498+0000 7fca17a1e640 1 -- 172.21.15.100:0/1483161939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca101597d0 msgr2=0x7fca1006c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.498+0000 7fca17a1e640 1 --2- 172.21.15.100:0/1483161939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca101597d0 0x7fca1006c450 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fca0c0097a0 tx=0x7fca0c02a210 comp rx=0 tx=0).stop 2024-09-15T21:48:39.512 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.498+0000 7f240b853640 1 -- start start 2024-09-15T21:48:39.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.498+0000 7f8e0f047640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9f00 0x7f8e08162a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.498+0000 7f8e0f047640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8e080c7a00 0x7f8e0815ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.513 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.498+0000 7f8e0f047640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e0815f4e0 con 0x7f8e080ca2e0 2024-09-15T21:48:39.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.498+0000 7f8e0f047640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f8e0815fb10 con 0x7f8e080c7a00 2024-09-15T21:48:39.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.498+0000 7f8e0f047640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8e080c7f60 con 0x7f8e080c9f00 2024-09-15T21:48:39.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.498+0000 7f8e0d5e4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f8e080c7a00 0x7f8e0815ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f8e0cde3640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9f00 0x7f8e08162a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f67f359e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f67f40c32f0 0x7f67f40c0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f67f359e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f40c3b10 0x7f67f4156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f67f359e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67f40c9c30 0x7f67f4073110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f8e0cde3640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9f00 0x7f8e08162a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:55690/0 (socket says 172.21.15.100:55690) 2024-09-15T21:48:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f8e0cde3640 1 -- 172.21.15.100:0/345143557 learned_addr learned my addr 172.21.15.100:0/345143557 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7fca17a1e640 1 -- 172.21.15.100:0/1483161939 shutdown_connections 2024-09-15T21:48:39.515 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7fca17a1e640 1 --2- 172.21.15.100:0/1483161939 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fca100c8770 0x7fca100c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7fca17a1e640 1 --2- 172.21.15.100:0/1483161939 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca101597d0 0x7fca1006c450 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7fca17a1e640 1 -- 172.21.15.100:0/1483161939 >> 172.21.15.100:0/1483161939 conn(0x7fca10067dd0 msgr2=0x7fca1006bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f67f359e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f67f41549d0 con 0x7f67f40c3b10 2024-09-15T21:48:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f67f359e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f67f4154850 con 0x7f67f40c9c30 2024-09-15T21:48:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f67f359e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f67f4154b50 con 0x7f67f40c32f0 2024-09-15T21:48:39.516 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f67f259c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f67f40c32f0 0x7f67f40c0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7fca17a1e640 1 -- 172.21.15.100:0/1483161939 shutdown_connections 2024-09-15T21:48:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7fca17a1e640 1 -- 172.21.15.100:0/1483161939 wait complete. 2024-09-15T21:48:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f67f1d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f40c3b10 0x7f67f4156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.517 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f67f1d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f40c3b10 0x7f67f4156640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54318/0 (socket says 172.21.15.100:54318) 2024-09-15T21:48:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f67f1d9b640 1 -- 172.21.15.100:0/3043466638 learned_addr learned my addr 172.21.15.100:0/3043466638 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f67f2d9d640 1 --2- 172.21.15.100:0/3043466638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67f40c9c30 0x7f67f4073110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.518 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f240b853640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24040d13d0 0x7f24040cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f240b853640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2404155130 0x7f24040c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f240b853640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f240415b830 0x7f24040ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f240b853640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f240415efc0 con 0x7f240415b830 2024-09-15T21:48:39.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f240b853640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f240415ece0 con 0x7f24040d13d0 2024-09-15T21:48:39.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f240b853640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f240415bc30 con 0x7f2404155130 2024-09-15T21:48:39.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.500+0000 7f24095ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24040d13d0 0x7f24040cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f8e0d5e4640 1 -- 172.21.15.100:0/345143557 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9f00 msgr2=0x7f8e08162a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:39.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f8e0d5e4640 1 --2- 172.21.15.100:0/345143557 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9f00 0x7f8e08162a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.519 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.499+0000 7f8e0d5e4640 1 -- 172.21.15.100:0/345143557 >> v1:172.21.15.100:6789/0 conn(0x7f8e080ca2e0 legacy=0x7f8e08148220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:39.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.500+0000 7f24095ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24040d13d0 0x7f24040cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:43002/0 (socket says 172.21.15.100:43002) 2024-09-15T21:48:39.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.500+0000 7f24095ef640 1 -- 172.21.15.100:0/899488568 learned_addr learned my addr 172.21.15.100:0/899488568 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:39.520 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.500+0000 7f67f1d9b640 1 -- 172.21.15.100:0/3043466638 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f67f40c32f0 msgr2=0x7f67f40c0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.500+0000 7f2409df0640 1 --2- 172.21.15.100:0/899488568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f240415b830 0x7f24040ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.500+0000 7f2408dee640 1 --2- 172.21.15.100:0/899488568 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2404155130 0x7f24040c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.521 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.500+0000 7f8e0d5e4640 1 -- 172.21.15.100:0/345143557 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e080c8240 con 0x7f8e080c7a00 2024-09-15T21:48:39.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.500+0000 7f2409df0640 1 -- 172.21.15.100:0/899488568 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2404155130 msgr2=0x7f24040c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.500+0000 7f2409df0640 1 --2- 172.21.15.100:0/899488568 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2404155130 0x7f24040c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.500+0000 7f2409df0640 1 -- 172.21.15.100:0/899488568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24040d13d0 msgr2=0x7f24040cf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.500+0000 7f2409df0640 1 --2- 172.21.15.100:0/899488568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24040d13d0 0x7f24040cf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.500+0000 7f2409df0640 1 -- 172.21.15.100:0/899488568 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240415df50 con 0x7f240415b830 2024-09-15T21:48:39.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.500+0000 7f8a44d66640 1 Processor -- start 2024-09-15T21:48:39.522 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.500+0000 7f8a44d66640 1 -- start start 2024-09-15T21:48:39.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.500+0000 7f67f1d9b640 1 --2- 172.21.15.100:0/3043466638 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f67f40c32f0 0x7f67f40c0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.500+0000 7f67f1d9b640 1 -- 172.21.15.100:0/3043466638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67f40c9c30 msgr2=0x7f67f4073110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.500+0000 7f67f1d9b640 1 --2- 172.21.15.100:0/3043466638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67f40c9c30 0x7f67f4073110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.500+0000 7f67f1d9b640 1 -- 172.21.15.100:0/3043466638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f4071460 con 0x7f67f40c3b10 2024-09-15T21:48:39.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.501+0000 7fca17a1e640 1 Processor -- start 2024-09-15T21:48:39.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.501+0000 7f8e0d5e4640 1 --2- 172.21.15.100:0/345143557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e080c7a00 0x7f8e0815ffa0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000b050 tx=0x7f8e0002a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e8670429e6da18 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.501+0000 7f67f1d9b640 1 --2- 172.21.15.100:0/3043466638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f40c3b10 0x7f67f4156640 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00cf90 tx=0x7f67ec00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.501+0000 7f2409df0640 1 --2- 172.21.15.100:0/899488568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f240415b830 0x7f24040ca250 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f23fc031000 tx=0x7f23fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.524 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.501+0000 7f67e2ffd640 1 -- 172.21.15.100:0/3043466638 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec01a070 con 0x7f67f40c3b10 2024-09-15T21:48:39.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7f67f359e640 1 -- 172.21.15.100:0/3043466638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f4070f40 con 0x7f67f40c3b10 2024-09-15T21:48:39.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7f67f359e640 1 -- 172.21.15.100:0/3043466638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f40c2fc0 con 0x7f67f40c3b10 2024-09-15T21:48:39.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7f67e2ffd640 1 -- 172.21.15.100:0/3043466638 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67ec00d840 con 0x7f67f40c3b10 2024-09-15T21:48:39.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7f67e2ffd640 1 -- 172.21.15.100:0/3043466638 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ec024060 con 0x7f67f40c3b10 2024-09-15T21:48:39.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7f8e077fe640 1 -- 172.21.15.100:0/345143557 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00032620 con 0x7f8e080c7a00 2024-09-15T21:48:39.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7f8e077fe640 1 -- 172.21.15.100:0/345143557 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e000327c0 con 0x7f8e080c7a00 2024-09-15T21:48:39.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7f8e077fe640 1 -- 172.21.15.100:0/345143557 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00032b90 con 0x7f8e080c7a00 2024-09-15T21:48:39.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.501+0000 7fca17a1e640 1 -- start start 2024-09-15T21:48:39.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7fca17a1e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca100c7550 0x7fca1014b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7fca17a1e640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fca100c8770 0x7fca10143950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7fca17a1e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca101597d0 0x7fca101482a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7fca17a1e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fca1015ffb0 con 0x7fca101597d0 2024-09-15T21:48:39.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7fca17a1e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fca1015fcd0 con 0x7fca100c7550 2024-09-15T21:48:39.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7fca17a1e640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fca100c9a90 con 0x7fca100c8770 2024-09-15T21:48:39.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7f8a44d66640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8a4006e2e0 0x7f8a400ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7f8a44d66640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a4006d820 con 0x7f8a401598a0 2024-09-15T21:48:39.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7f8a44d66640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a40163450 con 0x7f8a400c5bd0 2024-09-15T21:48:39.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7f8a44d66640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8a40163730 con 0x7f8a4006e2e0 2024-09-15T21:48:39.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7fca157ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca100c7550 0x7fca1014b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.527 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7fca157ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca100c7550 0x7fca1014b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:43010/0 (socket says 172.21.15.100:43010) 2024-09-15T21:48:39.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7fca157ba640 1 -- 172.21.15.100:0/4262878143 learned_addr learned my addr 172.21.15.100:0/4262878143 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:39.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7f8a3dd9b640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f8a400c5bd0 0x7f8a400d0e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.100:59414/0 (socket says 172.21.15.100:59414) 2024-09-15T21:48:39.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7f8a3dd9b640 1 -- 172.21.15.100:0/3976034704 learned_addr learned my addr 172.21.15.100:0/3976034704 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:39.528 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7f1813fff640 1 Processor -- start 2024-09-15T21:48:39.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.502+0000 7fca14fb9640 1 --2- 172.21.15.100:0/4262878143 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fca100c8770 0x7fca10143950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7fca157ba640 1 -- 172.21.15.100:0/4262878143 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fca100c8770 msgr2=0x7fca10143950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7fca157ba640 1 --2- 172.21.15.100:0/4262878143 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fca100c8770 0x7fca10143950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7fca157ba640 1 -- 172.21.15.100:0/4262878143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca101597d0 msgr2=0x7fca101482a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7fca157ba640 1 --2- 172.21.15.100:0/4262878143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca101597d0 0x7fca101482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7fca157ba640 1 -- 172.21.15.100:0/4262878143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca10142e90 con 0x7fca100c7550 2024-09-15T21:48:39.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7f8e0f047640 1 -- 172.21.15.100:0/345143557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e080c7a00 msgr2=0x7f8e0815ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.529 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7f8e0f047640 1 --2- 172.21.15.100:0/345143557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e080c7a00 0x7f8e0815ffa0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000b050 tx=0x7f8e0002a870 comp rx=0 tx=0).stop 2024-09-15T21:48:39.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7f8a3ed9d640 1 --2- 172.21.15.100:0/3976034704 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8a4006e2e0 0x7f8a400ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7f23f9ffb640 1 -- 172.21.15.100:0/899488568 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc004240 con 0x7f240415b830 2024-09-15T21:48:39.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7f23f9ffb640 1 -- 172.21.15.100:0/899488568 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23fc03e070 con 0x7f240415b830 2024-09-15T21:48:39.530 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7fca157ba640 1 --2- 172.21.15.100:0/4262878143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca100c7550 0x7fca1014b2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009890 tx=0x7fca0c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7f8a3d59a640 1 -- 172.21.15.100:0/3976034704 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3388735887 0 0) 0x7f8a40163450 con 0x7f8a400c5bd0 2024-09-15T21:48:39.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7f8a3d59a640 1 -- 172.21.15.100:0/3976034704 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a28003520 con 0x7f8a400c5bd0 2024-09-15T21:48:39.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7f8e0f047640 1 -- 172.21.15.100:0/345143557 shutdown_connections 2024-09-15T21:48:39.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7f8e0f047640 1 --2- 172.21.15.100:0/345143557 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e080c7a00 0x7f8e0815ffa0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7f8e0f047640 1 --2- 172.21.15.100:0/345143557 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9f00 0x7f8e08162a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.531 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7f8e0f047640 1 -- 172.21.15.100:0/345143557 >> 172.21.15.100:0/345143557 conn(0x7f8e08073820 msgr2=0x7f8e08073c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:39.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7fca05ffb640 1 -- 172.21.15.100:0/4262878143 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c037050 con 0x7fca100c7550 2024-09-15T21:48:39.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7f8e0f047640 1 -- 172.21.15.100:0/345143557 shutdown_connections 2024-09-15T21:48:39.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.504+0000 7fca05ffb640 1 -- 172.21.15.100:0/4262878143 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca0c043070 con 0x7fca100c7550 2024-09-15T21:48:39.532 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.504+0000 7fca05ffb640 1 -- 172.21.15.100:0/4262878143 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca0c03cb30 con 0x7fca100c7550 2024-09-15T21:48:39.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7f8a3ed9d640 1 -- 172.21.15.100:0/3976034704 >> v1:172.21.15.122:6789/0 conn(0x7f8a400c5bd0 legacy=0x7f8a400d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7f8a3ed9d640 1 -- 172.21.15.100:0/3976034704 >> v1:172.21.15.100:6789/0 conn(0x7f8a401598a0 legacy=0x7f8a40162bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:39.533 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.504+0000 7f8a3ed9d640 1 -- 172.21.15.100:0/3976034704 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a40163a10 con 0x7f8a4006e2e0 2024-09-15T21:48:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.504+0000 7f23f9ffb640 1 -- 172.21.15.100:0/899488568 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23fc004660 con 0x7f240415b830 2024-09-15T21:48:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.503+0000 7f1813fff640 1 -- start start 2024-09-15T21:48:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.504+0000 7f1813fff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18141597d0 0x7f1814072290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.504+0000 7f1813fff640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f181406cd50 con 0x7f18140c0170 2024-09-15T21:48:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.504+0000 7f1813fff640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1814072c60 con 0x7f18140727d0 2024-09-15T21:48:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.504+0000 7f1813fff640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f181406c790 con 0x7f18141597d0 2024-09-15T21:48:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.504+0000 7f240b853640 1 -- 172.21.15.100:0/899488568 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f240415c220 con 0x7f240415b830 2024-09-15T21:48:39.534 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.504+0000 7f240b853640 1 -- 172.21.15.100:0/899488568 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24040cff00 con 0x7f240415b830 2024-09-15T21:48:39.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.504+0000 7fca17a1e640 1 -- 172.21.15.100:0/4262878143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca101459b0 con 0x7fca100c7550 2024-09-15T21:48:39.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.505+0000 7f18137fe640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f18140727d0 0x7f1814163200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.100:59416/0 (socket says 172.21.15.100:59416) 2024-09-15T21:48:39.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.505+0000 7f18137fe640 1 -- 172.21.15.100:0/1932117412 learned_addr learned my addr 172.21.15.100:0/1932117412 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:39.535 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.504+0000 7f8e0f047640 1 -- 172.21.15.100:0/345143557 wait complete. 2024-09-15T21:48:39.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.505+0000 7f1811ffb640 1 -- 172.21.15.100:0/1932117412 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3644641482 0 0) 0x7f1814072c60 con 0x7f18140727d0 2024-09-15T21:48:39.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.505+0000 7f67f359e640 1 -- 172.21.15.100:0/3043466638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d8005290 con 0x7f67f40c3b10 2024-09-15T21:48:39.536 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.505+0000 7f1811ffb640 1 -- 172.21.15.100:0/1932117412 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f17fc003520 con 0x7f18140727d0 2024-09-15T21:48:39.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.505+0000 7f8e0f047640 1 Processor -- start 2024-09-15T21:48:39.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.504+0000 7fca17a1e640 1 -- 172.21.15.100:0/4262878143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca10143150 con 0x7fca100c7550 2024-09-15T21:48:39.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.505+0000 7f67e2ffd640 1 -- 172.21.15.100:0/3043466638 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f67ec01d070 con 0x7f67f40c3b10 2024-09-15T21:48:39.537 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.506+0000 7f93de586640 1 Processor -- start 2024-09-15T21:48:39.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.506+0000 7f93de586640 1 -- start start 2024-09-15T21:48:39.538 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.506+0000 7f8a3ed9d640 1 --2- 172.21.15.100:0/3976034704 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8a4006e2e0 0x7f8a400ccad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009dc0 tx=0x7f8a3802a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cd3aaafa588ccb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.505+0000 7f1812ffd640 1 --2- 172.21.15.100:0/1932117412 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18141597d0 0x7f1814072290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.506+0000 7f67e2ffd640 1 --2- 172.21.15.100:0/3043466638 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f67d406c2e0 0x7f67d406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.506+0000 7fca17a1e640 1 -- 172.21.15.100:0/4262878143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca10159bb0 con 0x7fca100c7550 2024-09-15T21:48:39.539 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.506+0000 7f240b853640 1 -- 172.21.15.100:0/899488568 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f240406d0e0 con 0x7f240415b830 2024-09-15T21:48:39.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.506+0000 7f1811ffb640 1 -- 172.21.15.100:0/1932117412 <== mon.1 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1797558168 0 0) 0x7f17fc003520 con 0x7f18140727d0 2024-09-15T21:48:39.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.506+0000 7f1811ffb640 1 -- 172.21.15.100:0/1932117412 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18141597d0 msgr2=0x7f1814072290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:39.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.506+0000 7f1811ffb640 1 --2- 172.21.15.100:0/1932117412 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18141597d0 0x7f1814072290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.506+0000 7f1811ffb640 1 -- 172.21.15.100:0/1932117412 >> v1:172.21.15.100:6789/0 conn(0x7f18140c0170 legacy=0x7f1814162af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:39.540 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.506+0000 7f67e2ffd640 1 -- 172.21.15.100:0/3043466638 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f67ec08bd70 con 0x7f67f40c3b10 2024-09-15T21:48:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.506+0000 7f1811ffb640 1 -- 172.21.15.100:0/1932117412 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181406ca70 con 0x7f18140727d0 2024-09-15T21:48:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.506+0000 7f8a3d59a640 1 -- 172.21.15.100:0/3976034704 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a38007740 con 0x7f8a4006e2e0 2024-09-15T21:48:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.506+0000 7f8a3d59a640 1 -- 172.21.15.100:0/3976034704 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a380078e0 con 0x7f8a4006e2e0 2024-09-15T21:48:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.506+0000 7f8a3d59a640 1 -- 172.21.15.100:0/3976034704 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a38007cf0 con 0x7f8a4006e2e0 2024-09-15T21:48:39.541 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.506+0000 7f1811ffb640 1 -- 172.21.15.100:0/1932117412 <== mon.1 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7f180c0019f0 con 0x7f18140727d0 2024-09-15T21:48:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f1811ffb640 1 -- 172.21.15.100:0/1932117412 >> v1:172.21.15.122:6789/0 conn(0x7f18140727d0 legacy=0x7f1814163200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f1811ffb640 1 --2- 172.21.15.100:0/1932117412 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f17fc003a20 0x7f17fc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f1811ffb640 1 --2- 172.21.15.100:0/1932117412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17fc004800 0x7f17fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f1811ffb640 1 --2- 172.21.15.100:0/1932117412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17fc0051c0 0x7f17fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f1811ffb640 1 -- 172.21.15.100:0/1932117412 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f181406c790 con 0x7f17fc0051c0 2024-09-15T21:48:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f1811ffb640 1 -- 172.21.15.100:0/1932117412 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f17fc00c040 con 0x7f17fc004800 2024-09-15T21:48:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f1811ffb640 1 -- 172.21.15.100:0/1932117412 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f17fc00c240 con 0x7f17fc003a20 2024-09-15T21:48:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f18127fc640 1 --2- 172.21.15.100:0/1932117412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17fc004800 0x7f17fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f93de586640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93d80c8a60 0x7f93d8160110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f93de586640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f93d80c8e40 0x7f93d80c2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f18137fe640 1 --2- 172.21.15.100:0/1932117412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17fc0051c0 0x7f17fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f93de586640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93d80c1f30 con 0x7f93d80b27a0 2024-09-15T21:48:39.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f93de586640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f93d80c2b00 con 0x7f93d80c8e40 2024-09-15T21:48:39.543 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7fca05ffb640 1 -- 172.21.15.100:0/4262878143 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fca0c03ccd0 con 0x7fca100c7550 2024-09-15T21:48:39.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f18127fc640 1 -- 172.21.15.100:0/1932117412 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f17fc003a20 msgr2=0x7f17fc003e20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-15T21:48:39.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f18127fc640 1 --2- 172.21.15.100:0/1932117412 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f17fc003a20 0x7f17fc003e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f18127fc640 1 -- 172.21.15.100:0/1932117412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17fc0051c0 msgr2=0x7f17fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:39.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f18127fc640 1 --2- 172.21.15.100:0/1932117412 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17fc0051c0 0x7f17fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f18127fc640 1 -- 172.21.15.100:0/1932117412 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17fc003520 con 0x7f17fc004800 2024-09-15T21:48:39.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f93de586640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f93d80c21d0 con 0x7f93d80c8a60 2024-09-15T21:48:39.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f8e0f047640 1 -- start start 2024-09-15T21:48:39.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7fca05ffb640 1 --2- 172.21.15.100:0/4262878143 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc9f406c3b0 0x7fc9f406e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f8a44d66640 1 -- 172.21.15.100:0/3976034704 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8a4006e2e0 msgr2=0x7f8a400ccad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f8a44d66640 1 --2- 172.21.15.100:0/3976034704 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8a4006e2e0 0x7f8a400ccad0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009dc0 tx=0x7f8a3802a6f0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7f8a44d66640 1 -- 172.21.15.100:0/3976034704 shutdown_connections 2024-09-15T21:48:39.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7f8a44d66640 1 --2- 172.21.15.100:0/3976034704 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8a4006e2e0 0x7f8a400ccad0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7f8a44d66640 1 -- 172.21.15.100:0/3976034704 >> 172.21.15.100:0/3976034704 conn(0x7f8a40067980 msgr2=0x7f8a40067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:39.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f93dcb23640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f93d80b27a0 0x7f93d8159420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.100:59432/0 (socket says 172.21.15.100:59432) 2024-09-15T21:48:39.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.507+0000 7f93dcb23640 1 -- 172.21.15.100:0/3431477320 learned_addr learned my addr 172.21.15.100:0/3431477320 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:39.546 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7f93d7fff640 1 --2- 172.21.15.100:0/3431477320 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93d80c8a60 0x7f93d8160110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7f8e0f047640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e080c7a00 0x7f8e0815ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7f8e0f047640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9f00 0x7f8e08161900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7f8e0f047640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e080ca2e0 0x7f8e08146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7f8e0f047640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8e08148930 con 0x7f8e080ca2e0 2024-09-15T21:48:39.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7f8e0f047640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0815f7a0 con 0x7f8e080c7a00 2024-09-15T21:48:39.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7f8e0f047640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8e08148ab0 con 0x7f8e080c9f00 2024-09-15T21:48:39.547 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7fca05ffb640 1 -- 172.21.15.100:0/4262878143 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7fca0c0ac030 con 0x7fca100c7550 2024-09-15T21:48:39.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7f8e07fff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9f00 0x7f8e08161900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7f8e0cde3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e080c7a00 0x7f8e0815ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7f8e07fff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9f00 0x7f8e08161900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:55750/0 (socket says 172.21.15.100:55750) 2024-09-15T21:48:39.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.509+0000 7f8e0cde3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e080c7a00 0x7f8e0815ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:43036/0 (socket says 172.21.15.100:43036) 2024-09-15T21:48:39.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.509+0000 7f8e0cde3640 1 -- 172.21.15.100:0/2317809547 learned_addr learned my addr 172.21.15.100:0/2317809547 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:39.548 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7fca14fb9640 1 --2- 172.21.15.100:0/4262878143 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc9f406c3b0 0x7fc9f406e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.509+0000 7f8e0d5e4640 1 --2- 172.21.15.100:0/2317809547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e080ca2e0 0x7f8e08146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7f67f259c640 1 --2- 172.21.15.100:0/3043466638 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f67d406c2e0 0x7f67d406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.549 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.509+0000 7f8a44d66640 1 -- 172.21.15.100:0/3976034704 shutdown_connections 2024-09-15T21:48:39.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7f93d77fe640 1 --2- 172.21.15.100:0/3431477320 >> v2:172.21.15.100:3300/0 conn(0x7f93d80c8e40 0x7f93d80c2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7f93d7fff640 1 -- 172.21.15.100:0/3431477320 >> v2:172.21.15.100:3300/0 conn(0x7f93d80c8e40 msgr2=0x7f93d80c2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:39.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7f93d7fff640 1 --2- 172.21.15.100:0/3431477320 >> v2:172.21.15.100:3300/0 conn(0x7f93d80c8e40 0x7f93d80c2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7f93d7fff640 1 -- 172.21.15.100:0/3431477320 >> v1:172.21.15.122:6789/0 conn(0x7f93d80b27a0 legacy=0x7f93d8159420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.508+0000 7f93d7fff640 1 -- 172.21.15.100:0/3431477320 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d80c24b0 con 0x7f93d80c8a60 2024-09-15T21:48:39.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.509+0000 7f18127fc640 1 --2- 172.21.15.100:0/1932117412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17fc004800 0x7f17fc004c80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1808003d80 tx=0x7f17fc005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bb67cca249f032 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.550 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.509+0000 7f8a44d66640 1 -- 172.21.15.100:0/3976034704 wait complete. 2024-09-15T21:48:39.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.509+0000 7f8e0cde3640 1 -- 172.21.15.100:0/2317809547 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9f00 msgr2=0x7f8e08161900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.509+0000 7f8e0cde3640 1 --2- 172.21.15.100:0/2317809547 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9f00 0x7f8e08161900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.509+0000 7f8e0cde3640 1 -- 172.21.15.100:0/2317809547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e080ca2e0 msgr2=0x7f8e08146ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.509+0000 7f8e0cde3640 1 --2- 172.21.15.100:0/2317809547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e080ca2e0 0x7f8e08146ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.551 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.509+0000 7f8e0cde3640 1 -- 172.21.15.100:0/2317809547 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0815bf30 con 0x7f8e080c7a00 2024-09-15T21:48:39.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.509+0000 7f67f259c640 1 --2- 172.21.15.100:0/3043466638 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f67d406c2e0 0x7f67d406e7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f67e4034000 tx=0x7f67e402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.509+0000 7f8e0cde3640 1 --2- 172.21.15.100:0/2317809547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e080c7a00 0x7f8e0815ab80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00e7e0 tx=0x7f8dfc00eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.509+0000 7f23f9ffb640 1 -- 172.21.15.100:0/899488568 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f23fc043470 con 0x7f240415b830 2024-09-15T21:48:39.552 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.510+0000 7fca14fb9640 1 --2- 172.21.15.100:0/4262878143 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc9f406c3b0 0x7fc9f406e870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fca1014b0b0 tx=0x7fca00009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.510+0000 7f93d7fff640 1 --2- 172.21.15.100:0/3431477320 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93d80c8a60 0x7f93d8160110 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f93c80097a0 tx=0x7f93c802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d19d193dfb612c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.510+0000 7f23f9ffb640 1 --2- 172.21.15.100:0/899488568 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f23e806c380 0x7f23e806e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.510+0000 7f23f9ffb640 1 -- 172.21.15.100:0/899488568 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f23fc0ac300 con 0x7f240415b830 2024-09-15T21:48:39.553 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.510+0000 7f24095ef640 1 --2- 172.21.15.100:0/899488568 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f23e806c380 0x7f23e806e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.510+0000 7f93d6ffd640 1 -- 172.21.15.100:0/3431477320 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8037070 con 0x7f93d80c8a60 2024-09-15T21:48:39.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.510+0000 7f93d6ffd640 1 -- 172.21.15.100:0/3431477320 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93c8032440 con 0x7f93d80c8a60 2024-09-15T21:48:39.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.510+0000 7f93d6ffd640 1 -- 172.21.15.100:0/3431477320 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93c8032810 con 0x7f93d80c8a60 2024-09-15T21:48:39.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.510+0000 7f24095ef640 1 --2- 172.21.15.100:0/899488568 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f23e806c380 0x7f23e806e840 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f23f4006fd0 tx=0x7f23f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.510+0000 7f8a44d66640 1 Processor -- start 2024-09-15T21:48:39.554 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f8a44d66640 1 -- start start 2024-09-15T21:48:39.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f93de586640 1 -- 172.21.15.100:0/3431477320 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93d80c8a60 msgr2=0x7f93d8160110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f93de586640 1 --2- 172.21.15.100:0/3431477320 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93d80c8a60 0x7f93d8160110 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f93c80097a0 tx=0x7f93c802a210 comp rx=0 tx=0).stop 2024-09-15T21:48:39.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f93de586640 1 -- 172.21.15.100:0/3431477320 shutdown_connections 2024-09-15T21:48:39.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f93de586640 1 --2- 172.21.15.100:0/3431477320 >> v2:172.21.15.100:3300/0 conn(0x7f93d80c8e40 0x7f93d80c2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f93de586640 1 --2- 172.21.15.100:0/3431477320 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93d80c8a60 0x7f93d8160110 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.555 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f93de586640 1 -- 172.21.15.100:0/3431477320 >> 172.21.15.100:0/3431477320 conn(0x7f93d806ed20 msgr2=0x7f93d806f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:39.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f93de586640 1 -- 172.21.15.100:0/3431477320 shutdown_connections 2024-09-15T21:48:39.556 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f93de586640 1 -- 172.21.15.100:0/3431477320 wait complete. 2024-09-15T21:48:39.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f8a44d66640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8a4006e2e0 0x7f8a400cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f8a44d66640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a400c5bd0 0x7f8a4006ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f8a44d66640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a401598a0 0x7f8a401f5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f8a44d66640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f8a40159c80 con 0x7f8a401598a0 2024-09-15T21:48:39.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f8a44d66640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4006e720 con 0x7f8a400c5bd0 2024-09-15T21:48:39.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f8a44d66640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f8a40159e00 con 0x7f8a4006e2e0 2024-09-15T21:48:39.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f8a3ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a401598a0 0x7f8a401f5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f8a3dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a400c5bd0 0x7f8a4006ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f8a3dd9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a400c5bd0 0x7f8a4006ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:43038/0 (socket says 172.21.15.100:43038) 2024-09-15T21:48:39.557 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f8a3ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a401598a0 0x7f8a401f5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54358/0 (socket says 172.21.15.100:54358) 2024-09-15T21:48:39.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.511+0000 7f8a3dd9b640 1 -- 172.21.15.100:0/1185695049 learned_addr learned my addr 172.21.15.100:0/1185695049 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:39.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.512+0000 7f8a3e59c640 1 --2- 172.21.15.100:0/1185695049 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8a4006e2e0 0x7f8a400cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.512+0000 7f8a3ed9d640 1 -- 172.21.15.100:0/1185695049 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8a4006e2e0 msgr2=0x7f8a400cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.512+0000 7f8a3ed9d640 1 --2- 172.21.15.100:0/1185695049 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8a4006e2e0 0x7f8a400cbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.512+0000 7f8a3ed9d640 1 -- 172.21.15.100:0/1185695049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a400c5bd0 msgr2=0x7f8a4006ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.512+0000 7f8a3ed9d640 1 --2- 172.21.15.100:0/1185695049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a400c5bd0 0x7f8a4006ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.512+0000 7f8a3ed9d640 1 -- 172.21.15.100:0/1185695049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a401f8ad0 con 0x7f8a401598a0 2024-09-15T21:48:39.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.512+0000 7f8a3dd9b640 1 --2- 172.21.15.100:0/1185695049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a400c5bd0 0x7f8a4006ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:48:39.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.512+0000 7f8a3ed9d640 1 --2- 172.21.15.100:0/1185695049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a401598a0 0x7f8a401f5af0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8a38031000 tx=0x7f8a38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.558 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.512+0000 7f1811ffb640 1 -- 172.21.15.100:0/1932117412 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808037070 con 0x7f17fc004800 2024-09-15T21:48:39.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.512+0000 7f1811ffb640 1 -- 172.21.15.100:0/1932117412 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f180802aa20 con 0x7f17fc004800 2024-09-15T21:48:39.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.512+0000 7f93de586640 1 Processor -- start 2024-09-15T21:48:39.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.512+0000 7f1811ffb640 1 -- 172.21.15.100:0/1932117412 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1808003980 con 0x7f17fc004800 2024-09-15T21:48:39.559 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.512+0000 7f8e057fa640 1 -- 172.21.15.100:0/2317809547 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc017070 con 0x7f8e080c7a00 2024-09-15T21:48:39.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.512+0000 7f8e057fa640 1 -- 172.21.15.100:0/2317809547 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dfc00c8d0 con 0x7f8e080c7a00 2024-09-15T21:48:39.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.512+0000 7f8e0f047640 1 -- 172.21.15.100:0/2317809547 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e081f5670 con 0x7f8e080c7a00 2024-09-15T21:48:39.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.512+0000 7fca05ffb640 1 -- 172.21.15.100:0/4262878143 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fca0c07a6e0 con 0x7fca100c7550 2024-09-15T21:48:39.560 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.513+0000 7f93de586640 1 -- start start 2024-09-15T21:48:39.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.513+0000 7f8e0f047640 1 -- 172.21.15.100:0/2317809547 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0815c4b0 con 0x7f8e080c7a00 2024-09-15T21:48:39.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.513+0000 7f1813fff640 1 -- 172.21.15.100:0/1932117412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17fc004800 msgr2=0x7f17fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.513+0000 7f1813fff640 1 --2- 172.21.15.100:0/1932117412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17fc004800 0x7f17fc004c80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1808003d80 tx=0x7f17fc005830 comp rx=0 tx=0).stop 2024-09-15T21:48:39.561 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.513+0000 7f18137fe640 1 -- 172.21.15.100:0/1932117412 reap_dead start 2024-09-15T21:48:39.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.512+0000 7f8e057fa640 1 -- 172.21.15.100:0/2317809547 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc00f040 con 0x7f8e080c7a00 2024-09-15T21:48:39.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.514+0000 7f1813fff640 1 -- 172.21.15.100:0/1932117412 shutdown_connections 2024-09-15T21:48:39.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.514+0000 7f1813fff640 1 -- 172.21.15.100:0/1932117412 >> 172.21.15.100:0/1932117412 conn(0x7f18140ccbd0 msgr2=0x7f18140ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:39.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.514+0000 7f1813fff640 1 -- 172.21.15.100:0/1932117412 shutdown_connections 2024-09-15T21:48:39.562 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.514+0000 7f1813fff640 1 -- 172.21.15.100:0/1932117412 wait complete. 2024-09-15T21:48:39.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.514+0000 7f93de586640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93d80b27a0 0x7f93d81ec210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.563 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.514+0000 7f8df2ffd640 1 -- 172.21.15.100:0/2317809547 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de0005290 con 0x7f8e080c7a00 2024-09-15T21:48:39.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.514+0000 7f93de586640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93d80c8a60 0x7f93d81522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.514+0000 7f93de586640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93d80c8e40 0x7f93d81527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.514+0000 7f93de586640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f93d8160a90 con 0x7f93d80b27a0 2024-09-15T21:48:39.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.514+0000 7f93d7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93d80b27a0 0x7f93d81ec210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.514+0000 7f93de586640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f93d80b31c0 con 0x7f93d80c8a60 2024-09-15T21:48:39.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.514+0000 7f93d7fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93d80b27a0 0x7f93d81ec210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54364/0 (socket says 172.21.15.100:54364) 2024-09-15T21:48:39.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.514+0000 7f93d7fff640 1 -- 172.21.15.100:0/1014339201 learned_addr learned my addr 172.21.15.100:0/1014339201 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:39.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.514+0000 7f93de586640 1 -- 172.21.15.100:0/1014339201 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f93d80c34d0 con 0x7f93d80c8e40 2024-09-15T21:48:39.564 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.514+0000 7f93d77fe640 1 --2- 172.21.15.100:0/1014339201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93d80c8a60 0x7f93d81522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.514+0000 7f93dcb23640 1 --2- 172.21.15.100:0/1014339201 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93d80c8e40 0x7f93d81527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.515+0000 7f8e057fa640 1 -- 172.21.15.100:0/2317809547 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8dfc01a070 con 0x7f8e080c7a00 2024-09-15T21:48:39.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.515+0000 7f93dcb23640 1 -- 172.21.15.100:0/1014339201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93d80c8a60 msgr2=0x7f93d81522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.515+0000 7f93dcb23640 1 --2- 172.21.15.100:0/1014339201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93d80c8a60 0x7f93d81522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.565 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.515+0000 7f93dcb23640 1 -- 172.21.15.100:0/1014339201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93d80b27a0 msgr2=0x7f93d81ec210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.515+0000 7f93dcb23640 1 --2- 172.21.15.100:0/1014339201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93d80b27a0 0x7f93d81ec210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.515+0000 7f93dcb23640 1 -- 172.21.15.100:0/1014339201 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93d80cc9a0 con 0x7f93d80c8e40 2024-09-15T21:48:39.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.515+0000 7f93d77fe640 1 --2- 172.21.15.100:0/1014339201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93d80c8a60 0x7f93d81522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:48:39.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.515+0000 7f93dcb23640 1 --2- 172.21.15.100:0/1014339201 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93d80c8e40 0x7f93d81527f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f93d000ca70 tx=0x7f93d000cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.515+0000 7f8e057fa640 1 --2- 172.21.15.100:0/2317809547 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8dec06c330 0x7f8dec06e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.515+0000 7f93d4ff9640 1 -- 172.21.15.100:0/1014339201 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d0017070 con 0x7f93d80c8e40 2024-09-15T21:48:39.566 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.515+0000 7f93de586640 1 -- 172.21.15.100:0/1014339201 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93d80cb7d0 con 0x7f93d80c8e40 2024-09-15T21:48:39.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.515+0000 7f8e07fff640 1 --2- 172.21.15.100:0/2317809547 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8dec06c330 0x7f8dec06e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.515+0000 7f8e057fa640 1 -- 172.21.15.100:0/2317809547 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f8dfc08afe0 con 0x7f8e080c7a00 2024-09-15T21:48:39.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.515+0000 7f93de586640 1 -- 172.21.15.100:0/1014339201 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f93d81ec750 con 0x7f93d80c8e40 2024-09-15T21:48:39.567 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.515+0000 7f1813fff640 1 Processor -- start 2024-09-15T21:48:39.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.516+0000 7f93d4ff9640 1 -- 172.21.15.100:0/1014339201 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d000ab70 con 0x7f93d80c8e40 2024-09-15T21:48:39.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.516+0000 7f93d4ff9640 1 -- 172.21.15.100:0/1014339201 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d0012400 con 0x7f93d80c8e40 2024-09-15T21:48:39.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.516+0000 7f1813fff640 1 -- start start 2024-09-15T21:48:39.569 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.516+0000 7f8e07fff640 1 --2- 172.21.15.100:0/2317809547 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8dec06c330 0x7f8dec06e7f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8e0815a460 tx=0x7f8df4009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.517+0000 7f1813fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17fc004800 0x7f1814148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.517+0000 7f1813fff640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18140cbd30 0x7f18140c25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.517+0000 7f1813fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18140cac80 0x7f1814150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.517+0000 7f1813fff640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f18140cd9c0 con 0x7f18140cac80 2024-09-15T21:48:39.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.517+0000 7f1813fff640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1814071d70 con 0x7f17fc004800 2024-09-15T21:48:39.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.517+0000 7f1813fff640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f18140cdb40 con 0x7f18140cbd30 2024-09-15T21:48:39.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.517+0000 7f1812ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17fc004800 0x7f1814148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.517+0000 7f1812ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17fc004800 0x7f1814148ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:43066/0 (socket says 172.21.15.100:43066) 2024-09-15T21:48:39.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.517+0000 7f1812ffd640 1 -- 172.21.15.100:0/218362826 learned_addr learned my addr 172.21.15.100:0/218362826 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:39.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.517+0000 7f18137fe640 1 --2- 172.21.15.100:0/218362826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18140cac80 0x7f1814150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.517+0000 7f18127fc640 1 --2- 172.21.15.100:0/218362826 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18140cbd30 0x7f18140c25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.517+0000 7f93de586640 1 -- 172.21.15.100:0/1014339201 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b8005290 con 0x7f93d80c8e40 2024-09-15T21:48:39.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.518+0000 7f1812ffd640 1 -- 172.21.15.100:0/218362826 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18140cbd30 msgr2=0x7f18140c25c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.518+0000 7f1812ffd640 1 --2- 172.21.15.100:0/218362826 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18140cbd30 0x7f18140c25c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.518+0000 7f1812ffd640 1 -- 172.21.15.100:0/218362826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18140cac80 msgr2=0x7f1814150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.518+0000 7f1812ffd640 1 --2- 172.21.15.100:0/218362826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18140cac80 0x7f1814150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.518+0000 7f93d4ff9640 1 -- 172.21.15.100:0/1014339201 <== mon.2 v2:172.21.15.122:3301/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f93d001a070 con 0x7f93d80c8e40 2024-09-15T21:48:39.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.518+0000 7f1812ffd640 1 -- 172.21.15.100:0/218362826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18140c12b0 con 0x7f17fc004800 2024-09-15T21:48:39.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.518+0000 7f18137fe640 1 --2- 172.21.15.100:0/218362826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18140cac80 0x7f1814150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:48:39.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.518+0000 7f1812ffd640 1 --2- 172.21.15.100:0/218362826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17fc004800 0x7f1814148ff0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f180400e830 tx=0x7f180400ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.518+0000 7f93d4ff9640 1 --2- 172.21.15.100:0/1014339201 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f93c406c2b0 0x7f93c406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.518+0000 7f93d4ff9640 1 -- 172.21.15.100:0/1014339201 <== mon.2 v2:172.21.15.122:3301/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f93d008b060 con 0x7f93d80c8e40 2024-09-15T21:48:39.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.518+0000 7f93d7fff640 1 --2- 172.21.15.100:0/1014339201 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f93c406c2b0 0x7f93c406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.519+0000 7f93d7fff640 1 --2- 172.21.15.100:0/1014339201 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f93c406c2b0 0x7f93c406e770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009560 tx=0x7f93c802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.519+0000 7f67e2ffd640 1 -- 172.21.15.100:0/3043466638 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f67ec090070 con 0x7f67f40c3b10 2024-09-15T21:48:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.520+0000 7f17fbfff640 1 -- 172.21.15.100:0/218362826 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1804017070 con 0x7f17fc004800 2024-09-15T21:48:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.520+0000 7f17fbfff640 1 -- 172.21.15.100:0/218362826 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f180400c8c0 con 0x7f17fc004800 2024-09-15T21:48:39.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.520+0000 7f17fbfff640 1 -- 172.21.15.100:0/218362826 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180400ccb0 con 0x7f17fc004800 2024-09-15T21:48:39.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.520+0000 7f1813fff640 1 -- 172.21.15.100:0/218362826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18140c1520 con 0x7f17fc004800 2024-09-15T21:48:39.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.520+0000 7f1813fff640 1 -- 172.21.15.100:0/218362826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18140c1ae0 con 0x7f17fc004800 2024-09-15T21:48:39.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.520+0000 7f8e057fa640 1 -- 172.21.15.100:0/2317809547 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8dfc058fd0 con 0x7f8e080c7a00 2024-09-15T21:48:39.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.521+0000 7f1813fff640 1 -- 172.21.15.100:0/218362826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f4005290 con 0x7f17fc004800 2024-09-15T21:48:39.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.522+0000 7f17fbfff640 1 -- 172.21.15.100:0/218362826 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f180401a070 con 0x7f17fc004800 2024-09-15T21:48:39.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.522+0000 7f17fbfff640 1 --2- 172.21.15.100:0/218362826 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f180006c2e0 0x7f180006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.522+0000 7f17fbfff640 1 -- 172.21.15.100:0/218362826 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f180408b580 con 0x7f17fc004800 2024-09-15T21:48:39.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.523+0000 7f18127fc640 1 --2- 172.21.15.100:0/218362826 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f180006c2e0 0x7f180006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.523+0000 7f93d4ff9640 1 -- 172.21.15.100:0/1014339201 <== mon.2 v2:172.21.15.122:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f93d00590d0 con 0x7f93d80c8e40 2024-09-15T21:48:39.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.523+0000 7f18127fc640 1 --2- 172.21.15.100:0/218362826 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f180006c2e0 0x7f180006e7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1808034000 tx=0x7f180802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.527+0000 7f17fbfff640 1 -- 172.21.15.100:0/218362826 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f1804059690 con 0x7f17fc004800 2024-09-15T21:48:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.529+0000 7f23f9ffb640 1 -- 172.21.15.100:0/899488568 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f23fc048060 con 0x7f240415b830 2024-09-15T21:48:39.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.529+0000 7f8a2effd640 1 -- 172.21.15.100:0/1185695049 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a38040070 con 0x7f8a401598a0 2024-09-15T21:48:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.529+0000 7f8a2effd640 1 -- 172.21.15.100:0/1185695049 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a38031b40 con 0x7f8a401598a0 2024-09-15T21:48:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.529+0000 7f8a2effd640 1 -- 172.21.15.100:0/1185695049 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a38045460 con 0x7f8a401598a0 2024-09-15T21:48:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.530+0000 7f8a44d66640 1 -- 172.21.15.100:0/1185695049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a40072f50 con 0x7f8a401598a0 2024-09-15T21:48:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.530+0000 7f8a44d66640 1 -- 172.21.15.100:0/1185695049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a401f9030 con 0x7f8a401598a0 2024-09-15T21:48:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.533+0000 7f8a2effd640 1 -- 172.21.15.100:0/1185695049 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f8a38039040 con 0x7f8a401598a0 2024-09-15T21:48:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.533+0000 7f8a44d66640 1 -- 172.21.15.100:0/1185695049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a401634b0 con 0x7f8a401598a0 2024-09-15T21:48:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.534+0000 7f8a2effd640 1 --2- 172.21.15.100:0/1185695049 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8a2006c380 0x7f8a2006e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.534+0000 7f8a2effd640 1 -- 172.21.15.100:0/1185695049 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f8a380acaf0 con 0x7f8a401598a0 2024-09-15T21:48:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.534+0000 7f8a3e59c640 1 --2- 172.21.15.100:0/1185695049 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8a2006c380 0x7f8a2006e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.535+0000 7f8a3e59c640 1 --2- 172.21.15.100:0/1185695049 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8a2006c380 0x7f8a2006e840 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8a30000f60 tx=0x7f8a30009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:39.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.543+0000 7f8a2effd640 1 -- 172.21.15.100:0/1185695049 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f8a401634b0 con 0x7f8a401598a0 2024-09-15T21:48:39.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.850+0000 7f552a039640 1 -- 172.21.15.100:0/806541077 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f5504005580 con 0x7f55240c3b10 2024-09-15T21:48:39.857 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.851+0000 7f5515ffb640 1 -- 172.21.15.100:0/806541077 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f551c05d5e0 con 0x7f55240c3b10 2024-09-15T21:48:39.858 INFO:teuthology.orchestra.run.smithi100.stdout:60129542177 2024-09-15T21:48:39.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.854+0000 7f550f7fe640 1 -- 172.21.15.100:0/806541077 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f550806c3b0 msgr2=0x7f550806e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.854+0000 7f550f7fe640 1 --2- 172.21.15.100:0/806541077 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f550806c3b0 0x7f550806e870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5518009670 tx=0x7f5518006bd0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.855+0000 7f550f7fe640 1 -- 172.21.15.100:0/806541077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55240c3b10 msgr2=0x7f55240c1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.855+0000 7f550f7fe640 1 --2- 172.21.15.100:0/806541077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55240c3b10 0x7f55240c1d40 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f551c00bd60 tx=0x7f551c009e90 comp rx=0 tx=0).stop 2024-09-15T21:48:39.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.855+0000 7f550f7fe640 1 -- 172.21.15.100:0/806541077 shutdown_connections 2024-09-15T21:48:39.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.855+0000 7f550f7fe640 1 --2- 172.21.15.100:0/806541077 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f550806c3b0 0x7f550806e870 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.855+0000 7f550f7fe640 1 --2- 172.21.15.100:0/806541077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f55240c3b10 0x7f55240c1d40 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.855+0000 7f550f7fe640 1 --2- 172.21.15.100:0/806541077 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f552406e9a0 0x7f55240c1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.855+0000 7f550f7fe640 1 --2- 172.21.15.100:0/806541077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f552406e5c0 0x7f55240cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.855+0000 7f550f7fe640 1 -- 172.21.15.100:0/806541077 >> 172.21.15.100:0/806541077 conn(0x7f55240c9110 msgr2=0x7f55240c8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:39.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.855+0000 7f550f7fe640 1 -- 172.21.15.100:0/806541077 shutdown_connections 2024-09-15T21:48:39.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.856+0000 7f550f7fe640 1 -- 172.21.15.100:0/806541077 wait complete. 2024-09-15T21:48:39.875 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542177 got 60129542177 for osd.1 2024-09-15T21:48:39.875 DEBUG:teuthology.parallel:result is None 2024-09-15T21:48:39.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.898+0000 7f8a44d66640 1 -- 172.21.15.100:0/1185695049 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f8a400cc680 con 0x7f8a401598a0 2024-09-15T21:48:39.901 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.898+0000 7f8a2effd640 1 -- 172.21.15.100:0/1185695049 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8a401f8ad0 con 0x7f8a401598a0 2024-09-15T21:48:39.901 INFO:teuthology.orchestra.run.smithi100.stdout:55834574881 2024-09-15T21:48:39.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.900+0000 7f8a44d66640 1 -- 172.21.15.100:0/1185695049 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8a2006c380 msgr2=0x7f8a2006e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.900+0000 7f8a44d66640 1 --2- 172.21.15.100:0/1185695049 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8a2006c380 0x7f8a2006e840 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8a30000f60 tx=0x7f8a30009040 comp rx=0 tx=0).stop 2024-09-15T21:48:39.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.900+0000 7f8a44d66640 1 -- 172.21.15.100:0/1185695049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a401598a0 msgr2=0x7f8a401f5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.900+0000 7f8a44d66640 1 --2- 172.21.15.100:0/1185695049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a401598a0 0x7f8a401f5af0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8a38031000 tx=0x7f8a38004a40 comp rx=0 tx=0).stop 2024-09-15T21:48:39.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.900+0000 7f8a44d66640 1 -- 172.21.15.100:0/1185695049 shutdown_connections 2024-09-15T21:48:39.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.900+0000 7f8a44d66640 1 --2- 172.21.15.100:0/1185695049 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8a2006c380 0x7f8a2006e840 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.904 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.901+0000 7f8a44d66640 1 --2- 172.21.15.100:0/1185695049 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8a400c5bd0 0x7f8a4006ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.901+0000 7f8a44d66640 1 --2- 172.21.15.100:0/1185695049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8a401598a0 0x7f8a401f5af0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.901+0000 7f8a44d66640 1 --2- 172.21.15.100:0/1185695049 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8a4006e2e0 0x7f8a400cbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.901+0000 7f8a44d66640 1 -- 172.21.15.100:0/1185695049 >> 172.21.15.100:0/1185695049 conn(0x7f8a40067980 msgr2=0x7f8a400ccd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:39.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.901+0000 7f8a44d66640 1 -- 172.21.15.100:0/1185695049 shutdown_connections 2024-09-15T21:48:39.905 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.902+0000 7f8a44d66640 1 -- 172.21.15.100:0/1185695049 wait complete. 2024-09-15T21:48:39.907 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.904+0000 7fca17a1e640 1 -- 172.21.15.100:0/4262878143 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fca100cc5d0 con 0x7fca100c7550 2024-09-15T21:48:39.908 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.905+0000 7fca05ffb640 1 -- 172.21.15.100:0/4262878143 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fca0c079fe0 con 0x7fca100c7550 2024-09-15T21:48:39.908 INFO:teuthology.orchestra.run.smithi100.stdout:64424509473 2024-09-15T21:48:39.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.908+0000 7fca17a1e640 1 -- 172.21.15.100:0/4262878143 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc9f406c3b0 msgr2=0x7fc9f406e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.909+0000 7fca17a1e640 1 --2- 172.21.15.100:0/4262878143 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc9f406c3b0 0x7fc9f406e870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fca1014b0b0 tx=0x7fca00009190 comp rx=0 tx=0).stop 2024-09-15T21:48:39.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.909+0000 7fca17a1e640 1 -- 172.21.15.100:0/4262878143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca100c7550 msgr2=0x7fca1014b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.909+0000 7fca17a1e640 1 --2- 172.21.15.100:0/4262878143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca100c7550 0x7fca1014b2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fca0c009890 tx=0x7fca0c0046c0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.909+0000 7fca17a1e640 1 -- 172.21.15.100:0/4262878143 shutdown_connections 2024-09-15T21:48:39.912 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.909+0000 7fca17a1e640 1 --2- 172.21.15.100:0/4262878143 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc9f406c3b0 0x7fc9f406e870 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.909+0000 7fca17a1e640 1 --2- 172.21.15.100:0/4262878143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fca101597d0 0x7fca101482a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.909+0000 7fca17a1e640 1 --2- 172.21.15.100:0/4262878143 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fca100c8770 0x7fca10143950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.909+0000 7fca17a1e640 1 --2- 172.21.15.100:0/4262878143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca100c7550 0x7fca1014b2f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.909+0000 7fca17a1e640 1 -- 172.21.15.100:0/4262878143 >> 172.21.15.100:0/4262878143 conn(0x7fca10067dd0 msgr2=0x7fca1014c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:39.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.909+0000 7fca17a1e640 1 -- 172.21.15.100:0/4262878143 shutdown_connections 2024-09-15T21:48:39.913 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.910+0000 7fca17a1e640 1 -- 172.21.15.100:0/4262878143 wait complete. 2024-09-15T21:48:39.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574881 got 55834574881 for osd.3 2024-09-15T21:48:39.918 DEBUG:teuthology.parallel:result is None 2024-09-15T21:48:39.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.921+0000 7f67f359e640 1 -- 172.21.15.100:0/3043466638 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f67d8005580 con 0x7f67f40c3b10 2024-09-15T21:48:39.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.921+0000 7f67e2ffd640 1 -- 172.21.15.100:0/3043466638 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f67ec059d10 con 0x7f67f40c3b10 2024-09-15T21:48:39.924 INFO:teuthology.orchestra.run.smithi100.stdout:64424509472 2024-09-15T21:48:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.921+0000 7f240b853640 1 -- 172.21.15.100:0/899488568 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f24041500f0 con 0x7f240415b830 2024-09-15T21:48:39.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.922+0000 7f23f9ffb640 1 -- 172.21.15.100:0/899488568 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f23fc07aa40 con 0x7f240415b830 2024-09-15T21:48:39.925 INFO:teuthology.orchestra.run.smithi100.stdout:64424509472 2024-09-15T21:48:39.926 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509473 got 64424509473 for osd.4 2024-09-15T21:48:39.926 DEBUG:teuthology.parallel:result is None 2024-09-15T21:48:39.927 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f67f359e640 1 -- 172.21.15.100:0/3043466638 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f67d406c2e0 msgr2=0x7f67d406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f67f359e640 1 --2- 172.21.15.100:0/3043466638 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f67d406c2e0 0x7f67d406e7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f67e4034000 tx=0x7f67e402e000 comp rx=0 tx=0).stop 2024-09-15T21:48:39.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f67f359e640 1 -- 172.21.15.100:0/3043466638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f40c3b10 msgr2=0x7f67f4156640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f67f359e640 1 --2- 172.21.15.100:0/3043466638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f40c3b10 0x7f67f4156640 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00cf90 tx=0x7f67ec00a7d0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f240b853640 1 -- 172.21.15.100:0/899488568 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f23e806c380 msgr2=0x7f23e806e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f240b853640 1 --2- 172.21.15.100:0/899488568 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f23e806c380 0x7f23e806e840 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f23f4006fd0 tx=0x7f23f4008040 comp rx=0 tx=0).stop 2024-09-15T21:48:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f240b853640 1 -- 172.21.15.100:0/899488568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f240415b830 msgr2=0x7f24040ca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f240b853640 1 --2- 172.21.15.100:0/899488568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f240415b830 0x7f24040ca250 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f23fc031000 tx=0x7f23fc004a40 comp rx=0 tx=0).stop 2024-09-15T21:48:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f240b853640 1 -- 172.21.15.100:0/899488568 shutdown_connections 2024-09-15T21:48:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f240b853640 1 --2- 172.21.15.100:0/899488568 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f23e806c380 0x7f23e806e840 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f240b853640 1 --2- 172.21.15.100:0/899488568 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f240415b830 0x7f24040ca250 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f240b853640 1 --2- 172.21.15.100:0/899488568 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2404155130 0x7f24040c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f240b853640 1 --2- 172.21.15.100:0/899488568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f24040d13d0 0x7f24040cf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f8df2ffd640 1 -- 172.21.15.100:0/2317809547 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f8de0005580 con 0x7f8e080c7a00 2024-09-15T21:48:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f240b853640 1 -- 172.21.15.100:0/899488568 >> 172.21.15.100:0/899488568 conn(0x7f2404067dd0 msgr2=0x7f240406e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f67f359e640 1 -- 172.21.15.100:0/3043466638 shutdown_connections 2024-09-15T21:48:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f67f359e640 1 --2- 172.21.15.100:0/3043466638 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f67d406c2e0 0x7f67d406e7a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f67f359e640 1 --2- 172.21.15.100:0/3043466638 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f67f40c9c30 0x7f67f4073110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f67f359e640 1 --2- 172.21.15.100:0/3043466638 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f40c3b10 0x7f67f4156640 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f67f359e640 1 --2- 172.21.15.100:0/3043466638 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f67f40c32f0 0x7f67f40c0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f67f359e640 1 -- 172.21.15.100:0/3043466638 >> 172.21.15.100:0/3043466638 conn(0x7f67f40c9110 msgr2=0x7f67f40cb330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f67f359e640 1 -- 172.21.15.100:0/3043466638 shutdown_connections 2024-09-15T21:48:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f240b853640 1 -- 172.21.15.100:0/899488568 shutdown_connections 2024-09-15T21:48:39.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f67f359e640 1 -- 172.21.15.100:0/3043466638 wait complete. 2024-09-15T21:48:39.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f240b853640 1 -- 172.21.15.100:0/899488568 wait complete. 2024-09-15T21:48:39.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.925+0000 7f8e057fa640 1 -- 172.21.15.100:0/2317809547 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8dfc05d2d0 con 0x7f8e080c7a00 2024-09-15T21:48:39.932 INFO:teuthology.orchestra.run.smithi100.stdout:68719476768 2024-09-15T21:48:39.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.928+0000 7f8df2ffd640 1 -- 172.21.15.100:0/2317809547 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8dec06c330 msgr2=0x7f8dec06e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.928+0000 7f8df2ffd640 1 --2- 172.21.15.100:0/2317809547 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8dec06c330 0x7f8dec06e7f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f8e0815a460 tx=0x7f8df4009060 comp rx=0 tx=0).stop 2024-09-15T21:48:39.933 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.928+0000 7f8df2ffd640 1 -- 172.21.15.100:0/2317809547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e080c7a00 msgr2=0x7f8e0815ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.928+0000 7f8df2ffd640 1 --2- 172.21.15.100:0/2317809547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e080c7a00 0x7f8e0815ab80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00e7e0 tx=0x7f8dfc00eca0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.928+0000 7f8df2ffd640 1 -- 172.21.15.100:0/2317809547 shutdown_connections 2024-09-15T21:48:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.928+0000 7f8df2ffd640 1 --2- 172.21.15.100:0/2317809547 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f8dec06c330 0x7f8dec06e7f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.928+0000 7f8df2ffd640 1 --2- 172.21.15.100:0/2317809547 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e080ca2e0 0x7f8e08146ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.928+0000 7f8df2ffd640 1 --2- 172.21.15.100:0/2317809547 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f8e080c9f00 0x7f8e08161900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.928+0000 7f8df2ffd640 1 --2- 172.21.15.100:0/2317809547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e080c7a00 0x7f8e0815ab80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.928+0000 7f8df2ffd640 1 -- 172.21.15.100:0/2317809547 >> 172.21.15.100:0/2317809547 conn(0x7f8e08073820 msgr2=0x7f8e080cc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.928+0000 7f8df2ffd640 1 -- 172.21.15.100:0/2317809547 shutdown_connections 2024-09-15T21:48:39.934 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.928+0000 7f8df2ffd640 1 -- 172.21.15.100:0/2317809547 wait complete. 2024-09-15T21:48:39.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.934+0000 7f1813fff640 1 -- 172.21.15.100:0/218362826 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f17f4005850 con 0x7f17fc004800 2024-09-15T21:48:39.938 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.935+0000 7f17fbfff640 1 -- 172.21.15.100:0/218362826 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f180405d990 con 0x7f17fc004800 2024-09-15T21:48:39.939 INFO:teuthology.orchestra.run.smithi100.stdout:244813135878 2024-09-15T21:48:39.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.938+0000 7f1813fff640 1 -- 172.21.15.100:0/218362826 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f180006c2e0 msgr2=0x7f180006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.941 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.938+0000 7f1813fff640 1 --2- 172.21.15.100:0/218362826 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f180006c2e0 0x7f180006e7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1808034000 tx=0x7f180802e000 comp rx=0 tx=0).stop 2024-09-15T21:48:39.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.938+0000 7f1813fff640 1 -- 172.21.15.100:0/218362826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17fc004800 msgr2=0x7f1814148ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.938+0000 7f1813fff640 1 --2- 172.21.15.100:0/218362826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17fc004800 0x7f1814148ff0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f180400e830 tx=0x7f180400ecf0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.938+0000 7f1813fff640 1 -- 172.21.15.100:0/218362826 shutdown_connections 2024-09-15T21:48:39.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.938+0000 7f1813fff640 1 --2- 172.21.15.100:0/218362826 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f180006c2e0 0x7f180006e7a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.938+0000 7f1813fff640 1 --2- 172.21.15.100:0/218362826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18140cac80 0x7f1814150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.938+0000 7f1813fff640 1 --2- 172.21.15.100:0/218362826 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f18140cbd30 0x7f18140c25c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.938+0000 7f1813fff640 1 --2- 172.21.15.100:0/218362826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17fc004800 0x7f1814148ff0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.938+0000 7f1813fff640 1 -- 172.21.15.100:0/218362826 >> 172.21.15.100:0/218362826 conn(0x7f18140ccbd0 msgr2=0x7f1814156ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:39.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.938+0000 7f1813fff640 1 -- 172.21.15.100:0/218362826 shutdown_connections 2024-09-15T21:48:39.942 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.938+0000 7f1813fff640 1 -- 172.21.15.100:0/218362826 wait complete. 2024-09-15T21:48:39.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.939+0000 7f93de586640 1 -- 172.21.15.100:0/1014339201 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f93b8005580 con 0x7f93d80c8e40 2024-09-15T21:48:39.943 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.939+0000 7f93d4ff9640 1 -- 172.21.15.100:0/1014339201 <== mon.2 v2:172.21.15.122:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f93d005d3d0 con 0x7f93d80c8e40 2024-09-15T21:48:39.943 INFO:teuthology.orchestra.run.smithi100.stdout:64424509474 2024-09-15T21:48:39.944 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476768 got 68719476768 for osd.0 2024-09-15T21:48:39.944 DEBUG:teuthology.parallel:result is None 2024-09-15T21:48:39.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.943+0000 7f93de586640 1 -- 172.21.15.100:0/1014339201 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f93c406c2b0 msgr2=0x7f93c406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.943+0000 7f93de586640 1 --2- 172.21.15.100:0/1014339201 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f93c406c2b0 0x7f93c406e770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009560 tx=0x7f93c802e000 comp rx=0 tx=0).stop 2024-09-15T21:48:39.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.943+0000 7f93de586640 1 -- 172.21.15.100:0/1014339201 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93d80c8e40 msgr2=0x7f93d81527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:39.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.943+0000 7f93de586640 1 --2- 172.21.15.100:0/1014339201 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93d80c8e40 0x7f93d81527f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f93d000ca70 tx=0x7f93d000cf30 comp rx=0 tx=0).stop 2024-09-15T21:48:39.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.943+0000 7f93de586640 1 -- 172.21.15.100:0/1014339201 shutdown_connections 2024-09-15T21:48:39.946 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.943+0000 7f93de586640 1 --2- 172.21.15.100:0/1014339201 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f93c406c2b0 0x7f93c406e770 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.943+0000 7f93de586640 1 --2- 172.21.15.100:0/1014339201 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93d80c8a60 0x7f93d81522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.943+0000 7f93de586640 1 --2- 172.21.15.100:0/1014339201 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93d80c8e40 0x7f93d81527f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.943+0000 7f93de586640 1 --2- 172.21.15.100:0/1014339201 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93d80b27a0 0x7f93d81ec210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:39.947 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.943+0000 7f93de586640 1 -- 172.21.15.100:0/1014339201 >> 172.21.15.100:0/1014339201 conn(0x7f93d806ed20 msgr2=0x7f93d8162270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:39.948 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509472 got 64424509472 for osd.7 2024-09-15T21:48:39.948 DEBUG:teuthology.parallel:result is None 2024-09-15T21:48:39.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.943+0000 7f93de586640 1 -- 172.21.15.100:0/1014339201 shutdown_connections 2024-09-15T21:48:39.948 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:39.943+0000 7f93de586640 1 -- 172.21.15.100:0/1014339201 wait complete. 2024-09-15T21:48:39.949 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509472 got 64424509472 for osd.2 2024-09-15T21:48:39.949 DEBUG:teuthology.parallel:result is None 2024-09-15T21:48:39.957 INFO:tasks.ceph.ceph_manager.ceph:need seq 244813135878 got 244813135878 for osd.5 2024-09-15T21:48:39.957 DEBUG:teuthology.parallel:result is None 2024-09-15T21:48:39.960 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509474 got 64424509474 for osd.6 2024-09-15T21:48:39.960 DEBUG:teuthology.parallel:result is None 2024-09-15T21:48:39.960 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-15T21:48:39.960 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:48:40.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.094+0000 7f2fa6f76640 1 Processor -- start 2024-09-15T21:48:40.097 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.094+0000 7f2fa6f76640 1 -- start start 2024-09-15T21:48:40.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.095+0000 7f2fa6f76640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2fa0072160 0x7f2fa00c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.095+0000 7f2fa6f76640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f2fa0154eb0 0x7f2fa0155300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.095+0000 7f2fa6f76640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fa015f1a0 con 0x7f2fa00c99e0 2024-09-15T21:48:40.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.095+0000 7f2fa6f76640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f2fa0155840 con 0x7f2fa0072160 2024-09-15T21:48:40.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.095+0000 7f2fa6f76640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f2fa015f440 con 0x7f2fa0154eb0 2024-09-15T21:48:40.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.095+0000 7f2fa4d12640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2fa0072160 0x7f2fa00c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.095+0000 7f2fa4d12640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2fa0072160 0x7f2fa00c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54382/0 (socket says 172.21.15.100:54382) 2024-09-15T21:48:40.099 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.095+0000 7f2fa4d12640 1 -- 172.21.15.100:0/2696831088 learned_addr learned my addr 172.21.15.100:0/2696831088 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:40.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.095+0000 7f2fa5513640 1 --2- 172.21.15.100:0/2696831088 >> v2:172.21.15.122:3300/0 conn(0x7f2fa0154eb0 0x7f2fa0155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.095+0000 7f2fa4d12640 1 -- 172.21.15.100:0/2696831088 >> v2:172.21.15.122:3300/0 conn(0x7f2fa0154eb0 msgr2=0x7f2fa0155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:40.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.095+0000 7f2fa4d12640 1 --2- 172.21.15.100:0/2696831088 >> v2:172.21.15.122:3300/0 conn(0x7f2fa0154eb0 0x7f2fa0155300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.095+0000 7f2fa4d12640 1 -- 172.21.15.100:0/2696831088 >> v1:172.21.15.122:6789/0 conn(0x7f2fa00c99e0 legacy=0x7f2fa0072540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:40.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.095+0000 7f2fa4d12640 1 -- 172.21.15.100:0/2696831088 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa00c83a0 con 0x7f2fa0072160 2024-09-15T21:48:40.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.096+0000 7f2fa4d12640 1 --2- 172.21.15.100:0/2696831088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa0072160 0x7f2fa00c9230 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2fa01559c0 tx=0x7f2f9c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709e4ab80dff7cf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:40.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.096+0000 7f2f977fe640 1 -- 172.21.15.100:0/2696831088 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c037070 con 0x7f2fa0072160 2024-09-15T21:48:40.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.096+0000 7f2f977fe640 1 -- 172.21.15.100:0/2696831088 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f9c032440 con 0x7f2fa0072160 2024-09-15T21:48:40.100 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.096+0000 7f2f977fe640 1 -- 172.21.15.100:0/2696831088 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c032810 con 0x7f2fa0072160 2024-09-15T21:48:40.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.096+0000 7f2fa6f76640 1 -- 172.21.15.100:0/2696831088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa0072160 msgr2=0x7f2fa00c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:40.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.096+0000 7f2fa6f76640 1 --2- 172.21.15.100:0/2696831088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa0072160 0x7f2fa00c9230 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2fa01559c0 tx=0x7f2f9c02a2c0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.097+0000 7f2fa6f76640 1 -- 172.21.15.100:0/2696831088 shutdown_connections 2024-09-15T21:48:40.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.097+0000 7f2fa6f76640 1 --2- 172.21.15.100:0/2696831088 >> v2:172.21.15.122:3300/0 conn(0x7f2fa0154eb0 0x7f2fa0155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.103 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.097+0000 7f2fa6f76640 1 --2- 172.21.15.100:0/2696831088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa0072160 0x7f2fa00c9230 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.097+0000 7f2fa6f76640 1 -- 172.21.15.100:0/2696831088 >> 172.21.15.100:0/2696831088 conn(0x7f2fa00cd820 msgr2=0x7f2fa00cdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.097+0000 7f2fa6f76640 1 -- 172.21.15.100:0/2696831088 shutdown_connections 2024-09-15T21:48:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.097+0000 7f2fa6f76640 1 -- 172.21.15.100:0/2696831088 wait complete. 2024-09-15T21:48:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.098+0000 7f2fa6f76640 1 Processor -- start 2024-09-15T21:48:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.098+0000 7f2fa6f76640 1 -- start start 2024-09-15T21:48:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.098+0000 7f2fa6f76640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa0072160 0x7f2fa0156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2fa6f76640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fa00c99e0 0x7f2fa01e8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2fa6f76640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2fa0154eb0 0x7f2fa01f4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2fa6f76640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa00ca010 con 0x7f2fa0072160 2024-09-15T21:48:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2fa6f76640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa0072d10 con 0x7f2fa00c99e0 2024-09-15T21:48:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2fa6f76640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa01573f0 con 0x7f2fa0154eb0 2024-09-15T21:48:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2fa4d12640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa0072160 0x7f2fa0156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.104 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2fa4d12640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa0072160 0x7f2fa0156eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54394/0 (socket says 172.21.15.100:54394) 2024-09-15T21:48:40.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2fa4d12640 1 -- 172.21.15.100:0/2984885613 learned_addr learned my addr 172.21.15.100:0/2984885613 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:40.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2f97fff640 1 --2- 172.21.15.100:0/2984885613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fa00c99e0 0x7f2fa01e8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2fa5513640 1 --2- 172.21.15.100:0/2984885613 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2fa0154eb0 0x7f2fa01f4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2fa4d12640 1 -- 172.21.15.100:0/2984885613 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2fa0154eb0 msgr2=0x7f2fa01f4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:40.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2fa4d12640 1 --2- 172.21.15.100:0/2984885613 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2fa0154eb0 0x7f2fa01f4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2fa4d12640 1 -- 172.21.15.100:0/2984885613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fa00c99e0 msgr2=0x7f2fa01e8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:40.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2fa4d12640 1 --2- 172.21.15.100:0/2984885613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fa00c99e0 0x7f2fa01e8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2fa4d12640 1 -- 172.21.15.100:0/2984885613 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa00c7250 con 0x7f2fa0072160 2024-09-15T21:48:40.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2fa4d12640 1 --2- 172.21.15.100:0/2984885613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa0072160 0x7f2fa0156eb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c02a300 tx=0x7f2f9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:40.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2f957fa640 1 -- 172.21.15.100:0/2984885613 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c037040 con 0x7f2fa0072160 2024-09-15T21:48:40.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2f957fa640 1 -- 172.21.15.100:0/2984885613 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f9c042070 con 0x7f2fa0072160 2024-09-15T21:48:40.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2f957fa640 1 -- 172.21.15.100:0/2984885613 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c030bb0 con 0x7f2fa0072160 2024-09-15T21:48:40.105 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2fa6f76640 1 -- 172.21.15.100:0/2984885613 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa01e4560 con 0x7f2fa0072160 2024-09-15T21:48:40.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.099+0000 7f2fa6f76640 1 -- 172.21.15.100:0/2984885613 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa0155b20 con 0x7f2fa0072160 2024-09-15T21:48:40.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.100+0000 7f2fa6f76640 1 -- 172.21.15.100:0/2984885613 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fa00c9230 con 0x7f2fa0072160 2024-09-15T21:48:40.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.101+0000 7f2f957fa640 1 -- 172.21.15.100:0/2984885613 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2f9c047460 con 0x7f2fa0072160 2024-09-15T21:48:40.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.101+0000 7f2f957fa640 1 --2- 172.21.15.100:0/2984885613 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2f8c06c2b0 0x7f2f8c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.106 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.101+0000 7f2f957fa640 1 -- 172.21.15.100:0/2984885613 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f2f9c0ac120 con 0x7f2fa0072160 2024-09-15T21:48:40.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.101+0000 7f2f97fff640 1 --2- 172.21.15.100:0/2984885613 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2f8c06c2b0 0x7f2f8c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.107 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.102+0000 7f2f97fff640 1 --2- 172.21.15.100:0/2984885613 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2f8c06c2b0 0x7f2f8c06e770 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0154a80 tx=0x7f2f90009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:40.108 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.106+0000 7f2f957fa640 1 -- 172.21.15.100:0/2984885613 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2f9c045360 con 0x7f2fa0072160 2024-09-15T21:48:40.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.269+0000 7f2fa6f76640 1 -- 172.21.15.100:0/2984885613 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2fa00c74f0 con 0x7f2f8c06c2b0 2024-09-15T21:48:40.278 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.275+0000 7f2f957fa640 1 -- 172.21.15.100:0/2984885613 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252302 (secure 0 0 0) 0x7f2fa00c74f0 con 0x7f2f8c06c2b0 2024-09-15T21:48:40.281 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:48:40.282 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-15T21:48:40.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.278+0000 7f2fa6f76640 1 -- 172.21.15.100:0/2984885613 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2f8c06c2b0 msgr2=0x7f2f8c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:40.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.278+0000 7f2fa6f76640 1 --2- 172.21.15.100:0/2984885613 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2f8c06c2b0 0x7f2f8c06e770 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0154a80 tx=0x7f2f90009040 comp rx=0 tx=0).stop 2024-09-15T21:48:40.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.278+0000 7f2fa6f76640 1 -- 172.21.15.100:0/2984885613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa0072160 msgr2=0x7f2fa0156eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:40.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.278+0000 7f2fa6f76640 1 --2- 172.21.15.100:0/2984885613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa0072160 0x7f2fa0156eb0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c02a300 tx=0x7f2f9c0046c0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.278+0000 7f2fa6f76640 1 -- 172.21.15.100:0/2984885613 shutdown_connections 2024-09-15T21:48:40.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.278+0000 7f2fa6f76640 1 --2- 172.21.15.100:0/2984885613 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2f8c06c2b0 0x7f2f8c06e770 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.284 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.278+0000 7f2fa6f76640 1 --2- 172.21.15.100:0/2984885613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2fa00c99e0 0x7f2fa01e8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.278+0000 7f2fa6f76640 1 --2- 172.21.15.100:0/2984885613 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2fa0154eb0 0x7f2fa01f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.278+0000 7f2fa6f76640 1 --2- 172.21.15.100:0/2984885613 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2fa0072160 0x7f2fa0156eb0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.278+0000 7f2fa6f76640 1 -- 172.21.15.100:0/2984885613 >> 172.21.15.100:0/2984885613 conn(0x7f2fa00cd820 msgr2=0x7f2fa0158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:40.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.279+0000 7f2fa6f76640 1 -- 172.21.15.100:0/2984885613 shutdown_connections 2024-09-15T21:48:40.285 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.279+0000 7f2fa6f76640 1 -- 172.21.15.100:0/2984885613 wait complete. 2024-09-15T21:48:40.292 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-15T21:48:39.832625+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633023,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":404,"num_write_kb":2533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1185538,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":335,"ondisk_log_size":335,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238220,"kb_used_data":8328,"kb_used_omap":18,"kb_used_meta":229677,"kb_avail":754736500,"statfs":{"total":773094113280,"available":772850176000,"internally_reserved":0,"allocated":8527872,"data_stored":4517856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19119,"internal_metadata":235189585},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001127"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.711284+0000","last_change":"2024-09-15T21:48:21.639361+0000","last_active":"2024-09-15T21:48:35.711284+0000","last_peered":"2024-09-15T21:48:35.711284+0000","last_clean":"2024-09-15T21:48:35.711284+0000","last_became_active":"2024-09-15T21:48:21.637402+0000","last_became_peered":"2024-09-15T21:48:21.637402+0000","last_unstale":"2024-09-15T21:48:35.711284+0000","last_undegraded":"2024-09-15T21:48:35.711284+0000","last_fullsized":"2024-09-15T21:48:35.711284+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:40.643861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726138+0000","last_change":"2024-09-15T21:46:23.418716+0000","last_active":"2024-09-15T21:48:36.726138+0000","last_peered":"2024-09-15T21:48:36.726138+0000","last_clean":"2024-09-15T21:48:36.726138+0000","last_became_active":"2024-09-15T21:46:23.418471+0000","last_became_peered":"2024-09-15T21:46:23.418471+0000","last_unstale":"2024-09-15T21:48:36.726138+0000","last_undegraded":"2024-09-15T21:48:36.726138+0000","last_fullsized":"2024-09-15T21:48:36.726138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:49.599801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726536+0000","last_change":"2024-09-15T21:46:23.426404+0000","last_active":"2024-09-15T21:48:36.726536+0000","last_peered":"2024-09-15T21:48:36.726536+0000","last_clean":"2024-09-15T21:48:36.726536+0000","last_became_active":"2024-09-15T21:46:23.426146+0000","last_became_peered":"2024-09-15T21:46:23.426146+0000","last_unstale":"2024-09-15T21:48:36.726536+0000","last_undegraded":"2024-09-15T21:48:36.726536+0000","last_fullsized":"2024-09-15T21:48:36.726536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:23.626109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022058+0000","last_change":"2024-09-15T21:48:17.624363+0000","last_active":"2024-09-15T21:48:37.022058+0000","last_peered":"2024-09-15T21:48:37.022058+0000","last_clean":"2024-09-15T21:48:37.022058+0000","last_became_active":"2024-09-15T21:48:17.623606+0000","last_became_peered":"2024-09-15T21:48:17.623606+0000","last_unstale":"2024-09-15T21:48:37.022058+0000","last_undegraded":"2024-09-15T21:48:37.022058+0000","last_fullsized":"2024-09-15T21:48:37.022058+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:03.294215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710447+0000","last_change":"2024-09-15T21:48:21.640215+0000","last_active":"2024-09-15T21:48:35.710447+0000","last_peered":"2024-09-15T21:48:35.710447+0000","last_clean":"2024-09-15T21:48:35.710447+0000","last_became_active":"2024-09-15T21:48:21.638410+0000","last_became_peered":"2024-09-15T21:48:21.638410+0000","last_unstale":"2024-09-15T21:48:35.710447+0000","last_undegraded":"2024-09-15T21:48:35.710447+0000","last_fullsized":"2024-09-15T21:48:35.710447+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:43.167624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022015+0000","last_change":"2024-09-15T21:48:17.625573+0000","last_active":"2024-09-15T21:48:37.022015+0000","last_peered":"2024-09-15T21:48:37.022015+0000","last_clean":"2024-09-15T21:48:37.022015+0000","last_became_active":"2024-09-15T21:48:17.624439+0000","last_became_peered":"2024-09-15T21:48:17.624439+0000","last_unstale":"2024-09-15T21:48:37.022015+0000","last_undegraded":"2024-09-15T21:48:37.022015+0000","last_fullsized":"2024-09-15T21:48:37.022015+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:04.063675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710549+0000","last_change":"2024-09-15T21:48:21.638612+0000","last_active":"2024-09-15T21:48:35.710549+0000","last_peered":"2024-09-15T21:48:35.710549+0000","last_clean":"2024-09-15T21:48:35.710549+0000","last_became_active":"2024-09-15T21:48:21.638310+0000","last_became_peered":"2024-09-15T21:48:21.638310+0000","last_unstale":"2024-09-15T21:48:35.710549+0000","last_undegraded":"2024-09-15T21:48:35.710549+0000","last_fullsized":"2024-09-15T21:48:35.710549+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:37.297103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":99,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850506+0000","last_change":"2024-09-15T21:48:27.657047+0000","last_active":"2024-09-15T21:48:36.850506+0000","last_peered":"2024-09-15T21:48:36.850506+0000","last_clean":"2024-09-15T21:48:36.850506+0000","last_became_active":"2024-09-15T21:48:27.656800+0000","last_became_peered":"2024-09-15T21:48:27.656800+0000","last_unstale":"2024-09-15T21:48:36.850506+0000","last_undegraded":"2024-09-15T21:48:36.850506+0000","last_fullsized":"2024-09-15T21:48:36.850506+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:17.342504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.711186+0000","last_change":"2024-09-15T21:48:21.642296+0000","last_active":"2024-09-15T21:48:35.711186+0000","last_peered":"2024-09-15T21:48:35.711186+0000","last_clean":"2024-09-15T21:48:35.711186+0000","last_became_active":"2024-09-15T21:48:21.641920+0000","last_became_peered":"2024-09-15T21:48:21.641920+0000","last_unstale":"2024-09-15T21:48:35.711186+0000","last_undegraded":"2024-09-15T21:48:35.711186+0000","last_fullsized":"2024-09-15T21:48:35.711186+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:45.342068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859014+0000","last_change":"2024-09-15T21:48:17.622439+0000","last_active":"2024-09-15T21:48:36.859014+0000","last_peered":"2024-09-15T21:48:36.859014+0000","last_clean":"2024-09-15T21:48:36.859014+0000","last_became_active":"2024-09-15T21:48:17.621800+0000","last_became_peered":"2024-09-15T21:48:17.621800+0000","last_unstale":"2024-09-15T21:48:36.859014+0000","last_undegraded":"2024-09-15T21:48:36.859014+0000","last_fullsized":"2024-09-15T21:48:36.859014+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:07.312656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":122,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858947+0000","last_change":"2024-09-15T21:47:05.735261+0000","last_active":"2024-09-15T21:48:36.858947+0000","last_peered":"2024-09-15T21:48:36.858947+0000","last_clean":"2024-09-15T21:48:36.858947+0000","last_became_active":"2024-09-15T21:47:05.734921+0000","last_became_peered":"2024-09-15T21:47:05.734921+0000","last_unstale":"2024-09-15T21:48:36.858947+0000","last_undegraded":"2024-09-15T21:48:36.858947+0000","last_fullsized":"2024-09-15T21:48:36.858947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:55.787926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726086+0000","last_change":"2024-09-15T21:46:23.432534+0000","last_active":"2024-09-15T21:48:36.726086+0000","last_peered":"2024-09-15T21:48:36.726086+0000","last_clean":"2024-09-15T21:48:36.726086+0000","last_became_active":"2024-09-15T21:46:23.430170+0000","last_became_peered":"2024-09-15T21:46:23.430170+0000","last_unstale":"2024-09-15T21:48:36.726086+0000","last_undegraded":"2024-09-15T21:48:36.726086+0000","last_fullsized":"2024-09-15T21:48:36.726086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:11.463937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":117,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710817+0000","last_change":"2024-09-15T21:46:23.427841+0000","last_active":"2024-09-15T21:48:35.710817+0000","last_peered":"2024-09-15T21:48:35.710817+0000","last_clean":"2024-09-15T21:48:35.710817+0000","last_became_active":"2024-09-15T21:46:23.427621+0000","last_became_peered":"2024-09-15T21:46:23.427621+0000","last_unstale":"2024-09-15T21:48:35.710817+0000","last_undegraded":"2024-09-15T21:48:35.710817+0000","last_fullsized":"2024-09-15T21:48:35.710817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:18.331943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"55'3","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726331+0000","last_change":"2024-09-15T21:46:23.424707+0000","last_active":"2024-09-15T21:48:36.726331+0000","last_peered":"2024-09-15T21:48:36.726331+0000","last_clean":"2024-09-15T21:48:36.726331+0000","last_became_active":"2024-09-15T21:46:23.424447+0000","last_became_peered":"2024-09-15T21:46:23.424447+0000","last_unstale":"2024-09-15T21:48:36.726331+0000","last_undegraded":"2024-09-15T21:48:36.726331+0000","last_fullsized":"2024-09-15T21:48:36.726331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:23.617214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725904+0000","last_change":"2024-09-15T21:46:23.430297+0000","last_active":"2024-09-15T21:48:36.725904+0000","last_peered":"2024-09-15T21:48:36.725904+0000","last_clean":"2024-09-15T21:48:36.725904+0000","last_became_active":"2024-09-15T21:46:23.430070+0000","last_became_peered":"2024-09-15T21:46:23.430070+0000","last_unstale":"2024-09-15T21:48:36.725904+0000","last_undegraded":"2024-09-15T21:48:36.725904+0000","last_fullsized":"2024-09-15T21:48:36.725904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:03:19.615022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021835+0000","last_change":"2024-09-15T21:48:17.625113+0000","last_active":"2024-09-15T21:48:37.021835+0000","last_peered":"2024-09-15T21:48:37.021835+0000","last_clean":"2024-09-15T21:48:37.021835+0000","last_became_active":"2024-09-15T21:48:17.624764+0000","last_became_peered":"2024-09-15T21:48:17.624764+0000","last_unstale":"2024-09-15T21:48:37.021835+0000","last_undegraded":"2024-09-15T21:48:37.021835+0000","last_fullsized":"2024-09-15T21:48:37.021835+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:28:44.142358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022276+0000","last_change":"2024-09-15T21:48:21.642646+0000","last_active":"2024-09-15T21:48:37.022276+0000","last_peered":"2024-09-15T21:48:37.022276+0000","last_clean":"2024-09-15T21:48:37.022276+0000","last_became_active":"2024-09-15T21:48:21.642303+0000","last_became_peered":"2024-09-15T21:48:21.642303+0000","last_unstale":"2024-09-15T21:48:37.022276+0000","last_undegraded":"2024-09-15T21:48:37.022276+0000","last_fullsized":"2024-09-15T21:48:37.022276+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:08:54.511282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":75,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850781+0000","last_change":"2024-09-15T21:48:27.661631+0000","last_active":"2024-09-15T21:48:36.850781+0000","last_peered":"2024-09-15T21:48:36.850781+0000","last_clean":"2024-09-15T21:48:36.850781+0000","last_became_active":"2024-09-15T21:48:27.661441+0000","last_became_peered":"2024-09-15T21:48:27.661441+0000","last_unstale":"2024-09-15T21:48:36.850781+0000","last_undegraded":"2024-09-15T21:48:36.850781+0000","last_fullsized":"2024-09-15T21:48:36.850781+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:23.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710445+0000","last_change":"2024-09-15T21:48:21.639602+0000","last_active":"2024-09-15T21:48:35.710445+0000","last_peered":"2024-09-15T21:48:35.710445+0000","last_clean":"2024-09-15T21:48:35.710445+0000","last_became_active":"2024-09-15T21:48:21.639009+0000","last_became_peered":"2024-09-15T21:48:21.639009+0000","last_unstale":"2024-09-15T21:48:35.710445+0000","last_undegraded":"2024-09-15T21:48:35.710445+0000","last_fullsized":"2024-09-15T21:48:35.710445+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:16.818783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858858+0000","last_change":"2024-09-15T21:46:23.434558+0000","last_active":"2024-09-15T21:48:36.858858+0000","last_peered":"2024-09-15T21:48:36.858858+0000","last_clean":"2024-09-15T21:48:36.858858+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:48:36.858858+0000","last_undegraded":"2024-09-15T21:48:36.858858+0000","last_fullsized":"2024-09-15T21:48:36.858858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:03:39.562977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726908+0000","last_change":"2024-09-15T21:48:21.642201+0000","last_active":"2024-09-15T21:48:36.726908+0000","last_peered":"2024-09-15T21:48:36.726908+0000","last_clean":"2024-09-15T21:48:36.726908+0000","last_became_active":"2024-09-15T21:48:21.641876+0000","last_became_peered":"2024-09-15T21:48:21.641876+0000","last_unstale":"2024-09-15T21:48:36.726908+0000","last_undegraded":"2024-09-15T21:48:36.726908+0000","last_fullsized":"2024-09-15T21:48:36.726908+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:41:59.764563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021785+0000","last_change":"2024-09-15T21:48:21.638398+0000","last_active":"2024-09-15T21:48:37.021785+0000","last_peered":"2024-09-15T21:48:37.021785+0000","last_clean":"2024-09-15T21:48:37.021785+0000","last_became_active":"2024-09-15T21:48:21.638073+0000","last_became_peered":"2024-09-15T21:48:21.638073+0000","last_unstale":"2024-09-15T21:48:37.021785+0000","last_undegraded":"2024-09-15T21:48:37.021785+0000","last_fullsized":"2024-09-15T21:48:37.021785+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:45.308950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":118,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710487+0000","last_change":"2024-09-15T21:46:23.431527+0000","last_active":"2024-09-15T21:48:35.710487+0000","last_peered":"2024-09-15T21:48:35.710487+0000","last_clean":"2024-09-15T21:48:35.710487+0000","last_became_active":"2024-09-15T21:46:23.430719+0000","last_became_peered":"2024-09-15T21:46:23.430719+0000","last_unstale":"2024-09-15T21:48:35.710487+0000","last_undegraded":"2024-09-15T21:48:35.710487+0000","last_fullsized":"2024-09-15T21:48:35.710487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:16.326529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021766+0000","last_change":"2024-09-15T21:48:17.622578+0000","last_active":"2024-09-15T21:48:37.021766+0000","last_peered":"2024-09-15T21:48:37.021766+0000","last_clean":"2024-09-15T21:48:37.021766+0000","last_became_active":"2024-09-15T21:48:17.621639+0000","last_became_peered":"2024-09-15T21:48:17.621639+0000","last_unstale":"2024-09-15T21:48:37.021766+0000","last_undegraded":"2024-09-15T21:48:37.021766+0000","last_fullsized":"2024-09-15T21:48:37.021766+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:57.099730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":118,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710554+0000","last_change":"2024-09-15T21:46:23.432646+0000","last_active":"2024-09-15T21:48:35.710554+0000","last_peered":"2024-09-15T21:48:35.710554+0000","last_clean":"2024-09-15T21:48:35.710554+0000","last_became_active":"2024-09-15T21:46:23.432317+0000","last_became_peered":"2024-09-15T21:46:23.432317+0000","last_unstale":"2024-09-15T21:48:35.710554+0000","last_undegraded":"2024-09-15T21:48:35.710554+0000","last_fullsized":"2024-09-15T21:48:35.710554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:49:07.706450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859042+0000","last_change":"2024-09-15T21:48:21.638549+0000","last_active":"2024-09-15T21:48:36.859042+0000","last_peered":"2024-09-15T21:48:36.859042+0000","last_clean":"2024-09-15T21:48:36.859042+0000","last_became_active":"2024-09-15T21:48:21.638212+0000","last_became_peered":"2024-09-15T21:48:21.638212+0000","last_unstale":"2024-09-15T21:48:36.859042+0000","last_undegraded":"2024-09-15T21:48:36.859042+0000","last_fullsized":"2024-09-15T21:48:36.859042+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:18.788661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726213+0000","last_change":"2024-09-15T21:46:23.421749+0000","last_active":"2024-09-15T21:48:36.726213+0000","last_peered":"2024-09-15T21:48:36.726213+0000","last_clean":"2024-09-15T21:48:36.726213+0000","last_became_active":"2024-09-15T21:46:23.421501+0000","last_became_peered":"2024-09-15T21:46:23.421501+0000","last_unstale":"2024-09-15T21:48:36.726213+0000","last_undegraded":"2024-09-15T21:48:36.726213+0000","last_fullsized":"2024-09-15T21:48:36.726213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:55.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":118,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710719+0000","last_change":"2024-09-15T21:46:23.432591+0000","last_active":"2024-09-15T21:48:35.710719+0000","last_peered":"2024-09-15T21:48:35.710719+0000","last_clean":"2024-09-15T21:48:35.710719+0000","last_became_active":"2024-09-15T21:46:23.432326+0000","last_became_peered":"2024-09-15T21:46:23.432326+0000","last_unstale":"2024-09-15T21:48:35.710719+0000","last_undegraded":"2024-09-15T21:48:35.710719+0000","last_fullsized":"2024-09-15T21:48:35.710719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:48.565579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.727013+0000","last_change":"2024-09-15T21:46:23.423428+0000","last_active":"2024-09-15T21:48:36.727013+0000","last_peered":"2024-09-15T21:48:36.727013+0000","last_clean":"2024-09-15T21:48:36.727013+0000","last_became_active":"2024-09-15T21:46:23.422734+0000","last_became_peered":"2024-09-15T21:46:23.422734+0000","last_unstale":"2024-09-15T21:48:36.727013+0000","last_undegraded":"2024-09-15T21:48:36.727013+0000","last_fullsized":"2024-09-15T21:48:36.727013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:06.592891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021899+0000","last_change":"2024-09-15T21:48:17.622914+0000","last_active":"2024-09-15T21:48:37.021899+0000","last_peered":"2024-09-15T21:48:37.021899+0000","last_clean":"2024-09-15T21:48:37.021899+0000","last_became_active":"2024-09-15T21:48:17.621473+0000","last_became_peered":"2024-09-15T21:48:17.621473+0000","last_unstale":"2024-09-15T21:48:37.021899+0000","last_undegraded":"2024-09-15T21:48:37.021899+0000","last_fullsized":"2024-09-15T21:48:37.021899+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:45.297321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850684+0000","last_change":"2024-09-15T21:48:17.616723+0000","last_active":"2024-09-15T21:48:36.850684+0000","last_peered":"2024-09-15T21:48:36.850684+0000","last_clean":"2024-09-15T21:48:36.850684+0000","last_became_active":"2024-09-15T21:48:17.616499+0000","last_became_peered":"2024-09-15T21:48:17.616499+0000","last_unstale":"2024-09-15T21:48:36.850684+0000","last_undegraded":"2024-09-15T21:48:36.850684+0000","last_fullsized":"2024-09-15T21:48:36.850684+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:29:36.886350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858637+0000","last_change":"2024-09-15T21:46:23.429648+0000","last_active":"2024-09-15T21:48:36.858637+0000","last_peered":"2024-09-15T21:48:36.858637+0000","last_clean":"2024-09-15T21:48:36.858637+0000","last_became_active":"2024-09-15T21:46:23.429376+0000","last_became_peered":"2024-09-15T21:46:23.429376+0000","last_unstale":"2024-09-15T21:48:36.858637+0000","last_undegraded":"2024-09-15T21:48:36.858637+0000","last_fullsized":"2024-09-15T21:48:36.858637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:31.278283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725723+0000","last_change":"2024-09-15T21:46:23.429847+0000","last_active":"2024-09-15T21:48:36.725723+0000","last_peered":"2024-09-15T21:48:36.725723+0000","last_clean":"2024-09-15T21:48:36.725723+0000","last_became_active":"2024-09-15T21:46:23.429442+0000","last_became_peered":"2024-09-15T21:46:23.429442+0000","last_unstale":"2024-09-15T21:48:36.725723+0000","last_undegraded":"2024-09-15T21:48:36.725723+0000","last_fullsized":"2024-09-15T21:48:36.725723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:31.232512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'6","reported_seq":147,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710932+0000","last_change":"2024-09-15T21:48:21.665708+0000","last_active":"2024-09-15T21:48:35.710932+0000","last_peered":"2024-09-15T21:48:35.710932+0000","last_clean":"2024-09-15T21:48:35.710932+0000","last_became_active":"2024-09-15T21:48:21.636972+0000","last_became_peered":"2024-09-15T21:48:21.636972+0000","last_unstale":"2024-09-15T21:48:35.710932+0000","last_undegraded":"2024-09-15T21:48:35.710932+0000","last_fullsized":"2024-09-15T21:48:35.710932+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:24:33.658234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1756,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":117,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.727058+0000","last_change":"2024-09-15T21:46:23.434198+0000","last_active":"2024-09-15T21:48:36.727058+0000","last_peered":"2024-09-15T21:48:36.727058+0000","last_clean":"2024-09-15T21:48:36.727058+0000","last_became_active":"2024-09-15T21:46:23.433944+0000","last_became_peered":"2024-09-15T21:46:23.433944+0000","last_unstale":"2024-09-15T21:48:36.727058+0000","last_undegraded":"2024-09-15T21:48:36.727058+0000","last_fullsized":"2024-09-15T21:48:36.727058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:25.412167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'7","reported_seq":126,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858780+0000","last_change":"2024-09-15T21:46:22.425571+0000","last_active":"2024-09-15T21:48:36.858780+0000","last_peered":"2024-09-15T21:48:36.858780+0000","last_clean":"2024-09-15T21:48:36.858780+0000","last_became_active":"2024-09-15T21:46:22.425315+0000","last_became_peered":"2024-09-15T21:46:22.425315+0000","last_unstale":"2024-09-15T21:48:36.858780+0000","last_undegraded":"2024-09-15T21:48:36.858780+0000","last_fullsized":"2024-09-15T21:48:36.858780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:31.983301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'3","reported_seq":122,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710544+0000","last_change":"2024-09-15T21:46:22.425492+0000","last_active":"2024-09-15T21:48:35.710544+0000","last_peered":"2024-09-15T21:48:35.710544+0000","last_clean":"2024-09-15T21:48:35.710544+0000","last_became_active":"2024-09-15T21:46:22.425226+0000","last_became_peered":"2024-09-15T21:46:22.425226+0000","last_unstale":"2024-09-15T21:48:35.710544+0000","last_undegraded":"2024-09-15T21:48:35.710544+0000","last_fullsized":"2024-09-15T21:48:35.710544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:24.322384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":117,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710324+0000","last_change":"2024-09-15T21:46:23.429140+0000","last_active":"2024-09-15T21:48:35.710324+0000","last_peered":"2024-09-15T21:48:35.710324+0000","last_clean":"2024-09-15T21:48:35.710324+0000","last_became_active":"2024-09-15T21:46:23.428902+0000","last_became_peered":"2024-09-15T21:46:23.428902+0000","last_unstale":"2024-09-15T21:48:35.710324+0000","last_undegraded":"2024-09-15T21:48:35.710324+0000","last_fullsized":"2024-09-15T21:48:35.710324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:11.256924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'3","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021677+0000","last_change":"2024-09-15T21:48:17.618127+0000","last_active":"2024-09-15T21:48:37.021677+0000","last_peered":"2024-09-15T21:48:37.021677+0000","last_clean":"2024-09-15T21:48:37.021677+0000","last_became_active":"2024-09-15T21:48:17.616704+0000","last_became_peered":"2024-09-15T21:48:17.616704+0000","last_unstale":"2024-09-15T21:48:37.021677+0000","last_undegraded":"2024-09-15T21:48:37.021677+0000","last_fullsized":"2024-09-15T21:48:37.021677+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:09.775586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725761+0000","last_change":"2024-09-15T21:46:23.427442+0000","last_active":"2024-09-15T21:48:36.725761+0000","last_peered":"2024-09-15T21:48:36.725761+0000","last_clean":"2024-09-15T21:48:36.725761+0000","last_became_active":"2024-09-15T21:46:23.427200+0000","last_became_peered":"2024-09-15T21:46:23.427200+0000","last_unstale":"2024-09-15T21:48:36.725761+0000","last_undegraded":"2024-09-15T21:48:36.725761+0000","last_fullsized":"2024-09-15T21:48:36.725761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:27.248545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"55'6","reported_seq":133,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859137+0000","last_change":"2024-09-15T21:48:17.622236+0000","last_active":"2024-09-15T21:48:36.859137+0000","last_peered":"2024-09-15T21:48:36.859137+0000","last_clean":"2024-09-15T21:48:36.859137+0000","last_became_active":"2024-09-15T21:48:17.621860+0000","last_became_peered":"2024-09-15T21:48:17.621860+0000","last_unstale":"2024-09-15T21:48:36.859137+0000","last_undegraded":"2024-09-15T21:48:36.859137+0000","last_fullsized":"2024-09-15T21:48:36.859137+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:00.032847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022154+0000","last_change":"2024-09-15T21:48:21.638639+0000","last_active":"2024-09-15T21:48:37.022154+0000","last_peered":"2024-09-15T21:48:37.022154+0000","last_clean":"2024-09-15T21:48:37.022154+0000","last_became_active":"2024-09-15T21:48:21.638300+0000","last_became_peered":"2024-09-15T21:48:21.638300+0000","last_unstale":"2024-09-15T21:48:37.022154+0000","last_undegraded":"2024-09-15T21:48:37.022154+0000","last_fullsized":"2024-09-15T21:48:37.022154+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:09:01.739184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'4","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022156+0000","last_change":"2024-09-15T21:48:17.623971+0000","last_active":"2024-09-15T21:48:37.022156+0000","last_peered":"2024-09-15T21:48:37.022156+0000","last_clean":"2024-09-15T21:48:37.022156+0000","last_became_active":"2024-09-15T21:48:17.623246+0000","last_became_peered":"2024-09-15T21:48:17.623246+0000","last_unstale":"2024-09-15T21:48:37.022156+0000","last_undegraded":"2024-09-15T21:48:37.022156+0000","last_fullsized":"2024-09-15T21:48:37.022156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:51:37.738581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726718+0000","last_change":"2024-09-15T21:48:17.625879+0000","last_active":"2024-09-15T21:48:36.726718+0000","last_peered":"2024-09-15T21:48:36.726718+0000","last_clean":"2024-09-15T21:48:36.726718+0000","last_became_active":"2024-09-15T21:48:17.625642+0000","last_became_peered":"2024-09-15T21:48:17.625642+0000","last_unstale":"2024-09-15T21:48:36.726718+0000","last_undegraded":"2024-09-15T21:48:36.726718+0000","last_fullsized":"2024-09-15T21:48:36.726718+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:41.396163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"55'7","reported_seq":139,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726301+0000","last_change":"2024-09-15T21:48:17.627957+0000","last_active":"2024-09-15T21:48:36.726301+0000","last_peered":"2024-09-15T21:48:36.726301+0000","last_clean":"2024-09-15T21:48:36.726301+0000","last_became_active":"2024-09-15T21:48:17.622409+0000","last_became_peered":"2024-09-15T21:48:17.622409+0000","last_unstale":"2024-09-15T21:48:36.726301+0000","last_undegraded":"2024-09-15T21:48:36.726301+0000","last_fullsized":"2024-09-15T21:48:36.726301+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:39.534804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":130,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726177+0000","last_change":"2024-09-15T21:48:21.637218+0000","last_active":"2024-09-15T21:48:36.726177+0000","last_peered":"2024-09-15T21:48:36.726177+0000","last_clean":"2024-09-15T21:48:36.726177+0000","last_became_active":"2024-09-15T21:48:21.636913+0000","last_became_peered":"2024-09-15T21:48:21.636913+0000","last_unstale":"2024-09-15T21:48:36.726177+0000","last_undegraded":"2024-09-15T21:48:36.726177+0000","last_fullsized":"2024-09-15T21:48:36.726177+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:36:05.649762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"55'24","reported_seq":143,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858914+0000","last_change":"2024-09-15T21:46:22.426151+0000","last_active":"2024-09-15T21:48:36.858914+0000","last_peered":"2024-09-15T21:48:36.858914+0000","last_clean":"2024-09-15T21:48:36.858914+0000","last_became_active":"2024-09-15T21:46:22.425866+0000","last_became_peered":"2024-09-15T21:46:22.425866+0000","last_unstale":"2024-09-15T21:48:36.858914+0000","last_undegraded":"2024-09-15T21:48:36.858914+0000","last_fullsized":"2024-09-15T21:48:36.858914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:10.684265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35573,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710799+0000","last_change":"2024-09-15T21:48:21.637466+0000","last_active":"2024-09-15T21:48:35.710799+0000","last_peered":"2024-09-15T21:48:35.710799+0000","last_clean":"2024-09-15T21:48:35.710799+0000","last_became_active":"2024-09-15T21:48:21.637151+0000","last_became_peered":"2024-09-15T21:48:21.637151+0000","last_unstale":"2024-09-15T21:48:35.710799+0000","last_undegraded":"2024-09-15T21:48:35.710799+0000","last_fullsized":"2024-09-15T21:48:35.710799+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:22:57.287004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'9","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.727151+0000","last_change":"2024-09-15T21:46:22.427415+0000","last_active":"2024-09-15T21:48:36.727151+0000","last_peered":"2024-09-15T21:48:36.727151+0000","last_clean":"2024-09-15T21:48:36.727151+0000","last_became_active":"2024-09-15T21:46:22.427087+0000","last_became_peered":"2024-09-15T21:46:22.427087+0000","last_unstale":"2024-09-15T21:48:36.727151+0000","last_undegraded":"2024-09-15T21:48:36.727151+0000","last_fullsized":"2024-09-15T21:48:36.727151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:09.365202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":127,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726487+0000","last_change":"2024-09-15T21:48:21.635299+0000","last_active":"2024-09-15T21:48:36.726487+0000","last_peered":"2024-09-15T21:48:36.726487+0000","last_clean":"2024-09-15T21:48:36.726487+0000","last_became_active":"2024-09-15T21:48:21.635022+0000","last_became_peered":"2024-09-15T21:48:21.635022+0000","last_unstale":"2024-09-15T21:48:36.726487+0000","last_undegraded":"2024-09-15T21:48:36.726487+0000","last_fullsized":"2024-09-15T21:48:36.726487+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:01.482588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"55'6","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021757+0000","last_change":"2024-09-15T21:48:17.625975+0000","last_active":"2024-09-15T21:48:37.021757+0000","last_peered":"2024-09-15T21:48:37.021757+0000","last_clean":"2024-09-15T21:48:37.021757+0000","last_became_active":"2024-09-15T21:48:17.625232+0000","last_became_peered":"2024-09-15T21:48:17.625232+0000","last_unstale":"2024-09-15T21:48:37.021757+0000","last_undegraded":"2024-09-15T21:48:37.021757+0000","last_fullsized":"2024-09-15T21:48:37.021757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:21.061733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710557+0000","last_change":"2024-09-15T21:48:21.639654+0000","last_active":"2024-09-15T21:48:35.710557+0000","last_peered":"2024-09-15T21:48:35.710557+0000","last_clean":"2024-09-15T21:48:35.710557+0000","last_became_active":"2024-09-15T21:48:21.638780+0000","last_became_peered":"2024-09-15T21:48:21.638780+0000","last_unstale":"2024-09-15T21:48:35.710557+0000","last_undegraded":"2024-09-15T21:48:35.710557+0000","last_fullsized":"2024-09-15T21:48:35.710557+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:16.549734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'5","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858678+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:48:36.858678+0000","last_peered":"2024-09-15T21:48:36.858678+0000","last_clean":"2024-09-15T21:48:36.858678+0000","last_became_active":"2024-09-15T21:46:22.425237+0000","last_became_peered":"2024-09-15T21:46:22.425237+0000","last_unstale":"2024-09-15T21:48:36.858678+0000","last_undegraded":"2024-09-15T21:48:36.858678+0000","last_fullsized":"2024-09-15T21:48:36.858678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:59.460420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":117,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858649+0000","last_change":"2024-09-15T21:46:23.427515+0000","last_active":"2024-09-15T21:48:36.858649+0000","last_peered":"2024-09-15T21:48:36.858649+0000","last_clean":"2024-09-15T21:48:36.858649+0000","last_became_active":"2024-09-15T21:46:23.427116+0000","last_became_peered":"2024-09-15T21:46:23.427116+0000","last_unstale":"2024-09-15T21:48:36.858649+0000","last_undegraded":"2024-09-15T21:48:36.858649+0000","last_fullsized":"2024-09-15T21:48:36.858649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:35.499970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"55'18","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850613+0000","last_change":"2024-09-15T21:46:22.419905+0000","last_active":"2024-09-15T21:48:36.850613+0000","last_peered":"2024-09-15T21:48:36.850613+0000","last_clean":"2024-09-15T21:48:36.850613+0000","last_became_active":"2024-09-15T21:46:22.419620+0000","last_became_peered":"2024-09-15T21:46:22.419620+0000","last_unstale":"2024-09-15T21:48:36.850613+0000","last_undegraded":"2024-09-15T21:48:36.850613+0000","last_fullsized":"2024-09-15T21:48:36.850613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:26:51.215156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710002+0000","last_change":"2024-09-15T21:48:21.639146+0000","last_active":"2024-09-15T21:48:35.710002+0000","last_peered":"2024-09-15T21:48:35.710002+0000","last_clean":"2024-09-15T21:48:35.710002+0000","last_became_active":"2024-09-15T21:48:21.637855+0000","last_became_peered":"2024-09-15T21:48:21.637855+0000","last_unstale":"2024-09-15T21:48:35.710002+0000","last_undegraded":"2024-09-15T21:48:35.710002+0000","last_fullsized":"2024-09-15T21:48:35.710002+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:35:29.305298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'2","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726069+0000","last_change":"2024-09-15T21:46:22.429661+0000","last_active":"2024-09-15T21:48:36.726069+0000","last_peered":"2024-09-15T21:48:36.726069+0000","last_clean":"2024-09-15T21:48:36.726069+0000","last_became_active":"2024-09-15T21:46:22.429250+0000","last_became_peered":"2024-09-15T21:46:22.429250+0000","last_unstale":"2024-09-15T21:48:36.726069+0000","last_undegraded":"2024-09-15T21:48:36.726069+0000","last_fullsized":"2024-09-15T21:48:36.726069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:29.341739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710978+0000","last_change":"2024-09-15T21:48:21.635384+0000","last_active":"2024-09-15T21:48:35.710978+0000","last_peered":"2024-09-15T21:48:35.710978+0000","last_clean":"2024-09-15T21:48:35.710978+0000","last_became_active":"2024-09-15T21:48:21.635051+0000","last_became_peered":"2024-09-15T21:48:21.635051+0000","last_unstale":"2024-09-15T21:48:35.710978+0000","last_undegraded":"2024-09-15T21:48:35.710978+0000","last_fullsized":"2024-09-15T21:48:35.710978+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:59.199568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'5","reported_seq":83,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850941+0000","last_change":"2024-09-15T21:48:27.660076+0000","last_active":"2024-09-15T21:48:36.850941+0000","last_peered":"2024-09-15T21:48:36.850941+0000","last_clean":"2024-09-15T21:48:36.850941+0000","last_became_active":"2024-09-15T21:48:27.659750+0000","last_became_peered":"2024-09-15T21:48:27.659750+0000","last_unstale":"2024-09-15T21:48:36.850941+0000","last_undegraded":"2024-09-15T21:48:36.850941+0000","last_fullsized":"2024-09-15T21:48:36.850941+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:06.505112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.711325+0000","last_change":"2024-09-15T21:48:21.641556+0000","last_active":"2024-09-15T21:48:35.711325+0000","last_peered":"2024-09-15T21:48:35.711325+0000","last_clean":"2024-09-15T21:48:35.711325+0000","last_became_active":"2024-09-15T21:48:21.639328+0000","last_became_peered":"2024-09-15T21:48:21.639328+0000","last_unstale":"2024-09-15T21:48:35.711325+0000","last_undegraded":"2024-09-15T21:48:35.711325+0000","last_fullsized":"2024-09-15T21:48:35.711325+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:36:11.470048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'3","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726437+0000","last_change":"2024-09-15T21:46:22.425072+0000","last_active":"2024-09-15T21:48:36.726437+0000","last_peered":"2024-09-15T21:48:36.726437+0000","last_clean":"2024-09-15T21:48:36.726437+0000","last_became_active":"2024-09-15T21:46:22.424771+0000","last_became_peered":"2024-09-15T21:46:22.424771+0000","last_unstale":"2024-09-15T21:48:36.726437+0000","last_undegraded":"2024-09-15T21:48:36.726437+0000","last_fullsized":"2024-09-15T21:48:36.726437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:26.256785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022156+0000","last_change":"2024-09-15T21:48:17.619698+0000","last_active":"2024-09-15T21:48:37.022156+0000","last_peered":"2024-09-15T21:48:37.022156+0000","last_clean":"2024-09-15T21:48:37.022156+0000","last_became_active":"2024-09-15T21:48:17.617701+0000","last_became_peered":"2024-09-15T21:48:17.617701+0000","last_unstale":"2024-09-15T21:48:37.022156+0000","last_undegraded":"2024-09-15T21:48:37.022156+0000","last_fullsized":"2024-09-15T21:48:37.022156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:00.170687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'5","reported_seq":132,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850794+0000","last_change":"2024-09-15T21:48:17.619089+0000","last_active":"2024-09-15T21:48:36.850794+0000","last_peered":"2024-09-15T21:48:36.850794+0000","last_clean":"2024-09-15T21:48:36.850794+0000","last_became_active":"2024-09-15T21:48:17.618782+0000","last_became_peered":"2024-09-15T21:48:17.618782+0000","last_unstale":"2024-09-15T21:48:36.850794+0000","last_undegraded":"2024-09-15T21:48:36.850794+0000","last_fullsized":"2024-09-15T21:48:36.850794+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:17.881528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710339+0000","last_change":"2024-09-15T21:48:21.639356+0000","last_active":"2024-09-15T21:48:35.710339+0000","last_peered":"2024-09-15T21:48:35.710339+0000","last_clean":"2024-09-15T21:48:35.710339+0000","last_became_active":"2024-09-15T21:48:21.637883+0000","last_became_peered":"2024-09-15T21:48:21.637883+0000","last_unstale":"2024-09-15T21:48:35.710339+0000","last_undegraded":"2024-09-15T21:48:35.710339+0000","last_fullsized":"2024-09-15T21:48:35.710339+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:27.061996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":236,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710002+0000","last_change":"2024-09-15T21:48:24.647129+0000","last_active":"2024-09-15T21:48:35.710002+0000","last_peered":"2024-09-15T21:48:35.710002+0000","last_clean":"2024-09-15T21:48:35.710002+0000","last_became_active":"2024-09-15T21:48:24.646783+0000","last_became_peered":"2024-09-15T21:48:24.646783+0000","last_unstale":"2024-09-15T21:48:35.710002+0000","last_undegraded":"2024-09-15T21:48:35.710002+0000","last_fullsized":"2024-09-15T21:48:35.710002+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_clean_scrub_stamp":"2024-09-15T21:46:12.223400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:20.467980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'6","reported_seq":68,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021854+0000","last_change":"2024-09-15T21:48:17.622619+0000","last_active":"2024-09-15T21:48:37.021854+0000","last_peered":"2024-09-15T21:48:37.021854+0000","last_clean":"2024-09-15T21:48:37.021854+0000","last_became_active":"2024-09-15T21:48:17.622159+0000","last_became_peered":"2024-09-15T21:48:17.622159+0000","last_unstale":"2024-09-15T21:48:37.021854+0000","last_undegraded":"2024-09-15T21:48:37.021854+0000","last_fullsized":"2024-09-15T21:48:37.021854+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:51.566702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":125,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710677+0000","last_change":"2024-09-15T21:48:17.621009+0000","last_active":"2024-09-15T21:48:35.710677+0000","last_peered":"2024-09-15T21:48:35.710677+0000","last_clean":"2024-09-15T21:48:35.710677+0000","last_became_active":"2024-09-15T21:48:17.620738+0000","last_became_peered":"2024-09-15T21:48:17.620738+0000","last_unstale":"2024-09-15T21:48:35.710677+0000","last_undegraded":"2024-09-15T21:48:35.710677+0000","last_fullsized":"2024-09-15T21:48:35.710677+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:20.095876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'4","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710013+0000","last_change":"2024-09-15T21:48:21.645645+0000","last_active":"2024-09-15T21:48:35.710013+0000","last_peered":"2024-09-15T21:48:35.710013+0000","last_clean":"2024-09-15T21:48:35.710013+0000","last_became_active":"2024-09-15T21:48:21.636886+0000","last_became_peered":"2024-09-15T21:48:21.636886+0000","last_unstale":"2024-09-15T21:48:35.710013+0000","last_undegraded":"2024-09-15T21:48:35.710013+0000","last_fullsized":"2024-09-15T21:48:35.710013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:12.676508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726029+0000","last_change":"2024-09-15T21:46:23.430396+0000","last_active":"2024-09-15T21:48:36.726029+0000","last_peered":"2024-09-15T21:48:36.726029+0000","last_clean":"2024-09-15T21:48:36.726029+0000","last_became_active":"2024-09-15T21:46:23.430138+0000","last_became_peered":"2024-09-15T21:46:23.430138+0000","last_unstale":"2024-09-15T21:48:36.726029+0000","last_undegraded":"2024-09-15T21:48:36.726029+0000","last_fullsized":"2024-09-15T21:48:36.726029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:50.622949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"55'7","reported_seq":80,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850706+0000","last_change":"2024-09-15T21:48:27.660216+0000","last_active":"2024-09-15T21:48:36.850706+0000","last_peered":"2024-09-15T21:48:36.850706+0000","last_clean":"2024-09-15T21:48:36.850706+0000","last_became_active":"2024-09-15T21:48:27.659880+0000","last_became_peered":"2024-09-15T21:48:27.659880+0000","last_unstale":"2024-09-15T21:48:36.850706+0000","last_undegraded":"2024-09-15T21:48:36.850706+0000","last_fullsized":"2024-09-15T21:48:36.850706+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:48.186260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":516,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725858+0000","last_change":"2024-09-15T21:46:23.427051+0000","last_active":"2024-09-15T21:48:36.725858+0000","last_peered":"2024-09-15T21:48:36.725858+0000","last_clean":"2024-09-15T21:48:36.725858+0000","last_became_active":"2024-09-15T21:46:23.426185+0000","last_became_peered":"2024-09-15T21:46:23.426185+0000","last_unstale":"2024-09-15T21:48:36.725858+0000","last_undegraded":"2024-09-15T21:48:36.725858+0000","last_fullsized":"2024-09-15T21:48:36.725858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:48.318405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'5","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021901+0000","last_change":"2024-09-15T21:48:17.625702+0000","last_active":"2024-09-15T21:48:37.021901+0000","last_peered":"2024-09-15T21:48:37.021901+0000","last_clean":"2024-09-15T21:48:37.021901+0000","last_became_active":"2024-09-15T21:48:17.623936+0000","last_became_peered":"2024-09-15T21:48:17.623936+0000","last_unstale":"2024-09-15T21:48:37.021901+0000","last_undegraded":"2024-09-15T21:48:37.021901+0000","last_fullsized":"2024-09-15T21:48:37.021901+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:53.114831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":121,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725863+0000","last_change":"2024-09-15T21:46:23.425699+0000","last_active":"2024-09-15T21:48:36.725863+0000","last_peered":"2024-09-15T21:48:36.725863+0000","last_clean":"2024-09-15T21:48:36.725863+0000","last_became_active":"2024-09-15T21:46:23.425416+0000","last_became_peered":"2024-09-15T21:46:23.425416+0000","last_unstale":"2024-09-15T21:48:36.725863+0000","last_undegraded":"2024-09-15T21:48:36.725863+0000","last_fullsized":"2024-09-15T21:48:36.725863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:00.325179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"22'1","reported_seq":122,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726645+0000","last_change":"2024-09-15T21:46:22.426492+0000","last_active":"2024-09-15T21:48:36.726645+0000","last_peered":"2024-09-15T21:48:36.726645+0000","last_clean":"2024-09-15T21:48:36.726645+0000","last_became_active":"2024-09-15T21:46:22.426210+0000","last_became_peered":"2024-09-15T21:46:22.426210+0000","last_unstale":"2024-09-15T21:48:36.726645+0000","last_undegraded":"2024-09-15T21:48:36.726645+0000","last_fullsized":"2024-09-15T21:48:36.726645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:05.248091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022064+0000","last_change":"2024-09-15T21:48:17.625501+0000","last_active":"2024-09-15T21:48:37.022064+0000","last_peered":"2024-09-15T21:48:37.022064+0000","last_clean":"2024-09-15T21:48:37.022064+0000","last_became_active":"2024-09-15T21:48:17.624852+0000","last_became_peered":"2024-09-15T21:48:17.624852+0000","last_unstale":"2024-09-15T21:48:37.022064+0000","last_undegraded":"2024-09-15T21:48:37.022064+0000","last_fullsized":"2024-09-15T21:48:37.022064+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:47:09.731108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"22'4","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710548+0000","last_change":"2024-09-15T21:48:21.668174+0000","last_active":"2024-09-15T21:48:35.710548+0000","last_peered":"2024-09-15T21:48:35.710548+0000","last_clean":"2024-09-15T21:48:35.710548+0000","last_became_active":"2024-09-15T21:48:21.639593+0000","last_became_peered":"2024-09-15T21:48:21.639593+0000","last_unstale":"2024-09-15T21:48:35.710548+0000","last_undegraded":"2024-09-15T21:48:35.710548+0000","last_fullsized":"2024-09-15T21:48:35.710548+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:05.341807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725928+0000","last_change":"2024-09-15T21:46:23.426729+0000","last_active":"2024-09-15T21:48:36.725928+0000","last_peered":"2024-09-15T21:48:36.725928+0000","last_clean":"2024-09-15T21:48:36.725928+0000","last_became_active":"2024-09-15T21:46:23.426495+0000","last_became_peered":"2024-09-15T21:46:23.426495+0000","last_unstale":"2024-09-15T21:48:36.725928+0000","last_undegraded":"2024-09-15T21:48:36.725928+0000","last_fullsized":"2024-09-15T21:48:36.725928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:03.277997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"55'16","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726226+0000","last_change":"2024-09-15T21:46:22.427137+0000","last_active":"2024-09-15T21:48:36.726226+0000","last_peered":"2024-09-15T21:48:36.726226+0000","last_clean":"2024-09-15T21:48:36.726226+0000","last_became_active":"2024-09-15T21:46:22.426855+0000","last_became_peered":"2024-09-15T21:46:22.426855+0000","last_unstale":"2024-09-15T21:48:36.726226+0000","last_undegraded":"2024-09-15T21:48:36.726226+0000","last_fullsized":"2024-09-15T21:48:36.726226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:40:14.277940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":128,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710689+0000","last_change":"2024-09-15T21:48:21.639860+0000","last_active":"2024-09-15T21:48:35.710689+0000","last_peered":"2024-09-15T21:48:35.710689+0000","last_clean":"2024-09-15T21:48:35.710689+0000","last_became_active":"2024-09-15T21:48:21.639541+0000","last_became_peered":"2024-09-15T21:48:21.639541+0000","last_unstale":"2024-09-15T21:48:35.710689+0000","last_undegraded":"2024-09-15T21:48:35.710689+0000","last_fullsized":"2024-09-15T21:48:35.710689+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:21.797034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"55'8","reported_seq":158,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859109+0000","last_change":"2024-09-15T21:48:21.687672+0000","last_active":"2024-09-15T21:48:36.859109+0000","last_peered":"2024-09-15T21:48:36.859109+0000","last_clean":"2024-09-15T21:48:36.859109+0000","last_became_active":"2024-09-15T21:48:21.643251+0000","last_became_peered":"2024-09-15T21:48:21.643251+0000","last_unstale":"2024-09-15T21:48:36.859109+0000","last_undegraded":"2024-09-15T21:48:36.859109+0000","last_fullsized":"2024-09-15T21:48:36.859109+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:40.697954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":44,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726888+0000","last_change":"2024-09-15T21:46:23.424266+0000","last_active":"2024-09-15T21:48:36.726888+0000","last_peered":"2024-09-15T21:48:36.726888+0000","last_clean":"2024-09-15T21:48:36.726888+0000","last_became_active":"2024-09-15T21:46:23.424007+0000","last_became_peered":"2024-09-15T21:46:23.424007+0000","last_unstale":"2024-09-15T21:48:36.726888+0000","last_undegraded":"2024-09-15T21:48:36.726888+0000","last_fullsized":"2024-09-15T21:48:36.726888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:23.492253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'2","reported_seq":135,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726262+0000","last_change":"2024-09-15T21:48:17.623800+0000","last_active":"2024-09-15T21:48:36.726262+0000","last_peered":"2024-09-15T21:48:36.726262+0000","last_clean":"2024-09-15T21:48:36.726262+0000","last_became_active":"2024-09-15T21:48:17.623453+0000","last_became_peered":"2024-09-15T21:48:17.623453+0000","last_unstale":"2024-09-15T21:48:36.726262+0000","last_undegraded":"2024-09-15T21:48:36.726262+0000","last_fullsized":"2024-09-15T21:48:36.726262+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:46:38.897647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":121,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725974+0000","last_change":"2024-09-15T21:46:23.430836+0000","last_active":"2024-09-15T21:48:36.725974+0000","last_peered":"2024-09-15T21:48:36.725974+0000","last_clean":"2024-09-15T21:48:36.725974+0000","last_became_active":"2024-09-15T21:46:23.429283+0000","last_became_peered":"2024-09-15T21:46:23.429283+0000","last_unstale":"2024-09-15T21:48:36.725974+0000","last_undegraded":"2024-09-15T21:48:36.725974+0000","last_fullsized":"2024-09-15T21:48:36.725974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:30.571376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'7","reported_seq":158,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726054+0000","last_change":"2024-09-15T21:48:21.660402+0000","last_active":"2024-09-15T21:48:36.726054+0000","last_peered":"2024-09-15T21:48:36.726054+0000","last_clean":"2024-09-15T21:48:36.726054+0000","last_became_active":"2024-09-15T21:48:21.635898+0000","last_became_peered":"2024-09-15T21:48:21.635898+0000","last_unstale":"2024-09-15T21:48:36.726054+0000","last_undegraded":"2024-09-15T21:48:36.726054+0000","last_fullsized":"2024-09-15T21:48:36.726054+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:29.173360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":180,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":118,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710424+0000","last_change":"2024-09-15T21:46:23.428634+0000","last_active":"2024-09-15T21:48:35.710424+0000","last_peered":"2024-09-15T21:48:35.710424+0000","last_clean":"2024-09-15T21:48:35.710424+0000","last_became_active":"2024-09-15T21:46:23.428301+0000","last_became_peered":"2024-09-15T21:46:23.428301+0000","last_unstale":"2024-09-15T21:48:35.710424+0000","last_undegraded":"2024-09-15T21:48:35.710424+0000","last_fullsized":"2024-09-15T21:48:35.710424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:59.492942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'4","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726625+0000","last_change":"2024-09-15T21:46:22.425810+0000","last_active":"2024-09-15T21:48:36.726625+0000","last_peered":"2024-09-15T21:48:36.726625+0000","last_clean":"2024-09-15T21:48:36.726625+0000","last_became_active":"2024-09-15T21:46:22.425478+0000","last_became_peered":"2024-09-15T21:46:22.425478+0000","last_unstale":"2024-09-15T21:48:36.726625+0000","last_undegraded":"2024-09-15T21:48:36.726625+0000","last_fullsized":"2024-09-15T21:48:36.726625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:53.296599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":118,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710948+0000","last_change":"2024-09-15T21:46:23.426054+0000","last_active":"2024-09-15T21:48:35.710948+0000","last_peered":"2024-09-15T21:48:35.710948+0000","last_clean":"2024-09-15T21:48:35.710948+0000","last_became_active":"2024-09-15T21:46:23.425547+0000","last_became_peered":"2024-09-15T21:46:23.425547+0000","last_unstale":"2024-09-15T21:48:35.710948+0000","last_undegraded":"2024-09-15T21:48:35.710948+0000","last_fullsized":"2024-09-15T21:48:35.710948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:56.838503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"55'5","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710664+0000","last_change":"2024-09-15T21:48:21.653804+0000","last_active":"2024-09-15T21:48:35.710664+0000","last_peered":"2024-09-15T21:48:35.710664+0000","last_clean":"2024-09-15T21:48:35.710664+0000","last_became_active":"2024-09-15T21:48:21.639223+0000","last_became_peered":"2024-09-15T21:48:21.639223+0000","last_unstale":"2024-09-15T21:48:35.710664+0000","last_undegraded":"2024-09-15T21:48:35.710664+0000","last_fullsized":"2024-09-15T21:48:35.710664+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:47:15.330440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858680+0000","last_change":"2024-09-15T21:46:23.434719+0000","last_active":"2024-09-15T21:48:36.858680+0000","last_peered":"2024-09-15T21:48:36.858680+0000","last_clean":"2024-09-15T21:48:36.858680+0000","last_became_active":"2024-09-15T21:46:23.434448+0000","last_became_peered":"2024-09-15T21:46:23.434448+0000","last_unstale":"2024-09-15T21:48:36.858680+0000","last_undegraded":"2024-09-15T21:48:36.858680+0000","last_fullsized":"2024-09-15T21:48:36.858680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:14.901671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":68,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850752+0000","last_change":"2024-09-15T21:48:27.657040+0000","last_active":"2024-09-15T21:48:36.850752+0000","last_peered":"2024-09-15T21:48:36.850752+0000","last_clean":"2024-09-15T21:48:36.850752+0000","last_became_active":"2024-09-15T21:48:27.656753+0000","last_became_peered":"2024-09-15T21:48:27.656753+0000","last_unstale":"2024-09-15T21:48:36.850752+0000","last_undegraded":"2024-09-15T21:48:36.850752+0000","last_fullsized":"2024-09-15T21:48:36.850752+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:14.458723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'5","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710454+0000","last_change":"2024-09-15T21:48:21.691408+0000","last_active":"2024-09-15T21:48:35.710454+0000","last_peered":"2024-09-15T21:48:35.710454+0000","last_clean":"2024-09-15T21:48:35.710454+0000","last_became_active":"2024-09-15T21:48:21.641517+0000","last_became_peered":"2024-09-15T21:48:21.641517+0000","last_unstale":"2024-09-15T21:48:35.710454+0000","last_undegraded":"2024-09-15T21:48:35.710454+0000","last_fullsized":"2024-09-15T21:48:35.710454+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:09.454433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022007+0000","last_change":"2024-09-15T21:48:17.623223+0000","last_active":"2024-09-15T21:48:37.022007+0000","last_peered":"2024-09-15T21:48:37.022007+0000","last_clean":"2024-09-15T21:48:37.022007+0000","last_became_active":"2024-09-15T21:48:17.622019+0000","last_became_peered":"2024-09-15T21:48:17.622019+0000","last_unstale":"2024-09-15T21:48:37.022007+0000","last_undegraded":"2024-09-15T21:48:37.022007+0000","last_fullsized":"2024-09-15T21:48:37.022007+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:39:57.643793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"55'13","reported_seq":53,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710695+0000","last_change":"2024-09-15T21:48:21.682609+0000","last_active":"2024-09-15T21:48:35.710695+0000","last_peered":"2024-09-15T21:48:35.710695+0000","last_clean":"2024-09-15T21:48:35.710695+0000","last_became_active":"2024-09-15T21:48:21.640156+0000","last_became_peered":"2024-09-15T21:48:21.640156+0000","last_unstale":"2024-09-15T21:48:35.710695+0000","last_undegraded":"2024-09-15T21:48:35.710695+0000","last_fullsized":"2024-09-15T21:48:35.710695+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:28.936509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726585+0000","last_change":"2024-09-15T21:48:17.623164+0000","last_active":"2024-09-15T21:48:36.726585+0000","last_peered":"2024-09-15T21:48:36.726585+0000","last_clean":"2024-09-15T21:48:36.726585+0000","last_became_active":"2024-09-15T21:48:17.622843+0000","last_became_peered":"2024-09-15T21:48:17.622843+0000","last_unstale":"2024-09-15T21:48:36.726585+0000","last_undegraded":"2024-09-15T21:48:36.726585+0000","last_fullsized":"2024-09-15T21:48:36.726585+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:06:30.745189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'2","reported_seq":122,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710725+0000","last_change":"2024-09-15T21:46:22.428529+0000","last_active":"2024-09-15T21:48:35.710725+0000","last_peered":"2024-09-15T21:48:35.710725+0000","last_clean":"2024-09-15T21:48:35.710725+0000","last_became_active":"2024-09-15T21:46:22.428263+0000","last_became_peered":"2024-09-15T21:46:22.428263+0000","last_unstale":"2024-09-15T21:48:35.710725+0000","last_undegraded":"2024-09-15T21:48:35.710725+0000","last_fullsized":"2024-09-15T21:48:35.710725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:22:40.160062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":118,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710647+0000","last_change":"2024-09-15T21:46:23.427968+0000","last_active":"2024-09-15T21:48:35.710647+0000","last_peered":"2024-09-15T21:48:35.710647+0000","last_clean":"2024-09-15T21:48:35.710647+0000","last_became_active":"2024-09-15T21:46:23.427546+0000","last_became_peered":"2024-09-15T21:46:23.427546+0000","last_unstale":"2024-09-15T21:48:35.710647+0000","last_undegraded":"2024-09-15T21:48:35.710647+0000","last_fullsized":"2024-09-15T21:48:35.710647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:47.226097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'13","reported_seq":171,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726869+0000","last_change":"2024-09-15T21:48:21.680908+0000","last_active":"2024-09-15T21:48:36.726869+0000","last_peered":"2024-09-15T21:48:36.726869+0000","last_clean":"2024-09-15T21:48:36.726869+0000","last_became_active":"2024-09-15T21:48:21.642406+0000","last_became_peered":"2024-09-15T21:48:21.642406+0000","last_unstale":"2024-09-15T21:48:36.726869+0000","last_undegraded":"2024-09-15T21:48:36.726869+0000","last_fullsized":"2024-09-15T21:48:36.726869+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:23:26.323380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1344,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":42655,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":243,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":4802,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":335872,"data_stored":85310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6368,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476769,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1507328,"data_stored":1002267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1503232,"data_stored":997684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509475,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":946176,"data_stored":447388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":57,"seq":244813135878,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29072,"kb_used_data":888,"kb_used_omap":7,"kb_used_meta":28152,"kb_avail":94342768,"statfs":{"total":96636764160,"available":96606994432,"internally_reserved":0,"allocated":909312,"data_stored":402827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7963,"internal_metadata":28827877},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509474,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":909312,"data_stored":412355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509473,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":921600,"data_stored":412891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":888832,"data_stored":400427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":942080,"data_stored":442017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":37057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6368,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":36363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:48:40.298 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:48:40.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.430+0000 7f2cf235f640 1 Processor -- start 2024-09-15T21:48:40.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.430+0000 7f2cf235f640 1 -- start start 2024-09-15T21:48:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.431+0000 7f2cf235f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2cec153490 0x7f2cec164880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.431+0000 7f2cf235f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cec0c8510 con 0x7f2cec073b80 2024-09-15T21:48:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.431+0000 7f2cf235f640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2cec159cb0 con 0x7f2cec0c9a50 2024-09-15T21:48:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.431+0000 7f2cf235f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f2cec0c82c0 con 0x7f2cec153490 2024-09-15T21:48:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.431+0000 7f2cf08fc640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2cec153490 0x7f2cec164880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.431+0000 7f2cebfff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f2cec073b80 0x7f2cec152d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42144/0 (socket says 172.21.15.100:42144) 2024-09-15T21:48:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.431+0000 7f2cebfff640 1 -- 172.21.15.100:0/3909582258 learned_addr learned my addr 172.21.15.100:0/3909582258 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.431+0000 7f2cf08fc640 1 -- 172.21.15.100:0/3909582258 >> v1:172.21.15.122:6789/0 conn(0x7f2cec0c9a50 legacy=0x7f2cec15c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:40.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.431+0000 7f2cf08fc640 1 -- 172.21.15.100:0/3909582258 >> v1:172.21.15.100:6789/0 conn(0x7f2cec073b80 legacy=0x7f2cec152d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:40.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.431+0000 7f2cf08fc640 1 -- 172.21.15.100:0/3909582258 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cec06c520 con 0x7f2cec153490 2024-09-15T21:48:40.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.432+0000 7f2cf08fc640 1 --2- 172.21.15.100:0/3909582258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cec153490 0x7f2cec164880 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009d90 tx=0x7f2ce402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2db037dc61b4113 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:40.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.432+0000 7f2ceaffd640 1 -- 172.21.15.100:0/3909582258 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4007740 con 0x7f2cec153490 2024-09-15T21:48:40.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.432+0000 7f2ceaffd640 1 -- 172.21.15.100:0/3909582258 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ce40078e0 con 0x7f2cec153490 2024-09-15T21:48:40.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.432+0000 7f2ceaffd640 1 -- 172.21.15.100:0/3909582258 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4007cb0 con 0x7f2cec153490 2024-09-15T21:48:40.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.432+0000 7f2cf235f640 1 -- 172.21.15.100:0/3909582258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cec153490 msgr2=0x7f2cec164880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:40.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.432+0000 7f2cf235f640 1 --2- 172.21.15.100:0/3909582258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cec153490 0x7f2cec164880 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009d90 tx=0x7f2ce402a730 comp rx=0 tx=0).stop 2024-09-15T21:48:40.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.433+0000 7f2cf235f640 1 -- 172.21.15.100:0/3909582258 shutdown_connections 2024-09-15T21:48:40.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.433+0000 7f2cf235f640 1 --2- 172.21.15.100:0/3909582258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cec153490 0x7f2cec164880 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.433+0000 7f2cf235f640 1 -- 172.21.15.100:0/3909582258 >> 172.21.15.100:0/3909582258 conn(0x7f2cec06d890 msgr2=0x7f2cec06bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:40.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.433+0000 7f2cf235f640 1 -- 172.21.15.100:0/3909582258 shutdown_connections 2024-09-15T21:48:40.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.433+0000 7f2cf235f640 1 -- 172.21.15.100:0/3909582258 wait complete. 2024-09-15T21:48:40.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.434+0000 7f2cf235f640 1 Processor -- start 2024-09-15T21:48:40.438 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.434+0000 7f2cf235f640 1 -- start start 2024-09-15T21:48:40.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2cf235f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2cec073b80 0x7f2cec0c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2cf235f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cec0c9a50 0x7f2cec15b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2cf235f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cec153490 0x7f2cec1ef3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2cf235f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec0c78c0 con 0x7f2cec0c9a50 2024-09-15T21:48:40.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2cf235f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2cec06c890 con 0x7f2cec153490 2024-09-15T21:48:40.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2cf235f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2cec0c7a40 con 0x7f2cec073b80 2024-09-15T21:48:40.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2ceb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cec0c9a50 0x7f2cec15b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2ceb7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cec0c9a50 0x7f2cec15b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54418/0 (socket says 172.21.15.100:54418) 2024-09-15T21:48:40.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2ceb7fe640 1 -- 172.21.15.100:0/3038342045 learned_addr learned my addr 172.21.15.100:0/3038342045 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:40.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2cf08fc640 1 --2- 172.21.15.100:0/3038342045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cec153490 0x7f2cec1ef3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2cebfff640 1 --2- 172.21.15.100:0/3038342045 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2cec073b80 0x7f2cec0c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2ceb7fe640 1 -- 172.21.15.100:0/3038342045 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2cec073b80 msgr2=0x7f2cec0c2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:40.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2ceb7fe640 1 --2- 172.21.15.100:0/3038342045 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2cec073b80 0x7f2cec0c2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2ceb7fe640 1 -- 172.21.15.100:0/3038342045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cec153490 msgr2=0x7f2cec1ef3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:40.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2ceb7fe640 1 --2- 172.21.15.100:0/3038342045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cec153490 0x7f2cec1ef3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2ceb7fe640 1 -- 172.21.15.100:0/3038342045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cec1f2eb0 con 0x7f2cec0c9a50 2024-09-15T21:48:40.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2ceb7fe640 1 --2- 172.21.15.100:0/3038342045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cec0c9a50 0x7f2cec15b790 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000b420 tx=0x7f2ce000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:40.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2ce8ff9640 1 -- 172.21.15.100:0/3038342045 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0011830 con 0x7f2cec0c9a50 2024-09-15T21:48:40.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2ce8ff9640 1 -- 172.21.15.100:0/3038342045 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ce0016070 con 0x7f2cec0c9a50 2024-09-15T21:48:40.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2cf235f640 1 -- 172.21.15.100:0/3038342045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cec1f3170 con 0x7f2cec0c9a50 2024-09-15T21:48:40.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2ce8ff9640 1 -- 172.21.15.100:0/3038342045 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0011c10 con 0x7f2cec0c9a50 2024-09-15T21:48:40.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.435+0000 7f2cf235f640 1 -- 172.21.15.100:0/3038342045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cec14d230 con 0x7f2cec0c9a50 2024-09-15T21:48:40.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.436+0000 7f2cf235f640 1 -- 172.21.15.100:0/3038342045 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cec15a680 con 0x7f2cec0c9a50 2024-09-15T21:48:40.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.437+0000 7f2ce8ff9640 1 -- 172.21.15.100:0/3038342045 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2ce001b460 con 0x7f2cec0c9a50 2024-09-15T21:48:40.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.437+0000 7f2ce8ff9640 1 --2- 172.21.15.100:0/3038342045 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2cd806c2b0 0x7f2cd806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.437+0000 7f2ce8ff9640 1 -- 172.21.15.100:0/3038342045 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f2ce008afe0 con 0x7f2cec0c9a50 2024-09-15T21:48:40.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.437+0000 7f2cebfff640 1 --2- 172.21.15.100:0/3038342045 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2cd806c2b0 0x7f2cd806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.438+0000 7f2cebfff640 1 --2- 172.21.15.100:0/3038342045 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2cd806c2b0 0x7f2cd806e770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc000f60 tx=0x7f2cdc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:40.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.441+0000 7f2ce8ff9640 1 -- 172.21.15.100:0/3038342045 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2cec15a680 con 0x7f2cec0c9a50 2024-09-15T21:48:40.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.606+0000 7f2cf235f640 1 -- 172.21.15.100:0/3038342045 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2cec073f60 con 0x7f2cd806c2b0 2024-09-15T21:48:40.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.611+0000 7f2ce8ff9640 1 -- 172.21.15.100:0/3038342045 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252302 (secure 0 0 0) 0x7f2cec073f60 con 0x7f2cd806c2b0 2024-09-15T21:48:40.617 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:48:40.618 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-15T21:48:40.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.614+0000 7f2cf235f640 1 -- 172.21.15.100:0/3038342045 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2cd806c2b0 msgr2=0x7f2cd806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:40.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.614+0000 7f2cf235f640 1 --2- 172.21.15.100:0/3038342045 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2cd806c2b0 0x7f2cd806e770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc000f60 tx=0x7f2cdc009040 comp rx=0 tx=0).stop 2024-09-15T21:48:40.619 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.614+0000 7f2cf235f640 1 -- 172.21.15.100:0/3038342045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cec0c9a50 msgr2=0x7f2cec15b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:40.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.614+0000 7f2cf235f640 1 --2- 172.21.15.100:0/3038342045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cec0c9a50 0x7f2cec15b790 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000b420 tx=0x7f2ce000b8e0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.614+0000 7f2cf235f640 1 -- 172.21.15.100:0/3038342045 shutdown_connections 2024-09-15T21:48:40.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.614+0000 7f2cf235f640 1 --2- 172.21.15.100:0/3038342045 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2cd806c2b0 0x7f2cd806e770 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.614+0000 7f2cf235f640 1 --2- 172.21.15.100:0/3038342045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2cec153490 0x7f2cec1ef3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.615+0000 7f2cf235f640 1 --2- 172.21.15.100:0/3038342045 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2cec0c9a50 0x7f2cec15b790 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.615+0000 7f2cf235f640 1 --2- 172.21.15.100:0/3038342045 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2cec073b80 0x7f2cec0c2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.615+0000 7f2cf235f640 1 -- 172.21.15.100:0/3038342045 >> 172.21.15.100:0/3038342045 conn(0x7f2cec06d890 msgr2=0x7f2cec157da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:40.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.615+0000 7f2cf235f640 1 -- 172.21.15.100:0/3038342045 shutdown_connections 2024-09-15T21:48:40.620 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.615+0000 7f2cf235f640 1 -- 172.21.15.100:0/3038342045 wait complete. 2024-09-15T21:48:40.628 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-15T21:48:39.832625+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633023,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":404,"num_write_kb":2533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1185538,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":335,"ondisk_log_size":335,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238220,"kb_used_data":8328,"kb_used_omap":18,"kb_used_meta":229677,"kb_avail":754736500,"statfs":{"total":773094113280,"available":772850176000,"internally_reserved":0,"allocated":8527872,"data_stored":4517856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19119,"internal_metadata":235189585},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001127"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.711284+0000","last_change":"2024-09-15T21:48:21.639361+0000","last_active":"2024-09-15T21:48:35.711284+0000","last_peered":"2024-09-15T21:48:35.711284+0000","last_clean":"2024-09-15T21:48:35.711284+0000","last_became_active":"2024-09-15T21:48:21.637402+0000","last_became_peered":"2024-09-15T21:48:21.637402+0000","last_unstale":"2024-09-15T21:48:35.711284+0000","last_undegraded":"2024-09-15T21:48:35.711284+0000","last_fullsized":"2024-09-15T21:48:35.711284+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:40.643861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726138+0000","last_change":"2024-09-15T21:46:23.418716+0000","last_active":"2024-09-15T21:48:36.726138+0000","last_peered":"2024-09-15T21:48:36.726138+0000","last_clean":"2024-09-15T21:48:36.726138+0000","last_became_active":"2024-09-15T21:46:23.418471+0000","last_became_peered":"2024-09-15T21:46:23.418471+0000","last_unstale":"2024-09-15T21:48:36.726138+0000","last_undegraded":"2024-09-15T21:48:36.726138+0000","last_fullsized":"2024-09-15T21:48:36.726138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:49.599801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726536+0000","last_change":"2024-09-15T21:46:23.426404+0000","last_active":"2024-09-15T21:48:36.726536+0000","last_peered":"2024-09-15T21:48:36.726536+0000","last_clean":"2024-09-15T21:48:36.726536+0000","last_became_active":"2024-09-15T21:46:23.426146+0000","last_became_peered":"2024-09-15T21:46:23.426146+0000","last_unstale":"2024-09-15T21:48:36.726536+0000","last_undegraded":"2024-09-15T21:48:36.726536+0000","last_fullsized":"2024-09-15T21:48:36.726536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:23.626109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022058+0000","last_change":"2024-09-15T21:48:17.624363+0000","last_active":"2024-09-15T21:48:37.022058+0000","last_peered":"2024-09-15T21:48:37.022058+0000","last_clean":"2024-09-15T21:48:37.022058+0000","last_became_active":"2024-09-15T21:48:17.623606+0000","last_became_peered":"2024-09-15T21:48:17.623606+0000","last_unstale":"2024-09-15T21:48:37.022058+0000","last_undegraded":"2024-09-15T21:48:37.022058+0000","last_fullsized":"2024-09-15T21:48:37.022058+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:03.294215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710447+0000","last_change":"2024-09-15T21:48:21.640215+0000","last_active":"2024-09-15T21:48:35.710447+0000","last_peered":"2024-09-15T21:48:35.710447+0000","last_clean":"2024-09-15T21:48:35.710447+0000","last_became_active":"2024-09-15T21:48:21.638410+0000","last_became_peered":"2024-09-15T21:48:21.638410+0000","last_unstale":"2024-09-15T21:48:35.710447+0000","last_undegraded":"2024-09-15T21:48:35.710447+0000","last_fullsized":"2024-09-15T21:48:35.710447+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:43.167624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022015+0000","last_change":"2024-09-15T21:48:17.625573+0000","last_active":"2024-09-15T21:48:37.022015+0000","last_peered":"2024-09-15T21:48:37.022015+0000","last_clean":"2024-09-15T21:48:37.022015+0000","last_became_active":"2024-09-15T21:48:17.624439+0000","last_became_peered":"2024-09-15T21:48:17.624439+0000","last_unstale":"2024-09-15T21:48:37.022015+0000","last_undegraded":"2024-09-15T21:48:37.022015+0000","last_fullsized":"2024-09-15T21:48:37.022015+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:04.063675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710549+0000","last_change":"2024-09-15T21:48:21.638612+0000","last_active":"2024-09-15T21:48:35.710549+0000","last_peered":"2024-09-15T21:48:35.710549+0000","last_clean":"2024-09-15T21:48:35.710549+0000","last_became_active":"2024-09-15T21:48:21.638310+0000","last_became_peered":"2024-09-15T21:48:21.638310+0000","last_unstale":"2024-09-15T21:48:35.710549+0000","last_undegraded":"2024-09-15T21:48:35.710549+0000","last_fullsized":"2024-09-15T21:48:35.710549+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:37.297103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":99,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850506+0000","last_change":"2024-09-15T21:48:27.657047+0000","last_active":"2024-09-15T21:48:36.850506+0000","last_peered":"2024-09-15T21:48:36.850506+0000","last_clean":"2024-09-15T21:48:36.850506+0000","last_became_active":"2024-09-15T21:48:27.656800+0000","last_became_peered":"2024-09-15T21:48:27.656800+0000","last_unstale":"2024-09-15T21:48:36.850506+0000","last_undegraded":"2024-09-15T21:48:36.850506+0000","last_fullsized":"2024-09-15T21:48:36.850506+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:17.342504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.711186+0000","last_change":"2024-09-15T21:48:21.642296+0000","last_active":"2024-09-15T21:48:35.711186+0000","last_peered":"2024-09-15T21:48:35.711186+0000","last_clean":"2024-09-15T21:48:35.711186+0000","last_became_active":"2024-09-15T21:48:21.641920+0000","last_became_peered":"2024-09-15T21:48:21.641920+0000","last_unstale":"2024-09-15T21:48:35.711186+0000","last_undegraded":"2024-09-15T21:48:35.711186+0000","last_fullsized":"2024-09-15T21:48:35.711186+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:45.342068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859014+0000","last_change":"2024-09-15T21:48:17.622439+0000","last_active":"2024-09-15T21:48:36.859014+0000","last_peered":"2024-09-15T21:48:36.859014+0000","last_clean":"2024-09-15T21:48:36.859014+0000","last_became_active":"2024-09-15T21:48:17.621800+0000","last_became_peered":"2024-09-15T21:48:17.621800+0000","last_unstale":"2024-09-15T21:48:36.859014+0000","last_undegraded":"2024-09-15T21:48:36.859014+0000","last_fullsized":"2024-09-15T21:48:36.859014+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:07.312656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":122,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858947+0000","last_change":"2024-09-15T21:47:05.735261+0000","last_active":"2024-09-15T21:48:36.858947+0000","last_peered":"2024-09-15T21:48:36.858947+0000","last_clean":"2024-09-15T21:48:36.858947+0000","last_became_active":"2024-09-15T21:47:05.734921+0000","last_became_peered":"2024-09-15T21:47:05.734921+0000","last_unstale":"2024-09-15T21:48:36.858947+0000","last_undegraded":"2024-09-15T21:48:36.858947+0000","last_fullsized":"2024-09-15T21:48:36.858947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:55.787926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726086+0000","last_change":"2024-09-15T21:46:23.432534+0000","last_active":"2024-09-15T21:48:36.726086+0000","last_peered":"2024-09-15T21:48:36.726086+0000","last_clean":"2024-09-15T21:48:36.726086+0000","last_became_active":"2024-09-15T21:46:23.430170+0000","last_became_peered":"2024-09-15T21:46:23.430170+0000","last_unstale":"2024-09-15T21:48:36.726086+0000","last_undegraded":"2024-09-15T21:48:36.726086+0000","last_fullsized":"2024-09-15T21:48:36.726086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:11.463937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":117,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710817+0000","last_change":"2024-09-15T21:46:23.427841+0000","last_active":"2024-09-15T21:48:35.710817+0000","last_peered":"2024-09-15T21:48:35.710817+0000","last_clean":"2024-09-15T21:48:35.710817+0000","last_became_active":"2024-09-15T21:46:23.427621+0000","last_became_peered":"2024-09-15T21:46:23.427621+0000","last_unstale":"2024-09-15T21:48:35.710817+0000","last_undegraded":"2024-09-15T21:48:35.710817+0000","last_fullsized":"2024-09-15T21:48:35.710817+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:18.331943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"55'3","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726331+0000","last_change":"2024-09-15T21:46:23.424707+0000","last_active":"2024-09-15T21:48:36.726331+0000","last_peered":"2024-09-15T21:48:36.726331+0000","last_clean":"2024-09-15T21:48:36.726331+0000","last_became_active":"2024-09-15T21:46:23.424447+0000","last_became_peered":"2024-09-15T21:46:23.424447+0000","last_unstale":"2024-09-15T21:48:36.726331+0000","last_undegraded":"2024-09-15T21:48:36.726331+0000","last_fullsized":"2024-09-15T21:48:36.726331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:23.617214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725904+0000","last_change":"2024-09-15T21:46:23.430297+0000","last_active":"2024-09-15T21:48:36.725904+0000","last_peered":"2024-09-15T21:48:36.725904+0000","last_clean":"2024-09-15T21:48:36.725904+0000","last_became_active":"2024-09-15T21:46:23.430070+0000","last_became_peered":"2024-09-15T21:46:23.430070+0000","last_unstale":"2024-09-15T21:48:36.725904+0000","last_undegraded":"2024-09-15T21:48:36.725904+0000","last_fullsized":"2024-09-15T21:48:36.725904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:03:19.615022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021835+0000","last_change":"2024-09-15T21:48:17.625113+0000","last_active":"2024-09-15T21:48:37.021835+0000","last_peered":"2024-09-15T21:48:37.021835+0000","last_clean":"2024-09-15T21:48:37.021835+0000","last_became_active":"2024-09-15T21:48:17.624764+0000","last_became_peered":"2024-09-15T21:48:17.624764+0000","last_unstale":"2024-09-15T21:48:37.021835+0000","last_undegraded":"2024-09-15T21:48:37.021835+0000","last_fullsized":"2024-09-15T21:48:37.021835+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:28:44.142358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022276+0000","last_change":"2024-09-15T21:48:21.642646+0000","last_active":"2024-09-15T21:48:37.022276+0000","last_peered":"2024-09-15T21:48:37.022276+0000","last_clean":"2024-09-15T21:48:37.022276+0000","last_became_active":"2024-09-15T21:48:21.642303+0000","last_became_peered":"2024-09-15T21:48:21.642303+0000","last_unstale":"2024-09-15T21:48:37.022276+0000","last_undegraded":"2024-09-15T21:48:37.022276+0000","last_fullsized":"2024-09-15T21:48:37.022276+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:08:54.511282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":75,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850781+0000","last_change":"2024-09-15T21:48:27.661631+0000","last_active":"2024-09-15T21:48:36.850781+0000","last_peered":"2024-09-15T21:48:36.850781+0000","last_clean":"2024-09-15T21:48:36.850781+0000","last_became_active":"2024-09-15T21:48:27.661441+0000","last_became_peered":"2024-09-15T21:48:27.661441+0000","last_unstale":"2024-09-15T21:48:36.850781+0000","last_undegraded":"2024-09-15T21:48:36.850781+0000","last_fullsized":"2024-09-15T21:48:36.850781+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:23.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710445+0000","last_change":"2024-09-15T21:48:21.639602+0000","last_active":"2024-09-15T21:48:35.710445+0000","last_peered":"2024-09-15T21:48:35.710445+0000","last_clean":"2024-09-15T21:48:35.710445+0000","last_became_active":"2024-09-15T21:48:21.639009+0000","last_became_peered":"2024-09-15T21:48:21.639009+0000","last_unstale":"2024-09-15T21:48:35.710445+0000","last_undegraded":"2024-09-15T21:48:35.710445+0000","last_fullsized":"2024-09-15T21:48:35.710445+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:16.818783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858858+0000","last_change":"2024-09-15T21:46:23.434558+0000","last_active":"2024-09-15T21:48:36.858858+0000","last_peered":"2024-09-15T21:48:36.858858+0000","last_clean":"2024-09-15T21:48:36.858858+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:48:36.858858+0000","last_undegraded":"2024-09-15T21:48:36.858858+0000","last_fullsized":"2024-09-15T21:48:36.858858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:03:39.562977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726908+0000","last_change":"2024-09-15T21:48:21.642201+0000","last_active":"2024-09-15T21:48:36.726908+0000","last_peered":"2024-09-15T21:48:36.726908+0000","last_clean":"2024-09-15T21:48:36.726908+0000","last_became_active":"2024-09-15T21:48:21.641876+0000","last_became_peered":"2024-09-15T21:48:21.641876+0000","last_unstale":"2024-09-15T21:48:36.726908+0000","last_undegraded":"2024-09-15T21:48:36.726908+0000","last_fullsized":"2024-09-15T21:48:36.726908+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:41:59.764563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021785+0000","last_change":"2024-09-15T21:48:21.638398+0000","last_active":"2024-09-15T21:48:37.021785+0000","last_peered":"2024-09-15T21:48:37.021785+0000","last_clean":"2024-09-15T21:48:37.021785+0000","last_became_active":"2024-09-15T21:48:21.638073+0000","last_became_peered":"2024-09-15T21:48:21.638073+0000","last_unstale":"2024-09-15T21:48:37.021785+0000","last_undegraded":"2024-09-15T21:48:37.021785+0000","last_fullsized":"2024-09-15T21:48:37.021785+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:45.308950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":118,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710487+0000","last_change":"2024-09-15T21:46:23.431527+0000","last_active":"2024-09-15T21:48:35.710487+0000","last_peered":"2024-09-15T21:48:35.710487+0000","last_clean":"2024-09-15T21:48:35.710487+0000","last_became_active":"2024-09-15T21:46:23.430719+0000","last_became_peered":"2024-09-15T21:46:23.430719+0000","last_unstale":"2024-09-15T21:48:35.710487+0000","last_undegraded":"2024-09-15T21:48:35.710487+0000","last_fullsized":"2024-09-15T21:48:35.710487+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:16.326529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021766+0000","last_change":"2024-09-15T21:48:17.622578+0000","last_active":"2024-09-15T21:48:37.021766+0000","last_peered":"2024-09-15T21:48:37.021766+0000","last_clean":"2024-09-15T21:48:37.021766+0000","last_became_active":"2024-09-15T21:48:17.621639+0000","last_became_peered":"2024-09-15T21:48:17.621639+0000","last_unstale":"2024-09-15T21:48:37.021766+0000","last_undegraded":"2024-09-15T21:48:37.021766+0000","last_fullsized":"2024-09-15T21:48:37.021766+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:57.099730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":118,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710554+0000","last_change":"2024-09-15T21:46:23.432646+0000","last_active":"2024-09-15T21:48:35.710554+0000","last_peered":"2024-09-15T21:48:35.710554+0000","last_clean":"2024-09-15T21:48:35.710554+0000","last_became_active":"2024-09-15T21:46:23.432317+0000","last_became_peered":"2024-09-15T21:46:23.432317+0000","last_unstale":"2024-09-15T21:48:35.710554+0000","last_undegraded":"2024-09-15T21:48:35.710554+0000","last_fullsized":"2024-09-15T21:48:35.710554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:49:07.706450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859042+0000","last_change":"2024-09-15T21:48:21.638549+0000","last_active":"2024-09-15T21:48:36.859042+0000","last_peered":"2024-09-15T21:48:36.859042+0000","last_clean":"2024-09-15T21:48:36.859042+0000","last_became_active":"2024-09-15T21:48:21.638212+0000","last_became_peered":"2024-09-15T21:48:21.638212+0000","last_unstale":"2024-09-15T21:48:36.859042+0000","last_undegraded":"2024-09-15T21:48:36.859042+0000","last_fullsized":"2024-09-15T21:48:36.859042+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:18.788661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726213+0000","last_change":"2024-09-15T21:46:23.421749+0000","last_active":"2024-09-15T21:48:36.726213+0000","last_peered":"2024-09-15T21:48:36.726213+0000","last_clean":"2024-09-15T21:48:36.726213+0000","last_became_active":"2024-09-15T21:46:23.421501+0000","last_became_peered":"2024-09-15T21:46:23.421501+0000","last_unstale":"2024-09-15T21:48:36.726213+0000","last_undegraded":"2024-09-15T21:48:36.726213+0000","last_fullsized":"2024-09-15T21:48:36.726213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:55.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":118,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710719+0000","last_change":"2024-09-15T21:46:23.432591+0000","last_active":"2024-09-15T21:48:35.710719+0000","last_peered":"2024-09-15T21:48:35.710719+0000","last_clean":"2024-09-15T21:48:35.710719+0000","last_became_active":"2024-09-15T21:46:23.432326+0000","last_became_peered":"2024-09-15T21:46:23.432326+0000","last_unstale":"2024-09-15T21:48:35.710719+0000","last_undegraded":"2024-09-15T21:48:35.710719+0000","last_fullsized":"2024-09-15T21:48:35.710719+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:48.565579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.727013+0000","last_change":"2024-09-15T21:46:23.423428+0000","last_active":"2024-09-15T21:48:36.727013+0000","last_peered":"2024-09-15T21:48:36.727013+0000","last_clean":"2024-09-15T21:48:36.727013+0000","last_became_active":"2024-09-15T21:46:23.422734+0000","last_became_peered":"2024-09-15T21:46:23.422734+0000","last_unstale":"2024-09-15T21:48:36.727013+0000","last_undegraded":"2024-09-15T21:48:36.727013+0000","last_fullsized":"2024-09-15T21:48:36.727013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:06.592891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021899+0000","last_change":"2024-09-15T21:48:17.622914+0000","last_active":"2024-09-15T21:48:37.021899+0000","last_peered":"2024-09-15T21:48:37.021899+0000","last_clean":"2024-09-15T21:48:37.021899+0000","last_became_active":"2024-09-15T21:48:17.621473+0000","last_became_peered":"2024-09-15T21:48:17.621473+0000","last_unstale":"2024-09-15T21:48:37.021899+0000","last_undegraded":"2024-09-15T21:48:37.021899+0000","last_fullsized":"2024-09-15T21:48:37.021899+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:45.297321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850684+0000","last_change":"2024-09-15T21:48:17.616723+0000","last_active":"2024-09-15T21:48:36.850684+0000","last_peered":"2024-09-15T21:48:36.850684+0000","last_clean":"2024-09-15T21:48:36.850684+0000","last_became_active":"2024-09-15T21:48:17.616499+0000","last_became_peered":"2024-09-15T21:48:17.616499+0000","last_unstale":"2024-09-15T21:48:36.850684+0000","last_undegraded":"2024-09-15T21:48:36.850684+0000","last_fullsized":"2024-09-15T21:48:36.850684+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:29:36.886350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858637+0000","last_change":"2024-09-15T21:46:23.429648+0000","last_active":"2024-09-15T21:48:36.858637+0000","last_peered":"2024-09-15T21:48:36.858637+0000","last_clean":"2024-09-15T21:48:36.858637+0000","last_became_active":"2024-09-15T21:46:23.429376+0000","last_became_peered":"2024-09-15T21:46:23.429376+0000","last_unstale":"2024-09-15T21:48:36.858637+0000","last_undegraded":"2024-09-15T21:48:36.858637+0000","last_fullsized":"2024-09-15T21:48:36.858637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:31.278283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725723+0000","last_change":"2024-09-15T21:46:23.429847+0000","last_active":"2024-09-15T21:48:36.725723+0000","last_peered":"2024-09-15T21:48:36.725723+0000","last_clean":"2024-09-15T21:48:36.725723+0000","last_became_active":"2024-09-15T21:46:23.429442+0000","last_became_peered":"2024-09-15T21:46:23.429442+0000","last_unstale":"2024-09-15T21:48:36.725723+0000","last_undegraded":"2024-09-15T21:48:36.725723+0000","last_fullsized":"2024-09-15T21:48:36.725723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:31.232512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'6","reported_seq":147,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710932+0000","last_change":"2024-09-15T21:48:21.665708+0000","last_active":"2024-09-15T21:48:35.710932+0000","last_peered":"2024-09-15T21:48:35.710932+0000","last_clean":"2024-09-15T21:48:35.710932+0000","last_became_active":"2024-09-15T21:48:21.636972+0000","last_became_peered":"2024-09-15T21:48:21.636972+0000","last_unstale":"2024-09-15T21:48:35.710932+0000","last_undegraded":"2024-09-15T21:48:35.710932+0000","last_fullsized":"2024-09-15T21:48:35.710932+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:24:33.658234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1756,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":117,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.727058+0000","last_change":"2024-09-15T21:46:23.434198+0000","last_active":"2024-09-15T21:48:36.727058+0000","last_peered":"2024-09-15T21:48:36.727058+0000","last_clean":"2024-09-15T21:48:36.727058+0000","last_became_active":"2024-09-15T21:46:23.433944+0000","last_became_peered":"2024-09-15T21:46:23.433944+0000","last_unstale":"2024-09-15T21:48:36.727058+0000","last_undegraded":"2024-09-15T21:48:36.727058+0000","last_fullsized":"2024-09-15T21:48:36.727058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:25.412167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'7","reported_seq":126,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858780+0000","last_change":"2024-09-15T21:46:22.425571+0000","last_active":"2024-09-15T21:48:36.858780+0000","last_peered":"2024-09-15T21:48:36.858780+0000","last_clean":"2024-09-15T21:48:36.858780+0000","last_became_active":"2024-09-15T21:46:22.425315+0000","last_became_peered":"2024-09-15T21:46:22.425315+0000","last_unstale":"2024-09-15T21:48:36.858780+0000","last_undegraded":"2024-09-15T21:48:36.858780+0000","last_fullsized":"2024-09-15T21:48:36.858780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:31.983301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'3","reported_seq":122,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710544+0000","last_change":"2024-09-15T21:46:22.425492+0000","last_active":"2024-09-15T21:48:35.710544+0000","last_peered":"2024-09-15T21:48:35.710544+0000","last_clean":"2024-09-15T21:48:35.710544+0000","last_became_active":"2024-09-15T21:46:22.425226+0000","last_became_peered":"2024-09-15T21:46:22.425226+0000","last_unstale":"2024-09-15T21:48:35.710544+0000","last_undegraded":"2024-09-15T21:48:35.710544+0000","last_fullsized":"2024-09-15T21:48:35.710544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:24.322384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":117,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710324+0000","last_change":"2024-09-15T21:46:23.429140+0000","last_active":"2024-09-15T21:48:35.710324+0000","last_peered":"2024-09-15T21:48:35.710324+0000","last_clean":"2024-09-15T21:48:35.710324+0000","last_became_active":"2024-09-15T21:46:23.428902+0000","last_became_peered":"2024-09-15T21:46:23.428902+0000","last_unstale":"2024-09-15T21:48:35.710324+0000","last_undegraded":"2024-09-15T21:48:35.710324+0000","last_fullsized":"2024-09-15T21:48:35.710324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:11.256924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'3","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021677+0000","last_change":"2024-09-15T21:48:17.618127+0000","last_active":"2024-09-15T21:48:37.021677+0000","last_peered":"2024-09-15T21:48:37.021677+0000","last_clean":"2024-09-15T21:48:37.021677+0000","last_became_active":"2024-09-15T21:48:17.616704+0000","last_became_peered":"2024-09-15T21:48:17.616704+0000","last_unstale":"2024-09-15T21:48:37.021677+0000","last_undegraded":"2024-09-15T21:48:37.021677+0000","last_fullsized":"2024-09-15T21:48:37.021677+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:09.775586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725761+0000","last_change":"2024-09-15T21:46:23.427442+0000","last_active":"2024-09-15T21:48:36.725761+0000","last_peered":"2024-09-15T21:48:36.725761+0000","last_clean":"2024-09-15T21:48:36.725761+0000","last_became_active":"2024-09-15T21:46:23.427200+0000","last_became_peered":"2024-09-15T21:46:23.427200+0000","last_unstale":"2024-09-15T21:48:36.725761+0000","last_undegraded":"2024-09-15T21:48:36.725761+0000","last_fullsized":"2024-09-15T21:48:36.725761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:27.248545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"55'6","reported_seq":133,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859137+0000","last_change":"2024-09-15T21:48:17.622236+0000","last_active":"2024-09-15T21:48:36.859137+0000","last_peered":"2024-09-15T21:48:36.859137+0000","last_clean":"2024-09-15T21:48:36.859137+0000","last_became_active":"2024-09-15T21:48:17.621860+0000","last_became_peered":"2024-09-15T21:48:17.621860+0000","last_unstale":"2024-09-15T21:48:36.859137+0000","last_undegraded":"2024-09-15T21:48:36.859137+0000","last_fullsized":"2024-09-15T21:48:36.859137+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:00.032847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022154+0000","last_change":"2024-09-15T21:48:21.638639+0000","last_active":"2024-09-15T21:48:37.022154+0000","last_peered":"2024-09-15T21:48:37.022154+0000","last_clean":"2024-09-15T21:48:37.022154+0000","last_became_active":"2024-09-15T21:48:21.638300+0000","last_became_peered":"2024-09-15T21:48:21.638300+0000","last_unstale":"2024-09-15T21:48:37.022154+0000","last_undegraded":"2024-09-15T21:48:37.022154+0000","last_fullsized":"2024-09-15T21:48:37.022154+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:09:01.739184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'4","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022156+0000","last_change":"2024-09-15T21:48:17.623971+0000","last_active":"2024-09-15T21:48:37.022156+0000","last_peered":"2024-09-15T21:48:37.022156+0000","last_clean":"2024-09-15T21:48:37.022156+0000","last_became_active":"2024-09-15T21:48:17.623246+0000","last_became_peered":"2024-09-15T21:48:17.623246+0000","last_unstale":"2024-09-15T21:48:37.022156+0000","last_undegraded":"2024-09-15T21:48:37.022156+0000","last_fullsized":"2024-09-15T21:48:37.022156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:51:37.738581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726718+0000","last_change":"2024-09-15T21:48:17.625879+0000","last_active":"2024-09-15T21:48:36.726718+0000","last_peered":"2024-09-15T21:48:36.726718+0000","last_clean":"2024-09-15T21:48:36.726718+0000","last_became_active":"2024-09-15T21:48:17.625642+0000","last_became_peered":"2024-09-15T21:48:17.625642+0000","last_unstale":"2024-09-15T21:48:36.726718+0000","last_undegraded":"2024-09-15T21:48:36.726718+0000","last_fullsized":"2024-09-15T21:48:36.726718+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:41.396163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"55'7","reported_seq":139,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726301+0000","last_change":"2024-09-15T21:48:17.627957+0000","last_active":"2024-09-15T21:48:36.726301+0000","last_peered":"2024-09-15T21:48:36.726301+0000","last_clean":"2024-09-15T21:48:36.726301+0000","last_became_active":"2024-09-15T21:48:17.622409+0000","last_became_peered":"2024-09-15T21:48:17.622409+0000","last_unstale":"2024-09-15T21:48:36.726301+0000","last_undegraded":"2024-09-15T21:48:36.726301+0000","last_fullsized":"2024-09-15T21:48:36.726301+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:39.534804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":130,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726177+0000","last_change":"2024-09-15T21:48:21.637218+0000","last_active":"2024-09-15T21:48:36.726177+0000","last_peered":"2024-09-15T21:48:36.726177+0000","last_clean":"2024-09-15T21:48:36.726177+0000","last_became_active":"2024-09-15T21:48:21.636913+0000","last_became_peered":"2024-09-15T21:48:21.636913+0000","last_unstale":"2024-09-15T21:48:36.726177+0000","last_undegraded":"2024-09-15T21:48:36.726177+0000","last_fullsized":"2024-09-15T21:48:36.726177+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:36:05.649762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"55'24","reported_seq":143,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858914+0000","last_change":"2024-09-15T21:46:22.426151+0000","last_active":"2024-09-15T21:48:36.858914+0000","last_peered":"2024-09-15T21:48:36.858914+0000","last_clean":"2024-09-15T21:48:36.858914+0000","last_became_active":"2024-09-15T21:46:22.425866+0000","last_became_peered":"2024-09-15T21:46:22.425866+0000","last_unstale":"2024-09-15T21:48:36.858914+0000","last_undegraded":"2024-09-15T21:48:36.858914+0000","last_fullsized":"2024-09-15T21:48:36.858914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:10.684265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35573,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710799+0000","last_change":"2024-09-15T21:48:21.637466+0000","last_active":"2024-09-15T21:48:35.710799+0000","last_peered":"2024-09-15T21:48:35.710799+0000","last_clean":"2024-09-15T21:48:35.710799+0000","last_became_active":"2024-09-15T21:48:21.637151+0000","last_became_peered":"2024-09-15T21:48:21.637151+0000","last_unstale":"2024-09-15T21:48:35.710799+0000","last_undegraded":"2024-09-15T21:48:35.710799+0000","last_fullsized":"2024-09-15T21:48:35.710799+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:22:57.287004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'9","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.727151+0000","last_change":"2024-09-15T21:46:22.427415+0000","last_active":"2024-09-15T21:48:36.727151+0000","last_peered":"2024-09-15T21:48:36.727151+0000","last_clean":"2024-09-15T21:48:36.727151+0000","last_became_active":"2024-09-15T21:46:22.427087+0000","last_became_peered":"2024-09-15T21:46:22.427087+0000","last_unstale":"2024-09-15T21:48:36.727151+0000","last_undegraded":"2024-09-15T21:48:36.727151+0000","last_fullsized":"2024-09-15T21:48:36.727151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:09.365202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":127,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726487+0000","last_change":"2024-09-15T21:48:21.635299+0000","last_active":"2024-09-15T21:48:36.726487+0000","last_peered":"2024-09-15T21:48:36.726487+0000","last_clean":"2024-09-15T21:48:36.726487+0000","last_became_active":"2024-09-15T21:48:21.635022+0000","last_became_peered":"2024-09-15T21:48:21.635022+0000","last_unstale":"2024-09-15T21:48:36.726487+0000","last_undegraded":"2024-09-15T21:48:36.726487+0000","last_fullsized":"2024-09-15T21:48:36.726487+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:01.482588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"55'6","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021757+0000","last_change":"2024-09-15T21:48:17.625975+0000","last_active":"2024-09-15T21:48:37.021757+0000","last_peered":"2024-09-15T21:48:37.021757+0000","last_clean":"2024-09-15T21:48:37.021757+0000","last_became_active":"2024-09-15T21:48:17.625232+0000","last_became_peered":"2024-09-15T21:48:17.625232+0000","last_unstale":"2024-09-15T21:48:37.021757+0000","last_undegraded":"2024-09-15T21:48:37.021757+0000","last_fullsized":"2024-09-15T21:48:37.021757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:21.061733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710557+0000","last_change":"2024-09-15T21:48:21.639654+0000","last_active":"2024-09-15T21:48:35.710557+0000","last_peered":"2024-09-15T21:48:35.710557+0000","last_clean":"2024-09-15T21:48:35.710557+0000","last_became_active":"2024-09-15T21:48:21.638780+0000","last_became_peered":"2024-09-15T21:48:21.638780+0000","last_unstale":"2024-09-15T21:48:35.710557+0000","last_undegraded":"2024-09-15T21:48:35.710557+0000","last_fullsized":"2024-09-15T21:48:35.710557+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:16.549734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'5","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858678+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:48:36.858678+0000","last_peered":"2024-09-15T21:48:36.858678+0000","last_clean":"2024-09-15T21:48:36.858678+0000","last_became_active":"2024-09-15T21:46:22.425237+0000","last_became_peered":"2024-09-15T21:46:22.425237+0000","last_unstale":"2024-09-15T21:48:36.858678+0000","last_undegraded":"2024-09-15T21:48:36.858678+0000","last_fullsized":"2024-09-15T21:48:36.858678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:59.460420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":117,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858649+0000","last_change":"2024-09-15T21:46:23.427515+0000","last_active":"2024-09-15T21:48:36.858649+0000","last_peered":"2024-09-15T21:48:36.858649+0000","last_clean":"2024-09-15T21:48:36.858649+0000","last_became_active":"2024-09-15T21:46:23.427116+0000","last_became_peered":"2024-09-15T21:46:23.427116+0000","last_unstale":"2024-09-15T21:48:36.858649+0000","last_undegraded":"2024-09-15T21:48:36.858649+0000","last_fullsized":"2024-09-15T21:48:36.858649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:35.499970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"55'18","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850613+0000","last_change":"2024-09-15T21:46:22.419905+0000","last_active":"2024-09-15T21:48:36.850613+0000","last_peered":"2024-09-15T21:48:36.850613+0000","last_clean":"2024-09-15T21:48:36.850613+0000","last_became_active":"2024-09-15T21:46:22.419620+0000","last_became_peered":"2024-09-15T21:46:22.419620+0000","last_unstale":"2024-09-15T21:48:36.850613+0000","last_undegraded":"2024-09-15T21:48:36.850613+0000","last_fullsized":"2024-09-15T21:48:36.850613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:26:51.215156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710002+0000","last_change":"2024-09-15T21:48:21.639146+0000","last_active":"2024-09-15T21:48:35.710002+0000","last_peered":"2024-09-15T21:48:35.710002+0000","last_clean":"2024-09-15T21:48:35.710002+0000","last_became_active":"2024-09-15T21:48:21.637855+0000","last_became_peered":"2024-09-15T21:48:21.637855+0000","last_unstale":"2024-09-15T21:48:35.710002+0000","last_undegraded":"2024-09-15T21:48:35.710002+0000","last_fullsized":"2024-09-15T21:48:35.710002+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:35:29.305298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'2","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726069+0000","last_change":"2024-09-15T21:46:22.429661+0000","last_active":"2024-09-15T21:48:36.726069+0000","last_peered":"2024-09-15T21:48:36.726069+0000","last_clean":"2024-09-15T21:48:36.726069+0000","last_became_active":"2024-09-15T21:46:22.429250+0000","last_became_peered":"2024-09-15T21:46:22.429250+0000","last_unstale":"2024-09-15T21:48:36.726069+0000","last_undegraded":"2024-09-15T21:48:36.726069+0000","last_fullsized":"2024-09-15T21:48:36.726069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:29.341739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710978+0000","last_change":"2024-09-15T21:48:21.635384+0000","last_active":"2024-09-15T21:48:35.710978+0000","last_peered":"2024-09-15T21:48:35.710978+0000","last_clean":"2024-09-15T21:48:35.710978+0000","last_became_active":"2024-09-15T21:48:21.635051+0000","last_became_peered":"2024-09-15T21:48:21.635051+0000","last_unstale":"2024-09-15T21:48:35.710978+0000","last_undegraded":"2024-09-15T21:48:35.710978+0000","last_fullsized":"2024-09-15T21:48:35.710978+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:59.199568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'5","reported_seq":83,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850941+0000","last_change":"2024-09-15T21:48:27.660076+0000","last_active":"2024-09-15T21:48:36.850941+0000","last_peered":"2024-09-15T21:48:36.850941+0000","last_clean":"2024-09-15T21:48:36.850941+0000","last_became_active":"2024-09-15T21:48:27.659750+0000","last_became_peered":"2024-09-15T21:48:27.659750+0000","last_unstale":"2024-09-15T21:48:36.850941+0000","last_undegraded":"2024-09-15T21:48:36.850941+0000","last_fullsized":"2024-09-15T21:48:36.850941+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:06.505112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.711325+0000","last_change":"2024-09-15T21:48:21.641556+0000","last_active":"2024-09-15T21:48:35.711325+0000","last_peered":"2024-09-15T21:48:35.711325+0000","last_clean":"2024-09-15T21:48:35.711325+0000","last_became_active":"2024-09-15T21:48:21.639328+0000","last_became_peered":"2024-09-15T21:48:21.639328+0000","last_unstale":"2024-09-15T21:48:35.711325+0000","last_undegraded":"2024-09-15T21:48:35.711325+0000","last_fullsized":"2024-09-15T21:48:35.711325+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:36:11.470048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'3","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726437+0000","last_change":"2024-09-15T21:46:22.425072+0000","last_active":"2024-09-15T21:48:36.726437+0000","last_peered":"2024-09-15T21:48:36.726437+0000","last_clean":"2024-09-15T21:48:36.726437+0000","last_became_active":"2024-09-15T21:46:22.424771+0000","last_became_peered":"2024-09-15T21:46:22.424771+0000","last_unstale":"2024-09-15T21:48:36.726437+0000","last_undegraded":"2024-09-15T21:48:36.726437+0000","last_fullsized":"2024-09-15T21:48:36.726437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:26.256785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022156+0000","last_change":"2024-09-15T21:48:17.619698+0000","last_active":"2024-09-15T21:48:37.022156+0000","last_peered":"2024-09-15T21:48:37.022156+0000","last_clean":"2024-09-15T21:48:37.022156+0000","last_became_active":"2024-09-15T21:48:17.617701+0000","last_became_peered":"2024-09-15T21:48:17.617701+0000","last_unstale":"2024-09-15T21:48:37.022156+0000","last_undegraded":"2024-09-15T21:48:37.022156+0000","last_fullsized":"2024-09-15T21:48:37.022156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:00.170687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'5","reported_seq":132,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850794+0000","last_change":"2024-09-15T21:48:17.619089+0000","last_active":"2024-09-15T21:48:36.850794+0000","last_peered":"2024-09-15T21:48:36.850794+0000","last_clean":"2024-09-15T21:48:36.850794+0000","last_became_active":"2024-09-15T21:48:17.618782+0000","last_became_peered":"2024-09-15T21:48:17.618782+0000","last_unstale":"2024-09-15T21:48:36.850794+0000","last_undegraded":"2024-09-15T21:48:36.850794+0000","last_fullsized":"2024-09-15T21:48:36.850794+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:17.881528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710339+0000","last_change":"2024-09-15T21:48:21.639356+0000","last_active":"2024-09-15T21:48:35.710339+0000","last_peered":"2024-09-15T21:48:35.710339+0000","last_clean":"2024-09-15T21:48:35.710339+0000","last_became_active":"2024-09-15T21:48:21.637883+0000","last_became_peered":"2024-09-15T21:48:21.637883+0000","last_unstale":"2024-09-15T21:48:35.710339+0000","last_undegraded":"2024-09-15T21:48:35.710339+0000","last_fullsized":"2024-09-15T21:48:35.710339+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:27.061996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":236,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710002+0000","last_change":"2024-09-15T21:48:24.647129+0000","last_active":"2024-09-15T21:48:35.710002+0000","last_peered":"2024-09-15T21:48:35.710002+0000","last_clean":"2024-09-15T21:48:35.710002+0000","last_became_active":"2024-09-15T21:48:24.646783+0000","last_became_peered":"2024-09-15T21:48:24.646783+0000","last_unstale":"2024-09-15T21:48:35.710002+0000","last_undegraded":"2024-09-15T21:48:35.710002+0000","last_fullsized":"2024-09-15T21:48:35.710002+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_clean_scrub_stamp":"2024-09-15T21:46:12.223400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:20.467980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'6","reported_seq":68,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021854+0000","last_change":"2024-09-15T21:48:17.622619+0000","last_active":"2024-09-15T21:48:37.021854+0000","last_peered":"2024-09-15T21:48:37.021854+0000","last_clean":"2024-09-15T21:48:37.021854+0000","last_became_active":"2024-09-15T21:48:17.622159+0000","last_became_peered":"2024-09-15T21:48:17.622159+0000","last_unstale":"2024-09-15T21:48:37.021854+0000","last_undegraded":"2024-09-15T21:48:37.021854+0000","last_fullsized":"2024-09-15T21:48:37.021854+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:51.566702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":125,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710677+0000","last_change":"2024-09-15T21:48:17.621009+0000","last_active":"2024-09-15T21:48:35.710677+0000","last_peered":"2024-09-15T21:48:35.710677+0000","last_clean":"2024-09-15T21:48:35.710677+0000","last_became_active":"2024-09-15T21:48:17.620738+0000","last_became_peered":"2024-09-15T21:48:17.620738+0000","last_unstale":"2024-09-15T21:48:35.710677+0000","last_undegraded":"2024-09-15T21:48:35.710677+0000","last_fullsized":"2024-09-15T21:48:35.710677+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:20.095876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'4","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710013+0000","last_change":"2024-09-15T21:48:21.645645+0000","last_active":"2024-09-15T21:48:35.710013+0000","last_peered":"2024-09-15T21:48:35.710013+0000","last_clean":"2024-09-15T21:48:35.710013+0000","last_became_active":"2024-09-15T21:48:21.636886+0000","last_became_peered":"2024-09-15T21:48:21.636886+0000","last_unstale":"2024-09-15T21:48:35.710013+0000","last_undegraded":"2024-09-15T21:48:35.710013+0000","last_fullsized":"2024-09-15T21:48:35.710013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:12.676508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726029+0000","last_change":"2024-09-15T21:46:23.430396+0000","last_active":"2024-09-15T21:48:36.726029+0000","last_peered":"2024-09-15T21:48:36.726029+0000","last_clean":"2024-09-15T21:48:36.726029+0000","last_became_active":"2024-09-15T21:46:23.430138+0000","last_became_peered":"2024-09-15T21:46:23.430138+0000","last_unstale":"2024-09-15T21:48:36.726029+0000","last_undegraded":"2024-09-15T21:48:36.726029+0000","last_fullsized":"2024-09-15T21:48:36.726029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:50.622949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"55'7","reported_seq":80,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850706+0000","last_change":"2024-09-15T21:48:27.660216+0000","last_active":"2024-09-15T21:48:36.850706+0000","last_peered":"2024-09-15T21:48:36.850706+0000","last_clean":"2024-09-15T21:48:36.850706+0000","last_became_active":"2024-09-15T21:48:27.659880+0000","last_became_peered":"2024-09-15T21:48:27.659880+0000","last_unstale":"2024-09-15T21:48:36.850706+0000","last_undegraded":"2024-09-15T21:48:36.850706+0000","last_fullsized":"2024-09-15T21:48:36.850706+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:48.186260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":516,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725858+0000","last_change":"2024-09-15T21:46:23.427051+0000","last_active":"2024-09-15T21:48:36.725858+0000","last_peered":"2024-09-15T21:48:36.725858+0000","last_clean":"2024-09-15T21:48:36.725858+0000","last_became_active":"2024-09-15T21:46:23.426185+0000","last_became_peered":"2024-09-15T21:46:23.426185+0000","last_unstale":"2024-09-15T21:48:36.725858+0000","last_undegraded":"2024-09-15T21:48:36.725858+0000","last_fullsized":"2024-09-15T21:48:36.725858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:48.318405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'5","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021901+0000","last_change":"2024-09-15T21:48:17.625702+0000","last_active":"2024-09-15T21:48:37.021901+0000","last_peered":"2024-09-15T21:48:37.021901+0000","last_clean":"2024-09-15T21:48:37.021901+0000","last_became_active":"2024-09-15T21:48:17.623936+0000","last_became_peered":"2024-09-15T21:48:17.623936+0000","last_unstale":"2024-09-15T21:48:37.021901+0000","last_undegraded":"2024-09-15T21:48:37.021901+0000","last_fullsized":"2024-09-15T21:48:37.021901+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:53.114831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":121,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725863+0000","last_change":"2024-09-15T21:46:23.425699+0000","last_active":"2024-09-15T21:48:36.725863+0000","last_peered":"2024-09-15T21:48:36.725863+0000","last_clean":"2024-09-15T21:48:36.725863+0000","last_became_active":"2024-09-15T21:46:23.425416+0000","last_became_peered":"2024-09-15T21:46:23.425416+0000","last_unstale":"2024-09-15T21:48:36.725863+0000","last_undegraded":"2024-09-15T21:48:36.725863+0000","last_fullsized":"2024-09-15T21:48:36.725863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:00.325179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"22'1","reported_seq":122,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726645+0000","last_change":"2024-09-15T21:46:22.426492+0000","last_active":"2024-09-15T21:48:36.726645+0000","last_peered":"2024-09-15T21:48:36.726645+0000","last_clean":"2024-09-15T21:48:36.726645+0000","last_became_active":"2024-09-15T21:46:22.426210+0000","last_became_peered":"2024-09-15T21:46:22.426210+0000","last_unstale":"2024-09-15T21:48:36.726645+0000","last_undegraded":"2024-09-15T21:48:36.726645+0000","last_fullsized":"2024-09-15T21:48:36.726645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:05.248091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022064+0000","last_change":"2024-09-15T21:48:17.625501+0000","last_active":"2024-09-15T21:48:37.022064+0000","last_peered":"2024-09-15T21:48:37.022064+0000","last_clean":"2024-09-15T21:48:37.022064+0000","last_became_active":"2024-09-15T21:48:17.624852+0000","last_became_peered":"2024-09-15T21:48:17.624852+0000","last_unstale":"2024-09-15T21:48:37.022064+0000","last_undegraded":"2024-09-15T21:48:37.022064+0000","last_fullsized":"2024-09-15T21:48:37.022064+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:47:09.731108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"22'4","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710548+0000","last_change":"2024-09-15T21:48:21.668174+0000","last_active":"2024-09-15T21:48:35.710548+0000","last_peered":"2024-09-15T21:48:35.710548+0000","last_clean":"2024-09-15T21:48:35.710548+0000","last_became_active":"2024-09-15T21:48:21.639593+0000","last_became_peered":"2024-09-15T21:48:21.639593+0000","last_unstale":"2024-09-15T21:48:35.710548+0000","last_undegraded":"2024-09-15T21:48:35.710548+0000","last_fullsized":"2024-09-15T21:48:35.710548+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:05.341807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725928+0000","last_change":"2024-09-15T21:46:23.426729+0000","last_active":"2024-09-15T21:48:36.725928+0000","last_peered":"2024-09-15T21:48:36.725928+0000","last_clean":"2024-09-15T21:48:36.725928+0000","last_became_active":"2024-09-15T21:46:23.426495+0000","last_became_peered":"2024-09-15T21:46:23.426495+0000","last_unstale":"2024-09-15T21:48:36.725928+0000","last_undegraded":"2024-09-15T21:48:36.725928+0000","last_fullsized":"2024-09-15T21:48:36.725928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:03.277997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"55'16","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726226+0000","last_change":"2024-09-15T21:46:22.427137+0000","last_active":"2024-09-15T21:48:36.726226+0000","last_peered":"2024-09-15T21:48:36.726226+0000","last_clean":"2024-09-15T21:48:36.726226+0000","last_became_active":"2024-09-15T21:46:22.426855+0000","last_became_peered":"2024-09-15T21:46:22.426855+0000","last_unstale":"2024-09-15T21:48:36.726226+0000","last_undegraded":"2024-09-15T21:48:36.726226+0000","last_fullsized":"2024-09-15T21:48:36.726226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:40:14.277940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":128,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710689+0000","last_change":"2024-09-15T21:48:21.639860+0000","last_active":"2024-09-15T21:48:35.710689+0000","last_peered":"2024-09-15T21:48:35.710689+0000","last_clean":"2024-09-15T21:48:35.710689+0000","last_became_active":"2024-09-15T21:48:21.639541+0000","last_became_peered":"2024-09-15T21:48:21.639541+0000","last_unstale":"2024-09-15T21:48:35.710689+0000","last_undegraded":"2024-09-15T21:48:35.710689+0000","last_fullsized":"2024-09-15T21:48:35.710689+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:21.797034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"55'8","reported_seq":158,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859109+0000","last_change":"2024-09-15T21:48:21.687672+0000","last_active":"2024-09-15T21:48:36.859109+0000","last_peered":"2024-09-15T21:48:36.859109+0000","last_clean":"2024-09-15T21:48:36.859109+0000","last_became_active":"2024-09-15T21:48:21.643251+0000","last_became_peered":"2024-09-15T21:48:21.643251+0000","last_unstale":"2024-09-15T21:48:36.859109+0000","last_undegraded":"2024-09-15T21:48:36.859109+0000","last_fullsized":"2024-09-15T21:48:36.859109+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:40.697954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":44,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726888+0000","last_change":"2024-09-15T21:46:23.424266+0000","last_active":"2024-09-15T21:48:36.726888+0000","last_peered":"2024-09-15T21:48:36.726888+0000","last_clean":"2024-09-15T21:48:36.726888+0000","last_became_active":"2024-09-15T21:46:23.424007+0000","last_became_peered":"2024-09-15T21:46:23.424007+0000","last_unstale":"2024-09-15T21:48:36.726888+0000","last_undegraded":"2024-09-15T21:48:36.726888+0000","last_fullsized":"2024-09-15T21:48:36.726888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:23.492253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'2","reported_seq":135,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726262+0000","last_change":"2024-09-15T21:48:17.623800+0000","last_active":"2024-09-15T21:48:36.726262+0000","last_peered":"2024-09-15T21:48:36.726262+0000","last_clean":"2024-09-15T21:48:36.726262+0000","last_became_active":"2024-09-15T21:48:17.623453+0000","last_became_peered":"2024-09-15T21:48:17.623453+0000","last_unstale":"2024-09-15T21:48:36.726262+0000","last_undegraded":"2024-09-15T21:48:36.726262+0000","last_fullsized":"2024-09-15T21:48:36.726262+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:46:38.897647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":121,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725974+0000","last_change":"2024-09-15T21:46:23.430836+0000","last_active":"2024-09-15T21:48:36.725974+0000","last_peered":"2024-09-15T21:48:36.725974+0000","last_clean":"2024-09-15T21:48:36.725974+0000","last_became_active":"2024-09-15T21:46:23.429283+0000","last_became_peered":"2024-09-15T21:46:23.429283+0000","last_unstale":"2024-09-15T21:48:36.725974+0000","last_undegraded":"2024-09-15T21:48:36.725974+0000","last_fullsized":"2024-09-15T21:48:36.725974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:30.571376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'7","reported_seq":158,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726054+0000","last_change":"2024-09-15T21:48:21.660402+0000","last_active":"2024-09-15T21:48:36.726054+0000","last_peered":"2024-09-15T21:48:36.726054+0000","last_clean":"2024-09-15T21:48:36.726054+0000","last_became_active":"2024-09-15T21:48:21.635898+0000","last_became_peered":"2024-09-15T21:48:21.635898+0000","last_unstale":"2024-09-15T21:48:36.726054+0000","last_undegraded":"2024-09-15T21:48:36.726054+0000","last_fullsized":"2024-09-15T21:48:36.726054+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:29.173360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":180,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":118,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710424+0000","last_change":"2024-09-15T21:46:23.428634+0000","last_active":"2024-09-15T21:48:35.710424+0000","last_peered":"2024-09-15T21:48:35.710424+0000","last_clean":"2024-09-15T21:48:35.710424+0000","last_became_active":"2024-09-15T21:46:23.428301+0000","last_became_peered":"2024-09-15T21:46:23.428301+0000","last_unstale":"2024-09-15T21:48:35.710424+0000","last_undegraded":"2024-09-15T21:48:35.710424+0000","last_fullsized":"2024-09-15T21:48:35.710424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:59.492942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'4","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726625+0000","last_change":"2024-09-15T21:46:22.425810+0000","last_active":"2024-09-15T21:48:36.726625+0000","last_peered":"2024-09-15T21:48:36.726625+0000","last_clean":"2024-09-15T21:48:36.726625+0000","last_became_active":"2024-09-15T21:46:22.425478+0000","last_became_peered":"2024-09-15T21:46:22.425478+0000","last_unstale":"2024-09-15T21:48:36.726625+0000","last_undegraded":"2024-09-15T21:48:36.726625+0000","last_fullsized":"2024-09-15T21:48:36.726625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:53.296599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":118,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710948+0000","last_change":"2024-09-15T21:46:23.426054+0000","last_active":"2024-09-15T21:48:35.710948+0000","last_peered":"2024-09-15T21:48:35.710948+0000","last_clean":"2024-09-15T21:48:35.710948+0000","last_became_active":"2024-09-15T21:46:23.425547+0000","last_became_peered":"2024-09-15T21:46:23.425547+0000","last_unstale":"2024-09-15T21:48:35.710948+0000","last_undegraded":"2024-09-15T21:48:35.710948+0000","last_fullsized":"2024-09-15T21:48:35.710948+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:56.838503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"55'5","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710664+0000","last_change":"2024-09-15T21:48:21.653804+0000","last_active":"2024-09-15T21:48:35.710664+0000","last_peered":"2024-09-15T21:48:35.710664+0000","last_clean":"2024-09-15T21:48:35.710664+0000","last_became_active":"2024-09-15T21:48:21.639223+0000","last_became_peered":"2024-09-15T21:48:21.639223+0000","last_unstale":"2024-09-15T21:48:35.710664+0000","last_undegraded":"2024-09-15T21:48:35.710664+0000","last_fullsized":"2024-09-15T21:48:35.710664+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:47:15.330440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858680+0000","last_change":"2024-09-15T21:46:23.434719+0000","last_active":"2024-09-15T21:48:36.858680+0000","last_peered":"2024-09-15T21:48:36.858680+0000","last_clean":"2024-09-15T21:48:36.858680+0000","last_became_active":"2024-09-15T21:46:23.434448+0000","last_became_peered":"2024-09-15T21:46:23.434448+0000","last_unstale":"2024-09-15T21:48:36.858680+0000","last_undegraded":"2024-09-15T21:48:36.858680+0000","last_fullsized":"2024-09-15T21:48:36.858680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:14.901671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":68,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850752+0000","last_change":"2024-09-15T21:48:27.657040+0000","last_active":"2024-09-15T21:48:36.850752+0000","last_peered":"2024-09-15T21:48:36.850752+0000","last_clean":"2024-09-15T21:48:36.850752+0000","last_became_active":"2024-09-15T21:48:27.656753+0000","last_became_peered":"2024-09-15T21:48:27.656753+0000","last_unstale":"2024-09-15T21:48:36.850752+0000","last_undegraded":"2024-09-15T21:48:36.850752+0000","last_fullsized":"2024-09-15T21:48:36.850752+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:14.458723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'5","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710454+0000","last_change":"2024-09-15T21:48:21.691408+0000","last_active":"2024-09-15T21:48:35.710454+0000","last_peered":"2024-09-15T21:48:35.710454+0000","last_clean":"2024-09-15T21:48:35.710454+0000","last_became_active":"2024-09-15T21:48:21.641517+0000","last_became_peered":"2024-09-15T21:48:21.641517+0000","last_unstale":"2024-09-15T21:48:35.710454+0000","last_undegraded":"2024-09-15T21:48:35.710454+0000","last_fullsized":"2024-09-15T21:48:35.710454+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:09.454433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022007+0000","last_change":"2024-09-15T21:48:17.623223+0000","last_active":"2024-09-15T21:48:37.022007+0000","last_peered":"2024-09-15T21:48:37.022007+0000","last_clean":"2024-09-15T21:48:37.022007+0000","last_became_active":"2024-09-15T21:48:17.622019+0000","last_became_peered":"2024-09-15T21:48:17.622019+0000","last_unstale":"2024-09-15T21:48:37.022007+0000","last_undegraded":"2024-09-15T21:48:37.022007+0000","last_fullsized":"2024-09-15T21:48:37.022007+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:39:57.643793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"55'13","reported_seq":53,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710695+0000","last_change":"2024-09-15T21:48:21.682609+0000","last_active":"2024-09-15T21:48:35.710695+0000","last_peered":"2024-09-15T21:48:35.710695+0000","last_clean":"2024-09-15T21:48:35.710695+0000","last_became_active":"2024-09-15T21:48:21.640156+0000","last_became_peered":"2024-09-15T21:48:21.640156+0000","last_unstale":"2024-09-15T21:48:35.710695+0000","last_undegraded":"2024-09-15T21:48:35.710695+0000","last_fullsized":"2024-09-15T21:48:35.710695+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:28.936509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726585+0000","last_change":"2024-09-15T21:48:17.623164+0000","last_active":"2024-09-15T21:48:36.726585+0000","last_peered":"2024-09-15T21:48:36.726585+0000","last_clean":"2024-09-15T21:48:36.726585+0000","last_became_active":"2024-09-15T21:48:17.622843+0000","last_became_peered":"2024-09-15T21:48:17.622843+0000","last_unstale":"2024-09-15T21:48:36.726585+0000","last_undegraded":"2024-09-15T21:48:36.726585+0000","last_fullsized":"2024-09-15T21:48:36.726585+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:06:30.745189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'2","reported_seq":122,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710725+0000","last_change":"2024-09-15T21:46:22.428529+0000","last_active":"2024-09-15T21:48:35.710725+0000","last_peered":"2024-09-15T21:48:35.710725+0000","last_clean":"2024-09-15T21:48:35.710725+0000","last_became_active":"2024-09-15T21:46:22.428263+0000","last_became_peered":"2024-09-15T21:46:22.428263+0000","last_unstale":"2024-09-15T21:48:35.710725+0000","last_undegraded":"2024-09-15T21:48:35.710725+0000","last_fullsized":"2024-09-15T21:48:35.710725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:22:40.160062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":118,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710647+0000","last_change":"2024-09-15T21:46:23.427968+0000","last_active":"2024-09-15T21:48:35.710647+0000","last_peered":"2024-09-15T21:48:35.710647+0000","last_clean":"2024-09-15T21:48:35.710647+0000","last_became_active":"2024-09-15T21:46:23.427546+0000","last_became_peered":"2024-09-15T21:46:23.427546+0000","last_unstale":"2024-09-15T21:48:35.710647+0000","last_undegraded":"2024-09-15T21:48:35.710647+0000","last_fullsized":"2024-09-15T21:48:35.710647+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:47.226097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'13","reported_seq":171,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726869+0000","last_change":"2024-09-15T21:48:21.680908+0000","last_active":"2024-09-15T21:48:36.726869+0000","last_peered":"2024-09-15T21:48:36.726869+0000","last_clean":"2024-09-15T21:48:36.726869+0000","last_became_active":"2024-09-15T21:48:21.642406+0000","last_became_peered":"2024-09-15T21:48:21.642406+0000","last_unstale":"2024-09-15T21:48:36.726869+0000","last_undegraded":"2024-09-15T21:48:36.726869+0000","last_fullsized":"2024-09-15T21:48:36.726869+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:23:26.323380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1344,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":42655,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":243,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":4802,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":335872,"data_stored":85310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6368,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476769,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1507328,"data_stored":1002267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1503232,"data_stored":997684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509475,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":946176,"data_stored":447388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":57,"seq":244813135878,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29072,"kb_used_data":888,"kb_used_omap":7,"kb_used_meta":28152,"kb_avail":94342768,"statfs":{"total":96636764160,"available":96606994432,"internally_reserved":0,"allocated":909312,"data_stored":402827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7963,"internal_metadata":28827877},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509474,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":909312,"data_stored":412355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509473,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":921600,"data_stored":412891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":888832,"data_stored":400427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":942080,"data_stored":442017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":37057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6368,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":36363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:48:40.633 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-15T21:48:40.633 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-15T21:48:40.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.770+0000 7f23e308d640 1 Processor -- start 2024-09-15T21:48:40.773 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.770+0000 7f23e308d640 1 -- start start 2024-09-15T21:48:40.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.771+0000 7f23e308d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23dc150020 0x7f23dc06bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.771+0000 7f23e308d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f23dc0ce430 0x7f23dc0c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.771+0000 7f23e308d640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23dc15c6b0 con 0x7f23dc06d480 2024-09-15T21:48:40.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.771+0000 7f23e308d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f23dc15d5a0 con 0x7f23dc0ce430 2024-09-15T21:48:40.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.771+0000 7f23e308d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f23dc15c990 con 0x7f23dc150020 2024-09-15T21:48:40.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.771+0000 7f23e162a640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f23dc06d480 0x7f23dc0c8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42152/0 (socket says 172.21.15.100:42152) 2024-09-15T21:48:40.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.771+0000 7f23e162a640 1 -- 172.21.15.100:0/256395556 learned_addr learned my addr 172.21.15.100:0/256395556 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:40.778 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.771+0000 7f23db7fe640 1 -- 172.21.15.100:0/256395556 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 589977910 0 0) 0x7f23dc15c6b0 con 0x7f23dc06d480 2024-09-15T21:48:40.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.771+0000 7f23dbfff640 1 --2- 172.21.15.100:0/256395556 >> v2:172.21.15.122:3300/0 conn(0x7f23dc0ce430 0x7f23dc0c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.771+0000 7f23e0e29640 1 --2- 172.21.15.100:0/256395556 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23dc150020 0x7f23dc06bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.771+0000 7f23db7fe640 1 -- 172.21.15.100:0/256395556 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23cc003520 con 0x7f23dc06d480 2024-09-15T21:48:40.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23db7fe640 1 -- 172.21.15.100:0/256395556 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1043057858 0 0) 0x7f23cc003520 con 0x7f23dc06d480 2024-09-15T21:48:40.779 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23db7fe640 1 -- 172.21.15.100:0/256395556 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23dc150020 msgr2=0x7f23dc06bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:40.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23db7fe640 1 --2- 172.21.15.100:0/256395556 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23dc150020 0x7f23dc06bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.780 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23db7fe640 1 -- 172.21.15.100:0/256395556 >> v2:172.21.15.122:3300/0 conn(0x7f23dc0ce430 msgr2=0x7f23dc0c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:40.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23db7fe640 1 --2- 172.21.15.100:0/256395556 >> v2:172.21.15.122:3300/0 conn(0x7f23dc0ce430 0x7f23dc0c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23db7fe640 1 -- 172.21.15.100:0/256395556 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc15cc70 con 0x7f23dc06d480 2024-09-15T21:48:40.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23db7fe640 1 -- 172.21.15.100:0/256395556 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7f23d40019f0 con 0x7f23dc06d480 2024-09-15T21:48:40.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23db7fe640 1 -- 172.21.15.100:0/256395556 >> v1:172.21.15.100:6789/0 conn(0x7f23dc06d480 legacy=0x7f23dc0c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:40.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23db7fe640 1 --2- 172.21.15.100:0/256395556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23cc0039b0 0x7f23cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23db7fe640 1 --2- 172.21.15.100:0/256395556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23cc004800 0x7f23cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23db7fe640 1 --2- 172.21.15.100:0/256395556 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23dc150020 0x7f23cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23db7fe640 1 -- 172.21.15.100:0/256395556 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc15d5a0 con 0x7f23cc004800 2024-09-15T21:48:40.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23db7fe640 1 -- 172.21.15.100:0/256395556 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc15c990 con 0x7f23cc0039b0 2024-09-15T21:48:40.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23db7fe640 1 -- 172.21.15.100:0/256395556 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f23cc00b9c0 con 0x7f23dc150020 2024-09-15T21:48:40.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23dbfff640 1 --2- 172.21.15.100:0/256395556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23cc004800 0x7f23cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.781 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23e0e29640 1 --2- 172.21.15.100:0/256395556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23cc0039b0 0x7f23cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23e162a640 1 --2- 172.21.15.100:0/256395556 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23dc150020 0x7f23cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23dbfff640 1 -- 172.21.15.100:0/256395556 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23dc150020 msgr2=0x7f23cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:40.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23dbfff640 1 --2- 172.21.15.100:0/256395556 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23dc150020 0x7f23cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23dbfff640 1 -- 172.21.15.100:0/256395556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23cc0039b0 msgr2=0x7f23cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:40.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23dbfff640 1 --2- 172.21.15.100:0/256395556 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23cc0039b0 0x7f23cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.772+0000 7f23dbfff640 1 -- 172.21.15.100:0/256395556 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23cc003520 con 0x7f23cc004800 2024-09-15T21:48:40.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.773+0000 7f23dbfff640 1 --2- 172.21.15.100:0/256395556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23cc004800 0x7f23cc004c80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009a90 tx=0x7f23cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51cb2a199b46925e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:40.782 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.773+0000 7f23db7fe640 1 -- 172.21.15.100:0/256395556 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8036070 con 0x7f23cc004800 2024-09-15T21:48:40.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.773+0000 7f23db7fe640 1 -- 172.21.15.100:0/256395556 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c8031400 con 0x7f23cc004800 2024-09-15T21:48:40.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.773+0000 7f23db7fe640 1 -- 172.21.15.100:0/256395556 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c8009690 con 0x7f23cc004800 2024-09-15T21:48:40.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.773+0000 7f23e308d640 1 -- 172.21.15.100:0/256395556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23cc004800 msgr2=0x7f23cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:40.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.773+0000 7f23e308d640 1 --2- 172.21.15.100:0/256395556 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23cc004800 0x7f23cc004c80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009a90 tx=0x7f23cc005830 comp rx=0 tx=0).stop 2024-09-15T21:48:40.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.773+0000 7f23e162a640 1 -- 172.21.15.100:0/256395556 reap_dead start 2024-09-15T21:48:40.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.774+0000 7f23e308d640 1 -- 172.21.15.100:0/256395556 shutdown_connections 2024-09-15T21:48:40.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.774+0000 7f23e308d640 1 -- 172.21.15.100:0/256395556 >> 172.21.15.100:0/256395556 conn(0x7f23dc0cd890 msgr2=0x7f23dc0cdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:40.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.774+0000 7f23e308d640 1 -- 172.21.15.100:0/256395556 shutdown_connections 2024-09-15T21:48:40.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.774+0000 7f23e308d640 1 -- 172.21.15.100:0/256395556 wait complete. 2024-09-15T21:48:40.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.775+0000 7f23e308d640 1 Processor -- start 2024-09-15T21:48:40.783 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.775+0000 7f23e308d640 1 -- start start 2024-09-15T21:48:40.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23e308d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23cc004800 0x7f23dc1e34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23e308d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23dc146530 0x7f23dc0cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23e308d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23dc1f2510 0x7f23dc1f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23e308d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc0c8c80 con 0x7f23cc004800 2024-09-15T21:48:40.787 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23e308d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc15d230 con 0x7f23dc1f2510 2024-09-15T21:48:40.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23e308d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f23dc0c8e00 con 0x7f23dc146530 2024-09-15T21:48:40.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23e0e29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23cc004800 0x7f23dc1e34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23e0e29640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23cc004800 0x7f23dc1e34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54428/0 (socket says 172.21.15.100:54428) 2024-09-15T21:48:40.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23e0e29640 1 -- 172.21.15.100:0/4032728396 learned_addr learned my addr 172.21.15.100:0/4032728396 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:40.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23dbfff640 1 --2- 172.21.15.100:0/4032728396 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23dc146530 0x7f23dc0cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23e162a640 1 --2- 172.21.15.100:0/4032728396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23dc1f2510 0x7f23dc1f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23e0e29640 1 -- 172.21.15.100:0/4032728396 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23dc146530 msgr2=0x7f23dc0cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:40.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23e0e29640 1 --2- 172.21.15.100:0/4032728396 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23dc146530 0x7f23dc0cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23e0e29640 1 -- 172.21.15.100:0/4032728396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23dc1f2510 msgr2=0x7f23dc1f2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:40.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23e0e29640 1 --2- 172.21.15.100:0/4032728396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23dc1f2510 0x7f23dc1f2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23e0e29640 1 -- 172.21.15.100:0/4032728396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc1f2ed0 con 0x7f23cc004800 2024-09-15T21:48:40.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23e0e29640 1 --2- 172.21.15.100:0/4032728396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23cc004800 0x7f23dc1e34d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f23d0011910 tx=0x7f23d0011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:40.788 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23d97fa640 1 -- 172.21.15.100:0/4032728396 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d000faf0 con 0x7f23cc004800 2024-09-15T21:48:40.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23d97fa640 1 -- 172.21.15.100:0/4032728396 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23d000fc90 con 0x7f23cc004800 2024-09-15T21:48:40.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23e308d640 1 -- 172.21.15.100:0/4032728396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23dc1e2580 con 0x7f23cc004800 2024-09-15T21:48:40.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.776+0000 7f23d97fa640 1 -- 172.21.15.100:0/4032728396 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d0014440 con 0x7f23cc004800 2024-09-15T21:48:40.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.777+0000 7f23e308d640 1 -- 172.21.15.100:0/4032728396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23dc1e4270 con 0x7f23cc004800 2024-09-15T21:48:40.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.777+0000 7f23e308d640 1 -- 172.21.15.100:0/4032728396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23dc06c400 con 0x7f23cc004800 2024-09-15T21:48:40.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.778+0000 7f23d97fa640 1 -- 172.21.15.100:0/4032728396 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f23d0012040 con 0x7f23cc004800 2024-09-15T21:48:40.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.778+0000 7f23d97fa640 1 --2- 172.21.15.100:0/4032728396 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f23c006c2b0 0x7f23c006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:40.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.778+0000 7f23d97fa640 1 -- 172.21.15.100:0/4032728396 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f23d008b8b0 con 0x7f23cc004800 2024-09-15T21:48:40.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.778+0000 7f23dbfff640 1 --2- 172.21.15.100:0/4032728396 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f23c006c2b0 0x7f23c006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:40.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.779+0000 7f23dbfff640 1 --2- 172.21.15.100:0/4032728396 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f23c006c2b0 0x7f23c006e770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f23c8033000 tx=0x7f23c8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:40.789 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.782+0000 7f23d97fa640 1 -- 172.21.15.100:0/4032728396 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f23dc06c400 con 0x7f23cc004800 2024-09-15T21:48:40.950 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.947+0000 7f23e308d640 1 -- 172.21.15.100:0/4032728396 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"} v 0) -- 0x7f23dc15c6b0 con 0x7f23cc004800 2024-09-15T21:48:40.953 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.950+0000 7f23d97fa640 1 -- 172.21.15.100:0/4032728396 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]=0 v11) ==== 118+0+0 (secure 0 0 0) 0x7f23dc1f2ed0 con 0x7f23cc004800 2024-09-15T21:48:40.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.952+0000 7f23e308d640 1 -- 172.21.15.100:0/4032728396 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f23c006c2b0 msgr2=0x7f23c006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:40.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.952+0000 7f23e308d640 1 --2- 172.21.15.100:0/4032728396 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f23c006c2b0 0x7f23c006e770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f23c8033000 tx=0x7f23c8008030 comp rx=0 tx=0).stop 2024-09-15T21:48:40.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.952+0000 7f23e308d640 1 -- 172.21.15.100:0/4032728396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23cc004800 msgr2=0x7f23dc1e34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:40.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.952+0000 7f23e308d640 1 --2- 172.21.15.100:0/4032728396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23cc004800 0x7f23dc1e34d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f23d0011910 tx=0x7f23d0011dd0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.952+0000 7f23e308d640 1 -- 172.21.15.100:0/4032728396 shutdown_connections 2024-09-15T21:48:40.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.952+0000 7f23e308d640 1 --2- 172.21.15.100:0/4032728396 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f23c006c2b0 0x7f23c006e770 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.952+0000 7f23e308d640 1 --2- 172.21.15.100:0/4032728396 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23dc1f2510 0x7f23dc1f2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.952+0000 7f23e308d640 1 --2- 172.21.15.100:0/4032728396 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23dc146530 0x7f23dc0cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.952+0000 7f23e308d640 1 --2- 172.21.15.100:0/4032728396 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23cc004800 0x7f23dc1e34d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:40.956 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.952+0000 7f23e308d640 1 -- 172.21.15.100:0/4032728396 >> 172.21.15.100:0/4032728396 conn(0x7f23dc0cd890 msgr2=0x7f23dc0c43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:40.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.952+0000 7f23e308d640 1 -- 172.21.15.100:0/4032728396 shutdown_connections 2024-09-15T21:48:40.957 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:40.952+0000 7f23e308d640 1 -- 172.21.15.100:0/4032728396 wait complete. 2024-09-15T21:48:40.966 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-15T21:48:40.976 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-15T21:48:40.976 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:48:40.976 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T21:48:40.993 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-09-15T21:48:40.993 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-15T21:48:41.049 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-15T21:48:41.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.185+0000 7fd49cc37640 1 Processor -- start 2024-09-15T21:48:41.188 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.185+0000 7fd49cc37640 1 -- start start 2024-09-15T21:48:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.186+0000 7fd49cc37640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd498151fd0 0x7fd49815bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.186+0000 7fd49cc37640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4980c8f40 con 0x7fd4980c86f0 2024-09-15T21:48:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.186+0000 7fd49cc37640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd498157d70 con 0x7fd498071b50 2024-09-15T21:48:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.186+0000 7fd49cc37640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fd498158030 con 0x7fd498151fd0 2024-09-15T21:48:41.191 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.186+0000 7fd48ffff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd4980c86f0 0x7fd4980cc770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42168/0 (socket says 172.21.15.100:42168) 2024-09-15T21:48:41.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.186+0000 7fd48ffff640 1 -- 172.21.15.100:0/3216435511 learned_addr learned my addr 172.21.15.100:0/3216435511 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:41.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.186+0000 7fd495d9b640 1 -- 172.21.15.100:0/3216435511 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2020894021 0 0) 0x7fd4980c8f40 con 0x7fd4980c86f0 2024-09-15T21:48:41.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.186+0000 7fd49659c640 1 --2- 172.21.15.100:0/3216435511 >> v2:172.21.15.122:3300/0 conn(0x7fd498151fd0 0x7fd49815bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:41.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.186+0000 7fd495d9b640 1 -- 172.21.15.100:0/3216435511 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd484003520 con 0x7fd4980c86f0 2024-09-15T21:48:41.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.186+0000 7fd495d9b640 1 -- 172.21.15.100:0/3216435511 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1891977847 0 0) 0x7fd484003520 con 0x7fd4980c86f0 2024-09-15T21:48:41.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.186+0000 7fd495d9b640 1 -- 172.21.15.100:0/3216435511 >> v2:172.21.15.122:3300/0 conn(0x7fd498151fd0 msgr2=0x7fd49815bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:41.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.186+0000 7fd495d9b640 1 --2- 172.21.15.100:0/3216435511 >> v2:172.21.15.122:3300/0 conn(0x7fd498151fd0 0x7fd49815bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.186+0000 7fd495d9b640 1 -- 172.21.15.100:0/3216435511 >> v1:172.21.15.122:6789/0 conn(0x7fd498071b50 legacy=0x7fd498071f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:41.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.186+0000 7fd495d9b640 1 -- 172.21.15.100:0/3216435511 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd498158310 con 0x7fd4980c86f0 2024-09-15T21:48:41.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.186+0000 7fd495d9b640 1 -- 172.21.15.100:0/3216435511 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7fd480001670 con 0x7fd4980c86f0 2024-09-15T21:48:41.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.186+0000 7fd495d9b640 1 -- 172.21.15.100:0/3216435511 >> v1:172.21.15.100:6789/0 conn(0x7fd4980c86f0 legacy=0x7fd4980cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:41.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.186+0000 7fd495d9b640 1 --2- 172.21.15.100:0/3216435511 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd4840039b0 0x7fd484003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:41.192 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.187+0000 7fd495d9b640 1 --2- 172.21.15.100:0/3216435511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd484004800 0x7fd484004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:41.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.187+0000 7fd495d9b640 1 --2- 172.21.15.100:0/3216435511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4840051c0 0x7fd48400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:41.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.187+0000 7fd495d9b640 1 -- 172.21.15.100:0/3216435511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd498158030 con 0x7fd484004800 2024-09-15T21:48:41.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.187+0000 7fd495d9b640 1 -- 172.21.15.100:0/3216435511 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd48400e270 con 0x7fd4840051c0 2024-09-15T21:48:41.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.187+0000 7fd495d9b640 1 -- 172.21.15.100:0/3216435511 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd48400e470 con 0x7fd4840039b0 2024-09-15T21:48:41.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.187+0000 7fd48ffff640 1 --2- 172.21.15.100:0/3216435511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd484004800 0x7fd484004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:41.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.187+0000 7fd49659c640 1 --2- 172.21.15.100:0/3216435511 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd4840039b0 0x7fd484003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:41.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.187+0000 7fd496d9d640 1 --2- 172.21.15.100:0/3216435511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4840051c0 0x7fd48400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:41.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.187+0000 7fd48ffff640 1 -- 172.21.15.100:0/3216435511 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd4840039b0 msgr2=0x7fd484003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:41.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.187+0000 7fd48ffff640 1 --2- 172.21.15.100:0/3216435511 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd4840039b0 0x7fd484003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.187+0000 7fd48ffff640 1 -- 172.21.15.100:0/3216435511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4840051c0 msgr2=0x7fd48400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:41.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.187+0000 7fd48ffff640 1 --2- 172.21.15.100:0/3216435511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4840051c0 0x7fd48400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.187+0000 7fd48ffff640 1 -- 172.21.15.100:0/3216435511 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd484003520 con 0x7fd484004800 2024-09-15T21:48:41.193 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.187+0000 7fd496d9d640 1 -- 172.21.15.100:0/3216435511 reap_dead start 2024-09-15T21:48:41.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.188+0000 7fd48ffff640 1 --2- 172.21.15.100:0/3216435511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd484004800 0x7fd484004c80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd48000c240 tx=0x7fd484005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203dde497aa24078 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:41.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.188+0000 7fd495d9b640 1 -- 172.21.15.100:0/3216435511 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480007cf0 con 0x7fd484004800 2024-09-15T21:48:41.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.188+0000 7fd495d9b640 1 -- 172.21.15.100:0/3216435511 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4800058d0 con 0x7fd484004800 2024-09-15T21:48:41.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.188+0000 7fd495d9b640 1 -- 172.21.15.100:0/3216435511 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480005ca0 con 0x7fd484004800 2024-09-15T21:48:41.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.188+0000 7fd49cc37640 1 -- 172.21.15.100:0/3216435511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd484004800 msgr2=0x7fd484004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:41.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.188+0000 7fd49cc37640 1 --2- 172.21.15.100:0/3216435511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd484004800 0x7fd484004c80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd48000c240 tx=0x7fd484005830 comp rx=0 tx=0).stop 2024-09-15T21:48:41.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.188+0000 7fd49cc37640 1 -- 172.21.15.100:0/3216435511 shutdown_connections 2024-09-15T21:48:41.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.188+0000 7fd49cc37640 1 --2- 172.21.15.100:0/3216435511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd484004800 0x7fd484004c80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.188+0000 7fd49cc37640 1 -- 172.21.15.100:0/3216435511 >> 172.21.15.100:0/3216435511 conn(0x7fd498067dd0 msgr2=0x7fd49806bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:41.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.188+0000 7fd49cc37640 1 -- 172.21.15.100:0/3216435511 shutdown_connections 2024-09-15T21:48:41.194 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.188+0000 7fd49cc37640 1 -- 172.21.15.100:0/3216435511 wait complete. 2024-09-15T21:48:41.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.190+0000 7fd49cc37640 1 Processor -- start 2024-09-15T21:48:41.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.190+0000 7fd49cc37640 1 -- start start 2024-09-15T21:48:41.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.190+0000 7fd49cc37640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd484004800 0x7fd498154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:41.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.190+0000 7fd49cc37640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd4980c9000 0x7fd4981ec5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:41.197 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.190+0000 7fd49cc37640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4980c9440 0x7fd4980bc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:41.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.190+0000 7fd49cc37640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd4981523b0 con 0x7fd4980c9440 2024-09-15T21:48:41.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.190+0000 7fd49cc37640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd4981588d0 con 0x7fd484004800 2024-09-15T21:48:41.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.190+0000 7fd49cc37640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd498152530 con 0x7fd4980c9000 2024-09-15T21:48:41.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.190+0000 7fd496d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4980c9440 0x7fd4980bc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:41.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.190+0000 7fd496d9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4980c9440 0x7fd4980bc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54446/0 (socket says 172.21.15.100:54446) 2024-09-15T21:48:41.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.190+0000 7fd496d9d640 1 -- 172.21.15.100:0/2737228846 learned_addr learned my addr 172.21.15.100:0/2737228846 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:41.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.190+0000 7fd48ffff640 1 --2- 172.21.15.100:0/2737228846 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd4980c9000 0x7fd4981ec5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:41.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.190+0000 7fd49659c640 1 --2- 172.21.15.100:0/2737228846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd484004800 0x7fd498154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:41.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.191+0000 7fd496d9d640 1 -- 172.21.15.100:0/2737228846 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd4980c9000 msgr2=0x7fd4981ec5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:41.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.191+0000 7fd496d9d640 1 --2- 172.21.15.100:0/2737228846 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd4980c9000 0x7fd4981ec5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.191+0000 7fd496d9d640 1 -- 172.21.15.100:0/2737228846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd484004800 msgr2=0x7fd498154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:41.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.191+0000 7fd496d9d640 1 --2- 172.21.15.100:0/2737228846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd484004800 0x7fd498154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.198 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.191+0000 7fd496d9d640 1 -- 172.21.15.100:0/2737228846 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd498154390 con 0x7fd4980c9440 2024-09-15T21:48:41.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.191+0000 7fd496d9d640 1 --2- 172.21.15.100:0/2737228846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4980c9440 0x7fd4980bc8d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd49815c2e0 tx=0x7fd49000fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:41.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.191+0000 7fd48effd640 1 -- 172.21.15.100:0/2737228846 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49001a070 con 0x7fd4980c9440 2024-09-15T21:48:41.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.191+0000 7fd48effd640 1 -- 172.21.15.100:0/2737228846 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd490006b50 con 0x7fd4980c9440 2024-09-15T21:48:41.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.191+0000 7fd48effd640 1 -- 172.21.15.100:0/2737228846 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd490003970 con 0x7fd4980c9440 2024-09-15T21:48:41.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.191+0000 7fd49cc37640 1 -- 172.21.15.100:0/2737228846 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd498154650 con 0x7fd4980c9440 2024-09-15T21:48:41.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.191+0000 7fd49cc37640 1 -- 172.21.15.100:0/2737228846 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4981efe40 con 0x7fd4980c9440 2024-09-15T21:48:41.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.192+0000 7fd49cc37640 1 -- 172.21.15.100:0/2737228846 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd49806c920 con 0x7fd4980c9440 2024-09-15T21:48:41.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.192+0000 7fd48effd640 1 -- 172.21.15.100:0/2737228846 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd49001d070 con 0x7fd4980c9440 2024-09-15T21:48:41.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.193+0000 7fd48effd640 1 --2- 172.21.15.100:0/2737228846 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd47806c2b0 0x7fd47806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:41.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.193+0000 7fd48effd640 1 -- 172.21.15.100:0/2737228846 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7fd49008a860 con 0x7fd4980c9440 2024-09-15T21:48:41.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.193+0000 7fd49659c640 1 --2- 172.21.15.100:0/2737228846 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd47806c2b0 0x7fd47806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:41.199 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.193+0000 7fd49659c640 1 --2- 172.21.15.100:0/2737228846 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd47806c2b0 0x7fd47806e770 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd488009c40 tx=0x7fd488009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:41.200 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.197+0000 7fd48effd640 1 -- 172.21.15.100:0/2737228846 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fd490058850 con 0x7fd4980c9440 2024-09-15T21:48:41.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.373+0000 7fd49cc37640 1 -- 172.21.15.100:0/2737228846 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd4980c99c0 con 0x7fd4980c9440 2024-09-15T21:48:41.376 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.373+0000 7fd48effd640 1 -- 172.21.15.100:0/2737228846 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v76) ==== 80+0+0 (secure 0 0 0) 0x7fd49005cb50 con 0x7fd4980c9440 2024-09-15T21:48:41.376 INFO:teuthology.orchestra.run.smithi100.stderr:listed 0 entries 2024-09-15T21:48:41.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.375+0000 7fd49cc37640 1 -- 172.21.15.100:0/2737228846 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd47806c2b0 msgr2=0x7fd47806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:41.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.375+0000 7fd49cc37640 1 --2- 172.21.15.100:0/2737228846 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd47806c2b0 0x7fd47806e770 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd488009c40 tx=0x7fd488009190 comp rx=0 tx=0).stop 2024-09-15T21:48:41.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.375+0000 7fd49cc37640 1 -- 172.21.15.100:0/2737228846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4980c9440 msgr2=0x7fd4980bc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:41.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.375+0000 7fd49cc37640 1 --2- 172.21.15.100:0/2737228846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4980c9440 0x7fd4980bc8d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd49815c2e0 tx=0x7fd49000fb40 comp rx=0 tx=0).stop 2024-09-15T21:48:41.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.375+0000 7fd49cc37640 1 -- 172.21.15.100:0/2737228846 shutdown_connections 2024-09-15T21:48:41.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.375+0000 7fd49cc37640 1 --2- 172.21.15.100:0/2737228846 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd47806c2b0 0x7fd47806e770 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.375+0000 7fd49cc37640 1 --2- 172.21.15.100:0/2737228846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd4980c9440 0x7fd4980bc8d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.378 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.375+0000 7fd49cc37640 1 --2- 172.21.15.100:0/2737228846 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd4980c9000 0x7fd4981ec5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.375+0000 7fd49cc37640 1 --2- 172.21.15.100:0/2737228846 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd484004800 0x7fd498154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.375+0000 7fd49cc37640 1 -- 172.21.15.100:0/2737228846 >> 172.21.15.100:0/2737228846 conn(0x7fd498067dd0 msgr2=0x7fd498073380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:41.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.375+0000 7fd49cc37640 1 -- 172.21.15.100:0/2737228846 shutdown_connections 2024-09-15T21:48:41.379 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.375+0000 7fd49cc37640 1 -- 172.21.15.100:0/2737228846 wait complete. 2024-09-15T21:48:41.389 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-15T21:48:41.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T21:48:41.389 DEBUG:teuthology.orchestra.run.smithi100:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-15T21:48:41.436 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi100.front.sepia.ceph.com 2024-09-15T21:48:41.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T21:48:41.437 DEBUG:teuthology.orchestra.run.smithi100:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-15T21:48:41.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T21:48:41.475 DEBUG:teuthology.orchestra.run.smithi100:> ip netns list 2024-09-15T21:48:41.531 INFO:teuthology.orchestra.run.smithi100.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-15T21:48:41.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T21:48:41.531 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-15T21:48:41.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-15T21:48:41.597 DEBUG:teuthology.orchestra.run.smithi100:> sudo ip link delete ceph-brx 2024-09-15T21:48:41.702 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-15T21:48:41.711 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:48:41.711 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:48:41.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.810+0000 7fc3df85c640 1 Processor -- start 2024-09-15T21:48:41.812 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.810+0000 7fc3df85c640 1 -- start start 2024-09-15T21:48:41.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.811+0000 7fc3df85c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc3d80cca30 0x7fc3d8071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:41.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.811+0000 7fc3df85c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc3d80cce10 0x7fc3d815af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:41.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.811+0000 7fc3df85c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3d80c91f0 con 0x7fc3d8070ee0 2024-09-15T21:48:41.813 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.811+0000 7fc3df85c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fc3d80c94d0 con 0x7fc3d80cca30 2024-09-15T21:48:41.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.811+0000 7fc3df85c640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7fc3d80c97b0 con 0x7fc3d80cce10 2024-09-15T21:48:41.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.811+0000 7fc3dd5f8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc3d80cca30 0x7fc3d8071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:41.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.811+0000 7fc3dd5f8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fc3d80cca30 0x7fc3d8071340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54448/0 (socket says 172.21.15.100:54448) 2024-09-15T21:48:41.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.811+0000 7fc3dd5f8640 1 -- 172.21.15.100:0/901134406 learned_addr learned my addr 172.21.15.100:0/901134406 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:41.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.811+0000 7fc3dddf9640 1 --2- 172.21.15.100:0/901134406 >> v2:172.21.15.122:3300/0 conn(0x7fc3d80cce10 0x7fc3d815af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:41.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.811+0000 7fc3dd5f8640 1 -- 172.21.15.100:0/901134406 >> v2:172.21.15.122:3300/0 conn(0x7fc3d80cce10 msgr2=0x7fc3d815af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:41.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.811+0000 7fc3dd5f8640 1 --2- 172.21.15.100:0/901134406 >> v2:172.21.15.122:3300/0 conn(0x7fc3d80cce10 0x7fc3d815af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.811+0000 7fc3dd5f8640 1 -- 172.21.15.100:0/901134406 >> v1:172.21.15.122:6789/0 conn(0x7fc3d8070ee0 legacy=0x7fc3d8151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:41.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.812+0000 7fc3dd5f8640 1 -- 172.21.15.100:0/901134406 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d80c9a90 con 0x7fc3d80cca30 2024-09-15T21:48:41.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.812+0000 7fc3dd5f8640 1 --2- 172.21.15.100:0/901134406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3d80cca30 0x7fc3d8071340 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80cd4c0 tx=0x7fc3d402b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81ebf7f74e46368 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:41.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.812+0000 7fc3cffff640 1 -- 172.21.15.100:0/901134406 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d400b040 con 0x7fc3d80cca30 2024-09-15T21:48:41.815 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.812+0000 7fc3cffff640 1 -- 172.21.15.100:0/901134406 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d4030710 con 0x7fc3d80cca30 2024-09-15T21:48:41.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.812+0000 7fc3cffff640 1 -- 172.21.15.100:0/901134406 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4030ae0 con 0x7fc3d80cca30 2024-09-15T21:48:41.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.813+0000 7fc3df85c640 1 -- 172.21.15.100:0/901134406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3d80cca30 msgr2=0x7fc3d8071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:41.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.813+0000 7fc3df85c640 1 --2- 172.21.15.100:0/901134406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3d80cca30 0x7fc3d8071340 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80cd4c0 tx=0x7fc3d402b6d0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.813+0000 7fc3df85c640 1 -- 172.21.15.100:0/901134406 shutdown_connections 2024-09-15T21:48:41.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.813+0000 7fc3df85c640 1 --2- 172.21.15.100:0/901134406 >> v2:172.21.15.122:3300/0 conn(0x7fc3d80cce10 0x7fc3d815af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.813+0000 7fc3df85c640 1 --2- 172.21.15.100:0/901134406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3d80cca30 0x7fc3d8071340 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.813+0000 7fc3df85c640 1 -- 172.21.15.100:0/901134406 >> 172.21.15.100:0/901134406 conn(0x7fc3d8067dd0 msgr2=0x7fc3d806bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:41.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.813+0000 7fc3df85c640 1 -- 172.21.15.100:0/901134406 shutdown_connections 2024-09-15T21:48:41.816 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.813+0000 7fc3df85c640 1 -- 172.21.15.100:0/901134406 wait complete. 2024-09-15T21:48:41.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.814+0000 7fc3df85c640 1 Processor -- start 2024-09-15T21:48:41.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.814+0000 7fc3df85c640 1 -- start start 2024-09-15T21:48:41.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.815+0000 7fc3df85c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3d8070ee0 0x7fc3d81f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:41.818 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.815+0000 7fc3df85c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3d80cca30 0x7fc3d8149020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.815+0000 7fc3df85c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc3d80cce10 0x7fc3d814ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.815+0000 7fc3df85c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d8151ce0 con 0x7fc3d8070ee0 2024-09-15T21:48:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.815+0000 7fc3df85c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d8151b60 con 0x7fc3d80cca30 2024-09-15T21:48:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.815+0000 7fc3df85c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fc3d8071990 con 0x7fc3d80cce10 2024-09-15T21:48:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.815+0000 7fc3dd5f8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3d8070ee0 0x7fc3d81f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.815+0000 7fc3dd5f8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3d8070ee0 0x7fc3d81f2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54464/0 (socket says 172.21.15.100:54464) 2024-09-15T21:48:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.815+0000 7fc3dd5f8640 1 -- 172.21.15.100:0/1450516200 learned_addr learned my addr 172.21.15.100:0/1450516200 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.815+0000 7fc3dcdf7640 1 --2- 172.21.15.100:0/1450516200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3d80cca30 0x7fc3d8149020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.815+0000 7fc3dddf9640 1 --2- 172.21.15.100:0/1450516200 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc3d80cce10 0x7fc3d814ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.815+0000 7fc3dd5f8640 1 -- 172.21.15.100:0/1450516200 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc3d80cce10 msgr2=0x7fc3d814ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.815+0000 7fc3dd5f8640 1 --2- 172.21.15.100:0/1450516200 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc3d80cce10 0x7fc3d814ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.815+0000 7fc3dd5f8640 1 -- 172.21.15.100:0/1450516200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3d80cca30 msgr2=0x7fc3d8149020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:41.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.815+0000 7fc3dd5f8640 1 --2- 172.21.15.100:0/1450516200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3d80cca30 0x7fc3d8149020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.815+0000 7fc3dd5f8640 1 -- 172.21.15.100:0/1450516200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d8155850 con 0x7fc3d8070ee0 2024-09-15T21:48:41.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.816+0000 7fc3dd5f8640 1 --2- 172.21.15.100:0/1450516200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3d8070ee0 0x7fc3d81f2990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400a1c0 tx=0x7fc3d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:41.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.816+0000 7fc3cdffb640 1 -- 172.21.15.100:0/1450516200 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d403f020 con 0x7fc3d8070ee0 2024-09-15T21:48:41.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.816+0000 7fc3cdffb640 1 -- 172.21.15.100:0/1450516200 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3d4030e30 con 0x7fc3d8070ee0 2024-09-15T21:48:41.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.816+0000 7fc3df85c640 1 -- 172.21.15.100:0/1450516200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d8155290 con 0x7fc3d8070ee0 2024-09-15T21:48:41.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.816+0000 7fc3cdffb640 1 -- 172.21.15.100:0/1450516200 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d403b430 con 0x7fc3d8070ee0 2024-09-15T21:48:41.820 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.816+0000 7fc3df85c640 1 -- 172.21.15.100:0/1450516200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d8148540 con 0x7fc3d8070ee0 2024-09-15T21:48:41.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.817+0000 7fc3df85c640 1 -- 172.21.15.100:0/1450516200 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d8148d10 con 0x7fc3d8070ee0 2024-09-15T21:48:41.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.817+0000 7fc3cdffb640 1 -- 172.21.15.100:0/1450516200 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fc3d4042020 con 0x7fc3d8070ee0 2024-09-15T21:48:41.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.818+0000 7fc3cdffb640 1 --2- 172.21.15.100:0/1450516200 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc3bc06c2b0 0x7fc3bc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:41.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.818+0000 7fc3cdffb640 1 -- 172.21.15.100:0/1450516200 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7fc3d40abc80 con 0x7fc3d8070ee0 2024-09-15T21:48:41.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.818+0000 7fc3dcdf7640 1 --2- 172.21.15.100:0/1450516200 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc3bc06c2b0 0x7fc3bc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:41.821 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.818+0000 7fc3dcdf7640 1 --2- 172.21.15.100:0/1450516200 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc3bc06c2b0 0x7fc3bc06e770 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc3d814c1e0 tx=0x7fc3c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:41.824 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.822+0000 7fc3cdffb640 1 -- 172.21.15.100:0/1450516200 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fc3d4049030 con 0x7fc3d8070ee0 2024-09-15T21:48:41.986 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.984+0000 7fc3df85c640 1 -- 172.21.15.100:0/1450516200 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc3d814bf30 con 0x7fc3bc06c2b0 2024-09-15T21:48:41.991 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.990+0000 7fc3cdffb640 1 -- 172.21.15.100:0/1450516200 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252278 (secure 0 0 0) 0x7fc3d814bf30 con 0x7fc3bc06c2b0 2024-09-15T21:48:41.994 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:48:41.995 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-15T21:48:41.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.992+0000 7fc3df85c640 1 -- 172.21.15.100:0/1450516200 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc3bc06c2b0 msgr2=0x7fc3bc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:41.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.992+0000 7fc3df85c640 1 --2- 172.21.15.100:0/1450516200 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc3bc06c2b0 0x7fc3bc06e770 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc3d814c1e0 tx=0x7fc3c8009040 comp rx=0 tx=0).stop 2024-09-15T21:48:41.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.992+0000 7fc3df85c640 1 -- 172.21.15.100:0/1450516200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3d8070ee0 msgr2=0x7fc3d81f2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:41.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.992+0000 7fc3df85c640 1 --2- 172.21.15.100:0/1450516200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3d8070ee0 0x7fc3d81f2990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400a1c0 tx=0x7fc3d40046c0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.997 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.993+0000 7fc3df85c640 1 -- 172.21.15.100:0/1450516200 shutdown_connections 2024-09-15T21:48:41.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.993+0000 7fc3df85c640 1 --2- 172.21.15.100:0/1450516200 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fc3bc06c2b0 0x7fc3bc06e770 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.993+0000 7fc3df85c640 1 --2- 172.21.15.100:0/1450516200 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3d80cca30 0x7fc3d8149020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.993+0000 7fc3df85c640 1 --2- 172.21.15.100:0/1450516200 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fc3d80cce10 0x7fc3d814ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.993+0000 7fc3df85c640 1 --2- 172.21.15.100:0/1450516200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3d8070ee0 0x7fc3d81f2990 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:41.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.993+0000 7fc3df85c640 1 -- 172.21.15.100:0/1450516200 >> 172.21.15.100:0/1450516200 conn(0x7fc3d8067dd0 msgr2=0x7fc3d815af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:41.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.993+0000 7fc3df85c640 1 -- 172.21.15.100:0/1450516200 shutdown_connections 2024-09-15T21:48:41.998 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:41.993+0000 7fc3df85c640 1 -- 172.21.15.100:0/1450516200 wait complete. 2024-09-15T21:48:42.006 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-15T21:48:41.833082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633023,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":404,"num_write_kb":2533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1185538,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":335,"ondisk_log_size":335,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238248,"kb_used_data":8364,"kb_used_omap":18,"kb_used_meta":229677,"kb_avail":754736472,"statfs":{"total":773094113280,"available":772850147328,"internally_reserved":0,"allocated":8564736,"data_stored":4536051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19119,"internal_metadata":235189585},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.130575"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.711284+0000","last_change":"2024-09-15T21:48:21.639361+0000","last_active":"2024-09-15T21:48:35.711284+0000","last_peered":"2024-09-15T21:48:35.711284+0000","last_clean":"2024-09-15T21:48:35.711284+0000","last_became_active":"2024-09-15T21:48:21.637402+0000","last_became_peered":"2024-09-15T21:48:21.637402+0000","last_unstale":"2024-09-15T21:48:35.711284+0000","last_undegraded":"2024-09-15T21:48:35.711284+0000","last_fullsized":"2024-09-15T21:48:35.711284+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:40.643861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726138+0000","last_change":"2024-09-15T21:46:23.418716+0000","last_active":"2024-09-15T21:48:36.726138+0000","last_peered":"2024-09-15T21:48:36.726138+0000","last_clean":"2024-09-15T21:48:36.726138+0000","last_became_active":"2024-09-15T21:46:23.418471+0000","last_became_peered":"2024-09-15T21:46:23.418471+0000","last_unstale":"2024-09-15T21:48:36.726138+0000","last_undegraded":"2024-09-15T21:48:36.726138+0000","last_fullsized":"2024-09-15T21:48:36.726138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:49.599801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726536+0000","last_change":"2024-09-15T21:46:23.426404+0000","last_active":"2024-09-15T21:48:36.726536+0000","last_peered":"2024-09-15T21:48:36.726536+0000","last_clean":"2024-09-15T21:48:36.726536+0000","last_became_active":"2024-09-15T21:46:23.426146+0000","last_became_peered":"2024-09-15T21:46:23.426146+0000","last_unstale":"2024-09-15T21:48:36.726536+0000","last_undegraded":"2024-09-15T21:48:36.726536+0000","last_fullsized":"2024-09-15T21:48:36.726536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:23.626109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022058+0000","last_change":"2024-09-15T21:48:17.624363+0000","last_active":"2024-09-15T21:48:37.022058+0000","last_peered":"2024-09-15T21:48:37.022058+0000","last_clean":"2024-09-15T21:48:37.022058+0000","last_became_active":"2024-09-15T21:48:17.623606+0000","last_became_peered":"2024-09-15T21:48:17.623606+0000","last_unstale":"2024-09-15T21:48:37.022058+0000","last_undegraded":"2024-09-15T21:48:37.022058+0000","last_fullsized":"2024-09-15T21:48:37.022058+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:03.294215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710447+0000","last_change":"2024-09-15T21:48:21.640215+0000","last_active":"2024-09-15T21:48:35.710447+0000","last_peered":"2024-09-15T21:48:35.710447+0000","last_clean":"2024-09-15T21:48:35.710447+0000","last_became_active":"2024-09-15T21:48:21.638410+0000","last_became_peered":"2024-09-15T21:48:21.638410+0000","last_unstale":"2024-09-15T21:48:35.710447+0000","last_undegraded":"2024-09-15T21:48:35.710447+0000","last_fullsized":"2024-09-15T21:48:35.710447+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:43.167624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022015+0000","last_change":"2024-09-15T21:48:17.625573+0000","last_active":"2024-09-15T21:48:37.022015+0000","last_peered":"2024-09-15T21:48:37.022015+0000","last_clean":"2024-09-15T21:48:37.022015+0000","last_became_active":"2024-09-15T21:48:17.624439+0000","last_became_peered":"2024-09-15T21:48:17.624439+0000","last_unstale":"2024-09-15T21:48:37.022015+0000","last_undegraded":"2024-09-15T21:48:37.022015+0000","last_fullsized":"2024-09-15T21:48:37.022015+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:04.063675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710549+0000","last_change":"2024-09-15T21:48:21.638612+0000","last_active":"2024-09-15T21:48:35.710549+0000","last_peered":"2024-09-15T21:48:35.710549+0000","last_clean":"2024-09-15T21:48:35.710549+0000","last_became_active":"2024-09-15T21:48:21.638310+0000","last_became_peered":"2024-09-15T21:48:21.638310+0000","last_unstale":"2024-09-15T21:48:35.710549+0000","last_undegraded":"2024-09-15T21:48:35.710549+0000","last_fullsized":"2024-09-15T21:48:35.710549+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:37.297103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":99,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850506+0000","last_change":"2024-09-15T21:48:27.657047+0000","last_active":"2024-09-15T21:48:36.850506+0000","last_peered":"2024-09-15T21:48:36.850506+0000","last_clean":"2024-09-15T21:48:36.850506+0000","last_became_active":"2024-09-15T21:48:27.656800+0000","last_became_peered":"2024-09-15T21:48:27.656800+0000","last_unstale":"2024-09-15T21:48:36.850506+0000","last_undegraded":"2024-09-15T21:48:36.850506+0000","last_fullsized":"2024-09-15T21:48:36.850506+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:17.342504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.711186+0000","last_change":"2024-09-15T21:48:21.642296+0000","last_active":"2024-09-15T21:48:35.711186+0000","last_peered":"2024-09-15T21:48:35.711186+0000","last_clean":"2024-09-15T21:48:35.711186+0000","last_became_active":"2024-09-15T21:48:21.641920+0000","last_became_peered":"2024-09-15T21:48:21.641920+0000","last_unstale":"2024-09-15T21:48:35.711186+0000","last_undegraded":"2024-09-15T21:48:35.711186+0000","last_fullsized":"2024-09-15T21:48:35.711186+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:45.342068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859014+0000","last_change":"2024-09-15T21:48:17.622439+0000","last_active":"2024-09-15T21:48:36.859014+0000","last_peered":"2024-09-15T21:48:36.859014+0000","last_clean":"2024-09-15T21:48:36.859014+0000","last_became_active":"2024-09-15T21:48:17.621800+0000","last_became_peered":"2024-09-15T21:48:17.621800+0000","last_unstale":"2024-09-15T21:48:36.859014+0000","last_undegraded":"2024-09-15T21:48:36.859014+0000","last_fullsized":"2024-09-15T21:48:36.859014+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:07.312656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":122,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858947+0000","last_change":"2024-09-15T21:47:05.735261+0000","last_active":"2024-09-15T21:48:36.858947+0000","last_peered":"2024-09-15T21:48:36.858947+0000","last_clean":"2024-09-15T21:48:36.858947+0000","last_became_active":"2024-09-15T21:47:05.734921+0000","last_became_peered":"2024-09-15T21:47:05.734921+0000","last_unstale":"2024-09-15T21:48:36.858947+0000","last_undegraded":"2024-09-15T21:48:36.858947+0000","last_fullsized":"2024-09-15T21:48:36.858947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:55.787926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726086+0000","last_change":"2024-09-15T21:46:23.432534+0000","last_active":"2024-09-15T21:48:36.726086+0000","last_peered":"2024-09-15T21:48:36.726086+0000","last_clean":"2024-09-15T21:48:36.726086+0000","last_became_active":"2024-09-15T21:46:23.430170+0000","last_became_peered":"2024-09-15T21:46:23.430170+0000","last_unstale":"2024-09-15T21:48:36.726086+0000","last_undegraded":"2024-09-15T21:48:36.726086+0000","last_fullsized":"2024-09-15T21:48:36.726086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:11.463937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387398+0000","last_change":"2024-09-15T21:46:23.427841+0000","last_active":"2024-09-15T21:48:38.387398+0000","last_peered":"2024-09-15T21:48:38.387398+0000","last_clean":"2024-09-15T21:48:38.387398+0000","last_became_active":"2024-09-15T21:46:23.427621+0000","last_became_peered":"2024-09-15T21:46:23.427621+0000","last_unstale":"2024-09-15T21:48:38.387398+0000","last_undegraded":"2024-09-15T21:48:38.387398+0000","last_fullsized":"2024-09-15T21:48:38.387398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:18.331943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"55'3","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726331+0000","last_change":"2024-09-15T21:46:23.424707+0000","last_active":"2024-09-15T21:48:36.726331+0000","last_peered":"2024-09-15T21:48:36.726331+0000","last_clean":"2024-09-15T21:48:36.726331+0000","last_became_active":"2024-09-15T21:46:23.424447+0000","last_became_peered":"2024-09-15T21:46:23.424447+0000","last_unstale":"2024-09-15T21:48:36.726331+0000","last_undegraded":"2024-09-15T21:48:36.726331+0000","last_fullsized":"2024-09-15T21:48:36.726331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:23.617214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725904+0000","last_change":"2024-09-15T21:46:23.430297+0000","last_active":"2024-09-15T21:48:36.725904+0000","last_peered":"2024-09-15T21:48:36.725904+0000","last_clean":"2024-09-15T21:48:36.725904+0000","last_became_active":"2024-09-15T21:46:23.430070+0000","last_became_peered":"2024-09-15T21:46:23.430070+0000","last_unstale":"2024-09-15T21:48:36.725904+0000","last_undegraded":"2024-09-15T21:48:36.725904+0000","last_fullsized":"2024-09-15T21:48:36.725904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:03:19.615022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021835+0000","last_change":"2024-09-15T21:48:17.625113+0000","last_active":"2024-09-15T21:48:37.021835+0000","last_peered":"2024-09-15T21:48:37.021835+0000","last_clean":"2024-09-15T21:48:37.021835+0000","last_became_active":"2024-09-15T21:48:17.624764+0000","last_became_peered":"2024-09-15T21:48:17.624764+0000","last_unstale":"2024-09-15T21:48:37.021835+0000","last_undegraded":"2024-09-15T21:48:37.021835+0000","last_fullsized":"2024-09-15T21:48:37.021835+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:28:44.142358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022276+0000","last_change":"2024-09-15T21:48:21.642646+0000","last_active":"2024-09-15T21:48:37.022276+0000","last_peered":"2024-09-15T21:48:37.022276+0000","last_clean":"2024-09-15T21:48:37.022276+0000","last_became_active":"2024-09-15T21:48:21.642303+0000","last_became_peered":"2024-09-15T21:48:21.642303+0000","last_unstale":"2024-09-15T21:48:37.022276+0000","last_undegraded":"2024-09-15T21:48:37.022276+0000","last_fullsized":"2024-09-15T21:48:37.022276+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:08:54.511282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":75,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850781+0000","last_change":"2024-09-15T21:48:27.661631+0000","last_active":"2024-09-15T21:48:36.850781+0000","last_peered":"2024-09-15T21:48:36.850781+0000","last_clean":"2024-09-15T21:48:36.850781+0000","last_became_active":"2024-09-15T21:48:27.661441+0000","last_became_peered":"2024-09-15T21:48:27.661441+0000","last_unstale":"2024-09-15T21:48:36.850781+0000","last_undegraded":"2024-09-15T21:48:36.850781+0000","last_fullsized":"2024-09-15T21:48:36.850781+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:23.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710445+0000","last_change":"2024-09-15T21:48:21.639602+0000","last_active":"2024-09-15T21:48:35.710445+0000","last_peered":"2024-09-15T21:48:35.710445+0000","last_clean":"2024-09-15T21:48:35.710445+0000","last_became_active":"2024-09-15T21:48:21.639009+0000","last_became_peered":"2024-09-15T21:48:21.639009+0000","last_unstale":"2024-09-15T21:48:35.710445+0000","last_undegraded":"2024-09-15T21:48:35.710445+0000","last_fullsized":"2024-09-15T21:48:35.710445+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:16.818783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858858+0000","last_change":"2024-09-15T21:46:23.434558+0000","last_active":"2024-09-15T21:48:36.858858+0000","last_peered":"2024-09-15T21:48:36.858858+0000","last_clean":"2024-09-15T21:48:36.858858+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:48:36.858858+0000","last_undegraded":"2024-09-15T21:48:36.858858+0000","last_fullsized":"2024-09-15T21:48:36.858858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:03:39.562977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726908+0000","last_change":"2024-09-15T21:48:21.642201+0000","last_active":"2024-09-15T21:48:36.726908+0000","last_peered":"2024-09-15T21:48:36.726908+0000","last_clean":"2024-09-15T21:48:36.726908+0000","last_became_active":"2024-09-15T21:48:21.641876+0000","last_became_peered":"2024-09-15T21:48:21.641876+0000","last_unstale":"2024-09-15T21:48:36.726908+0000","last_undegraded":"2024-09-15T21:48:36.726908+0000","last_fullsized":"2024-09-15T21:48:36.726908+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:41:59.764563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021785+0000","last_change":"2024-09-15T21:48:21.638398+0000","last_active":"2024-09-15T21:48:37.021785+0000","last_peered":"2024-09-15T21:48:37.021785+0000","last_clean":"2024-09-15T21:48:37.021785+0000","last_became_active":"2024-09-15T21:48:21.638073+0000","last_became_peered":"2024-09-15T21:48:21.638073+0000","last_unstale":"2024-09-15T21:48:37.021785+0000","last_undegraded":"2024-09-15T21:48:37.021785+0000","last_fullsized":"2024-09-15T21:48:37.021785+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:45.308950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387331+0000","last_change":"2024-09-15T21:46:23.431527+0000","last_active":"2024-09-15T21:48:38.387331+0000","last_peered":"2024-09-15T21:48:38.387331+0000","last_clean":"2024-09-15T21:48:38.387331+0000","last_became_active":"2024-09-15T21:46:23.430719+0000","last_became_peered":"2024-09-15T21:46:23.430719+0000","last_unstale":"2024-09-15T21:48:38.387331+0000","last_undegraded":"2024-09-15T21:48:38.387331+0000","last_fullsized":"2024-09-15T21:48:38.387331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:16.326529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021766+0000","last_change":"2024-09-15T21:48:17.622578+0000","last_active":"2024-09-15T21:48:37.021766+0000","last_peered":"2024-09-15T21:48:37.021766+0000","last_clean":"2024-09-15T21:48:37.021766+0000","last_became_active":"2024-09-15T21:48:17.621639+0000","last_became_peered":"2024-09-15T21:48:17.621639+0000","last_unstale":"2024-09-15T21:48:37.021766+0000","last_undegraded":"2024-09-15T21:48:37.021766+0000","last_fullsized":"2024-09-15T21:48:37.021766+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:57.099730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387622+0000","last_change":"2024-09-15T21:46:23.432646+0000","last_active":"2024-09-15T21:48:38.387622+0000","last_peered":"2024-09-15T21:48:38.387622+0000","last_clean":"2024-09-15T21:48:38.387622+0000","last_became_active":"2024-09-15T21:46:23.432317+0000","last_became_peered":"2024-09-15T21:46:23.432317+0000","last_unstale":"2024-09-15T21:48:38.387622+0000","last_undegraded":"2024-09-15T21:48:38.387622+0000","last_fullsized":"2024-09-15T21:48:38.387622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:49:07.706450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859042+0000","last_change":"2024-09-15T21:48:21.638549+0000","last_active":"2024-09-15T21:48:36.859042+0000","last_peered":"2024-09-15T21:48:36.859042+0000","last_clean":"2024-09-15T21:48:36.859042+0000","last_became_active":"2024-09-15T21:48:21.638212+0000","last_became_peered":"2024-09-15T21:48:21.638212+0000","last_unstale":"2024-09-15T21:48:36.859042+0000","last_undegraded":"2024-09-15T21:48:36.859042+0000","last_fullsized":"2024-09-15T21:48:36.859042+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:18.788661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726213+0000","last_change":"2024-09-15T21:46:23.421749+0000","last_active":"2024-09-15T21:48:36.726213+0000","last_peered":"2024-09-15T21:48:36.726213+0000","last_clean":"2024-09-15T21:48:36.726213+0000","last_became_active":"2024-09-15T21:46:23.421501+0000","last_became_peered":"2024-09-15T21:46:23.421501+0000","last_unstale":"2024-09-15T21:48:36.726213+0000","last_undegraded":"2024-09-15T21:48:36.726213+0000","last_fullsized":"2024-09-15T21:48:36.726213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:55.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387611+0000","last_change":"2024-09-15T21:46:23.432591+0000","last_active":"2024-09-15T21:48:38.387611+0000","last_peered":"2024-09-15T21:48:38.387611+0000","last_clean":"2024-09-15T21:48:38.387611+0000","last_became_active":"2024-09-15T21:46:23.432326+0000","last_became_peered":"2024-09-15T21:46:23.432326+0000","last_unstale":"2024-09-15T21:48:38.387611+0000","last_undegraded":"2024-09-15T21:48:38.387611+0000","last_fullsized":"2024-09-15T21:48:38.387611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:48.565579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.727013+0000","last_change":"2024-09-15T21:46:23.423428+0000","last_active":"2024-09-15T21:48:36.727013+0000","last_peered":"2024-09-15T21:48:36.727013+0000","last_clean":"2024-09-15T21:48:36.727013+0000","last_became_active":"2024-09-15T21:46:23.422734+0000","last_became_peered":"2024-09-15T21:46:23.422734+0000","last_unstale":"2024-09-15T21:48:36.727013+0000","last_undegraded":"2024-09-15T21:48:36.727013+0000","last_fullsized":"2024-09-15T21:48:36.727013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:06.592891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021899+0000","last_change":"2024-09-15T21:48:17.622914+0000","last_active":"2024-09-15T21:48:37.021899+0000","last_peered":"2024-09-15T21:48:37.021899+0000","last_clean":"2024-09-15T21:48:37.021899+0000","last_became_active":"2024-09-15T21:48:17.621473+0000","last_became_peered":"2024-09-15T21:48:17.621473+0000","last_unstale":"2024-09-15T21:48:37.021899+0000","last_undegraded":"2024-09-15T21:48:37.021899+0000","last_fullsized":"2024-09-15T21:48:37.021899+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:45.297321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850684+0000","last_change":"2024-09-15T21:48:17.616723+0000","last_active":"2024-09-15T21:48:36.850684+0000","last_peered":"2024-09-15T21:48:36.850684+0000","last_clean":"2024-09-15T21:48:36.850684+0000","last_became_active":"2024-09-15T21:48:17.616499+0000","last_became_peered":"2024-09-15T21:48:17.616499+0000","last_unstale":"2024-09-15T21:48:36.850684+0000","last_undegraded":"2024-09-15T21:48:36.850684+0000","last_fullsized":"2024-09-15T21:48:36.850684+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:29:36.886350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858637+0000","last_change":"2024-09-15T21:46:23.429648+0000","last_active":"2024-09-15T21:48:36.858637+0000","last_peered":"2024-09-15T21:48:36.858637+0000","last_clean":"2024-09-15T21:48:36.858637+0000","last_became_active":"2024-09-15T21:46:23.429376+0000","last_became_peered":"2024-09-15T21:46:23.429376+0000","last_unstale":"2024-09-15T21:48:36.858637+0000","last_undegraded":"2024-09-15T21:48:36.858637+0000","last_fullsized":"2024-09-15T21:48:36.858637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:31.278283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725723+0000","last_change":"2024-09-15T21:46:23.429847+0000","last_active":"2024-09-15T21:48:36.725723+0000","last_peered":"2024-09-15T21:48:36.725723+0000","last_clean":"2024-09-15T21:48:36.725723+0000","last_became_active":"2024-09-15T21:46:23.429442+0000","last_became_peered":"2024-09-15T21:46:23.429442+0000","last_unstale":"2024-09-15T21:48:36.725723+0000","last_undegraded":"2024-09-15T21:48:36.725723+0000","last_fullsized":"2024-09-15T21:48:36.725723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:31.232512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'6","reported_seq":149,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387490+0000","last_change":"2024-09-15T21:48:21.665708+0000","last_active":"2024-09-15T21:48:38.387490+0000","last_peered":"2024-09-15T21:48:38.387490+0000","last_clean":"2024-09-15T21:48:38.387490+0000","last_became_active":"2024-09-15T21:48:21.636972+0000","last_became_peered":"2024-09-15T21:48:21.636972+0000","last_unstale":"2024-09-15T21:48:38.387490+0000","last_undegraded":"2024-09-15T21:48:38.387490+0000","last_fullsized":"2024-09-15T21:48:38.387490+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:24:33.658234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1756,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":117,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.727058+0000","last_change":"2024-09-15T21:46:23.434198+0000","last_active":"2024-09-15T21:48:36.727058+0000","last_peered":"2024-09-15T21:48:36.727058+0000","last_clean":"2024-09-15T21:48:36.727058+0000","last_became_active":"2024-09-15T21:46:23.433944+0000","last_became_peered":"2024-09-15T21:46:23.433944+0000","last_unstale":"2024-09-15T21:48:36.727058+0000","last_undegraded":"2024-09-15T21:48:36.727058+0000","last_fullsized":"2024-09-15T21:48:36.727058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:25.412167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'7","reported_seq":126,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858780+0000","last_change":"2024-09-15T21:46:22.425571+0000","last_active":"2024-09-15T21:48:36.858780+0000","last_peered":"2024-09-15T21:48:36.858780+0000","last_clean":"2024-09-15T21:48:36.858780+0000","last_became_active":"2024-09-15T21:46:22.425315+0000","last_became_peered":"2024-09-15T21:46:22.425315+0000","last_unstale":"2024-09-15T21:48:36.858780+0000","last_undegraded":"2024-09-15T21:48:36.858780+0000","last_fullsized":"2024-09-15T21:48:36.858780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:31.983301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'3","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387375+0000","last_change":"2024-09-15T21:46:22.425492+0000","last_active":"2024-09-15T21:48:38.387375+0000","last_peered":"2024-09-15T21:48:38.387375+0000","last_clean":"2024-09-15T21:48:38.387375+0000","last_became_active":"2024-09-15T21:46:22.425226+0000","last_became_peered":"2024-09-15T21:46:22.425226+0000","last_unstale":"2024-09-15T21:48:38.387375+0000","last_undegraded":"2024-09-15T21:48:38.387375+0000","last_fullsized":"2024-09-15T21:48:38.387375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:24.322384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387260+0000","last_change":"2024-09-15T21:46:23.429140+0000","last_active":"2024-09-15T21:48:38.387260+0000","last_peered":"2024-09-15T21:48:38.387260+0000","last_clean":"2024-09-15T21:48:38.387260+0000","last_became_active":"2024-09-15T21:46:23.428902+0000","last_became_peered":"2024-09-15T21:46:23.428902+0000","last_unstale":"2024-09-15T21:48:38.387260+0000","last_undegraded":"2024-09-15T21:48:38.387260+0000","last_fullsized":"2024-09-15T21:48:38.387260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:11.256924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'3","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021677+0000","last_change":"2024-09-15T21:48:17.618127+0000","last_active":"2024-09-15T21:48:37.021677+0000","last_peered":"2024-09-15T21:48:37.021677+0000","last_clean":"2024-09-15T21:48:37.021677+0000","last_became_active":"2024-09-15T21:48:17.616704+0000","last_became_peered":"2024-09-15T21:48:17.616704+0000","last_unstale":"2024-09-15T21:48:37.021677+0000","last_undegraded":"2024-09-15T21:48:37.021677+0000","last_fullsized":"2024-09-15T21:48:37.021677+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:09.775586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725761+0000","last_change":"2024-09-15T21:46:23.427442+0000","last_active":"2024-09-15T21:48:36.725761+0000","last_peered":"2024-09-15T21:48:36.725761+0000","last_clean":"2024-09-15T21:48:36.725761+0000","last_became_active":"2024-09-15T21:46:23.427200+0000","last_became_peered":"2024-09-15T21:46:23.427200+0000","last_unstale":"2024-09-15T21:48:36.725761+0000","last_undegraded":"2024-09-15T21:48:36.725761+0000","last_fullsized":"2024-09-15T21:48:36.725761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:27.248545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"55'6","reported_seq":133,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859137+0000","last_change":"2024-09-15T21:48:17.622236+0000","last_active":"2024-09-15T21:48:36.859137+0000","last_peered":"2024-09-15T21:48:36.859137+0000","last_clean":"2024-09-15T21:48:36.859137+0000","last_became_active":"2024-09-15T21:48:17.621860+0000","last_became_peered":"2024-09-15T21:48:17.621860+0000","last_unstale":"2024-09-15T21:48:36.859137+0000","last_undegraded":"2024-09-15T21:48:36.859137+0000","last_fullsized":"2024-09-15T21:48:36.859137+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:00.032847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022154+0000","last_change":"2024-09-15T21:48:21.638639+0000","last_active":"2024-09-15T21:48:37.022154+0000","last_peered":"2024-09-15T21:48:37.022154+0000","last_clean":"2024-09-15T21:48:37.022154+0000","last_became_active":"2024-09-15T21:48:21.638300+0000","last_became_peered":"2024-09-15T21:48:21.638300+0000","last_unstale":"2024-09-15T21:48:37.022154+0000","last_undegraded":"2024-09-15T21:48:37.022154+0000","last_fullsized":"2024-09-15T21:48:37.022154+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:09:01.739184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'4","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022156+0000","last_change":"2024-09-15T21:48:17.623971+0000","last_active":"2024-09-15T21:48:37.022156+0000","last_peered":"2024-09-15T21:48:37.022156+0000","last_clean":"2024-09-15T21:48:37.022156+0000","last_became_active":"2024-09-15T21:48:17.623246+0000","last_became_peered":"2024-09-15T21:48:17.623246+0000","last_unstale":"2024-09-15T21:48:37.022156+0000","last_undegraded":"2024-09-15T21:48:37.022156+0000","last_fullsized":"2024-09-15T21:48:37.022156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:51:37.738581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726718+0000","last_change":"2024-09-15T21:48:17.625879+0000","last_active":"2024-09-15T21:48:36.726718+0000","last_peered":"2024-09-15T21:48:36.726718+0000","last_clean":"2024-09-15T21:48:36.726718+0000","last_became_active":"2024-09-15T21:48:17.625642+0000","last_became_peered":"2024-09-15T21:48:17.625642+0000","last_unstale":"2024-09-15T21:48:36.726718+0000","last_undegraded":"2024-09-15T21:48:36.726718+0000","last_fullsized":"2024-09-15T21:48:36.726718+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:41.396163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"55'7","reported_seq":139,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726301+0000","last_change":"2024-09-15T21:48:17.627957+0000","last_active":"2024-09-15T21:48:36.726301+0000","last_peered":"2024-09-15T21:48:36.726301+0000","last_clean":"2024-09-15T21:48:36.726301+0000","last_became_active":"2024-09-15T21:48:17.622409+0000","last_became_peered":"2024-09-15T21:48:17.622409+0000","last_unstale":"2024-09-15T21:48:36.726301+0000","last_undegraded":"2024-09-15T21:48:36.726301+0000","last_fullsized":"2024-09-15T21:48:36.726301+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:39.534804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":130,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726177+0000","last_change":"2024-09-15T21:48:21.637218+0000","last_active":"2024-09-15T21:48:36.726177+0000","last_peered":"2024-09-15T21:48:36.726177+0000","last_clean":"2024-09-15T21:48:36.726177+0000","last_became_active":"2024-09-15T21:48:21.636913+0000","last_became_peered":"2024-09-15T21:48:21.636913+0000","last_unstale":"2024-09-15T21:48:36.726177+0000","last_undegraded":"2024-09-15T21:48:36.726177+0000","last_fullsized":"2024-09-15T21:48:36.726177+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:36:05.649762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"55'24","reported_seq":143,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858914+0000","last_change":"2024-09-15T21:46:22.426151+0000","last_active":"2024-09-15T21:48:36.858914+0000","last_peered":"2024-09-15T21:48:36.858914+0000","last_clean":"2024-09-15T21:48:36.858914+0000","last_became_active":"2024-09-15T21:46:22.425866+0000","last_became_peered":"2024-09-15T21:46:22.425866+0000","last_unstale":"2024-09-15T21:48:36.858914+0000","last_undegraded":"2024-09-15T21:48:36.858914+0000","last_fullsized":"2024-09-15T21:48:36.858914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:10.684265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35573,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710799+0000","last_change":"2024-09-15T21:48:21.637466+0000","last_active":"2024-09-15T21:48:35.710799+0000","last_peered":"2024-09-15T21:48:35.710799+0000","last_clean":"2024-09-15T21:48:35.710799+0000","last_became_active":"2024-09-15T21:48:21.637151+0000","last_became_peered":"2024-09-15T21:48:21.637151+0000","last_unstale":"2024-09-15T21:48:35.710799+0000","last_undegraded":"2024-09-15T21:48:35.710799+0000","last_fullsized":"2024-09-15T21:48:35.710799+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:22:57.287004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'9","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.727151+0000","last_change":"2024-09-15T21:46:22.427415+0000","last_active":"2024-09-15T21:48:36.727151+0000","last_peered":"2024-09-15T21:48:36.727151+0000","last_clean":"2024-09-15T21:48:36.727151+0000","last_became_active":"2024-09-15T21:46:22.427087+0000","last_became_peered":"2024-09-15T21:46:22.427087+0000","last_unstale":"2024-09-15T21:48:36.727151+0000","last_undegraded":"2024-09-15T21:48:36.727151+0000","last_fullsized":"2024-09-15T21:48:36.727151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:09.365202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":127,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726487+0000","last_change":"2024-09-15T21:48:21.635299+0000","last_active":"2024-09-15T21:48:36.726487+0000","last_peered":"2024-09-15T21:48:36.726487+0000","last_clean":"2024-09-15T21:48:36.726487+0000","last_became_active":"2024-09-15T21:48:21.635022+0000","last_became_peered":"2024-09-15T21:48:21.635022+0000","last_unstale":"2024-09-15T21:48:36.726487+0000","last_undegraded":"2024-09-15T21:48:36.726487+0000","last_fullsized":"2024-09-15T21:48:36.726487+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:01.482588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"55'6","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021757+0000","last_change":"2024-09-15T21:48:17.625975+0000","last_active":"2024-09-15T21:48:37.021757+0000","last_peered":"2024-09-15T21:48:37.021757+0000","last_clean":"2024-09-15T21:48:37.021757+0000","last_became_active":"2024-09-15T21:48:17.625232+0000","last_became_peered":"2024-09-15T21:48:17.625232+0000","last_unstale":"2024-09-15T21:48:37.021757+0000","last_undegraded":"2024-09-15T21:48:37.021757+0000","last_fullsized":"2024-09-15T21:48:37.021757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:21.061733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710557+0000","last_change":"2024-09-15T21:48:21.639654+0000","last_active":"2024-09-15T21:48:35.710557+0000","last_peered":"2024-09-15T21:48:35.710557+0000","last_clean":"2024-09-15T21:48:35.710557+0000","last_became_active":"2024-09-15T21:48:21.638780+0000","last_became_peered":"2024-09-15T21:48:21.638780+0000","last_unstale":"2024-09-15T21:48:35.710557+0000","last_undegraded":"2024-09-15T21:48:35.710557+0000","last_fullsized":"2024-09-15T21:48:35.710557+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:16.549734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'5","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858678+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:48:36.858678+0000","last_peered":"2024-09-15T21:48:36.858678+0000","last_clean":"2024-09-15T21:48:36.858678+0000","last_became_active":"2024-09-15T21:46:22.425237+0000","last_became_peered":"2024-09-15T21:46:22.425237+0000","last_unstale":"2024-09-15T21:48:36.858678+0000","last_undegraded":"2024-09-15T21:48:36.858678+0000","last_fullsized":"2024-09-15T21:48:36.858678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:59.460420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":117,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858649+0000","last_change":"2024-09-15T21:46:23.427515+0000","last_active":"2024-09-15T21:48:36.858649+0000","last_peered":"2024-09-15T21:48:36.858649+0000","last_clean":"2024-09-15T21:48:36.858649+0000","last_became_active":"2024-09-15T21:46:23.427116+0000","last_became_peered":"2024-09-15T21:46:23.427116+0000","last_unstale":"2024-09-15T21:48:36.858649+0000","last_undegraded":"2024-09-15T21:48:36.858649+0000","last_fullsized":"2024-09-15T21:48:36.858649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:35.499970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"55'18","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850613+0000","last_change":"2024-09-15T21:46:22.419905+0000","last_active":"2024-09-15T21:48:36.850613+0000","last_peered":"2024-09-15T21:48:36.850613+0000","last_clean":"2024-09-15T21:48:36.850613+0000","last_became_active":"2024-09-15T21:46:22.419620+0000","last_became_peered":"2024-09-15T21:46:22.419620+0000","last_unstale":"2024-09-15T21:48:36.850613+0000","last_undegraded":"2024-09-15T21:48:36.850613+0000","last_fullsized":"2024-09-15T21:48:36.850613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:26:51.215156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710002+0000","last_change":"2024-09-15T21:48:21.639146+0000","last_active":"2024-09-15T21:48:35.710002+0000","last_peered":"2024-09-15T21:48:35.710002+0000","last_clean":"2024-09-15T21:48:35.710002+0000","last_became_active":"2024-09-15T21:48:21.637855+0000","last_became_peered":"2024-09-15T21:48:21.637855+0000","last_unstale":"2024-09-15T21:48:35.710002+0000","last_undegraded":"2024-09-15T21:48:35.710002+0000","last_fullsized":"2024-09-15T21:48:35.710002+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:35:29.305298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'2","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726069+0000","last_change":"2024-09-15T21:46:22.429661+0000","last_active":"2024-09-15T21:48:36.726069+0000","last_peered":"2024-09-15T21:48:36.726069+0000","last_clean":"2024-09-15T21:48:36.726069+0000","last_became_active":"2024-09-15T21:46:22.429250+0000","last_became_peered":"2024-09-15T21:46:22.429250+0000","last_unstale":"2024-09-15T21:48:36.726069+0000","last_undegraded":"2024-09-15T21:48:36.726069+0000","last_fullsized":"2024-09-15T21:48:36.726069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:29.341739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710978+0000","last_change":"2024-09-15T21:48:21.635384+0000","last_active":"2024-09-15T21:48:35.710978+0000","last_peered":"2024-09-15T21:48:35.710978+0000","last_clean":"2024-09-15T21:48:35.710978+0000","last_became_active":"2024-09-15T21:48:21.635051+0000","last_became_peered":"2024-09-15T21:48:21.635051+0000","last_unstale":"2024-09-15T21:48:35.710978+0000","last_undegraded":"2024-09-15T21:48:35.710978+0000","last_fullsized":"2024-09-15T21:48:35.710978+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:59.199568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'5","reported_seq":83,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850941+0000","last_change":"2024-09-15T21:48:27.660076+0000","last_active":"2024-09-15T21:48:36.850941+0000","last_peered":"2024-09-15T21:48:36.850941+0000","last_clean":"2024-09-15T21:48:36.850941+0000","last_became_active":"2024-09-15T21:48:27.659750+0000","last_became_peered":"2024-09-15T21:48:27.659750+0000","last_unstale":"2024-09-15T21:48:36.850941+0000","last_undegraded":"2024-09-15T21:48:36.850941+0000","last_fullsized":"2024-09-15T21:48:36.850941+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:06.505112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.711325+0000","last_change":"2024-09-15T21:48:21.641556+0000","last_active":"2024-09-15T21:48:35.711325+0000","last_peered":"2024-09-15T21:48:35.711325+0000","last_clean":"2024-09-15T21:48:35.711325+0000","last_became_active":"2024-09-15T21:48:21.639328+0000","last_became_peered":"2024-09-15T21:48:21.639328+0000","last_unstale":"2024-09-15T21:48:35.711325+0000","last_undegraded":"2024-09-15T21:48:35.711325+0000","last_fullsized":"2024-09-15T21:48:35.711325+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:36:11.470048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'3","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726437+0000","last_change":"2024-09-15T21:46:22.425072+0000","last_active":"2024-09-15T21:48:36.726437+0000","last_peered":"2024-09-15T21:48:36.726437+0000","last_clean":"2024-09-15T21:48:36.726437+0000","last_became_active":"2024-09-15T21:46:22.424771+0000","last_became_peered":"2024-09-15T21:46:22.424771+0000","last_unstale":"2024-09-15T21:48:36.726437+0000","last_undegraded":"2024-09-15T21:48:36.726437+0000","last_fullsized":"2024-09-15T21:48:36.726437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:26.256785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022156+0000","last_change":"2024-09-15T21:48:17.619698+0000","last_active":"2024-09-15T21:48:37.022156+0000","last_peered":"2024-09-15T21:48:37.022156+0000","last_clean":"2024-09-15T21:48:37.022156+0000","last_became_active":"2024-09-15T21:48:17.617701+0000","last_became_peered":"2024-09-15T21:48:17.617701+0000","last_unstale":"2024-09-15T21:48:37.022156+0000","last_undegraded":"2024-09-15T21:48:37.022156+0000","last_fullsized":"2024-09-15T21:48:37.022156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:00.170687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'5","reported_seq":132,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850794+0000","last_change":"2024-09-15T21:48:17.619089+0000","last_active":"2024-09-15T21:48:36.850794+0000","last_peered":"2024-09-15T21:48:36.850794+0000","last_clean":"2024-09-15T21:48:36.850794+0000","last_became_active":"2024-09-15T21:48:17.618782+0000","last_became_peered":"2024-09-15T21:48:17.618782+0000","last_unstale":"2024-09-15T21:48:36.850794+0000","last_undegraded":"2024-09-15T21:48:36.850794+0000","last_fullsized":"2024-09-15T21:48:36.850794+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:17.881528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710339+0000","last_change":"2024-09-15T21:48:21.639356+0000","last_active":"2024-09-15T21:48:35.710339+0000","last_peered":"2024-09-15T21:48:35.710339+0000","last_clean":"2024-09-15T21:48:35.710339+0000","last_became_active":"2024-09-15T21:48:21.637883+0000","last_became_peered":"2024-09-15T21:48:21.637883+0000","last_unstale":"2024-09-15T21:48:35.710339+0000","last_undegraded":"2024-09-15T21:48:35.710339+0000","last_fullsized":"2024-09-15T21:48:35.710339+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:27.061996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":236,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710002+0000","last_change":"2024-09-15T21:48:24.647129+0000","last_active":"2024-09-15T21:48:35.710002+0000","last_peered":"2024-09-15T21:48:35.710002+0000","last_clean":"2024-09-15T21:48:35.710002+0000","last_became_active":"2024-09-15T21:48:24.646783+0000","last_became_peered":"2024-09-15T21:48:24.646783+0000","last_unstale":"2024-09-15T21:48:35.710002+0000","last_undegraded":"2024-09-15T21:48:35.710002+0000","last_fullsized":"2024-09-15T21:48:35.710002+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_clean_scrub_stamp":"2024-09-15T21:46:12.223400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:20.467980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'6","reported_seq":68,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021854+0000","last_change":"2024-09-15T21:48:17.622619+0000","last_active":"2024-09-15T21:48:37.021854+0000","last_peered":"2024-09-15T21:48:37.021854+0000","last_clean":"2024-09-15T21:48:37.021854+0000","last_became_active":"2024-09-15T21:48:17.622159+0000","last_became_peered":"2024-09-15T21:48:17.622159+0000","last_unstale":"2024-09-15T21:48:37.021854+0000","last_undegraded":"2024-09-15T21:48:37.021854+0000","last_fullsized":"2024-09-15T21:48:37.021854+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:51.566702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":127,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387385+0000","last_change":"2024-09-15T21:48:17.621009+0000","last_active":"2024-09-15T21:48:38.387385+0000","last_peered":"2024-09-15T21:48:38.387385+0000","last_clean":"2024-09-15T21:48:38.387385+0000","last_became_active":"2024-09-15T21:48:17.620738+0000","last_became_peered":"2024-09-15T21:48:17.620738+0000","last_unstale":"2024-09-15T21:48:38.387385+0000","last_undegraded":"2024-09-15T21:48:38.387385+0000","last_fullsized":"2024-09-15T21:48:38.387385+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:20.095876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'4","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710013+0000","last_change":"2024-09-15T21:48:21.645645+0000","last_active":"2024-09-15T21:48:35.710013+0000","last_peered":"2024-09-15T21:48:35.710013+0000","last_clean":"2024-09-15T21:48:35.710013+0000","last_became_active":"2024-09-15T21:48:21.636886+0000","last_became_peered":"2024-09-15T21:48:21.636886+0000","last_unstale":"2024-09-15T21:48:35.710013+0000","last_undegraded":"2024-09-15T21:48:35.710013+0000","last_fullsized":"2024-09-15T21:48:35.710013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:12.676508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726029+0000","last_change":"2024-09-15T21:46:23.430396+0000","last_active":"2024-09-15T21:48:36.726029+0000","last_peered":"2024-09-15T21:48:36.726029+0000","last_clean":"2024-09-15T21:48:36.726029+0000","last_became_active":"2024-09-15T21:46:23.430138+0000","last_became_peered":"2024-09-15T21:46:23.430138+0000","last_unstale":"2024-09-15T21:48:36.726029+0000","last_undegraded":"2024-09-15T21:48:36.726029+0000","last_fullsized":"2024-09-15T21:48:36.726029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:50.622949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"55'7","reported_seq":80,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850706+0000","last_change":"2024-09-15T21:48:27.660216+0000","last_active":"2024-09-15T21:48:36.850706+0000","last_peered":"2024-09-15T21:48:36.850706+0000","last_clean":"2024-09-15T21:48:36.850706+0000","last_became_active":"2024-09-15T21:48:27.659880+0000","last_became_peered":"2024-09-15T21:48:27.659880+0000","last_unstale":"2024-09-15T21:48:36.850706+0000","last_undegraded":"2024-09-15T21:48:36.850706+0000","last_fullsized":"2024-09-15T21:48:36.850706+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:48.186260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":516,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725858+0000","last_change":"2024-09-15T21:46:23.427051+0000","last_active":"2024-09-15T21:48:36.725858+0000","last_peered":"2024-09-15T21:48:36.725858+0000","last_clean":"2024-09-15T21:48:36.725858+0000","last_became_active":"2024-09-15T21:46:23.426185+0000","last_became_peered":"2024-09-15T21:46:23.426185+0000","last_unstale":"2024-09-15T21:48:36.725858+0000","last_undegraded":"2024-09-15T21:48:36.725858+0000","last_fullsized":"2024-09-15T21:48:36.725858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:48.318405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'5","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021901+0000","last_change":"2024-09-15T21:48:17.625702+0000","last_active":"2024-09-15T21:48:37.021901+0000","last_peered":"2024-09-15T21:48:37.021901+0000","last_clean":"2024-09-15T21:48:37.021901+0000","last_became_active":"2024-09-15T21:48:17.623936+0000","last_became_peered":"2024-09-15T21:48:17.623936+0000","last_unstale":"2024-09-15T21:48:37.021901+0000","last_undegraded":"2024-09-15T21:48:37.021901+0000","last_fullsized":"2024-09-15T21:48:37.021901+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:53.114831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":121,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725863+0000","last_change":"2024-09-15T21:46:23.425699+0000","last_active":"2024-09-15T21:48:36.725863+0000","last_peered":"2024-09-15T21:48:36.725863+0000","last_clean":"2024-09-15T21:48:36.725863+0000","last_became_active":"2024-09-15T21:46:23.425416+0000","last_became_peered":"2024-09-15T21:46:23.425416+0000","last_unstale":"2024-09-15T21:48:36.725863+0000","last_undegraded":"2024-09-15T21:48:36.725863+0000","last_fullsized":"2024-09-15T21:48:36.725863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:00.325179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"22'1","reported_seq":122,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726645+0000","last_change":"2024-09-15T21:46:22.426492+0000","last_active":"2024-09-15T21:48:36.726645+0000","last_peered":"2024-09-15T21:48:36.726645+0000","last_clean":"2024-09-15T21:48:36.726645+0000","last_became_active":"2024-09-15T21:46:22.426210+0000","last_became_peered":"2024-09-15T21:46:22.426210+0000","last_unstale":"2024-09-15T21:48:36.726645+0000","last_undegraded":"2024-09-15T21:48:36.726645+0000","last_fullsized":"2024-09-15T21:48:36.726645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:05.248091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022064+0000","last_change":"2024-09-15T21:48:17.625501+0000","last_active":"2024-09-15T21:48:37.022064+0000","last_peered":"2024-09-15T21:48:37.022064+0000","last_clean":"2024-09-15T21:48:37.022064+0000","last_became_active":"2024-09-15T21:48:17.624852+0000","last_became_peered":"2024-09-15T21:48:17.624852+0000","last_unstale":"2024-09-15T21:48:37.022064+0000","last_undegraded":"2024-09-15T21:48:37.022064+0000","last_fullsized":"2024-09-15T21:48:37.022064+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:47:09.731108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"22'4","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710548+0000","last_change":"2024-09-15T21:48:21.668174+0000","last_active":"2024-09-15T21:48:35.710548+0000","last_peered":"2024-09-15T21:48:35.710548+0000","last_clean":"2024-09-15T21:48:35.710548+0000","last_became_active":"2024-09-15T21:48:21.639593+0000","last_became_peered":"2024-09-15T21:48:21.639593+0000","last_unstale":"2024-09-15T21:48:35.710548+0000","last_undegraded":"2024-09-15T21:48:35.710548+0000","last_fullsized":"2024-09-15T21:48:35.710548+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:05.341807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725928+0000","last_change":"2024-09-15T21:46:23.426729+0000","last_active":"2024-09-15T21:48:36.725928+0000","last_peered":"2024-09-15T21:48:36.725928+0000","last_clean":"2024-09-15T21:48:36.725928+0000","last_became_active":"2024-09-15T21:46:23.426495+0000","last_became_peered":"2024-09-15T21:46:23.426495+0000","last_unstale":"2024-09-15T21:48:36.725928+0000","last_undegraded":"2024-09-15T21:48:36.725928+0000","last_fullsized":"2024-09-15T21:48:36.725928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:03.277997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"55'16","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726226+0000","last_change":"2024-09-15T21:46:22.427137+0000","last_active":"2024-09-15T21:48:36.726226+0000","last_peered":"2024-09-15T21:48:36.726226+0000","last_clean":"2024-09-15T21:48:36.726226+0000","last_became_active":"2024-09-15T21:46:22.426855+0000","last_became_peered":"2024-09-15T21:46:22.426855+0000","last_unstale":"2024-09-15T21:48:36.726226+0000","last_undegraded":"2024-09-15T21:48:36.726226+0000","last_fullsized":"2024-09-15T21:48:36.726226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:40:14.277940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":130,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387631+0000","last_change":"2024-09-15T21:48:21.639860+0000","last_active":"2024-09-15T21:48:38.387631+0000","last_peered":"2024-09-15T21:48:38.387631+0000","last_clean":"2024-09-15T21:48:38.387631+0000","last_became_active":"2024-09-15T21:48:21.639541+0000","last_became_peered":"2024-09-15T21:48:21.639541+0000","last_unstale":"2024-09-15T21:48:38.387631+0000","last_undegraded":"2024-09-15T21:48:38.387631+0000","last_fullsized":"2024-09-15T21:48:38.387631+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:21.797034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"55'8","reported_seq":158,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859109+0000","last_change":"2024-09-15T21:48:21.687672+0000","last_active":"2024-09-15T21:48:36.859109+0000","last_peered":"2024-09-15T21:48:36.859109+0000","last_clean":"2024-09-15T21:48:36.859109+0000","last_became_active":"2024-09-15T21:48:21.643251+0000","last_became_peered":"2024-09-15T21:48:21.643251+0000","last_unstale":"2024-09-15T21:48:36.859109+0000","last_undegraded":"2024-09-15T21:48:36.859109+0000","last_fullsized":"2024-09-15T21:48:36.859109+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:40.697954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":44,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726888+0000","last_change":"2024-09-15T21:46:23.424266+0000","last_active":"2024-09-15T21:48:36.726888+0000","last_peered":"2024-09-15T21:48:36.726888+0000","last_clean":"2024-09-15T21:48:36.726888+0000","last_became_active":"2024-09-15T21:46:23.424007+0000","last_became_peered":"2024-09-15T21:46:23.424007+0000","last_unstale":"2024-09-15T21:48:36.726888+0000","last_undegraded":"2024-09-15T21:48:36.726888+0000","last_fullsized":"2024-09-15T21:48:36.726888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:23.492253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'2","reported_seq":135,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726262+0000","last_change":"2024-09-15T21:48:17.623800+0000","last_active":"2024-09-15T21:48:36.726262+0000","last_peered":"2024-09-15T21:48:36.726262+0000","last_clean":"2024-09-15T21:48:36.726262+0000","last_became_active":"2024-09-15T21:48:17.623453+0000","last_became_peered":"2024-09-15T21:48:17.623453+0000","last_unstale":"2024-09-15T21:48:36.726262+0000","last_undegraded":"2024-09-15T21:48:36.726262+0000","last_fullsized":"2024-09-15T21:48:36.726262+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:46:38.897647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":121,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725974+0000","last_change":"2024-09-15T21:46:23.430836+0000","last_active":"2024-09-15T21:48:36.725974+0000","last_peered":"2024-09-15T21:48:36.725974+0000","last_clean":"2024-09-15T21:48:36.725974+0000","last_became_active":"2024-09-15T21:46:23.429283+0000","last_became_peered":"2024-09-15T21:46:23.429283+0000","last_unstale":"2024-09-15T21:48:36.725974+0000","last_undegraded":"2024-09-15T21:48:36.725974+0000","last_fullsized":"2024-09-15T21:48:36.725974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:30.571376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'7","reported_seq":158,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726054+0000","last_change":"2024-09-15T21:48:21.660402+0000","last_active":"2024-09-15T21:48:36.726054+0000","last_peered":"2024-09-15T21:48:36.726054+0000","last_clean":"2024-09-15T21:48:36.726054+0000","last_became_active":"2024-09-15T21:48:21.635898+0000","last_became_peered":"2024-09-15T21:48:21.635898+0000","last_unstale":"2024-09-15T21:48:36.726054+0000","last_undegraded":"2024-09-15T21:48:36.726054+0000","last_fullsized":"2024-09-15T21:48:36.726054+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:29.173360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":180,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387326+0000","last_change":"2024-09-15T21:46:23.428634+0000","last_active":"2024-09-15T21:48:38.387326+0000","last_peered":"2024-09-15T21:48:38.387326+0000","last_clean":"2024-09-15T21:48:38.387326+0000","last_became_active":"2024-09-15T21:46:23.428301+0000","last_became_peered":"2024-09-15T21:46:23.428301+0000","last_unstale":"2024-09-15T21:48:38.387326+0000","last_undegraded":"2024-09-15T21:48:38.387326+0000","last_fullsized":"2024-09-15T21:48:38.387326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:59.492942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'4","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726625+0000","last_change":"2024-09-15T21:46:22.425810+0000","last_active":"2024-09-15T21:48:36.726625+0000","last_peered":"2024-09-15T21:48:36.726625+0000","last_clean":"2024-09-15T21:48:36.726625+0000","last_became_active":"2024-09-15T21:46:22.425478+0000","last_became_peered":"2024-09-15T21:46:22.425478+0000","last_unstale":"2024-09-15T21:48:36.726625+0000","last_undegraded":"2024-09-15T21:48:36.726625+0000","last_fullsized":"2024-09-15T21:48:36.726625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:53.296599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387462+0000","last_change":"2024-09-15T21:46:23.426054+0000","last_active":"2024-09-15T21:48:38.387462+0000","last_peered":"2024-09-15T21:48:38.387462+0000","last_clean":"2024-09-15T21:48:38.387462+0000","last_became_active":"2024-09-15T21:46:23.425547+0000","last_became_peered":"2024-09-15T21:46:23.425547+0000","last_unstale":"2024-09-15T21:48:38.387462+0000","last_undegraded":"2024-09-15T21:48:38.387462+0000","last_fullsized":"2024-09-15T21:48:38.387462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:56.838503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"55'5","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710664+0000","last_change":"2024-09-15T21:48:21.653804+0000","last_active":"2024-09-15T21:48:35.710664+0000","last_peered":"2024-09-15T21:48:35.710664+0000","last_clean":"2024-09-15T21:48:35.710664+0000","last_became_active":"2024-09-15T21:48:21.639223+0000","last_became_peered":"2024-09-15T21:48:21.639223+0000","last_unstale":"2024-09-15T21:48:35.710664+0000","last_undegraded":"2024-09-15T21:48:35.710664+0000","last_fullsized":"2024-09-15T21:48:35.710664+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:47:15.330440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858680+0000","last_change":"2024-09-15T21:46:23.434719+0000","last_active":"2024-09-15T21:48:36.858680+0000","last_peered":"2024-09-15T21:48:36.858680+0000","last_clean":"2024-09-15T21:48:36.858680+0000","last_became_active":"2024-09-15T21:46:23.434448+0000","last_became_peered":"2024-09-15T21:46:23.434448+0000","last_unstale":"2024-09-15T21:48:36.858680+0000","last_undegraded":"2024-09-15T21:48:36.858680+0000","last_fullsized":"2024-09-15T21:48:36.858680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:14.901671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":68,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850752+0000","last_change":"2024-09-15T21:48:27.657040+0000","last_active":"2024-09-15T21:48:36.850752+0000","last_peered":"2024-09-15T21:48:36.850752+0000","last_clean":"2024-09-15T21:48:36.850752+0000","last_became_active":"2024-09-15T21:48:27.656753+0000","last_became_peered":"2024-09-15T21:48:27.656753+0000","last_unstale":"2024-09-15T21:48:36.850752+0000","last_undegraded":"2024-09-15T21:48:36.850752+0000","last_fullsized":"2024-09-15T21:48:36.850752+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:14.458723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'5","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710454+0000","last_change":"2024-09-15T21:48:21.691408+0000","last_active":"2024-09-15T21:48:35.710454+0000","last_peered":"2024-09-15T21:48:35.710454+0000","last_clean":"2024-09-15T21:48:35.710454+0000","last_became_active":"2024-09-15T21:48:21.641517+0000","last_became_peered":"2024-09-15T21:48:21.641517+0000","last_unstale":"2024-09-15T21:48:35.710454+0000","last_undegraded":"2024-09-15T21:48:35.710454+0000","last_fullsized":"2024-09-15T21:48:35.710454+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:09.454433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022007+0000","last_change":"2024-09-15T21:48:17.623223+0000","last_active":"2024-09-15T21:48:37.022007+0000","last_peered":"2024-09-15T21:48:37.022007+0000","last_clean":"2024-09-15T21:48:37.022007+0000","last_became_active":"2024-09-15T21:48:17.622019+0000","last_became_peered":"2024-09-15T21:48:17.622019+0000","last_unstale":"2024-09-15T21:48:37.022007+0000","last_undegraded":"2024-09-15T21:48:37.022007+0000","last_fullsized":"2024-09-15T21:48:37.022007+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:39:57.643793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"55'13","reported_seq":53,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710695+0000","last_change":"2024-09-15T21:48:21.682609+0000","last_active":"2024-09-15T21:48:35.710695+0000","last_peered":"2024-09-15T21:48:35.710695+0000","last_clean":"2024-09-15T21:48:35.710695+0000","last_became_active":"2024-09-15T21:48:21.640156+0000","last_became_peered":"2024-09-15T21:48:21.640156+0000","last_unstale":"2024-09-15T21:48:35.710695+0000","last_undegraded":"2024-09-15T21:48:35.710695+0000","last_fullsized":"2024-09-15T21:48:35.710695+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:28.936509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726585+0000","last_change":"2024-09-15T21:48:17.623164+0000","last_active":"2024-09-15T21:48:36.726585+0000","last_peered":"2024-09-15T21:48:36.726585+0000","last_clean":"2024-09-15T21:48:36.726585+0000","last_became_active":"2024-09-15T21:48:17.622843+0000","last_became_peered":"2024-09-15T21:48:17.622843+0000","last_unstale":"2024-09-15T21:48:36.726585+0000","last_undegraded":"2024-09-15T21:48:36.726585+0000","last_fullsized":"2024-09-15T21:48:36.726585+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:06:30.745189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'2","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387645+0000","last_change":"2024-09-15T21:46:22.428529+0000","last_active":"2024-09-15T21:48:38.387645+0000","last_peered":"2024-09-15T21:48:38.387645+0000","last_clean":"2024-09-15T21:48:38.387645+0000","last_became_active":"2024-09-15T21:46:22.428263+0000","last_became_peered":"2024-09-15T21:46:22.428263+0000","last_unstale":"2024-09-15T21:48:38.387645+0000","last_undegraded":"2024-09-15T21:48:38.387645+0000","last_fullsized":"2024-09-15T21:48:38.387645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:22:40.160062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387575+0000","last_change":"2024-09-15T21:46:23.427968+0000","last_active":"2024-09-15T21:48:38.387575+0000","last_peered":"2024-09-15T21:48:38.387575+0000","last_clean":"2024-09-15T21:48:38.387575+0000","last_became_active":"2024-09-15T21:46:23.427546+0000","last_became_peered":"2024-09-15T21:46:23.427546+0000","last_unstale":"2024-09-15T21:48:38.387575+0000","last_undegraded":"2024-09-15T21:48:38.387575+0000","last_fullsized":"2024-09-15T21:48:38.387575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:47.226097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'13","reported_seq":171,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726869+0000","last_change":"2024-09-15T21:48:21.680908+0000","last_active":"2024-09-15T21:48:36.726869+0000","last_peered":"2024-09-15T21:48:36.726869+0000","last_clean":"2024-09-15T21:48:36.726869+0000","last_became_active":"2024-09-15T21:48:21.642406+0000","last_became_peered":"2024-09-15T21:48:21.642406+0000","last_unstale":"2024-09-15T21:48:36.726869+0000","last_undegraded":"2024-09-15T21:48:36.726869+0000","last_fullsized":"2024-09-15T21:48:36.726869+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:23:26.323380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1344,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":42655,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":243,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":4802,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":335872,"data_stored":85310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6368,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476769,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1507328,"data_stored":1002267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1503232,"data_stored":997684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509475,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":946176,"data_stored":447388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":57,"seq":244813135879,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29096,"kb_used_data":912,"kb_used_omap":7,"kb_used_meta":28152,"kb_avail":94342744,"statfs":{"total":96636764160,"available":96606969856,"internally_reserved":0,"allocated":933888,"data_stored":414957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7963,"internal_metadata":28827877},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509474,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":909312,"data_stored":412355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509473,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":921600,"data_stored":412891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":888832,"data_stored":400427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":954368,"data_stored":448082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":37057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6368,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":36363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:48:42.009 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:48:42.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.142+0000 7f18a3903640 1 Processor -- start 2024-09-15T21:48:42.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.142+0000 7f18a3903640 1 -- start start 2024-09-15T21:48:42.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.143+0000 7f18a3903640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f189c155440 0x7f189c0cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.143+0000 7f18a3903640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f189c15e7a0 con 0x7f189c0cb7d0 2024-09-15T21:48:42.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.143+0000 7f18a3903640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f189c15ea40 con 0x7f189c0cc8f0 2024-09-15T21:48:42.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.143+0000 7f18a3903640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f189c15b470 con 0x7f189c155440 2024-09-15T21:48:42.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.143+0000 7f18a0e9e640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f189c0cb7d0 0x7f189c0cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42174/0 (socket says 172.21.15.100:42174) 2024-09-15T21:48:42.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.143+0000 7f18a0e9e640 1 -- 172.21.15.100:0/1951525220 learned_addr learned my addr 172.21.15.100:0/1951525220 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:42.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.143+0000 7f18a169f640 1 --2- 172.21.15.100:0/1951525220 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f189c155440 0x7f189c0cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:42.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.144+0000 7f1893fff640 1 -- 172.21.15.100:0/1951525220 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2679368276 0 0) 0x7f189c15e7a0 con 0x7f189c0cb7d0 2024-09-15T21:48:42.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.144+0000 7f1893fff640 1 -- 172.21.15.100:0/1951525220 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1888003520 con 0x7f189c0cb7d0 2024-09-15T21:48:42.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.144+0000 7f1893fff640 1 -- 172.21.15.100:0/1951525220 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1548170954 0 0) 0x7f189c15ea40 con 0x7f189c0cc8f0 2024-09-15T21:48:42.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.144+0000 7f1893fff640 1 -- 172.21.15.100:0/1951525220 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f189c15e7a0 con 0x7f189c0cc8f0 2024-09-15T21:48:42.148 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.144+0000 7f18a169f640 1 -- 172.21.15.100:0/1951525220 >> v1:172.21.15.122:6789/0 conn(0x7f189c0cc8f0 legacy=0x7f189c165990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:42.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.144+0000 7f18a169f640 1 -- 172.21.15.100:0/1951525220 >> v1:172.21.15.100:6789/0 conn(0x7f189c0cb7d0 legacy=0x7f189c0cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:42.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.144+0000 7f18a169f640 1 -- 172.21.15.100:0/1951525220 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189c15ed20 con 0x7f189c155440 2024-09-15T21:48:42.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.145+0000 7f18a169f640 1 --2- 172.21.15.100:0/1951525220 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f189c155440 0x7f189c0cd420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18940097d0 tx=0x7f189402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd0e38fb6d4900e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:42.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.145+0000 7f1893fff640 1 -- 172.21.15.100:0/1951525220 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894037070 con 0x7f189c155440 2024-09-15T21:48:42.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.145+0000 7f1893fff640 1 -- 172.21.15.100:0/1951525220 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1894032440 con 0x7f189c155440 2024-09-15T21:48:42.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.145+0000 7f1893fff640 1 -- 172.21.15.100:0/1951525220 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18940327f0 con 0x7f189c155440 2024-09-15T21:48:42.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.145+0000 7f18a3903640 1 -- 172.21.15.100:0/1951525220 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f189c155440 msgr2=0x7f189c0cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:42.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.145+0000 7f18a3903640 1 --2- 172.21.15.100:0/1951525220 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f189c155440 0x7f189c0cd420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f18940097d0 tx=0x7f189402a210 comp rx=0 tx=0).stop 2024-09-15T21:48:42.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.145+0000 7f18a3903640 1 -- 172.21.15.100:0/1951525220 shutdown_connections 2024-09-15T21:48:42.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.145+0000 7f18a3903640 1 --2- 172.21.15.100:0/1951525220 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f189c155440 0x7f189c0cd420 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.145+0000 7f18a3903640 1 -- 172.21.15.100:0/1951525220 >> 172.21.15.100:0/1951525220 conn(0x7f189c0680d0 msgr2=0x7f189c06d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:42.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.145+0000 7f18a3903640 1 -- 172.21.15.100:0/1951525220 shutdown_connections 2024-09-15T21:48:42.149 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.146+0000 7f18a3903640 1 -- 172.21.15.100:0/1951525220 wait complete. 2024-09-15T21:48:42.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.147+0000 7f18a3903640 1 Processor -- start 2024-09-15T21:48:42.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.147+0000 7f18a3903640 1 -- start start 2024-09-15T21:48:42.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.147+0000 7f18a3903640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f189c0cb7d0 0x7f189c0c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.147+0000 7f18a3903640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f189c0cc8f0 0x7f189c154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.147+0000 7f18a3903640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f189c155440 0x7f189c1f1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.147+0000 7f18a3903640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f189c15f2e0 con 0x7f189c155440 2024-09-15T21:48:42.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.147+0000 7f18a3903640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f189c155820 con 0x7f189c0cc8f0 2024-09-15T21:48:42.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.147+0000 7f18a3903640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f189c15b070 con 0x7f189c0cb7d0 2024-09-15T21:48:42.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.147+0000 7f18a1ea0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f189c155440 0x7f189c1f1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:42.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.147+0000 7f18a1ea0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f189c155440 0x7f189c1f1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54470/0 (socket says 172.21.15.100:54470) 2024-09-15T21:48:42.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.147+0000 7f18a1ea0640 1 -- 172.21.15.100:0/2402223445 learned_addr learned my addr 172.21.15.100:0/2402223445 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:42.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.148+0000 7f18a0e9e640 1 --2- 172.21.15.100:0/2402223445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f189c0cc8f0 0x7f189c154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:42.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.148+0000 7f18a169f640 1 --2- 172.21.15.100:0/2402223445 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f189c0cb7d0 0x7f189c0c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:42.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.148+0000 7f18a1ea0640 1 -- 172.21.15.100:0/2402223445 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f189c0cb7d0 msgr2=0x7f189c0c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:42.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.148+0000 7f18a1ea0640 1 --2- 172.21.15.100:0/2402223445 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f189c0cb7d0 0x7f189c0c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.148+0000 7f18a1ea0640 1 -- 172.21.15.100:0/2402223445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f189c0cc8f0 msgr2=0x7f189c154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:42.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.148+0000 7f18a1ea0640 1 --2- 172.21.15.100:0/2402223445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f189c0cc8f0 0x7f189c154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.148+0000 7f18a1ea0640 1 -- 172.21.15.100:0/2402223445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189c0ca880 con 0x7f189c155440 2024-09-15T21:48:42.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.148+0000 7f18a1ea0640 1 --2- 172.21.15.100:0/2402223445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f189c155440 0x7f189c1f1270 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f189800c7a0 tx=0x7f189800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:42.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.148+0000 7f1891ffb640 1 -- 172.21.15.100:0/2402223445 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1898003b70 con 0x7f189c155440 2024-09-15T21:48:42.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.148+0000 7f1891ffb640 1 -- 172.21.15.100:0/2402223445 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1898003d10 con 0x7f189c155440 2024-09-15T21:48:42.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.148+0000 7f1891ffb640 1 -- 172.21.15.100:0/2402223445 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189800aa90 con 0x7f189c155440 2024-09-15T21:48:42.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.148+0000 7f18a3903640 1 -- 172.21.15.100:0/2402223445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189c0cab60 con 0x7f189c155440 2024-09-15T21:48:42.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.148+0000 7f18a3903640 1 -- 172.21.15.100:0/2402223445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189c1f4d10 con 0x7f189c155440 2024-09-15T21:48:42.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.149+0000 7f18a3903640 1 -- 172.21.15.100:0/2402223445 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f189c0b3940 con 0x7f189c155440 2024-09-15T21:48:42.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.150+0000 7f1891ffb640 1 -- 172.21.15.100:0/2402223445 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f189800ac30 con 0x7f189c155440 2024-09-15T21:48:42.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.150+0000 7f1891ffb640 1 --2- 172.21.15.100:0/2402223445 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f188006c2b0 0x7f188006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.150+0000 7f18a169f640 1 --2- 172.21.15.100:0/2402223445 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f188006c2b0 0x7f188006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:42.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.150+0000 7f1891ffb640 1 -- 172.21.15.100:0/2402223445 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f189808b360 con 0x7f189c155440 2024-09-15T21:48:42.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.151+0000 7f18a169f640 1 --2- 172.21.15.100:0/2402223445 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f188006c2b0 0x7f188006e770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1894034000 tx=0x7f189402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:42.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.155+0000 7f1891ffb640 1 -- 172.21.15.100:0/2402223445 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f18980593e0 con 0x7f189c155440 2024-09-15T21:48:42.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.318+0000 7f18a3903640 1 -- 172.21.15.100:0/2402223445 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f189c0ca630 con 0x7f188006c2b0 2024-09-15T21:48:42.325 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.324+0000 7f1891ffb640 1 -- 172.21.15.100:0/2402223445 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252278 (secure 0 0 0) 0x7f189c0ca630 con 0x7f188006c2b0 2024-09-15T21:48:42.328 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:48:42.330 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-15T21:48:42.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.327+0000 7f18a3903640 1 -- 172.21.15.100:0/2402223445 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f188006c2b0 msgr2=0x7f188006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:42.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.327+0000 7f18a3903640 1 --2- 172.21.15.100:0/2402223445 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f188006c2b0 0x7f188006e770 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1894034000 tx=0x7f189402e000 comp rx=0 tx=0).stop 2024-09-15T21:48:42.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.327+0000 7f18a3903640 1 -- 172.21.15.100:0/2402223445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f189c155440 msgr2=0x7f189c1f1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:42.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.327+0000 7f18a3903640 1 --2- 172.21.15.100:0/2402223445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f189c155440 0x7f189c1f1270 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f189800c7a0 tx=0x7f189800cc60 comp rx=0 tx=0).stop 2024-09-15T21:48:42.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.327+0000 7f18a3903640 1 -- 172.21.15.100:0/2402223445 shutdown_connections 2024-09-15T21:48:42.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.327+0000 7f18a3903640 1 --2- 172.21.15.100:0/2402223445 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f188006c2b0 0x7f188006e770 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.327+0000 7f18a3903640 1 --2- 172.21.15.100:0/2402223445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f189c0cc8f0 0x7f189c154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.327+0000 7f18a3903640 1 --2- 172.21.15.100:0/2402223445 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f189c155440 0x7f189c1f1270 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.327+0000 7f18a3903640 1 --2- 172.21.15.100:0/2402223445 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f189c0cb7d0 0x7f189c0c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.327+0000 7f18a3903640 1 -- 172.21.15.100:0/2402223445 >> 172.21.15.100:0/2402223445 conn(0x7f189c0680d0 msgr2=0x7f189c06c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:42.332 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.327+0000 7f18a3903640 1 -- 172.21.15.100:0/2402223445 shutdown_connections 2024-09-15T21:48:42.333 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.327+0000 7f18a3903640 1 -- 172.21.15.100:0/2402223445 wait complete. 2024-09-15T21:48:42.339 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-15T21:48:41.833082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633023,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":404,"num_write_kb":2533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1185538,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":335,"ondisk_log_size":335,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238248,"kb_used_data":8364,"kb_used_omap":18,"kb_used_meta":229677,"kb_avail":754736472,"statfs":{"total":773094113280,"available":772850147328,"internally_reserved":0,"allocated":8564736,"data_stored":4536051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19119,"internal_metadata":235189585},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.130575"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.711284+0000","last_change":"2024-09-15T21:48:21.639361+0000","last_active":"2024-09-15T21:48:35.711284+0000","last_peered":"2024-09-15T21:48:35.711284+0000","last_clean":"2024-09-15T21:48:35.711284+0000","last_became_active":"2024-09-15T21:48:21.637402+0000","last_became_peered":"2024-09-15T21:48:21.637402+0000","last_unstale":"2024-09-15T21:48:35.711284+0000","last_undegraded":"2024-09-15T21:48:35.711284+0000","last_fullsized":"2024-09-15T21:48:35.711284+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:40.643861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726138+0000","last_change":"2024-09-15T21:46:23.418716+0000","last_active":"2024-09-15T21:48:36.726138+0000","last_peered":"2024-09-15T21:48:36.726138+0000","last_clean":"2024-09-15T21:48:36.726138+0000","last_became_active":"2024-09-15T21:46:23.418471+0000","last_became_peered":"2024-09-15T21:46:23.418471+0000","last_unstale":"2024-09-15T21:48:36.726138+0000","last_undegraded":"2024-09-15T21:48:36.726138+0000","last_fullsized":"2024-09-15T21:48:36.726138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:49.599801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726536+0000","last_change":"2024-09-15T21:46:23.426404+0000","last_active":"2024-09-15T21:48:36.726536+0000","last_peered":"2024-09-15T21:48:36.726536+0000","last_clean":"2024-09-15T21:48:36.726536+0000","last_became_active":"2024-09-15T21:46:23.426146+0000","last_became_peered":"2024-09-15T21:46:23.426146+0000","last_unstale":"2024-09-15T21:48:36.726536+0000","last_undegraded":"2024-09-15T21:48:36.726536+0000","last_fullsized":"2024-09-15T21:48:36.726536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:23.626109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022058+0000","last_change":"2024-09-15T21:48:17.624363+0000","last_active":"2024-09-15T21:48:37.022058+0000","last_peered":"2024-09-15T21:48:37.022058+0000","last_clean":"2024-09-15T21:48:37.022058+0000","last_became_active":"2024-09-15T21:48:17.623606+0000","last_became_peered":"2024-09-15T21:48:17.623606+0000","last_unstale":"2024-09-15T21:48:37.022058+0000","last_undegraded":"2024-09-15T21:48:37.022058+0000","last_fullsized":"2024-09-15T21:48:37.022058+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:03.294215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710447+0000","last_change":"2024-09-15T21:48:21.640215+0000","last_active":"2024-09-15T21:48:35.710447+0000","last_peered":"2024-09-15T21:48:35.710447+0000","last_clean":"2024-09-15T21:48:35.710447+0000","last_became_active":"2024-09-15T21:48:21.638410+0000","last_became_peered":"2024-09-15T21:48:21.638410+0000","last_unstale":"2024-09-15T21:48:35.710447+0000","last_undegraded":"2024-09-15T21:48:35.710447+0000","last_fullsized":"2024-09-15T21:48:35.710447+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:43.167624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022015+0000","last_change":"2024-09-15T21:48:17.625573+0000","last_active":"2024-09-15T21:48:37.022015+0000","last_peered":"2024-09-15T21:48:37.022015+0000","last_clean":"2024-09-15T21:48:37.022015+0000","last_became_active":"2024-09-15T21:48:17.624439+0000","last_became_peered":"2024-09-15T21:48:17.624439+0000","last_unstale":"2024-09-15T21:48:37.022015+0000","last_undegraded":"2024-09-15T21:48:37.022015+0000","last_fullsized":"2024-09-15T21:48:37.022015+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:04.063675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710549+0000","last_change":"2024-09-15T21:48:21.638612+0000","last_active":"2024-09-15T21:48:35.710549+0000","last_peered":"2024-09-15T21:48:35.710549+0000","last_clean":"2024-09-15T21:48:35.710549+0000","last_became_active":"2024-09-15T21:48:21.638310+0000","last_became_peered":"2024-09-15T21:48:21.638310+0000","last_unstale":"2024-09-15T21:48:35.710549+0000","last_undegraded":"2024-09-15T21:48:35.710549+0000","last_fullsized":"2024-09-15T21:48:35.710549+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:37.297103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":99,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850506+0000","last_change":"2024-09-15T21:48:27.657047+0000","last_active":"2024-09-15T21:48:36.850506+0000","last_peered":"2024-09-15T21:48:36.850506+0000","last_clean":"2024-09-15T21:48:36.850506+0000","last_became_active":"2024-09-15T21:48:27.656800+0000","last_became_peered":"2024-09-15T21:48:27.656800+0000","last_unstale":"2024-09-15T21:48:36.850506+0000","last_undegraded":"2024-09-15T21:48:36.850506+0000","last_fullsized":"2024-09-15T21:48:36.850506+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:17.342504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.711186+0000","last_change":"2024-09-15T21:48:21.642296+0000","last_active":"2024-09-15T21:48:35.711186+0000","last_peered":"2024-09-15T21:48:35.711186+0000","last_clean":"2024-09-15T21:48:35.711186+0000","last_became_active":"2024-09-15T21:48:21.641920+0000","last_became_peered":"2024-09-15T21:48:21.641920+0000","last_unstale":"2024-09-15T21:48:35.711186+0000","last_undegraded":"2024-09-15T21:48:35.711186+0000","last_fullsized":"2024-09-15T21:48:35.711186+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:45.342068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859014+0000","last_change":"2024-09-15T21:48:17.622439+0000","last_active":"2024-09-15T21:48:36.859014+0000","last_peered":"2024-09-15T21:48:36.859014+0000","last_clean":"2024-09-15T21:48:36.859014+0000","last_became_active":"2024-09-15T21:48:17.621800+0000","last_became_peered":"2024-09-15T21:48:17.621800+0000","last_unstale":"2024-09-15T21:48:36.859014+0000","last_undegraded":"2024-09-15T21:48:36.859014+0000","last_fullsized":"2024-09-15T21:48:36.859014+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:07.312656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":122,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858947+0000","last_change":"2024-09-15T21:47:05.735261+0000","last_active":"2024-09-15T21:48:36.858947+0000","last_peered":"2024-09-15T21:48:36.858947+0000","last_clean":"2024-09-15T21:48:36.858947+0000","last_became_active":"2024-09-15T21:47:05.734921+0000","last_became_peered":"2024-09-15T21:47:05.734921+0000","last_unstale":"2024-09-15T21:48:36.858947+0000","last_undegraded":"2024-09-15T21:48:36.858947+0000","last_fullsized":"2024-09-15T21:48:36.858947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:55.787926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726086+0000","last_change":"2024-09-15T21:46:23.432534+0000","last_active":"2024-09-15T21:48:36.726086+0000","last_peered":"2024-09-15T21:48:36.726086+0000","last_clean":"2024-09-15T21:48:36.726086+0000","last_became_active":"2024-09-15T21:46:23.430170+0000","last_became_peered":"2024-09-15T21:46:23.430170+0000","last_unstale":"2024-09-15T21:48:36.726086+0000","last_undegraded":"2024-09-15T21:48:36.726086+0000","last_fullsized":"2024-09-15T21:48:36.726086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:11.463937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387398+0000","last_change":"2024-09-15T21:46:23.427841+0000","last_active":"2024-09-15T21:48:38.387398+0000","last_peered":"2024-09-15T21:48:38.387398+0000","last_clean":"2024-09-15T21:48:38.387398+0000","last_became_active":"2024-09-15T21:46:23.427621+0000","last_became_peered":"2024-09-15T21:46:23.427621+0000","last_unstale":"2024-09-15T21:48:38.387398+0000","last_undegraded":"2024-09-15T21:48:38.387398+0000","last_fullsized":"2024-09-15T21:48:38.387398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:18.331943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"55'3","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726331+0000","last_change":"2024-09-15T21:46:23.424707+0000","last_active":"2024-09-15T21:48:36.726331+0000","last_peered":"2024-09-15T21:48:36.726331+0000","last_clean":"2024-09-15T21:48:36.726331+0000","last_became_active":"2024-09-15T21:46:23.424447+0000","last_became_peered":"2024-09-15T21:46:23.424447+0000","last_unstale":"2024-09-15T21:48:36.726331+0000","last_undegraded":"2024-09-15T21:48:36.726331+0000","last_fullsized":"2024-09-15T21:48:36.726331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:23.617214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725904+0000","last_change":"2024-09-15T21:46:23.430297+0000","last_active":"2024-09-15T21:48:36.725904+0000","last_peered":"2024-09-15T21:48:36.725904+0000","last_clean":"2024-09-15T21:48:36.725904+0000","last_became_active":"2024-09-15T21:46:23.430070+0000","last_became_peered":"2024-09-15T21:46:23.430070+0000","last_unstale":"2024-09-15T21:48:36.725904+0000","last_undegraded":"2024-09-15T21:48:36.725904+0000","last_fullsized":"2024-09-15T21:48:36.725904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:03:19.615022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021835+0000","last_change":"2024-09-15T21:48:17.625113+0000","last_active":"2024-09-15T21:48:37.021835+0000","last_peered":"2024-09-15T21:48:37.021835+0000","last_clean":"2024-09-15T21:48:37.021835+0000","last_became_active":"2024-09-15T21:48:17.624764+0000","last_became_peered":"2024-09-15T21:48:17.624764+0000","last_unstale":"2024-09-15T21:48:37.021835+0000","last_undegraded":"2024-09-15T21:48:37.021835+0000","last_fullsized":"2024-09-15T21:48:37.021835+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:28:44.142358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022276+0000","last_change":"2024-09-15T21:48:21.642646+0000","last_active":"2024-09-15T21:48:37.022276+0000","last_peered":"2024-09-15T21:48:37.022276+0000","last_clean":"2024-09-15T21:48:37.022276+0000","last_became_active":"2024-09-15T21:48:21.642303+0000","last_became_peered":"2024-09-15T21:48:21.642303+0000","last_unstale":"2024-09-15T21:48:37.022276+0000","last_undegraded":"2024-09-15T21:48:37.022276+0000","last_fullsized":"2024-09-15T21:48:37.022276+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:08:54.511282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":75,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850781+0000","last_change":"2024-09-15T21:48:27.661631+0000","last_active":"2024-09-15T21:48:36.850781+0000","last_peered":"2024-09-15T21:48:36.850781+0000","last_clean":"2024-09-15T21:48:36.850781+0000","last_became_active":"2024-09-15T21:48:27.661441+0000","last_became_peered":"2024-09-15T21:48:27.661441+0000","last_unstale":"2024-09-15T21:48:36.850781+0000","last_undegraded":"2024-09-15T21:48:36.850781+0000","last_fullsized":"2024-09-15T21:48:36.850781+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:23.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710445+0000","last_change":"2024-09-15T21:48:21.639602+0000","last_active":"2024-09-15T21:48:35.710445+0000","last_peered":"2024-09-15T21:48:35.710445+0000","last_clean":"2024-09-15T21:48:35.710445+0000","last_became_active":"2024-09-15T21:48:21.639009+0000","last_became_peered":"2024-09-15T21:48:21.639009+0000","last_unstale":"2024-09-15T21:48:35.710445+0000","last_undegraded":"2024-09-15T21:48:35.710445+0000","last_fullsized":"2024-09-15T21:48:35.710445+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:16.818783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858858+0000","last_change":"2024-09-15T21:46:23.434558+0000","last_active":"2024-09-15T21:48:36.858858+0000","last_peered":"2024-09-15T21:48:36.858858+0000","last_clean":"2024-09-15T21:48:36.858858+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:48:36.858858+0000","last_undegraded":"2024-09-15T21:48:36.858858+0000","last_fullsized":"2024-09-15T21:48:36.858858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:03:39.562977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726908+0000","last_change":"2024-09-15T21:48:21.642201+0000","last_active":"2024-09-15T21:48:36.726908+0000","last_peered":"2024-09-15T21:48:36.726908+0000","last_clean":"2024-09-15T21:48:36.726908+0000","last_became_active":"2024-09-15T21:48:21.641876+0000","last_became_peered":"2024-09-15T21:48:21.641876+0000","last_unstale":"2024-09-15T21:48:36.726908+0000","last_undegraded":"2024-09-15T21:48:36.726908+0000","last_fullsized":"2024-09-15T21:48:36.726908+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:41:59.764563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021785+0000","last_change":"2024-09-15T21:48:21.638398+0000","last_active":"2024-09-15T21:48:37.021785+0000","last_peered":"2024-09-15T21:48:37.021785+0000","last_clean":"2024-09-15T21:48:37.021785+0000","last_became_active":"2024-09-15T21:48:21.638073+0000","last_became_peered":"2024-09-15T21:48:21.638073+0000","last_unstale":"2024-09-15T21:48:37.021785+0000","last_undegraded":"2024-09-15T21:48:37.021785+0000","last_fullsized":"2024-09-15T21:48:37.021785+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:45.308950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387331+0000","last_change":"2024-09-15T21:46:23.431527+0000","last_active":"2024-09-15T21:48:38.387331+0000","last_peered":"2024-09-15T21:48:38.387331+0000","last_clean":"2024-09-15T21:48:38.387331+0000","last_became_active":"2024-09-15T21:46:23.430719+0000","last_became_peered":"2024-09-15T21:46:23.430719+0000","last_unstale":"2024-09-15T21:48:38.387331+0000","last_undegraded":"2024-09-15T21:48:38.387331+0000","last_fullsized":"2024-09-15T21:48:38.387331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:16.326529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021766+0000","last_change":"2024-09-15T21:48:17.622578+0000","last_active":"2024-09-15T21:48:37.021766+0000","last_peered":"2024-09-15T21:48:37.021766+0000","last_clean":"2024-09-15T21:48:37.021766+0000","last_became_active":"2024-09-15T21:48:17.621639+0000","last_became_peered":"2024-09-15T21:48:17.621639+0000","last_unstale":"2024-09-15T21:48:37.021766+0000","last_undegraded":"2024-09-15T21:48:37.021766+0000","last_fullsized":"2024-09-15T21:48:37.021766+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:57.099730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387622+0000","last_change":"2024-09-15T21:46:23.432646+0000","last_active":"2024-09-15T21:48:38.387622+0000","last_peered":"2024-09-15T21:48:38.387622+0000","last_clean":"2024-09-15T21:48:38.387622+0000","last_became_active":"2024-09-15T21:46:23.432317+0000","last_became_peered":"2024-09-15T21:46:23.432317+0000","last_unstale":"2024-09-15T21:48:38.387622+0000","last_undegraded":"2024-09-15T21:48:38.387622+0000","last_fullsized":"2024-09-15T21:48:38.387622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:49:07.706450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859042+0000","last_change":"2024-09-15T21:48:21.638549+0000","last_active":"2024-09-15T21:48:36.859042+0000","last_peered":"2024-09-15T21:48:36.859042+0000","last_clean":"2024-09-15T21:48:36.859042+0000","last_became_active":"2024-09-15T21:48:21.638212+0000","last_became_peered":"2024-09-15T21:48:21.638212+0000","last_unstale":"2024-09-15T21:48:36.859042+0000","last_undegraded":"2024-09-15T21:48:36.859042+0000","last_fullsized":"2024-09-15T21:48:36.859042+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:18.788661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726213+0000","last_change":"2024-09-15T21:46:23.421749+0000","last_active":"2024-09-15T21:48:36.726213+0000","last_peered":"2024-09-15T21:48:36.726213+0000","last_clean":"2024-09-15T21:48:36.726213+0000","last_became_active":"2024-09-15T21:46:23.421501+0000","last_became_peered":"2024-09-15T21:46:23.421501+0000","last_unstale":"2024-09-15T21:48:36.726213+0000","last_undegraded":"2024-09-15T21:48:36.726213+0000","last_fullsized":"2024-09-15T21:48:36.726213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:55.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387611+0000","last_change":"2024-09-15T21:46:23.432591+0000","last_active":"2024-09-15T21:48:38.387611+0000","last_peered":"2024-09-15T21:48:38.387611+0000","last_clean":"2024-09-15T21:48:38.387611+0000","last_became_active":"2024-09-15T21:46:23.432326+0000","last_became_peered":"2024-09-15T21:46:23.432326+0000","last_unstale":"2024-09-15T21:48:38.387611+0000","last_undegraded":"2024-09-15T21:48:38.387611+0000","last_fullsized":"2024-09-15T21:48:38.387611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:48.565579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.727013+0000","last_change":"2024-09-15T21:46:23.423428+0000","last_active":"2024-09-15T21:48:36.727013+0000","last_peered":"2024-09-15T21:48:36.727013+0000","last_clean":"2024-09-15T21:48:36.727013+0000","last_became_active":"2024-09-15T21:46:23.422734+0000","last_became_peered":"2024-09-15T21:46:23.422734+0000","last_unstale":"2024-09-15T21:48:36.727013+0000","last_undegraded":"2024-09-15T21:48:36.727013+0000","last_fullsized":"2024-09-15T21:48:36.727013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:06.592891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021899+0000","last_change":"2024-09-15T21:48:17.622914+0000","last_active":"2024-09-15T21:48:37.021899+0000","last_peered":"2024-09-15T21:48:37.021899+0000","last_clean":"2024-09-15T21:48:37.021899+0000","last_became_active":"2024-09-15T21:48:17.621473+0000","last_became_peered":"2024-09-15T21:48:17.621473+0000","last_unstale":"2024-09-15T21:48:37.021899+0000","last_undegraded":"2024-09-15T21:48:37.021899+0000","last_fullsized":"2024-09-15T21:48:37.021899+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:45.297321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850684+0000","last_change":"2024-09-15T21:48:17.616723+0000","last_active":"2024-09-15T21:48:36.850684+0000","last_peered":"2024-09-15T21:48:36.850684+0000","last_clean":"2024-09-15T21:48:36.850684+0000","last_became_active":"2024-09-15T21:48:17.616499+0000","last_became_peered":"2024-09-15T21:48:17.616499+0000","last_unstale":"2024-09-15T21:48:36.850684+0000","last_undegraded":"2024-09-15T21:48:36.850684+0000","last_fullsized":"2024-09-15T21:48:36.850684+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:29:36.886350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858637+0000","last_change":"2024-09-15T21:46:23.429648+0000","last_active":"2024-09-15T21:48:36.858637+0000","last_peered":"2024-09-15T21:48:36.858637+0000","last_clean":"2024-09-15T21:48:36.858637+0000","last_became_active":"2024-09-15T21:46:23.429376+0000","last_became_peered":"2024-09-15T21:46:23.429376+0000","last_unstale":"2024-09-15T21:48:36.858637+0000","last_undegraded":"2024-09-15T21:48:36.858637+0000","last_fullsized":"2024-09-15T21:48:36.858637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:31.278283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725723+0000","last_change":"2024-09-15T21:46:23.429847+0000","last_active":"2024-09-15T21:48:36.725723+0000","last_peered":"2024-09-15T21:48:36.725723+0000","last_clean":"2024-09-15T21:48:36.725723+0000","last_became_active":"2024-09-15T21:46:23.429442+0000","last_became_peered":"2024-09-15T21:46:23.429442+0000","last_unstale":"2024-09-15T21:48:36.725723+0000","last_undegraded":"2024-09-15T21:48:36.725723+0000","last_fullsized":"2024-09-15T21:48:36.725723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:31.232512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'6","reported_seq":149,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387490+0000","last_change":"2024-09-15T21:48:21.665708+0000","last_active":"2024-09-15T21:48:38.387490+0000","last_peered":"2024-09-15T21:48:38.387490+0000","last_clean":"2024-09-15T21:48:38.387490+0000","last_became_active":"2024-09-15T21:48:21.636972+0000","last_became_peered":"2024-09-15T21:48:21.636972+0000","last_unstale":"2024-09-15T21:48:38.387490+0000","last_undegraded":"2024-09-15T21:48:38.387490+0000","last_fullsized":"2024-09-15T21:48:38.387490+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:24:33.658234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1756,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":117,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.727058+0000","last_change":"2024-09-15T21:46:23.434198+0000","last_active":"2024-09-15T21:48:36.727058+0000","last_peered":"2024-09-15T21:48:36.727058+0000","last_clean":"2024-09-15T21:48:36.727058+0000","last_became_active":"2024-09-15T21:46:23.433944+0000","last_became_peered":"2024-09-15T21:46:23.433944+0000","last_unstale":"2024-09-15T21:48:36.727058+0000","last_undegraded":"2024-09-15T21:48:36.727058+0000","last_fullsized":"2024-09-15T21:48:36.727058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:25.412167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'7","reported_seq":126,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858780+0000","last_change":"2024-09-15T21:46:22.425571+0000","last_active":"2024-09-15T21:48:36.858780+0000","last_peered":"2024-09-15T21:48:36.858780+0000","last_clean":"2024-09-15T21:48:36.858780+0000","last_became_active":"2024-09-15T21:46:22.425315+0000","last_became_peered":"2024-09-15T21:46:22.425315+0000","last_unstale":"2024-09-15T21:48:36.858780+0000","last_undegraded":"2024-09-15T21:48:36.858780+0000","last_fullsized":"2024-09-15T21:48:36.858780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:31.983301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'3","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387375+0000","last_change":"2024-09-15T21:46:22.425492+0000","last_active":"2024-09-15T21:48:38.387375+0000","last_peered":"2024-09-15T21:48:38.387375+0000","last_clean":"2024-09-15T21:48:38.387375+0000","last_became_active":"2024-09-15T21:46:22.425226+0000","last_became_peered":"2024-09-15T21:46:22.425226+0000","last_unstale":"2024-09-15T21:48:38.387375+0000","last_undegraded":"2024-09-15T21:48:38.387375+0000","last_fullsized":"2024-09-15T21:48:38.387375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:24.322384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387260+0000","last_change":"2024-09-15T21:46:23.429140+0000","last_active":"2024-09-15T21:48:38.387260+0000","last_peered":"2024-09-15T21:48:38.387260+0000","last_clean":"2024-09-15T21:48:38.387260+0000","last_became_active":"2024-09-15T21:46:23.428902+0000","last_became_peered":"2024-09-15T21:46:23.428902+0000","last_unstale":"2024-09-15T21:48:38.387260+0000","last_undegraded":"2024-09-15T21:48:38.387260+0000","last_fullsized":"2024-09-15T21:48:38.387260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:11.256924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'3","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021677+0000","last_change":"2024-09-15T21:48:17.618127+0000","last_active":"2024-09-15T21:48:37.021677+0000","last_peered":"2024-09-15T21:48:37.021677+0000","last_clean":"2024-09-15T21:48:37.021677+0000","last_became_active":"2024-09-15T21:48:17.616704+0000","last_became_peered":"2024-09-15T21:48:17.616704+0000","last_unstale":"2024-09-15T21:48:37.021677+0000","last_undegraded":"2024-09-15T21:48:37.021677+0000","last_fullsized":"2024-09-15T21:48:37.021677+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:09.775586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725761+0000","last_change":"2024-09-15T21:46:23.427442+0000","last_active":"2024-09-15T21:48:36.725761+0000","last_peered":"2024-09-15T21:48:36.725761+0000","last_clean":"2024-09-15T21:48:36.725761+0000","last_became_active":"2024-09-15T21:46:23.427200+0000","last_became_peered":"2024-09-15T21:46:23.427200+0000","last_unstale":"2024-09-15T21:48:36.725761+0000","last_undegraded":"2024-09-15T21:48:36.725761+0000","last_fullsized":"2024-09-15T21:48:36.725761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:27.248545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"55'6","reported_seq":133,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859137+0000","last_change":"2024-09-15T21:48:17.622236+0000","last_active":"2024-09-15T21:48:36.859137+0000","last_peered":"2024-09-15T21:48:36.859137+0000","last_clean":"2024-09-15T21:48:36.859137+0000","last_became_active":"2024-09-15T21:48:17.621860+0000","last_became_peered":"2024-09-15T21:48:17.621860+0000","last_unstale":"2024-09-15T21:48:36.859137+0000","last_undegraded":"2024-09-15T21:48:36.859137+0000","last_fullsized":"2024-09-15T21:48:36.859137+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:00.032847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022154+0000","last_change":"2024-09-15T21:48:21.638639+0000","last_active":"2024-09-15T21:48:37.022154+0000","last_peered":"2024-09-15T21:48:37.022154+0000","last_clean":"2024-09-15T21:48:37.022154+0000","last_became_active":"2024-09-15T21:48:21.638300+0000","last_became_peered":"2024-09-15T21:48:21.638300+0000","last_unstale":"2024-09-15T21:48:37.022154+0000","last_undegraded":"2024-09-15T21:48:37.022154+0000","last_fullsized":"2024-09-15T21:48:37.022154+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:09:01.739184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'4","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022156+0000","last_change":"2024-09-15T21:48:17.623971+0000","last_active":"2024-09-15T21:48:37.022156+0000","last_peered":"2024-09-15T21:48:37.022156+0000","last_clean":"2024-09-15T21:48:37.022156+0000","last_became_active":"2024-09-15T21:48:17.623246+0000","last_became_peered":"2024-09-15T21:48:17.623246+0000","last_unstale":"2024-09-15T21:48:37.022156+0000","last_undegraded":"2024-09-15T21:48:37.022156+0000","last_fullsized":"2024-09-15T21:48:37.022156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:51:37.738581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726718+0000","last_change":"2024-09-15T21:48:17.625879+0000","last_active":"2024-09-15T21:48:36.726718+0000","last_peered":"2024-09-15T21:48:36.726718+0000","last_clean":"2024-09-15T21:48:36.726718+0000","last_became_active":"2024-09-15T21:48:17.625642+0000","last_became_peered":"2024-09-15T21:48:17.625642+0000","last_unstale":"2024-09-15T21:48:36.726718+0000","last_undegraded":"2024-09-15T21:48:36.726718+0000","last_fullsized":"2024-09-15T21:48:36.726718+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:41.396163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"55'7","reported_seq":139,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726301+0000","last_change":"2024-09-15T21:48:17.627957+0000","last_active":"2024-09-15T21:48:36.726301+0000","last_peered":"2024-09-15T21:48:36.726301+0000","last_clean":"2024-09-15T21:48:36.726301+0000","last_became_active":"2024-09-15T21:48:17.622409+0000","last_became_peered":"2024-09-15T21:48:17.622409+0000","last_unstale":"2024-09-15T21:48:36.726301+0000","last_undegraded":"2024-09-15T21:48:36.726301+0000","last_fullsized":"2024-09-15T21:48:36.726301+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:39.534804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":130,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726177+0000","last_change":"2024-09-15T21:48:21.637218+0000","last_active":"2024-09-15T21:48:36.726177+0000","last_peered":"2024-09-15T21:48:36.726177+0000","last_clean":"2024-09-15T21:48:36.726177+0000","last_became_active":"2024-09-15T21:48:21.636913+0000","last_became_peered":"2024-09-15T21:48:21.636913+0000","last_unstale":"2024-09-15T21:48:36.726177+0000","last_undegraded":"2024-09-15T21:48:36.726177+0000","last_fullsized":"2024-09-15T21:48:36.726177+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:36:05.649762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"55'24","reported_seq":143,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858914+0000","last_change":"2024-09-15T21:46:22.426151+0000","last_active":"2024-09-15T21:48:36.858914+0000","last_peered":"2024-09-15T21:48:36.858914+0000","last_clean":"2024-09-15T21:48:36.858914+0000","last_became_active":"2024-09-15T21:46:22.425866+0000","last_became_peered":"2024-09-15T21:46:22.425866+0000","last_unstale":"2024-09-15T21:48:36.858914+0000","last_undegraded":"2024-09-15T21:48:36.858914+0000","last_fullsized":"2024-09-15T21:48:36.858914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:10.684265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35573,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710799+0000","last_change":"2024-09-15T21:48:21.637466+0000","last_active":"2024-09-15T21:48:35.710799+0000","last_peered":"2024-09-15T21:48:35.710799+0000","last_clean":"2024-09-15T21:48:35.710799+0000","last_became_active":"2024-09-15T21:48:21.637151+0000","last_became_peered":"2024-09-15T21:48:21.637151+0000","last_unstale":"2024-09-15T21:48:35.710799+0000","last_undegraded":"2024-09-15T21:48:35.710799+0000","last_fullsized":"2024-09-15T21:48:35.710799+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:22:57.287004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'9","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.727151+0000","last_change":"2024-09-15T21:46:22.427415+0000","last_active":"2024-09-15T21:48:36.727151+0000","last_peered":"2024-09-15T21:48:36.727151+0000","last_clean":"2024-09-15T21:48:36.727151+0000","last_became_active":"2024-09-15T21:46:22.427087+0000","last_became_peered":"2024-09-15T21:46:22.427087+0000","last_unstale":"2024-09-15T21:48:36.727151+0000","last_undegraded":"2024-09-15T21:48:36.727151+0000","last_fullsized":"2024-09-15T21:48:36.727151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:09.365202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":127,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726487+0000","last_change":"2024-09-15T21:48:21.635299+0000","last_active":"2024-09-15T21:48:36.726487+0000","last_peered":"2024-09-15T21:48:36.726487+0000","last_clean":"2024-09-15T21:48:36.726487+0000","last_became_active":"2024-09-15T21:48:21.635022+0000","last_became_peered":"2024-09-15T21:48:21.635022+0000","last_unstale":"2024-09-15T21:48:36.726487+0000","last_undegraded":"2024-09-15T21:48:36.726487+0000","last_fullsized":"2024-09-15T21:48:36.726487+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:01.482588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"55'6","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021757+0000","last_change":"2024-09-15T21:48:17.625975+0000","last_active":"2024-09-15T21:48:37.021757+0000","last_peered":"2024-09-15T21:48:37.021757+0000","last_clean":"2024-09-15T21:48:37.021757+0000","last_became_active":"2024-09-15T21:48:17.625232+0000","last_became_peered":"2024-09-15T21:48:17.625232+0000","last_unstale":"2024-09-15T21:48:37.021757+0000","last_undegraded":"2024-09-15T21:48:37.021757+0000","last_fullsized":"2024-09-15T21:48:37.021757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:21.061733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710557+0000","last_change":"2024-09-15T21:48:21.639654+0000","last_active":"2024-09-15T21:48:35.710557+0000","last_peered":"2024-09-15T21:48:35.710557+0000","last_clean":"2024-09-15T21:48:35.710557+0000","last_became_active":"2024-09-15T21:48:21.638780+0000","last_became_peered":"2024-09-15T21:48:21.638780+0000","last_unstale":"2024-09-15T21:48:35.710557+0000","last_undegraded":"2024-09-15T21:48:35.710557+0000","last_fullsized":"2024-09-15T21:48:35.710557+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:16.549734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'5","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858678+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:48:36.858678+0000","last_peered":"2024-09-15T21:48:36.858678+0000","last_clean":"2024-09-15T21:48:36.858678+0000","last_became_active":"2024-09-15T21:46:22.425237+0000","last_became_peered":"2024-09-15T21:46:22.425237+0000","last_unstale":"2024-09-15T21:48:36.858678+0000","last_undegraded":"2024-09-15T21:48:36.858678+0000","last_fullsized":"2024-09-15T21:48:36.858678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:59.460420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":117,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858649+0000","last_change":"2024-09-15T21:46:23.427515+0000","last_active":"2024-09-15T21:48:36.858649+0000","last_peered":"2024-09-15T21:48:36.858649+0000","last_clean":"2024-09-15T21:48:36.858649+0000","last_became_active":"2024-09-15T21:46:23.427116+0000","last_became_peered":"2024-09-15T21:46:23.427116+0000","last_unstale":"2024-09-15T21:48:36.858649+0000","last_undegraded":"2024-09-15T21:48:36.858649+0000","last_fullsized":"2024-09-15T21:48:36.858649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:35.499970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"55'18","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850613+0000","last_change":"2024-09-15T21:46:22.419905+0000","last_active":"2024-09-15T21:48:36.850613+0000","last_peered":"2024-09-15T21:48:36.850613+0000","last_clean":"2024-09-15T21:48:36.850613+0000","last_became_active":"2024-09-15T21:46:22.419620+0000","last_became_peered":"2024-09-15T21:46:22.419620+0000","last_unstale":"2024-09-15T21:48:36.850613+0000","last_undegraded":"2024-09-15T21:48:36.850613+0000","last_fullsized":"2024-09-15T21:48:36.850613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:26:51.215156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710002+0000","last_change":"2024-09-15T21:48:21.639146+0000","last_active":"2024-09-15T21:48:35.710002+0000","last_peered":"2024-09-15T21:48:35.710002+0000","last_clean":"2024-09-15T21:48:35.710002+0000","last_became_active":"2024-09-15T21:48:21.637855+0000","last_became_peered":"2024-09-15T21:48:21.637855+0000","last_unstale":"2024-09-15T21:48:35.710002+0000","last_undegraded":"2024-09-15T21:48:35.710002+0000","last_fullsized":"2024-09-15T21:48:35.710002+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:35:29.305298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'2","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726069+0000","last_change":"2024-09-15T21:46:22.429661+0000","last_active":"2024-09-15T21:48:36.726069+0000","last_peered":"2024-09-15T21:48:36.726069+0000","last_clean":"2024-09-15T21:48:36.726069+0000","last_became_active":"2024-09-15T21:46:22.429250+0000","last_became_peered":"2024-09-15T21:46:22.429250+0000","last_unstale":"2024-09-15T21:48:36.726069+0000","last_undegraded":"2024-09-15T21:48:36.726069+0000","last_fullsized":"2024-09-15T21:48:36.726069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:29.341739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710978+0000","last_change":"2024-09-15T21:48:21.635384+0000","last_active":"2024-09-15T21:48:35.710978+0000","last_peered":"2024-09-15T21:48:35.710978+0000","last_clean":"2024-09-15T21:48:35.710978+0000","last_became_active":"2024-09-15T21:48:21.635051+0000","last_became_peered":"2024-09-15T21:48:21.635051+0000","last_unstale":"2024-09-15T21:48:35.710978+0000","last_undegraded":"2024-09-15T21:48:35.710978+0000","last_fullsized":"2024-09-15T21:48:35.710978+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:59.199568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'5","reported_seq":83,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850941+0000","last_change":"2024-09-15T21:48:27.660076+0000","last_active":"2024-09-15T21:48:36.850941+0000","last_peered":"2024-09-15T21:48:36.850941+0000","last_clean":"2024-09-15T21:48:36.850941+0000","last_became_active":"2024-09-15T21:48:27.659750+0000","last_became_peered":"2024-09-15T21:48:27.659750+0000","last_unstale":"2024-09-15T21:48:36.850941+0000","last_undegraded":"2024-09-15T21:48:36.850941+0000","last_fullsized":"2024-09-15T21:48:36.850941+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:06.505112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.711325+0000","last_change":"2024-09-15T21:48:21.641556+0000","last_active":"2024-09-15T21:48:35.711325+0000","last_peered":"2024-09-15T21:48:35.711325+0000","last_clean":"2024-09-15T21:48:35.711325+0000","last_became_active":"2024-09-15T21:48:21.639328+0000","last_became_peered":"2024-09-15T21:48:21.639328+0000","last_unstale":"2024-09-15T21:48:35.711325+0000","last_undegraded":"2024-09-15T21:48:35.711325+0000","last_fullsized":"2024-09-15T21:48:35.711325+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:36:11.470048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'3","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726437+0000","last_change":"2024-09-15T21:46:22.425072+0000","last_active":"2024-09-15T21:48:36.726437+0000","last_peered":"2024-09-15T21:48:36.726437+0000","last_clean":"2024-09-15T21:48:36.726437+0000","last_became_active":"2024-09-15T21:46:22.424771+0000","last_became_peered":"2024-09-15T21:46:22.424771+0000","last_unstale":"2024-09-15T21:48:36.726437+0000","last_undegraded":"2024-09-15T21:48:36.726437+0000","last_fullsized":"2024-09-15T21:48:36.726437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:26.256785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022156+0000","last_change":"2024-09-15T21:48:17.619698+0000","last_active":"2024-09-15T21:48:37.022156+0000","last_peered":"2024-09-15T21:48:37.022156+0000","last_clean":"2024-09-15T21:48:37.022156+0000","last_became_active":"2024-09-15T21:48:17.617701+0000","last_became_peered":"2024-09-15T21:48:17.617701+0000","last_unstale":"2024-09-15T21:48:37.022156+0000","last_undegraded":"2024-09-15T21:48:37.022156+0000","last_fullsized":"2024-09-15T21:48:37.022156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:00.170687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'5","reported_seq":132,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850794+0000","last_change":"2024-09-15T21:48:17.619089+0000","last_active":"2024-09-15T21:48:36.850794+0000","last_peered":"2024-09-15T21:48:36.850794+0000","last_clean":"2024-09-15T21:48:36.850794+0000","last_became_active":"2024-09-15T21:48:17.618782+0000","last_became_peered":"2024-09-15T21:48:17.618782+0000","last_unstale":"2024-09-15T21:48:36.850794+0000","last_undegraded":"2024-09-15T21:48:36.850794+0000","last_fullsized":"2024-09-15T21:48:36.850794+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:17.881528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710339+0000","last_change":"2024-09-15T21:48:21.639356+0000","last_active":"2024-09-15T21:48:35.710339+0000","last_peered":"2024-09-15T21:48:35.710339+0000","last_clean":"2024-09-15T21:48:35.710339+0000","last_became_active":"2024-09-15T21:48:21.637883+0000","last_became_peered":"2024-09-15T21:48:21.637883+0000","last_unstale":"2024-09-15T21:48:35.710339+0000","last_undegraded":"2024-09-15T21:48:35.710339+0000","last_fullsized":"2024-09-15T21:48:35.710339+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:27.061996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":236,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710002+0000","last_change":"2024-09-15T21:48:24.647129+0000","last_active":"2024-09-15T21:48:35.710002+0000","last_peered":"2024-09-15T21:48:35.710002+0000","last_clean":"2024-09-15T21:48:35.710002+0000","last_became_active":"2024-09-15T21:48:24.646783+0000","last_became_peered":"2024-09-15T21:48:24.646783+0000","last_unstale":"2024-09-15T21:48:35.710002+0000","last_undegraded":"2024-09-15T21:48:35.710002+0000","last_fullsized":"2024-09-15T21:48:35.710002+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_clean_scrub_stamp":"2024-09-15T21:46:12.223400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:20.467980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'6","reported_seq":68,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021854+0000","last_change":"2024-09-15T21:48:17.622619+0000","last_active":"2024-09-15T21:48:37.021854+0000","last_peered":"2024-09-15T21:48:37.021854+0000","last_clean":"2024-09-15T21:48:37.021854+0000","last_became_active":"2024-09-15T21:48:17.622159+0000","last_became_peered":"2024-09-15T21:48:17.622159+0000","last_unstale":"2024-09-15T21:48:37.021854+0000","last_undegraded":"2024-09-15T21:48:37.021854+0000","last_fullsized":"2024-09-15T21:48:37.021854+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:51.566702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":127,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387385+0000","last_change":"2024-09-15T21:48:17.621009+0000","last_active":"2024-09-15T21:48:38.387385+0000","last_peered":"2024-09-15T21:48:38.387385+0000","last_clean":"2024-09-15T21:48:38.387385+0000","last_became_active":"2024-09-15T21:48:17.620738+0000","last_became_peered":"2024-09-15T21:48:17.620738+0000","last_unstale":"2024-09-15T21:48:38.387385+0000","last_undegraded":"2024-09-15T21:48:38.387385+0000","last_fullsized":"2024-09-15T21:48:38.387385+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:20.095876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'4","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710013+0000","last_change":"2024-09-15T21:48:21.645645+0000","last_active":"2024-09-15T21:48:35.710013+0000","last_peered":"2024-09-15T21:48:35.710013+0000","last_clean":"2024-09-15T21:48:35.710013+0000","last_became_active":"2024-09-15T21:48:21.636886+0000","last_became_peered":"2024-09-15T21:48:21.636886+0000","last_unstale":"2024-09-15T21:48:35.710013+0000","last_undegraded":"2024-09-15T21:48:35.710013+0000","last_fullsized":"2024-09-15T21:48:35.710013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:12.676508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726029+0000","last_change":"2024-09-15T21:46:23.430396+0000","last_active":"2024-09-15T21:48:36.726029+0000","last_peered":"2024-09-15T21:48:36.726029+0000","last_clean":"2024-09-15T21:48:36.726029+0000","last_became_active":"2024-09-15T21:46:23.430138+0000","last_became_peered":"2024-09-15T21:46:23.430138+0000","last_unstale":"2024-09-15T21:48:36.726029+0000","last_undegraded":"2024-09-15T21:48:36.726029+0000","last_fullsized":"2024-09-15T21:48:36.726029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:50.622949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"55'7","reported_seq":80,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850706+0000","last_change":"2024-09-15T21:48:27.660216+0000","last_active":"2024-09-15T21:48:36.850706+0000","last_peered":"2024-09-15T21:48:36.850706+0000","last_clean":"2024-09-15T21:48:36.850706+0000","last_became_active":"2024-09-15T21:48:27.659880+0000","last_became_peered":"2024-09-15T21:48:27.659880+0000","last_unstale":"2024-09-15T21:48:36.850706+0000","last_undegraded":"2024-09-15T21:48:36.850706+0000","last_fullsized":"2024-09-15T21:48:36.850706+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:48.186260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":516,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725858+0000","last_change":"2024-09-15T21:46:23.427051+0000","last_active":"2024-09-15T21:48:36.725858+0000","last_peered":"2024-09-15T21:48:36.725858+0000","last_clean":"2024-09-15T21:48:36.725858+0000","last_became_active":"2024-09-15T21:46:23.426185+0000","last_became_peered":"2024-09-15T21:46:23.426185+0000","last_unstale":"2024-09-15T21:48:36.725858+0000","last_undegraded":"2024-09-15T21:48:36.725858+0000","last_fullsized":"2024-09-15T21:48:36.725858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:48.318405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'5","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021901+0000","last_change":"2024-09-15T21:48:17.625702+0000","last_active":"2024-09-15T21:48:37.021901+0000","last_peered":"2024-09-15T21:48:37.021901+0000","last_clean":"2024-09-15T21:48:37.021901+0000","last_became_active":"2024-09-15T21:48:17.623936+0000","last_became_peered":"2024-09-15T21:48:17.623936+0000","last_unstale":"2024-09-15T21:48:37.021901+0000","last_undegraded":"2024-09-15T21:48:37.021901+0000","last_fullsized":"2024-09-15T21:48:37.021901+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:53.114831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":121,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725863+0000","last_change":"2024-09-15T21:46:23.425699+0000","last_active":"2024-09-15T21:48:36.725863+0000","last_peered":"2024-09-15T21:48:36.725863+0000","last_clean":"2024-09-15T21:48:36.725863+0000","last_became_active":"2024-09-15T21:46:23.425416+0000","last_became_peered":"2024-09-15T21:46:23.425416+0000","last_unstale":"2024-09-15T21:48:36.725863+0000","last_undegraded":"2024-09-15T21:48:36.725863+0000","last_fullsized":"2024-09-15T21:48:36.725863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:00.325179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"22'1","reported_seq":122,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726645+0000","last_change":"2024-09-15T21:46:22.426492+0000","last_active":"2024-09-15T21:48:36.726645+0000","last_peered":"2024-09-15T21:48:36.726645+0000","last_clean":"2024-09-15T21:48:36.726645+0000","last_became_active":"2024-09-15T21:46:22.426210+0000","last_became_peered":"2024-09-15T21:46:22.426210+0000","last_unstale":"2024-09-15T21:48:36.726645+0000","last_undegraded":"2024-09-15T21:48:36.726645+0000","last_fullsized":"2024-09-15T21:48:36.726645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:05.248091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022064+0000","last_change":"2024-09-15T21:48:17.625501+0000","last_active":"2024-09-15T21:48:37.022064+0000","last_peered":"2024-09-15T21:48:37.022064+0000","last_clean":"2024-09-15T21:48:37.022064+0000","last_became_active":"2024-09-15T21:48:17.624852+0000","last_became_peered":"2024-09-15T21:48:17.624852+0000","last_unstale":"2024-09-15T21:48:37.022064+0000","last_undegraded":"2024-09-15T21:48:37.022064+0000","last_fullsized":"2024-09-15T21:48:37.022064+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:47:09.731108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"22'4","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710548+0000","last_change":"2024-09-15T21:48:21.668174+0000","last_active":"2024-09-15T21:48:35.710548+0000","last_peered":"2024-09-15T21:48:35.710548+0000","last_clean":"2024-09-15T21:48:35.710548+0000","last_became_active":"2024-09-15T21:48:21.639593+0000","last_became_peered":"2024-09-15T21:48:21.639593+0000","last_unstale":"2024-09-15T21:48:35.710548+0000","last_undegraded":"2024-09-15T21:48:35.710548+0000","last_fullsized":"2024-09-15T21:48:35.710548+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:05.341807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725928+0000","last_change":"2024-09-15T21:46:23.426729+0000","last_active":"2024-09-15T21:48:36.725928+0000","last_peered":"2024-09-15T21:48:36.725928+0000","last_clean":"2024-09-15T21:48:36.725928+0000","last_became_active":"2024-09-15T21:46:23.426495+0000","last_became_peered":"2024-09-15T21:46:23.426495+0000","last_unstale":"2024-09-15T21:48:36.725928+0000","last_undegraded":"2024-09-15T21:48:36.725928+0000","last_fullsized":"2024-09-15T21:48:36.725928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:03.277997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"55'16","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726226+0000","last_change":"2024-09-15T21:46:22.427137+0000","last_active":"2024-09-15T21:48:36.726226+0000","last_peered":"2024-09-15T21:48:36.726226+0000","last_clean":"2024-09-15T21:48:36.726226+0000","last_became_active":"2024-09-15T21:46:22.426855+0000","last_became_peered":"2024-09-15T21:46:22.426855+0000","last_unstale":"2024-09-15T21:48:36.726226+0000","last_undegraded":"2024-09-15T21:48:36.726226+0000","last_fullsized":"2024-09-15T21:48:36.726226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:40:14.277940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":130,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387631+0000","last_change":"2024-09-15T21:48:21.639860+0000","last_active":"2024-09-15T21:48:38.387631+0000","last_peered":"2024-09-15T21:48:38.387631+0000","last_clean":"2024-09-15T21:48:38.387631+0000","last_became_active":"2024-09-15T21:48:21.639541+0000","last_became_peered":"2024-09-15T21:48:21.639541+0000","last_unstale":"2024-09-15T21:48:38.387631+0000","last_undegraded":"2024-09-15T21:48:38.387631+0000","last_fullsized":"2024-09-15T21:48:38.387631+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:21.797034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"55'8","reported_seq":158,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859109+0000","last_change":"2024-09-15T21:48:21.687672+0000","last_active":"2024-09-15T21:48:36.859109+0000","last_peered":"2024-09-15T21:48:36.859109+0000","last_clean":"2024-09-15T21:48:36.859109+0000","last_became_active":"2024-09-15T21:48:21.643251+0000","last_became_peered":"2024-09-15T21:48:21.643251+0000","last_unstale":"2024-09-15T21:48:36.859109+0000","last_undegraded":"2024-09-15T21:48:36.859109+0000","last_fullsized":"2024-09-15T21:48:36.859109+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:40.697954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":44,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726888+0000","last_change":"2024-09-15T21:46:23.424266+0000","last_active":"2024-09-15T21:48:36.726888+0000","last_peered":"2024-09-15T21:48:36.726888+0000","last_clean":"2024-09-15T21:48:36.726888+0000","last_became_active":"2024-09-15T21:46:23.424007+0000","last_became_peered":"2024-09-15T21:46:23.424007+0000","last_unstale":"2024-09-15T21:48:36.726888+0000","last_undegraded":"2024-09-15T21:48:36.726888+0000","last_fullsized":"2024-09-15T21:48:36.726888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:23.492253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'2","reported_seq":135,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726262+0000","last_change":"2024-09-15T21:48:17.623800+0000","last_active":"2024-09-15T21:48:36.726262+0000","last_peered":"2024-09-15T21:48:36.726262+0000","last_clean":"2024-09-15T21:48:36.726262+0000","last_became_active":"2024-09-15T21:48:17.623453+0000","last_became_peered":"2024-09-15T21:48:17.623453+0000","last_unstale":"2024-09-15T21:48:36.726262+0000","last_undegraded":"2024-09-15T21:48:36.726262+0000","last_fullsized":"2024-09-15T21:48:36.726262+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:46:38.897647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":121,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725974+0000","last_change":"2024-09-15T21:46:23.430836+0000","last_active":"2024-09-15T21:48:36.725974+0000","last_peered":"2024-09-15T21:48:36.725974+0000","last_clean":"2024-09-15T21:48:36.725974+0000","last_became_active":"2024-09-15T21:46:23.429283+0000","last_became_peered":"2024-09-15T21:46:23.429283+0000","last_unstale":"2024-09-15T21:48:36.725974+0000","last_undegraded":"2024-09-15T21:48:36.725974+0000","last_fullsized":"2024-09-15T21:48:36.725974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:30.571376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'7","reported_seq":158,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726054+0000","last_change":"2024-09-15T21:48:21.660402+0000","last_active":"2024-09-15T21:48:36.726054+0000","last_peered":"2024-09-15T21:48:36.726054+0000","last_clean":"2024-09-15T21:48:36.726054+0000","last_became_active":"2024-09-15T21:48:21.635898+0000","last_became_peered":"2024-09-15T21:48:21.635898+0000","last_unstale":"2024-09-15T21:48:36.726054+0000","last_undegraded":"2024-09-15T21:48:36.726054+0000","last_fullsized":"2024-09-15T21:48:36.726054+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:29.173360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":180,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387326+0000","last_change":"2024-09-15T21:46:23.428634+0000","last_active":"2024-09-15T21:48:38.387326+0000","last_peered":"2024-09-15T21:48:38.387326+0000","last_clean":"2024-09-15T21:48:38.387326+0000","last_became_active":"2024-09-15T21:46:23.428301+0000","last_became_peered":"2024-09-15T21:46:23.428301+0000","last_unstale":"2024-09-15T21:48:38.387326+0000","last_undegraded":"2024-09-15T21:48:38.387326+0000","last_fullsized":"2024-09-15T21:48:38.387326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:59.492942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'4","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726625+0000","last_change":"2024-09-15T21:46:22.425810+0000","last_active":"2024-09-15T21:48:36.726625+0000","last_peered":"2024-09-15T21:48:36.726625+0000","last_clean":"2024-09-15T21:48:36.726625+0000","last_became_active":"2024-09-15T21:46:22.425478+0000","last_became_peered":"2024-09-15T21:46:22.425478+0000","last_unstale":"2024-09-15T21:48:36.726625+0000","last_undegraded":"2024-09-15T21:48:36.726625+0000","last_fullsized":"2024-09-15T21:48:36.726625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:53.296599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387462+0000","last_change":"2024-09-15T21:46:23.426054+0000","last_active":"2024-09-15T21:48:38.387462+0000","last_peered":"2024-09-15T21:48:38.387462+0000","last_clean":"2024-09-15T21:48:38.387462+0000","last_became_active":"2024-09-15T21:46:23.425547+0000","last_became_peered":"2024-09-15T21:46:23.425547+0000","last_unstale":"2024-09-15T21:48:38.387462+0000","last_undegraded":"2024-09-15T21:48:38.387462+0000","last_fullsized":"2024-09-15T21:48:38.387462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:56.838503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"55'5","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710664+0000","last_change":"2024-09-15T21:48:21.653804+0000","last_active":"2024-09-15T21:48:35.710664+0000","last_peered":"2024-09-15T21:48:35.710664+0000","last_clean":"2024-09-15T21:48:35.710664+0000","last_became_active":"2024-09-15T21:48:21.639223+0000","last_became_peered":"2024-09-15T21:48:21.639223+0000","last_unstale":"2024-09-15T21:48:35.710664+0000","last_undegraded":"2024-09-15T21:48:35.710664+0000","last_fullsized":"2024-09-15T21:48:35.710664+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:47:15.330440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858680+0000","last_change":"2024-09-15T21:46:23.434719+0000","last_active":"2024-09-15T21:48:36.858680+0000","last_peered":"2024-09-15T21:48:36.858680+0000","last_clean":"2024-09-15T21:48:36.858680+0000","last_became_active":"2024-09-15T21:46:23.434448+0000","last_became_peered":"2024-09-15T21:46:23.434448+0000","last_unstale":"2024-09-15T21:48:36.858680+0000","last_undegraded":"2024-09-15T21:48:36.858680+0000","last_fullsized":"2024-09-15T21:48:36.858680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:14.901671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":68,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850752+0000","last_change":"2024-09-15T21:48:27.657040+0000","last_active":"2024-09-15T21:48:36.850752+0000","last_peered":"2024-09-15T21:48:36.850752+0000","last_clean":"2024-09-15T21:48:36.850752+0000","last_became_active":"2024-09-15T21:48:27.656753+0000","last_became_peered":"2024-09-15T21:48:27.656753+0000","last_unstale":"2024-09-15T21:48:36.850752+0000","last_undegraded":"2024-09-15T21:48:36.850752+0000","last_fullsized":"2024-09-15T21:48:36.850752+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:14.458723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'5","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710454+0000","last_change":"2024-09-15T21:48:21.691408+0000","last_active":"2024-09-15T21:48:35.710454+0000","last_peered":"2024-09-15T21:48:35.710454+0000","last_clean":"2024-09-15T21:48:35.710454+0000","last_became_active":"2024-09-15T21:48:21.641517+0000","last_became_peered":"2024-09-15T21:48:21.641517+0000","last_unstale":"2024-09-15T21:48:35.710454+0000","last_undegraded":"2024-09-15T21:48:35.710454+0000","last_fullsized":"2024-09-15T21:48:35.710454+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:09.454433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022007+0000","last_change":"2024-09-15T21:48:17.623223+0000","last_active":"2024-09-15T21:48:37.022007+0000","last_peered":"2024-09-15T21:48:37.022007+0000","last_clean":"2024-09-15T21:48:37.022007+0000","last_became_active":"2024-09-15T21:48:17.622019+0000","last_became_peered":"2024-09-15T21:48:17.622019+0000","last_unstale":"2024-09-15T21:48:37.022007+0000","last_undegraded":"2024-09-15T21:48:37.022007+0000","last_fullsized":"2024-09-15T21:48:37.022007+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:39:57.643793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"55'13","reported_seq":53,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710695+0000","last_change":"2024-09-15T21:48:21.682609+0000","last_active":"2024-09-15T21:48:35.710695+0000","last_peered":"2024-09-15T21:48:35.710695+0000","last_clean":"2024-09-15T21:48:35.710695+0000","last_became_active":"2024-09-15T21:48:21.640156+0000","last_became_peered":"2024-09-15T21:48:21.640156+0000","last_unstale":"2024-09-15T21:48:35.710695+0000","last_undegraded":"2024-09-15T21:48:35.710695+0000","last_fullsized":"2024-09-15T21:48:35.710695+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:28.936509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726585+0000","last_change":"2024-09-15T21:48:17.623164+0000","last_active":"2024-09-15T21:48:36.726585+0000","last_peered":"2024-09-15T21:48:36.726585+0000","last_clean":"2024-09-15T21:48:36.726585+0000","last_became_active":"2024-09-15T21:48:17.622843+0000","last_became_peered":"2024-09-15T21:48:17.622843+0000","last_unstale":"2024-09-15T21:48:36.726585+0000","last_undegraded":"2024-09-15T21:48:36.726585+0000","last_fullsized":"2024-09-15T21:48:36.726585+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:06:30.745189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'2","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387645+0000","last_change":"2024-09-15T21:46:22.428529+0000","last_active":"2024-09-15T21:48:38.387645+0000","last_peered":"2024-09-15T21:48:38.387645+0000","last_clean":"2024-09-15T21:48:38.387645+0000","last_became_active":"2024-09-15T21:46:22.428263+0000","last_became_peered":"2024-09-15T21:46:22.428263+0000","last_unstale":"2024-09-15T21:48:38.387645+0000","last_undegraded":"2024-09-15T21:48:38.387645+0000","last_fullsized":"2024-09-15T21:48:38.387645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:22:40.160062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387575+0000","last_change":"2024-09-15T21:46:23.427968+0000","last_active":"2024-09-15T21:48:38.387575+0000","last_peered":"2024-09-15T21:48:38.387575+0000","last_clean":"2024-09-15T21:48:38.387575+0000","last_became_active":"2024-09-15T21:46:23.427546+0000","last_became_peered":"2024-09-15T21:46:23.427546+0000","last_unstale":"2024-09-15T21:48:38.387575+0000","last_undegraded":"2024-09-15T21:48:38.387575+0000","last_fullsized":"2024-09-15T21:48:38.387575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:47.226097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'13","reported_seq":171,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726869+0000","last_change":"2024-09-15T21:48:21.680908+0000","last_active":"2024-09-15T21:48:36.726869+0000","last_peered":"2024-09-15T21:48:36.726869+0000","last_clean":"2024-09-15T21:48:36.726869+0000","last_became_active":"2024-09-15T21:48:21.642406+0000","last_became_peered":"2024-09-15T21:48:21.642406+0000","last_unstale":"2024-09-15T21:48:36.726869+0000","last_undegraded":"2024-09-15T21:48:36.726869+0000","last_fullsized":"2024-09-15T21:48:36.726869+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:23:26.323380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1344,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":42655,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":243,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":4802,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":335872,"data_stored":85310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6368,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476769,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1507328,"data_stored":1002267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1503232,"data_stored":997684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509475,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":946176,"data_stored":447388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":57,"seq":244813135879,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29096,"kb_used_data":912,"kb_used_omap":7,"kb_used_meta":28152,"kb_avail":94342744,"statfs":{"total":96636764160,"available":96606969856,"internally_reserved":0,"allocated":933888,"data_stored":414957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7963,"internal_metadata":28827877},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509474,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":909312,"data_stored":412355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509473,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":921600,"data_stored":412891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":888832,"data_stored":400427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":954368,"data_stored":448082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":37057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6368,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":36363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:48:42.344 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:48:42.344 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:48:42.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.475+0000 7f02df4d9640 1 Processor -- start 2024-09-15T21:48:42.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.476+0000 7f02df4d9640 1 -- start start 2024-09-15T21:48:42.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.476+0000 7f02df4d9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f02d8155440 0x7f02d80cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.479 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.476+0000 7f02df4d9640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f02d80cc8f0 0x7f02d80cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.476+0000 7f02df4d9640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02d815e7a0 con 0x7f02d80cb7d0 2024-09-15T21:48:42.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.476+0000 7f02df4d9640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f02d815b470 con 0x7f02d8155440 2024-09-15T21:48:42.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.476+0000 7f02df4d9640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f02d8155820 con 0x7f02d80cc8f0 2024-09-15T21:48:42.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.476+0000 7f02dd275640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f02d8155440 0x7f02d80cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:42.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.477+0000 7f02dd275640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f02d8155440 0x7f02d80cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54476/0 (socket says 172.21.15.100:54476) 2024-09-15T21:48:42.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.477+0000 7f02dd275640 1 -- 172.21.15.100:0/1287427217 learned_addr learned my addr 172.21.15.100:0/1287427217 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:42.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.477+0000 7f02dda76640 1 --2- 172.21.15.100:0/1287427217 >> v2:172.21.15.122:3300/0 conn(0x7f02d80cc8f0 0x7f02d80cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:42.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.477+0000 7f02dd275640 1 -- 172.21.15.100:0/1287427217 >> v2:172.21.15.122:3300/0 conn(0x7f02d80cc8f0 msgr2=0x7f02d80cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:42.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.477+0000 7f02dd275640 1 --2- 172.21.15.100:0/1287427217 >> v2:172.21.15.122:3300/0 conn(0x7f02d80cc8f0 0x7f02d80cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.477+0000 7f02dd275640 1 -- 172.21.15.100:0/1287427217 >> v1:172.21.15.122:6789/0 conn(0x7f02d80cb7d0 legacy=0x7f02d80cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:42.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.477+0000 7f02dd275640 1 -- 172.21.15.100:0/1287427217 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d815ea40 con 0x7f02d8155440 2024-09-15T21:48:42.480 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.478+0000 7f02dd275640 1 --2- 172.21.15.100:0/1287427217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02d8155440 0x7f02d80cd420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f02d80ce140 tx=0x7f02c802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42dba9eabb736876 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:42.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.478+0000 7f02d7fff640 1 -- 172.21.15.100:0/1287427217 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8036070 con 0x7f02d8155440 2024-09-15T21:48:42.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.478+0000 7f02d7fff640 1 -- 172.21.15.100:0/1287427217 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02c8031600 con 0x7f02d8155440 2024-09-15T21:48:42.481 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.478+0000 7f02d7fff640 1 -- 172.21.15.100:0/1287427217 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c80319d0 con 0x7f02d8155440 2024-09-15T21:48:42.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.478+0000 7f02df4d9640 1 -- 172.21.15.100:0/1287427217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02d8155440 msgr2=0x7f02d80cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:42.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.478+0000 7f02df4d9640 1 --2- 172.21.15.100:0/1287427217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02d8155440 0x7f02d80cd420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f02d80ce140 tx=0x7f02c802a6f0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.478+0000 7f02df4d9640 1 -- 172.21.15.100:0/1287427217 shutdown_connections 2024-09-15T21:48:42.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.478+0000 7f02df4d9640 1 --2- 172.21.15.100:0/1287427217 >> v2:172.21.15.122:3300/0 conn(0x7f02d80cc8f0 0x7f02d80cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.478+0000 7f02df4d9640 1 --2- 172.21.15.100:0/1287427217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02d8155440 0x7f02d80cd420 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.478+0000 7f02df4d9640 1 -- 172.21.15.100:0/1287427217 >> 172.21.15.100:0/1287427217 conn(0x7f02d80680d0 msgr2=0x7f02d806d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:42.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.478+0000 7f02df4d9640 1 -- 172.21.15.100:0/1287427217 shutdown_connections 2024-09-15T21:48:42.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.478+0000 7f02df4d9640 1 -- 172.21.15.100:0/1287427217 wait complete. 2024-09-15T21:48:42.484 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.479+0000 7f02df4d9640 1 Processor -- start 2024-09-15T21:48:42.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.480+0000 7f02df4d9640 1 -- start start 2024-09-15T21:48:42.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.480+0000 7f02df4d9640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f02d80cb7d0 0x7f02d815e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.480+0000 7f02df4d9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02d80cc8f0 0x7f02d815d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.480+0000 7f02df4d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02d8155440 0x7f02d8159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.480+0000 7f02df4d9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f02d815f2e0 con 0x7f02d80cc8f0 2024-09-15T21:48:42.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.480+0000 7f02df4d9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f02d815f000 con 0x7f02d8155440 2024-09-15T21:48:42.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.480+0000 7f02df4d9640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f02d815b070 con 0x7f02d80cb7d0 2024-09-15T21:48:42.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.480+0000 7f02dca74640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02d80cc8f0 0x7f02d815d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:42.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.480+0000 7f02dca74640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02d80cc8f0 0x7f02d815d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54480/0 (socket says 172.21.15.100:54480) 2024-09-15T21:48:42.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.480+0000 7f02dca74640 1 -- 172.21.15.100:0/3379546306 learned_addr learned my addr 172.21.15.100:0/3379546306 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:42.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.480+0000 7f02dda76640 1 --2- 172.21.15.100:0/3379546306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02d8155440 0x7f02d8159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:42.485 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.480+0000 7f02dd275640 1 --2- 172.21.15.100:0/3379546306 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f02d80cb7d0 0x7f02d815e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.481+0000 7f02dca74640 1 -- 172.21.15.100:0/3379546306 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f02d80cb7d0 msgr2=0x7f02d815e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.481+0000 7f02dca74640 1 --2- 172.21.15.100:0/3379546306 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f02d80cb7d0 0x7f02d815e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.481+0000 7f02dca74640 1 -- 172.21.15.100:0/3379546306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02d8155440 msgr2=0x7f02d8159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.481+0000 7f02dca74640 1 --2- 172.21.15.100:0/3379546306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02d8155440 0x7f02d8159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.481+0000 7f02dca74640 1 -- 172.21.15.100:0/3379546306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d81e7570 con 0x7f02d80cc8f0 2024-09-15T21:48:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.481+0000 7f02dca74640 1 --2- 172.21.15.100:0/3379546306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02d80cc8f0 0x7f02d815d850 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00c430 tx=0x7f02cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.481+0000 7f02d5ffb640 1 -- 172.21.15.100:0/3379546306 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc011790 con 0x7f02d80cc8f0 2024-09-15T21:48:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.481+0000 7f02d5ffb640 1 -- 172.21.15.100:0/3379546306 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02cc017070 con 0x7f02d80cc8f0 2024-09-15T21:48:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.481+0000 7f02df4d9640 1 -- 172.21.15.100:0/3379546306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d81e5130 con 0x7f02d80cc8f0 2024-09-15T21:48:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.481+0000 7f02d5ffb640 1 -- 172.21.15.100:0/3379546306 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc011bb0 con 0x7f02d80cc8f0 2024-09-15T21:48:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.481+0000 7f02df4d9640 1 -- 172.21.15.100:0/3379546306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d81e56f0 con 0x7f02d80cc8f0 2024-09-15T21:48:42.486 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.482+0000 7f02df4d9640 1 -- 172.21.15.100:0/3379546306 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d806bbe0 con 0x7f02d80cc8f0 2024-09-15T21:48:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.483+0000 7f02d5ffb640 1 -- 172.21.15.100:0/3379546306 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f02cc01c460 con 0x7f02d80cc8f0 2024-09-15T21:48:42.487 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.483+0000 7f02d5ffb640 1 --2- 172.21.15.100:0/3379546306 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f02c406c2b0 0x7f02c406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.483+0000 7f02d5ffb640 1 -- 172.21.15.100:0/3379546306 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f02cc08b1d0 con 0x7f02d80cc8f0 2024-09-15T21:48:42.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.483+0000 7f02dd275640 1 --2- 172.21.15.100:0/3379546306 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f02c406c2b0 0x7f02c406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:42.488 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.483+0000 7f02dd275640 1 --2- 172.21.15.100:0/3379546306 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f02c406c2b0 0x7f02c406e770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f02c8033000 tx=0x7f02c8009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:42.489 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.487+0000 7f02d5ffb640 1 -- 172.21.15.100:0/3379546306 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f02d806bbe0 con 0x7f02d80cc8f0 2024-09-15T21:48:42.652 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.650+0000 7f02df4d9640 1 -- 172.21.15.100:0/3379546306 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f02d80b3940 con 0x7f02c406c2b0 2024-09-15T21:48:42.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.656+0000 7f02d5ffb640 1 -- 172.21.15.100:0/3379546306 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252278 (secure 0 0 0) 0x7f02d80b3940 con 0x7f02c406c2b0 2024-09-15T21:48:42.660 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:48:42.661 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-15T21:48:42.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.659+0000 7f02df4d9640 1 -- 172.21.15.100:0/3379546306 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f02c406c2b0 msgr2=0x7f02c406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:42.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.659+0000 7f02df4d9640 1 --2- 172.21.15.100:0/3379546306 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f02c406c2b0 0x7f02c406e770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f02c8033000 tx=0x7f02c8009000 comp rx=0 tx=0).stop 2024-09-15T21:48:42.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.659+0000 7f02df4d9640 1 -- 172.21.15.100:0/3379546306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02d80cc8f0 msgr2=0x7f02d815d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:42.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.659+0000 7f02df4d9640 1 --2- 172.21.15.100:0/3379546306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02d80cc8f0 0x7f02d815d850 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00c430 tx=0x7f02cc00c8f0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.659+0000 7f02df4d9640 1 -- 172.21.15.100:0/3379546306 shutdown_connections 2024-09-15T21:48:42.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.659+0000 7f02df4d9640 1 --2- 172.21.15.100:0/3379546306 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f02c406c2b0 0x7f02c406e770 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.659+0000 7f02df4d9640 1 --2- 172.21.15.100:0/3379546306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02d8155440 0x7f02d8159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.659+0000 7f02df4d9640 1 --2- 172.21.15.100:0/3379546306 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f02d80cc8f0 0x7f02d815d850 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.659+0000 7f02df4d9640 1 --2- 172.21.15.100:0/3379546306 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f02d80cb7d0 0x7f02d815e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.659+0000 7f02df4d9640 1 -- 172.21.15.100:0/3379546306 >> 172.21.15.100:0/3379546306 conn(0x7f02d80680d0 msgr2=0x7f02d815b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:42.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.659+0000 7f02df4d9640 1 -- 172.21.15.100:0/3379546306 shutdown_connections 2024-09-15T21:48:42.664 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.659+0000 7f02df4d9640 1 -- 172.21.15.100:0/3379546306 wait complete. 2024-09-15T21:48:42.672 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-15T21:48:41.833082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633023,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":404,"num_write_kb":2533,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1185538,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":335,"ondisk_log_size":335,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238248,"kb_used_data":8364,"kb_used_omap":18,"kb_used_meta":229677,"kb_avail":754736472,"statfs":{"total":773094113280,"available":772850147328,"internally_reserved":0,"allocated":8564736,"data_stored":4536051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19119,"internal_metadata":235189585},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.130575"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.711284+0000","last_change":"2024-09-15T21:48:21.639361+0000","last_active":"2024-09-15T21:48:35.711284+0000","last_peered":"2024-09-15T21:48:35.711284+0000","last_clean":"2024-09-15T21:48:35.711284+0000","last_became_active":"2024-09-15T21:48:21.637402+0000","last_became_peered":"2024-09-15T21:48:21.637402+0000","last_unstale":"2024-09-15T21:48:35.711284+0000","last_undegraded":"2024-09-15T21:48:35.711284+0000","last_fullsized":"2024-09-15T21:48:35.711284+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:40.643861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726138+0000","last_change":"2024-09-15T21:46:23.418716+0000","last_active":"2024-09-15T21:48:36.726138+0000","last_peered":"2024-09-15T21:48:36.726138+0000","last_clean":"2024-09-15T21:48:36.726138+0000","last_became_active":"2024-09-15T21:46:23.418471+0000","last_became_peered":"2024-09-15T21:46:23.418471+0000","last_unstale":"2024-09-15T21:48:36.726138+0000","last_undegraded":"2024-09-15T21:48:36.726138+0000","last_fullsized":"2024-09-15T21:48:36.726138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:49.599801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726536+0000","last_change":"2024-09-15T21:46:23.426404+0000","last_active":"2024-09-15T21:48:36.726536+0000","last_peered":"2024-09-15T21:48:36.726536+0000","last_clean":"2024-09-15T21:48:36.726536+0000","last_became_active":"2024-09-15T21:46:23.426146+0000","last_became_peered":"2024-09-15T21:46:23.426146+0000","last_unstale":"2024-09-15T21:48:36.726536+0000","last_undegraded":"2024-09-15T21:48:36.726536+0000","last_fullsized":"2024-09-15T21:48:36.726536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:23.626109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022058+0000","last_change":"2024-09-15T21:48:17.624363+0000","last_active":"2024-09-15T21:48:37.022058+0000","last_peered":"2024-09-15T21:48:37.022058+0000","last_clean":"2024-09-15T21:48:37.022058+0000","last_became_active":"2024-09-15T21:48:17.623606+0000","last_became_peered":"2024-09-15T21:48:17.623606+0000","last_unstale":"2024-09-15T21:48:37.022058+0000","last_undegraded":"2024-09-15T21:48:37.022058+0000","last_fullsized":"2024-09-15T21:48:37.022058+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:03.294215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710447+0000","last_change":"2024-09-15T21:48:21.640215+0000","last_active":"2024-09-15T21:48:35.710447+0000","last_peered":"2024-09-15T21:48:35.710447+0000","last_clean":"2024-09-15T21:48:35.710447+0000","last_became_active":"2024-09-15T21:48:21.638410+0000","last_became_peered":"2024-09-15T21:48:21.638410+0000","last_unstale":"2024-09-15T21:48:35.710447+0000","last_undegraded":"2024-09-15T21:48:35.710447+0000","last_fullsized":"2024-09-15T21:48:35.710447+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:43.167624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022015+0000","last_change":"2024-09-15T21:48:17.625573+0000","last_active":"2024-09-15T21:48:37.022015+0000","last_peered":"2024-09-15T21:48:37.022015+0000","last_clean":"2024-09-15T21:48:37.022015+0000","last_became_active":"2024-09-15T21:48:17.624439+0000","last_became_peered":"2024-09-15T21:48:17.624439+0000","last_unstale":"2024-09-15T21:48:37.022015+0000","last_undegraded":"2024-09-15T21:48:37.022015+0000","last_fullsized":"2024-09-15T21:48:37.022015+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:04.063675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710549+0000","last_change":"2024-09-15T21:48:21.638612+0000","last_active":"2024-09-15T21:48:35.710549+0000","last_peered":"2024-09-15T21:48:35.710549+0000","last_clean":"2024-09-15T21:48:35.710549+0000","last_became_active":"2024-09-15T21:48:21.638310+0000","last_became_peered":"2024-09-15T21:48:21.638310+0000","last_unstale":"2024-09-15T21:48:35.710549+0000","last_undegraded":"2024-09-15T21:48:35.710549+0000","last_fullsized":"2024-09-15T21:48:35.710549+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:37.297103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":99,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850506+0000","last_change":"2024-09-15T21:48:27.657047+0000","last_active":"2024-09-15T21:48:36.850506+0000","last_peered":"2024-09-15T21:48:36.850506+0000","last_clean":"2024-09-15T21:48:36.850506+0000","last_became_active":"2024-09-15T21:48:27.656800+0000","last_became_peered":"2024-09-15T21:48:27.656800+0000","last_unstale":"2024-09-15T21:48:36.850506+0000","last_undegraded":"2024-09-15T21:48:36.850506+0000","last_fullsized":"2024-09-15T21:48:36.850506+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:17.342504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.711186+0000","last_change":"2024-09-15T21:48:21.642296+0000","last_active":"2024-09-15T21:48:35.711186+0000","last_peered":"2024-09-15T21:48:35.711186+0000","last_clean":"2024-09-15T21:48:35.711186+0000","last_became_active":"2024-09-15T21:48:21.641920+0000","last_became_peered":"2024-09-15T21:48:21.641920+0000","last_unstale":"2024-09-15T21:48:35.711186+0000","last_undegraded":"2024-09-15T21:48:35.711186+0000","last_fullsized":"2024-09-15T21:48:35.711186+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:45.342068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859014+0000","last_change":"2024-09-15T21:48:17.622439+0000","last_active":"2024-09-15T21:48:36.859014+0000","last_peered":"2024-09-15T21:48:36.859014+0000","last_clean":"2024-09-15T21:48:36.859014+0000","last_became_active":"2024-09-15T21:48:17.621800+0000","last_became_peered":"2024-09-15T21:48:17.621800+0000","last_unstale":"2024-09-15T21:48:36.859014+0000","last_undegraded":"2024-09-15T21:48:36.859014+0000","last_fullsized":"2024-09-15T21:48:36.859014+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:07.312656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":122,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858947+0000","last_change":"2024-09-15T21:47:05.735261+0000","last_active":"2024-09-15T21:48:36.858947+0000","last_peered":"2024-09-15T21:48:36.858947+0000","last_clean":"2024-09-15T21:48:36.858947+0000","last_became_active":"2024-09-15T21:47:05.734921+0000","last_became_peered":"2024-09-15T21:47:05.734921+0000","last_unstale":"2024-09-15T21:48:36.858947+0000","last_undegraded":"2024-09-15T21:48:36.858947+0000","last_fullsized":"2024-09-15T21:48:36.858947+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:55.787926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726086+0000","last_change":"2024-09-15T21:46:23.432534+0000","last_active":"2024-09-15T21:48:36.726086+0000","last_peered":"2024-09-15T21:48:36.726086+0000","last_clean":"2024-09-15T21:48:36.726086+0000","last_became_active":"2024-09-15T21:46:23.430170+0000","last_became_peered":"2024-09-15T21:46:23.430170+0000","last_unstale":"2024-09-15T21:48:36.726086+0000","last_undegraded":"2024-09-15T21:48:36.726086+0000","last_fullsized":"2024-09-15T21:48:36.726086+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:11.463937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387398+0000","last_change":"2024-09-15T21:46:23.427841+0000","last_active":"2024-09-15T21:48:38.387398+0000","last_peered":"2024-09-15T21:48:38.387398+0000","last_clean":"2024-09-15T21:48:38.387398+0000","last_became_active":"2024-09-15T21:46:23.427621+0000","last_became_peered":"2024-09-15T21:46:23.427621+0000","last_unstale":"2024-09-15T21:48:38.387398+0000","last_undegraded":"2024-09-15T21:48:38.387398+0000","last_fullsized":"2024-09-15T21:48:38.387398+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:18.331943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"55'3","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726331+0000","last_change":"2024-09-15T21:46:23.424707+0000","last_active":"2024-09-15T21:48:36.726331+0000","last_peered":"2024-09-15T21:48:36.726331+0000","last_clean":"2024-09-15T21:48:36.726331+0000","last_became_active":"2024-09-15T21:46:23.424447+0000","last_became_peered":"2024-09-15T21:46:23.424447+0000","last_unstale":"2024-09-15T21:48:36.726331+0000","last_undegraded":"2024-09-15T21:48:36.726331+0000","last_fullsized":"2024-09-15T21:48:36.726331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:23.617214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725904+0000","last_change":"2024-09-15T21:46:23.430297+0000","last_active":"2024-09-15T21:48:36.725904+0000","last_peered":"2024-09-15T21:48:36.725904+0000","last_clean":"2024-09-15T21:48:36.725904+0000","last_became_active":"2024-09-15T21:46:23.430070+0000","last_became_peered":"2024-09-15T21:46:23.430070+0000","last_unstale":"2024-09-15T21:48:36.725904+0000","last_undegraded":"2024-09-15T21:48:36.725904+0000","last_fullsized":"2024-09-15T21:48:36.725904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:03:19.615022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021835+0000","last_change":"2024-09-15T21:48:17.625113+0000","last_active":"2024-09-15T21:48:37.021835+0000","last_peered":"2024-09-15T21:48:37.021835+0000","last_clean":"2024-09-15T21:48:37.021835+0000","last_became_active":"2024-09-15T21:48:17.624764+0000","last_became_peered":"2024-09-15T21:48:17.624764+0000","last_unstale":"2024-09-15T21:48:37.021835+0000","last_undegraded":"2024-09-15T21:48:37.021835+0000","last_fullsized":"2024-09-15T21:48:37.021835+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:28:44.142358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022276+0000","last_change":"2024-09-15T21:48:21.642646+0000","last_active":"2024-09-15T21:48:37.022276+0000","last_peered":"2024-09-15T21:48:37.022276+0000","last_clean":"2024-09-15T21:48:37.022276+0000","last_became_active":"2024-09-15T21:48:21.642303+0000","last_became_peered":"2024-09-15T21:48:21.642303+0000","last_unstale":"2024-09-15T21:48:37.022276+0000","last_undegraded":"2024-09-15T21:48:37.022276+0000","last_fullsized":"2024-09-15T21:48:37.022276+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:08:54.511282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":75,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850781+0000","last_change":"2024-09-15T21:48:27.661631+0000","last_active":"2024-09-15T21:48:36.850781+0000","last_peered":"2024-09-15T21:48:36.850781+0000","last_clean":"2024-09-15T21:48:36.850781+0000","last_became_active":"2024-09-15T21:48:27.661441+0000","last_became_peered":"2024-09-15T21:48:27.661441+0000","last_unstale":"2024-09-15T21:48:36.850781+0000","last_undegraded":"2024-09-15T21:48:36.850781+0000","last_fullsized":"2024-09-15T21:48:36.850781+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:23.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710445+0000","last_change":"2024-09-15T21:48:21.639602+0000","last_active":"2024-09-15T21:48:35.710445+0000","last_peered":"2024-09-15T21:48:35.710445+0000","last_clean":"2024-09-15T21:48:35.710445+0000","last_became_active":"2024-09-15T21:48:21.639009+0000","last_became_peered":"2024-09-15T21:48:21.639009+0000","last_unstale":"2024-09-15T21:48:35.710445+0000","last_undegraded":"2024-09-15T21:48:35.710445+0000","last_fullsized":"2024-09-15T21:48:35.710445+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:16.818783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858858+0000","last_change":"2024-09-15T21:46:23.434558+0000","last_active":"2024-09-15T21:48:36.858858+0000","last_peered":"2024-09-15T21:48:36.858858+0000","last_clean":"2024-09-15T21:48:36.858858+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:48:36.858858+0000","last_undegraded":"2024-09-15T21:48:36.858858+0000","last_fullsized":"2024-09-15T21:48:36.858858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:03:39.562977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726908+0000","last_change":"2024-09-15T21:48:21.642201+0000","last_active":"2024-09-15T21:48:36.726908+0000","last_peered":"2024-09-15T21:48:36.726908+0000","last_clean":"2024-09-15T21:48:36.726908+0000","last_became_active":"2024-09-15T21:48:21.641876+0000","last_became_peered":"2024-09-15T21:48:21.641876+0000","last_unstale":"2024-09-15T21:48:36.726908+0000","last_undegraded":"2024-09-15T21:48:36.726908+0000","last_fullsized":"2024-09-15T21:48:36.726908+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:41:59.764563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021785+0000","last_change":"2024-09-15T21:48:21.638398+0000","last_active":"2024-09-15T21:48:37.021785+0000","last_peered":"2024-09-15T21:48:37.021785+0000","last_clean":"2024-09-15T21:48:37.021785+0000","last_became_active":"2024-09-15T21:48:21.638073+0000","last_became_peered":"2024-09-15T21:48:21.638073+0000","last_unstale":"2024-09-15T21:48:37.021785+0000","last_undegraded":"2024-09-15T21:48:37.021785+0000","last_fullsized":"2024-09-15T21:48:37.021785+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:45.308950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387331+0000","last_change":"2024-09-15T21:46:23.431527+0000","last_active":"2024-09-15T21:48:38.387331+0000","last_peered":"2024-09-15T21:48:38.387331+0000","last_clean":"2024-09-15T21:48:38.387331+0000","last_became_active":"2024-09-15T21:46:23.430719+0000","last_became_peered":"2024-09-15T21:46:23.430719+0000","last_unstale":"2024-09-15T21:48:38.387331+0000","last_undegraded":"2024-09-15T21:48:38.387331+0000","last_fullsized":"2024-09-15T21:48:38.387331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:16.326529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021766+0000","last_change":"2024-09-15T21:48:17.622578+0000","last_active":"2024-09-15T21:48:37.021766+0000","last_peered":"2024-09-15T21:48:37.021766+0000","last_clean":"2024-09-15T21:48:37.021766+0000","last_became_active":"2024-09-15T21:48:17.621639+0000","last_became_peered":"2024-09-15T21:48:17.621639+0000","last_unstale":"2024-09-15T21:48:37.021766+0000","last_undegraded":"2024-09-15T21:48:37.021766+0000","last_fullsized":"2024-09-15T21:48:37.021766+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:57.099730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387622+0000","last_change":"2024-09-15T21:46:23.432646+0000","last_active":"2024-09-15T21:48:38.387622+0000","last_peered":"2024-09-15T21:48:38.387622+0000","last_clean":"2024-09-15T21:48:38.387622+0000","last_became_active":"2024-09-15T21:46:23.432317+0000","last_became_peered":"2024-09-15T21:46:23.432317+0000","last_unstale":"2024-09-15T21:48:38.387622+0000","last_undegraded":"2024-09-15T21:48:38.387622+0000","last_fullsized":"2024-09-15T21:48:38.387622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:49:07.706450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859042+0000","last_change":"2024-09-15T21:48:21.638549+0000","last_active":"2024-09-15T21:48:36.859042+0000","last_peered":"2024-09-15T21:48:36.859042+0000","last_clean":"2024-09-15T21:48:36.859042+0000","last_became_active":"2024-09-15T21:48:21.638212+0000","last_became_peered":"2024-09-15T21:48:21.638212+0000","last_unstale":"2024-09-15T21:48:36.859042+0000","last_undegraded":"2024-09-15T21:48:36.859042+0000","last_fullsized":"2024-09-15T21:48:36.859042+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:18.788661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726213+0000","last_change":"2024-09-15T21:46:23.421749+0000","last_active":"2024-09-15T21:48:36.726213+0000","last_peered":"2024-09-15T21:48:36.726213+0000","last_clean":"2024-09-15T21:48:36.726213+0000","last_became_active":"2024-09-15T21:46:23.421501+0000","last_became_peered":"2024-09-15T21:46:23.421501+0000","last_unstale":"2024-09-15T21:48:36.726213+0000","last_undegraded":"2024-09-15T21:48:36.726213+0000","last_fullsized":"2024-09-15T21:48:36.726213+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:55.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387611+0000","last_change":"2024-09-15T21:46:23.432591+0000","last_active":"2024-09-15T21:48:38.387611+0000","last_peered":"2024-09-15T21:48:38.387611+0000","last_clean":"2024-09-15T21:48:38.387611+0000","last_became_active":"2024-09-15T21:46:23.432326+0000","last_became_peered":"2024-09-15T21:46:23.432326+0000","last_unstale":"2024-09-15T21:48:38.387611+0000","last_undegraded":"2024-09-15T21:48:38.387611+0000","last_fullsized":"2024-09-15T21:48:38.387611+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:48.565579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.727013+0000","last_change":"2024-09-15T21:46:23.423428+0000","last_active":"2024-09-15T21:48:36.727013+0000","last_peered":"2024-09-15T21:48:36.727013+0000","last_clean":"2024-09-15T21:48:36.727013+0000","last_became_active":"2024-09-15T21:46:23.422734+0000","last_became_peered":"2024-09-15T21:46:23.422734+0000","last_unstale":"2024-09-15T21:48:36.727013+0000","last_undegraded":"2024-09-15T21:48:36.727013+0000","last_fullsized":"2024-09-15T21:48:36.727013+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:06.592891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021899+0000","last_change":"2024-09-15T21:48:17.622914+0000","last_active":"2024-09-15T21:48:37.021899+0000","last_peered":"2024-09-15T21:48:37.021899+0000","last_clean":"2024-09-15T21:48:37.021899+0000","last_became_active":"2024-09-15T21:48:17.621473+0000","last_became_peered":"2024-09-15T21:48:17.621473+0000","last_unstale":"2024-09-15T21:48:37.021899+0000","last_undegraded":"2024-09-15T21:48:37.021899+0000","last_fullsized":"2024-09-15T21:48:37.021899+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:45.297321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850684+0000","last_change":"2024-09-15T21:48:17.616723+0000","last_active":"2024-09-15T21:48:36.850684+0000","last_peered":"2024-09-15T21:48:36.850684+0000","last_clean":"2024-09-15T21:48:36.850684+0000","last_became_active":"2024-09-15T21:48:17.616499+0000","last_became_peered":"2024-09-15T21:48:17.616499+0000","last_unstale":"2024-09-15T21:48:36.850684+0000","last_undegraded":"2024-09-15T21:48:36.850684+0000","last_fullsized":"2024-09-15T21:48:36.850684+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:29:36.886350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858637+0000","last_change":"2024-09-15T21:46:23.429648+0000","last_active":"2024-09-15T21:48:36.858637+0000","last_peered":"2024-09-15T21:48:36.858637+0000","last_clean":"2024-09-15T21:48:36.858637+0000","last_became_active":"2024-09-15T21:46:23.429376+0000","last_became_peered":"2024-09-15T21:46:23.429376+0000","last_unstale":"2024-09-15T21:48:36.858637+0000","last_undegraded":"2024-09-15T21:48:36.858637+0000","last_fullsized":"2024-09-15T21:48:36.858637+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:31.278283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725723+0000","last_change":"2024-09-15T21:46:23.429847+0000","last_active":"2024-09-15T21:48:36.725723+0000","last_peered":"2024-09-15T21:48:36.725723+0000","last_clean":"2024-09-15T21:48:36.725723+0000","last_became_active":"2024-09-15T21:46:23.429442+0000","last_became_peered":"2024-09-15T21:46:23.429442+0000","last_unstale":"2024-09-15T21:48:36.725723+0000","last_undegraded":"2024-09-15T21:48:36.725723+0000","last_fullsized":"2024-09-15T21:48:36.725723+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:31.232512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'6","reported_seq":149,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387490+0000","last_change":"2024-09-15T21:48:21.665708+0000","last_active":"2024-09-15T21:48:38.387490+0000","last_peered":"2024-09-15T21:48:38.387490+0000","last_clean":"2024-09-15T21:48:38.387490+0000","last_became_active":"2024-09-15T21:48:21.636972+0000","last_became_peered":"2024-09-15T21:48:21.636972+0000","last_unstale":"2024-09-15T21:48:38.387490+0000","last_undegraded":"2024-09-15T21:48:38.387490+0000","last_fullsized":"2024-09-15T21:48:38.387490+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:24:33.658234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":878,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1756,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":117,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.727058+0000","last_change":"2024-09-15T21:46:23.434198+0000","last_active":"2024-09-15T21:48:36.727058+0000","last_peered":"2024-09-15T21:48:36.727058+0000","last_clean":"2024-09-15T21:48:36.727058+0000","last_became_active":"2024-09-15T21:46:23.433944+0000","last_became_peered":"2024-09-15T21:46:23.433944+0000","last_unstale":"2024-09-15T21:48:36.727058+0000","last_undegraded":"2024-09-15T21:48:36.727058+0000","last_fullsized":"2024-09-15T21:48:36.727058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:25.412167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'7","reported_seq":126,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858780+0000","last_change":"2024-09-15T21:46:22.425571+0000","last_active":"2024-09-15T21:48:36.858780+0000","last_peered":"2024-09-15T21:48:36.858780+0000","last_clean":"2024-09-15T21:48:36.858780+0000","last_became_active":"2024-09-15T21:46:22.425315+0000","last_became_peered":"2024-09-15T21:46:22.425315+0000","last_unstale":"2024-09-15T21:48:36.858780+0000","last_undegraded":"2024-09-15T21:48:36.858780+0000","last_fullsized":"2024-09-15T21:48:36.858780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:31.983301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'3","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387375+0000","last_change":"2024-09-15T21:46:22.425492+0000","last_active":"2024-09-15T21:48:38.387375+0000","last_peered":"2024-09-15T21:48:38.387375+0000","last_clean":"2024-09-15T21:48:38.387375+0000","last_became_active":"2024-09-15T21:46:22.425226+0000","last_became_peered":"2024-09-15T21:46:22.425226+0000","last_unstale":"2024-09-15T21:48:38.387375+0000","last_undegraded":"2024-09-15T21:48:38.387375+0000","last_fullsized":"2024-09-15T21:48:38.387375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:24.322384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387260+0000","last_change":"2024-09-15T21:46:23.429140+0000","last_active":"2024-09-15T21:48:38.387260+0000","last_peered":"2024-09-15T21:48:38.387260+0000","last_clean":"2024-09-15T21:48:38.387260+0000","last_became_active":"2024-09-15T21:46:23.428902+0000","last_became_peered":"2024-09-15T21:46:23.428902+0000","last_unstale":"2024-09-15T21:48:38.387260+0000","last_undegraded":"2024-09-15T21:48:38.387260+0000","last_fullsized":"2024-09-15T21:48:38.387260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:11.256924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'3","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021677+0000","last_change":"2024-09-15T21:48:17.618127+0000","last_active":"2024-09-15T21:48:37.021677+0000","last_peered":"2024-09-15T21:48:37.021677+0000","last_clean":"2024-09-15T21:48:37.021677+0000","last_became_active":"2024-09-15T21:48:17.616704+0000","last_became_peered":"2024-09-15T21:48:17.616704+0000","last_unstale":"2024-09-15T21:48:37.021677+0000","last_undegraded":"2024-09-15T21:48:37.021677+0000","last_fullsized":"2024-09-15T21:48:37.021677+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:09.775586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725761+0000","last_change":"2024-09-15T21:46:23.427442+0000","last_active":"2024-09-15T21:48:36.725761+0000","last_peered":"2024-09-15T21:48:36.725761+0000","last_clean":"2024-09-15T21:48:36.725761+0000","last_became_active":"2024-09-15T21:46:23.427200+0000","last_became_peered":"2024-09-15T21:46:23.427200+0000","last_unstale":"2024-09-15T21:48:36.725761+0000","last_undegraded":"2024-09-15T21:48:36.725761+0000","last_fullsized":"2024-09-15T21:48:36.725761+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:27.248545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"55'6","reported_seq":133,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859137+0000","last_change":"2024-09-15T21:48:17.622236+0000","last_active":"2024-09-15T21:48:36.859137+0000","last_peered":"2024-09-15T21:48:36.859137+0000","last_clean":"2024-09-15T21:48:36.859137+0000","last_became_active":"2024-09-15T21:48:17.621860+0000","last_became_peered":"2024-09-15T21:48:17.621860+0000","last_unstale":"2024-09-15T21:48:36.859137+0000","last_undegraded":"2024-09-15T21:48:36.859137+0000","last_fullsized":"2024-09-15T21:48:36.859137+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:00.032847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022154+0000","last_change":"2024-09-15T21:48:21.638639+0000","last_active":"2024-09-15T21:48:37.022154+0000","last_peered":"2024-09-15T21:48:37.022154+0000","last_clean":"2024-09-15T21:48:37.022154+0000","last_became_active":"2024-09-15T21:48:21.638300+0000","last_became_peered":"2024-09-15T21:48:21.638300+0000","last_unstale":"2024-09-15T21:48:37.022154+0000","last_undegraded":"2024-09-15T21:48:37.022154+0000","last_fullsized":"2024-09-15T21:48:37.022154+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:09:01.739184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'4","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022156+0000","last_change":"2024-09-15T21:48:17.623971+0000","last_active":"2024-09-15T21:48:37.022156+0000","last_peered":"2024-09-15T21:48:37.022156+0000","last_clean":"2024-09-15T21:48:37.022156+0000","last_became_active":"2024-09-15T21:48:17.623246+0000","last_became_peered":"2024-09-15T21:48:17.623246+0000","last_unstale":"2024-09-15T21:48:37.022156+0000","last_undegraded":"2024-09-15T21:48:37.022156+0000","last_fullsized":"2024-09-15T21:48:37.022156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:51:37.738581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726718+0000","last_change":"2024-09-15T21:48:17.625879+0000","last_active":"2024-09-15T21:48:36.726718+0000","last_peered":"2024-09-15T21:48:36.726718+0000","last_clean":"2024-09-15T21:48:36.726718+0000","last_became_active":"2024-09-15T21:48:17.625642+0000","last_became_peered":"2024-09-15T21:48:17.625642+0000","last_unstale":"2024-09-15T21:48:36.726718+0000","last_undegraded":"2024-09-15T21:48:36.726718+0000","last_fullsized":"2024-09-15T21:48:36.726718+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:41.396163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"55'7","reported_seq":139,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726301+0000","last_change":"2024-09-15T21:48:17.627957+0000","last_active":"2024-09-15T21:48:36.726301+0000","last_peered":"2024-09-15T21:48:36.726301+0000","last_clean":"2024-09-15T21:48:36.726301+0000","last_became_active":"2024-09-15T21:48:17.622409+0000","last_became_peered":"2024-09-15T21:48:17.622409+0000","last_unstale":"2024-09-15T21:48:36.726301+0000","last_undegraded":"2024-09-15T21:48:36.726301+0000","last_fullsized":"2024-09-15T21:48:36.726301+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:39.534804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":130,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726177+0000","last_change":"2024-09-15T21:48:21.637218+0000","last_active":"2024-09-15T21:48:36.726177+0000","last_peered":"2024-09-15T21:48:36.726177+0000","last_clean":"2024-09-15T21:48:36.726177+0000","last_became_active":"2024-09-15T21:48:21.636913+0000","last_became_peered":"2024-09-15T21:48:21.636913+0000","last_unstale":"2024-09-15T21:48:36.726177+0000","last_undegraded":"2024-09-15T21:48:36.726177+0000","last_fullsized":"2024-09-15T21:48:36.726177+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:36:05.649762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"55'24","reported_seq":143,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858914+0000","last_change":"2024-09-15T21:46:22.426151+0000","last_active":"2024-09-15T21:48:36.858914+0000","last_peered":"2024-09-15T21:48:36.858914+0000","last_clean":"2024-09-15T21:48:36.858914+0000","last_became_active":"2024-09-15T21:46:22.425866+0000","last_became_peered":"2024-09-15T21:46:22.425866+0000","last_unstale":"2024-09-15T21:48:36.858914+0000","last_undegraded":"2024-09-15T21:48:36.858914+0000","last_fullsized":"2024-09-15T21:48:36.858914+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:10.684265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35573,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":44,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710799+0000","last_change":"2024-09-15T21:48:21.637466+0000","last_active":"2024-09-15T21:48:35.710799+0000","last_peered":"2024-09-15T21:48:35.710799+0000","last_clean":"2024-09-15T21:48:35.710799+0000","last_became_active":"2024-09-15T21:48:21.637151+0000","last_became_peered":"2024-09-15T21:48:21.637151+0000","last_unstale":"2024-09-15T21:48:35.710799+0000","last_undegraded":"2024-09-15T21:48:35.710799+0000","last_fullsized":"2024-09-15T21:48:35.710799+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:22:57.287004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'9","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.727151+0000","last_change":"2024-09-15T21:46:22.427415+0000","last_active":"2024-09-15T21:48:36.727151+0000","last_peered":"2024-09-15T21:48:36.727151+0000","last_clean":"2024-09-15T21:48:36.727151+0000","last_became_active":"2024-09-15T21:46:22.427087+0000","last_became_peered":"2024-09-15T21:46:22.427087+0000","last_unstale":"2024-09-15T21:48:36.727151+0000","last_undegraded":"2024-09-15T21:48:36.727151+0000","last_fullsized":"2024-09-15T21:48:36.727151+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:09.365202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":127,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726487+0000","last_change":"2024-09-15T21:48:21.635299+0000","last_active":"2024-09-15T21:48:36.726487+0000","last_peered":"2024-09-15T21:48:36.726487+0000","last_clean":"2024-09-15T21:48:36.726487+0000","last_became_active":"2024-09-15T21:48:21.635022+0000","last_became_peered":"2024-09-15T21:48:21.635022+0000","last_unstale":"2024-09-15T21:48:36.726487+0000","last_undegraded":"2024-09-15T21:48:36.726487+0000","last_fullsized":"2024-09-15T21:48:36.726487+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:01.482588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"55'6","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021757+0000","last_change":"2024-09-15T21:48:17.625975+0000","last_active":"2024-09-15T21:48:37.021757+0000","last_peered":"2024-09-15T21:48:37.021757+0000","last_clean":"2024-09-15T21:48:37.021757+0000","last_became_active":"2024-09-15T21:48:17.625232+0000","last_became_peered":"2024-09-15T21:48:17.625232+0000","last_unstale":"2024-09-15T21:48:37.021757+0000","last_undegraded":"2024-09-15T21:48:37.021757+0000","last_fullsized":"2024-09-15T21:48:37.021757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:21.061733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710557+0000","last_change":"2024-09-15T21:48:21.639654+0000","last_active":"2024-09-15T21:48:35.710557+0000","last_peered":"2024-09-15T21:48:35.710557+0000","last_clean":"2024-09-15T21:48:35.710557+0000","last_became_active":"2024-09-15T21:48:21.638780+0000","last_became_peered":"2024-09-15T21:48:21.638780+0000","last_unstale":"2024-09-15T21:48:35.710557+0000","last_undegraded":"2024-09-15T21:48:35.710557+0000","last_fullsized":"2024-09-15T21:48:35.710557+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:16.549734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'5","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858678+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:48:36.858678+0000","last_peered":"2024-09-15T21:48:36.858678+0000","last_clean":"2024-09-15T21:48:36.858678+0000","last_became_active":"2024-09-15T21:46:22.425237+0000","last_became_peered":"2024-09-15T21:46:22.425237+0000","last_unstale":"2024-09-15T21:48:36.858678+0000","last_undegraded":"2024-09-15T21:48:36.858678+0000","last_fullsized":"2024-09-15T21:48:36.858678+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:59.460420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":117,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858649+0000","last_change":"2024-09-15T21:46:23.427515+0000","last_active":"2024-09-15T21:48:36.858649+0000","last_peered":"2024-09-15T21:48:36.858649+0000","last_clean":"2024-09-15T21:48:36.858649+0000","last_became_active":"2024-09-15T21:46:23.427116+0000","last_became_peered":"2024-09-15T21:46:23.427116+0000","last_unstale":"2024-09-15T21:48:36.858649+0000","last_undegraded":"2024-09-15T21:48:36.858649+0000","last_fullsized":"2024-09-15T21:48:36.858649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:35.499970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"55'18","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850613+0000","last_change":"2024-09-15T21:46:22.419905+0000","last_active":"2024-09-15T21:48:36.850613+0000","last_peered":"2024-09-15T21:48:36.850613+0000","last_clean":"2024-09-15T21:48:36.850613+0000","last_became_active":"2024-09-15T21:46:22.419620+0000","last_became_peered":"2024-09-15T21:46:22.419620+0000","last_unstale":"2024-09-15T21:48:36.850613+0000","last_undegraded":"2024-09-15T21:48:36.850613+0000","last_fullsized":"2024-09-15T21:48:36.850613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:26:51.215156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710002+0000","last_change":"2024-09-15T21:48:21.639146+0000","last_active":"2024-09-15T21:48:35.710002+0000","last_peered":"2024-09-15T21:48:35.710002+0000","last_clean":"2024-09-15T21:48:35.710002+0000","last_became_active":"2024-09-15T21:48:21.637855+0000","last_became_peered":"2024-09-15T21:48:21.637855+0000","last_unstale":"2024-09-15T21:48:35.710002+0000","last_undegraded":"2024-09-15T21:48:35.710002+0000","last_fullsized":"2024-09-15T21:48:35.710002+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:35:29.305298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'2","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726069+0000","last_change":"2024-09-15T21:46:22.429661+0000","last_active":"2024-09-15T21:48:36.726069+0000","last_peered":"2024-09-15T21:48:36.726069+0000","last_clean":"2024-09-15T21:48:36.726069+0000","last_became_active":"2024-09-15T21:46:22.429250+0000","last_became_peered":"2024-09-15T21:46:22.429250+0000","last_unstale":"2024-09-15T21:48:36.726069+0000","last_undegraded":"2024-09-15T21:48:36.726069+0000","last_fullsized":"2024-09-15T21:48:36.726069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:29.341739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710978+0000","last_change":"2024-09-15T21:48:21.635384+0000","last_active":"2024-09-15T21:48:35.710978+0000","last_peered":"2024-09-15T21:48:35.710978+0000","last_clean":"2024-09-15T21:48:35.710978+0000","last_became_active":"2024-09-15T21:48:21.635051+0000","last_became_peered":"2024-09-15T21:48:21.635051+0000","last_unstale":"2024-09-15T21:48:35.710978+0000","last_undegraded":"2024-09-15T21:48:35.710978+0000","last_fullsized":"2024-09-15T21:48:35.710978+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:59.199568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'5","reported_seq":83,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850941+0000","last_change":"2024-09-15T21:48:27.660076+0000","last_active":"2024-09-15T21:48:36.850941+0000","last_peered":"2024-09-15T21:48:36.850941+0000","last_clean":"2024-09-15T21:48:36.850941+0000","last_became_active":"2024-09-15T21:48:27.659750+0000","last_became_peered":"2024-09-15T21:48:27.659750+0000","last_unstale":"2024-09-15T21:48:36.850941+0000","last_undegraded":"2024-09-15T21:48:36.850941+0000","last_fullsized":"2024-09-15T21:48:36.850941+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:06.505112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.711325+0000","last_change":"2024-09-15T21:48:21.641556+0000","last_active":"2024-09-15T21:48:35.711325+0000","last_peered":"2024-09-15T21:48:35.711325+0000","last_clean":"2024-09-15T21:48:35.711325+0000","last_became_active":"2024-09-15T21:48:21.639328+0000","last_became_peered":"2024-09-15T21:48:21.639328+0000","last_unstale":"2024-09-15T21:48:35.711325+0000","last_undegraded":"2024-09-15T21:48:35.711325+0000","last_fullsized":"2024-09-15T21:48:35.711325+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:36:11.470048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'3","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726437+0000","last_change":"2024-09-15T21:46:22.425072+0000","last_active":"2024-09-15T21:48:36.726437+0000","last_peered":"2024-09-15T21:48:36.726437+0000","last_clean":"2024-09-15T21:48:36.726437+0000","last_became_active":"2024-09-15T21:46:22.424771+0000","last_became_peered":"2024-09-15T21:46:22.424771+0000","last_unstale":"2024-09-15T21:48:36.726437+0000","last_undegraded":"2024-09-15T21:48:36.726437+0000","last_fullsized":"2024-09-15T21:48:36.726437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:26.256785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022156+0000","last_change":"2024-09-15T21:48:17.619698+0000","last_active":"2024-09-15T21:48:37.022156+0000","last_peered":"2024-09-15T21:48:37.022156+0000","last_clean":"2024-09-15T21:48:37.022156+0000","last_became_active":"2024-09-15T21:48:17.617701+0000","last_became_peered":"2024-09-15T21:48:17.617701+0000","last_unstale":"2024-09-15T21:48:37.022156+0000","last_undegraded":"2024-09-15T21:48:37.022156+0000","last_fullsized":"2024-09-15T21:48:37.022156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:00.170687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'5","reported_seq":132,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850794+0000","last_change":"2024-09-15T21:48:17.619089+0000","last_active":"2024-09-15T21:48:36.850794+0000","last_peered":"2024-09-15T21:48:36.850794+0000","last_clean":"2024-09-15T21:48:36.850794+0000","last_became_active":"2024-09-15T21:48:17.618782+0000","last_became_peered":"2024-09-15T21:48:17.618782+0000","last_unstale":"2024-09-15T21:48:36.850794+0000","last_undegraded":"2024-09-15T21:48:36.850794+0000","last_fullsized":"2024-09-15T21:48:36.850794+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:17.881528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710339+0000","last_change":"2024-09-15T21:48:21.639356+0000","last_active":"2024-09-15T21:48:35.710339+0000","last_peered":"2024-09-15T21:48:35.710339+0000","last_clean":"2024-09-15T21:48:35.710339+0000","last_became_active":"2024-09-15T21:48:21.637883+0000","last_became_peered":"2024-09-15T21:48:21.637883+0000","last_unstale":"2024-09-15T21:48:35.710339+0000","last_undegraded":"2024-09-15T21:48:35.710339+0000","last_fullsized":"2024-09-15T21:48:35.710339+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:27.061996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":236,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710002+0000","last_change":"2024-09-15T21:48:24.647129+0000","last_active":"2024-09-15T21:48:35.710002+0000","last_peered":"2024-09-15T21:48:35.710002+0000","last_clean":"2024-09-15T21:48:35.710002+0000","last_became_active":"2024-09-15T21:48:24.646783+0000","last_became_peered":"2024-09-15T21:48:24.646783+0000","last_unstale":"2024-09-15T21:48:35.710002+0000","last_undegraded":"2024-09-15T21:48:35.710002+0000","last_fullsized":"2024-09-15T21:48:35.710002+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_clean_scrub_stamp":"2024-09-15T21:46:12.223400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:20.467980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'6","reported_seq":68,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021854+0000","last_change":"2024-09-15T21:48:17.622619+0000","last_active":"2024-09-15T21:48:37.021854+0000","last_peered":"2024-09-15T21:48:37.021854+0000","last_clean":"2024-09-15T21:48:37.021854+0000","last_became_active":"2024-09-15T21:48:17.622159+0000","last_became_peered":"2024-09-15T21:48:17.622159+0000","last_unstale":"2024-09-15T21:48:37.021854+0000","last_undegraded":"2024-09-15T21:48:37.021854+0000","last_fullsized":"2024-09-15T21:48:37.021854+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:51.566702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":127,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387385+0000","last_change":"2024-09-15T21:48:17.621009+0000","last_active":"2024-09-15T21:48:38.387385+0000","last_peered":"2024-09-15T21:48:38.387385+0000","last_clean":"2024-09-15T21:48:38.387385+0000","last_became_active":"2024-09-15T21:48:17.620738+0000","last_became_peered":"2024-09-15T21:48:17.620738+0000","last_unstale":"2024-09-15T21:48:38.387385+0000","last_undegraded":"2024-09-15T21:48:38.387385+0000","last_fullsized":"2024-09-15T21:48:38.387385+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:20.095876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'4","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710013+0000","last_change":"2024-09-15T21:48:21.645645+0000","last_active":"2024-09-15T21:48:35.710013+0000","last_peered":"2024-09-15T21:48:35.710013+0000","last_clean":"2024-09-15T21:48:35.710013+0000","last_became_active":"2024-09-15T21:48:21.636886+0000","last_became_peered":"2024-09-15T21:48:21.636886+0000","last_unstale":"2024-09-15T21:48:35.710013+0000","last_undegraded":"2024-09-15T21:48:35.710013+0000","last_fullsized":"2024-09-15T21:48:35.710013+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:12.676508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726029+0000","last_change":"2024-09-15T21:46:23.430396+0000","last_active":"2024-09-15T21:48:36.726029+0000","last_peered":"2024-09-15T21:48:36.726029+0000","last_clean":"2024-09-15T21:48:36.726029+0000","last_became_active":"2024-09-15T21:46:23.430138+0000","last_became_peered":"2024-09-15T21:46:23.430138+0000","last_unstale":"2024-09-15T21:48:36.726029+0000","last_undegraded":"2024-09-15T21:48:36.726029+0000","last_fullsized":"2024-09-15T21:48:36.726029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:50.622949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"55'7","reported_seq":80,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850706+0000","last_change":"2024-09-15T21:48:27.660216+0000","last_active":"2024-09-15T21:48:36.850706+0000","last_peered":"2024-09-15T21:48:36.850706+0000","last_clean":"2024-09-15T21:48:36.850706+0000","last_became_active":"2024-09-15T21:48:27.659880+0000","last_became_peered":"2024-09-15T21:48:27.659880+0000","last_unstale":"2024-09-15T21:48:36.850706+0000","last_undegraded":"2024-09-15T21:48:36.850706+0000","last_fullsized":"2024-09-15T21:48:36.850706+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:48.186260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":516,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725858+0000","last_change":"2024-09-15T21:46:23.427051+0000","last_active":"2024-09-15T21:48:36.725858+0000","last_peered":"2024-09-15T21:48:36.725858+0000","last_clean":"2024-09-15T21:48:36.725858+0000","last_became_active":"2024-09-15T21:46:23.426185+0000","last_became_peered":"2024-09-15T21:46:23.426185+0000","last_unstale":"2024-09-15T21:48:36.725858+0000","last_undegraded":"2024-09-15T21:48:36.725858+0000","last_fullsized":"2024-09-15T21:48:36.725858+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:48.318405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'5","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021901+0000","last_change":"2024-09-15T21:48:17.625702+0000","last_active":"2024-09-15T21:48:37.021901+0000","last_peered":"2024-09-15T21:48:37.021901+0000","last_clean":"2024-09-15T21:48:37.021901+0000","last_became_active":"2024-09-15T21:48:17.623936+0000","last_became_peered":"2024-09-15T21:48:17.623936+0000","last_unstale":"2024-09-15T21:48:37.021901+0000","last_undegraded":"2024-09-15T21:48:37.021901+0000","last_fullsized":"2024-09-15T21:48:37.021901+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:53.114831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":121,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725863+0000","last_change":"2024-09-15T21:46:23.425699+0000","last_active":"2024-09-15T21:48:36.725863+0000","last_peered":"2024-09-15T21:48:36.725863+0000","last_clean":"2024-09-15T21:48:36.725863+0000","last_became_active":"2024-09-15T21:46:23.425416+0000","last_became_peered":"2024-09-15T21:46:23.425416+0000","last_unstale":"2024-09-15T21:48:36.725863+0000","last_undegraded":"2024-09-15T21:48:36.725863+0000","last_fullsized":"2024-09-15T21:48:36.725863+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:00.325179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"22'1","reported_seq":122,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726645+0000","last_change":"2024-09-15T21:46:22.426492+0000","last_active":"2024-09-15T21:48:36.726645+0000","last_peered":"2024-09-15T21:48:36.726645+0000","last_clean":"2024-09-15T21:48:36.726645+0000","last_became_active":"2024-09-15T21:46:22.426210+0000","last_became_peered":"2024-09-15T21:46:22.426210+0000","last_unstale":"2024-09-15T21:48:36.726645+0000","last_undegraded":"2024-09-15T21:48:36.726645+0000","last_fullsized":"2024-09-15T21:48:36.726645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:05.248091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022064+0000","last_change":"2024-09-15T21:48:17.625501+0000","last_active":"2024-09-15T21:48:37.022064+0000","last_peered":"2024-09-15T21:48:37.022064+0000","last_clean":"2024-09-15T21:48:37.022064+0000","last_became_active":"2024-09-15T21:48:17.624852+0000","last_became_peered":"2024-09-15T21:48:17.624852+0000","last_unstale":"2024-09-15T21:48:37.022064+0000","last_undegraded":"2024-09-15T21:48:37.022064+0000","last_fullsized":"2024-09-15T21:48:37.022064+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:47:09.731108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"22'4","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710548+0000","last_change":"2024-09-15T21:48:21.668174+0000","last_active":"2024-09-15T21:48:35.710548+0000","last_peered":"2024-09-15T21:48:35.710548+0000","last_clean":"2024-09-15T21:48:35.710548+0000","last_became_active":"2024-09-15T21:48:21.639593+0000","last_became_peered":"2024-09-15T21:48:21.639593+0000","last_unstale":"2024-09-15T21:48:35.710548+0000","last_undegraded":"2024-09-15T21:48:35.710548+0000","last_fullsized":"2024-09-15T21:48:35.710548+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:05.341807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":119,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725928+0000","last_change":"2024-09-15T21:46:23.426729+0000","last_active":"2024-09-15T21:48:36.725928+0000","last_peered":"2024-09-15T21:48:36.725928+0000","last_clean":"2024-09-15T21:48:36.725928+0000","last_became_active":"2024-09-15T21:46:23.426495+0000","last_became_peered":"2024-09-15T21:46:23.426495+0000","last_unstale":"2024-09-15T21:48:36.725928+0000","last_undegraded":"2024-09-15T21:48:36.725928+0000","last_fullsized":"2024-09-15T21:48:36.725928+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:03.277997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"55'16","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726226+0000","last_change":"2024-09-15T21:46:22.427137+0000","last_active":"2024-09-15T21:48:36.726226+0000","last_peered":"2024-09-15T21:48:36.726226+0000","last_clean":"2024-09-15T21:48:36.726226+0000","last_became_active":"2024-09-15T21:46:22.426855+0000","last_became_peered":"2024-09-15T21:46:22.426855+0000","last_unstale":"2024-09-15T21:48:36.726226+0000","last_undegraded":"2024-09-15T21:48:36.726226+0000","last_fullsized":"2024-09-15T21:48:36.726226+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:40:14.277940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":130,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387631+0000","last_change":"2024-09-15T21:48:21.639860+0000","last_active":"2024-09-15T21:48:38.387631+0000","last_peered":"2024-09-15T21:48:38.387631+0000","last_clean":"2024-09-15T21:48:38.387631+0000","last_became_active":"2024-09-15T21:48:21.639541+0000","last_became_peered":"2024-09-15T21:48:21.639541+0000","last_unstale":"2024-09-15T21:48:38.387631+0000","last_undegraded":"2024-09-15T21:48:38.387631+0000","last_fullsized":"2024-09-15T21:48:38.387631+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:21.797034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"55'8","reported_seq":158,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859109+0000","last_change":"2024-09-15T21:48:21.687672+0000","last_active":"2024-09-15T21:48:36.859109+0000","last_peered":"2024-09-15T21:48:36.859109+0000","last_clean":"2024-09-15T21:48:36.859109+0000","last_became_active":"2024-09-15T21:48:21.643251+0000","last_became_peered":"2024-09-15T21:48:21.643251+0000","last_unstale":"2024-09-15T21:48:36.859109+0000","last_undegraded":"2024-09-15T21:48:36.859109+0000","last_fullsized":"2024-09-15T21:48:36.859109+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:40.697954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":44,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726888+0000","last_change":"2024-09-15T21:46:23.424266+0000","last_active":"2024-09-15T21:48:36.726888+0000","last_peered":"2024-09-15T21:48:36.726888+0000","last_clean":"2024-09-15T21:48:36.726888+0000","last_became_active":"2024-09-15T21:46:23.424007+0000","last_became_peered":"2024-09-15T21:46:23.424007+0000","last_unstale":"2024-09-15T21:48:36.726888+0000","last_undegraded":"2024-09-15T21:48:36.726888+0000","last_fullsized":"2024-09-15T21:48:36.726888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:23.492253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'2","reported_seq":135,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726262+0000","last_change":"2024-09-15T21:48:17.623800+0000","last_active":"2024-09-15T21:48:36.726262+0000","last_peered":"2024-09-15T21:48:36.726262+0000","last_clean":"2024-09-15T21:48:36.726262+0000","last_became_active":"2024-09-15T21:48:17.623453+0000","last_became_peered":"2024-09-15T21:48:17.623453+0000","last_unstale":"2024-09-15T21:48:36.726262+0000","last_undegraded":"2024-09-15T21:48:36.726262+0000","last_fullsized":"2024-09-15T21:48:36.726262+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:46:38.897647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":121,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.725974+0000","last_change":"2024-09-15T21:46:23.430836+0000","last_active":"2024-09-15T21:48:36.725974+0000","last_peered":"2024-09-15T21:48:36.725974+0000","last_clean":"2024-09-15T21:48:36.725974+0000","last_became_active":"2024-09-15T21:46:23.429283+0000","last_became_peered":"2024-09-15T21:46:23.429283+0000","last_unstale":"2024-09-15T21:48:36.725974+0000","last_undegraded":"2024-09-15T21:48:36.725974+0000","last_fullsized":"2024-09-15T21:48:36.725974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:30.571376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'7","reported_seq":158,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726054+0000","last_change":"2024-09-15T21:48:21.660402+0000","last_active":"2024-09-15T21:48:36.726054+0000","last_peered":"2024-09-15T21:48:36.726054+0000","last_clean":"2024-09-15T21:48:36.726054+0000","last_became_active":"2024-09-15T21:48:21.635898+0000","last_became_peered":"2024-09-15T21:48:21.635898+0000","last_unstale":"2024-09-15T21:48:36.726054+0000","last_undegraded":"2024-09-15T21:48:36.726054+0000","last_fullsized":"2024-09-15T21:48:36.726054+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:29.173360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":180,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387326+0000","last_change":"2024-09-15T21:46:23.428634+0000","last_active":"2024-09-15T21:48:38.387326+0000","last_peered":"2024-09-15T21:48:38.387326+0000","last_clean":"2024-09-15T21:48:38.387326+0000","last_became_active":"2024-09-15T21:46:23.428301+0000","last_became_peered":"2024-09-15T21:46:23.428301+0000","last_unstale":"2024-09-15T21:48:38.387326+0000","last_undegraded":"2024-09-15T21:48:38.387326+0000","last_fullsized":"2024-09-15T21:48:38.387326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:59.492942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"22'4","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726625+0000","last_change":"2024-09-15T21:46:22.425810+0000","last_active":"2024-09-15T21:48:36.726625+0000","last_peered":"2024-09-15T21:48:36.726625+0000","last_clean":"2024-09-15T21:48:36.726625+0000","last_became_active":"2024-09-15T21:46:22.425478+0000","last_became_peered":"2024-09-15T21:46:22.425478+0000","last_unstale":"2024-09-15T21:48:36.726625+0000","last_undegraded":"2024-09-15T21:48:36.726625+0000","last_fullsized":"2024-09-15T21:48:36.726625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:53.296599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387462+0000","last_change":"2024-09-15T21:46:23.426054+0000","last_active":"2024-09-15T21:48:38.387462+0000","last_peered":"2024-09-15T21:48:38.387462+0000","last_clean":"2024-09-15T21:48:38.387462+0000","last_became_active":"2024-09-15T21:46:23.425547+0000","last_became_peered":"2024-09-15T21:46:23.425547+0000","last_unstale":"2024-09-15T21:48:38.387462+0000","last_undegraded":"2024-09-15T21:48:38.387462+0000","last_fullsized":"2024-09-15T21:48:38.387462+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:56.838503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"55'5","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710664+0000","last_change":"2024-09-15T21:48:21.653804+0000","last_active":"2024-09-15T21:48:35.710664+0000","last_peered":"2024-09-15T21:48:35.710664+0000","last_clean":"2024-09-15T21:48:35.710664+0000","last_became_active":"2024-09-15T21:48:21.639223+0000","last_became_peered":"2024-09-15T21:48:21.639223+0000","last_unstale":"2024-09-15T21:48:35.710664+0000","last_undegraded":"2024-09-15T21:48:35.710664+0000","last_fullsized":"2024-09-15T21:48:35.710664+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:47:15.330440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":118,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858680+0000","last_change":"2024-09-15T21:46:23.434719+0000","last_active":"2024-09-15T21:48:36.858680+0000","last_peered":"2024-09-15T21:48:36.858680+0000","last_clean":"2024-09-15T21:48:36.858680+0000","last_became_active":"2024-09-15T21:46:23.434448+0000","last_became_peered":"2024-09-15T21:46:23.434448+0000","last_unstale":"2024-09-15T21:48:36.858680+0000","last_undegraded":"2024-09-15T21:48:36.858680+0000","last_fullsized":"2024-09-15T21:48:36.858680+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:14.901671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":68,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850752+0000","last_change":"2024-09-15T21:48:27.657040+0000","last_active":"2024-09-15T21:48:36.850752+0000","last_peered":"2024-09-15T21:48:36.850752+0000","last_clean":"2024-09-15T21:48:36.850752+0000","last_became_active":"2024-09-15T21:48:27.656753+0000","last_became_peered":"2024-09-15T21:48:27.656753+0000","last_unstale":"2024-09-15T21:48:36.850752+0000","last_undegraded":"2024-09-15T21:48:36.850752+0000","last_fullsized":"2024-09-15T21:48:36.850752+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:14.458723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'5","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710454+0000","last_change":"2024-09-15T21:48:21.691408+0000","last_active":"2024-09-15T21:48:35.710454+0000","last_peered":"2024-09-15T21:48:35.710454+0000","last_clean":"2024-09-15T21:48:35.710454+0000","last_became_active":"2024-09-15T21:48:21.641517+0000","last_became_peered":"2024-09-15T21:48:21.641517+0000","last_unstale":"2024-09-15T21:48:35.710454+0000","last_undegraded":"2024-09-15T21:48:35.710454+0000","last_fullsized":"2024-09-15T21:48:35.710454+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:09.454433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022007+0000","last_change":"2024-09-15T21:48:17.623223+0000","last_active":"2024-09-15T21:48:37.022007+0000","last_peered":"2024-09-15T21:48:37.022007+0000","last_clean":"2024-09-15T21:48:37.022007+0000","last_became_active":"2024-09-15T21:48:17.622019+0000","last_became_peered":"2024-09-15T21:48:17.622019+0000","last_unstale":"2024-09-15T21:48:37.022007+0000","last_undegraded":"2024-09-15T21:48:37.022007+0000","last_fullsized":"2024-09-15T21:48:37.022007+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:39:57.643793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"55'13","reported_seq":53,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-15T21:48:35.710695+0000","last_change":"2024-09-15T21:48:21.682609+0000","last_active":"2024-09-15T21:48:35.710695+0000","last_peered":"2024-09-15T21:48:35.710695+0000","last_clean":"2024-09-15T21:48:35.710695+0000","last_became_active":"2024-09-15T21:48:21.640156+0000","last_became_peered":"2024-09-15T21:48:21.640156+0000","last_unstale":"2024-09-15T21:48:35.710695+0000","last_undegraded":"2024-09-15T21:48:35.710695+0000","last_fullsized":"2024-09-15T21:48:35.710695+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:28.936509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726585+0000","last_change":"2024-09-15T21:48:17.623164+0000","last_active":"2024-09-15T21:48:36.726585+0000","last_peered":"2024-09-15T21:48:36.726585+0000","last_clean":"2024-09-15T21:48:36.726585+0000","last_became_active":"2024-09-15T21:48:17.622843+0000","last_became_peered":"2024-09-15T21:48:17.622843+0000","last_unstale":"2024-09-15T21:48:36.726585+0000","last_undegraded":"2024-09-15T21:48:36.726585+0000","last_fullsized":"2024-09-15T21:48:36.726585+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:06:30.745189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'2","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387645+0000","last_change":"2024-09-15T21:46:22.428529+0000","last_active":"2024-09-15T21:48:38.387645+0000","last_peered":"2024-09-15T21:48:38.387645+0000","last_clean":"2024-09-15T21:48:38.387645+0000","last_became_active":"2024-09-15T21:46:22.428263+0000","last_became_peered":"2024-09-15T21:46:22.428263+0000","last_unstale":"2024-09-15T21:48:38.387645+0000","last_undegraded":"2024-09-15T21:48:38.387645+0000","last_fullsized":"2024-09-15T21:48:38.387645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:22:40.160062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":120,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387575+0000","last_change":"2024-09-15T21:46:23.427968+0000","last_active":"2024-09-15T21:48:38.387575+0000","last_peered":"2024-09-15T21:48:38.387575+0000","last_clean":"2024-09-15T21:48:38.387575+0000","last_became_active":"2024-09-15T21:46:23.427546+0000","last_became_peered":"2024-09-15T21:46:23.427546+0000","last_unstale":"2024-09-15T21:48:38.387575+0000","last_undegraded":"2024-09-15T21:48:38.387575+0000","last_fullsized":"2024-09-15T21:48:38.387575+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:47.226097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'13","reported_seq":171,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726869+0000","last_change":"2024-09-15T21:48:21.680908+0000","last_active":"2024-09-15T21:48:36.726869+0000","last_peered":"2024-09-15T21:48:36.726869+0000","last_clean":"2024-09-15T21:48:36.726869+0000","last_became_active":"2024-09-15T21:48:21.642406+0000","last_became_peered":"2024-09-15T21:48:21.642406+0000","last_unstale":"2024-09-15T21:48:36.726869+0000","last_undegraded":"2024-09-15T21:48:36.726869+0000","last_fullsized":"2024-09-15T21:48:36.726869+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:23:26.323380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1344,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":42655,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":243,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":4802,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":335872,"data_stored":85310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6368,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476769,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1507328,"data_stored":1002267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509473,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32660,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1503232,"data_stored":997684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509475,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":946176,"data_stored":447388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":57,"seq":244813135879,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29096,"kb_used_data":912,"kb_used_omap":7,"kb_used_meta":28152,"kb_avail":94342744,"statfs":{"total":96636764160,"available":96606969856,"internally_reserved":0,"allocated":933888,"data_stored":414957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7963,"internal_metadata":28827877},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509474,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":909312,"data_stored":412355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509473,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":921600,"data_stored":412891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":888832,"data_stored":400427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":954368,"data_stored":448082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":37057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6368,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":36363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:48:42.750 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:48:42.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.846+0000 7f89c8d18640 1 Processor -- start 2024-09-15T21:48:42.848 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.846+0000 7f89c8d18640 1 -- start start 2024-09-15T21:48:42.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.847+0000 7f89c8d18640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f89c4155440 0x7f89c40cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.847+0000 7f89c8d18640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f89c40cc8f0 0x7f89c40cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.847+0000 7f89c8d18640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89c415e7a0 con 0x7f89c40cb7d0 2024-09-15T21:48:42.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.847+0000 7f89c8d18640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f89c415b470 con 0x7f89c4155440 2024-09-15T21:48:42.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.847+0000 7f89c8d18640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f89c4155820 con 0x7f89c40cc8f0 2024-09-15T21:48:42.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c1d9b640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f89c40cb7d0 0x7f89c40cc1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42180/0 (socket says 172.21.15.100:42180) 2024-09-15T21:48:42.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c1d9b640 1 -- 172.21.15.100:0/1981589657 learned_addr learned my addr 172.21.15.100:0/1981589657 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:42.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c2d9d640 1 --2- 172.21.15.100:0/1981589657 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f89c40cc8f0 0x7f89c40cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:42.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c259c640 1 --2- 172.21.15.100:0/1981589657 >> v2:172.21.15.122:3300/0 conn(0x7f89c4155440 0x7f89c40cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:42.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c159a640 1 -- 172.21.15.100:0/1981589657 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3930360180 0 0) 0x7f89c415e7a0 con 0x7f89c40cb7d0 2024-09-15T21:48:42.853 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c159a640 1 -- 172.21.15.100:0/1981589657 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89b0003520 con 0x7f89c40cb7d0 2024-09-15T21:48:42.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c159a640 1 -- 172.21.15.100:0/1981589657 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 51164118 0 0) 0x7f89b0003520 con 0x7f89c40cb7d0 2024-09-15T21:48:42.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c159a640 1 -- 172.21.15.100:0/1981589657 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f89c40cc8f0 msgr2=0x7f89c40cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:42.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c159a640 1 --2- 172.21.15.100:0/1981589657 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f89c40cc8f0 0x7f89c40cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c159a640 1 -- 172.21.15.100:0/1981589657 >> v2:172.21.15.122:3300/0 conn(0x7f89c4155440 msgr2=0x7f89c40cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:42.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c159a640 1 --2- 172.21.15.100:0/1981589657 >> v2:172.21.15.122:3300/0 conn(0x7f89c4155440 0x7f89c40cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c159a640 1 -- 172.21.15.100:0/1981589657 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c415ea40 con 0x7f89c40cb7d0 2024-09-15T21:48:42.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c159a640 1 -- 172.21.15.100:0/1981589657 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7f89b4001670 con 0x7f89c40cb7d0 2024-09-15T21:48:42.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c259c640 1 --2- 172.21.15.100:0/1981589657 >> v2:172.21.15.122:3300/0 conn(0x7f89c4155440 0x7f89c40cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:48:42.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c159a640 1 -- 172.21.15.100:0/1981589657 >> v1:172.21.15.100:6789/0 conn(0x7f89c40cb7d0 legacy=0x7f89c40cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:42.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c159a640 1 --2- 172.21.15.100:0/1981589657 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f89c40cc8f0 0x7f89b00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c159a640 1 --2- 172.21.15.100:0/1981589657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89b0003f30 0x7f89b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c159a640 1 --2- 172.21.15.100:0/1981589657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b0004d40 0x7f89b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.854 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c159a640 1 -- 172.21.15.100:0/1981589657 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f89c415b470 con 0x7f89b0004d40 2024-09-15T21:48:42.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c159a640 1 -- 172.21.15.100:0/1981589657 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f89c4155820 con 0x7f89b0003f30 2024-09-15T21:48:42.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c159a640 1 -- 172.21.15.100:0/1981589657 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f89b000b9c0 con 0x7f89c40cc8f0 2024-09-15T21:48:42.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c2d9d640 1 --2- 172.21.15.100:0/1981589657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b0004d40 0x7f89b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:42.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c259c640 1 --2- 172.21.15.100:0/1981589657 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f89c40cc8f0 0x7f89b00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:42.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.848+0000 7f89c1d9b640 1 --2- 172.21.15.100:0/1981589657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89b0003f30 0x7f89b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:42.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.849+0000 7f89c2d9d640 1 -- 172.21.15.100:0/1981589657 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f89c40cc8f0 msgr2=0x7f89b00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:42.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.849+0000 7f89c2d9d640 1 --2- 172.21.15.100:0/1981589657 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f89c40cc8f0 0x7f89b00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.849+0000 7f89c2d9d640 1 -- 172.21.15.100:0/1981589657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89b0003f30 msgr2=0x7f89b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:42.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.849+0000 7f89c2d9d640 1 --2- 172.21.15.100:0/1981589657 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89b0003f30 0x7f89b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.849+0000 7f89c2d9d640 1 -- 172.21.15.100:0/1981589657 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b0003520 con 0x7f89b0004d40 2024-09-15T21:48:42.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.849+0000 7f89c259c640 1 --2- 172.21.15.100:0/1981589657 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f89c40cc8f0 0x7f89b00039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:48:42.855 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.849+0000 7f89c2d9d640 1 --2- 172.21.15.100:0/1981589657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b0004d40 0x7f89b00051c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009fa0 tx=0x7f89b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821fbc8cdaaab721 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:42.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.850+0000 7f89c159a640 1 -- 172.21.15.100:0/1981589657 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc009bf0 con 0x7f89b0004d40 2024-09-15T21:48:42.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.850+0000 7f89c159a640 1 -- 172.21.15.100:0/1981589657 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89bc036070 con 0x7f89b0004d40 2024-09-15T21:48:42.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.850+0000 7f89c159a640 1 -- 172.21.15.100:0/1981589657 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89bc030670 con 0x7f89b0004d40 2024-09-15T21:48:42.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.850+0000 7f89c8d18640 1 -- 172.21.15.100:0/1981589657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b0004d40 msgr2=0x7f89b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:42.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.850+0000 7f89c8d18640 1 --2- 172.21.15.100:0/1981589657 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b0004d40 0x7f89b00051c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009fa0 tx=0x7f89b0005830 comp rx=0 tx=0).stop 2024-09-15T21:48:42.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.850+0000 7f89c2d9d640 1 -- 172.21.15.100:0/1981589657 reap_dead start 2024-09-15T21:48:42.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.850+0000 7f89c8d18640 1 -- 172.21.15.100:0/1981589657 shutdown_connections 2024-09-15T21:48:42.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.850+0000 7f89c8d18640 1 -- 172.21.15.100:0/1981589657 >> 172.21.15.100:0/1981589657 conn(0x7f89c40680d0 msgr2=0x7f89c406d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:42.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.850+0000 7f89c8d18640 1 -- 172.21.15.100:0/1981589657 shutdown_connections 2024-09-15T21:48:42.856 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.850+0000 7f89c8d18640 1 -- 172.21.15.100:0/1981589657 wait complete. 2024-09-15T21:48:42.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.851+0000 7f89c8d18640 1 Processor -- start 2024-09-15T21:48:42.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.851+0000 7f89c8d18640 1 -- start start 2024-09-15T21:48:42.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.852+0000 7f89c8d18640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b0004d40 0x7f89c415c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.852+0000 7f89c8d18640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89c4150a30 0x7f89c4153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.852+0000 7f89c8d18640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f89c40c8b00 0x7f89c41f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.852+0000 7f89c8d18640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f89c415f2e0 con 0x7f89b0004d40 2024-09-15T21:48:42.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.852+0000 7f89c8d18640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f89c415f000 con 0x7f89c4150a30 2024-09-15T21:48:42.859 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.852+0000 7f89c8d18640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f89c415b070 con 0x7f89c40c8b00 2024-09-15T21:48:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.852+0000 7f89c259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b0004d40 0x7f89c415c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.852+0000 7f89c259c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b0004d40 0x7f89c415c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54498/0 (socket says 172.21.15.100:54498) 2024-09-15T21:48:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.852+0000 7f89c259c640 1 -- 172.21.15.100:0/1411660982 learned_addr learned my addr 172.21.15.100:0/1411660982 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.852+0000 7f89c259c640 1 -- 172.21.15.100:0/1411660982 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f89c40c8b00 msgr2=0x7f89c41f3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.852+0000 7f89c259c640 1 --2- 172.21.15.100:0/1411660982 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f89c40c8b00 0x7f89c41f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.852+0000 7f89c259c640 1 -- 172.21.15.100:0/1411660982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89c4150a30 msgr2=0x7f89c4153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.852+0000 7f89c259c640 1 --2- 172.21.15.100:0/1411660982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89c4150a30 0x7f89c4153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.852+0000 7f89c259c640 1 -- 172.21.15.100:0/1411660982 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c415e4d0 con 0x7f89b0004d40 2024-09-15T21:48:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.853+0000 7f89c259c640 1 --2- 172.21.15.100:0/1411660982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b0004d40 0x7f89c415c1d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f89ac011e10 tx=0x7f89ac00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.853+0000 7f89baffd640 1 -- 172.21.15.100:0/1411660982 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac014440 con 0x7f89b0004d40 2024-09-15T21:48:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.853+0000 7f89baffd640 1 -- 172.21.15.100:0/1411660982 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89ac0145e0 con 0x7f89b0004d40 2024-09-15T21:48:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.853+0000 7f89c8d18640 1 -- 172.21.15.100:0/1411660982 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c40c80a0 con 0x7f89b0004d40 2024-09-15T21:48:42.860 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.853+0000 7f89baffd640 1 -- 172.21.15.100:0/1411660982 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89ac014990 con 0x7f89b0004d40 2024-09-15T21:48:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.853+0000 7f89c8d18640 1 -- 172.21.15.100:0/1411660982 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c414fe00 con 0x7f89b0004d40 2024-09-15T21:48:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.854+0000 7f89c8d18640 1 -- 172.21.15.100:0/1411660982 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89c40b3940 con 0x7f89b0004d40 2024-09-15T21:48:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.854+0000 7f89baffd640 1 -- 172.21.15.100:0/1411660982 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f89ac012040 con 0x7f89b0004d40 2024-09-15T21:48:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.855+0000 7f89baffd640 1 --2- 172.21.15.100:0/1411660982 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f89a406c2b0 0x7f89a406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.855+0000 7f89baffd640 1 -- 172.21.15.100:0/1411660982 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f89ac08bf40 con 0x7f89b0004d40 2024-09-15T21:48:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.855+0000 7f89c1d9b640 1 --2- 172.21.15.100:0/1411660982 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f89a406c2b0 0x7f89a406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.855+0000 7f89c1d9b640 1 --2- 172.21.15.100:0/1411660982 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f89a406c2b0 0x7f89a406e770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f89b4001be0 tx=0x7f89b4001b70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:42.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:42.859+0000 7f89baffd640 1 -- 172.21.15.100:0/1411660982 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f89ac019030 con 0x7f89b0004d40 2024-09-15T21:48:43.026 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:43.025+0000 7f89c8d18640 1 -- 172.21.15.100:0/1411660982 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89c40caed0 con 0x7f89b0004d40 2024-09-15T21:48:43.028 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:43.026+0000 7f89baffd640 1 -- 172.21.15.100:0/1411660982 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+16959 (secure 0 0 0) 0x7f89ac059f30 con 0x7f89b0004d40 2024-09-15T21:48:43.029 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:48:43.029 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":76,"fsid":"30e2fff4-2171-4531-8806-0e03b63f494f","created":"2024-09-15T21:45:59.746897+0000","modified":"2024-09-15T21:48:36.714481+0000","last_up_change":"2024-09-15T21:48:16.603329+0000","last_in_change":"2024-09-15T21:48:20.610986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:46:11.837603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T21:46:20.577982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T21:46:21.729231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5887900b-522d-476f-9704-cc2c5ee87a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6819","nonce":3920435546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6821","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6822","nonce":3920435546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6829","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6831","nonce":3920435546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6825","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6827","nonce":3920435546}]},"public_addr":"172.21.15.100:6819/3920435546","cluster_addr":"172.21.15.100:6822/3920435546","heartbeat_back_addr":"172.21.15.100:6831/3920435546","heartbeat_front_addr":"172.21.15.100:6827/3920435546","state":["exists","up"]},{"osd":1,"uuid":"adc17993-5c50-4251-85e1-4c7ac62e25e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6801","nonce":3802679972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6803","nonce":3802679972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6807","nonce":3802679972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6805","nonce":3802679972}]},"public_addr":"172.21.15.100:6801/3802679972","cluster_addr":"172.21.15.100:6803/3802679972","heartbeat_back_addr":"172.21.15.100:6807/3802679972","heartbeat_front_addr":"172.21.15.100:6805/3802679972","state":["exists","up"]},{"osd":2,"uuid":"8de3b7d5-1980-4a3b-a934-264f867c7c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6823","nonce":2316787974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6826","nonce":2316787974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6833","nonce":2316787974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6830","nonce":2316787974}]},"public_addr":"172.21.15.100:6823/2316787974","cluster_addr":"172.21.15.100:6826/2316787974","heartbeat_back_addr":"172.21.15.100:6833/2316787974","heartbeat_front_addr":"172.21.15.100:6830/2316787974","state":["exists","up"]},{"osd":3,"uuid":"93dbcd1c-6a6f-49cf-bc94-e64b883de5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6811","nonce":1287562741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6813","nonce":1287562741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6817","nonce":1287562741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6815","nonce":1287562741}]},"public_addr":"172.21.15.100:6811/1287562741","cluster_addr":"172.21.15.100:6813/1287562741","heartbeat_back_addr":"172.21.15.100:6817/1287562741","heartbeat_front_addr":"172.21.15.100:6815/1287562741","state":["exists","up"]},{"osd":4,"uuid":"7caee2a0-bab4-45a9-8109-724948c863c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6801","nonce":4045445512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6803","nonce":4045445512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6807","nonce":4045445512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6805","nonce":4045445512}]},"public_addr":"172.21.15.122:6801/4045445512","cluster_addr":"172.21.15.122:6803/4045445512","heartbeat_back_addr":"172.21.15.122:6807/4045445512","heartbeat_front_addr":"172.21.15.122:6805/4045445512","state":["exists","up"]},{"osd":5,"uuid":"6e64070a-671a-444b-a061-b3051a29b121","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":29,"up_from":57,"up_thru":60,"down_at":30,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":822572374},{"type":"v1","addr":"172.21.15.122:6825","nonce":822572374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":822572374},{"type":"v1","addr":"172.21.15.122:6827","nonce":822572374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":822572374},{"type":"v1","addr":"172.21.15.122:6831","nonce":822572374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":822572374},{"type":"v1","addr":"172.21.15.122:6829","nonce":822572374}]},"public_addr":"172.21.15.122:6825/822572374","cluster_addr":"172.21.15.122:6827/822572374","heartbeat_back_addr":"172.21.15.122:6831/822572374","heartbeat_front_addr":"172.21.15.122:6829/822572374","state":["exists","up"]},{"osd":6,"uuid":"47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6809","nonce":1437167311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6811","nonce":1437167311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6815","nonce":1437167311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6813","nonce":1437167311}]},"public_addr":"172.21.15.122:6809/1437167311","cluster_addr":"172.21.15.122:6811/1437167311","heartbeat_back_addr":"172.21.15.122:6815/1437167311","heartbeat_front_addr":"172.21.15.122:6813/1437167311","state":["exists","up"]},{"osd":7,"uuid":"9a916b6d-af82-486f-af40-4a7291c900fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6817","nonce":2431377162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6819","nonce":2431377162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6823","nonce":2431377162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6821","nonce":2431377162}]},"public_addr":"172.21.15.122:6817/2431377162","cluster_addr":"172.21.15.122:6819/2431377162","heartbeat_back_addr":"172.21.15.122:6823/2431377162","heartbeat_front_addr":"172.21.15.122:6821/2431377162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.168771+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.886614+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.154129+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.570531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.926773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.146320+0000","dead_epoch":30},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.409137+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.373359+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.8","osds":[0,5]}],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:48:43.031 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:43.028+0000 7f89c8d18640 1 -- 172.21.15.100:0/1411660982 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f89a406c2b0 msgr2=0x7f89a406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:43.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:43.028+0000 7f89c8d18640 1 --2- 172.21.15.100:0/1411660982 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f89a406c2b0 0x7f89a406e770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f89b4001be0 tx=0x7f89b4001b70 comp rx=0 tx=0).stop 2024-09-15T21:48:43.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:43.028+0000 7f89c8d18640 1 -- 172.21.15.100:0/1411660982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b0004d40 msgr2=0x7f89c415c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:43.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:43.028+0000 7f89c8d18640 1 --2- 172.21.15.100:0/1411660982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b0004d40 0x7f89c415c1d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f89ac011e10 tx=0x7f89ac00f610 comp rx=0 tx=0).stop 2024-09-15T21:48:43.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:43.028+0000 7f89c8d18640 1 -- 172.21.15.100:0/1411660982 shutdown_connections 2024-09-15T21:48:43.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:43.028+0000 7f89c8d18640 1 --2- 172.21.15.100:0/1411660982 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f89a406c2b0 0x7f89a406e770 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:43.032 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:43.028+0000 7f89c8d18640 1 --2- 172.21.15.100:0/1411660982 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89c4150a30 0x7f89c4153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:43.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:43.028+0000 7f89c8d18640 1 --2- 172.21.15.100:0/1411660982 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f89c40c8b00 0x7f89c41f3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:43.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:43.028+0000 7f89c8d18640 1 --2- 172.21.15.100:0/1411660982 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89b0004d40 0x7f89c415c1d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:43.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:43.028+0000 7f89c8d18640 1 -- 172.21.15.100:0/1411660982 >> 172.21.15.100:0/1411660982 conn(0x7f89c40680d0 msgr2=0x7f89c406ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:43.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:43.028+0000 7f89c8d18640 1 -- 172.21.15.100:0/1411660982 shutdown_connections 2024-09-15T21:48:43.033 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:43.028+0000 7f89c8d18640 1 -- 172.21.15.100:0/1411660982 wait complete. 2024-09-15T21:48:44.043 INFO:tasks.ceph:Scrubbing osd.0 2024-09-15T21:48:44.044 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:48:44.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.157+0000 7ff35496d640 1 Processor -- start 2024-09-15T21:48:44.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.157+0000 7ff35496d640 1 -- start start 2024-09-15T21:48:44.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.158+0000 7ff35496d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff350067bf0 0x7ff35015b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.158+0000 7ff35496d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff35015bc70 0x7ff3500ccf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.158+0000 7ff35496d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff35006c5b0 con 0x7ff3500c6ee0 2024-09-15T21:48:44.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.158+0000 7ff35496d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff3500c7ba0 con 0x7ff350067bf0 2024-09-15T21:48:44.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.158+0000 7ff35496d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7ff3500c7e80 con 0x7ff35015bc70 2024-09-15T21:48:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.158+0000 7ff345d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff350067bf0 0x7ff35015b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.158+0000 7ff345d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff350067bf0 0x7ff35015b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54500/0 (socket says 172.21.15.100:54500) 2024-09-15T21:48:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.158+0000 7ff345d9b640 1 -- 172.21.15.100:0/383795474 learned_addr learned my addr 172.21.15.100:0/383795474 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.158+0000 7ff34ed9d640 1 --2- 172.21.15.100:0/383795474 >> v2:172.21.15.122:3300/0 conn(0x7ff35015bc70 0x7ff3500ccf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.158+0000 7ff345d9b640 1 -- 172.21.15.100:0/383795474 >> v2:172.21.15.122:3300/0 conn(0x7ff35015bc70 msgr2=0x7ff3500ccf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.158+0000 7ff345d9b640 1 --2- 172.21.15.100:0/383795474 >> v2:172.21.15.122:3300/0 conn(0x7ff35015bc70 0x7ff3500ccf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.158+0000 7ff345d9b640 1 -- 172.21.15.100:0/383795474 >> v1:172.21.15.122:6789/0 conn(0x7ff3500c6ee0 legacy=0x7ff350071870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.158+0000 7ff345d9b640 1 -- 172.21.15.100:0/383795474 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3500c8160 con 0x7ff350067bf0 2024-09-15T21:48:44.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.159+0000 7ff345d9b640 1 --2- 172.21.15.100:0/383795474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff350067bf0 0x7ff35015b730 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff3501558a0 tx=0x7ff33802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35a9a11350b4951 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:44.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.159+0000 7ff34dd9b640 1 -- 172.21.15.100:0/383795474 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338037070 con 0x7ff350067bf0 2024-09-15T21:48:44.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.159+0000 7ff34dd9b640 1 -- 172.21.15.100:0/383795474 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff338032440 con 0x7ff350067bf0 2024-09-15T21:48:44.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.159+0000 7ff34dd9b640 1 -- 172.21.15.100:0/383795474 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338032810 con 0x7ff350067bf0 2024-09-15T21:48:44.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.159+0000 7ff35496d640 1 -- 172.21.15.100:0/383795474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff350067bf0 msgr2=0x7ff35015b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.159+0000 7ff35496d640 1 --2- 172.21.15.100:0/383795474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff350067bf0 0x7ff35015b730 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7ff3501558a0 tx=0x7ff33802a230 comp rx=0 tx=0).stop 2024-09-15T21:48:44.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.159+0000 7ff35496d640 1 -- 172.21.15.100:0/383795474 shutdown_connections 2024-09-15T21:48:44.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.159+0000 7ff35496d640 1 --2- 172.21.15.100:0/383795474 >> v2:172.21.15.122:3300/0 conn(0x7ff35015bc70 0x7ff3500ccf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.159+0000 7ff35496d640 1 --2- 172.21.15.100:0/383795474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff350067bf0 0x7ff35015b730 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.160+0000 7ff35496d640 1 -- 172.21.15.100:0/383795474 >> 172.21.15.100:0/383795474 conn(0x7ff35006da00 msgr2=0x7ff35006bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:44.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.160+0000 7ff35496d640 1 -- 172.21.15.100:0/383795474 shutdown_connections 2024-09-15T21:48:44.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.160+0000 7ff35496d640 1 -- 172.21.15.100:0/383795474 wait complete. 2024-09-15T21:48:44.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.161+0000 7ff35496d640 1 Processor -- start 2024-09-15T21:48:44.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.161+0000 7ff35496d640 1 -- start start 2024-09-15T21:48:44.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.162+0000 7ff35496d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff350067bf0 0x7ff35006f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.162+0000 7ff35496d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3500c6ee0 0x7ff350138b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.162+0000 7ff35496d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff35015bc70 0x7ff350139080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.162+0000 7ff35496d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff3500c72c0 con 0x7ff35015bc70 2024-09-15T21:48:44.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.162+0000 7ff35496d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff35015b210 con 0x7ff3500c6ee0 2024-09-15T21:48:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.162+0000 7ff35496d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff3500c7440 con 0x7ff350067bf0 2024-09-15T21:48:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.162+0000 7ff34ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff35015bc70 0x7ff350139080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.162+0000 7ff34ed9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff35015bc70 0x7ff350139080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54506/0 (socket says 172.21.15.100:54506) 2024-09-15T21:48:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.162+0000 7ff34ed9d640 1 -- 172.21.15.100:0/2614927705 learned_addr learned my addr 172.21.15.100:0/2614927705 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.162+0000 7ff34e59c640 1 --2- 172.21.15.100:0/2614927705 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff350067bf0 0x7ff35006f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.162+0000 7ff345d9b640 1 --2- 172.21.15.100:0/2614927705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3500c6ee0 0x7ff350138b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.162+0000 7ff34ed9d640 1 -- 172.21.15.100:0/2614927705 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff350067bf0 msgr2=0x7ff35006f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.162+0000 7ff34ed9d640 1 --2- 172.21.15.100:0/2614927705 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff350067bf0 0x7ff35006f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.162+0000 7ff34ed9d640 1 -- 172.21.15.100:0/2614927705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3500c6ee0 msgr2=0x7ff350138b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.162+0000 7ff34ed9d640 1 --2- 172.21.15.100:0/2614927705 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3500c6ee0 0x7ff350138b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.162+0000 7ff34ed9d640 1 -- 172.21.15.100:0/2614927705 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3501e6650 con 0x7ff35015bc70 2024-09-15T21:48:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.162+0000 7ff34ed9d640 1 --2- 172.21.15.100:0/2614927705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff35015bc70 0x7ff350139080 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff348007e90 tx=0x7ff34800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.163+0000 7ff3477fe640 1 -- 172.21.15.100:0/2614927705 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348014420 con 0x7ff35015bc70 2024-09-15T21:48:44.167 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.163+0000 7ff3477fe640 1 -- 172.21.15.100:0/2614927705 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff348019070 con 0x7ff35015bc70 2024-09-15T21:48:44.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.163+0000 7ff35496d640 1 -- 172.21.15.100:0/2614927705 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff35015abf0 con 0x7ff35015bc70 2024-09-15T21:48:44.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.163+0000 7ff3477fe640 1 -- 172.21.15.100:0/2614927705 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff348014800 con 0x7ff35015bc70 2024-09-15T21:48:44.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.163+0000 7ff35496d640 1 -- 172.21.15.100:0/2614927705 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff35015c1a0 con 0x7ff35015bc70 2024-09-15T21:48:44.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.163+0000 7ff35496d640 1 -- 172.21.15.100:0/2614927705 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff35015cb40 con 0x7ff35015bc70 2024-09-15T21:48:44.168 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.164+0000 7ff3477fe640 1 -- 172.21.15.100:0/2614927705 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff3480149a0 con 0x7ff35015bc70 2024-09-15T21:48:44.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.165+0000 7ff3477fe640 1 --2- 172.21.15.100:0/2614927705 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff33006c2e0 0x7ff33006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.165+0000 7ff3477fe640 1 -- 172.21.15.100:0/2614927705 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7ff34808bb60 con 0x7ff35015bc70 2024-09-15T21:48:44.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.165+0000 7ff34e59c640 1 --2- 172.21.15.100:0/2614927705 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff33006c2e0 0x7ff33006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.165+0000 7ff3477fe640 1 --2- 172.21.15.100:0/2614927705 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7ff330074660 0x7ff330076b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.169 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.165+0000 7ff3477fe640 1 -- 172.21.15.100:0/2614927705 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff3300771d0 con 0x7ff330074660 2024-09-15T21:48:44.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.165+0000 7ff3477fe640 1 -- 172.21.15.100:0/2614927705 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7ff3480902c0 con 0x7ff35015bc70 2024-09-15T21:48:44.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.165+0000 7ff345d9b640 1 --2- 172.21.15.100:0/2614927705 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7ff330074660 0x7ff330076b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.165+0000 7ff34e59c640 1 --2- 172.21.15.100:0/2614927705 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff33006c2e0 0x7ff33006e7a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff340004f60 tx=0x7ff340007e30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:44.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.165+0000 7ff345d9b640 1 --2- 172.21.15.100:0/2614927705 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7ff330074660 0x7ff330076b00 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:44.170 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.166+0000 7ff3477fe640 1 -- 172.21.15.100:0/2614927705 <== osd.0 v2:172.21.15.100:6818/3920435546 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff3300771d0 con 0x7ff330074660 2024-09-15T21:48:44.180 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.178+0000 7ff35496d640 1 -- 172.21.15.100:0/2614927705 --> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff35015b730 con 0x7ff330074660 2024-09-15T21:48:44.181 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.179+0000 7ff3477fe640 1 -- 172.21.15.100:0/2614927705 <== osd.0 v2:172.21.15.100:6818/3920435546 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff35015b730 con 0x7ff330074660 2024-09-15T21:48:44.181 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:44.181 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:48:44.181 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:44.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.179+0000 7ff35496d640 1 -- 172.21.15.100:0/2614927705 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7ff330074660 msgr2=0x7ff330076b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.179+0000 7ff35496d640 1 --2- 172.21.15.100:0/2614927705 >> [v2:172.21.15.100:6818/3920435546,v1:172.21.15.100:6819/3920435546] conn(0x7ff330074660 0x7ff330076b00 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.179+0000 7ff35496d640 1 -- 172.21.15.100:0/2614927705 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff33006c2e0 msgr2=0x7ff33006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.179+0000 7ff35496d640 1 --2- 172.21.15.100:0/2614927705 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff33006c2e0 0x7ff33006e7a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff340004f60 tx=0x7ff340007e30 comp rx=0 tx=0).stop 2024-09-15T21:48:44.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.179+0000 7ff35496d640 1 -- 172.21.15.100:0/2614927705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff35015bc70 msgr2=0x7ff350139080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.179+0000 7ff35496d640 1 --2- 172.21.15.100:0/2614927705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff35015bc70 0x7ff350139080 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7ff348007e90 tx=0x7ff34800e4b0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.179+0000 7ff34ed9d640 1 -- 172.21.15.100:0/2614927705 reap_dead start 2024-09-15T21:48:44.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.179+0000 7ff35496d640 1 -- 172.21.15.100:0/2614927705 shutdown_connections 2024-09-15T21:48:44.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.179+0000 7ff35496d640 1 -- 172.21.15.100:0/2614927705 >> 172.21.15.100:0/2614927705 conn(0x7ff35006da00 msgr2=0x7ff3500cc790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:44.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.179+0000 7ff35496d640 1 -- 172.21.15.100:0/2614927705 shutdown_connections 2024-09-15T21:48:44.184 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.179+0000 7ff35496d640 1 -- 172.21.15.100:0/2614927705 wait complete. 2024-09-15T21:48:44.191 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-15T21:48:44.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.289+0000 7f931dbe4640 1 Processor -- start 2024-09-15T21:48:44.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.289+0000 7f931dbe4640 1 -- start start 2024-09-15T21:48:44.294 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.290+0000 7f931dbe4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9318073790 0x7f93180c4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.290+0000 7f931dbe4640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f931806ccf0 0x7f931806d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.290+0000 7f931dbe4640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9318157ce0 con 0x7f9318152160 2024-09-15T21:48:44.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.290+0000 7f931dbe4640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f9318072bf0 con 0x7f931806ccf0 2024-09-15T21:48:44.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.290+0000 7f931dbe4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f9318157fa0 con 0x7f9318073790 2024-09-15T21:48:44.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.290+0000 7f9317fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f931806ccf0 0x7f931806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.290+0000 7f9317fff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f931806ccf0 0x7f931806d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54516/0 (socket says 172.21.15.100:54516) 2024-09-15T21:48:44.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.290+0000 7f9317fff640 1 -- 172.21.15.100:0/4047115281 learned_addr learned my addr 172.21.15.100:0/4047115281 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:44.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.290+0000 7f930ffff640 1 --2- 172.21.15.100:0/4047115281 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9318073790 0x7f93180c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.290+0000 7f9317fff640 1 -- 172.21.15.100:0/4047115281 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9318073790 msgr2=0x7f93180c4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:44.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.290+0000 7f9317fff640 1 --2- 172.21.15.100:0/4047115281 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9318073790 0x7f93180c4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.290+0000 7f9317fff640 1 -- 172.21.15.100:0/4047115281 >> v1:172.21.15.100:6789/0 conn(0x7f9318152160 legacy=0x7f9318073080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.290+0000 7f9317fff640 1 -- 172.21.15.100:0/4047115281 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9318158280 con 0x7f931806ccf0 2024-09-15T21:48:44.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.291+0000 7f9317fff640 1 --2- 172.21.15.100:0/4047115281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931806ccf0 0x7f931806d160 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9310009e50 tx=0x7f931002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d74758eabd23623e server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:44.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.291+0000 7f9316ffd640 1 -- 172.21.15.100:0/4047115281 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9310036070 con 0x7f931806ccf0 2024-09-15T21:48:44.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.291+0000 7f9316ffd640 1 -- 172.21.15.100:0/4047115281 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9310031600 con 0x7f931806ccf0 2024-09-15T21:48:44.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.291+0000 7f9316ffd640 1 -- 172.21.15.100:0/4047115281 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93100319d0 con 0x7f931806ccf0 2024-09-15T21:48:44.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.291+0000 7f931dbe4640 1 -- 172.21.15.100:0/4047115281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931806ccf0 msgr2=0x7f931806d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.291+0000 7f931dbe4640 1 --2- 172.21.15.100:0/4047115281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931806ccf0 0x7f931806d160 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9310009e50 tx=0x7f931002a710 comp rx=0 tx=0).stop 2024-09-15T21:48:44.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.291+0000 7f931dbe4640 1 -- 172.21.15.100:0/4047115281 shutdown_connections 2024-09-15T21:48:44.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.291+0000 7f931dbe4640 1 --2- 172.21.15.100:0/4047115281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931806ccf0 0x7f931806d160 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.291+0000 7f931dbe4640 1 --2- 172.21.15.100:0/4047115281 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9318073790 0x7f93180c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.291+0000 7f931dbe4640 1 -- 172.21.15.100:0/4047115281 >> 172.21.15.100:0/4047115281 conn(0x7f93180ca390 msgr2=0x7f93180ca770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:44.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.292+0000 7f931dbe4640 1 -- 172.21.15.100:0/4047115281 shutdown_connections 2024-09-15T21:48:44.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.292+0000 7f931dbe4640 1 -- 172.21.15.100:0/4047115281 wait complete. 2024-09-15T21:48:44.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.293+0000 7f931dbe4640 1 Processor -- start 2024-09-15T21:48:44.297 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.293+0000 7f931dbe4640 1 -- start start 2024-09-15T21:48:44.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.293+0000 7f931dbe4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931806ccf0 0x7f9318153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.293+0000 7f931dbe4640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9318073790 0x7f93181ef610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.293+0000 7f931dbe4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9318152160 0x7f93180c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.293+0000 7f931dbe4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9318158b20 con 0x7f931806ccf0 2024-09-15T21:48:44.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.293+0000 7f931dbe4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9318158840 con 0x7f9318152160 2024-09-15T21:48:44.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.293+0000 7f931dbe4640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f931806efd0 con 0x7f9318073790 2024-09-15T21:48:44.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.293+0000 7f93177fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931806ccf0 0x7f9318153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.293+0000 7f93177fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931806ccf0 0x7f9318153710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54530/0 (socket says 172.21.15.100:54530) 2024-09-15T21:48:44.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.293+0000 7f93177fe640 1 -- 172.21.15.100:0/3802683057 learned_addr learned my addr 172.21.15.100:0/3802683057 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:44.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.294+0000 7f930ffff640 1 --2- 172.21.15.100:0/3802683057 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9318073790 0x7f93181ef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.294+0000 7f9317fff640 1 --2- 172.21.15.100:0/3802683057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9318152160 0x7f93180c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.294+0000 7f93177fe640 1 -- 172.21.15.100:0/3802683057 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9318073790 msgr2=0x7f93181ef610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.294+0000 7f93177fe640 1 --2- 172.21.15.100:0/3802683057 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9318073790 0x7f93181ef610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.294+0000 7f93177fe640 1 -- 172.21.15.100:0/3802683057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9318152160 msgr2=0x7f93180c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.294+0000 7f93177fe640 1 --2- 172.21.15.100:0/3802683057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9318152160 0x7f93180c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.294+0000 7f93177fe640 1 -- 172.21.15.100:0/3802683057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93181f2fc0 con 0x7f931806ccf0 2024-09-15T21:48:44.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.294+0000 7f93177fe640 1 --2- 172.21.15.100:0/3802683057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931806ccf0 0x7f9318153710 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f930800c3f0 tx=0x7f930800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:44.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.294+0000 7f9314ff9640 1 -- 172.21.15.100:0/3802683057 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93080052d0 con 0x7f931806ccf0 2024-09-15T21:48:44.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.294+0000 7f9314ff9640 1 -- 172.21.15.100:0/3802683057 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9308005490 con 0x7f931806ccf0 2024-09-15T21:48:44.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.294+0000 7f931dbe4640 1 -- 172.21.15.100:0/3802683057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93180728a0 con 0x7f931806ccf0 2024-09-15T21:48:44.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.294+0000 7f9314ff9640 1 -- 172.21.15.100:0/3802683057 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308007260 con 0x7f931806ccf0 2024-09-15T21:48:44.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.294+0000 7f931dbe4640 1 -- 172.21.15.100:0/3802683057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93181eeb50 con 0x7f931806ccf0 2024-09-15T21:48:44.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.295+0000 7f931dbe4640 1 -- 172.21.15.100:0/3802683057 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93180c4600 con 0x7f931806ccf0 2024-09-15T21:48:44.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.296+0000 7f9314ff9640 1 -- 172.21.15.100:0/3802683057 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9308007400 con 0x7f931806ccf0 2024-09-15T21:48:44.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.296+0000 7f9314ff9640 1 --2- 172.21.15.100:0/3802683057 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f92f806c2b0 0x7f92f806e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.296+0000 7f9314ff9640 1 -- 172.21.15.100:0/3802683057 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f930808a910 con 0x7f931806ccf0 2024-09-15T21:48:44.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.296+0000 7f930ffff640 1 --2- 172.21.15.100:0/3802683057 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f92f806c2b0 0x7f92f806e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.296+0000 7f930ffff640 1 --2- 172.21.15.100:0/3802683057 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f92f806c2b0 0x7f92f806e770 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f93181497d0 tx=0x7f9300008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:44.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.300+0000 7f9314ff9640 1 -- 172.21.15.100:0/3802683057 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9308059070 con 0x7f931806ccf0 2024-09-15T21:48:44.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.467+0000 7f931dbe4640 1 -- 172.21.15.100:0/3802683057 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f931806dd40 con 0x7f92f806c2b0 2024-09-15T21:48:44.469 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.467+0000 7f9314ff9640 1 -- 172.21.15.100:0/3802683057 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f931806dd40 con 0x7f92f806c2b0 2024-09-15T21:48:44.469 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 0 to deep-scrub 2024-09-15T21:48:44.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.469+0000 7f931dbe4640 1 -- 172.21.15.100:0/3802683057 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f92f806c2b0 msgr2=0x7f92f806e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.469+0000 7f931dbe4640 1 --2- 172.21.15.100:0/3802683057 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f92f806c2b0 0x7f92f806e770 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f93181497d0 tx=0x7f9300008040 comp rx=0 tx=0).stop 2024-09-15T21:48:44.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.469+0000 7f931dbe4640 1 -- 172.21.15.100:0/3802683057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931806ccf0 msgr2=0x7f9318153710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.469+0000 7f931dbe4640 1 --2- 172.21.15.100:0/3802683057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931806ccf0 0x7f9318153710 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f930800c3f0 tx=0x7f930800c8b0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.469+0000 7f931dbe4640 1 -- 172.21.15.100:0/3802683057 shutdown_connections 2024-09-15T21:48:44.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.469+0000 7f931dbe4640 1 --2- 172.21.15.100:0/3802683057 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f92f806c2b0 0x7f92f806e770 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.469+0000 7f931dbe4640 1 --2- 172.21.15.100:0/3802683057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9318152160 0x7f93180c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.469+0000 7f931dbe4640 1 --2- 172.21.15.100:0/3802683057 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9318073790 0x7f93181ef610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.469+0000 7f931dbe4640 1 --2- 172.21.15.100:0/3802683057 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f931806ccf0 0x7f9318153710 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.469+0000 7f931dbe4640 1 -- 172.21.15.100:0/3802683057 >> 172.21.15.100:0/3802683057 conn(0x7f93180ca390 msgr2=0x7f9318073080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:44.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.469+0000 7f931dbe4640 1 -- 172.21.15.100:0/3802683057 shutdown_connections 2024-09-15T21:48:44.472 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.470+0000 7f931dbe4640 1 -- 172.21.15.100:0/3802683057 wait complete. 2024-09-15T21:48:44.482 INFO:tasks.ceph:Scrubbing osd.1 2024-09-15T21:48:44.482 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:48:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.584+0000 7ff7c71f6640 1 Processor -- start 2024-09-15T21:48:44.586 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.584+0000 7ff7c71f6640 1 -- start start 2024-09-15T21:48:44.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.585+0000 7ff7c71f6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff7c0072b40 0x7ff7c015c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.585+0000 7ff7c71f6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7c00c8f40 0x7ff7c00c8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.585+0000 7ff7c71f6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7c0072210 con 0x7ff7c0152380 2024-09-15T21:48:44.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.585+0000 7ff7c71f6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff7c0071fc0 con 0x7ff7c0072b40 2024-09-15T21:48:44.587 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.585+0000 7ff7c71f6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff7c0158090 con 0x7ff7c00c8f40 2024-09-15T21:48:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.585+0000 7ff7bffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff7c0072b40 0x7ff7c015c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.585+0000 7ff7bffff640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff7c0072b40 0x7ff7c015c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54534/0 (socket says 172.21.15.100:54534) 2024-09-15T21:48:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.585+0000 7ff7bffff640 1 -- 172.21.15.100:0/2280776560 learned_addr learned my addr 172.21.15.100:0/2280776560 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.585+0000 7ff7c5793640 1 --2- 172.21.15.100:0/2280776560 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7c00c8f40 0x7ff7c00c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.585+0000 7ff7bffff640 1 -- 172.21.15.100:0/2280776560 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7c00c8f40 msgr2=0x7ff7c00c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.585+0000 7ff7bffff640 1 --2- 172.21.15.100:0/2280776560 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7c00c8f40 0x7ff7c00c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.585+0000 7ff7bffff640 1 -- 172.21.15.100:0/2280776560 >> v1:172.21.15.100:6789/0 conn(0x7ff7c0152380 legacy=0x7ff7c0158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.585+0000 7ff7bffff640 1 -- 172.21.15.100:0/2280776560 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c00724f0 con 0x7ff7c0072b40 2024-09-15T21:48:44.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.586+0000 7ff7bffff640 1 --2- 172.21.15.100:0/2280776560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c0072b40 0x7ff7c015c090 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0072140 tx=0x7ff7ac02a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56854b49380fbf18 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:44.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.586+0000 7ff7bf7fe640 1 -- 172.21.15.100:0/2280776560 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac037070 con 0x7ff7c0072b40 2024-09-15T21:48:44.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.586+0000 7ff7bf7fe640 1 -- 172.21.15.100:0/2280776560 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7ac032440 con 0x7ff7c0072b40 2024-09-15T21:48:44.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.586+0000 7ff7bf7fe640 1 -- 172.21.15.100:0/2280776560 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac032810 con 0x7ff7c0072b40 2024-09-15T21:48:44.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.587+0000 7ff7c71f6640 1 -- 172.21.15.100:0/2280776560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c0072b40 msgr2=0x7ff7c015c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.587+0000 7ff7c71f6640 1 --2- 172.21.15.100:0/2280776560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c0072b40 0x7ff7c015c090 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0072140 tx=0x7ff7ac02a250 comp rx=0 tx=0).stop 2024-09-15T21:48:44.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.587+0000 7ff7c71f6640 1 -- 172.21.15.100:0/2280776560 shutdown_connections 2024-09-15T21:48:44.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.587+0000 7ff7c71f6640 1 --2- 172.21.15.100:0/2280776560 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7c00c8f40 0x7ff7c00c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.587+0000 7ff7c71f6640 1 --2- 172.21.15.100:0/2280776560 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c0072b40 0x7ff7c015c090 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.587+0000 7ff7c71f6640 1 -- 172.21.15.100:0/2280776560 >> 172.21.15.100:0/2280776560 conn(0x7ff7c00c9380 msgr2=0x7ff7c00c9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:44.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.587+0000 7ff7c71f6640 1 -- 172.21.15.100:0/2280776560 shutdown_connections 2024-09-15T21:48:44.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.587+0000 7ff7c71f6640 1 -- 172.21.15.100:0/2280776560 wait complete. 2024-09-15T21:48:44.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.588+0000 7ff7c71f6640 1 Processor -- start 2024-09-15T21:48:44.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.588+0000 7ff7c71f6640 1 -- start start 2024-09-15T21:48:44.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.589+0000 7ff7c71f6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7c0072b40 0x7ff7c00c1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.592 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.589+0000 7ff7c71f6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7c00c8f40 0x7ff7c01ef160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.589+0000 7ff7c71f6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c0152380 0x7ff7c014c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.589+0000 7ff7c71f6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0152760 con 0x7ff7c0152380 2024-09-15T21:48:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.589+0000 7ff7c71f6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c0158650 con 0x7ff7c0072b40 2024-09-15T21:48:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.589+0000 7ff7c71f6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff7c006de80 con 0x7ff7c00c8f40 2024-09-15T21:48:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.589+0000 7ff7c5793640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c0152380 0x7ff7c014c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.589+0000 7ff7c5793640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c0152380 0x7ff7c014c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54550/0 (socket says 172.21.15.100:54550) 2024-09-15T21:48:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.589+0000 7ff7c5793640 1 -- 172.21.15.100:0/2591719784 learned_addr learned my addr 172.21.15.100:0/2591719784 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.589+0000 7ff7bffff640 1 --2- 172.21.15.100:0/2591719784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7c00c8f40 0x7ff7c01ef160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.589+0000 7ff7c4f92640 1 --2- 172.21.15.100:0/2591719784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7c0072b40 0x7ff7c00c1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.589+0000 7ff7c5793640 1 -- 172.21.15.100:0/2591719784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7c00c8f40 msgr2=0x7ff7c01ef160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.589+0000 7ff7c5793640 1 --2- 172.21.15.100:0/2591719784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7c00c8f40 0x7ff7c01ef160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.589+0000 7ff7c5793640 1 -- 172.21.15.100:0/2591719784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7c0072b40 msgr2=0x7ff7c00c1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.593 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.589+0000 7ff7c5793640 1 --2- 172.21.15.100:0/2591719784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff7c0072b40 0x7ff7c00c1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.589+0000 7ff7c5793640 1 -- 172.21.15.100:0/2591719784 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c01f35c0 con 0x7ff7c0152380 2024-09-15T21:48:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.589+0000 7ff7bffff640 1 --2- 172.21.15.100:0/2591719784 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff7c00c8f40 0x7ff7c01ef160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:48:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.589+0000 7ff7c5793640 1 --2- 172.21.15.100:0/2591719784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c0152380 0x7ff7c014c0b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8007de0 tx=0x7ff7b800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.589+0000 7ff7bd7fa640 1 -- 172.21.15.100:0/2591719784 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8014620 con 0x7ff7c0152380 2024-09-15T21:48:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.590+0000 7ff7bd7fa640 1 -- 172.21.15.100:0/2591719784 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff7b8019070 con 0x7ff7c0152380 2024-09-15T21:48:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.590+0000 7ff7c71f6640 1 -- 172.21.15.100:0/2591719784 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c00cc350 con 0x7ff7c0152380 2024-09-15T21:48:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.590+0000 7ff7bd7fa640 1 -- 172.21.15.100:0/2591719784 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8014a20 con 0x7ff7c0152380 2024-09-15T21:48:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.590+0000 7ff7c71f6640 1 -- 172.21.15.100:0/2591719784 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c01f2ae0 con 0x7ff7c0152380 2024-09-15T21:48:44.594 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.590+0000 7ff7c71f6640 1 -- 172.21.15.100:0/2591719784 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff7c00cc630 con 0x7ff7c0152380 2024-09-15T21:48:44.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.591+0000 7ff7bd7fa640 1 -- 172.21.15.100:0/2591719784 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff7b8014bc0 con 0x7ff7c0152380 2024-09-15T21:48:44.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.591+0000 7ff7bd7fa640 1 --2- 172.21.15.100:0/2591719784 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff7a406c2b0 0x7ff7a406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.595 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.591+0000 7ff7bd7fa640 1 -- 172.21.15.100:0/2591719784 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7ff7b808be50 con 0x7ff7c0152380 2024-09-15T21:48:44.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.592+0000 7ff7c4f92640 1 --2- 172.21.15.100:0/2591719784 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff7a406c2b0 0x7ff7a406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.592+0000 7ff7bd7fa640 1 --2- 172.21.15.100:0/2591719784 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7ff7a4074690 0x7ff7a4076b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.592+0000 7ff7bd7fa640 1 -- 172.21.15.100:0/2591719784 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff7b800ce50 con 0x7ff7a4074690 2024-09-15T21:48:44.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.592+0000 7ff7bd7fa640 1 -- 172.21.15.100:0/2591719784 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7ff7b808e550 con 0x7ff7c0152380 2024-09-15T21:48:44.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.592+0000 7ff7bffff640 1 --2- 172.21.15.100:0/2591719784 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7ff7a4074690 0x7ff7a4076b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.592+0000 7ff7c4f92640 1 --2- 172.21.15.100:0/2591719784 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff7a406c2b0 0x7ff7a406e770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4007fd0 tx=0x7ff7b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:44.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.592+0000 7ff7bffff640 1 --2- 172.21.15.100:0/2591719784 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7ff7a4074690 0x7ff7a4076b30 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:44.596 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.593+0000 7ff7bd7fa640 1 -- 172.21.15.100:0/2591719784 <== osd.1 v2:172.21.15.100:6800/3802679972 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff7b800ce50 con 0x7ff7a4074690 2024-09-15T21:48:44.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.605+0000 7ff7c71f6640 1 -- 172.21.15.100:0/2591719784 --> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff7c0068b40 con 0x7ff7a4074690 2024-09-15T21:48:44.607 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.605+0000 7ff7bd7fa640 1 -- 172.21.15.100:0/2591719784 <== osd.1 v2:172.21.15.100:6800/3802679972 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff7c0068b40 con 0x7ff7a4074690 2024-09-15T21:48:44.607 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:44.607 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:48:44.607 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:44.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.605+0000 7ff7c71f6640 1 -- 172.21.15.100:0/2591719784 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7ff7a4074690 msgr2=0x7ff7a4076b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.605+0000 7ff7c71f6640 1 --2- 172.21.15.100:0/2591719784 >> [v2:172.21.15.100:6800/3802679972,v1:172.21.15.100:6801/3802679972] conn(0x7ff7a4074690 0x7ff7a4076b30 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.606+0000 7ff7c71f6640 1 -- 172.21.15.100:0/2591719784 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff7a406c2b0 msgr2=0x7ff7a406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.608 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.606+0000 7ff7c71f6640 1 --2- 172.21.15.100:0/2591719784 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff7a406c2b0 0x7ff7a406e770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4007fd0 tx=0x7ff7b4009040 comp rx=0 tx=0).stop 2024-09-15T21:48:44.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.606+0000 7ff7c71f6640 1 -- 172.21.15.100:0/2591719784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c0152380 msgr2=0x7ff7c014c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.606+0000 7ff7c71f6640 1 --2- 172.21.15.100:0/2591719784 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff7c0152380 0x7ff7c014c0b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8007de0 tx=0x7ff7b800e4b0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.606+0000 7ff7c5793640 1 -- 172.21.15.100:0/2591719784 reap_dead start 2024-09-15T21:48:44.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.606+0000 7ff7c71f6640 1 -- 172.21.15.100:0/2591719784 shutdown_connections 2024-09-15T21:48:44.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.606+0000 7ff7c71f6640 1 -- 172.21.15.100:0/2591719784 >> 172.21.15.100:0/2591719784 conn(0x7ff7c00c9380 msgr2=0x7ff7c0158ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:44.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.606+0000 7ff7c71f6640 1 -- 172.21.15.100:0/2591719784 shutdown_connections 2024-09-15T21:48:44.609 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.606+0000 7ff7c71f6640 1 -- 172.21.15.100:0/2591719784 wait complete. 2024-09-15T21:48:44.618 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-15T21:48:44.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.715+0000 7f4e42da6640 1 Processor -- start 2024-09-15T21:48:44.717 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.715+0000 7f4e42da6640 1 -- start start 2024-09-15T21:48:44.718 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.716+0000 7f4e42da6640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4e3c153040 0x7f4e3c0c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.716+0000 7f4e42da6640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f4e3c0c4820 0x7f4e3c0c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.716+0000 7f4e42da6640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e3c06f040 con 0x7f4e3c0caec0 2024-09-15T21:48:44.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.716+0000 7f4e42da6640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f4e3c073ca0 con 0x7f4e3c153040 2024-09-15T21:48:44.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.716+0000 7f4e42da6640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f4e3c0c9c40 con 0x7f4e3c0c4820 2024-09-15T21:48:44.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.716+0000 7f4e40b42640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4e3c153040 0x7f4e3c0c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.716+0000 7f4e40b42640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f4e3c153040 0x7f4e3c0c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54558/0 (socket says 172.21.15.100:54558) 2024-09-15T21:48:44.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.716+0000 7f4e40b42640 1 -- 172.21.15.100:0/2716479120 learned_addr learned my addr 172.21.15.100:0/2716479120 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:44.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.716+0000 7f4e3bfff640 1 --2- 172.21.15.100:0/2716479120 >> v2:172.21.15.122:3300/0 conn(0x7f4e3c0c4820 0x7f4e3c0c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.717+0000 7f4e40b42640 1 -- 172.21.15.100:0/2716479120 >> v2:172.21.15.122:3300/0 conn(0x7f4e3c0c4820 msgr2=0x7f4e3c0c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:44.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.717+0000 7f4e40b42640 1 --2- 172.21.15.100:0/2716479120 >> v2:172.21.15.122:3300/0 conn(0x7f4e3c0c4820 0x7f4e3c0c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.717+0000 7f4e40b42640 1 -- 172.21.15.100:0/2716479120 >> v1:172.21.15.122:6789/0 conn(0x7f4e3c0caec0 legacy=0x7f4e3c0720c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.719 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.717+0000 7f4e40b42640 1 -- 172.21.15.100:0/2716479120 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3c0c9f20 con 0x7f4e3c153040 2024-09-15T21:48:44.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.718+0000 7f4e40b42640 1 --2- 172.21.15.100:0/2716479120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c153040 0x7f4e3c0c6fa0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0c7f50 tx=0x7f4e3002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e365f040e56c2f88 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:44.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.718+0000 7f4e3b7fe640 1 -- 172.21.15.100:0/2716479120 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30037070 con 0x7f4e3c153040 2024-09-15T21:48:44.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.718+0000 7f4e3b7fe640 1 -- 172.21.15.100:0/2716479120 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e3002ae50 con 0x7f4e3c153040 2024-09-15T21:48:44.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.718+0000 7f4e3b7fe640 1 -- 172.21.15.100:0/2716479120 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30032860 con 0x7f4e3c153040 2024-09-15T21:48:44.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.718+0000 7f4e42da6640 1 -- 172.21.15.100:0/2716479120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c153040 msgr2=0x7f4e3c0c6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.718+0000 7f4e42da6640 1 --2- 172.21.15.100:0/2716479120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c153040 0x7f4e3c0c6fa0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0c7f50 tx=0x7f4e3002a300 comp rx=0 tx=0).stop 2024-09-15T21:48:44.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.718+0000 7f4e42da6640 1 -- 172.21.15.100:0/2716479120 shutdown_connections 2024-09-15T21:48:44.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.718+0000 7f4e42da6640 1 --2- 172.21.15.100:0/2716479120 >> v2:172.21.15.122:3300/0 conn(0x7f4e3c0c4820 0x7f4e3c0c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.718+0000 7f4e42da6640 1 --2- 172.21.15.100:0/2716479120 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c153040 0x7f4e3c0c6fa0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.718+0000 7f4e42da6640 1 -- 172.21.15.100:0/2716479120 >> 172.21.15.100:0/2716479120 conn(0x7f4e3c0ca390 msgr2=0x7f4e3c0ca770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:44.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.718+0000 7f4e42da6640 1 -- 172.21.15.100:0/2716479120 shutdown_connections 2024-09-15T21:48:44.721 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.718+0000 7f4e42da6640 1 -- 172.21.15.100:0/2716479120 wait complete. 2024-09-15T21:48:44.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.720+0000 7f4e42da6640 1 Processor -- start 2024-09-15T21:48:44.723 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.720+0000 7f4e42da6640 1 -- start start 2024-09-15T21:48:44.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.720+0000 7f4e42da6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e3c0c4820 0x7f4e3c1eeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.720+0000 7f4e42da6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c0caec0 0x7f4e3c1e11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.720+0000 7f4e42da6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e3c153040 0x7f4e3c14a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.720+0000 7f4e42da6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c073930 con 0x7f4e3c0caec0 2024-09-15T21:48:44.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.720+0000 7f4e42da6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c073650 con 0x7f4e3c153040 2024-09-15T21:48:44.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.720+0000 7f4e42da6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f4e3c153420 con 0x7f4e3c0c4820 2024-09-15T21:48:44.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.721+0000 7f4e3bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c0caec0 0x7f4e3c1e11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.721+0000 7f4e3bfff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c0caec0 0x7f4e3c1e11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54560/0 (socket says 172.21.15.100:54560) 2024-09-15T21:48:44.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.721+0000 7f4e3bfff640 1 -- 172.21.15.100:0/841860378 learned_addr learned my addr 172.21.15.100:0/841860378 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:44.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.721+0000 7f4e41343640 1 --2- 172.21.15.100:0/841860378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e3c153040 0x7f4e3c14a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.721+0000 7f4e40b42640 1 --2- 172.21.15.100:0/841860378 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e3c0c4820 0x7f4e3c1eeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.724 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.721+0000 7f4e3bfff640 1 -- 172.21.15.100:0/841860378 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e3c0c4820 msgr2=0x7f4e3c1eeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.721+0000 7f4e3bfff640 1 --2- 172.21.15.100:0/841860378 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e3c0c4820 0x7f4e3c1eeb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.721+0000 7f4e3bfff640 1 -- 172.21.15.100:0/841860378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e3c153040 msgr2=0x7f4e3c14a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.721+0000 7f4e3bfff640 1 --2- 172.21.15.100:0/841860378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e3c153040 0x7f4e3c14a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.721+0000 7f4e3bfff640 1 -- 172.21.15.100:0/841860378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3c151590 con 0x7f4e3c0caec0 2024-09-15T21:48:44.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.721+0000 7f4e3bfff640 1 --2- 172.21.15.100:0/841860378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c0caec0 0x7f4e3c1e11d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800e9e0 tx=0x7f4e2800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:44.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.721+0000 7f4e397fa640 1 -- 172.21.15.100:0/841860378 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e28017070 con 0x7f4e3c0caec0 2024-09-15T21:48:44.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.721+0000 7f4e397fa640 1 -- 172.21.15.100:0/841860378 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e2800c970 con 0x7f4e3c0caec0 2024-09-15T21:48:44.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.721+0000 7f4e42da6640 1 -- 172.21.15.100:0/841860378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e3c1e32b0 con 0x7f4e3c0caec0 2024-09-15T21:48:44.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.721+0000 7f4e397fa640 1 -- 172.21.15.100:0/841860378 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2800cd60 con 0x7f4e3c0caec0 2024-09-15T21:48:44.725 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.721+0000 7f4e42da6640 1 -- 172.21.15.100:0/841860378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e3c1e3830 con 0x7f4e3c0caec0 2024-09-15T21:48:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.722+0000 7f4e42da6640 1 -- 172.21.15.100:0/841860378 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e3c06dcd0 con 0x7f4e3c0caec0 2024-09-15T21:48:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.723+0000 7f4e397fa640 1 -- 172.21.15.100:0/841860378 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f4e2801a070 con 0x7f4e3c0caec0 2024-09-15T21:48:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.723+0000 7f4e397fa640 1 --2- 172.21.15.100:0/841860378 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4e2006c2b0 0x7f4e2006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.723+0000 7f4e397fa640 1 -- 172.21.15.100:0/841860378 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f4e2808b4b0 con 0x7f4e3c0caec0 2024-09-15T21:48:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.723+0000 7f4e40b42640 1 --2- 172.21.15.100:0/841860378 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4e2006c2b0 0x7f4e2006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:44.726 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.724+0000 7f4e40b42640 1 --2- 172.21.15.100:0/841860378 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4e2006c2b0 0x7f4e2006e770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4e30034000 tx=0x7f4e3002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:44.729 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.728+0000 7f4e397fa640 1 -- 172.21.15.100:0/841860378 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f4e280594a0 con 0x7f4e3c0caec0 2024-09-15T21:48:44.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.894+0000 7f4e42da6640 1 -- 172.21.15.100:0/841860378 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f4e3c151030 con 0x7f4e2006c2b0 2024-09-15T21:48:44.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.894+0000 7f4e397fa640 1 -- 172.21.15.100:0/841860378 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4e3c151030 con 0x7f4e2006c2b0 2024-09-15T21:48:44.896 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 1 to deep-scrub 2024-09-15T21:48:44.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.896+0000 7f4e42da6640 1 -- 172.21.15.100:0/841860378 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4e2006c2b0 msgr2=0x7f4e2006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.898 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.896+0000 7f4e42da6640 1 --2- 172.21.15.100:0/841860378 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4e2006c2b0 0x7f4e2006e770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4e30034000 tx=0x7f4e3002e000 comp rx=0 tx=0).stop 2024-09-15T21:48:44.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.896+0000 7f4e42da6640 1 -- 172.21.15.100:0/841860378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c0caec0 msgr2=0x7f4e3c1e11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:44.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.896+0000 7f4e42da6640 1 --2- 172.21.15.100:0/841860378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c0caec0 0x7f4e3c1e11d0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800e9e0 tx=0x7f4e2800eea0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.896+0000 7f4e42da6640 1 -- 172.21.15.100:0/841860378 shutdown_connections 2024-09-15T21:48:44.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.896+0000 7f4e42da6640 1 --2- 172.21.15.100:0/841860378 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f4e2006c2b0 0x7f4e2006e770 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.896+0000 7f4e42da6640 1 --2- 172.21.15.100:0/841860378 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e3c153040 0x7f4e3c14a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.896+0000 7f4e42da6640 1 --2- 172.21.15.100:0/841860378 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4e3c0caec0 0x7f4e3c1e11d0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.896+0000 7f4e42da6640 1 --2- 172.21.15.100:0/841860378 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f4e3c0c4820 0x7f4e3c1eeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:44.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.896+0000 7f4e42da6640 1 -- 172.21.15.100:0/841860378 >> 172.21.15.100:0/841860378 conn(0x7f4e3c0ca390 msgr2=0x7f4e3c0c6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:44.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.896+0000 7f4e42da6640 1 -- 172.21.15.100:0/841860378 shutdown_connections 2024-09-15T21:48:44.899 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:44.896+0000 7f4e42da6640 1 -- 172.21.15.100:0/841860378 wait complete. 2024-09-15T21:48:44.909 INFO:tasks.ceph:Scrubbing osd.2 2024-09-15T21:48:44.909 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:48:45.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.010+0000 7ff1544a6640 1 Processor -- start 2024-09-15T21:48:45.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.010+0000 7ff1544a6640 1 -- start start 2024-09-15T21:48:45.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff1544a6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff14c0cc2f0 0x7ff14c15b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff1544a6640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff14c0cd190 con 0x7ff14c15b7e0 2024-09-15T21:48:45.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff1544a6640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff14c15e9b0 con 0x7ff14c155500 2024-09-15T21:48:45.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff1544a6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff14c15f770 con 0x7ff14c0cc2f0 2024-09-15T21:48:45.015 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff152a43640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7ff14c15b7e0 0x7ff14c0d0b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42198/0 (socket says 172.21.15.100:42198) 2024-09-15T21:48:45.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff152a43640 1 -- 172.21.15.100:0/2885003033 learned_addr learned my addr 172.21.15.100:0/2885003033 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:45.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff151a41640 1 --2- 172.21.15.100:0/2885003033 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff14c0cc2f0 0x7ff14c15b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff151240640 1 -- 172.21.15.100:0/2885003033 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1704001645 0 0) 0x7ff14c0cd190 con 0x7ff14c15b7e0 2024-09-15T21:48:45.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff151240640 1 -- 172.21.15.100:0/2885003033 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff138003520 con 0x7ff14c15b7e0 2024-09-15T21:48:45.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff151240640 1 -- 172.21.15.100:0/2885003033 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4123701234 0 0) 0x7ff138003520 con 0x7ff14c15b7e0 2024-09-15T21:48:45.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff151240640 1 -- 172.21.15.100:0/2885003033 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff14c0cc2f0 msgr2=0x7ff14c15b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:45.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff151240640 1 --2- 172.21.15.100:0/2885003033 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff14c0cc2f0 0x7ff14c15b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff151240640 1 -- 172.21.15.100:0/2885003033 >> v1:172.21.15.122:6789/0 conn(0x7ff14c155500 legacy=0x7ff14c0cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff151240640 1 -- 172.21.15.100:0/2885003033 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14c15ec90 con 0x7ff14c15b7e0 2024-09-15T21:48:45.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff151240640 1 -- 172.21.15.100:0/2885003033 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7ff1440019f0 con 0x7ff14c15b7e0 2024-09-15T21:48:45.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff151240640 1 -- 172.21.15.100:0/2885003033 >> v1:172.21.15.100:6789/0 conn(0x7ff14c15b7e0 legacy=0x7ff14c0d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff151240640 1 --2- 172.21.15.100:0/2885003033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff1380039e0 0x7ff138003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff151240640 1 --2- 172.21.15.100:0/2885003033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff138004800 0x7ff138004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff151240640 1 --2- 172.21.15.100:0/2885003033 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff14c0cc2f0 0x7ff1380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff151240640 1 -- 172.21.15.100:0/2885003033 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff13800bb00 con 0x7ff138004800 2024-09-15T21:48:45.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff151240640 1 -- 172.21.15.100:0/2885003033 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff13800bcb0 con 0x7ff1380039e0 2024-09-15T21:48:45.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.011+0000 7ff151240640 1 -- 172.21.15.100:0/2885003033 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff13800beb0 con 0x7ff14c0cc2f0 2024-09-15T21:48:45.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.012+0000 7ff151a41640 1 --2- 172.21.15.100:0/2885003033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff138004800 0x7ff138004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.012+0000 7ff152242640 1 --2- 172.21.15.100:0/2885003033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff1380039e0 0x7ff138003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.012+0000 7ff152a43640 1 --2- 172.21.15.100:0/2885003033 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff14c0cc2f0 0x7ff1380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.012+0000 7ff151a41640 1 -- 172.21.15.100:0/2885003033 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff14c0cc2f0 msgr2=0x7ff1380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:45.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.012+0000 7ff151a41640 1 --2- 172.21.15.100:0/2885003033 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff14c0cc2f0 0x7ff1380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.012+0000 7ff151a41640 1 -- 172.21.15.100:0/2885003033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff1380039e0 msgr2=0x7ff138003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:45.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.012+0000 7ff151a41640 1 --2- 172.21.15.100:0/2885003033 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff1380039e0 0x7ff138003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.012+0000 7ff151a41640 1 -- 172.21.15.100:0/2885003033 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff138003520 con 0x7ff138004800 2024-09-15T21:48:45.017 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.013+0000 7ff151a41640 1 --2- 172.21.15.100:0/2885003033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff138004800 0x7ff138004c80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff14c158ed0 tx=0x7ff138005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b179d03d22c82bde server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:45.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.013+0000 7ff151240640 1 -- 172.21.15.100:0/2885003033 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c036070 con 0x7ff138004800 2024-09-15T21:48:45.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.013+0000 7ff151240640 1 -- 172.21.15.100:0/2885003033 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff13c031400 con 0x7ff138004800 2024-09-15T21:48:45.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.013+0000 7ff151240640 1 -- 172.21.15.100:0/2885003033 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c009690 con 0x7ff138004800 2024-09-15T21:48:45.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.013+0000 7ff1544a6640 1 -- 172.21.15.100:0/2885003033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff138004800 msgr2=0x7ff138004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.013+0000 7ff1544a6640 1 --2- 172.21.15.100:0/2885003033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff138004800 0x7ff138004c80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff14c158ed0 tx=0x7ff138005830 comp rx=0 tx=0).stop 2024-09-15T21:48:45.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.013+0000 7ff152a43640 1 -- 172.21.15.100:0/2885003033 reap_dead start 2024-09-15T21:48:45.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.013+0000 7ff1544a6640 1 -- 172.21.15.100:0/2885003033 shutdown_connections 2024-09-15T21:48:45.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.013+0000 7ff1544a6640 1 -- 172.21.15.100:0/2885003033 >> 172.21.15.100:0/2885003033 conn(0x7ff14c067980 msgr2=0x7ff14c067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:45.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.013+0000 7ff1544a6640 1 -- 172.21.15.100:0/2885003033 shutdown_connections 2024-09-15T21:48:45.018 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.013+0000 7ff1544a6640 1 -- 172.21.15.100:0/2885003033 wait complete. 2024-09-15T21:48:45.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.014+0000 7ff1544a6640 1 Processor -- start 2024-09-15T21:48:45.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.015+0000 7ff1544a6640 1 -- start start 2024-09-15T21:48:45.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.015+0000 7ff1544a6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff138004800 0x7ff14c152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.015+0000 7ff1544a6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff14c150230 0x7ff14c155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.015+0000 7ff1544a6640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff14c153030 0x7ff14c159df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.015+0000 7ff1544a6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff14c0b6c50 con 0x7ff14c150230 2024-09-15T21:48:45.022 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.015+0000 7ff1544a6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff14c15f2e0 con 0x7ff138004800 2024-09-15T21:48:45.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.015+0000 7ff1544a6640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff14c1558e0 con 0x7ff14c153030 2024-09-15T21:48:45.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.015+0000 7ff151a41640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff14c150230 0x7ff14c155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.015+0000 7ff151a41640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff14c150230 0x7ff14c155e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54588/0 (socket says 172.21.15.100:54588) 2024-09-15T21:48:45.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.015+0000 7ff151a41640 1 -- 172.21.15.100:0/2054496888 learned_addr learned my addr 172.21.15.100:0/2054496888 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:45.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.015+0000 7ff152242640 1 --2- 172.21.15.100:0/2054496888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff138004800 0x7ff14c152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.015+0000 7ff152a43640 1 --2- 172.21.15.100:0/2054496888 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff14c153030 0x7ff14c159df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.016+0000 7ff151a41640 1 -- 172.21.15.100:0/2054496888 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff14c153030 msgr2=0x7ff14c159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.016+0000 7ff151a41640 1 --2- 172.21.15.100:0/2054496888 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff14c153030 0x7ff14c159df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.016+0000 7ff151a41640 1 -- 172.21.15.100:0/2054496888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff138004800 msgr2=0x7ff14c152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.016+0000 7ff151a41640 1 --2- 172.21.15.100:0/2054496888 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff138004800 0x7ff14c152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.016+0000 7ff151a41640 1 -- 172.21.15.100:0/2054496888 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff14c1f0a90 con 0x7ff14c150230 2024-09-15T21:48:45.023 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.016+0000 7ff151a41640 1 --2- 172.21.15.100:0/2054496888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff14c150230 0x7ff14c155e80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff13c02ab40 tx=0x7ff13c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:45.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.016+0000 7ff142ffd640 1 -- 172.21.15.100:0/2054496888 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c041020 con 0x7ff14c150230 2024-09-15T21:48:45.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.016+0000 7ff142ffd640 1 -- 172.21.15.100:0/2054496888 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff13c03c470 con 0x7ff14c150230 2024-09-15T21:48:45.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.016+0000 7ff1544a6640 1 -- 172.21.15.100:0/2054496888 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff14c150720 con 0x7ff14c150230 2024-09-15T21:48:45.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.016+0000 7ff142ffd640 1 -- 172.21.15.100:0/2054496888 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13c036040 con 0x7ff14c150230 2024-09-15T21:48:45.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.016+0000 7ff1544a6640 1 -- 172.21.15.100:0/2054496888 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff14c1f0fe0 con 0x7ff14c150230 2024-09-15T21:48:45.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.017+0000 7ff1544a6640 1 -- 172.21.15.100:0/2054496888 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff14c0ccdc0 con 0x7ff14c150230 2024-09-15T21:48:45.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.017+0000 7ff142ffd640 1 -- 172.21.15.100:0/2054496888 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff13c044020 con 0x7ff14c150230 2024-09-15T21:48:45.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.018+0000 7ff142ffd640 1 --2- 172.21.15.100:0/2054496888 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff13006c2b0 0x7ff13006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.018+0000 7ff142ffd640 1 -- 172.21.15.100:0/2054496888 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7ff13c0ac0f0 con 0x7ff14c150230 2024-09-15T21:48:45.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.018+0000 7ff152242640 1 --2- 172.21.15.100:0/2054496888 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff13006c2b0 0x7ff13006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.018+0000 7ff142ffd640 1 --2- 172.21.15.100:0/2054496888 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7ff130074630 0x7ff130076ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.018+0000 7ff142ffd640 1 -- 172.21.15.100:0/2054496888 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff13c033e90 con 0x7ff130074630 2024-09-15T21:48:45.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.018+0000 7ff142ffd640 1 -- 172.21.15.100:0/2054496888 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7ff13c0b3050 con 0x7ff14c150230 2024-09-15T21:48:45.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.018+0000 7ff152a43640 1 --2- 172.21.15.100:0/2054496888 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7ff130074630 0x7ff130076ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.024 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.018+0000 7ff152242640 1 --2- 172.21.15.100:0/2054496888 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff13006c2b0 0x7ff13006e770 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff1480017e0 tx=0x7ff148009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:45.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.018+0000 7ff152a43640 1 --2- 172.21.15.100:0/2054496888 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7ff130074630 0x7ff130076ad0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:45.025 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.019+0000 7ff142ffd640 1 -- 172.21.15.100:0/2054496888 <== osd.2 v2:172.21.15.100:6820/2316787974 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff13c033e90 con 0x7ff130074630 2024-09-15T21:48:45.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.031+0000 7ff1544a6640 1 -- 172.21.15.100:0/2054496888 --> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff14c0cf1f0 con 0x7ff130074630 2024-09-15T21:48:45.034 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.032+0000 7ff142ffd640 1 -- 172.21.15.100:0/2054496888 <== osd.2 v2:172.21.15.100:6820/2316787974 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff14c0cf1f0 con 0x7ff130074630 2024-09-15T21:48:45.034 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:45.034 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:48:45.034 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:45.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.032+0000 7ff1544a6640 1 -- 172.21.15.100:0/2054496888 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7ff130074630 msgr2=0x7ff130076ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.035 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.032+0000 7ff1544a6640 1 --2- 172.21.15.100:0/2054496888 >> [v2:172.21.15.100:6820/2316787974,v1:172.21.15.100:6823/2316787974] conn(0x7ff130074630 0x7ff130076ad0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.032+0000 7ff1544a6640 1 -- 172.21.15.100:0/2054496888 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff13006c2b0 msgr2=0x7ff13006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.032+0000 7ff1544a6640 1 --2- 172.21.15.100:0/2054496888 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff13006c2b0 0x7ff13006e770 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff1480017e0 tx=0x7ff148009040 comp rx=0 tx=0).stop 2024-09-15T21:48:45.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.032+0000 7ff1544a6640 1 -- 172.21.15.100:0/2054496888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff14c150230 msgr2=0x7ff14c155e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.032+0000 7ff1544a6640 1 --2- 172.21.15.100:0/2054496888 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff14c150230 0x7ff14c155e80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff13c02ab40 tx=0x7ff13c004730 comp rx=0 tx=0).stop 2024-09-15T21:48:45.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.032+0000 7ff152a43640 1 -- 172.21.15.100:0/2054496888 reap_dead start 2024-09-15T21:48:45.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.032+0000 7ff1544a6640 1 -- 172.21.15.100:0/2054496888 shutdown_connections 2024-09-15T21:48:45.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.032+0000 7ff1544a6640 1 -- 172.21.15.100:0/2054496888 >> 172.21.15.100:0/2054496888 conn(0x7ff14c067980 msgr2=0x7ff14c0d0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:45.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.032+0000 7ff1544a6640 1 -- 172.21.15.100:0/2054496888 shutdown_connections 2024-09-15T21:48:45.036 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.032+0000 7ff1544a6640 1 -- 172.21.15.100:0/2054496888 wait complete. 2024-09-15T21:48:45.044 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-15T21:48:45.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.141+0000 7f2f1e023640 1 Processor -- start 2024-09-15T21:48:45.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.142+0000 7f2f1e023640 1 -- start start 2024-09-15T21:48:45.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.142+0000 7f2f1e023640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f2f181552e0 0x7f2f1806d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.142+0000 7f2f1e023640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f1806ec60 con 0x7f2f1815c5e0 2024-09-15T21:48:45.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.142+0000 7f2f1e023640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f1815af40 con 0x7f2f180cee00 2024-09-15T21:48:45.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.142+0000 7f2f1e023640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f2f1815b200 con 0x7f2f181552e0 2024-09-15T21:48:45.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.142+0000 7f2f17fff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f2f1815c5e0 0x7f2f1815c9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42214/0 (socket says 172.21.15.100:42214) 2024-09-15T21:48:45.151 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.142+0000 7f2f17fff640 1 -- 172.21.15.100:0/2545363582 learned_addr learned my addr 172.21.15.100:0/2545363582 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:45.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f177fe640 1 --2- 172.21.15.100:0/2545363582 >> v2:172.21.15.122:3300/0 conn(0x7f2f181552e0 0x7f2f1806d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f167fc640 1 -- 172.21.15.100:0/2545363582 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3528935281 0 0) 0x7f2f10001110 con 0x7f2f1815c5e0 2024-09-15T21:48:45.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f167fc640 1 -- 172.21.15.100:0/2545363582 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f00003520 con 0x7f2f1815c5e0 2024-09-15T21:48:45.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f167fc640 1 -- 172.21.15.100:0/2545363582 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 521243022 0 0) 0x7f2f00003520 con 0x7f2f1815c5e0 2024-09-15T21:48:45.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f167fc640 1 -- 172.21.15.100:0/2545363582 >> v2:172.21.15.122:3300/0 conn(0x7f2f181552e0 msgr2=0x7f2f1806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:45.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f167fc640 1 --2- 172.21.15.100:0/2545363582 >> v2:172.21.15.122:3300/0 conn(0x7f2f181552e0 0x7f2f1806d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f167fc640 1 -- 172.21.15.100:0/2545363582 >> v1:172.21.15.122:6789/0 conn(0x7f2f180cee00 legacy=0x7f2f180730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f167fc640 1 -- 172.21.15.100:0/2545363582 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f1815b4e0 con 0x7f2f1815c5e0 2024-09-15T21:48:45.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f167fc640 1 -- 172.21.15.100:0/2545363582 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7f2f10001840 con 0x7f2f1815c5e0 2024-09-15T21:48:45.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f167fc640 1 -- 172.21.15.100:0/2545363582 >> v1:172.21.15.100:6789/0 conn(0x7f2f1815c5e0 legacy=0x7f2f1815c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f167fc640 1 --2- 172.21.15.100:0/2545363582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f000039b0 0x7f2f00003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.152 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f167fc640 1 --2- 172.21.15.100:0/2545363582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f000043b0 0x7f2f00004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f167fc640 1 --2- 172.21.15.100:0/2545363582 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f00005080 0x7f2f0000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f167fc640 1 -- 172.21.15.100:0/2545363582 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1815b200 con 0x7f2f000039b0 2024-09-15T21:48:45.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f167fc640 1 -- 172.21.15.100:0/2545363582 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2f0000bfe0 con 0x7f2f000043b0 2024-09-15T21:48:45.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f167fc640 1 -- 172.21.15.100:0/2545363582 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2f0000c1e0 con 0x7f2f00005080 2024-09-15T21:48:45.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f177fe640 1 --2- 172.21.15.100:0/2545363582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f000039b0 0x7f2f00003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.153 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f16ffd640 1 --2- 172.21.15.100:0/2545363582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f000043b0 0x7f2f00004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f17fff640 1 --2- 172.21.15.100:0/2545363582 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f00005080 0x7f2f0000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f177fe640 1 -- 172.21.15.100:0/2545363582 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f00005080 msgr2=0x7f2f0000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:45.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f177fe640 1 --2- 172.21.15.100:0/2545363582 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f00005080 0x7f2f0000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f177fe640 1 -- 172.21.15.100:0/2545363582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f000043b0 msgr2=0x7f2f00004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:45.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f177fe640 1 --2- 172.21.15.100:0/2545363582 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f000043b0 0x7f2f00004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f177fe640 1 -- 172.21.15.100:0/2545363582 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f00003520 con 0x7f2f000039b0 2024-09-15T21:48:45.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.143+0000 7f2f17fff640 1 -- 172.21.15.100:0/2545363582 reap_dead start 2024-09-15T21:48:45.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.144+0000 7f2f177fe640 1 --2- 172.21.15.100:0/2545363582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f000039b0 0x7f2f00003d90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009fc0 tx=0x7f2f00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559a785aca3eb9b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:45.154 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.144+0000 7f2f167fc640 1 -- 172.21.15.100:0/2545363582 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08009c10 con 0x7f2f000039b0 2024-09-15T21:48:45.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.144+0000 7f2f167fc640 1 -- 172.21.15.100:0/2545363582 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f08036070 con 0x7f2f000039b0 2024-09-15T21:48:45.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.144+0000 7f2f167fc640 1 -- 172.21.15.100:0/2545363582 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08031880 con 0x7f2f000039b0 2024-09-15T21:48:45.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.145+0000 7f2f1e023640 1 -- 172.21.15.100:0/2545363582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f000039b0 msgr2=0x7f2f00003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.145+0000 7f2f1e023640 1 --2- 172.21.15.100:0/2545363582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f000039b0 0x7f2f00003d90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009fc0 tx=0x7f2f00005830 comp rx=0 tx=0).stop 2024-09-15T21:48:45.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.145+0000 7f2f1e023640 1 -- 172.21.15.100:0/2545363582 shutdown_connections 2024-09-15T21:48:45.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.145+0000 7f2f1e023640 1 --2- 172.21.15.100:0/2545363582 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f000039b0 0x7f2f00003d90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.145+0000 7f2f1e023640 1 -- 172.21.15.100:0/2545363582 >> 172.21.15.100:0/2545363582 conn(0x7f2f18070ca0 msgr2=0x7f2f18071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:45.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.145+0000 7f2f1e023640 1 -- 172.21.15.100:0/2545363582 shutdown_connections 2024-09-15T21:48:45.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.145+0000 7f2f1e023640 1 -- 172.21.15.100:0/2545363582 wait complete. 2024-09-15T21:48:45.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.146+0000 7f2f1e023640 1 Processor -- start 2024-09-15T21:48:45.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.146+0000 7f2f1e023640 1 -- start start 2024-09-15T21:48:45.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2f1e023640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f000039b0 0x7f2f181f1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2f1e023640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f1815ace0 0x7f2f180cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2f1e023640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f18159d00 0x7f2f181492e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2f1e023640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2f180c91a0 con 0x7f2f18159d00 2024-09-15T21:48:45.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2f1e023640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1815baa0 con 0x7f2f000039b0 2024-09-15T21:48:45.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2f1e023640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2f180cf250 con 0x7f2f1815ace0 2024-09-15T21:48:45.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2f17fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f18159d00 0x7f2f181492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2f17fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f18159d00 0x7f2f181492e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54618/0 (socket says 172.21.15.100:54618) 2024-09-15T21:48:45.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2f17fff640 1 -- 172.21.15.100:0/459127520 learned_addr learned my addr 172.21.15.100:0/459127520 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:45.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2f177fe640 1 --2- 172.21.15.100:0/459127520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f000039b0 0x7f2f181f1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2f16ffd640 1 --2- 172.21.15.100:0/459127520 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f1815ace0 0x7f2f180cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2f17fff640 1 -- 172.21.15.100:0/459127520 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f1815ace0 msgr2=0x7f2f180cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2f17fff640 1 --2- 172.21.15.100:0/459127520 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f1815ace0 0x7f2f180cabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2f17fff640 1 -- 172.21.15.100:0/459127520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f000039b0 msgr2=0x7f2f181f1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2f17fff640 1 --2- 172.21.15.100:0/459127520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f000039b0 0x7f2f181f1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2f17fff640 1 -- 172.21.15.100:0/459127520 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f180cc700 con 0x7f2f18159d00 2024-09-15T21:48:45.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2f17fff640 1 --2- 172.21.15.100:0/459127520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f18159d00 0x7f2f181492e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2f1815e490 tx=0x7f2f1000e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:45.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2efffff640 1 -- 172.21.15.100:0/459127520 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10019070 con 0x7f2f18159d00 2024-09-15T21:48:45.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2efffff640 1 -- 172.21.15.100:0/459127520 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f100105d0 con 0x7f2f18159d00 2024-09-15T21:48:45.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2f1e023640 1 -- 172.21.15.100:0/459127520 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f180ca310 con 0x7f2f18159d00 2024-09-15T21:48:45.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.147+0000 7f2efffff640 1 -- 172.21.15.100:0/459127520 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f100109a0 con 0x7f2f18159d00 2024-09-15T21:48:45.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.148+0000 7f2f1e023640 1 -- 172.21.15.100:0/459127520 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f18158f40 con 0x7f2f18159d00 2024-09-15T21:48:45.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.148+0000 7f2f1e023640 1 -- 172.21.15.100:0/459127520 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f180c5480 con 0x7f2f18159d00 2024-09-15T21:48:45.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.149+0000 7f2efffff640 1 -- 172.21.15.100:0/459127520 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2f1001c070 con 0x7f2f18159d00 2024-09-15T21:48:45.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.149+0000 7f2efffff640 1 --2- 172.21.15.100:0/459127520 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2f0406c2b0 0x7f2f0406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.150+0000 7f2efffff640 1 -- 172.21.15.100:0/459127520 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f2f1008a990 con 0x7f2f18159d00 2024-09-15T21:48:45.158 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.150+0000 7f2f177fe640 1 --2- 172.21.15.100:0/459127520 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2f0406c2b0 0x7f2f0406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.150+0000 7f2f177fe640 1 --2- 172.21.15.100:0/459127520 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2f0406c2b0 0x7f2f0406e770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2f08035000 tx=0x7f2f0802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:45.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.154+0000 7f2efffff640 1 -- 172.21.15.100:0/459127520 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2f180c5480 con 0x7f2f18159d00 2024-09-15T21:48:45.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.319+0000 7f2f1e023640 1 -- 172.21.15.100:0/459127520 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f2f1806d700 con 0x7f2f0406c2b0 2024-09-15T21:48:45.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.319+0000 7f2efffff640 1 -- 172.21.15.100:0/459127520 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2f1806d700 con 0x7f2f0406c2b0 2024-09-15T21:48:45.321 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 2 to deep-scrub 2024-09-15T21:48:45.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.321+0000 7f2f1e023640 1 -- 172.21.15.100:0/459127520 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2f0406c2b0 msgr2=0x7f2f0406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.321+0000 7f2f1e023640 1 --2- 172.21.15.100:0/459127520 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2f0406c2b0 0x7f2f0406e770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2f08035000 tx=0x7f2f0802f000 comp rx=0 tx=0).stop 2024-09-15T21:48:45.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.321+0000 7f2f1e023640 1 -- 172.21.15.100:0/459127520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f18159d00 msgr2=0x7f2f181492e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.321+0000 7f2f1e023640 1 --2- 172.21.15.100:0/459127520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f18159d00 0x7f2f181492e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2f1815e490 tx=0x7f2f1000e960 comp rx=0 tx=0).stop 2024-09-15T21:48:45.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.321+0000 7f2f1e023640 1 -- 172.21.15.100:0/459127520 shutdown_connections 2024-09-15T21:48:45.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.321+0000 7f2f1e023640 1 --2- 172.21.15.100:0/459127520 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2f0406c2b0 0x7f2f0406e770 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.321+0000 7f2f1e023640 1 --2- 172.21.15.100:0/459127520 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f18159d00 0x7f2f181492e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.321+0000 7f2f1e023640 1 --2- 172.21.15.100:0/459127520 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f1815ace0 0x7f2f180cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.321+0000 7f2f1e023640 1 --2- 172.21.15.100:0/459127520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f000039b0 0x7f2f181f1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.321+0000 7f2f1e023640 1 -- 172.21.15.100:0/459127520 >> 172.21.15.100:0/459127520 conn(0x7f2f18070ca0 msgr2=0x7f2f1806ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:45.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.321+0000 7f2f1e023640 1 -- 172.21.15.100:0/459127520 shutdown_connections 2024-09-15T21:48:45.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.321+0000 7f2f1e023640 1 -- 172.21.15.100:0/459127520 wait complete. 2024-09-15T21:48:45.335 INFO:tasks.ceph:Scrubbing osd.3 2024-09-15T21:48:45.335 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:48:45.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.438+0000 7f5a50b2a640 1 Processor -- start 2024-09-15T21:48:45.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.438+0000 7f5a50b2a640 1 -- start start 2024-09-15T21:48:45.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.439+0000 7f5a50b2a640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f5a4c152380 0x7f5a4c15c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.439+0000 7f5a50b2a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5a4c0c8f40 0x7f5a4c0c8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.439+0000 7f5a50b2a640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a4c072210 con 0x7f5a4c072b40 2024-09-15T21:48:45.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.439+0000 7f5a50b2a640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f5a4c071fc0 con 0x7f5a4c152380 2024-09-15T21:48:45.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.439+0000 7f5a50b2a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5a4c158090 con 0x7f5a4c0c8f40 2024-09-15T21:48:45.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.439+0000 7f5a4ad9d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5a4c0c8f40 0x7f5a4c0c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.439+0000 7f5a4a59c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f5a4c152380 0x7f5a4c15c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.439+0000 7f5a4ad9d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5a4c0c8f40 0x7f5a4c0c8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:56012/0 (socket says 172.21.15.100:56012) 2024-09-15T21:48:45.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.439+0000 7f5a4a59c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f5a4c152380 0x7f5a4c15c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:43338/0 (socket says 172.21.15.100:43338) 2024-09-15T21:48:45.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.439+0000 7f5a4ad9d640 1 -- 172.21.15.100:0/718323174 learned_addr learned my addr 172.21.15.100:0/718323174 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:45.445 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.439+0000 7f5a4a59c640 1 -- 172.21.15.100:0/718323174 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5a4c0c8f40 msgr2=0x7f5a4c0c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:45.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.439+0000 7f5a4a59c640 1 --2- 172.21.15.100:0/718323174 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5a4c0c8f40 0x7f5a4c0c8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.439+0000 7f5a4a59c640 1 -- 172.21.15.100:0/718323174 >> v1:172.21.15.122:6789/0 conn(0x7f5a4c072b40 legacy=0x7f5a4c158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.439+0000 7f5a4a59c640 1 -- 172.21.15.100:0/718323174 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4c0724f0 con 0x7f5a4c152380 2024-09-15T21:48:45.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.439+0000 7f5a4959a640 1 -- 172.21.15.100:0/718323174 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2310613446 0 0) 0x7f5a4c072210 con 0x7f5a4c072b40 2024-09-15T21:48:45.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.439+0000 7f5a4ad9d640 1 --2- 172.21.15.100:0/718323174 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5a4c0c8f40 0x7f5a4c0c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-15T21:48:45.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.440+0000 7f5a4a59c640 1 --2- 172.21.15.100:0/718323174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a4c152380 0x7f5a4c15c090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c072140 tx=0x7f5a3c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6461a0c4797eb583 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:45.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.440+0000 7f5a4959a640 1 -- 172.21.15.100:0/718323174 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c037070 con 0x7f5a4c152380 2024-09-15T21:48:45.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.440+0000 7f5a4959a640 1 -- 172.21.15.100:0/718323174 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a3c032440 con 0x7f5a4c152380 2024-09-15T21:48:45.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.440+0000 7f5a4959a640 1 -- 172.21.15.100:0/718323174 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c032810 con 0x7f5a4c152380 2024-09-15T21:48:45.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.441+0000 7f5a50b2a640 1 -- 172.21.15.100:0/718323174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a4c152380 msgr2=0x7f5a4c15c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.446 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.441+0000 7f5a50b2a640 1 --2- 172.21.15.100:0/718323174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a4c152380 0x7f5a4c15c090 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c072140 tx=0x7f5a3c02a180 comp rx=0 tx=0).stop 2024-09-15T21:48:45.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.441+0000 7f5a50b2a640 1 -- 172.21.15.100:0/718323174 shutdown_connections 2024-09-15T21:48:45.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.441+0000 7f5a50b2a640 1 --2- 172.21.15.100:0/718323174 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5a4c0c8f40 0x7f5a4c0c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.441+0000 7f5a50b2a640 1 --2- 172.21.15.100:0/718323174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a4c152380 0x7f5a4c15c090 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.441+0000 7f5a50b2a640 1 -- 172.21.15.100:0/718323174 >> 172.21.15.100:0/718323174 conn(0x7f5a4c0c9380 msgr2=0x7f5a4c0c9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:45.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.441+0000 7f5a50b2a640 1 -- 172.21.15.100:0/718323174 shutdown_connections 2024-09-15T21:48:45.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.441+0000 7f5a50b2a640 1 -- 172.21.15.100:0/718323174 wait complete. 2024-09-15T21:48:45.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.442+0000 7f5a50b2a640 1 Processor -- start 2024-09-15T21:48:45.447 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.442+0000 7f5a50b2a640 1 -- start start 2024-09-15T21:48:45.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.443+0000 7f5a50b2a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a4c072b40 0x7f5a4c0c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.443+0000 7f5a50b2a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a4c0c8f40 0x7f5a4c06e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.443+0000 7f5a50b2a640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5a4c152380 0x7f5a4c14efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.443+0000 7f5a50b2a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c152760 con 0x7f5a4c072b40 2024-09-15T21:48:45.450 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.443+0000 7f5a50b2a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5a4c158650 con 0x7f5a4c0c8f40 2024-09-15T21:48:45.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.443+0000 7f5a50b2a640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5a4c06de80 con 0x7f5a4c152380 2024-09-15T21:48:45.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.443+0000 7f5a4a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a4c072b40 0x7f5a4c0c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.443+0000 7f5a4a59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a4c072b40 0x7f5a4c0c2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54628/0 (socket says 172.21.15.100:54628) 2024-09-15T21:48:45.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.443+0000 7f5a4a59c640 1 -- 172.21.15.100:0/399895403 learned_addr learned my addr 172.21.15.100:0/399895403 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:45.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.443+0000 7f5a49d9b640 1 --2- 172.21.15.100:0/399895403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a4c0c8f40 0x7f5a4c06e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.443+0000 7f5a4ad9d640 1 --2- 172.21.15.100:0/399895403 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5a4c152380 0x7f5a4c14efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.443+0000 7f5a4a59c640 1 -- 172.21.15.100:0/399895403 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5a4c152380 msgr2=0x7f5a4c14efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.443+0000 7f5a4a59c640 1 --2- 172.21.15.100:0/399895403 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5a4c152380 0x7f5a4c14efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.443+0000 7f5a4a59c640 1 -- 172.21.15.100:0/399895403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a4c0c8f40 msgr2=0x7f5a4c06e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.443+0000 7f5a4a59c640 1 --2- 172.21.15.100:0/399895403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5a4c0c8f40 0x7f5a4c06e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.443+0000 7f5a4a59c640 1 -- 172.21.15.100:0/399895403 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a4c0c1140 con 0x7f5a4c072b40 2024-09-15T21:48:45.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.444+0000 7f5a4a59c640 1 --2- 172.21.15.100:0/399895403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a4c072b40 0x7f5a4c0c2380 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c000c00 tx=0x7f5a3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:45.451 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.444+0000 7f5a3affd640 1 -- 172.21.15.100:0/399895403 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c037050 con 0x7f5a4c072b40 2024-09-15T21:48:45.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.444+0000 7f5a3affd640 1 -- 172.21.15.100:0/399895403 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a3c038040 con 0x7f5a4c072b40 2024-09-15T21:48:45.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.444+0000 7f5a50b2a640 1 -- 172.21.15.100:0/399895403 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4c0c28c0 con 0x7f5a4c072b40 2024-09-15T21:48:45.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.444+0000 7f5a3affd640 1 -- 172.21.15.100:0/399895403 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c004470 con 0x7f5a4c072b40 2024-09-15T21:48:45.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.444+0000 7f5a50b2a640 1 -- 172.21.15.100:0/399895403 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a4c0c0b80 con 0x7f5a4c072b40 2024-09-15T21:48:45.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.445+0000 7f5a50b2a640 1 -- 172.21.15.100:0/399895403 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5a4c0c0650 con 0x7f5a4c072b40 2024-09-15T21:48:45.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.445+0000 7f5a3affd640 1 -- 172.21.15.100:0/399895403 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5a3c003610 con 0x7f5a4c072b40 2024-09-15T21:48:45.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.446+0000 7f5a3affd640 1 --2- 172.21.15.100:0/399895403 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5a2c06c2e0 0x7f5a2c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.446+0000 7f5a3affd640 1 -- 172.21.15.100:0/399895403 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f5a3c0abd50 con 0x7f5a4c072b40 2024-09-15T21:48:45.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.446+0000 7f5a49d9b640 1 --2- 172.21.15.100:0/399895403 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5a2c06c2e0 0x7f5a2c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.446+0000 7f5a3affd640 1 --2- 172.21.15.100:0/399895403 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f5a2c0746c0 0x7f5a2c076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.446+0000 7f5a3affd640 1 -- 172.21.15.100:0/399895403 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5a3c048e50 con 0x7f5a2c0746c0 2024-09-15T21:48:45.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.446+0000 7f5a3affd640 1 -- 172.21.15.100:0/399895403 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f5a3c0ae450 con 0x7f5a4c072b40 2024-09-15T21:48:45.452 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.446+0000 7f5a4ad9d640 1 --2- 172.21.15.100:0/399895403 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f5a2c0746c0 0x7f5a2c076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.446+0000 7f5a49d9b640 1 --2- 172.21.15.100:0/399895403 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5a2c06c2e0 0x7f5a2c06e7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c15bb50 tx=0x7f5a40009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:45.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.446+0000 7f5a4ad9d640 1 --2- 172.21.15.100:0/399895403 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f5a2c0746c0 0x7f5a2c076b60 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:45.453 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.447+0000 7f5a3affd640 1 -- 172.21.15.100:0/399895403 <== osd.3 v2:172.21.15.100:6810/1287562741 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5a3c048e50 con 0x7f5a2c0746c0 2024-09-15T21:48:45.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.459+0000 7f5a50b2a640 1 -- 172.21.15.100:0/399895403 --> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5a4c073aa0 con 0x7f5a2c0746c0 2024-09-15T21:48:45.461 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.459+0000 7f5a3affd640 1 -- 172.21.15.100:0/399895403 <== osd.3 v2:172.21.15.100:6810/1287562741 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5a4c073aa0 con 0x7f5a2c0746c0 2024-09-15T21:48:45.462 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:45.462 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:48:45.462 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:45.462 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.460+0000 7f5a50b2a640 1 -- 172.21.15.100:0/399895403 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f5a2c0746c0 msgr2=0x7f5a2c076b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.460+0000 7f5a50b2a640 1 --2- 172.21.15.100:0/399895403 >> [v2:172.21.15.100:6810/1287562741,v1:172.21.15.100:6811/1287562741] conn(0x7f5a2c0746c0 0x7f5a2c076b60 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.460+0000 7f5a50b2a640 1 -- 172.21.15.100:0/399895403 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5a2c06c2e0 msgr2=0x7f5a2c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.460+0000 7f5a50b2a640 1 --2- 172.21.15.100:0/399895403 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5a2c06c2e0 0x7f5a2c06e7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c15bb50 tx=0x7f5a40009040 comp rx=0 tx=0).stop 2024-09-15T21:48:45.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.460+0000 7f5a50b2a640 1 -- 172.21.15.100:0/399895403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a4c072b40 msgr2=0x7f5a4c0c2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.460+0000 7f5a50b2a640 1 --2- 172.21.15.100:0/399895403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5a4c072b40 0x7f5a4c0c2380 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c000c00 tx=0x7f5a3c0046c0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.460+0000 7f5a4ad9d640 1 -- 172.21.15.100:0/399895403 reap_dead start 2024-09-15T21:48:45.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.460+0000 7f5a50b2a640 1 -- 172.21.15.100:0/399895403 shutdown_connections 2024-09-15T21:48:45.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.460+0000 7f5a50b2a640 1 -- 172.21.15.100:0/399895403 >> 172.21.15.100:0/399895403 conn(0x7f5a4c0c9380 msgr2=0x7f5a4c14fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:45.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.460+0000 7f5a50b2a640 1 -- 172.21.15.100:0/399895403 shutdown_connections 2024-09-15T21:48:45.463 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.460+0000 7f5a50b2a640 1 -- 172.21.15.100:0/399895403 wait complete. 2024-09-15T21:48:45.472 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-15T21:48:45.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.571+0000 7ff0d8d6f640 1 Processor -- start 2024-09-15T21:48:45.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.571+0000 7ff0d8d6f640 1 -- start start 2024-09-15T21:48:45.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.572+0000 7ff0d8d6f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff0d40c9710 0x7ff0d40730f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.572+0000 7ff0d8d6f640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff0d40c9b40 0x7ff0d4073630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.574 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.572+0000 7ff0d8d6f640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0d415c530 con 0x7ff0d4152fe0 2024-09-15T21:48:45.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.572+0000 7ff0d8d6f640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff0d4159ec0 con 0x7ff0d40c9b40 2024-09-15T21:48:45.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.572+0000 7ff0d8d6f640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7ff0d415c830 con 0x7ff0d40c9710 2024-09-15T21:48:45.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.572+0000 7ff0d2d9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff0d40c9b40 0x7ff0d4073630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.572+0000 7ff0d2d9d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff0d40c9b40 0x7ff0d4073630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54632/0 (socket says 172.21.15.100:54632) 2024-09-15T21:48:45.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.572+0000 7ff0d2d9d640 1 -- 172.21.15.100:0/181269013 learned_addr learned my addr 172.21.15.100:0/181269013 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:45.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.572+0000 7ff0d1d9b640 1 --2- 172.21.15.100:0/181269013 >> v2:172.21.15.122:3300/0 conn(0x7ff0d40c9710 0x7ff0d40730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.572+0000 7ff0d2d9d640 1 -- 172.21.15.100:0/181269013 >> v2:172.21.15.122:3300/0 conn(0x7ff0d40c9710 msgr2=0x7ff0d40730f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:45.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.572+0000 7ff0d2d9d640 1 --2- 172.21.15.100:0/181269013 >> v2:172.21.15.122:3300/0 conn(0x7ff0d40c9710 0x7ff0d40730f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.572+0000 7ff0d2d9d640 1 -- 172.21.15.100:0/181269013 >> v1:172.21.15.122:6789/0 conn(0x7ff0d4152fe0 legacy=0x7ff0d415ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.572+0000 7ff0d2d9d640 1 -- 172.21.15.100:0/181269013 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d415d3d0 con 0x7ff0d40c9b40 2024-09-15T21:48:45.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.573+0000 7ff0d2d9d640 1 --2- 172.21.15.100:0/181269013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0d40c9b40 0x7ff0d4073630 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff0d406e580 tx=0x7ff0cc02a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e1ed748fd4ff1d server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:45.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.573+0000 7ff0d159a640 1 -- 172.21.15.100:0/181269013 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc036070 con 0x7ff0d40c9b40 2024-09-15T21:48:45.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.573+0000 7ff0d159a640 1 -- 172.21.15.100:0/181269013 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0cc031600 con 0x7ff0d40c9b40 2024-09-15T21:48:45.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.573+0000 7ff0d159a640 1 -- 172.21.15.100:0/181269013 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc0319f0 con 0x7ff0d40c9b40 2024-09-15T21:48:45.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.574+0000 7ff0d8d6f640 1 -- 172.21.15.100:0/181269013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0d40c9b40 msgr2=0x7ff0d4073630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.574+0000 7ff0d8d6f640 1 --2- 172.21.15.100:0/181269013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0d40c9b40 0x7ff0d4073630 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff0d406e580 tx=0x7ff0cc02a7c0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.577 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.574+0000 7ff0d8d6f640 1 -- 172.21.15.100:0/181269013 shutdown_connections 2024-09-15T21:48:45.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.574+0000 7ff0d8d6f640 1 --2- 172.21.15.100:0/181269013 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0d40c9b40 0x7ff0d4073630 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.574+0000 7ff0d8d6f640 1 --2- 172.21.15.100:0/181269013 >> v2:172.21.15.122:3300/0 conn(0x7ff0d40c9710 0x7ff0d40730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.574+0000 7ff0d8d6f640 1 -- 172.21.15.100:0/181269013 >> 172.21.15.100:0/181269013 conn(0x7ff0d4067dd0 msgr2=0x7ff0d406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:45.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.574+0000 7ff0d8d6f640 1 -- 172.21.15.100:0/181269013 shutdown_connections 2024-09-15T21:48:45.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.574+0000 7ff0d8d6f640 1 -- 172.21.15.100:0/181269013 wait complete. 2024-09-15T21:48:45.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.575+0000 7ff0d8d6f640 1 Processor -- start 2024-09-15T21:48:45.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.575+0000 7ff0d8d6f640 1 -- start start 2024-09-15T21:48:45.582 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.576+0000 7ff0d8d6f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0d40c9710 0x7ff0d41ef6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.576+0000 7ff0d8d6f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0d40c9b40 0x7ff0d415b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.576+0000 7ff0d8d6f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff0d4152fe0 0x7ff0d4155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.576+0000 7ff0d8d6f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d40c44f0 con 0x7ff0d40c9b40 2024-09-15T21:48:45.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.576+0000 7ff0d8d6f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d40c4210 con 0x7ff0d40c9710 2024-09-15T21:48:45.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.576+0000 7ff0d8d6f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff0d4158ea0 con 0x7ff0d4152fe0 2024-09-15T21:48:45.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.576+0000 7ff0d1d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0d40c9b40 0x7ff0d415b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.576+0000 7ff0d1d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0d40c9b40 0x7ff0d415b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54642/0 (socket says 172.21.15.100:54642) 2024-09-15T21:48:45.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.576+0000 7ff0d1d9b640 1 -- 172.21.15.100:0/2467397467 learned_addr learned my addr 172.21.15.100:0/2467397467 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:45.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.576+0000 7ff0d259c640 1 --2- 172.21.15.100:0/2467397467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0d40c9710 0x7ff0d41ef6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.576+0000 7ff0d2d9d640 1 --2- 172.21.15.100:0/2467397467 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff0d4152fe0 0x7ff0d4155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.576+0000 7ff0d1d9b640 1 -- 172.21.15.100:0/2467397467 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff0d4152fe0 msgr2=0x7ff0d4155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.583 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.576+0000 7ff0d1d9b640 1 --2- 172.21.15.100:0/2467397467 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff0d4152fe0 0x7ff0d4155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.576+0000 7ff0d1d9b640 1 -- 172.21.15.100:0/2467397467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0d40c9710 msgr2=0x7ff0d41ef6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.576+0000 7ff0d1d9b640 1 --2- 172.21.15.100:0/2467397467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0d40c9710 0x7ff0d41ef6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.576+0000 7ff0d1d9b640 1 -- 172.21.15.100:0/2467397467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d41571e0 con 0x7ff0d40c9b40 2024-09-15T21:48:45.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.576+0000 7ff0d1d9b640 1 --2- 172.21.15.100:0/2467397467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0d40c9b40 0x7ff0d415b320 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800e9e0 tx=0x7ff0c800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:45.584 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.577+0000 7ff0c2ffd640 1 -- 172.21.15.100:0/2467397467 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8017070 con 0x7ff0d40c9b40 2024-09-15T21:48:45.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.577+0000 7ff0c2ffd640 1 -- 172.21.15.100:0/2467397467 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0c800c9d0 con 0x7ff0d40c9b40 2024-09-15T21:48:45.585 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.577+0000 7ff0d8d6f640 1 -- 172.21.15.100:0/2467397467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d4153870 con 0x7ff0d40c9b40 2024-09-15T21:48:45.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.577+0000 7ff0c2ffd640 1 -- 172.21.15.100:0/2467397467 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c800cdc0 con 0x7ff0d40c9b40 2024-09-15T21:48:45.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.577+0000 7ff0d8d6f640 1 -- 172.21.15.100:0/2467397467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d4153df0 con 0x7ff0d40c9b40 2024-09-15T21:48:45.588 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.578+0000 7ff0d8d6f640 1 -- 172.21.15.100:0/2467397467 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d40c47d0 con 0x7ff0d40c9b40 2024-09-15T21:48:45.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.578+0000 7ff0c2ffd640 1 -- 172.21.15.100:0/2467397467 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff0c801a070 con 0x7ff0d40c9b40 2024-09-15T21:48:45.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.579+0000 7ff0c2ffd640 1 --2- 172.21.15.100:0/2467397467 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff0b406c2e0 0x7ff0b406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.579+0000 7ff0c2ffd640 1 -- 172.21.15.100:0/2467397467 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7ff0c808b410 con 0x7ff0d40c9b40 2024-09-15T21:48:45.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.579+0000 7ff0d259c640 1 --2- 172.21.15.100:0/2467397467 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff0b406c2e0 0x7ff0b406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.589 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.579+0000 7ff0d259c640 1 --2- 172.21.15.100:0/2467397467 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff0b406c2e0 0x7ff0b406e7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40016e0 tx=0x7ff0c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:45.590 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.583+0000 7ff0c2ffd640 1 -- 172.21.15.100:0/2467397467 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff0c8059490 con 0x7ff0d40c9b40 2024-09-15T21:48:45.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.753+0000 7ff0d8d6f640 1 -- 172.21.15.100:0/2467397467 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7ff0d4158bc0 con 0x7ff0b406c2e0 2024-09-15T21:48:45.756 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.754+0000 7ff0c2ffd640 1 -- 172.21.15.100:0/2467397467 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7ff0d4158bc0 con 0x7ff0b406c2e0 2024-09-15T21:48:45.756 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 3 to deep-scrub 2024-09-15T21:48:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.756+0000 7ff0d8d6f640 1 -- 172.21.15.100:0/2467397467 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff0b406c2e0 msgr2=0x7ff0b406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.756+0000 7ff0d8d6f640 1 --2- 172.21.15.100:0/2467397467 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff0b406c2e0 0x7ff0b406e7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40016e0 tx=0x7ff0c4009040 comp rx=0 tx=0).stop 2024-09-15T21:48:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.756+0000 7ff0d8d6f640 1 -- 172.21.15.100:0/2467397467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0d40c9b40 msgr2=0x7ff0d415b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.756+0000 7ff0d8d6f640 1 --2- 172.21.15.100:0/2467397467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0d40c9b40 0x7ff0d415b320 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800e9e0 tx=0x7ff0c800eea0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.756+0000 7ff0d8d6f640 1 -- 172.21.15.100:0/2467397467 shutdown_connections 2024-09-15T21:48:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.756+0000 7ff0d8d6f640 1 --2- 172.21.15.100:0/2467397467 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff0b406c2e0 0x7ff0b406e7a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.758 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.756+0000 7ff0d8d6f640 1 --2- 172.21.15.100:0/2467397467 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff0d4152fe0 0x7ff0d4155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.756+0000 7ff0d8d6f640 1 --2- 172.21.15.100:0/2467397467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff0d40c9b40 0x7ff0d415b320 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.756+0000 7ff0d8d6f640 1 --2- 172.21.15.100:0/2467397467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0d40c9710 0x7ff0d41ef6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.756+0000 7ff0d8d6f640 1 -- 172.21.15.100:0/2467397467 >> 172.21.15.100:0/2467397467 conn(0x7ff0d4067dd0 msgr2=0x7ff0d4151480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:45.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.756+0000 7ff0d8d6f640 1 -- 172.21.15.100:0/2467397467 shutdown_connections 2024-09-15T21:48:45.759 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.756+0000 7ff0d8d6f640 1 -- 172.21.15.100:0/2467397467 wait complete. 2024-09-15T21:48:45.768 INFO:tasks.ceph:Scrubbing osd.4 2024-09-15T21:48:45.769 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:48:45.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.870+0000 7f5ad75e8640 1 Processor -- start 2024-09-15T21:48:45.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.871+0000 7f5ad75e8640 1 -- start start 2024-09-15T21:48:45.873 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.871+0000 7f5ad75e8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f5ad00cebc0 0x7f5ad0073a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.871+0000 7f5ad75e8640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5ad0073fd0 0x7f5ad00c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.871+0000 7f5ad75e8640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ad015b480 con 0x7f5ad00cefa0 2024-09-15T21:48:45.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.871+0000 7f5ad75e8640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f5ad015b250 con 0x7f5ad0073fd0 2024-09-15T21:48:45.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.871+0000 7f5ad75e8640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f5ad015e800 con 0x7f5ad00cebc0 2024-09-15T21:48:45.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.871+0000 7f5ad4b83640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5ad0073fd0 0x7f5ad00c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.872+0000 7f5ad4b83640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f5ad0073fd0 0x7f5ad00c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54646/0 (socket says 172.21.15.100:54646) 2024-09-15T21:48:45.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.872+0000 7f5ad4b83640 1 -- 172.21.15.100:0/273430635 learned_addr learned my addr 172.21.15.100:0/273430635 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:45.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.872+0000 7f5ad5b85640 1 --1- 172.21.15.100:0/273430635 >> v1:172.21.15.100:6789/0 conn(0x7f5ad00cefa0 0x7f5ad015ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42224/0 (socket says 172.21.15.100:42224) 2024-09-15T21:48:45.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.872+0000 7f5ad5384640 1 --2- 172.21.15.100:0/273430635 >> v2:172.21.15.122:3300/0 conn(0x7f5ad00cebc0 0x7f5ad0073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.872+0000 7f5ad4b83640 1 -- 172.21.15.100:0/273430635 >> v2:172.21.15.122:3300/0 conn(0x7f5ad00cebc0 msgr2=0x7f5ad0073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:45.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.872+0000 7f5ad4b83640 1 --2- 172.21.15.100:0/273430635 >> v2:172.21.15.122:3300/0 conn(0x7f5ad00cebc0 0x7f5ad0073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.872+0000 7f5ad4b83640 1 -- 172.21.15.100:0/273430635 >> v1:172.21.15.100:6789/0 conn(0x7f5ad00cefa0 legacy=0x7f5ad015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.874 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.872+0000 7f5ad4b83640 1 -- 172.21.15.100:0/273430635 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad015b760 con 0x7f5ad0073fd0 2024-09-15T21:48:45.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.872+0000 7f5ad4b83640 1 --2- 172.21.15.100:0/273430635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ad0073fd0 0x7f5ad00c8590 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00cc480 tx=0x7f5ac402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8b63e9e86d69b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:45.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.873+0000 7f5acffff640 1 -- 172.21.15.100:0/273430635 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4037070 con 0x7f5ad0073fd0 2024-09-15T21:48:45.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.873+0000 7f5acffff640 1 -- 172.21.15.100:0/273430635 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac4032440 con 0x7f5ad0073fd0 2024-09-15T21:48:45.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.873+0000 7f5acffff640 1 -- 172.21.15.100:0/273430635 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4032810 con 0x7f5ad0073fd0 2024-09-15T21:48:45.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.873+0000 7f5ad75e8640 1 -- 172.21.15.100:0/273430635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ad0073fd0 msgr2=0x7f5ad00c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.873+0000 7f5ad75e8640 1 --2- 172.21.15.100:0/273430635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ad0073fd0 0x7f5ad00c8590 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5ad00cc480 tx=0x7f5ac402a270 comp rx=0 tx=0).stop 2024-09-15T21:48:45.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.873+0000 7f5ad75e8640 1 -- 172.21.15.100:0/273430635 shutdown_connections 2024-09-15T21:48:45.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.873+0000 7f5ad75e8640 1 --2- 172.21.15.100:0/273430635 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ad0073fd0 0x7f5ad00c8590 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.873+0000 7f5ad75e8640 1 --2- 172.21.15.100:0/273430635 >> v2:172.21.15.122:3300/0 conn(0x7f5ad00cebc0 0x7f5ad0073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.873+0000 7f5ad75e8640 1 -- 172.21.15.100:0/273430635 >> 172.21.15.100:0/273430635 conn(0x7f5ad0067980 msgr2=0x7f5ad0067d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:45.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.873+0000 7f5ad75e8640 1 -- 172.21.15.100:0/273430635 shutdown_connections 2024-09-15T21:48:45.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.873+0000 7f5ad75e8640 1 -- 172.21.15.100:0/273430635 wait complete. 2024-09-15T21:48:45.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.874+0000 7f5ad75e8640 1 Processor -- start 2024-09-15T21:48:45.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.874+0000 7f5ad75e8640 1 -- start start 2024-09-15T21:48:45.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.875+0000 7f5ad75e8640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ad0073fd0 0x7f5ad01e5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.875+0000 7f5ad75e8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ad00cebc0 0x7f5ad01e4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.875+0000 7f5ad75e8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ad00cefa0 0x7f5ad01e4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.875+0000 7f5ad75e8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad015c000 con 0x7f5ad00cebc0 2024-09-15T21:48:45.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.875+0000 7f5ad75e8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad015bd20 con 0x7f5ad00cefa0 2024-09-15T21:48:45.879 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.875+0000 7f5ad75e8640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad015f3a0 con 0x7f5ad0073fd0 2024-09-15T21:48:45.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.875+0000 7f5ad4b83640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ad00cebc0 0x7f5ad01e4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.875+0000 7f5ad4b83640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ad00cebc0 0x7f5ad01e4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54656/0 (socket says 172.21.15.100:54656) 2024-09-15T21:48:45.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.875+0000 7f5ad4b83640 1 -- 172.21.15.100:0/3175469088 learned_addr learned my addr 172.21.15.100:0/3175469088 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:45.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.875+0000 7f5ad5384640 1 --2- 172.21.15.100:0/3175469088 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ad0073fd0 0x7f5ad01e5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.875+0000 7f5ad5b85640 1 --2- 172.21.15.100:0/3175469088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ad00cefa0 0x7f5ad01e4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.876+0000 7f5ad4b83640 1 -- 172.21.15.100:0/3175469088 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ad0073fd0 msgr2=0x7f5ad01e5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.876+0000 7f5ad4b83640 1 --2- 172.21.15.100:0/3175469088 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f5ad0073fd0 0x7f5ad01e5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.876+0000 7f5ad4b83640 1 -- 172.21.15.100:0/3175469088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ad00cefa0 msgr2=0x7f5ad01e4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.876+0000 7f5ad4b83640 1 --2- 172.21.15.100:0/3175469088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5ad00cefa0 0x7f5ad01e4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.876+0000 7f5ad4b83640 1 -- 172.21.15.100:0/3175469088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad0155bc0 con 0x7f5ad00cebc0 2024-09-15T21:48:45.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.876+0000 7f5ad4b83640 1 --2- 172.21.15.100:0/3175469088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ad00cebc0 0x7f5ad01e4820 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5ac402a2b0 tx=0x7f5ac40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:45.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.876+0000 7f5acdffb640 1 -- 172.21.15.100:0/3175469088 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4041020 con 0x7f5ad00cebc0 2024-09-15T21:48:45.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.876+0000 7f5acdffb640 1 -- 172.21.15.100:0/3175469088 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac40306c0 con 0x7f5ad00cebc0 2024-09-15T21:48:45.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.876+0000 7f5ad75e8640 1 -- 172.21.15.100:0/3175469088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad0155e60 con 0x7f5ad00cebc0 2024-09-15T21:48:45.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.876+0000 7f5acdffb640 1 -- 172.21.15.100:0/3175469088 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4037040 con 0x7f5ad00cebc0 2024-09-15T21:48:45.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.876+0000 7f5ad75e8640 1 -- 172.21.15.100:0/3175469088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad01e6620 con 0x7f5ad00cebc0 2024-09-15T21:48:45.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.877+0000 7f5ad75e8640 1 -- 172.21.15.100:0/3175469088 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ad00c9340 con 0x7f5ad00cebc0 2024-09-15T21:48:45.881 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.878+0000 7f5acdffb640 1 -- 172.21.15.100:0/3175469088 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f5ac4044020 con 0x7f5ad00cebc0 2024-09-15T21:48:45.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.878+0000 7f5acdffb640 1 --2- 172.21.15.100:0/3175469088 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5abc06c2b0 0x7f5abc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.878+0000 7f5acdffb640 1 -- 172.21.15.100:0/3175469088 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f5ac40ac060 con 0x7f5ad00cebc0 2024-09-15T21:48:45.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.878+0000 7f5ad5384640 1 --2- 172.21.15.100:0/3175469088 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5abc06c2b0 0x7f5abc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.878+0000 7f5acdffb640 1 --2- 172.21.15.100:0/3175469088 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f5abc074630 0x7f5abc076ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:45.882 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.878+0000 7f5acdffb640 1 -- 172.21.15.100:0/3175469088 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ac4001d30 con 0x7f5abc074630 2024-09-15T21:48:45.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.878+0000 7f5acdffb640 1 -- 172.21.15.100:0/3175469088 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f5ac40ac4f0 con 0x7f5ad00cebc0 2024-09-15T21:48:45.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.878+0000 7f5ad5384640 1 --2- 172.21.15.100:0/3175469088 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5abc06c2b0 0x7f5abc06e770 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0003b80 tx=0x7f5ac00098d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:45.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.879+0000 7f5ad5b85640 1 --2- 172.21.15.100:0/3175469088 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f5abc074630 0x7f5abc076ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:45.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.879+0000 7f5ad5b85640 1 --2- 172.21.15.100:0/3175469088 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f5abc074630 0x7f5abc076ad0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:45.883 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.880+0000 7f5acdffb640 1 -- 172.21.15.100:0/3175469088 <== osd.4 v2:172.21.15.122:6800/4045445512 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5ac4001d30 con 0x7f5abc074630 2024-09-15T21:48:45.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.892+0000 7f5ad75e8640 1 -- 172.21.15.100:0/3175469088 --> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f5ad0068250 con 0x7f5abc074630 2024-09-15T21:48:45.894 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.892+0000 7f5acdffb640 1 -- 172.21.15.100:0/3175469088 <== osd.4 v2:172.21.15.122:6800/4045445512 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f5ad0068250 con 0x7f5abc074630 2024-09-15T21:48:45.894 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:45.894 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:48:45.895 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.893+0000 7f5ad75e8640 1 -- 172.21.15.100:0/3175469088 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f5abc074630 msgr2=0x7f5abc076ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.893+0000 7f5ad75e8640 1 --2- 172.21.15.100:0/3175469088 >> [v2:172.21.15.122:6800/4045445512,v1:172.21.15.122:6801/4045445512] conn(0x7f5abc074630 0x7f5abc076ad0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.893+0000 7f5ad75e8640 1 -- 172.21.15.100:0/3175469088 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5abc06c2b0 msgr2=0x7f5abc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.893+0000 7f5ad75e8640 1 --2- 172.21.15.100:0/3175469088 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f5abc06c2b0 0x7f5abc06e770 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0003b80 tx=0x7f5ac00098d0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.893+0000 7f5ad75e8640 1 -- 172.21.15.100:0/3175469088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ad00cebc0 msgr2=0x7f5ad01e4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.893+0000 7f5ad75e8640 1 --2- 172.21.15.100:0/3175469088 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5ad00cebc0 0x7f5ad01e4820 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5ac402a2b0 tx=0x7f5ac40046c0 comp rx=0 tx=0).stop 2024-09-15T21:48:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.893+0000 7f5ad5b85640 1 -- 172.21.15.100:0/3175469088 reap_dead start 2024-09-15T21:48:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.893+0000 7f5ad75e8640 1 -- 172.21.15.100:0/3175469088 shutdown_connections 2024-09-15T21:48:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.893+0000 7f5ad75e8640 1 -- 172.21.15.100:0/3175469088 >> 172.21.15.100:0/3175469088 conn(0x7f5ad0067980 msgr2=0x7f5ad0158eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.893+0000 7f5ad75e8640 1 -- 172.21.15.100:0/3175469088 shutdown_connections 2024-09-15T21:48:45.896 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:45.893+0000 7f5ad75e8640 1 -- 172.21.15.100:0/3175469088 wait complete. 2024-09-15T21:48:45.905 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-15T21:48:46.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.003+0000 7face3ed0640 1 Processor -- start 2024-09-15T21:48:46.005 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.003+0000 7face3ed0640 1 -- start start 2024-09-15T21:48:46.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.004+0000 7face3ed0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7facdc1555c0 0x7facdc0c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.004+0000 7face3ed0640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facdc15f1a0 con 0x7facdc072160 2024-09-15T21:48:46.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.004+0000 7face3ed0640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facdc15f440 con 0x7facdc0c99e0 2024-09-15T21:48:46.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.004+0000 7face3ed0640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7facdc0c83a0 con 0x7facdc1555c0 2024-09-15T21:48:46.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.004+0000 7face1c6c640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7facdc072160 0x7facdc072540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42228/0 (socket says 172.21.15.100:42228) 2024-09-15T21:48:46.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.004+0000 7face1c6c640 1 -- 172.21.15.100:0/1779379311 learned_addr learned my addr 172.21.15.100:0/1779379311 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:46.006 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.004+0000 7face246d640 1 --2- 172.21.15.100:0/1779379311 >> v2:172.21.15.100:3300/0 conn(0x7facdc1555c0 0x7facdc0c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.004+0000 7face246d640 1 -- 172.21.15.100:0/1779379311 >> v1:172.21.15.122:6789/0 conn(0x7facdc0c99e0 legacy=0x7facdc154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.004+0000 7face246d640 1 -- 172.21.15.100:0/1779379311 >> v1:172.21.15.100:6789/0 conn(0x7facdc072160 legacy=0x7facdc072540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.007 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.004+0000 7face246d640 1 -- 172.21.15.100:0/1779379311 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc0c8680 con 0x7facdc1555c0 2024-09-15T21:48:46.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.005+0000 7face246d640 1 --2- 172.21.15.100:0/1779379311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facdc1555c0 0x7facdc0c9230 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7facd4009d90 tx=0x7facd402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70d9581dd718689 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:46.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.005+0000 7face0c6a640 1 -- 172.21.15.100:0/1779379311 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4037070 con 0x7facdc1555c0 2024-09-15T21:48:46.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.005+0000 7face0c6a640 1 -- 172.21.15.100:0/1779379311 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd4007740 con 0x7facdc1555c0 2024-09-15T21:48:46.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.005+0000 7face0c6a640 1 -- 172.21.15.100:0/1779379311 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4007b10 con 0x7facdc1555c0 2024-09-15T21:48:46.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.006+0000 7face3ed0640 1 -- 172.21.15.100:0/1779379311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facdc1555c0 msgr2=0x7facdc0c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.006+0000 7face3ed0640 1 --2- 172.21.15.100:0/1779379311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facdc1555c0 0x7facdc0c9230 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7facd4009d90 tx=0x7facd402a780 comp rx=0 tx=0).stop 2024-09-15T21:48:46.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.006+0000 7face3ed0640 1 -- 172.21.15.100:0/1779379311 shutdown_connections 2024-09-15T21:48:46.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.006+0000 7face3ed0640 1 --2- 172.21.15.100:0/1779379311 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facdc1555c0 0x7facdc0c9230 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.006+0000 7face3ed0640 1 -- 172.21.15.100:0/1779379311 >> 172.21.15.100:0/1779379311 conn(0x7facdc0cd820 msgr2=0x7facdc0cdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:46.008 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.006+0000 7face3ed0640 1 -- 172.21.15.100:0/1779379311 shutdown_connections 2024-09-15T21:48:46.009 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.006+0000 7face3ed0640 1 -- 172.21.15.100:0/1779379311 wait complete. 2024-09-15T21:48:46.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.007+0000 7face3ed0640 1 Processor -- start 2024-09-15T21:48:46.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.007+0000 7face3ed0640 1 -- start start 2024-09-15T21:48:46.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7face3ed0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7facdc072160 0x7facdc14f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7face3ed0640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7facdc0c99e0 0x7facdc1f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7face3ed0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facdc1555c0 0x7facdc14ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7face3ed0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7facdc072e90 con 0x7facdc1555c0 2024-09-15T21:48:46.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7face3ed0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7facdc072d10 con 0x7facdc072160 2024-09-15T21:48:46.011 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7face3ed0640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7facdc15b3b0 con 0x7facdc0c99e0 2024-09-15T21:48:46.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7face246d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facdc1555c0 0x7facdc14ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7face246d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facdc1555c0 0x7facdc14ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54670/0 (socket says 172.21.15.100:54670) 2024-09-15T21:48:46.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7face246d640 1 -- 172.21.15.100:0/2328131235 learned_addr learned my addr 172.21.15.100:0/2328131235 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:46.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7face1c6c640 1 --2- 172.21.15.100:0/2328131235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7facdc072160 0x7facdc14f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7face146b640 1 --2- 172.21.15.100:0/2328131235 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7facdc0c99e0 0x7facdc1f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7face246d640 1 -- 172.21.15.100:0/2328131235 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7facdc0c99e0 msgr2=0x7facdc1f43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7face246d640 1 --2- 172.21.15.100:0/2328131235 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7facdc0c99e0 0x7facdc1f43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7face246d640 1 -- 172.21.15.100:0/2328131235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7facdc072160 msgr2=0x7facdc14f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7face246d640 1 --2- 172.21.15.100:0/2328131235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7facdc072160 0x7facdc14f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7face246d640 1 -- 172.21.15.100:0/2328131235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc14f9e0 con 0x7facdc1555c0 2024-09-15T21:48:46.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7face246d640 1 --2- 172.21.15.100:0/2328131235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facdc1555c0 0x7facdc14ac70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7facd4009eb0 tx=0x7facd4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:46.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7facd27fc640 1 -- 172.21.15.100:0/2328131235 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4042020 con 0x7facdc1555c0 2024-09-15T21:48:46.012 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7facd27fc640 1 -- 172.21.15.100:0/2328131235 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facd4032990 con 0x7facdc1555c0 2024-09-15T21:48:46.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7face3ed0640 1 -- 172.21.15.100:0/2328131235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facdc1f4d10 con 0x7facdc1555c0 2024-09-15T21:48:46.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7facd27fc640 1 -- 172.21.15.100:0/2328131235 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd4037040 con 0x7facdc1555c0 2024-09-15T21:48:46.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.008+0000 7face3ed0640 1 -- 172.21.15.100:0/2328131235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facdc0ccb70 con 0x7facdc1555c0 2024-09-15T21:48:46.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.009+0000 7face3ed0640 1 -- 172.21.15.100:0/2328131235 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facdc0c5e70 con 0x7facdc1555c0 2024-09-15T21:48:46.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.010+0000 7facd27fc640 1 -- 172.21.15.100:0/2328131235 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7facd4045020 con 0x7facdc1555c0 2024-09-15T21:48:46.013 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.010+0000 7facd27fc640 1 --2- 172.21.15.100:0/2328131235 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7facc006c2e0 0x7facc006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.010+0000 7facd27fc640 1 -- 172.21.15.100:0/2328131235 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7facd40ac600 con 0x7facdc1555c0 2024-09-15T21:48:46.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.010+0000 7face1c6c640 1 --2- 172.21.15.100:0/2328131235 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7facc006c2e0 0x7facc006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.014 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.011+0000 7face1c6c640 1 --2- 172.21.15.100:0/2328131235 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7facc006c2e0 0x7facc006e7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7facd8005880 tx=0x7facd8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:46.016 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.015+0000 7facd27fc640 1 -- 172.21.15.100:0/2328131235 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7facd404c030 con 0x7facdc1555c0 2024-09-15T21:48:46.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.181+0000 7face3ed0640 1 -- 172.21.15.100:0/2328131235 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7facdc0769a0 con 0x7facc006c2e0 2024-09-15T21:48:46.183 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.181+0000 7facd27fc640 1 -- 172.21.15.100:0/2328131235 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7facdc0769a0 con 0x7facc006c2e0 2024-09-15T21:48:46.183 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 4 to deep-scrub 2024-09-15T21:48:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.183+0000 7face3ed0640 1 -- 172.21.15.100:0/2328131235 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7facc006c2e0 msgr2=0x7facc006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.183+0000 7face3ed0640 1 --2- 172.21.15.100:0/2328131235 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7facc006c2e0 0x7facc006e7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7facd8005880 tx=0x7facd8009040 comp rx=0 tx=0).stop 2024-09-15T21:48:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.183+0000 7face3ed0640 1 -- 172.21.15.100:0/2328131235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facdc1555c0 msgr2=0x7facdc14ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.185 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.183+0000 7face3ed0640 1 --2- 172.21.15.100:0/2328131235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facdc1555c0 0x7facdc14ac70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7facd4009eb0 tx=0x7facd4004a40 comp rx=0 tx=0).stop 2024-09-15T21:48:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.183+0000 7face3ed0640 1 -- 172.21.15.100:0/2328131235 shutdown_connections 2024-09-15T21:48:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.183+0000 7face3ed0640 1 --2- 172.21.15.100:0/2328131235 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7facc006c2e0 0x7facc006e7a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.183+0000 7face3ed0640 1 --2- 172.21.15.100:0/2328131235 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7facdc1555c0 0x7facdc14ac70 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.183+0000 7face3ed0640 1 --2- 172.21.15.100:0/2328131235 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7facdc0c99e0 0x7facdc1f43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.183+0000 7face3ed0640 1 --2- 172.21.15.100:0/2328131235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7facdc072160 0x7facdc14f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.183+0000 7face3ed0640 1 -- 172.21.15.100:0/2328131235 >> 172.21.15.100:0/2328131235 conn(0x7facdc0cd820 msgr2=0x7facdc154b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.183+0000 7face3ed0640 1 -- 172.21.15.100:0/2328131235 shutdown_connections 2024-09-15T21:48:46.186 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.183+0000 7face3ed0640 1 -- 172.21.15.100:0/2328131235 wait complete. 2024-09-15T21:48:46.196 INFO:tasks.ceph:Scrubbing osd.5 2024-09-15T21:48:46.196 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:48:46.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.296+0000 7f2150a77640 1 Processor -- start 2024-09-15T21:48:46.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.296+0000 7f2150a77640 1 -- start start 2024-09-15T21:48:46.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.297+0000 7f2150a77640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f214c152380 0x7f214c15c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.297+0000 7f2150a77640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f214c072210 con 0x7f214c072b40 2024-09-15T21:48:46.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.297+0000 7f2150a77640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f214c0724f0 con 0x7f214c0c8f40 2024-09-15T21:48:46.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.297+0000 7f2150a77640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f214c071fc0 con 0x7f214c152380 2024-09-15T21:48:46.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.297+0000 7f2149d9b640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f214c072b40 0x7f214c158ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42242/0 (socket says 172.21.15.100:42242) 2024-09-15T21:48:46.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.297+0000 7f2149d9b640 1 -- 172.21.15.100:0/2147023614 learned_addr learned my addr 172.21.15.100:0/2147023614 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:46.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f214a59c640 1 --2- 172.21.15.100:0/2147023614 >> v2:172.21.15.122:3300/0 conn(0x7f214c152380 0x7f214c15c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f214959a640 1 -- 172.21.15.100:0/2147023614 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 731363482 0 0) 0x7f214c072210 con 0x7f214c072b40 2024-09-15T21:48:46.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f214959a640 1 -- 172.21.15.100:0/2147023614 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2138003520 con 0x7f214c072b40 2024-09-15T21:48:46.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f214959a640 1 -- 172.21.15.100:0/2147023614 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 474219256 0 0) 0x7f2138003520 con 0x7f214c072b40 2024-09-15T21:48:46.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f214959a640 1 -- 172.21.15.100:0/2147023614 >> v2:172.21.15.122:3300/0 conn(0x7f214c152380 msgr2=0x7f214c15c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:46.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f214959a640 1 --2- 172.21.15.100:0/2147023614 >> v2:172.21.15.122:3300/0 conn(0x7f214c152380 0x7f214c15c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f214959a640 1 -- 172.21.15.100:0/2147023614 >> v1:172.21.15.122:6789/0 conn(0x7f214c0c8f40 legacy=0x7f214c0c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f214959a640 1 -- 172.21.15.100:0/2147023614 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214c1582e0 con 0x7f214c072b40 2024-09-15T21:48:46.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f214959a640 1 -- 172.21.15.100:0/2147023614 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7f2134001670 con 0x7f214c072b40 2024-09-15T21:48:46.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f214959a640 1 -- 172.21.15.100:0/2147023614 >> v1:172.21.15.100:6789/0 conn(0x7f214c072b40 legacy=0x7f214c158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f214959a640 1 --2- 172.21.15.100:0/2147023614 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21380039b0 0x7f2138003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f214959a640 1 --2- 172.21.15.100:0/2147023614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2138004800 0x7f2138004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f214959a640 1 --2- 172.21.15.100:0/2147023614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21380051c0 0x7f213800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f214959a640 1 -- 172.21.15.100:0/2147023614 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f214c071fc0 con 0x7f2138004800 2024-09-15T21:48:46.303 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f214959a640 1 -- 172.21.15.100:0/2147023614 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f213800c060 con 0x7f21380051c0 2024-09-15T21:48:46.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f214959a640 1 -- 172.21.15.100:0/2147023614 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f213800c260 con 0x7f21380039b0 2024-09-15T21:48:46.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f2149d9b640 1 --2- 172.21.15.100:0/2147023614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2138004800 0x7f2138004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f214a59c640 1 --2- 172.21.15.100:0/2147023614 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21380039b0 0x7f2138003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f214ad9d640 1 --2- 172.21.15.100:0/2147023614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21380051c0 0x7f213800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f2149d9b640 1 -- 172.21.15.100:0/2147023614 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21380039b0 msgr2=0x7f2138003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:46.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f2149d9b640 1 --2- 172.21.15.100:0/2147023614 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21380039b0 0x7f2138003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f2149d9b640 1 -- 172.21.15.100:0/2147023614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21380051c0 msgr2=0x7f213800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:46.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f2149d9b640 1 --2- 172.21.15.100:0/2147023614 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21380051c0 0x7f213800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f214ad9d640 1 -- 172.21.15.100:0/2147023614 reap_dead start 2024-09-15T21:48:46.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.298+0000 7f2149d9b640 1 -- 172.21.15.100:0/2147023614 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2138003520 con 0x7f2138004800 2024-09-15T21:48:46.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.299+0000 7f2149d9b640 1 --2- 172.21.15.100:0/2147023614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2138004800 0x7f2138004c80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f213400c210 tx=0x7f2138005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad71471b1a241de7 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:46.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.299+0000 7f214959a640 1 -- 172.21.15.100:0/2147023614 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2134007d10 con 0x7f2138004800 2024-09-15T21:48:46.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.299+0000 7f214959a640 1 -- 172.21.15.100:0/2147023614 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21340058d0 con 0x7f2138004800 2024-09-15T21:48:46.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.299+0000 7f214959a640 1 -- 172.21.15.100:0/2147023614 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2134005d40 con 0x7f2138004800 2024-09-15T21:48:46.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.300+0000 7f2150a77640 1 -- 172.21.15.100:0/2147023614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2138004800 msgr2=0x7f2138004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.300+0000 7f2150a77640 1 --2- 172.21.15.100:0/2147023614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2138004800 0x7f2138004c80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f213400c210 tx=0x7f2138005830 comp rx=0 tx=0).stop 2024-09-15T21:48:46.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.300+0000 7f2150a77640 1 -- 172.21.15.100:0/2147023614 shutdown_connections 2024-09-15T21:48:46.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.300+0000 7f2150a77640 1 --2- 172.21.15.100:0/2147023614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2138004800 0x7f2138004c80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.300+0000 7f2150a77640 1 -- 172.21.15.100:0/2147023614 >> 172.21.15.100:0/2147023614 conn(0x7f214c0c9380 msgr2=0x7f214c0c9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:46.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.300+0000 7f2150a77640 1 -- 172.21.15.100:0/2147023614 shutdown_connections 2024-09-15T21:48:46.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.300+0000 7f2150a77640 1 -- 172.21.15.100:0/2147023614 wait complete. 2024-09-15T21:48:46.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.301+0000 7f2150a77640 1 Processor -- start 2024-09-15T21:48:46.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.301+0000 7f2150a77640 1 -- start start 2024-09-15T21:48:46.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.302+0000 7f2150a77640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2138004800 0x7f214c1e4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.302+0000 7f2150a77640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f214c1e50b0 0x7f214c1ef5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.302+0000 7f2150a77640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f214c1efae0 0x7f214c14fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.302+0000 7f2150a77640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f214c158650 con 0x7f214c1efae0 2024-09-15T21:48:46.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.302+0000 7f2150a77640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f214c158090 con 0x7f214c1e50b0 2024-09-15T21:48:46.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.302+0000 7f2150a77640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f214c06de80 con 0x7f2138004800 2024-09-15T21:48:46.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.302+0000 7f214ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f214c1efae0 0x7f214c14fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.302+0000 7f214ad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f214c1efae0 0x7f214c14fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54694/0 (socket says 172.21.15.100:54694) 2024-09-15T21:48:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.302+0000 7f214ad9d640 1 -- 172.21.15.100:0/1753134258 learned_addr learned my addr 172.21.15.100:0/1753134258 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.302+0000 7f214a59c640 1 --2- 172.21.15.100:0/1753134258 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2138004800 0x7f214c1e4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.302+0000 7f2149d9b640 1 --2- 172.21.15.100:0/1753134258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f214c1e50b0 0x7f214c1ef5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.302+0000 7f214ad9d640 1 -- 172.21.15.100:0/1753134258 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2138004800 msgr2=0x7f214c1e4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.302+0000 7f214ad9d640 1 --2- 172.21.15.100:0/1753134258 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2138004800 0x7f214c1e4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.302+0000 7f214ad9d640 1 -- 172.21.15.100:0/1753134258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f214c1e50b0 msgr2=0x7f214c1ef5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.302+0000 7f214ad9d640 1 --2- 172.21.15.100:0/1753134258 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f214c1e50b0 0x7f214c1ef5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.302+0000 7f214ad9d640 1 -- 172.21.15.100:0/1753134258 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f214c15b650 con 0x7f214c1efae0 2024-09-15T21:48:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.302+0000 7f214ad9d640 1 --2- 172.21.15.100:0/1753134258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f214c1efae0 0x7f214c14fb10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f214c0c9350 tx=0x7f214400f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.302+0000 7f2142ffd640 1 -- 172.21.15.100:0/1753134258 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214401a070 con 0x7f214c1efae0 2024-09-15T21:48:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.303+0000 7f2142ffd640 1 -- 172.21.15.100:0/1753134258 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2144005280 con 0x7f214c1efae0 2024-09-15T21:48:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.303+0000 7f2150a77640 1 -- 172.21.15.100:0/1753134258 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f214c1f3270 con 0x7f214c1efae0 2024-09-15T21:48:46.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.303+0000 7f2142ffd640 1 -- 172.21.15.100:0/1753134258 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2144007030 con 0x7f214c1efae0 2024-09-15T21:48:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.303+0000 7f2150a77640 1 -- 172.21.15.100:0/1753134258 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f214c1572b0 con 0x7f214c1efae0 2024-09-15T21:48:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.304+0000 7f2150a77640 1 -- 172.21.15.100:0/1753134258 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f214c1eff90 con 0x7f214c1efae0 2024-09-15T21:48:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.304+0000 7f2142ffd640 1 -- 172.21.15.100:0/1753134258 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f21440071d0 con 0x7f214c1efae0 2024-09-15T21:48:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.304+0000 7f2142ffd640 1 --2- 172.21.15.100:0/1753134258 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f212c06c2e0 0x7f212c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.304+0000 7f2142ffd640 1 -- 172.21.15.100:0/1753134258 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f214408a3c0 con 0x7f214c1efae0 2024-09-15T21:48:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.304+0000 7f214a59c640 1 --2- 172.21.15.100:0/1753134258 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f212c06c2e0 0x7f212c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.305+0000 7f2142ffd640 1 --2- 172.21.15.100:0/1753134258 >> [v2:172.21.15.122:6824/822572374,v1:172.21.15.122:6825/822572374] conn(0x7f212c0746c0 0x7f212c076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.305+0000 7f2142ffd640 1 -- 172.21.15.100:0/1753134258 --> [v2:172.21.15.122:6824/822572374,v1:172.21.15.122:6825/822572374] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f212c077230 con 0x7f212c0746c0 2024-09-15T21:48:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.305+0000 7f2142ffd640 1 -- 172.21.15.100:0/1753134258 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f2144091050 con 0x7f214c1efae0 2024-09-15T21:48:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.305+0000 7f214a59c640 1 --2- 172.21.15.100:0/1753134258 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f212c06c2e0 0x7f212c06e7a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f213c014510 tx=0x7f213c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.305+0000 7f2149d9b640 1 --2- 172.21.15.100:0/1753134258 >> [v2:172.21.15.122:6824/822572374,v1:172.21.15.122:6825/822572374] conn(0x7f212c0746c0 0x7f212c076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.305+0000 7f2149d9b640 1 --2- 172.21.15.100:0/1753134258 >> [v2:172.21.15.122:6824/822572374,v1:172.21.15.122:6825/822572374] conn(0x7f212c0746c0 0x7f212c076b60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:46.311 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.306+0000 7f2142ffd640 1 -- 172.21.15.100:0/1753134258 <== osd.5 v2:172.21.15.122:6824/822572374 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f212c077230 con 0x7f212c0746c0 2024-09-15T21:48:46.320 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.318+0000 7f2150a77640 1 -- 172.21.15.100:0/1753134258 --> [v2:172.21.15.122:6824/822572374,v1:172.21.15.122:6825/822572374] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f214c0c9ea0 con 0x7f212c0746c0 2024-09-15T21:48:46.321 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.319+0000 7f2142ffd640 1 -- 172.21.15.100:0/1753134258 <== osd.5 v2:172.21.15.122:6824/822572374 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f214c0c9ea0 con 0x7f212c0746c0 2024-09-15T21:48:46.321 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:46.321 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-09-15T21:48:46.322 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:46.323 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.319+0000 7f2150a77640 1 -- 172.21.15.100:0/1753134258 >> [v2:172.21.15.122:6824/822572374,v1:172.21.15.122:6825/822572374] conn(0x7f212c0746c0 msgr2=0x7f212c076b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.319+0000 7f2150a77640 1 --2- 172.21.15.100:0/1753134258 >> [v2:172.21.15.122:6824/822572374,v1:172.21.15.122:6825/822572374] conn(0x7f212c0746c0 0x7f212c076b60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.320+0000 7f2150a77640 1 -- 172.21.15.100:0/1753134258 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f212c06c2e0 msgr2=0x7f212c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.320+0000 7f2150a77640 1 --2- 172.21.15.100:0/1753134258 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f212c06c2e0 0x7f212c06e7a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f213c014510 tx=0x7f213c00b040 comp rx=0 tx=0).stop 2024-09-15T21:48:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.320+0000 7f2150a77640 1 -- 172.21.15.100:0/1753134258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f214c1efae0 msgr2=0x7f214c14fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.320+0000 7f2150a77640 1 --2- 172.21.15.100:0/1753134258 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f214c1efae0 0x7f214c14fb10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f214c0c9350 tx=0x7f214400f890 comp rx=0 tx=0).stop 2024-09-15T21:48:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.320+0000 7f214ad9d640 1 -- 172.21.15.100:0/1753134258 reap_dead start 2024-09-15T21:48:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.320+0000 7f2150a77640 1 -- 172.21.15.100:0/1753134258 shutdown_connections 2024-09-15T21:48:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.320+0000 7f2150a77640 1 -- 172.21.15.100:0/1753134258 >> 172.21.15.100:0/1753134258 conn(0x7f214c0c9380 msgr2=0x7f214c0c8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.320+0000 7f2150a77640 1 -- 172.21.15.100:0/1753134258 shutdown_connections 2024-09-15T21:48:46.324 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.320+0000 7f2150a77640 1 -- 172.21.15.100:0/1753134258 wait complete. 2024-09-15T21:48:46.332 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-15T21:48:46.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.430+0000 7f936a18f640 1 Processor -- start 2024-09-15T21:48:46.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.430+0000 7f936a18f640 1 -- start start 2024-09-15T21:48:46.433 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.431+0000 7f936a18f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93640cce10 0x7f9364071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.431+0000 7f936a18f640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93640c91f0 con 0x7f9364070ee0 2024-09-15T21:48:46.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.431+0000 7f936a18f640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93640c94d0 con 0x7f93640cca30 2024-09-15T21:48:46.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.431+0000 7f936a18f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f93640c97b0 con 0x7f93640cce10 2024-09-15T21:48:46.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.431+0000 7f9362ffd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f9364070ee0 0x7f936415af50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:42250/0 (socket says 172.21.15.100:42250) 2024-09-15T21:48:46.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.431+0000 7f9362ffd640 1 -- 172.21.15.100:0/611443291 learned_addr learned my addr 172.21.15.100:0/611443291 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:46.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.431+0000 7f9363fff640 1 --2- 172.21.15.100:0/611443291 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93640cce10 0x7f9364071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.431+0000 7f93627fc640 1 -- 172.21.15.100:0/611443291 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3860077920 0 0) 0x7f93640c91f0 con 0x7f9364070ee0 2024-09-15T21:48:46.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.431+0000 7f93627fc640 1 -- 172.21.15.100:0/611443291 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f934c003520 con 0x7f9364070ee0 2024-09-15T21:48:46.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.431+0000 7f93627fc640 1 -- 172.21.15.100:0/611443291 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1360058907 0 0) 0x7f93640c94d0 con 0x7f93640cca30 2024-09-15T21:48:46.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.431+0000 7f93627fc640 1 -- 172.21.15.100:0/611443291 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93640c91f0 con 0x7f93640cca30 2024-09-15T21:48:46.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.431+0000 7f9363fff640 1 -- 172.21.15.100:0/611443291 >> v1:172.21.15.122:6789/0 conn(0x7f93640cca30 legacy=0x7f9364151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.434 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.431+0000 7f9363fff640 1 -- 172.21.15.100:0/611443291 >> v1:172.21.15.100:6789/0 conn(0x7f9364070ee0 legacy=0x7f936415af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.431+0000 7f9363fff640 1 -- 172.21.15.100:0/611443291 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93640c9a90 con 0x7f93640cce10 2024-09-15T21:48:46.435 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.432+0000 7f9363fff640 1 --2- 172.21.15.100:0/611443291 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93640cce10 0x7f9364071340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f935c009dc0 tx=0x7f935c02a760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0c7453412076625 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:46.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.432+0000 7f93627fc640 1 -- 172.21.15.100:0/611443291 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c007740 con 0x7f93640cce10 2024-09-15T21:48:46.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.432+0000 7f93627fc640 1 -- 172.21.15.100:0/611443291 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f935c0078e0 con 0x7f93640cce10 2024-09-15T21:48:46.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.432+0000 7f93627fc640 1 -- 172.21.15.100:0/611443291 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c007c90 con 0x7f93640cce10 2024-09-15T21:48:46.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.433+0000 7f936a18f640 1 -- 172.21.15.100:0/611443291 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93640cce10 msgr2=0x7f9364071340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.433+0000 7f936a18f640 1 --2- 172.21.15.100:0/611443291 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93640cce10 0x7f9364071340 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f935c009dc0 tx=0x7f935c02a760 comp rx=0 tx=0).stop 2024-09-15T21:48:46.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.433+0000 7f936a18f640 1 -- 172.21.15.100:0/611443291 shutdown_connections 2024-09-15T21:48:46.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.433+0000 7f936a18f640 1 --2- 172.21.15.100:0/611443291 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f93640cce10 0x7f9364071340 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.433+0000 7f936a18f640 1 -- 172.21.15.100:0/611443291 >> 172.21.15.100:0/611443291 conn(0x7f9364067dd0 msgr2=0x7f936406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:46.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.433+0000 7f936a18f640 1 -- 172.21.15.100:0/611443291 shutdown_connections 2024-09-15T21:48:46.436 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.433+0000 7f936a18f640 1 -- 172.21.15.100:0/611443291 wait complete. 2024-09-15T21:48:46.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.434+0000 7f936a18f640 1 Processor -- start 2024-09-15T21:48:46.437 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.434+0000 7f936a18f640 1 -- start start 2024-09-15T21:48:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.435+0000 7f936a18f640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9364070ee0 0x7f9364153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.435+0000 7f936a18f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93640cca30 0x7f9364141a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.435+0000 7f936a18f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93640cce10 0x7f9364152bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.435+0000 7f936a18f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9364151ce0 con 0x7f93640cca30 2024-09-15T21:48:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.435+0000 7f936a18f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9364151b60 con 0x7f93640cce10 2024-09-15T21:48:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.435+0000 7f936a18f640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f936415b660 con 0x7f9364070ee0 2024-09-15T21:48:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.435+0000 7f9362ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93640cca30 0x7f9364141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.435+0000 7f9362ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93640cca30 0x7f9364141a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:54696/0 (socket says 172.21.15.100:54696) 2024-09-15T21:48:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.435+0000 7f9362ffd640 1 -- 172.21.15.100:0/1359852140 learned_addr learned my addr 172.21.15.100:0/1359852140 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:46.439 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.435+0000 7f93637fe640 1 --2- 172.21.15.100:0/1359852140 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9364070ee0 0x7f9364153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.435+0000 7f9363fff640 1 --2- 172.21.15.100:0/1359852140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93640cce10 0x7f9364152bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.435+0000 7f9362ffd640 1 -- 172.21.15.100:0/1359852140 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9364070ee0 msgr2=0x7f9364153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.435+0000 7f9362ffd640 1 --2- 172.21.15.100:0/1359852140 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9364070ee0 0x7f9364153e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.435+0000 7f9362ffd640 1 -- 172.21.15.100:0/1359852140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93640cce10 msgr2=0x7f9364152bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.435+0000 7f9362ffd640 1 --2- 172.21.15.100:0/1359852140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93640cce10 0x7f9364152bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.436+0000 7f9362ffd640 1 -- 172.21.15.100:0/1359852140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9364154d00 con 0x7f93640cca30 2024-09-15T21:48:46.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.436+0000 7f9362ffd640 1 --2- 172.21.15.100:0/1359852140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93640cca30 0x7f9364141a60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f935800e970 tx=0x7f935800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:46.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.436+0000 7f934bfff640 1 -- 172.21.15.100:0/1359852140 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9358015070 con 0x7f93640cca30 2024-09-15T21:48:46.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.436+0000 7f934bfff640 1 -- 172.21.15.100:0/1359852140 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9358005ce0 con 0x7f93640cca30 2024-09-15T21:48:46.440 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.436+0000 7f934bfff640 1 -- 172.21.15.100:0/1359852140 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935800ca50 con 0x7f93640cca30 2024-09-15T21:48:46.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.436+0000 7f936a18f640 1 -- 172.21.15.100:0/1359852140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9364154a60 con 0x7f93640cca30 2024-09-15T21:48:46.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.436+0000 7f936a18f640 1 -- 172.21.15.100:0/1359852140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93641f1bf0 con 0x7f93640cca30 2024-09-15T21:48:46.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.437+0000 7f936a18f640 1 -- 172.21.15.100:0/1359852140 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9364071990 con 0x7f93640cca30 2024-09-15T21:48:46.441 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.437+0000 7f934bfff640 1 -- 172.21.15.100:0/1359852140 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f9358018070 con 0x7f93640cca30 2024-09-15T21:48:46.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.438+0000 7f934bfff640 1 --2- 172.21.15.100:0/1359852140 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f935006c2b0 0x7f935006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.438+0000 7f934bfff640 1 -- 172.21.15.100:0/1359852140 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f935808d700 con 0x7f93640cca30 2024-09-15T21:48:46.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.438+0000 7f93637fe640 1 --2- 172.21.15.100:0/1359852140 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f935006c2b0 0x7f935006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.442 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.438+0000 7f93637fe640 1 --2- 172.21.15.100:0/1359852140 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f935006c2b0 0x7f935006e770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f93540011e0 tx=0x7f9354009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:46.444 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.442+0000 7f934bfff640 1 -- 172.21.15.100:0/1359852140 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f935805b780 con 0x7f93640cca30 2024-09-15T21:48:46.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.609+0000 7f936a18f640 1 -- 172.21.15.100:0/1359852140 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f93640c8ed0 con 0x7f935006c2b0 2024-09-15T21:48:46.611 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.610+0000 7f934bfff640 1 -- 172.21.15.100:0/1359852140 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f93640c8ed0 con 0x7f935006c2b0 2024-09-15T21:48:46.612 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 5 to deep-scrub 2024-09-15T21:48:46.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.612+0000 7f936a18f640 1 -- 172.21.15.100:0/1359852140 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f935006c2b0 msgr2=0x7f935006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.612+0000 7f936a18f640 1 --2- 172.21.15.100:0/1359852140 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f935006c2b0 0x7f935006e770 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f93540011e0 tx=0x7f9354009040 comp rx=0 tx=0).stop 2024-09-15T21:48:46.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.612+0000 7f936a18f640 1 -- 172.21.15.100:0/1359852140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93640cca30 msgr2=0x7f9364141a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.612+0000 7f936a18f640 1 --2- 172.21.15.100:0/1359852140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93640cca30 0x7f9364141a60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f935800e970 tx=0x7f935800ee30 comp rx=0 tx=0).stop 2024-09-15T21:48:46.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.612+0000 7f936a18f640 1 -- 172.21.15.100:0/1359852140 shutdown_connections 2024-09-15T21:48:46.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.612+0000 7f936a18f640 1 --2- 172.21.15.100:0/1359852140 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f935006c2b0 0x7f935006e770 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.612+0000 7f936a18f640 1 --2- 172.21.15.100:0/1359852140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93640cce10 0x7f9364152bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.612+0000 7f936a18f640 1 --2- 172.21.15.100:0/1359852140 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f93640cca30 0x7f9364141a60 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.612+0000 7f936a18f640 1 --2- 172.21.15.100:0/1359852140 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f9364070ee0 0x7f9364153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.612+0000 7f936a18f640 1 -- 172.21.15.100:0/1359852140 >> 172.21.15.100:0/1359852140 conn(0x7f9364067dd0 msgr2=0x7f93640c46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:46.614 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.612+0000 7f936a18f640 1 -- 172.21.15.100:0/1359852140 shutdown_connections 2024-09-15T21:48:46.615 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.612+0000 7f936a18f640 1 -- 172.21.15.100:0/1359852140 wait complete. 2024-09-15T21:48:46.624 INFO:tasks.ceph:Scrubbing osd.6 2024-09-15T21:48:46.625 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:48:46.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.726+0000 7fefff9bd640 1 Processor -- start 2024-09-15T21:48:46.728 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.726+0000 7fefff9bd640 1 -- start start 2024-09-15T21:48:46.730 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.727+0000 7fefff9bd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7feff8155500 0x7feff80cba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.727+0000 7fefff9bd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feff80cc2f0 0x7feff815b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.727+0000 7fefff9bd640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feff80cd190 con 0x7feff815b7e0 2024-09-15T21:48:46.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.727+0000 7fefff9bd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7feff815f770 con 0x7feff8155500 2024-09-15T21:48:46.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.727+0000 7fefff9bd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7feff815f2e0 con 0x7feff80cc2f0 2024-09-15T21:48:46.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.727+0000 7feffd759640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7feff8155500 0x7feff80cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.727+0000 7feffd759640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7feff8155500 0x7feff80cba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45278/0 (socket says 172.21.15.100:45278) 2024-09-15T21:48:46.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.727+0000 7feffd759640 1 -- 172.21.15.100:0/2702152745 learned_addr learned my addr 172.21.15.100:0/2702152745 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:46.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.727+0000 7feffdf5a640 1 --1- 172.21.15.100:0/2702152745 >> v1:172.21.15.100:6789/0 conn(0x7feff815b7e0 0x7feff80d0b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48040/0 (socket says 172.21.15.100:48040) 2024-09-15T21:48:46.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.727+0000 7feffcf58640 1 --2- 172.21.15.100:0/2702152745 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feff80cc2f0 0x7feff815b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.727+0000 7feffd759640 1 -- 172.21.15.100:0/2702152745 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feff80cc2f0 msgr2=0x7feff815b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:46.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.727+0000 7feffd759640 1 --2- 172.21.15.100:0/2702152745 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feff80cc2f0 0x7feff815b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.731 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.727+0000 7feffd759640 1 -- 172.21.15.100:0/2702152745 >> v1:172.21.15.100:6789/0 conn(0x7feff815b7e0 legacy=0x7feff80d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.727+0000 7feffd759640 1 -- 172.21.15.100:0/2702152745 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff815e9b0 con 0x7feff8155500 2024-09-15T21:48:46.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.727+0000 7feffcf58640 1 --2- 172.21.15.100:0/2702152745 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feff80cc2f0 0x7feff815b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:48:46.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.728+0000 7feffd759640 1 --2- 172.21.15.100:0/2702152745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff8155500 0x7feff80cba30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7feff4009870 tx=0x7feff402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a0994df8e309a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:46.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.728+0000 7fefeffff640 1 -- 172.21.15.100:0/2702152745 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4037070 con 0x7feff8155500 2024-09-15T21:48:46.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.728+0000 7fefeffff640 1 -- 172.21.15.100:0/2702152745 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff402ae50 con 0x7feff8155500 2024-09-15T21:48:46.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.728+0000 7fefeffff640 1 -- 172.21.15.100:0/2702152745 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4032860 con 0x7feff8155500 2024-09-15T21:48:46.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.729+0000 7fefff9bd640 1 -- 172.21.15.100:0/2702152745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff8155500 msgr2=0x7feff80cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.729+0000 7fefff9bd640 1 --2- 172.21.15.100:0/2702152745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff8155500 0x7feff80cba30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7feff4009870 tx=0x7feff402a300 comp rx=0 tx=0).stop 2024-09-15T21:48:46.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.729+0000 7fefff9bd640 1 -- 172.21.15.100:0/2702152745 shutdown_connections 2024-09-15T21:48:46.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.729+0000 7fefff9bd640 1 --2- 172.21.15.100:0/2702152745 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feff80cc2f0 0x7feff815b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.729+0000 7fefff9bd640 1 --2- 172.21.15.100:0/2702152745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff8155500 0x7feff80cba30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.729+0000 7fefff9bd640 1 -- 172.21.15.100:0/2702152745 >> 172.21.15.100:0/2702152745 conn(0x7feff8067980 msgr2=0x7feff8067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:46.732 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.729+0000 7fefff9bd640 1 -- 172.21.15.100:0/2702152745 shutdown_connections 2024-09-15T21:48:46.733 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.729+0000 7fefff9bd640 1 -- 172.21.15.100:0/2702152745 wait complete. 2024-09-15T21:48:46.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.730+0000 7fefff9bd640 1 Processor -- start 2024-09-15T21:48:46.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.730+0000 7fefff9bd640 1 -- start start 2024-09-15T21:48:46.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7fefff9bd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff80cc2f0 0x7feff815df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7fefff9bd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feff8155500 0x7feff814ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7fefff9bd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff815b7e0 0x7feff8150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7fefff9bd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7feff815ef70 con 0x7feff80cc2f0 2024-09-15T21:48:46.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7fefff9bd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feff80b6c50 con 0x7feff815b7e0 2024-09-15T21:48:46.735 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7fefff9bd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7feff80cbf70 con 0x7feff8155500 2024-09-15T21:48:46.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7feffd759640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff80cc2f0 0x7feff815df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7feffd759640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff80cc2f0 0x7feff815df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45282/0 (socket says 172.21.15.100:45282) 2024-09-15T21:48:46.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7feffd759640 1 -- 172.21.15.100:0/513468709 learned_addr learned my addr 172.21.15.100:0/513468709 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:46.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7feffcf58640 1 --2- 172.21.15.100:0/513468709 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feff8155500 0x7feff814ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7feffdf5a640 1 --2- 172.21.15.100:0/513468709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff815b7e0 0x7feff8150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7feffd759640 1 -- 172.21.15.100:0/513468709 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feff8155500 msgr2=0x7feff814ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7feffd759640 1 --2- 172.21.15.100:0/513468709 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7feff8155500 0x7feff814ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7feffd759640 1 -- 172.21.15.100:0/513468709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff815b7e0 msgr2=0x7feff8150440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7feffd759640 1 --2- 172.21.15.100:0/513468709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff815b7e0 0x7feff8150440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7feffd759640 1 -- 172.21.15.100:0/513468709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feff814f770 con 0x7feff80cc2f0 2024-09-15T21:48:46.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7feffdf5a640 1 --2- 172.21.15.100:0/513468709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feff815b7e0 0x7feff8150440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:48:46.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7feffd759640 1 --2- 172.21.15.100:0/513468709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff80cc2f0 0x7feff815df60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7feff4000c00 tx=0x7feff40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:46.736 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7fefedffb640 1 -- 172.21.15.100:0/513468709 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4041070 con 0x7feff80cc2f0 2024-09-15T21:48:46.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7fefedffb640 1 -- 172.21.15.100:0/513468709 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feff403c670 con 0x7feff80cc2f0 2024-09-15T21:48:46.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7fefff9bd640 1 -- 172.21.15.100:0/513468709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feff81f4cd0 con 0x7feff80cc2f0 2024-09-15T21:48:46.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.731+0000 7fefedffb640 1 -- 172.21.15.100:0/513468709 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4037040 con 0x7feff80cc2f0 2024-09-15T21:48:46.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.732+0000 7fefff9bd640 1 -- 172.21.15.100:0/513468709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feff8153580 con 0x7feff80cc2f0 2024-09-15T21:48:46.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.732+0000 7fefff9bd640 1 -- 172.21.15.100:0/513468709 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feff8153050 con 0x7feff80cc2f0 2024-09-15T21:48:46.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.733+0000 7fefedffb640 1 -- 172.21.15.100:0/513468709 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7feff4044070 con 0x7feff80cc2f0 2024-09-15T21:48:46.737 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.734+0000 7fefedffb640 1 --2- 172.21.15.100:0/513468709 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fefdc06c2b0 0x7fefdc06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.734+0000 7fefedffb640 1 -- 172.21.15.100:0/513468709 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7feff40abae0 con 0x7feff80cc2f0 2024-09-15T21:48:46.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.734+0000 7feffcf58640 1 --2- 172.21.15.100:0/513468709 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fefdc06c2b0 0x7fefdc06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.734+0000 7fefedffb640 1 --2- 172.21.15.100:0/513468709 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fefdc074690 0x7fefdc076b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.734+0000 7fefedffb640 1 -- 172.21.15.100:0/513468709 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feff400af30 con 0x7fefdc074690 2024-09-15T21:48:46.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.734+0000 7fefedffb640 1 -- 172.21.15.100:0/513468709 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7feff40b02c0 con 0x7feff80cc2f0 2024-09-15T21:48:46.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.734+0000 7feffdf5a640 1 --2- 172.21.15.100:0/513468709 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fefdc074690 0x7fefdc076b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.734+0000 7feffcf58640 1 --2- 172.21.15.100:0/513468709 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fefdc06c2b0 0x7fefdc06e770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7feff815b570 tx=0x7fefe8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:46.738 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.734+0000 7feffdf5a640 1 --2- 172.21.15.100:0/513468709 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fefdc074690 0x7fefdc076b30 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:46.739 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.735+0000 7fefedffb640 1 -- 172.21.15.100:0/513468709 <== osd.6 v2:172.21.15.122:6808/1437167311 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feff400af30 con 0x7fefdc074690 2024-09-15T21:48:46.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.748+0000 7fefff9bd640 1 -- 172.21.15.100:0/513468709 --> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7feff81558e0 con 0x7fefdc074690 2024-09-15T21:48:46.750 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.748+0000 7fefedffb640 1 -- 172.21.15.100:0/513468709 <== osd.6 v2:172.21.15.122:6808/1437167311 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7feff81558e0 con 0x7fefdc074690 2024-09-15T21:48:46.750 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:46.750 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:48:46.751 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:46.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.749+0000 7fefff9bd640 1 -- 172.21.15.100:0/513468709 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fefdc074690 msgr2=0x7fefdc076b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.749+0000 7fefff9bd640 1 --2- 172.21.15.100:0/513468709 >> [v2:172.21.15.122:6808/1437167311,v1:172.21.15.122:6809/1437167311] conn(0x7fefdc074690 0x7fefdc076b30 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.749+0000 7fefff9bd640 1 -- 172.21.15.100:0/513468709 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fefdc06c2b0 msgr2=0x7fefdc06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.749+0000 7fefff9bd640 1 --2- 172.21.15.100:0/513468709 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fefdc06c2b0 0x7fefdc06e770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7feff815b570 tx=0x7fefe8008040 comp rx=0 tx=0).stop 2024-09-15T21:48:46.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.749+0000 7fefff9bd640 1 -- 172.21.15.100:0/513468709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff80cc2f0 msgr2=0x7feff815df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.749+0000 7fefff9bd640 1 --2- 172.21.15.100:0/513468709 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7feff80cc2f0 0x7feff815df60 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7feff4000c00 tx=0x7feff40046c0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.749+0000 7feffdf5a640 1 -- 172.21.15.100:0/513468709 reap_dead start 2024-09-15T21:48:46.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.749+0000 7fefff9bd640 1 -- 172.21.15.100:0/513468709 shutdown_connections 2024-09-15T21:48:46.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.749+0000 7fefff9bd640 1 -- 172.21.15.100:0/513468709 >> 172.21.15.100:0/513468709 conn(0x7feff8067980 msgr2=0x7feff815d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:46.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.749+0000 7fefff9bd640 1 -- 172.21.15.100:0/513468709 shutdown_connections 2024-09-15T21:48:46.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.749+0000 7fefff9bd640 1 -- 172.21.15.100:0/513468709 wait complete. 2024-09-15T21:48:46.761 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-15T21:48:46.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.859+0000 7f8554653640 1 Processor -- start 2024-09-15T21:48:46.861 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.859+0000 7f8554653640 1 -- start start 2024-09-15T21:48:46.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.860+0000 7f8554653640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f854c0cca30 0x7f854c071340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.860+0000 7f8554653640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f854c0cce10 0x7f854c15af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.862 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.860+0000 7f8554653640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f854c0c91f0 con 0x7f854c070ee0 2024-09-15T21:48:46.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.860+0000 7f8554653640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f854c0c94d0 con 0x7f854c0cce10 2024-09-15T21:48:46.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.860+0000 7f8554653640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f854c0c97b0 con 0x7f854c0cca30 2024-09-15T21:48:46.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.860+0000 7f8552bf0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f854c0cce10 0x7f854c15af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.860+0000 7f8552bf0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f854c0cce10 0x7f854c15af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45286/0 (socket says 172.21.15.100:45286) 2024-09-15T21:48:46.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.860+0000 7f8552bf0640 1 -- 172.21.15.100:0/679429195 learned_addr learned my addr 172.21.15.100:0/679429195 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:46.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.860+0000 7f85523ef640 1 --2- 172.21.15.100:0/679429195 >> v2:172.21.15.122:3300/0 conn(0x7f854c0cca30 0x7f854c071340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.860+0000 7f8552bf0640 1 -- 172.21.15.100:0/679429195 >> v2:172.21.15.122:3300/0 conn(0x7f854c0cca30 msgr2=0x7f854c071340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:46.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.860+0000 7f8552bf0640 1 --2- 172.21.15.100:0/679429195 >> v2:172.21.15.122:3300/0 conn(0x7f854c0cca30 0x7f854c071340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.861+0000 7f8552bf0640 1 -- 172.21.15.100:0/679429195 >> v1:172.21.15.122:6789/0 conn(0x7f854c070ee0 legacy=0x7f854c151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:46.863 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.861+0000 7f8552bf0640 1 -- 172.21.15.100:0/679429195 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854c0c9a90 con 0x7f854c0cce10 2024-09-15T21:48:46.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.861+0000 7f8552bf0640 1 --2- 172.21.15.100:0/679429195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f854c0cce10 0x7f854c15af50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f854c0cc790 tx=0x7f854402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857da02a5f481e9c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:46.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.861+0000 7f85513ed640 1 -- 172.21.15.100:0/679429195 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8544036070 con 0x7f854c0cce10 2024-09-15T21:48:46.864 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.861+0000 7f85513ed640 1 -- 172.21.15.100:0/679429195 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8544031600 con 0x7f854c0cce10 2024-09-15T21:48:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.861+0000 7f85513ed640 1 -- 172.21.15.100:0/679429195 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85440319d0 con 0x7f854c0cce10 2024-09-15T21:48:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.862+0000 7f8554653640 1 -- 172.21.15.100:0/679429195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f854c0cce10 msgr2=0x7f854c15af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.862+0000 7f8554653640 1 --2- 172.21.15.100:0/679429195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f854c0cce10 0x7f854c15af50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f854c0cc790 tx=0x7f854402a780 comp rx=0 tx=0).stop 2024-09-15T21:48:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.862+0000 7f8554653640 1 -- 172.21.15.100:0/679429195 shutdown_connections 2024-09-15T21:48:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.862+0000 7f8554653640 1 --2- 172.21.15.100:0/679429195 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f854c0cce10 0x7f854c15af50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.862+0000 7f8554653640 1 --2- 172.21.15.100:0/679429195 >> v2:172.21.15.122:3300/0 conn(0x7f854c0cca30 0x7f854c071340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.862+0000 7f8554653640 1 -- 172.21.15.100:0/679429195 >> 172.21.15.100:0/679429195 conn(0x7f854c067dd0 msgr2=0x7f854c06bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.862+0000 7f8554653640 1 -- 172.21.15.100:0/679429195 shutdown_connections 2024-09-15T21:48:46.865 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.862+0000 7f8554653640 1 -- 172.21.15.100:0/679429195 wait complete. 2024-09-15T21:48:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.863+0000 7f8554653640 1 Processor -- start 2024-09-15T21:48:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.863+0000 7f8554653640 1 -- start start 2024-09-15T21:48:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.864+0000 7f8554653640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f854c070ee0 0x7f854c159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.864+0000 7f8554653640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f854c0cca30 0x7f854c0cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.864+0000 7f8554653640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f854c0cce10 0x7f854c159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.864+0000 7f8554653640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f854c151ce0 con 0x7f854c0cca30 2024-09-15T21:48:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.864+0000 7f8554653640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f854c151b60 con 0x7f854c0cce10 2024-09-15T21:48:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.864+0000 7f8554653640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f854c071990 con 0x7f854c070ee0 2024-09-15T21:48:46.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.864+0000 7f8551bee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f854c0cca30 0x7f854c0cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.864+0000 7f8551bee640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f854c0cca30 0x7f854c0cb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45292/0 (socket says 172.21.15.100:45292) 2024-09-15T21:48:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.864+0000 7f8551bee640 1 -- 172.21.15.100:0/2956836755 learned_addr learned my addr 172.21.15.100:0/2956836755 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.864+0000 7f8552bf0640 1 --2- 172.21.15.100:0/2956836755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f854c0cce10 0x7f854c159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.864+0000 7f85523ef640 1 --2- 172.21.15.100:0/2956836755 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f854c070ee0 0x7f854c159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.864+0000 7f8551bee640 1 -- 172.21.15.100:0/2956836755 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f854c070ee0 msgr2=0x7f854c159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.864+0000 7f8551bee640 1 --2- 172.21.15.100:0/2956836755 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f854c070ee0 0x7f854c159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.864+0000 7f8551bee640 1 -- 172.21.15.100:0/2956836755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f854c0cce10 msgr2=0x7f854c159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.864+0000 7f8551bee640 1 --2- 172.21.15.100:0/2956836755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f854c0cce10 0x7f854c159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.864+0000 7f8551bee640 1 -- 172.21.15.100:0/2956836755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854c15a260 con 0x7f854c0cca30 2024-09-15T21:48:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.865+0000 7f8551bee640 1 --2- 172.21.15.100:0/2956836755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f854c0cca30 0x7f854c0cb860 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f853c00c430 tx=0x7f853c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.865+0000 7f8542ffd640 1 -- 172.21.15.100:0/2956836755 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c011790 con 0x7f854c0cca30 2024-09-15T21:48:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.865+0000 7f8542ffd640 1 -- 172.21.15.100:0/2956836755 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f853c017070 con 0x7f854c0cca30 2024-09-15T21:48:46.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.865+0000 7f8554653640 1 -- 172.21.15.100:0/2956836755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854c1f2b50 con 0x7f854c0cca30 2024-09-15T21:48:46.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.865+0000 7f8542ffd640 1 -- 172.21.15.100:0/2956836755 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c011b70 con 0x7f854c0cca30 2024-09-15T21:48:46.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.865+0000 7f8554653640 1 -- 172.21.15.100:0/2956836755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f854c1f3110 con 0x7f854c0cca30 2024-09-15T21:48:46.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.866+0000 7f8554653640 1 -- 172.21.15.100:0/2956836755 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f854c151930 con 0x7f854c0cca30 2024-09-15T21:48:46.869 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.866+0000 7f8542ffd640 1 -- 172.21.15.100:0/2956836755 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f853c011d10 con 0x7f854c0cca30 2024-09-15T21:48:46.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.866+0000 7f8542ffd640 1 --2- 172.21.15.100:0/2956836755 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f853006c2b0 0x7f853006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:46.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.867+0000 7f8542ffd640 1 -- 172.21.15.100:0/2956836755 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f853c08b1e0 con 0x7f854c0cca30 2024-09-15T21:48:46.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.867+0000 7f85523ef640 1 --2- 172.21.15.100:0/2956836755 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f853006c2b0 0x7f853006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:46.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.867+0000 7f85523ef640 1 --2- 172.21.15.100:0/2956836755 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f853006c2b0 0x7f853006e770 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f854800aac0 tx=0x7f8548008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:46.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:46.871+0000 7f8542ffd640 1 -- 172.21.15.100:0/2956836755 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f854c151930 con 0x7f854c0cca30 2024-09-15T21:48:47.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.035+0000 7f8554653640 1 -- 172.21.15.100:0/2956836755 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f854c1f2900 con 0x7f853006c2b0 2024-09-15T21:48:47.037 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.035+0000 7f8542ffd640 1 -- 172.21.15.100:0/2956836755 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f854c1f2900 con 0x7f853006c2b0 2024-09-15T21:48:47.037 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 6 to deep-scrub 2024-09-15T21:48:47.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.037+0000 7f8554653640 1 -- 172.21.15.100:0/2956836755 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f853006c2b0 msgr2=0x7f853006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.039 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.037+0000 7f8554653640 1 --2- 172.21.15.100:0/2956836755 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f853006c2b0 0x7f853006e770 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f854800aac0 tx=0x7f8548008040 comp rx=0 tx=0).stop 2024-09-15T21:48:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.037+0000 7f8554653640 1 -- 172.21.15.100:0/2956836755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f854c0cca30 msgr2=0x7f854c0cb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.037+0000 7f8554653640 1 --2- 172.21.15.100:0/2956836755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f854c0cca30 0x7f854c0cb860 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f853c00c430 tx=0x7f853c00c8f0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.037+0000 7f8554653640 1 -- 172.21.15.100:0/2956836755 shutdown_connections 2024-09-15T21:48:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.037+0000 7f8554653640 1 --2- 172.21.15.100:0/2956836755 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f853006c2b0 0x7f853006e770 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.037+0000 7f8554653640 1 --2- 172.21.15.100:0/2956836755 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f854c0cce10 0x7f854c159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.037+0000 7f8554653640 1 --2- 172.21.15.100:0/2956836755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f854c0cca30 0x7f854c0cb860 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.037+0000 7f8554653640 1 --2- 172.21.15.100:0/2956836755 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f854c070ee0 0x7f854c159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.037+0000 7f8554653640 1 -- 172.21.15.100:0/2956836755 >> 172.21.15.100:0/2956836755 conn(0x7f854c067dd0 msgr2=0x7f854c151220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.037+0000 7f8554653640 1 -- 172.21.15.100:0/2956836755 shutdown_connections 2024-09-15T21:48:47.040 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.037+0000 7f8554653640 1 -- 172.21.15.100:0/2956836755 wait complete. 2024-09-15T21:48:47.050 INFO:tasks.ceph:Scrubbing osd.7 2024-09-15T21:48:47.051 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:48:47.107 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T21:48:47.116 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T21:48:47.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.153+0000 7fd07f8ba640 1 Processor -- start 2024-09-15T21:48:47.155 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.153+0000 7fd07f8ba640 1 -- start start 2024-09-15T21:48:47.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.154+0000 7fd07f8ba640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd078152f30 0x7fd07806c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:47.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.154+0000 7fd07f8ba640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd07806cec0 0x7fd0780c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:47.156 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.154+0000 7fd07f8ba640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd07815c400 con 0x7fd078153310 2024-09-15T21:48:47.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.154+0000 7fd07f8ba640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7fd0780c4ba0 con 0x7fd078152f30 2024-09-15T21:48:47.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.154+0000 7fd07f8ba640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd07815c6a0 con 0x7fd07806cec0 2024-09-15T21:48:47.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.154+0000 7fd07d656640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd078152f30 0x7fd07806c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:47.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.154+0000 7fd07d656640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7fd078152f30 0x7fd07806c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45306/0 (socket says 172.21.15.100:45306) 2024-09-15T21:48:47.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.154+0000 7fd07ce55640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fd078153310 0x7fd0781593a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48056/0 (socket says 172.21.15.100:48056) 2024-09-15T21:48:47.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.154+0000 7fd07d656640 1 -- 172.21.15.100:0/4164976320 learned_addr learned my addr 172.21.15.100:0/4164976320 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:47.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.154+0000 7fd07d656640 1 -- 172.21.15.100:0/4164976320 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd07806cec0 msgr2=0x7fd0780c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:48:47.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.154+0000 7fd07de57640 1 --2- 172.21.15.100:0/4164976320 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd07806cec0 0x7fd0780c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:47.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.154+0000 7fd07d656640 1 --2- 172.21.15.100:0/4164976320 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd07806cec0 0x7fd0780c72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.155+0000 7fd07d656640 1 -- 172.21.15.100:0/4164976320 >> v1:172.21.15.100:6789/0 conn(0x7fd078153310 legacy=0x7fd0781593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.155+0000 7fd06ffff640 1 -- 172.21.15.100:0/4164976320 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1414964824 0 0) 0x7fd07815c400 con 0x7fd078153310 2024-09-15T21:48:47.157 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.155+0000 7fd07d656640 1 -- 172.21.15.100:0/4164976320 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07815c980 con 0x7fd078152f30 2024-09-15T21:48:47.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.155+0000 7fd07d656640 1 --2- 172.21.15.100:0/4164976320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd078152f30 0x7fd07806c980 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd074009870 tx=0x7fd07402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d1a0dde1ebd624 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:47.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.155+0000 7fd06ffff640 1 -- 172.21.15.100:0/4164976320 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd074037070 con 0x7fd078152f30 2024-09-15T21:48:47.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.155+0000 7fd06ffff640 1 -- 172.21.15.100:0/4164976320 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd07402ae50 con 0x7fd078152f30 2024-09-15T21:48:47.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.155+0000 7fd06ffff640 1 -- 172.21.15.100:0/4164976320 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd074032860 con 0x7fd078152f30 2024-09-15T21:48:47.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.156+0000 7fd07f8ba640 1 -- 172.21.15.100:0/4164976320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd078152f30 msgr2=0x7fd07806c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.156+0000 7fd07f8ba640 1 --2- 172.21.15.100:0/4164976320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd078152f30 0x7fd07806c980 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd074009870 tx=0x7fd07402a300 comp rx=0 tx=0).stop 2024-09-15T21:48:47.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.156+0000 7fd07f8ba640 1 -- 172.21.15.100:0/4164976320 shutdown_connections 2024-09-15T21:48:47.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.156+0000 7fd07f8ba640 1 --2- 172.21.15.100:0/4164976320 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd07806cec0 0x7fd0780c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.156+0000 7fd07f8ba640 1 --2- 172.21.15.100:0/4164976320 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd078152f30 0x7fd07806c980 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.156+0000 7fd07f8ba640 1 -- 172.21.15.100:0/4164976320 >> 172.21.15.100:0/4164976320 conn(0x7fd0780cd930 msgr2=0x7fd0780cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:47.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.156+0000 7fd07f8ba640 1 -- 172.21.15.100:0/4164976320 shutdown_connections 2024-09-15T21:48:47.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.156+0000 7fd07f8ba640 1 -- 172.21.15.100:0/4164976320 wait complete. 2024-09-15T21:48:47.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.157+0000 7fd07f8ba640 1 Processor -- start 2024-09-15T21:48:47.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.157+0000 7fd07f8ba640 1 -- start start 2024-09-15T21:48:47.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.158+0000 7fd07f8ba640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd07806cec0 0x7fd078148c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:47.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.158+0000 7fd07f8ba640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd078152f30 0x7fd078157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:47.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.158+0000 7fd07f8ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0781e59a0 0x7fd0781e5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:47.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.158+0000 7fd07f8ba640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fd07815d220 con 0x7fd078152f30 2024-09-15T21:48:47.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.158+0000 7fd07f8ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd07815cf40 con 0x7fd0781e59a0 2024-09-15T21:48:47.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.158+0000 7fd07f8ba640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fd078158e10 con 0x7fd07806cec0 2024-09-15T21:48:47.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.158+0000 7fd07ce55640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd078152f30 0x7fd078157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:47.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.158+0000 7fd07ce55640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd078152f30 0x7fd078157340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45312/0 (socket says 172.21.15.100:45312) 2024-09-15T21:48:47.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.158+0000 7fd07ce55640 1 -- 172.21.15.100:0/1973637464 learned_addr learned my addr 172.21.15.100:0/1973637464 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:47.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.158+0000 7fd07d656640 1 --2- 172.21.15.100:0/1973637464 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd07806cec0 0x7fd078148c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:47.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.158+0000 7fd07de57640 1 --2- 172.21.15.100:0/1973637464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0781e59a0 0x7fd0781e5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:47.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.158+0000 7fd07ce55640 1 -- 172.21.15.100:0/1973637464 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd07806cec0 msgr2=0x7fd078148c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.158+0000 7fd07ce55640 1 --2- 172.21.15.100:0/1973637464 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fd07806cec0 0x7fd078148c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.158+0000 7fd07ce55640 1 -- 172.21.15.100:0/1973637464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0781e59a0 msgr2=0x7fd0781e5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.158+0000 7fd07ce55640 1 --2- 172.21.15.100:0/1973637464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0781e59a0 0x7fd0781e5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.158+0000 7fd07ce55640 1 -- 172.21.15.100:0/1973637464 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd078158070 con 0x7fd078152f30 2024-09-15T21:48:47.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.158+0000 7fd07de57640 1 --2- 172.21.15.100:0/1973637464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd0781e59a0 0x7fd0781e5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:48:47.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.158+0000 7fd07ce55640 1 --2- 172.21.15.100:0/1973637464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd078152f30 0x7fd078157340 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd06800c4e0 tx=0x7fd06800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:47.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.158+0000 7fd06dffb640 1 -- 172.21.15.100:0/1973637464 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068018070 con 0x7fd078152f30 2024-09-15T21:48:47.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.159+0000 7fd06dffb640 1 -- 172.21.15.100:0/1973637464 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0680074c0 con 0x7fd078152f30 2024-09-15T21:48:47.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.159+0000 7fd07f8ba640 1 -- 172.21.15.100:0/1973637464 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0781e6370 con 0x7fd078152f30 2024-09-15T21:48:47.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.159+0000 7fd06dffb640 1 -- 172.21.15.100:0/1973637464 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd068004a20 con 0x7fd078152f30 2024-09-15T21:48:47.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.159+0000 7fd07f8ba640 1 -- 172.21.15.100:0/1973637464 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0781582e0 con 0x7fd078152f30 2024-09-15T21:48:47.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.159+0000 7fd07f8ba640 1 -- 172.21.15.100:0/1973637464 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd078159b40 con 0x7fd078152f30 2024-09-15T21:48:47.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.160+0000 7fd06dffb640 1 -- 172.21.15.100:0/1973637464 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fd06801b070 con 0x7fd078152f30 2024-09-15T21:48:47.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.160+0000 7fd06dffb640 1 --2- 172.21.15.100:0/1973637464 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd05c06c2e0 0x7fd05c06e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:47.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.161+0000 7fd07d656640 1 --2- 172.21.15.100:0/1973637464 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd05c06c2e0 0x7fd05c06e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:47.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.161+0000 7fd06dffb640 1 -- 172.21.15.100:0/1973637464 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7fd06808a820 con 0x7fd078152f30 2024-09-15T21:48:47.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.161+0000 7fd06dffb640 1 --2- 172.21.15.100:0/1973637464 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fd05c074660 0x7fd05c076b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:47.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.161+0000 7fd06dffb640 1 -- 172.21.15.100:0/1973637464 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd068016e70 con 0x7fd05c074660 2024-09-15T21:48:47.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.161+0000 7fd06dffb640 1 -- 172.21.15.100:0/1973637464 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fd06808f2c0 con 0x7fd078152f30 2024-09-15T21:48:47.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.161+0000 7fd07d656640 1 --2- 172.21.15.100:0/1973637464 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd05c06c2e0 0x7fd05c06e7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd074034000 tx=0x7fd07402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:47.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.161+0000 7fd07de57640 1 --2- 172.21.15.100:0/1973637464 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fd05c074660 0x7fd05c076b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:47.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.161+0000 7fd07de57640 1 --2- 172.21.15.100:0/1973637464 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fd05c074660 0x7fd05c076b00 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:47.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.162+0000 7fd06dffb640 1 -- 172.21.15.100:0/1973637464 <== osd.7 v2:172.21.15.122:6816/2431377162 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd068016e70 con 0x7fd05c074660 2024-09-15T21:48:47.176 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.174+0000 7fd07f8ba640 1 -- 172.21.15.100:0/1973637464 --> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd0780c4690 con 0x7fd05c074660 2024-09-15T21:48:47.177 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.175+0000 7fd06dffb640 1 -- 172.21.15.100:0/1973637464 <== osd.7 v2:172.21.15.122:6816/2431377162 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd0780c4690 con 0x7fd05c074660 2024-09-15T21:48:47.177 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-09-15T21:48:47.177 INFO:teuthology.orchestra.run.smithi100.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:48:47.177 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-09-15T21:48:47.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.175+0000 7fd07f8ba640 1 -- 172.21.15.100:0/1973637464 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fd05c074660 msgr2=0x7fd05c076b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.175+0000 7fd07f8ba640 1 --2- 172.21.15.100:0/1973637464 >> [v2:172.21.15.122:6816/2431377162,v1:172.21.15.122:6817/2431377162] conn(0x7fd05c074660 0x7fd05c076b00 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.176+0000 7fd07f8ba640 1 -- 172.21.15.100:0/1973637464 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd05c06c2e0 msgr2=0x7fd05c06e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.176+0000 7fd07f8ba640 1 --2- 172.21.15.100:0/1973637464 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fd05c06c2e0 0x7fd05c06e7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd074034000 tx=0x7fd07402e000 comp rx=0 tx=0).stop 2024-09-15T21:48:47.178 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.176+0000 7fd07f8ba640 1 -- 172.21.15.100:0/1973637464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd078152f30 msgr2=0x7fd078157340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.176+0000 7fd07f8ba640 1 --2- 172.21.15.100:0/1973637464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd078152f30 0x7fd078157340 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd06800c4e0 tx=0x7fd06800c9a0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.176+0000 7fd07de57640 1 -- 172.21.15.100:0/1973637464 reap_dead start 2024-09-15T21:48:47.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.176+0000 7fd07f8ba640 1 -- 172.21.15.100:0/1973637464 shutdown_connections 2024-09-15T21:48:47.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.176+0000 7fd07f8ba640 1 -- 172.21.15.100:0/1973637464 >> 172.21.15.100:0/1973637464 conn(0x7fd0780cd930 msgr2=0x7fd07806d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:47.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.176+0000 7fd07f8ba640 1 -- 172.21.15.100:0/1973637464 shutdown_connections 2024-09-15T21:48:47.179 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.176+0000 7fd07f8ba640 1 -- 172.21.15.100:0/1973637464 wait complete. 2024-09-15T21:48:47.188 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-15T21:48:47.357 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.340+0000 7f986993b640 1 Processor -- start 2024-09-15T21:48:47.358 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.341+0000 7f986993b640 1 -- start start 2024-09-15T21:48:47.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.341+0000 7f986993b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98640c99e0 0x7f98640c9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:47.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.341+0000 7f986993b640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f986415f1a0 con 0x7f9864072160 2024-09-15T21:48:47.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.341+0000 7f986993b640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f986415f440 con 0x7f9864154eb0 2024-09-15T21:48:47.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.341+0000 7f986993b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f98640c83a0 con 0x7f98640c99e0 2024-09-15T21:48:47.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.341+0000 7f9862ffd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f9864072160 0x7f9864072540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:48070/0 (socket says 172.21.15.100:48070) 2024-09-15T21:48:47.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.341+0000 7f9862ffd640 1 -- 172.21.15.100:0/3454533513 learned_addr learned my addr 172.21.15.100:0/3454533513 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:48:47.359 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.342+0000 7f98627fc640 1 --2- 172.21.15.100:0/3454533513 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98640c99e0 0x7f98640c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:47.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.342+0000 7f9861ffb640 1 -- 172.21.15.100:0/3454533513 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4092745010 0 0) 0x7f986415f1a0 con 0x7f9864072160 2024-09-15T21:48:47.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.342+0000 7f9861ffb640 1 -- 172.21.15.100:0/3454533513 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f984c003520 con 0x7f9864072160 2024-09-15T21:48:47.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.342+0000 7f9861ffb640 1 -- 172.21.15.100:0/3454533513 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 46880596 0 0) 0x7f986415f440 con 0x7f9864154eb0 2024-09-15T21:48:47.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.342+0000 7f98627fc640 1 -- 172.21.15.100:0/3454533513 >> v1:172.21.15.122:6789/0 conn(0x7f9864154eb0 legacy=0x7f98641552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.342+0000 7f98627fc640 1 -- 172.21.15.100:0/3454533513 >> v1:172.21.15.100:6789/0 conn(0x7f9864072160 legacy=0x7f9864072540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.342+0000 7f98627fc640 1 -- 172.21.15.100:0/3454533513 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98640c8680 con 0x7f98640c99e0 2024-09-15T21:48:47.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.343+0000 7f98627fc640 1 --2- 172.21.15.100:0/3454533513 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98640c99e0 0x7f98640c9230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f98580097d0 tx=0x7f985802a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=474064640bfd8097 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:47.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.343+0000 7f9861ffb640 1 -- 172.21.15.100:0/3454533513 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858037070 con 0x7f98640c99e0 2024-09-15T21:48:47.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.343+0000 7f9861ffb640 1 -- 172.21.15.100:0/3454533513 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9858032440 con 0x7f98640c99e0 2024-09-15T21:48:47.360 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.343+0000 7f9861ffb640 1 -- 172.21.15.100:0/3454533513 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98580327f0 con 0x7f98640c99e0 2024-09-15T21:48:47.364 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.343+0000 7f986993b640 1 -- 172.21.15.100:0/3454533513 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98640c99e0 msgr2=0x7f98640c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.343+0000 7f986993b640 1 --2- 172.21.15.100:0/3454533513 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98640c99e0 0x7f98640c9230 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f98580097d0 tx=0x7f985802a290 comp rx=0 tx=0).stop 2024-09-15T21:48:47.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.343+0000 7f986993b640 1 -- 172.21.15.100:0/3454533513 shutdown_connections 2024-09-15T21:48:47.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.343+0000 7f986993b640 1 --2- 172.21.15.100:0/3454533513 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98640c99e0 0x7f98640c9230 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.343+0000 7f986993b640 1 -- 172.21.15.100:0/3454533513 >> 172.21.15.100:0/3454533513 conn(0x7f98640cd820 msgr2=0x7f98640cdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:47.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.343+0000 7f986993b640 1 -- 172.21.15.100:0/3454533513 shutdown_connections 2024-09-15T21:48:47.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.343+0000 7f986993b640 1 -- 172.21.15.100:0/3454533513 wait complete. 2024-09-15T21:48:47.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.345+0000 7f986993b640 1 Processor -- start 2024-09-15T21:48:47.365 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.345+0000 7f986993b640 1 -- start start 2024-09-15T21:48:47.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.345+0000 7f986993b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9864072160 0x7f986414f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:47.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.345+0000 7f986993b640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98640c99e0 0x7f986414fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:47.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.345+0000 7f986993b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9864154eb0 0x7f98641f28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:47.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.345+0000 7f986993b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f9864072e90 con 0x7f9864072160 2024-09-15T21:48:47.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.345+0000 7f986993b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9864072d10 con 0x7f9864154eb0 2024-09-15T21:48:47.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.345+0000 7f986993b640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f986415b3b0 con 0x7f98640c99e0 2024-09-15T21:48:47.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.346+0000 7f9862ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9864072160 0x7f986414f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:47.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.346+0000 7f9862ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9864072160 0x7f986414f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45320/0 (socket says 172.21.15.100:45320) 2024-09-15T21:48:47.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.346+0000 7f9862ffd640 1 -- 172.21.15.100:0/3024357825 learned_addr learned my addr 172.21.15.100:0/3024357825 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:47.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.346+0000 7f98627fc640 1 --2- 172.21.15.100:0/3024357825 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98640c99e0 0x7f986414fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:47.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.346+0000 7f98637fe640 1 --2- 172.21.15.100:0/3024357825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9864154eb0 0x7f98641f28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:47.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.346+0000 7f9862ffd640 1 -- 172.21.15.100:0/3024357825 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98640c99e0 msgr2=0x7f986414fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.346+0000 7f9862ffd640 1 --2- 172.21.15.100:0/3024357825 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98640c99e0 0x7f986414fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.366 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.346+0000 7f9862ffd640 1 -- 172.21.15.100:0/3024357825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9864154eb0 msgr2=0x7f98641f28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.346+0000 7f9862ffd640 1 --2- 172.21.15.100:0/3024357825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9864154eb0 0x7f98641f28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.346+0000 7f9862ffd640 1 -- 172.21.15.100:0/3024357825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98641f2100 con 0x7f9864072160 2024-09-15T21:48:47.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.346+0000 7f9862ffd640 1 --2- 172.21.15.100:0/3024357825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9864072160 0x7f986414f9b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f98540010e0 tx=0x7f985400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:47.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.346+0000 7f984bfff640 1 -- 172.21.15.100:0/3024357825 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9854017070 con 0x7f9864072160 2024-09-15T21:48:47.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.346+0000 7f984bfff640 1 -- 172.21.15.100:0/3024357825 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9854005d00 con 0x7f9864072160 2024-09-15T21:48:47.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.346+0000 7f984bfff640 1 -- 172.21.15.100:0/3024357825 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98540072b0 con 0x7f9864072160 2024-09-15T21:48:47.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.346+0000 7f986993b640 1 -- 172.21.15.100:0/3024357825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98640ca880 con 0x7f9864072160 2024-09-15T21:48:47.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.346+0000 7f986993b640 1 -- 172.21.15.100:0/3024357825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98641518f0 con 0x7f9864072160 2024-09-15T21:48:47.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.347+0000 7f986993b640 1 -- 172.21.15.100:0/3024357825 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98640c9230 con 0x7f9864072160 2024-09-15T21:48:47.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.348+0000 7f984bfff640 1 -- 172.21.15.100:0/3024357825 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f985401a070 con 0x7f9864072160 2024-09-15T21:48:47.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.348+0000 7f984bfff640 1 --2- 172.21.15.100:0/3024357825 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f985006c2b0 0x7f985006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:47.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.348+0000 7f984bfff640 1 -- 172.21.15.100:0/3024357825 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f985408a8a0 con 0x7f9864072160 2024-09-15T21:48:47.367 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.348+0000 7f98627fc640 1 --2- 172.21.15.100:0/3024357825 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f985006c2b0 0x7f985006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:47.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.349+0000 7f98627fc640 1 --2- 172.21.15.100:0/3024357825 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f985006c2b0 0x7f985006e770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9858034000 tx=0x7f985802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:47.368 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.353+0000 7f984bfff640 1 -- 172.21.15.100:0/3024357825 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f9854059000 con 0x7f9864072160 2024-09-15T21:48:47.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.520+0000 7f986993b640 1 -- 172.21.15.100:0/3024357825 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f98640c9610 con 0x7f985006c2b0 2024-09-15T21:48:47.523 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.521+0000 7f984bfff640 1 -- 172.21.15.100:0/3024357825 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f98640c9610 con 0x7f985006c2b0 2024-09-15T21:48:47.523 INFO:teuthology.orchestra.run.smithi100.stderr:instructed osd(s) 7 to deep-scrub 2024-09-15T21:48:47.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.522+0000 7f986993b640 1 -- 172.21.15.100:0/3024357825 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f985006c2b0 msgr2=0x7f985006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.522+0000 7f986993b640 1 --2- 172.21.15.100:0/3024357825 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f985006c2b0 0x7f985006e770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9858034000 tx=0x7f985802e000 comp rx=0 tx=0).stop 2024-09-15T21:48:47.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.522+0000 7f986993b640 1 -- 172.21.15.100:0/3024357825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9864072160 msgr2=0x7f986414f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.522+0000 7f986993b640 1 --2- 172.21.15.100:0/3024357825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9864072160 0x7f986414f9b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f98540010e0 tx=0x7f985400c960 comp rx=0 tx=0).stop 2024-09-15T21:48:47.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.523+0000 7f986993b640 1 -- 172.21.15.100:0/3024357825 shutdown_connections 2024-09-15T21:48:47.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.523+0000 7f986993b640 1 --2- 172.21.15.100:0/3024357825 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f985006c2b0 0x7f985006e770 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.523+0000 7f986993b640 1 --2- 172.21.15.100:0/3024357825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9864154eb0 0x7f98641f28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.523+0000 7f986993b640 1 --2- 172.21.15.100:0/3024357825 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f98640c99e0 0x7f986414fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.523+0000 7f986993b640 1 --2- 172.21.15.100:0/3024357825 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9864072160 0x7f986414f9b0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.525 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.523+0000 7f986993b640 1 -- 172.21.15.100:0/3024357825 >> 172.21.15.100:0/3024357825 conn(0x7f98640cd820 msgr2=0x7f98641552c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:47.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.523+0000 7f986993b640 1 -- 172.21.15.100:0/3024357825 shutdown_connections 2024-09-15T21:48:47.526 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.523+0000 7f986993b640 1 -- 172.21.15.100:0/3024357825 wait complete. 2024-09-15T21:48:47.535 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:48:47.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.636+0000 7f3a82b03640 1 Processor -- start 2024-09-15T21:48:47.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.636+0000 7f3a82b03640 1 -- start start 2024-09-15T21:48:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.637+0000 7f3a82b03640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3a7c1522e0 0x7f3a7c0c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.637+0000 7f3a82b03640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a7c157ef0 con 0x7f3a7c0c92d0 2024-09-15T21:48:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.637+0000 7f3a82b03640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a7c1581b0 con 0x7f3a7c0c96b0 2024-09-15T21:48:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.637+0000 7f3a82b03640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f3a7c0c4980 con 0x7f3a7c1522e0 2024-09-15T21:48:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.637+0000 7f3a810a0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3a7c1522e0 0x7f3a7c0c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.637+0000 7f3a810a0640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f3a7c1522e0 0x7f3a7c0c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45326/0 (socket says 172.21.15.100:45326) 2024-09-15T21:48:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.637+0000 7f3a810a0640 1 -- 172.21.15.100:0/2149873259 learned_addr learned my addr 172.21.15.100:0/2149873259 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:47.639 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.637+0000 7f3a810a0640 1 -- 172.21.15.100:0/2149873259 >> v1:172.21.15.122:6789/0 conn(0x7f3a7c0c96b0 legacy=0x7f3a7c0c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.637+0000 7f3a810a0640 1 -- 172.21.15.100:0/2149873259 >> v1:172.21.15.100:6789/0 conn(0x7f3a7c0c92d0 legacy=0x7f3a7c151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.640 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.637+0000 7f3a810a0640 1 -- 172.21.15.100:0/2149873259 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7c158490 con 0x7f3a7c1522e0 2024-09-15T21:48:47.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.638+0000 7f3a810a0640 1 --2- 172.21.15.100:0/2149873259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c1522e0 0x7f3a7c0c8740 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009e50 tx=0x7f3a7402c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1e8242737c1961 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:47.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.638+0000 7f3a7b7fe640 1 -- 172.21.15.100:0/2149873259 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74032430 con 0x7f3a7c1522e0 2024-09-15T21:48:47.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.638+0000 7f3a7b7fe640 1 -- 172.21.15.100:0/2149873259 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a74037070 con 0x7f3a7c1522e0 2024-09-15T21:48:47.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.638+0000 7f3a7b7fe640 1 -- 172.21.15.100:0/2149873259 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a740327f0 con 0x7f3a7c1522e0 2024-09-15T21:48:47.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.639+0000 7f3a82b03640 1 -- 172.21.15.100:0/2149873259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c1522e0 msgr2=0x7f3a7c0c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.641 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.639+0000 7f3a82b03640 1 --2- 172.21.15.100:0/2149873259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c1522e0 0x7f3a7c0c8740 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3a74009e50 tx=0x7f3a7402c940 comp rx=0 tx=0).stop 2024-09-15T21:48:47.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.639+0000 7f3a82b03640 1 -- 172.21.15.100:0/2149873259 shutdown_connections 2024-09-15T21:48:47.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.639+0000 7f3a82b03640 1 --2- 172.21.15.100:0/2149873259 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c1522e0 0x7f3a7c0c8740 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.639+0000 7f3a82b03640 1 -- 172.21.15.100:0/2149873259 >> 172.21.15.100:0/2149873259 conn(0x7f3a7c067dd0 msgr2=0x7f3a7c06bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:47.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.639+0000 7f3a82b03640 1 -- 172.21.15.100:0/2149873259 shutdown_connections 2024-09-15T21:48:47.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.639+0000 7f3a82b03640 1 -- 172.21.15.100:0/2149873259 wait complete. 2024-09-15T21:48:47.642 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.640+0000 7f3a82b03640 1 Processor -- start 2024-09-15T21:48:47.643 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.640+0000 7f3a82b03640 1 -- start start 2024-09-15T21:48:47.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.641+0000 7f3a82b03640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a7c0c92d0 0x7f3a7c1e2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:47.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.641+0000 7f3a82b03640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3a7c0c96b0 0x7f3a7c14c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:47.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.641+0000 7f3a82b03640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c1522e0 0x7f3a7c1f0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:47.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.641+0000 7f3a82b03640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c0c8d30 con 0x7f3a7c1522e0 2024-09-15T21:48:47.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.641+0000 7f3a82b03640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c158a50 con 0x7f3a7c0c92d0 2024-09-15T21:48:47.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.641+0000 7f3a82b03640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f3a7c0c8eb0 con 0x7f3a7c0c96b0 2024-09-15T21:48:47.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.641+0000 7f3a810a0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c1522e0 0x7f3a7c1f0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:47.645 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.641+0000 7f3a810a0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c1522e0 0x7f3a7c1f0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:45336/0 (socket says 172.21.15.100:45336) 2024-09-15T21:48:47.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.641+0000 7f3a810a0640 1 -- 172.21.15.100:0/2600820211 learned_addr learned my addr 172.21.15.100:0/2600820211 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:48:47.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.641+0000 7f3a8089f640 1 --2- 172.21.15.100:0/2600820211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a7c0c92d0 0x7f3a7c1e2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:47.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.641+0000 7f3a7bfff640 1 --2- 172.21.15.100:0/2600820211 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3a7c0c96b0 0x7f3a7c14c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:47.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.641+0000 7f3a810a0640 1 -- 172.21.15.100:0/2600820211 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3a7c0c96b0 msgr2=0x7f3a7c14c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.641+0000 7f3a810a0640 1 --2- 172.21.15.100:0/2600820211 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3a7c0c96b0 0x7f3a7c14c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.641+0000 7f3a810a0640 1 -- 172.21.15.100:0/2600820211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a7c0c92d0 msgr2=0x7f3a7c1e2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.641+0000 7f3a810a0640 1 --2- 172.21.15.100:0/2600820211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a7c0c92d0 0x7f3a7c1e2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.641+0000 7f3a810a0640 1 -- 172.21.15.100:0/2600820211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7c072d30 con 0x7f3a7c1522e0 2024-09-15T21:48:47.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.642+0000 7f3a810a0640 1 --2- 172.21.15.100:0/2600820211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c1522e0 0x7f3a7c1f0a90 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3a74034040 tx=0x7f3a74004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:47.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.642+0000 7f3a797fa640 1 -- 172.21.15.100:0/2600820211 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74041070 con 0x7f3a7c1522e0 2024-09-15T21:48:47.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.642+0000 7f3a797fa640 1 -- 172.21.15.100:0/2600820211 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a7403b830 con 0x7f3a7c1522e0 2024-09-15T21:48:47.646 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.642+0000 7f3a82b03640 1 -- 172.21.15.100:0/2600820211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7c157410 con 0x7f3a7c1522e0 2024-09-15T21:48:47.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.642+0000 7f3a797fa640 1 -- 172.21.15.100:0/2600820211 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74009510 con 0x7f3a7c1522e0 2024-09-15T21:48:47.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.642+0000 7f3a82b03640 1 -- 172.21.15.100:0/2600820211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7c1579d0 con 0x7f3a7c1522e0 2024-09-15T21:48:47.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.643+0000 7f3a82b03640 1 -- 172.21.15.100:0/2600820211 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a7c0c8740 con 0x7f3a7c1522e0 2024-09-15T21:48:47.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.643+0000 7f3a797fa640 1 -- 172.21.15.100:0/2600820211 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f3a74005d10 con 0x7f3a7c1522e0 2024-09-15T21:48:47.647 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.644+0000 7f3a797fa640 1 --2- 172.21.15.100:0/2600820211 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3a6006c2b0 0x7f3a6006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:48:47.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.644+0000 7f3a797fa640 1 -- 172.21.15.100:0/2600820211 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5857+0+0 (secure 0 0 0) 0x7f3a740aba80 con 0x7f3a7c1522e0 2024-09-15T21:48:47.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.644+0000 7f3a8089f640 1 --2- 172.21.15.100:0/2600820211 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3a6006c2b0 0x7f3a6006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:48:47.648 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.644+0000 7f3a8089f640 1 --2- 172.21.15.100:0/2600820211 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3a6006c2b0 0x7f3a6006e770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3a70000f60 tx=0x7f3a70009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:48:47.649 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.648+0000 7f3a797fa640 1 -- 172.21.15.100:0/2600820211 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f3a74079a70 con 0x7f3a7c1522e0 2024-09-15T21:48:47.814 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.812+0000 7f3a82b03640 1 -- 172.21.15.100:0/2600820211 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3a7c071580 con 0x7f3a6006c2b0 2024-09-15T21:48:47.819 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.818+0000 7f3a797fa640 1 -- 172.21.15.100:0/2600820211 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252509 (secure 0 0 0) 0x7f3a7c071580 con 0x7f3a6006c2b0 2024-09-15T21:48:47.822 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:48:47.823 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-15T21:48:47.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.821+0000 7f3a82b03640 1 -- 172.21.15.100:0/2600820211 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3a6006c2b0 msgr2=0x7f3a6006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.821+0000 7f3a82b03640 1 --2- 172.21.15.100:0/2600820211 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3a6006c2b0 0x7f3a6006e770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3a70000f60 tx=0x7f3a70009040 comp rx=0 tx=0).stop 2024-09-15T21:48:47.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.821+0000 7f3a82b03640 1 -- 172.21.15.100:0/2600820211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c1522e0 msgr2=0x7f3a7c1f0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:48:47.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.821+0000 7f3a82b03640 1 --2- 172.21.15.100:0/2600820211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c1522e0 0x7f3a7c1f0a90 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3a74034040 tx=0x7f3a74004a40 comp rx=0 tx=0).stop 2024-09-15T21:48:47.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.821+0000 7f3a82b03640 1 -- 172.21.15.100:0/2600820211 shutdown_connections 2024-09-15T21:48:47.825 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.821+0000 7f3a82b03640 1 --2- 172.21.15.100:0/2600820211 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f3a6006c2b0 0x7f3a6006e770 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.821+0000 7f3a82b03640 1 --2- 172.21.15.100:0/2600820211 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a7c1522e0 0x7f3a7c1f0a90 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.821+0000 7f3a82b03640 1 --2- 172.21.15.100:0/2600820211 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f3a7c0c96b0 0x7f3a7c14c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.821+0000 7f3a82b03640 1 --2- 172.21.15.100:0/2600820211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a7c0c92d0 0x7f3a7c1e2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:48:47.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.821+0000 7f3a82b03640 1 -- 172.21.15.100:0/2600820211 >> 172.21.15.100:0/2600820211 conn(0x7f3a7c067dd0 msgr2=0x7f3a7c073610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:48:47.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.821+0000 7f3a82b03640 1 -- 172.21.15.100:0/2600820211 shutdown_connections 2024-09-15T21:48:47.826 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:48:47.821+0000 7f3a82b03640 1 -- 172.21.15.100:0/2600820211 wait complete. 2024-09-15T21:48:47.834 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-15T21:48:45.833852+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633413,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":413,"num_write_kb":2538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1185538,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":842,"ondisk_log_size":842,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238224,"kb_used_data":8400,"kb_used_omap":18,"kb_used_meta":229677,"kb_avail":754736496,"statfs":{"total":773094113280,"available":772850171904,"internally_reserved":0,"allocated":8601600,"data_stored":4555026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19119,"internal_metadata":235189585},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":390,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002087"},"pg_stats":[{"pgid":"3.3f","version":"76'8","reported_seq":47,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.454919+0000","last_change":"2024-09-15T21:48:21.639361+0000","last_active":"2024-09-15T21:48:41.454919+0000","last_peered":"2024-09-15T21:48:41.454919+0000","last_clean":"2024-09-15T21:48:41.454919+0000","last_became_active":"2024-09-15T21:48:21.637402+0000","last_became_peered":"2024-09-15T21:48:21.637402+0000","last_unstale":"2024-09-15T21:48:41.454919+0000","last_undegraded":"2024-09-15T21:48:41.454919+0000","last_fullsized":"2024-09-15T21:48:41.454919+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:45:40.643861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"76'4","reported_seq":123,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.457366+0000","last_change":"2024-09-15T21:46:23.418716+0000","last_active":"2024-09-15T21:48:41.457366+0000","last_peered":"2024-09-15T21:48:41.457366+0000","last_clean":"2024-09-15T21:48:41.457366+0000","last_became_active":"2024-09-15T21:46:23.418471+0000","last_became_peered":"2024-09-15T21:46:23.418471+0000","last_unstale":"2024-09-15T21:48:41.457366+0000","last_undegraded":"2024-09-15T21:48:41.457366+0000","last_fullsized":"2024-09-15T21:48:41.457366+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:49.599801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"76'7","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.446202+0000","last_change":"2024-09-15T21:46:23.426404+0000","last_active":"2024-09-15T21:48:41.446202+0000","last_peered":"2024-09-15T21:48:41.446202+0000","last_clean":"2024-09-15T21:48:41.446202+0000","last_became_active":"2024-09-15T21:46:23.426146+0000","last_became_peered":"2024-09-15T21:46:23.426146+0000","last_unstale":"2024-09-15T21:48:41.446202+0000","last_undegraded":"2024-09-15T21:48:41.446202+0000","last_fullsized":"2024-09-15T21:48:41.446202+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:19:23.626109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"76'8","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.454789+0000","last_change":"2024-09-15T21:48:17.624363+0000","last_active":"2024-09-15T21:48:41.454789+0000","last_peered":"2024-09-15T21:48:41.454789+0000","last_clean":"2024-09-15T21:48:41.454789+0000","last_became_active":"2024-09-15T21:48:17.623606+0000","last_became_peered":"2024-09-15T21:48:17.623606+0000","last_unstale":"2024-09-15T21:48:41.454789+0000","last_undegraded":"2024-09-15T21:48:41.454789+0000","last_fullsized":"2024-09-15T21:48:41.454789+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:17:03.294215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"76'4","reported_seq":43,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.462247+0000","last_change":"2024-09-15T21:48:21.640215+0000","last_active":"2024-09-15T21:48:41.462247+0000","last_peered":"2024-09-15T21:48:41.462247+0000","last_clean":"2024-09-15T21:48:41.462247+0000","last_became_active":"2024-09-15T21:48:21.638410+0000","last_became_peered":"2024-09-15T21:48:21.638410+0000","last_unstale":"2024-09-15T21:48:41.462247+0000","last_undegraded":"2024-09-15T21:48:41.462247+0000","last_fullsized":"2024-09-15T21:48:41.462247+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:43.167624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"76'6","reported_seq":63,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.457720+0000","last_change":"2024-09-15T21:48:17.625573+0000","last_active":"2024-09-15T21:48:41.457720+0000","last_peered":"2024-09-15T21:48:41.457720+0000","last_clean":"2024-09-15T21:48:41.457720+0000","last_became_active":"2024-09-15T21:48:17.624439+0000","last_became_peered":"2024-09-15T21:48:17.624439+0000","last_unstale":"2024-09-15T21:48:41.457720+0000","last_undegraded":"2024-09-15T21:48:41.457720+0000","last_fullsized":"2024-09-15T21:48:41.457720+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:04.063675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"76'5","reported_seq":44,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.464903+0000","last_change":"2024-09-15T21:48:21.638612+0000","last_active":"2024-09-15T21:48:41.464903+0000","last_peered":"2024-09-15T21:48:41.464903+0000","last_clean":"2024-09-15T21:48:41.464903+0000","last_became_active":"2024-09-15T21:48:21.638310+0000","last_became_peered":"2024-09-15T21:48:21.638310+0000","last_unstale":"2024-09-15T21:48:41.464903+0000","last_undegraded":"2024-09-15T21:48:41.464903+0000","last_fullsized":"2024-09-15T21:48:41.464903+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:37.297103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"76'9","reported_seq":108,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.446611+0000","last_change":"2024-09-15T21:48:27.657047+0000","last_active":"2024-09-15T21:48:41.446611+0000","last_peered":"2024-09-15T21:48:41.446611+0000","last_clean":"2024-09-15T21:48:41.446611+0000","last_became_active":"2024-09-15T21:48:27.656800+0000","last_became_peered":"2024-09-15T21:48:27.656800+0000","last_unstale":"2024-09-15T21:48:41.446611+0000","last_undegraded":"2024-09-15T21:48:41.446611+0000","last_fullsized":"2024-09-15T21:48:41.446611+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:17.342504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"76'11","reported_seq":50,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.465415+0000","last_change":"2024-09-15T21:48:21.642296+0000","last_active":"2024-09-15T21:48:41.465415+0000","last_peered":"2024-09-15T21:48:41.465415+0000","last_clean":"2024-09-15T21:48:41.465415+0000","last_became_active":"2024-09-15T21:48:21.641920+0000","last_became_peered":"2024-09-15T21:48:21.641920+0000","last_unstale":"2024-09-15T21:48:41.465415+0000","last_undegraded":"2024-09-15T21:48:41.465415+0000","last_fullsized":"2024-09-15T21:48:41.465415+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:45.342068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"76'16","reported_seq":140,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.456553+0000","last_change":"2024-09-15T21:48:17.622439+0000","last_active":"2024-09-15T21:48:41.456553+0000","last_peered":"2024-09-15T21:48:41.456553+0000","last_clean":"2024-09-15T21:48:41.456553+0000","last_became_active":"2024-09-15T21:48:17.621800+0000","last_became_peered":"2024-09-15T21:48:17.621800+0000","last_unstale":"2024-09-15T21:48:41.456553+0000","last_undegraded":"2024-09-15T21:48:41.456553+0000","last_fullsized":"2024-09-15T21:48:41.456553+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:07.312656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"76'6","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.448946+0000","last_change":"2024-09-15T21:47:05.735261+0000","last_active":"2024-09-15T21:48:41.448946+0000","last_peered":"2024-09-15T21:48:41.448946+0000","last_clean":"2024-09-15T21:48:41.448946+0000","last_became_active":"2024-09-15T21:47:05.734921+0000","last_became_peered":"2024-09-15T21:47:05.734921+0000","last_unstale":"2024-09-15T21:48:41.448946+0000","last_undegraded":"2024-09-15T21:48:41.448946+0000","last_fullsized":"2024-09-15T21:48:41.448946+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:55.787926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"76'7","reported_seq":127,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.451499+0000","last_change":"2024-09-15T21:46:23.432534+0000","last_active":"2024-09-15T21:48:41.451499+0000","last_peered":"2024-09-15T21:48:41.451499+0000","last_clean":"2024-09-15T21:48:41.451499+0000","last_became_active":"2024-09-15T21:46:23.430170+0000","last_became_peered":"2024-09-15T21:46:23.430170+0000","last_unstale":"2024-09-15T21:48:41.451499+0000","last_undegraded":"2024-09-15T21:48:41.451499+0000","last_fullsized":"2024-09-15T21:48:41.451499+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:11.463937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"76'10","reported_seq":129,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.449784+0000","last_change":"2024-09-15T21:46:23.427841+0000","last_active":"2024-09-15T21:48:41.449784+0000","last_peered":"2024-09-15T21:48:41.449784+0000","last_clean":"2024-09-15T21:48:41.449784+0000","last_became_active":"2024-09-15T21:46:23.427621+0000","last_became_peered":"2024-09-15T21:46:23.427621+0000","last_unstale":"2024-09-15T21:48:41.449784+0000","last_undegraded":"2024-09-15T21:48:41.449784+0000","last_fullsized":"2024-09-15T21:48:41.449784+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:21:18.331943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"76'10","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.442612+0000","last_change":"2024-09-15T21:46:23.424707+0000","last_active":"2024-09-15T21:48:41.442612+0000","last_peered":"2024-09-15T21:48:41.442612+0000","last_clean":"2024-09-15T21:48:41.442612+0000","last_became_active":"2024-09-15T21:46:23.424447+0000","last_became_peered":"2024-09-15T21:46:23.424447+0000","last_unstale":"2024-09-15T21:48:41.442612+0000","last_undegraded":"2024-09-15T21:48:41.442612+0000","last_fullsized":"2024-09-15T21:48:41.442612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:23.617214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"76'7","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.459255+0000","last_change":"2024-09-15T21:46:23.430297+0000","last_active":"2024-09-15T21:48:41.459255+0000","last_peered":"2024-09-15T21:48:41.459255+0000","last_clean":"2024-09-15T21:48:41.459255+0000","last_became_active":"2024-09-15T21:46:23.430070+0000","last_became_peered":"2024-09-15T21:46:23.430070+0000","last_unstale":"2024-09-15T21:48:41.459255+0000","last_undegraded":"2024-09-15T21:48:41.459255+0000","last_fullsized":"2024-09-15T21:48:41.459255+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:03:19.615022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"76'9","reported_seq":66,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.457414+0000","last_change":"2024-09-15T21:48:17.625113+0000","last_active":"2024-09-15T21:48:41.457414+0000","last_peered":"2024-09-15T21:48:41.457414+0000","last_clean":"2024-09-15T21:48:41.457414+0000","last_became_active":"2024-09-15T21:48:17.624764+0000","last_became_peered":"2024-09-15T21:48:17.624764+0000","last_unstale":"2024-09-15T21:48:41.457414+0000","last_undegraded":"2024-09-15T21:48:41.457414+0000","last_fullsized":"2024-09-15T21:48:41.457414+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:28:44.142358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"76'8","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.460258+0000","last_change":"2024-09-15T21:48:21.642646+0000","last_active":"2024-09-15T21:48:41.460258+0000","last_peered":"2024-09-15T21:48:41.460258+0000","last_clean":"2024-09-15T21:48:41.460258+0000","last_became_active":"2024-09-15T21:48:21.642303+0000","last_became_peered":"2024-09-15T21:48:21.642303+0000","last_unstale":"2024-09-15T21:48:41.460258+0000","last_undegraded":"2024-09-15T21:48:41.460258+0000","last_fullsized":"2024-09-15T21:48:41.460258+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:08:54.511282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"76'10","reported_seq":86,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:42.209597+0000","last_change":"2024-09-15T21:48:27.661631+0000","last_active":"2024-09-15T21:48:42.209597+0000","last_peered":"2024-09-15T21:48:42.209597+0000","last_clean":"2024-09-15T21:48:42.209597+0000","last_became_active":"2024-09-15T21:48:27.661441+0000","last_became_peered":"2024-09-15T21:48:27.661441+0000","last_unstale":"2024-09-15T21:48:42.209597+0000","last_undegraded":"2024-09-15T21:48:42.209597+0000","last_fullsized":"2024-09-15T21:48:42.209597+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:29:23.865954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"76'7","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.462340+0000","last_change":"2024-09-15T21:48:21.639602+0000","last_active":"2024-09-15T21:48:41.462340+0000","last_peered":"2024-09-15T21:48:41.462340+0000","last_clean":"2024-09-15T21:48:41.462340+0000","last_became_active":"2024-09-15T21:48:21.639009+0000","last_became_peered":"2024-09-15T21:48:21.639009+0000","last_unstale":"2024-09-15T21:48:41.462340+0000","last_undegraded":"2024-09-15T21:48:41.462340+0000","last_fullsized":"2024-09-15T21:48:41.462340+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:38:16.818783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"76'8","reported_seq":126,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.459010+0000","last_change":"2024-09-15T21:46:23.434558+0000","last_active":"2024-09-15T21:48:41.459010+0000","last_peered":"2024-09-15T21:48:41.459010+0000","last_clean":"2024-09-15T21:48:41.459010+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:48:41.459010+0000","last_undegraded":"2024-09-15T21:48:41.459010+0000","last_fullsized":"2024-09-15T21:48:41.459010+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:03:39.562977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"76'11","reported_seq":139,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.461154+0000","last_change":"2024-09-15T21:48:21.642201+0000","last_active":"2024-09-15T21:48:41.461154+0000","last_peered":"2024-09-15T21:48:41.461154+0000","last_clean":"2024-09-15T21:48:41.461154+0000","last_became_active":"2024-09-15T21:48:21.641876+0000","last_became_peered":"2024-09-15T21:48:21.641876+0000","last_unstale":"2024-09-15T21:48:41.461154+0000","last_undegraded":"2024-09-15T21:48:41.461154+0000","last_fullsized":"2024-09-15T21:48:41.461154+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:41:59.764563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"76'10","reported_seq":69,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.460749+0000","last_change":"2024-09-15T21:48:21.638398+0000","last_active":"2024-09-15T21:48:41.460749+0000","last_peered":"2024-09-15T21:48:41.460749+0000","last_clean":"2024-09-15T21:48:41.460749+0000","last_became_active":"2024-09-15T21:48:21.638073+0000","last_became_peered":"2024-09-15T21:48:21.638073+0000","last_unstale":"2024-09-15T21:48:41.460749+0000","last_undegraded":"2024-09-15T21:48:41.460749+0000","last_fullsized":"2024-09-15T21:48:41.460749+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:16:45.308950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"76'8","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.446199+0000","last_change":"2024-09-15T21:46:23.431527+0000","last_active":"2024-09-15T21:48:41.446199+0000","last_peered":"2024-09-15T21:48:41.446199+0000","last_clean":"2024-09-15T21:48:41.446199+0000","last_became_active":"2024-09-15T21:46:23.430719+0000","last_became_peered":"2024-09-15T21:46:23.430719+0000","last_unstale":"2024-09-15T21:48:41.446199+0000","last_undegraded":"2024-09-15T21:48:41.446199+0000","last_fullsized":"2024-09-15T21:48:41.446199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:16.326529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"76'6","reported_seq":63,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.454926+0000","last_change":"2024-09-15T21:48:17.622578+0000","last_active":"2024-09-15T21:48:41.454926+0000","last_peered":"2024-09-15T21:48:41.454926+0000","last_clean":"2024-09-15T21:48:41.454926+0000","last_became_active":"2024-09-15T21:48:17.621639+0000","last_became_peered":"2024-09-15T21:48:17.621639+0000","last_unstale":"2024-09-15T21:48:41.454926+0000","last_undegraded":"2024-09-15T21:48:41.454926+0000","last_fullsized":"2024-09-15T21:48:41.454926+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:05:57.099730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"76'2","reported_seq":122,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.450303+0000","last_change":"2024-09-15T21:46:23.432646+0000","last_active":"2024-09-15T21:48:41.450303+0000","last_peered":"2024-09-15T21:48:41.450303+0000","last_clean":"2024-09-15T21:48:41.450303+0000","last_became_active":"2024-09-15T21:46:23.432317+0000","last_became_peered":"2024-09-15T21:46:23.432317+0000","last_unstale":"2024-09-15T21:48:41.450303+0000","last_undegraded":"2024-09-15T21:48:41.450303+0000","last_fullsized":"2024-09-15T21:48:41.450303+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:49:07.706450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"76'15","reported_seq":143,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.457343+0000","last_change":"2024-09-15T21:48:21.638549+0000","last_active":"2024-09-15T21:48:41.457343+0000","last_peered":"2024-09-15T21:48:41.457343+0000","last_clean":"2024-09-15T21:48:41.457343+0000","last_became_active":"2024-09-15T21:48:21.638212+0000","last_became_peered":"2024-09-15T21:48:21.638212+0000","last_unstale":"2024-09-15T21:48:41.457343+0000","last_undegraded":"2024-09-15T21:48:41.457343+0000","last_fullsized":"2024-09-15T21:48:41.457343+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:03:18.788661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"76'6","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.457376+0000","last_change":"2024-09-15T21:46:23.421749+0000","last_active":"2024-09-15T21:48:41.457376+0000","last_peered":"2024-09-15T21:48:41.457376+0000","last_clean":"2024-09-15T21:48:41.457376+0000","last_became_active":"2024-09-15T21:46:23.421501+0000","last_became_peered":"2024-09-15T21:46:23.421501+0000","last_unstale":"2024-09-15T21:48:41.457376+0000","last_undegraded":"2024-09-15T21:48:41.457376+0000","last_fullsized":"2024-09-15T21:48:41.457376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:55.728181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"76'4","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.450897+0000","last_change":"2024-09-15T21:46:23.432591+0000","last_active":"2024-09-15T21:48:41.450897+0000","last_peered":"2024-09-15T21:48:41.450897+0000","last_clean":"2024-09-15T21:48:41.450897+0000","last_became_active":"2024-09-15T21:46:23.432326+0000","last_became_peered":"2024-09-15T21:46:23.432326+0000","last_unstale":"2024-09-15T21:48:41.450897+0000","last_undegraded":"2024-09-15T21:48:41.450897+0000","last_fullsized":"2024-09-15T21:48:41.450897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:38:48.565579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"76'7","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.442747+0000","last_change":"2024-09-15T21:46:23.423428+0000","last_active":"2024-09-15T21:48:41.442747+0000","last_peered":"2024-09-15T21:48:41.442747+0000","last_clean":"2024-09-15T21:48:41.442747+0000","last_became_active":"2024-09-15T21:46:23.422734+0000","last_became_peered":"2024-09-15T21:46:23.422734+0000","last_unstale":"2024-09-15T21:48:41.442747+0000","last_undegraded":"2024-09-15T21:48:41.442747+0000","last_fullsized":"2024-09-15T21:48:41.442747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:48:06.592891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"76'8","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.454838+0000","last_change":"2024-09-15T21:48:17.622914+0000","last_active":"2024-09-15T21:48:41.454838+0000","last_peered":"2024-09-15T21:48:41.454838+0000","last_clean":"2024-09-15T21:48:41.454838+0000","last_became_active":"2024-09-15T21:48:17.621473+0000","last_became_peered":"2024-09-15T21:48:17.621473+0000","last_unstale":"2024-09-15T21:48:41.454838+0000","last_undegraded":"2024-09-15T21:48:41.454838+0000","last_fullsized":"2024-09-15T21:48:41.454838+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:36:45.297321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"76'12","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.441469+0000","last_change":"2024-09-15T21:48:17.616723+0000","last_active":"2024-09-15T21:48:41.441469+0000","last_peered":"2024-09-15T21:48:41.441469+0000","last_clean":"2024-09-15T21:48:41.441469+0000","last_became_active":"2024-09-15T21:48:17.616499+0000","last_became_peered":"2024-09-15T21:48:17.616499+0000","last_unstale":"2024-09-15T21:48:41.441469+0000","last_undegraded":"2024-09-15T21:48:41.441469+0000","last_fullsized":"2024-09-15T21:48:41.441469+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:29:36.886350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"76'9","reported_seq":127,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.453137+0000","last_change":"2024-09-15T21:46:23.429648+0000","last_active":"2024-09-15T21:48:41.453137+0000","last_peered":"2024-09-15T21:48:41.453137+0000","last_clean":"2024-09-15T21:48:41.453137+0000","last_became_active":"2024-09-15T21:46:23.429376+0000","last_became_peered":"2024-09-15T21:46:23.429376+0000","last_unstale":"2024-09-15T21:48:41.453137+0000","last_undegraded":"2024-09-15T21:48:41.453137+0000","last_fullsized":"2024-09-15T21:48:41.453137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:31.278283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"76'7","reported_seq":127,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.446357+0000","last_change":"2024-09-15T21:46:23.429847+0000","last_active":"2024-09-15T21:48:41.446357+0000","last_peered":"2024-09-15T21:48:41.446357+0000","last_clean":"2024-09-15T21:48:41.446357+0000","last_became_active":"2024-09-15T21:46:23.429442+0000","last_became_peered":"2024-09-15T21:46:23.429442+0000","last_unstale":"2024-09-15T21:48:41.446357+0000","last_undegraded":"2024-09-15T21:48:41.446357+0000","last_fullsized":"2024-09-15T21:48:41.446357+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:55:31.232512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"76'7","reported_seq":150,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387537+0000","last_change":"2024-09-15T21:48:21.665708+0000","last_active":"2024-09-15T21:48:38.387537+0000","last_peered":"2024-09-15T21:48:38.387537+0000","last_clean":"2024-09-15T21:48:38.387537+0000","last_became_active":"2024-09-15T21:48:21.636972+0000","last_became_peered":"2024-09-15T21:48:21.636972+0000","last_unstale":"2024-09-15T21:48:38.387537+0000","last_undegraded":"2024-09-15T21:48:38.387537+0000","last_fullsized":"2024-09-15T21:48:38.387537+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:24:33.658234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1019,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1756,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"76'8","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.450965+0000","last_change":"2024-09-15T21:46:23.434198+0000","last_active":"2024-09-15T21:48:41.450965+0000","last_peered":"2024-09-15T21:48:41.450965+0000","last_clean":"2024-09-15T21:48:41.450965+0000","last_became_active":"2024-09-15T21:46:23.433944+0000","last_became_peered":"2024-09-15T21:46:23.433944+0000","last_unstale":"2024-09-15T21:48:41.450965+0000","last_undegraded":"2024-09-15T21:48:41.450965+0000","last_fullsized":"2024-09-15T21:48:41.450965+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:03:25.412167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'7","reported_seq":126,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858780+0000","last_change":"2024-09-15T21:46:22.425571+0000","last_active":"2024-09-15T21:48:36.858780+0000","last_peered":"2024-09-15T21:48:36.858780+0000","last_clean":"2024-09-15T21:48:36.858780+0000","last_became_active":"2024-09-15T21:46:22.425315+0000","last_became_peered":"2024-09-15T21:46:22.425315+0000","last_unstale":"2024-09-15T21:48:36.858780+0000","last_undegraded":"2024-09-15T21:48:36.858780+0000","last_fullsized":"2024-09-15T21:48:36.858780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:19:31.983301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"22'3","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387375+0000","last_change":"2024-09-15T21:46:22.425492+0000","last_active":"2024-09-15T21:48:38.387375+0000","last_peered":"2024-09-15T21:48:38.387375+0000","last_clean":"2024-09-15T21:48:38.387375+0000","last_became_active":"2024-09-15T21:46:22.425226+0000","last_became_peered":"2024-09-15T21:46:22.425226+0000","last_unstale":"2024-09-15T21:48:38.387375+0000","last_undegraded":"2024-09-15T21:48:38.387375+0000","last_fullsized":"2024-09-15T21:48:38.387375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:53:24.322384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"76'8","reported_seq":127,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.446756+0000","last_change":"2024-09-15T21:46:23.429140+0000","last_active":"2024-09-15T21:48:41.446756+0000","last_peered":"2024-09-15T21:48:41.446756+0000","last_clean":"2024-09-15T21:48:41.446756+0000","last_became_active":"2024-09-15T21:46:23.428902+0000","last_became_peered":"2024-09-15T21:46:23.428902+0000","last_unstale":"2024-09-15T21:48:41.446756+0000","last_undegraded":"2024-09-15T21:48:41.446756+0000","last_fullsized":"2024-09-15T21:48:41.446756+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:50:11.256924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"22'3","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021677+0000","last_change":"2024-09-15T21:48:17.618127+0000","last_active":"2024-09-15T21:48:37.021677+0000","last_peered":"2024-09-15T21:48:37.021677+0000","last_clean":"2024-09-15T21:48:37.021677+0000","last_became_active":"2024-09-15T21:48:17.616704+0000","last_became_peered":"2024-09-15T21:48:17.616704+0000","last_unstale":"2024-09-15T21:48:37.021677+0000","last_undegraded":"2024-09-15T21:48:37.021677+0000","last_fullsized":"2024-09-15T21:48:37.021677+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:09.775586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"76'10","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.457810+0000","last_change":"2024-09-15T21:46:23.427442+0000","last_active":"2024-09-15T21:48:41.457810+0000","last_peered":"2024-09-15T21:48:41.457810+0000","last_clean":"2024-09-15T21:48:41.457810+0000","last_became_active":"2024-09-15T21:46:23.427200+0000","last_became_peered":"2024-09-15T21:46:23.427200+0000","last_unstale":"2024-09-15T21:48:41.457810+0000","last_undegraded":"2024-09-15T21:48:41.457810+0000","last_fullsized":"2024-09-15T21:48:41.457810+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:38:27.248545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"76'7","reported_seq":134,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859187+0000","last_change":"2024-09-15T21:48:17.622236+0000","last_active":"2024-09-15T21:48:36.859187+0000","last_peered":"2024-09-15T21:48:36.859187+0000","last_clean":"2024-09-15T21:48:36.859187+0000","last_became_active":"2024-09-15T21:48:17.621860+0000","last_became_peered":"2024-09-15T21:48:17.621860+0000","last_unstale":"2024-09-15T21:48:36.859187+0000","last_undegraded":"2024-09-15T21:48:36.859187+0000","last_fullsized":"2024-09-15T21:48:36.859187+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:00.032847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"76'10","reported_seq":69,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.460543+0000","last_change":"2024-09-15T21:48:21.638639+0000","last_active":"2024-09-15T21:48:41.460543+0000","last_peered":"2024-09-15T21:48:41.460543+0000","last_clean":"2024-09-15T21:48:41.460543+0000","last_became_active":"2024-09-15T21:48:21.638300+0000","last_became_peered":"2024-09-15T21:48:21.638300+0000","last_unstale":"2024-09-15T21:48:41.460543+0000","last_undegraded":"2024-09-15T21:48:41.460543+0000","last_fullsized":"2024-09-15T21:48:41.460543+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:09:01.739184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"22'4","reported_seq":65,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.022156+0000","last_change":"2024-09-15T21:48:17.623971+0000","last_active":"2024-09-15T21:48:37.022156+0000","last_peered":"2024-09-15T21:48:37.022156+0000","last_clean":"2024-09-15T21:48:37.022156+0000","last_became_active":"2024-09-15T21:48:17.623246+0000","last_became_peered":"2024-09-15T21:48:17.623246+0000","last_unstale":"2024-09-15T21:48:37.022156+0000","last_undegraded":"2024-09-15T21:48:37.022156+0000","last_fullsized":"2024-09-15T21:48:37.022156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:51:37.738581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"76'9","reported_seq":133,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.436624+0000","last_change":"2024-09-15T21:48:17.625879+0000","last_active":"2024-09-15T21:48:41.436624+0000","last_peered":"2024-09-15T21:48:41.436624+0000","last_clean":"2024-09-15T21:48:41.436624+0000","last_became_active":"2024-09-15T21:48:17.625642+0000","last_became_peered":"2024-09-15T21:48:17.625642+0000","last_unstale":"2024-09-15T21:48:41.436624+0000","last_undegraded":"2024-09-15T21:48:41.436624+0000","last_fullsized":"2024-09-15T21:48:41.436624+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:41.396163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"55'7","reported_seq":139,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726301+0000","last_change":"2024-09-15T21:48:17.627957+0000","last_active":"2024-09-15T21:48:36.726301+0000","last_peered":"2024-09-15T21:48:36.726301+0000","last_clean":"2024-09-15T21:48:36.726301+0000","last_became_active":"2024-09-15T21:48:17.622409+0000","last_became_peered":"2024-09-15T21:48:17.622409+0000","last_unstale":"2024-09-15T21:48:36.726301+0000","last_undegraded":"2024-09-15T21:48:36.726301+0000","last_fullsized":"2024-09-15T21:48:36.726301+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:01:39.534804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"76'11","reported_seq":142,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.457231+0000","last_change":"2024-09-15T21:48:21.637218+0000","last_active":"2024-09-15T21:48:41.457231+0000","last_peered":"2024-09-15T21:48:41.457231+0000","last_clean":"2024-09-15T21:48:41.457231+0000","last_became_active":"2024-09-15T21:48:21.636913+0000","last_became_peered":"2024-09-15T21:48:21.636913+0000","last_unstale":"2024-09-15T21:48:41.457231+0000","last_undegraded":"2024-09-15T21:48:41.457231+0000","last_fullsized":"2024-09-15T21:48:41.457231+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:36:05.649762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"76'27","reported_seq":147,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:42.211193+0000","last_change":"2024-09-15T21:46:22.426151+0000","last_active":"2024-09-15T21:48:42.211193+0000","last_peered":"2024-09-15T21:48:42.211193+0000","last_clean":"2024-09-15T21:48:42.211193+0000","last_became_active":"2024-09-15T21:46:22.425866+0000","last_became_peered":"2024-09-15T21:46:22.425866+0000","last_unstale":"2024-09-15T21:48:42.211193+0000","last_undegraded":"2024-09-15T21:48:42.211193+0000","last_fullsized":"2024-09-15T21:48:42.211193+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:00:10.684265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35822,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"76'6","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.457188+0000","last_change":"2024-09-15T21:48:21.637466+0000","last_active":"2024-09-15T21:48:41.457188+0000","last_peered":"2024-09-15T21:48:41.457188+0000","last_clean":"2024-09-15T21:48:41.457188+0000","last_became_active":"2024-09-15T21:48:21.637151+0000","last_became_peered":"2024-09-15T21:48:21.637151+0000","last_unstale":"2024-09-15T21:48:41.457188+0000","last_undegraded":"2024-09-15T21:48:41.457188+0000","last_fullsized":"2024-09-15T21:48:41.457188+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:22:57.287004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'9","reported_seq":129,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.727202+0000","last_change":"2024-09-15T21:46:22.427415+0000","last_active":"2024-09-15T21:48:36.727202+0000","last_peered":"2024-09-15T21:48:36.727202+0000","last_clean":"2024-09-15T21:48:36.727202+0000","last_became_active":"2024-09-15T21:46:22.427087+0000","last_became_peered":"2024-09-15T21:46:22.427087+0000","last_unstale":"2024-09-15T21:48:36.727202+0000","last_undegraded":"2024-09-15T21:48:36.727202+0000","last_fullsized":"2024-09-15T21:48:36.727202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:42:09.365202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"76'9","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.461043+0000","last_change":"2024-09-15T21:48:21.635299+0000","last_active":"2024-09-15T21:48:41.461043+0000","last_peered":"2024-09-15T21:48:41.461043+0000","last_clean":"2024-09-15T21:48:41.461043+0000","last_became_active":"2024-09-15T21:48:21.635022+0000","last_became_peered":"2024-09-15T21:48:21.635022+0000","last_unstale":"2024-09-15T21:48:41.461043+0000","last_undegraded":"2024-09-15T21:48:41.461043+0000","last_fullsized":"2024-09-15T21:48:41.461043+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:01.482588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"55'6","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021757+0000","last_change":"2024-09-15T21:48:17.625975+0000","last_active":"2024-09-15T21:48:37.021757+0000","last_peered":"2024-09-15T21:48:37.021757+0000","last_clean":"2024-09-15T21:48:37.021757+0000","last_became_active":"2024-09-15T21:48:17.625232+0000","last_became_peered":"2024-09-15T21:48:17.625232+0000","last_unstale":"2024-09-15T21:48:37.021757+0000","last_undegraded":"2024-09-15T21:48:37.021757+0000","last_fullsized":"2024-09-15T21:48:37.021757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:09:21.061733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"76'15","reported_seq":54,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.456206+0000","last_change":"2024-09-15T21:48:21.639654+0000","last_active":"2024-09-15T21:48:41.456206+0000","last_peered":"2024-09-15T21:48:41.456206+0000","last_clean":"2024-09-15T21:48:41.456206+0000","last_became_active":"2024-09-15T21:48:21.638780+0000","last_became_peered":"2024-09-15T21:48:21.638780+0000","last_unstale":"2024-09-15T21:48:41.456206+0000","last_undegraded":"2024-09-15T21:48:41.456206+0000","last_fullsized":"2024-09-15T21:48:41.456206+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:16.549734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"22'5","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.858720+0000","last_change":"2024-09-15T21:46:22.425517+0000","last_active":"2024-09-15T21:48:36.858720+0000","last_peered":"2024-09-15T21:48:36.858720+0000","last_clean":"2024-09-15T21:48:36.858720+0000","last_became_active":"2024-09-15T21:46:22.425237+0000","last_became_peered":"2024-09-15T21:46:22.425237+0000","last_unstale":"2024-09-15T21:48:36.858720+0000","last_undegraded":"2024-09-15T21:48:36.858720+0000","last_fullsized":"2024-09-15T21:48:36.858720+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:59.460420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"76'6","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.448829+0000","last_change":"2024-09-15T21:46:23.427515+0000","last_active":"2024-09-15T21:48:41.448829+0000","last_peered":"2024-09-15T21:48:41.448829+0000","last_clean":"2024-09-15T21:48:41.448829+0000","last_became_active":"2024-09-15T21:46:23.427116+0000","last_became_peered":"2024-09-15T21:46:23.427116+0000","last_unstale":"2024-09-15T21:48:41.448829+0000","last_undegraded":"2024-09-15T21:48:41.448829+0000","last_fullsized":"2024-09-15T21:48:41.448829+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:35.499970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"55'18","reported_seq":138,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850656+0000","last_change":"2024-09-15T21:46:22.419905+0000","last_active":"2024-09-15T21:48:36.850656+0000","last_peered":"2024-09-15T21:48:36.850656+0000","last_clean":"2024-09-15T21:48:36.850656+0000","last_became_active":"2024-09-15T21:46:22.419620+0000","last_became_peered":"2024-09-15T21:46:22.419620+0000","last_unstale":"2024-09-15T21:48:36.850656+0000","last_undegraded":"2024-09-15T21:48:36.850656+0000","last_fullsized":"2024-09-15T21:48:36.850656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:26:51.215156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"76'6","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.455276+0000","last_change":"2024-09-15T21:48:21.639146+0000","last_active":"2024-09-15T21:48:41.455276+0000","last_peered":"2024-09-15T21:48:41.455276+0000","last_clean":"2024-09-15T21:48:41.455276+0000","last_became_active":"2024-09-15T21:48:21.637855+0000","last_became_peered":"2024-09-15T21:48:21.637855+0000","last_unstale":"2024-09-15T21:48:41.455276+0000","last_undegraded":"2024-09-15T21:48:41.455276+0000","last_fullsized":"2024-09-15T21:48:41.455276+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:35:29.305298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'2","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726069+0000","last_change":"2024-09-15T21:46:22.429661+0000","last_active":"2024-09-15T21:48:36.726069+0000","last_peered":"2024-09-15T21:48:36.726069+0000","last_clean":"2024-09-15T21:48:36.726069+0000","last_became_active":"2024-09-15T21:46:22.429250+0000","last_became_peered":"2024-09-15T21:46:22.429250+0000","last_unstale":"2024-09-15T21:48:36.726069+0000","last_undegraded":"2024-09-15T21:48:36.726069+0000","last_fullsized":"2024-09-15T21:48:36.726069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:29.341739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"76'5","reported_seq":44,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.454911+0000","last_change":"2024-09-15T21:48:21.635384+0000","last_active":"2024-09-15T21:48:41.454911+0000","last_peered":"2024-09-15T21:48:41.454911+0000","last_clean":"2024-09-15T21:48:41.454911+0000","last_became_active":"2024-09-15T21:48:21.635051+0000","last_became_peered":"2024-09-15T21:48:21.635051+0000","last_unstale":"2024-09-15T21:48:41.454911+0000","last_undegraded":"2024-09-15T21:48:41.454911+0000","last_fullsized":"2024-09-15T21:48:41.454911+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:59.199568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'5","reported_seq":83,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850941+0000","last_change":"2024-09-15T21:48:27.660076+0000","last_active":"2024-09-15T21:48:36.850941+0000","last_peered":"2024-09-15T21:48:36.850941+0000","last_clean":"2024-09-15T21:48:36.850941+0000","last_became_active":"2024-09-15T21:48:27.659750+0000","last_became_peered":"2024-09-15T21:48:27.659750+0000","last_unstale":"2024-09-15T21:48:36.850941+0000","last_undegraded":"2024-09-15T21:48:36.850941+0000","last_fullsized":"2024-09-15T21:48:36.850941+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:06.505112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"76'9","reported_seq":48,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.455712+0000","last_change":"2024-09-15T21:48:21.641556+0000","last_active":"2024-09-15T21:48:41.455712+0000","last_peered":"2024-09-15T21:48:41.455712+0000","last_clean":"2024-09-15T21:48:41.455712+0000","last_became_active":"2024-09-15T21:48:21.639328+0000","last_became_peered":"2024-09-15T21:48:21.639328+0000","last_unstale":"2024-09-15T21:48:41.455712+0000","last_undegraded":"2024-09-15T21:48:41.455712+0000","last_fullsized":"2024-09-15T21:48:41.455712+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:36:11.470048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'3","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726437+0000","last_change":"2024-09-15T21:46:22.425072+0000","last_active":"2024-09-15T21:48:36.726437+0000","last_peered":"2024-09-15T21:48:36.726437+0000","last_clean":"2024-09-15T21:48:36.726437+0000","last_became_active":"2024-09-15T21:46:22.424771+0000","last_became_peered":"2024-09-15T21:46:22.424771+0000","last_unstale":"2024-09-15T21:48:36.726437+0000","last_undegraded":"2024-09-15T21:48:36.726437+0000","last_fullsized":"2024-09-15T21:48:36.726437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:26.256785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"76'6","reported_seq":63,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.454532+0000","last_change":"2024-09-15T21:48:17.619698+0000","last_active":"2024-09-15T21:48:41.454532+0000","last_peered":"2024-09-15T21:48:41.454532+0000","last_clean":"2024-09-15T21:48:41.454532+0000","last_became_active":"2024-09-15T21:48:17.617701+0000","last_became_peered":"2024-09-15T21:48:17.617701+0000","last_unstale":"2024-09-15T21:48:41.454532+0000","last_undegraded":"2024-09-15T21:48:41.454532+0000","last_fullsized":"2024-09-15T21:48:41.454532+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:21:00.170687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"22'5","reported_seq":132,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850794+0000","last_change":"2024-09-15T21:48:17.619089+0000","last_active":"2024-09-15T21:48:36.850794+0000","last_peered":"2024-09-15T21:48:36.850794+0000","last_clean":"2024-09-15T21:48:36.850794+0000","last_became_active":"2024-09-15T21:48:17.618782+0000","last_became_peered":"2024-09-15T21:48:17.618782+0000","last_unstale":"2024-09-15T21:48:36.850794+0000","last_undegraded":"2024-09-15T21:48:36.850794+0000","last_fullsized":"2024-09-15T21:48:36.850794+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:46:17.881528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"76'7","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.465233+0000","last_change":"2024-09-15T21:48:21.639356+0000","last_active":"2024-09-15T21:48:41.465233+0000","last_peered":"2024-09-15T21:48:41.465233+0000","last_clean":"2024-09-15T21:48:41.465233+0000","last_became_active":"2024-09-15T21:48:21.637883+0000","last_became_peered":"2024-09-15T21:48:21.637883+0000","last_unstale":"2024-09-15T21:48:41.465233+0000","last_undegraded":"2024-09-15T21:48:41.465233+0000","last_fullsized":"2024-09-15T21:48:41.465233+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:27.061996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":238,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.612519+0000","last_change":"2024-09-15T21:48:24.647129+0000","last_active":"2024-09-15T21:48:38.612519+0000","last_peered":"2024-09-15T21:48:38.612519+0000","last_clean":"2024-09-15T21:48:38.612519+0000","last_became_active":"2024-09-15T21:48:24.646783+0000","last_became_peered":"2024-09-15T21:48:24.646783+0000","last_unstale":"2024-09-15T21:48:38.612519+0000","last_undegraded":"2024-09-15T21:48:38.612519+0000","last_fullsized":"2024-09-15T21:48:38.612519+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_clean_scrub_stamp":"2024-09-15T21:46:12.223400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:11:20.467980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"22'6","reported_seq":68,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021854+0000","last_change":"2024-09-15T21:48:17.622619+0000","last_active":"2024-09-15T21:48:37.021854+0000","last_peered":"2024-09-15T21:48:37.021854+0000","last_clean":"2024-09-15T21:48:37.021854+0000","last_became_active":"2024-09-15T21:48:17.622159+0000","last_became_peered":"2024-09-15T21:48:17.622159+0000","last_unstale":"2024-09-15T21:48:37.021854+0000","last_undegraded":"2024-09-15T21:48:37.021854+0000","last_fullsized":"2024-09-15T21:48:37.021854+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:14:51.566702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"76'8","reported_seq":135,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.455877+0000","last_change":"2024-09-15T21:48:17.621009+0000","last_active":"2024-09-15T21:48:41.455877+0000","last_peered":"2024-09-15T21:48:41.455877+0000","last_clean":"2024-09-15T21:48:41.455877+0000","last_became_active":"2024-09-15T21:48:17.620738+0000","last_became_peered":"2024-09-15T21:48:17.620738+0000","last_unstale":"2024-09-15T21:48:41.455877+0000","last_undegraded":"2024-09-15T21:48:41.455877+0000","last_fullsized":"2024-09-15T21:48:41.455877+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:20.095876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"22'4","reported_seq":43,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.611793+0000","last_change":"2024-09-15T21:48:21.645645+0000","last_active":"2024-09-15T21:48:38.611793+0000","last_peered":"2024-09-15T21:48:38.611793+0000","last_clean":"2024-09-15T21:48:38.611793+0000","last_became_active":"2024-09-15T21:48:21.636886+0000","last_became_peered":"2024-09-15T21:48:21.636886+0000","last_unstale":"2024-09-15T21:48:38.611793+0000","last_undegraded":"2024-09-15T21:48:38.611793+0000","last_fullsized":"2024-09-15T21:48:38.611793+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:18:12.676508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"76'5","reported_seq":123,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.445112+0000","last_change":"2024-09-15T21:46:23.430396+0000","last_active":"2024-09-15T21:48:41.445112+0000","last_peered":"2024-09-15T21:48:41.445112+0000","last_clean":"2024-09-15T21:48:41.445112+0000","last_became_active":"2024-09-15T21:46:23.430138+0000","last_became_peered":"2024-09-15T21:46:23.430138+0000","last_unstale":"2024-09-15T21:48:41.445112+0000","last_undegraded":"2024-09-15T21:48:41.445112+0000","last_fullsized":"2024-09-15T21:48:41.445112+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:27:50.622949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"55'7","reported_seq":80,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.850706+0000","last_change":"2024-09-15T21:48:27.660216+0000","last_active":"2024-09-15T21:48:36.850706+0000","last_peered":"2024-09-15T21:48:36.850706+0000","last_clean":"2024-09-15T21:48:36.850706+0000","last_became_active":"2024-09-15T21:48:27.659880+0000","last_became_peered":"2024-09-15T21:48:27.659880+0000","last_unstale":"2024-09-15T21:48:36.850706+0000","last_undegraded":"2024-09-15T21:48:36.850706+0000","last_fullsized":"2024-09-15T21:48:36.850706+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:06:48.186260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":516,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'8","reported_seq":127,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.457142+0000","last_change":"2024-09-15T21:46:23.427051+0000","last_active":"2024-09-15T21:48:41.457142+0000","last_peered":"2024-09-15T21:48:41.457142+0000","last_clean":"2024-09-15T21:48:41.457142+0000","last_became_active":"2024-09-15T21:46:23.426185+0000","last_became_peered":"2024-09-15T21:46:23.426185+0000","last_unstale":"2024-09-15T21:48:41.457142+0000","last_undegraded":"2024-09-15T21:48:41.457142+0000","last_fullsized":"2024-09-15T21:48:41.457142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:27:48.318405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'5","reported_seq":67,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:37.021901+0000","last_change":"2024-09-15T21:48:17.625702+0000","last_active":"2024-09-15T21:48:37.021901+0000","last_peered":"2024-09-15T21:48:37.021901+0000","last_clean":"2024-09-15T21:48:37.021901+0000","last_became_active":"2024-09-15T21:48:17.623936+0000","last_became_peered":"2024-09-15T21:48:17.623936+0000","last_unstale":"2024-09-15T21:48:37.021901+0000","last_undegraded":"2024-09-15T21:48:37.021901+0000","last_fullsized":"2024-09-15T21:48:37.021901+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:53.114831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"76'7","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.444539+0000","last_change":"2024-09-15T21:46:23.425699+0000","last_active":"2024-09-15T21:48:41.444539+0000","last_peered":"2024-09-15T21:48:41.444539+0000","last_clean":"2024-09-15T21:48:41.444539+0000","last_became_active":"2024-09-15T21:46:23.425416+0000","last_became_peered":"2024-09-15T21:46:23.425416+0000","last_unstale":"2024-09-15T21:48:41.444539+0000","last_undegraded":"2024-09-15T21:48:41.444539+0000","last_fullsized":"2024-09-15T21:48:41.444539+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:24:00.325179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"22'1","reported_seq":122,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726645+0000","last_change":"2024-09-15T21:46:22.426492+0000","last_active":"2024-09-15T21:48:36.726645+0000","last_peered":"2024-09-15T21:48:36.726645+0000","last_clean":"2024-09-15T21:48:36.726645+0000","last_became_active":"2024-09-15T21:46:22.426210+0000","last_became_peered":"2024-09-15T21:46:22.426210+0000","last_unstale":"2024-09-15T21:48:36.726645+0000","last_undegraded":"2024-09-15T21:48:36.726645+0000","last_fullsized":"2024-09-15T21:48:36.726645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:46:05.248091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"76'5","reported_seq":62,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.454158+0000","last_change":"2024-09-15T21:48:17.625501+0000","last_active":"2024-09-15T21:48:41.454158+0000","last_peered":"2024-09-15T21:48:41.454158+0000","last_clean":"2024-09-15T21:48:41.454158+0000","last_became_active":"2024-09-15T21:48:17.624852+0000","last_became_peered":"2024-09-15T21:48:17.624852+0000","last_unstale":"2024-09-15T21:48:41.454158+0000","last_undegraded":"2024-09-15T21:48:41.454158+0000","last_fullsized":"2024-09-15T21:48:41.454158+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:47:09.731108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"22'4","reported_seq":47,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.612123+0000","last_change":"2024-09-15T21:48:21.668174+0000","last_active":"2024-09-15T21:48:38.612123+0000","last_peered":"2024-09-15T21:48:38.612123+0000","last_clean":"2024-09-15T21:48:38.612123+0000","last_became_active":"2024-09-15T21:48:21.639593+0000","last_became_peered":"2024-09-15T21:48:21.639593+0000","last_unstale":"2024-09-15T21:48:38.612123+0000","last_undegraded":"2024-09-15T21:48:38.612123+0000","last_fullsized":"2024-09-15T21:48:38.612123+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:40:05.341807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"76'9","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.457586+0000","last_change":"2024-09-15T21:46:23.426729+0000","last_active":"2024-09-15T21:48:41.457586+0000","last_peered":"2024-09-15T21:48:41.457586+0000","last_clean":"2024-09-15T21:48:41.457586+0000","last_became_active":"2024-09-15T21:46:23.426495+0000","last_became_peered":"2024-09-15T21:46:23.426495+0000","last_unstale":"2024-09-15T21:48:41.457586+0000","last_undegraded":"2024-09-15T21:48:41.457586+0000","last_fullsized":"2024-09-15T21:48:41.457586+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:44:03.277997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"76'17","reported_seq":138,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726265+0000","last_change":"2024-09-15T21:46:22.427137+0000","last_active":"2024-09-15T21:48:36.726265+0000","last_peered":"2024-09-15T21:48:36.726265+0000","last_clean":"2024-09-15T21:48:36.726265+0000","last_became_active":"2024-09-15T21:46:22.426855+0000","last_became_peered":"2024-09-15T21:46:22.426855+0000","last_unstale":"2024-09-15T21:48:36.726265+0000","last_undegraded":"2024-09-15T21:48:36.726265+0000","last_fullsized":"2024-09-15T21:48:36.726265+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:40:14.277940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"76'5","reported_seq":135,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.455647+0000","last_change":"2024-09-15T21:48:21.639860+0000","last_active":"2024-09-15T21:48:41.455647+0000","last_peered":"2024-09-15T21:48:41.455647+0000","last_clean":"2024-09-15T21:48:41.455647+0000","last_became_active":"2024-09-15T21:48:21.639541+0000","last_became_peered":"2024-09-15T21:48:21.639541+0000","last_unstale":"2024-09-15T21:48:41.455647+0000","last_undegraded":"2024-09-15T21:48:41.455647+0000","last_fullsized":"2024-09-15T21:48:41.455647+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:45:21.797034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"55'8","reported_seq":158,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.859109+0000","last_change":"2024-09-15T21:48:21.687672+0000","last_active":"2024-09-15T21:48:36.859109+0000","last_peered":"2024-09-15T21:48:36.859109+0000","last_clean":"2024-09-15T21:48:36.859109+0000","last_became_active":"2024-09-15T21:48:21.643251+0000","last_became_peered":"2024-09-15T21:48:21.643251+0000","last_unstale":"2024-09-15T21:48:36.859109+0000","last_undegraded":"2024-09-15T21:48:36.859109+0000","last_fullsized":"2024-09-15T21:48:36.859109+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:40.697954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":44,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"76'7","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.442631+0000","last_change":"2024-09-15T21:46:23.424266+0000","last_active":"2024-09-15T21:48:41.442631+0000","last_peered":"2024-09-15T21:48:41.442631+0000","last_clean":"2024-09-15T21:48:41.442631+0000","last_became_active":"2024-09-15T21:46:23.424007+0000","last_became_peered":"2024-09-15T21:46:23.424007+0000","last_unstale":"2024-09-15T21:48:41.442631+0000","last_undegraded":"2024-09-15T21:48:41.442631+0000","last_fullsized":"2024-09-15T21:48:41.442631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:56:23.492253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"22'2","reported_seq":135,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726262+0000","last_change":"2024-09-15T21:48:17.623800+0000","last_active":"2024-09-15T21:48:36.726262+0000","last_peered":"2024-09-15T21:48:36.726262+0000","last_clean":"2024-09-15T21:48:36.726262+0000","last_became_active":"2024-09-15T21:48:17.623453+0000","last_became_peered":"2024-09-15T21:48:17.623453+0000","last_unstale":"2024-09-15T21:48:36.726262+0000","last_undegraded":"2024-09-15T21:48:36.726262+0000","last_fullsized":"2024-09-15T21:48:36.726262+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:46:38.897647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"76'2","reported_seq":123,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.446764+0000","last_change":"2024-09-15T21:46:23.430836+0000","last_active":"2024-09-15T21:48:41.446764+0000","last_peered":"2024-09-15T21:48:41.446764+0000","last_clean":"2024-09-15T21:48:41.446764+0000","last_became_active":"2024-09-15T21:46:23.429283+0000","last_became_peered":"2024-09-15T21:46:23.429283+0000","last_unstale":"2024-09-15T21:48:41.446764+0000","last_undegraded":"2024-09-15T21:48:41.446764+0000","last_fullsized":"2024-09-15T21:48:41.446764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:04:30.571376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"22'7","reported_seq":158,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726054+0000","last_change":"2024-09-15T21:48:21.660402+0000","last_active":"2024-09-15T21:48:36.726054+0000","last_peered":"2024-09-15T21:48:36.726054+0000","last_clean":"2024-09-15T21:48:36.726054+0000","last_became_active":"2024-09-15T21:48:21.635898+0000","last_became_peered":"2024-09-15T21:48:21.635898+0000","last_unstale":"2024-09-15T21:48:36.726054+0000","last_undegraded":"2024-09-15T21:48:36.726054+0000","last_fullsized":"2024-09-15T21:48:36.726054+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:29.173360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":180,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"76'9","reported_seq":129,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.450531+0000","last_change":"2024-09-15T21:46:23.428634+0000","last_active":"2024-09-15T21:48:41.450531+0000","last_peered":"2024-09-15T21:48:41.450531+0000","last_clean":"2024-09-15T21:48:41.450531+0000","last_became_active":"2024-09-15T21:46:23.428301+0000","last_became_peered":"2024-09-15T21:46:23.428301+0000","last_unstale":"2024-09-15T21:48:41.450531+0000","last_undegraded":"2024-09-15T21:48:41.450531+0000","last_fullsized":"2024-09-15T21:48:41.450531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:59.492942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"76'5","reported_seq":126,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726666+0000","last_change":"2024-09-15T21:46:22.425810+0000","last_active":"2024-09-15T21:48:36.726666+0000","last_peered":"2024-09-15T21:48:36.726666+0000","last_clean":"2024-09-15T21:48:36.726666+0000","last_became_active":"2024-09-15T21:46:22.425478+0000","last_became_peered":"2024-09-15T21:46:22.425478+0000","last_unstale":"2024-09-15T21:48:36.726666+0000","last_undegraded":"2024-09-15T21:48:36.726666+0000","last_fullsized":"2024-09-15T21:48:36.726666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:13:53.296599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"76'8","reported_seq":128,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.445537+0000","last_change":"2024-09-15T21:46:23.426054+0000","last_active":"2024-09-15T21:48:41.445537+0000","last_peered":"2024-09-15T21:48:41.445537+0000","last_clean":"2024-09-15T21:48:41.445537+0000","last_became_active":"2024-09-15T21:46:23.425547+0000","last_became_peered":"2024-09-15T21:46:23.425547+0000","last_unstale":"2024-09-15T21:48:41.445537+0000","last_undegraded":"2024-09-15T21:48:41.445537+0000","last_fullsized":"2024-09-15T21:48:41.445537+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:58:56.838503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"55'5","reported_seq":45,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.611899+0000","last_change":"2024-09-15T21:48:21.653804+0000","last_active":"2024-09-15T21:48:38.611899+0000","last_peered":"2024-09-15T21:48:38.611899+0000","last_clean":"2024-09-15T21:48:38.611899+0000","last_became_active":"2024-09-15T21:48:21.639223+0000","last_became_peered":"2024-09-15T21:48:21.639223+0000","last_unstale":"2024-09-15T21:48:38.611899+0000","last_undegraded":"2024-09-15T21:48:38.611899+0000","last_fullsized":"2024-09-15T21:48:38.611899+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:47:15.330440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"76'5","reported_seq":123,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.457827+0000","last_change":"2024-09-15T21:46:23.434719+0000","last_active":"2024-09-15T21:48:41.457827+0000","last_peered":"2024-09-15T21:48:41.457827+0000","last_clean":"2024-09-15T21:48:41.457827+0000","last_became_active":"2024-09-15T21:46:23.434448+0000","last_became_peered":"2024-09-15T21:46:23.434448+0000","last_unstale":"2024-09-15T21:48:41.457827+0000","last_undegraded":"2024-09-15T21:48:41.457827+0000","last_fullsized":"2024-09-15T21:48:41.457827+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:14.901671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"76'15","reported_seq":83,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.453545+0000","last_change":"2024-09-15T21:48:27.657040+0000","last_active":"2024-09-15T21:48:41.453545+0000","last_peered":"2024-09-15T21:48:41.453545+0000","last_clean":"2024-09-15T21:48:41.453545+0000","last_became_active":"2024-09-15T21:48:27.656753+0000","last_became_peered":"2024-09-15T21:48:27.656753+0000","last_unstale":"2024-09-15T21:48:41.453545+0000","last_undegraded":"2024-09-15T21:48:41.453545+0000","last_fullsized":"2024-09-15T21:48:41.453545+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:02:14.458723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'5","reported_seq":47,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.611895+0000","last_change":"2024-09-15T21:48:21.691408+0000","last_active":"2024-09-15T21:48:38.611895+0000","last_peered":"2024-09-15T21:48:38.611895+0000","last_clean":"2024-09-15T21:48:38.611895+0000","last_became_active":"2024-09-15T21:48:21.641517+0000","last_became_peered":"2024-09-15T21:48:21.641517+0000","last_unstale":"2024-09-15T21:48:38.611895+0000","last_undegraded":"2024-09-15T21:48:38.611895+0000","last_fullsized":"2024-09-15T21:48:38.611895+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:31:09.454433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"76'9","reported_seq":66,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.453694+0000","last_change":"2024-09-15T21:48:17.623223+0000","last_active":"2024-09-15T21:48:41.453694+0000","last_peered":"2024-09-15T21:48:41.453694+0000","last_clean":"2024-09-15T21:48:41.453694+0000","last_became_active":"2024-09-15T21:48:17.622019+0000","last_became_peered":"2024-09-15T21:48:17.622019+0000","last_unstale":"2024-09-15T21:48:41.453694+0000","last_undegraded":"2024-09-15T21:48:41.453694+0000","last_fullsized":"2024-09-15T21:48:41.453694+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:39:57.643793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"55'13","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.612166+0000","last_change":"2024-09-15T21:48:21.682609+0000","last_active":"2024-09-15T21:48:38.612166+0000","last_peered":"2024-09-15T21:48:38.612166+0000","last_clean":"2024-09-15T21:48:38.612166+0000","last_became_active":"2024-09-15T21:48:21.640156+0000","last_became_peered":"2024-09-15T21:48:21.640156+0000","last_unstale":"2024-09-15T21:48:38.612166+0000","last_undegraded":"2024-09-15T21:48:38.612166+0000","last_fullsized":"2024-09-15T21:48:38.612166+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:05:28.936509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"76'6","reported_seq":131,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.439398+0000","last_change":"2024-09-15T21:48:17.623164+0000","last_active":"2024-09-15T21:48:41.439398+0000","last_peered":"2024-09-15T21:48:41.439398+0000","last_clean":"2024-09-15T21:48:41.439398+0000","last_became_active":"2024-09-15T21:48:17.622843+0000","last_became_peered":"2024-09-15T21:48:17.622843+0000","last_unstale":"2024-09-15T21:48:41.439398+0000","last_undegraded":"2024-09-15T21:48:41.439398+0000","last_fullsized":"2024-09-15T21:48:41.439398+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:06:30.745189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'2","reported_seq":124,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.387645+0000","last_change":"2024-09-15T21:46:22.428529+0000","last_active":"2024-09-15T21:48:38.387645+0000","last_peered":"2024-09-15T21:48:38.387645+0000","last_clean":"2024-09-15T21:48:38.387645+0000","last_became_active":"2024-09-15T21:46:22.428263+0000","last_became_peered":"2024-09-15T21:46:22.428263+0000","last_unstale":"2024-09-15T21:48:38.387645+0000","last_undegraded":"2024-09-15T21:48:38.387645+0000","last_fullsized":"2024-09-15T21:48:38.387645+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:22:40.160062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"76'5","reported_seq":125,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.449671+0000","last_change":"2024-09-15T21:46:23.427968+0000","last_active":"2024-09-15T21:48:41.449671+0000","last_peered":"2024-09-15T21:48:41.449671+0000","last_clean":"2024-09-15T21:48:41.449671+0000","last_became_active":"2024-09-15T21:46:23.427546+0000","last_became_peered":"2024-09-15T21:46:23.427546+0000","last_unstale":"2024-09-15T21:48:41.449671+0000","last_undegraded":"2024-09-15T21:48:41.449671+0000","last_fullsized":"2024-09-15T21:48:41.449671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:26:47.226097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'13","reported_seq":171,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:36.726869+0000","last_change":"2024-09-15T21:48:21.680908+0000","last_active":"2024-09-15T21:48:36.726869+0000","last_peered":"2024-09-15T21:48:36.726869+0000","last_clean":"2024-09-15T21:48:36.726869+0000","last_became_active":"2024-09-15T21:48:21.642406+0000","last_became_peered":"2024-09-15T21:48:21.642406+0000","last_unstale":"2024-09-15T21:48:36.726869+0000","last_undegraded":"2024-09-15T21:48:36.726869+0000","last_fullsized":"2024-09-15T21:48:36.726869+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:21.396803+0000","last_clean_scrub_stamp":"2024-09-15T21:46:21.396803+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:23:26.323380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1344,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":503,"ondisk_log_size":503,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43045,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":252,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":4802,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":335872,"data_stored":86090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6368,"internal_metadata":0},"log_size":222,"ondisk_log_size":222,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476770,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1507328,"data_stored":1002267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509474,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":1003890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509476,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":946176,"data_stored":447637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":57,"seq":244813135880,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29088,"kb_used_data":912,"kb_used_omap":7,"kb_used_meta":28152,"kb_avail":94342752,"statfs":{"total":96636764160,"available":96606978048,"internally_reserved":0,"allocated":933888,"data_stored":414957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7963,"internal_metadata":28827877},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509475,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":909312,"data_stored":412355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509474,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":921600,"data_stored":412891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":913408,"data_stored":412557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574883,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":954368,"data_stored":448472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":37447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6368,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":36612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:48:47.842 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.842 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.843 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.843 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.843 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.843 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.843 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.843 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.843 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.843 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.843 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.844 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.844 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.844 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.844 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.844 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.844 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.844 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.844 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.844 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.845 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.845 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.845 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.845 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.845 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.845 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.845 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.845 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.845 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.845 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.846 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.846 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.846 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.846 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.846 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.846 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.846 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.846 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.846 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.847 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.847 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.847 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.847 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.847 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.847 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.847 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.848 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.848 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.848 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.848 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.848 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.848 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.848 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.848 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.848 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.849 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.849 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.849 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.849 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.849 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.849 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.849 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.849 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.850 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.851 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-15T21:46:12.223400+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.851 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.851 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.851 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.851 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.851 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.852 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.852 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.852 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.852 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.852 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.852 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.852 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.853 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.853 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.853 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.853 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.853 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.853 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.853 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.853 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.854 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.854 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.854 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.854 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.855 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.855 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.855 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.855 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.855 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.855 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.855 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.855 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-09-15T21:46:21.396803+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=21, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:48:47.856 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T21:49:07.857 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:49:07.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.968+0000 7ff8a792c640 1 Processor -- start 2024-09-15T21:49:07.970 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.968+0000 7ff8a792c640 1 -- start start 2024-09-15T21:49:07.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.969+0000 7ff8a792c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff8a00c96b0 0x7ff8a00c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:07.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.969+0000 7ff8a792c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff8a01522e0 0x7ff8a00723c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:07.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.969+0000 7ff8a792c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8a0157ef0 con 0x7ff8a00c92d0 2024-09-15T21:49:07.971 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.969+0000 7ff8a792c640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7ff8a0072900 con 0x7ff8a01522e0 2024-09-15T21:49:07.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.969+0000 7ff8a792c640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7ff8a00c4980 con 0x7ff8a00c96b0 2024-09-15T21:49:07.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.969+0000 7ff8a5ec9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff8a01522e0 0x7ff8a00723c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:07.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.969+0000 7ff8a5ec9640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7ff8a01522e0 0x7ff8a00723c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42948/0 (socket says 172.21.15.100:42948) 2024-09-15T21:49:07.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.969+0000 7ff8a5ec9640 1 -- 172.21.15.100:0/498468901 learned_addr learned my addr 172.21.15.100:0/498468901 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:49:07.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.969+0000 7ff8a4ec7640 1 --2- 172.21.15.100:0/498468901 >> v2:172.21.15.122:3300/0 conn(0x7ff8a00c96b0 0x7ff8a00c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:07.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.969+0000 7ff8a5ec9640 1 -- 172.21.15.100:0/498468901 >> v2:172.21.15.122:3300/0 conn(0x7ff8a00c96b0 msgr2=0x7ff8a00c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:49:07.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.969+0000 7ff8a5ec9640 1 --2- 172.21.15.100:0/498468901 >> v2:172.21.15.122:3300/0 conn(0x7ff8a00c96b0 0x7ff8a00c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:07.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.969+0000 7ff8a5ec9640 1 -- 172.21.15.100:0/498468901 >> v1:172.21.15.122:6789/0 conn(0x7ff8a00c92d0 legacy=0x7ff8a0151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:07.972 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.969+0000 7ff8a5ec9640 1 -- 172.21.15.100:0/498468901 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a01581b0 con 0x7ff8a01522e0 2024-09-15T21:49:07.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.970+0000 7ff8a5ec9640 1 --2- 172.21.15.100:0/498468901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a01522e0 0x7ff8a00723c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00c86d0 tx=0x7ff89802c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a310dffac5a75b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:07.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.970+0000 7ff897fff640 1 -- 172.21.15.100:0/498468901 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89800b2d0 con 0x7ff8a01522e0 2024-09-15T21:49:07.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.970+0000 7ff897fff640 1 -- 172.21.15.100:0/498468901 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff89800b470 con 0x7ff8a01522e0 2024-09-15T21:49:07.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.970+0000 7ff897fff640 1 -- 172.21.15.100:0/498468901 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898009820 con 0x7ff8a01522e0 2024-09-15T21:49:07.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.971+0000 7ff8a792c640 1 -- 172.21.15.100:0/498468901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a01522e0 msgr2=0x7ff8a00723c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:07.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.971+0000 7ff8a792c640 1 --2- 172.21.15.100:0/498468901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a01522e0 0x7ff8a00723c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff8a00c86d0 tx=0x7ff89802c8b0 comp rx=0 tx=0).stop 2024-09-15T21:49:07.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.971+0000 7ff8a792c640 1 -- 172.21.15.100:0/498468901 shutdown_connections 2024-09-15T21:49:07.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.971+0000 7ff8a792c640 1 --2- 172.21.15.100:0/498468901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a01522e0 0x7ff8a00723c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:07.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.971+0000 7ff8a792c640 1 --2- 172.21.15.100:0/498468901 >> v2:172.21.15.122:3300/0 conn(0x7ff8a00c96b0 0x7ff8a00c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:07.974 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.971+0000 7ff8a792c640 1 -- 172.21.15.100:0/498468901 >> 172.21.15.100:0/498468901 conn(0x7ff8a0067dd0 msgr2=0x7ff8a006bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:07.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.971+0000 7ff8a792c640 1 -- 172.21.15.100:0/498468901 shutdown_connections 2024-09-15T21:49:07.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.971+0000 7ff8a792c640 1 -- 172.21.15.100:0/498468901 wait complete. 2024-09-15T21:49:07.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.972+0000 7ff8a792c640 1 Processor -- start 2024-09-15T21:49:07.975 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.972+0000 7ff8a792c640 1 -- start start 2024-09-15T21:49:07.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.973+0000 7ff8a792c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8a00c92d0 0x7ff8a00c3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:07.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.973+0000 7ff8a792c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a00c96b0 0x7ff8a014dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:07.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.973+0000 7ff8a792c640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff8a01522e0 0x7ff8a014a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:07.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.973+0000 7ff8a792c640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a0158a50 con 0x7ff8a00c96b0 2024-09-15T21:49:07.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.973+0000 7ff8a792c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a0158770 con 0x7ff8a00c92d0 2024-09-15T21:49:07.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.973+0000 7ff8a792c640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff8a00c8d30 con 0x7ff8a01522e0 2024-09-15T21:49:07.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.973+0000 7ff8a4ec7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a00c96b0 0x7ff8a014dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:07.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.973+0000 7ff8a4ec7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a00c96b0 0x7ff8a014dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:42950/0 (socket says 172.21.15.100:42950) 2024-09-15T21:49:07.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.973+0000 7ff8a4ec7640 1 -- 172.21.15.100:0/3793062813 learned_addr learned my addr 172.21.15.100:0/3793062813 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:49:07.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.973+0000 7ff8a5ec9640 1 --2- 172.21.15.100:0/3793062813 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff8a01522e0 0x7ff8a014a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:07.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.973+0000 7ff8a56c8640 1 --2- 172.21.15.100:0/3793062813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8a00c92d0 0x7ff8a00c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:07.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.973+0000 7ff8a4ec7640 1 -- 172.21.15.100:0/3793062813 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff8a01522e0 msgr2=0x7ff8a014a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:07.977 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.973+0000 7ff8a4ec7640 1 --2- 172.21.15.100:0/3793062813 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff8a01522e0 0x7ff8a014a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:07.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.973+0000 7ff8a4ec7640 1 -- 172.21.15.100:0/3793062813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8a00c92d0 msgr2=0x7ff8a00c3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:07.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.973+0000 7ff8a4ec7640 1 --2- 172.21.15.100:0/3793062813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8a00c92d0 0x7ff8a00c3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:07.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.973+0000 7ff8a4ec7640 1 -- 172.21.15.100:0/3793062813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a0071eb0 con 0x7ff8a00c96b0 2024-09-15T21:49:07.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.973+0000 7ff8a4ec7640 1 --2- 172.21.15.100:0/3793062813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a00c96b0 0x7ff8a014dc00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff89000e870 tx=0x7ff89000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:07.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.974+0000 7ff895ffb640 1 -- 172.21.15.100:0/3793062813 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff890017070 con 0x7ff8a00c96b0 2024-09-15T21:49:07.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.974+0000 7ff895ffb640 1 -- 172.21.15.100:0/3793062813 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff89000c940 con 0x7ff8a00c96b0 2024-09-15T21:49:07.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.974+0000 7ff8a792c640 1 -- 172.21.15.100:0/3793062813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a014e640 con 0x7ff8a00c96b0 2024-09-15T21:49:07.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.974+0000 7ff895ffb640 1 -- 172.21.15.100:0/3793062813 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89000cd10 con 0x7ff8a00c96b0 2024-09-15T21:49:07.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.974+0000 7ff8a792c640 1 -- 172.21.15.100:0/3793062813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a01df190 con 0x7ff8a00c96b0 2024-09-15T21:49:07.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.975+0000 7ff8a792c640 1 -- 172.21.15.100:0/3793062813 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8a00c8740 con 0x7ff8a00c96b0 2024-09-15T21:49:07.978 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.975+0000 7ff895ffb640 1 -- 172.21.15.100:0/3793062813 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff89001a070 con 0x7ff8a00c96b0 2024-09-15T21:49:07.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.976+0000 7ff895ffb640 1 --2- 172.21.15.100:0/3793062813 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff88406c2b0 0x7ff88406e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:07.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.976+0000 7ff895ffb640 1 -- 172.21.15.100:0/3793062813 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5886+0+0 (secure 0 0 0) 0x7ff89008b6e0 con 0x7ff8a00c96b0 2024-09-15T21:49:07.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.976+0000 7ff8a56c8640 1 --2- 172.21.15.100:0/3793062813 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff88406c2b0 0x7ff88406e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:07.979 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.976+0000 7ff8a56c8640 1 --2- 172.21.15.100:0/3793062813 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff88406c2b0 0x7ff88406e770 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0016e0 tx=0x7ff89c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:07.982 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:07.980+0000 7ff895ffb640 1 -- 172.21.15.100:0/3793062813 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff8900596c0 con 0x7ff8a00c96b0 2024-09-15T21:49:08.144 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:08.142+0000 7ff8a792c640 1 -- 172.21.15.100:0/3793062813 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff8a01f0d10 con 0x7ff88406c2b0 2024-09-15T21:49:08.150 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:08.148+0000 7ff895ffb640 1 -- 172.21.15.100:0/3793062813 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252649 (secure 0 0 0) 0x7ff8a01f0d10 con 0x7ff88406c2b0 2024-09-15T21:49:08.156 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:49:08.157 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-15T21:49:08.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:08.151+0000 7ff8a792c640 1 -- 172.21.15.100:0/3793062813 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff88406c2b0 msgr2=0x7ff88406e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:08.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:08.151+0000 7ff8a792c640 1 --2- 172.21.15.100:0/3793062813 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff88406c2b0 0x7ff88406e770 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0016e0 tx=0x7ff89c009040 comp rx=0 tx=0).stop 2024-09-15T21:49:08.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:08.151+0000 7ff8a792c640 1 -- 172.21.15.100:0/3793062813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a00c96b0 msgr2=0x7ff8a014dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:08.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:08.151+0000 7ff8a792c640 1 --2- 172.21.15.100:0/3793062813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a00c96b0 0x7ff8a014dc00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff89000e870 tx=0x7ff89000ed30 comp rx=0 tx=0).stop 2024-09-15T21:49:08.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:08.152+0000 7ff8a792c640 1 -- 172.21.15.100:0/3793062813 shutdown_connections 2024-09-15T21:49:08.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:08.152+0000 7ff8a792c640 1 --2- 172.21.15.100:0/3793062813 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff88406c2b0 0x7ff88406e770 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:08.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:08.152+0000 7ff8a792c640 1 --2- 172.21.15.100:0/3793062813 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff8a01522e0 0x7ff8a014a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:08.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:08.152+0000 7ff8a792c640 1 --2- 172.21.15.100:0/3793062813 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff8a00c96b0 0x7ff8a014dc00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:08.159 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:08.152+0000 7ff8a792c640 1 --2- 172.21.15.100:0/3793062813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8a00c92d0 0x7ff8a00c3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:08.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:08.152+0000 7ff8a792c640 1 -- 172.21.15.100:0/3793062813 >> 172.21.15.100:0/3793062813 conn(0x7ff8a0067dd0 msgr2=0x7ff8a00c6770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:08.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:08.152+0000 7ff8a792c640 1 -- 172.21.15.100:0/3793062813 shutdown_connections 2024-09-15T21:49:08.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:08.152+0000 7ff8a792c640 1 -- 172.21.15.100:0/3793062813 wait complete. 2024-09-15T21:49:08.165 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-15T21:49:07.839179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633413,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":413,"num_write_kb":2538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1185538,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":935,"ondisk_log_size":935,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238256,"kb_used_data":8424,"kb_used_omap":18,"kb_used_meta":229677,"kb_avail":754736464,"statfs":{"total":773094113280,"available":772850139136,"internally_reserved":0,"allocated":8626176,"data_stored":4567256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19119,"internal_metadata":235189585},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001814"},"pg_stats":[{"pgid":"3.3f","version":"76'8","reported_seq":56,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:47.609936+0000","last_change":"2024-09-15T21:48:47.609967+0000","last_active":"2024-09-15T21:48:47.609936+0000","last_peered":"2024-09-15T21:48:47.609936+0000","last_clean":"2024-09-15T21:48:47.609936+0000","last_became_active":"2024-09-15T21:48:21.637402+0000","last_became_peered":"2024-09-15T21:48:21.637402+0000","last_unstale":"2024-09-15T21:48:47.609936+0000","last_undegraded":"2024-09-15T21:48:47.609936+0000","last_fullsized":"2024-09-15T21:48:47.609936+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:47.609834+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:47.609834+0000","last_clean_scrub_stamp":"2024-09-15T21:48:47.609834+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:25:11.657264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"76'4","reported_seq":134,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.811028+0000","last_change":"2024-09-15T21:48:49.661819+0000","last_active":"2024-09-15T21:49:04.811028+0000","last_peered":"2024-09-15T21:49:04.811028+0000","last_clean":"2024-09-15T21:49:04.811028+0000","last_became_active":"2024-09-15T21:46:23.418471+0000","last_became_peered":"2024-09-15T21:46:23.418471+0000","last_unstale":"2024-09-15T21:49:04.811028+0000","last_undegraded":"2024-09-15T21:49:04.811028+0000","last_fullsized":"2024-09-15T21:49:04.811028+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'4","last_scrub_stamp":"2024-09-15T21:48:49.661665+0000","last_deep_scrub":"76'4","last_deep_scrub_stamp":"2024-09-15T21:48:49.661665+0000","last_clean_scrub_stamp":"2024-09-15T21:48:49.661665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:06:35.692625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"76'7","reported_seq":134,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:48.719336+0000","last_change":"2024-09-15T21:48:48.719368+0000","last_active":"2024-09-15T21:48:48.719336+0000","last_peered":"2024-09-15T21:48:48.719336+0000","last_clean":"2024-09-15T21:48:48.719336+0000","last_became_active":"2024-09-15T21:46:23.426146+0000","last_became_peered":"2024-09-15T21:46:23.426146+0000","last_unstale":"2024-09-15T21:48:48.719336+0000","last_undegraded":"2024-09-15T21:48:48.719336+0000","last_fullsized":"2024-09-15T21:48:48.719336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:48.719251+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:48.719251+0000","last_clean_scrub_stamp":"2024-09-15T21:48:48.719251+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:17.487930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"76'8","reported_seq":76,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.811820+0000","last_change":"2024-09-15T21:48:53.078030+0000","last_active":"2024-09-15T21:49:04.811820+0000","last_peered":"2024-09-15T21:49:04.811820+0000","last_clean":"2024-09-15T21:49:04.811820+0000","last_became_active":"2024-09-15T21:48:17.623606+0000","last_became_peered":"2024-09-15T21:48:17.623606+0000","last_unstale":"2024-09-15T21:49:04.811820+0000","last_undegraded":"2024-09-15T21:49:04.811820+0000","last_fullsized":"2024-09-15T21:49:04.811820+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:53.077912+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:53.077912+0000","last_clean_scrub_stamp":"2024-09-15T21:48:53.077912+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:28:10.764425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"76'4","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:55.677512+0000","last_change":"2024-09-15T21:48:55.677538+0000","last_active":"2024-09-15T21:48:55.677512+0000","last_peered":"2024-09-15T21:48:55.677512+0000","last_clean":"2024-09-15T21:48:55.677512+0000","last_became_active":"2024-09-15T21:48:21.638410+0000","last_became_peered":"2024-09-15T21:48:21.638410+0000","last_unstale":"2024-09-15T21:48:55.677512+0000","last_undegraded":"2024-09-15T21:48:55.677512+0000","last_fullsized":"2024-09-15T21:48:55.677512+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'4","last_scrub_stamp":"2024-09-15T21:48:55.677402+0000","last_deep_scrub":"76'4","last_deep_scrub_stamp":"2024-09-15T21:48:55.677402+0000","last_clean_scrub_stamp":"2024-09-15T21:48:55.677402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:42.615279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"76'6","reported_seq":74,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.812428+0000","last_change":"2024-09-15T21:48:57.067351+0000","last_active":"2024-09-15T21:49:04.812428+0000","last_peered":"2024-09-15T21:49:04.812428+0000","last_clean":"2024-09-15T21:49:04.812428+0000","last_became_active":"2024-09-15T21:48:17.624439+0000","last_became_peered":"2024-09-15T21:48:17.624439+0000","last_unstale":"2024-09-15T21:49:04.812428+0000","last_undegraded":"2024-09-15T21:49:04.812428+0000","last_fullsized":"2024-09-15T21:49:04.812428+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:57.067212+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:57.067212+0000","last_clean_scrub_stamp":"2024-09-15T21:48:57.067212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:24.410710+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"76'5","reported_seq":53,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:50.591947+0000","last_change":"2024-09-15T21:48:50.591979+0000","last_active":"2024-09-15T21:48:50.591947+0000","last_peered":"2024-09-15T21:48:50.591947+0000","last_clean":"2024-09-15T21:48:50.591947+0000","last_became_active":"2024-09-15T21:48:21.638310+0000","last_became_peered":"2024-09-15T21:48:21.638310+0000","last_unstale":"2024-09-15T21:48:50.591947+0000","last_undegraded":"2024-09-15T21:48:50.591947+0000","last_fullsized":"2024-09-15T21:48:50.591947+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'5","last_scrub_stamp":"2024-09-15T21:48:50.591846+0000","last_deep_scrub":"76'5","last_deep_scrub_stamp":"2024-09-15T21:48:50.591846+0000","last_clean_scrub_stamp":"2024-09-15T21:48:50.591846+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:37:32.407604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"76'9","reported_seq":117,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:49.465733+0000","last_change":"2024-09-15T21:48:49.465760+0000","last_active":"2024-09-15T21:48:49.465733+0000","last_peered":"2024-09-15T21:48:49.465733+0000","last_clean":"2024-09-15T21:48:49.465733+0000","last_became_active":"2024-09-15T21:48:27.656800+0000","last_became_peered":"2024-09-15T21:48:27.656800+0000","last_unstale":"2024-09-15T21:48:49.465733+0000","last_undegraded":"2024-09-15T21:48:49.465733+0000","last_fullsized":"2024-09-15T21:48:49.465733+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:49.465653+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:49.465653+0000","last_clean_scrub_stamp":"2024-09-15T21:48:49.465653+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:22:51.070326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"76'11","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:57.641087+0000","last_change":"2024-09-15T21:48:57.641112+0000","last_active":"2024-09-15T21:48:57.641087+0000","last_peered":"2024-09-15T21:48:57.641087+0000","last_clean":"2024-09-15T21:48:57.641087+0000","last_became_active":"2024-09-15T21:48:21.641920+0000","last_became_peered":"2024-09-15T21:48:21.641920+0000","last_unstale":"2024-09-15T21:48:57.641087+0000","last_undegraded":"2024-09-15T21:48:57.641087+0000","last_fullsized":"2024-09-15T21:48:57.641087+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'11","last_scrub_stamp":"2024-09-15T21:48:57.640998+0000","last_deep_scrub":"76'11","last_deep_scrub_stamp":"2024-09-15T21:48:57.640998+0000","last_clean_scrub_stamp":"2024-09-15T21:48:57.640998+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:10.988113+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"76'16","reported_seq":150,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:56.402537+0000","last_change":"2024-09-15T21:48:56.402567+0000","last_active":"2024-09-15T21:48:56.402537+0000","last_peered":"2024-09-15T21:48:56.402537+0000","last_clean":"2024-09-15T21:48:56.402537+0000","last_became_active":"2024-09-15T21:48:17.621800+0000","last_became_peered":"2024-09-15T21:48:17.621800+0000","last_unstale":"2024-09-15T21:48:56.402537+0000","last_undegraded":"2024-09-15T21:48:56.402537+0000","last_fullsized":"2024-09-15T21:48:56.402537+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'16","last_scrub_stamp":"2024-09-15T21:48:56.402433+0000","last_deep_scrub":"76'16","last_deep_scrub_stamp":"2024-09-15T21:48:56.402433+0000","last_clean_scrub_stamp":"2024-09-15T21:48:56.402433+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:47.418031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"76'6","reported_seq":138,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:54.409794+0000","last_change":"2024-09-15T21:48:54.409819+0000","last_active":"2024-09-15T21:48:54.409794+0000","last_peered":"2024-09-15T21:48:54.409794+0000","last_clean":"2024-09-15T21:48:54.409794+0000","last_became_active":"2024-09-15T21:47:05.734921+0000","last_became_peered":"2024-09-15T21:47:05.734921+0000","last_unstale":"2024-09-15T21:48:54.409794+0000","last_undegraded":"2024-09-15T21:48:54.409794+0000","last_fullsized":"2024-09-15T21:48:54.409794+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:54.409705+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:54.409705+0000","last_clean_scrub_stamp":"2024-09-15T21:48:54.409705+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:04.597525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"76'7","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:50.443535+0000","last_change":"2024-09-15T21:48:50.443565+0000","last_active":"2024-09-15T21:48:50.443535+0000","last_peered":"2024-09-15T21:48:50.443535+0000","last_clean":"2024-09-15T21:48:50.443535+0000","last_became_active":"2024-09-15T21:46:23.430170+0000","last_became_peered":"2024-09-15T21:46:23.430170+0000","last_unstale":"2024-09-15T21:48:50.443535+0000","last_undegraded":"2024-09-15T21:48:50.443535+0000","last_fullsized":"2024-09-15T21:48:50.443535+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:50.443445+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:50.443445+0000","last_clean_scrub_stamp":"2024-09-15T21:48:50.443445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:02.921106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"76'10","reported_seq":139,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:49.365097+0000","last_change":"2024-09-15T21:48:49.365097+0000","last_active":"2024-09-15T21:48:49.365097+0000","last_peered":"2024-09-15T21:48:49.365097+0000","last_clean":"2024-09-15T21:48:49.365097+0000","last_became_active":"2024-09-15T21:46:23.427621+0000","last_became_peered":"2024-09-15T21:46:23.427621+0000","last_unstale":"2024-09-15T21:48:49.365097+0000","last_undegraded":"2024-09-15T21:48:49.365097+0000","last_fullsized":"2024-09-15T21:48:49.365097+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'10","last_scrub_stamp":"2024-09-15T21:48:49.364982+0000","last_deep_scrub":"76'10","last_deep_scrub_stamp":"2024-09-15T21:48:49.364982+0000","last_clean_scrub_stamp":"2024-09-15T21:48:49.364982+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:25:08.887670+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"76'10","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:47.718653+0000","last_change":"2024-09-15T21:48:47.718675+0000","last_active":"2024-09-15T21:48:47.718653+0000","last_peered":"2024-09-15T21:48:47.718653+0000","last_clean":"2024-09-15T21:48:47.718653+0000","last_became_active":"2024-09-15T21:46:23.424447+0000","last_became_peered":"2024-09-15T21:46:23.424447+0000","last_unstale":"2024-09-15T21:48:47.718653+0000","last_undegraded":"2024-09-15T21:48:47.718653+0000","last_fullsized":"2024-09-15T21:48:47.718653+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'10","last_scrub_stamp":"2024-09-15T21:48:47.718575+0000","last_deep_scrub":"76'10","last_deep_scrub_stamp":"2024-09-15T21:48:47.718575+0000","last_clean_scrub_stamp":"2024-09-15T21:48:47.718575+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:48:01.821709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"76'7","reported_seq":137,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.810731+0000","last_change":"2024-09-15T21:48:54.618833+0000","last_active":"2024-09-15T21:49:04.810731+0000","last_peered":"2024-09-15T21:49:04.810731+0000","last_clean":"2024-09-15T21:49:04.810731+0000","last_became_active":"2024-09-15T21:46:23.430070+0000","last_became_peered":"2024-09-15T21:46:23.430070+0000","last_unstale":"2024-09-15T21:49:04.810731+0000","last_undegraded":"2024-09-15T21:49:04.810731+0000","last_fullsized":"2024-09-15T21:49:04.810731+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:54.618713+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:54.618713+0000","last_clean_scrub_stamp":"2024-09-15T21:48:54.618713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:01.651899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"76'9","reported_seq":77,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.811218+0000","last_change":"2024-09-15T21:48:55.108052+0000","last_active":"2024-09-15T21:49:04.811218+0000","last_peered":"2024-09-15T21:49:04.811218+0000","last_clean":"2024-09-15T21:49:04.811218+0000","last_became_active":"2024-09-15T21:48:17.624764+0000","last_became_peered":"2024-09-15T21:48:17.624764+0000","last_unstale":"2024-09-15T21:49:04.811218+0000","last_undegraded":"2024-09-15T21:49:04.811218+0000","last_fullsized":"2024-09-15T21:49:04.811218+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:55.107907+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:55.107907+0000","last_clean_scrub_stamp":"2024-09-15T21:48:55.107907+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:44:35.180572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"76'8","reported_seq":68,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:41.460414+0000","last_change":"2024-09-15T21:48:21.642646+0000","last_active":"2024-09-15T21:48:41.460414+0000","last_peered":"2024-09-15T21:48:41.460414+0000","last_clean":"2024-09-15T21:48:41.460414+0000","last_became_active":"2024-09-15T21:48:21.642303+0000","last_became_peered":"2024-09-15T21:48:21.642303+0000","last_unstale":"2024-09-15T21:48:41.460414+0000","last_undegraded":"2024-09-15T21:48:41.460414+0000","last_fullsized":"2024-09-15T21:48:41.460414+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:22.402110+0000","last_clean_scrub_stamp":"2024-09-15T21:46:22.402110+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"76'10","reported_seq":95,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:52.489056+0000","last_change":"2024-09-15T21:48:52.489084+0000","last_active":"2024-09-15T21:48:52.489056+0000","last_peered":"2024-09-15T21:48:52.489056+0000","last_clean":"2024-09-15T21:48:52.489056+0000","last_became_active":"2024-09-15T21:48:27.661441+0000","last_became_peered":"2024-09-15T21:48:27.661441+0000","last_unstale":"2024-09-15T21:48:52.489056+0000","last_undegraded":"2024-09-15T21:48:52.489056+0000","last_fullsized":"2024-09-15T21:48:52.489056+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'10","last_scrub_stamp":"2024-09-15T21:48:52.488966+0000","last_deep_scrub":"76'10","last_deep_scrub_stamp":"2024-09-15T21:48:52.488966+0000","last_clean_scrub_stamp":"2024-09-15T21:48:52.488966+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:18:52.212403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"76'7","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:53.606227+0000","last_change":"2024-09-15T21:48:53.606284+0000","last_active":"2024-09-15T21:48:53.606227+0000","last_peered":"2024-09-15T21:48:53.606227+0000","last_clean":"2024-09-15T21:48:53.606227+0000","last_became_active":"2024-09-15T21:48:21.639009+0000","last_became_peered":"2024-09-15T21:48:21.639009+0000","last_unstale":"2024-09-15T21:48:53.606227+0000","last_undegraded":"2024-09-15T21:48:53.606227+0000","last_fullsized":"2024-09-15T21:48:53.606227+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:53.606083+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:53.606083+0000","last_clean_scrub_stamp":"2024-09-15T21:48:53.606083+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:38.744519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"76'8","reported_seq":135,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:52.421061+0000","last_change":"2024-09-15T21:48:52.421114+0000","last_active":"2024-09-15T21:48:52.421061+0000","last_peered":"2024-09-15T21:48:52.421061+0000","last_clean":"2024-09-15T21:48:52.421061+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:48:52.421061+0000","last_undegraded":"2024-09-15T21:48:52.421061+0000","last_fullsized":"2024-09-15T21:48:52.421061+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:52.420959+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:52.420959+0000","last_clean_scrub_stamp":"2024-09-15T21:48:52.420959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:41.523791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"76'11","reported_seq":148,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:55.715193+0000","last_change":"2024-09-15T21:48:55.715221+0000","last_active":"2024-09-15T21:48:55.715193+0000","last_peered":"2024-09-15T21:48:55.715193+0000","last_clean":"2024-09-15T21:48:55.715193+0000","last_became_active":"2024-09-15T21:48:21.641876+0000","last_became_peered":"2024-09-15T21:48:21.641876+0000","last_unstale":"2024-09-15T21:48:55.715193+0000","last_undegraded":"2024-09-15T21:48:55.715193+0000","last_fullsized":"2024-09-15T21:48:55.715193+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'11","last_scrub_stamp":"2024-09-15T21:48:55.715108+0000","last_deep_scrub":"76'11","last_deep_scrub_stamp":"2024-09-15T21:48:55.715108+0000","last_clean_scrub_stamp":"2024-09-15T21:48:55.715108+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:09:45.242437+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"76'10","reported_seq":80,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.811477+0000","last_change":"2024-09-15T21:49:00.105773+0000","last_active":"2024-09-15T21:49:04.811477+0000","last_peered":"2024-09-15T21:49:04.811477+0000","last_clean":"2024-09-15T21:49:04.811477+0000","last_became_active":"2024-09-15T21:48:21.638073+0000","last_became_peered":"2024-09-15T21:48:21.638073+0000","last_unstale":"2024-09-15T21:49:04.811477+0000","last_undegraded":"2024-09-15T21:49:04.811477+0000","last_fullsized":"2024-09-15T21:49:04.811477+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'10","last_scrub_stamp":"2024-09-15T21:49:00.105642+0000","last_deep_scrub":"76'10","last_deep_scrub_stamp":"2024-09-15T21:49:00.105642+0000","last_clean_scrub_stamp":"2024-09-15T21:49:00.105642+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:39.997437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"76'8","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:53.385830+0000","last_change":"2024-09-15T21:48:53.385859+0000","last_active":"2024-09-15T21:48:53.385830+0000","last_peered":"2024-09-15T21:48:53.385830+0000","last_clean":"2024-09-15T21:48:53.385830+0000","last_became_active":"2024-09-15T21:46:23.430719+0000","last_became_peered":"2024-09-15T21:46:23.430719+0000","last_unstale":"2024-09-15T21:48:53.385830+0000","last_undegraded":"2024-09-15T21:48:53.385830+0000","last_fullsized":"2024-09-15T21:48:53.385830+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:53.385747+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:53.385747+0000","last_clean_scrub_stamp":"2024-09-15T21:48:53.385747+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:16.486598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"76'6","reported_seq":74,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.811047+0000","last_change":"2024-09-15T21:48:49.107693+0000","last_active":"2024-09-15T21:49:04.811047+0000","last_peered":"2024-09-15T21:49:04.811047+0000","last_clean":"2024-09-15T21:49:04.811047+0000","last_became_active":"2024-09-15T21:48:17.621639+0000","last_became_peered":"2024-09-15T21:48:17.621639+0000","last_unstale":"2024-09-15T21:49:04.811047+0000","last_undegraded":"2024-09-15T21:49:04.811047+0000","last_fullsized":"2024-09-15T21:49:04.811047+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:49.107527+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:49.107527+0000","last_clean_scrub_stamp":"2024-09-15T21:48:49.107527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:32:03.896706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"76'2","reported_seq":131,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:55.308528+0000","last_change":"2024-09-15T21:48:55.308557+0000","last_active":"2024-09-15T21:48:55.308528+0000","last_peered":"2024-09-15T21:48:55.308528+0000","last_clean":"2024-09-15T21:48:55.308528+0000","last_became_active":"2024-09-15T21:46:23.432317+0000","last_became_peered":"2024-09-15T21:46:23.432317+0000","last_unstale":"2024-09-15T21:48:55.308528+0000","last_undegraded":"2024-09-15T21:48:55.308528+0000","last_fullsized":"2024-09-15T21:48:55.308528+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'2","last_scrub_stamp":"2024-09-15T21:48:55.308414+0000","last_deep_scrub":"76'2","last_deep_scrub_stamp":"2024-09-15T21:48:55.308414+0000","last_clean_scrub_stamp":"2024-09-15T21:48:55.308414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:14:12.319169+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"76'15","reported_seq":152,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:57.379011+0000","last_change":"2024-09-15T21:48:57.379036+0000","last_active":"2024-09-15T21:48:57.379011+0000","last_peered":"2024-09-15T21:48:57.379011+0000","last_clean":"2024-09-15T21:48:57.379011+0000","last_became_active":"2024-09-15T21:48:21.638212+0000","last_became_peered":"2024-09-15T21:48:21.638212+0000","last_unstale":"2024-09-15T21:48:57.379011+0000","last_undegraded":"2024-09-15T21:48:57.379011+0000","last_fullsized":"2024-09-15T21:48:57.379011+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'15","last_scrub_stamp":"2024-09-15T21:48:57.378918+0000","last_deep_scrub":"76'15","last_deep_scrub_stamp":"2024-09-15T21:48:57.378918+0000","last_clean_scrub_stamp":"2024-09-15T21:48:57.378918+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:18.293141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"76'6","reported_seq":136,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.810937+0000","last_change":"2024-09-15T21:48:50.631757+0000","last_active":"2024-09-15T21:49:04.810937+0000","last_peered":"2024-09-15T21:49:04.810937+0000","last_clean":"2024-09-15T21:49:04.810937+0000","last_became_active":"2024-09-15T21:46:23.421501+0000","last_became_peered":"2024-09-15T21:46:23.421501+0000","last_unstale":"2024-09-15T21:49:04.810937+0000","last_undegraded":"2024-09-15T21:49:04.810937+0000","last_fullsized":"2024-09-15T21:49:04.810937+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:50.631641+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:50.631641+0000","last_clean_scrub_stamp":"2024-09-15T21:48:50.631641+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:20:22.937617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"76'4","reported_seq":133,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:54.340079+0000","last_change":"2024-09-15T21:48:54.340105+0000","last_active":"2024-09-15T21:48:54.340079+0000","last_peered":"2024-09-15T21:48:54.340079+0000","last_clean":"2024-09-15T21:48:54.340079+0000","last_became_active":"2024-09-15T21:46:23.432326+0000","last_became_peered":"2024-09-15T21:46:23.432326+0000","last_unstale":"2024-09-15T21:48:54.340079+0000","last_undegraded":"2024-09-15T21:48:54.340079+0000","last_fullsized":"2024-09-15T21:48:54.340079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'4","last_scrub_stamp":"2024-09-15T21:48:54.339983+0000","last_deep_scrub":"76'4","last_deep_scrub_stamp":"2024-09-15T21:48:54.339983+0000","last_clean_scrub_stamp":"2024-09-15T21:48:54.339983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:09:02.395350+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"76'7","reported_seq":134,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:45.670978+0000","last_change":"2024-09-15T21:48:45.671000+0000","last_active":"2024-09-15T21:48:45.670978+0000","last_peered":"2024-09-15T21:48:45.670978+0000","last_clean":"2024-09-15T21:48:45.670978+0000","last_became_active":"2024-09-15T21:46:23.422734+0000","last_became_peered":"2024-09-15T21:46:23.422734+0000","last_unstale":"2024-09-15T21:48:45.670978+0000","last_undegraded":"2024-09-15T21:48:45.670978+0000","last_fullsized":"2024-09-15T21:48:45.670978+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:45.670897+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:45.670897+0000","last_clean_scrub_stamp":"2024-09-15T21:48:45.670897+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:30:38.392752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"76'8","reported_seq":76,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.812234+0000","last_change":"2024-09-15T21:48:51.066043+0000","last_active":"2024-09-15T21:49:04.812234+0000","last_peered":"2024-09-15T21:49:04.812234+0000","last_clean":"2024-09-15T21:49:04.812234+0000","last_became_active":"2024-09-15T21:48:17.621473+0000","last_became_peered":"2024-09-15T21:48:17.621473+0000","last_unstale":"2024-09-15T21:49:04.812234+0000","last_undegraded":"2024-09-15T21:49:04.812234+0000","last_fullsized":"2024-09-15T21:49:04.812234+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:51.065913+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:51.065913+0000","last_clean_scrub_stamp":"2024-09-15T21:48:51.065913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:05:51.774365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"76'12","reported_seq":146,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:47.449142+0000","last_change":"2024-09-15T21:48:47.449165+0000","last_active":"2024-09-15T21:48:47.449142+0000","last_peered":"2024-09-15T21:48:47.449142+0000","last_clean":"2024-09-15T21:48:47.449142+0000","last_became_active":"2024-09-15T21:48:17.616499+0000","last_became_peered":"2024-09-15T21:48:17.616499+0000","last_unstale":"2024-09-15T21:48:47.449142+0000","last_undegraded":"2024-09-15T21:48:47.449142+0000","last_fullsized":"2024-09-15T21:48:47.449142+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'12","last_scrub_stamp":"2024-09-15T21:48:47.449074+0000","last_deep_scrub":"76'12","last_deep_scrub_stamp":"2024-09-15T21:48:47.449074+0000","last_clean_scrub_stamp":"2024-09-15T21:48:47.449074+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:45:43.158651+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"76'9","reported_seq":136,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:51.414622+0000","last_change":"2024-09-15T21:48:51.414676+0000","last_active":"2024-09-15T21:48:51.414622+0000","last_peered":"2024-09-15T21:48:51.414622+0000","last_clean":"2024-09-15T21:48:51.414622+0000","last_became_active":"2024-09-15T21:46:23.429376+0000","last_became_peered":"2024-09-15T21:46:23.429376+0000","last_unstale":"2024-09-15T21:48:51.414622+0000","last_undegraded":"2024-09-15T21:48:51.414622+0000","last_fullsized":"2024-09-15T21:48:51.414622+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:51.414516+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:51.414516+0000","last_clean_scrub_stamp":"2024-09-15T21:48:51.414516+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:12:44.641131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"76'7","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:48.503982+0000","last_change":"2024-09-15T21:48:48.503982+0000","last_active":"2024-09-15T21:48:48.503982+0000","last_peered":"2024-09-15T21:48:48.503982+0000","last_clean":"2024-09-15T21:48:48.503982+0000","last_became_active":"2024-09-15T21:46:23.429442+0000","last_became_peered":"2024-09-15T21:46:23.429442+0000","last_unstale":"2024-09-15T21:48:48.503982+0000","last_undegraded":"2024-09-15T21:48:48.503982+0000","last_fullsized":"2024-09-15T21:48:48.503982+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:48.503827+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:48.503827+0000","last_clean_scrub_stamp":"2024-09-15T21:48:48.503827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:10:11.107071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"76'8","reported_seq":161,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:58.317588+0000","last_change":"2024-09-15T21:48:58.317631+0000","last_active":"2024-09-15T21:48:58.317588+0000","last_peered":"2024-09-15T21:48:58.317588+0000","last_clean":"2024-09-15T21:48:58.317588+0000","last_became_active":"2024-09-15T21:48:21.636972+0000","last_became_peered":"2024-09-15T21:48:21.636972+0000","last_unstale":"2024-09-15T21:48:58.317588+0000","last_undegraded":"2024-09-15T21:48:58.317588+0000","last_fullsized":"2024-09-15T21:48:58.317588+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:58.317488+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:58.317488+0000","last_clean_scrub_stamp":"2024-09-15T21:48:58.317488+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:30.154332+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1019,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1756,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"76'8","reported_seq":135,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:50.758175+0000","last_change":"2024-09-15T21:48:50.758175+0000","last_active":"2024-09-15T21:48:50.758175+0000","last_peered":"2024-09-15T21:48:50.758175+0000","last_clean":"2024-09-15T21:48:50.758175+0000","last_became_active":"2024-09-15T21:46:23.433944+0000","last_became_peered":"2024-09-15T21:46:23.433944+0000","last_unstale":"2024-09-15T21:48:50.758175+0000","last_undegraded":"2024-09-15T21:48:50.758175+0000","last_fullsized":"2024-09-15T21:48:50.758175+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:50.758058+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:50.758058+0000","last_clean_scrub_stamp":"2024-09-15T21:48:50.758058+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:25:56.670468+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"76'10","reported_seq":139,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:48.512029+0000","last_change":"2024-09-15T21:48:48.512029+0000","last_active":"2024-09-15T21:48:48.512029+0000","last_peered":"2024-09-15T21:48:48.512029+0000","last_clean":"2024-09-15T21:48:48.512029+0000","last_became_active":"2024-09-15T21:46:22.425315+0000","last_became_peered":"2024-09-15T21:46:22.425315+0000","last_unstale":"2024-09-15T21:48:48.512029+0000","last_undegraded":"2024-09-15T21:48:48.512029+0000","last_fullsized":"2024-09-15T21:48:48.512029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'10","last_scrub_stamp":"2024-09-15T21:48:48.511883+0000","last_deep_scrub":"76'10","last_deep_scrub_stamp":"2024-09-15T21:48:48.511883+0000","last_clean_scrub_stamp":"2024-09-15T21:48:48.511883+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:14.895078+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"76'5","reported_seq":136,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:46.431777+0000","last_change":"2024-09-15T21:48:46.431777+0000","last_active":"2024-09-15T21:48:46.431777+0000","last_peered":"2024-09-15T21:48:46.431777+0000","last_clean":"2024-09-15T21:48:46.431777+0000","last_became_active":"2024-09-15T21:46:22.425226+0000","last_became_peered":"2024-09-15T21:46:22.425226+0000","last_unstale":"2024-09-15T21:48:46.431777+0000","last_undegraded":"2024-09-15T21:48:46.431777+0000","last_fullsized":"2024-09-15T21:48:46.431777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'5","last_scrub_stamp":"2024-09-15T21:48:46.431641+0000","last_deep_scrub":"76'5","last_deep_scrub_stamp":"2024-09-15T21:48:46.431641+0000","last_clean_scrub_stamp":"2024-09-15T21:48:46.431641+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:18:53.830908+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"76'8","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:52.415821+0000","last_change":"2024-09-15T21:48:52.415850+0000","last_active":"2024-09-15T21:48:52.415821+0000","last_peered":"2024-09-15T21:48:52.415821+0000","last_clean":"2024-09-15T21:48:52.415821+0000","last_became_active":"2024-09-15T21:46:23.428902+0000","last_became_peered":"2024-09-15T21:46:23.428902+0000","last_unstale":"2024-09-15T21:48:52.415821+0000","last_undegraded":"2024-09-15T21:48:52.415821+0000","last_fullsized":"2024-09-15T21:48:52.415821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:52.415717+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:52.415717+0000","last_clean_scrub_stamp":"2024-09-15T21:48:52.415717+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:32:42.694715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"76'4","reported_seq":77,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.811060+0000","last_change":"2024-09-15T21:48:47.101716+0000","last_active":"2024-09-15T21:49:04.811060+0000","last_peered":"2024-09-15T21:49:04.811060+0000","last_clean":"2024-09-15T21:49:04.811060+0000","last_became_active":"2024-09-15T21:48:17.616704+0000","last_became_peered":"2024-09-15T21:48:17.616704+0000","last_unstale":"2024-09-15T21:49:04.811060+0000","last_undegraded":"2024-09-15T21:49:04.811060+0000","last_fullsized":"2024-09-15T21:49:04.811060+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'4","last_scrub_stamp":"2024-09-15T21:48:47.101544+0000","last_deep_scrub":"76'4","last_deep_scrub_stamp":"2024-09-15T21:48:47.101544+0000","last_clean_scrub_stamp":"2024-09-15T21:48:47.101544+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:28.355881+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"76'10","reported_seq":140,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.810728+0000","last_change":"2024-09-15T21:48:53.616901+0000","last_active":"2024-09-15T21:49:04.810728+0000","last_peered":"2024-09-15T21:49:04.810728+0000","last_clean":"2024-09-15T21:49:04.810728+0000","last_became_active":"2024-09-15T21:46:23.427200+0000","last_became_peered":"2024-09-15T21:46:23.427200+0000","last_unstale":"2024-09-15T21:49:04.810728+0000","last_undegraded":"2024-09-15T21:49:04.810728+0000","last_fullsized":"2024-09-15T21:49:04.810728+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'10","last_scrub_stamp":"2024-09-15T21:48:53.616791+0000","last_deep_scrub":"76'10","last_deep_scrub_stamp":"2024-09-15T21:48:53.616791+0000","last_clean_scrub_stamp":"2024-09-15T21:48:53.616791+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:40:13.547220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"76'9","reported_seq":146,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:55.414256+0000","last_change":"2024-09-15T21:48:55.414295+0000","last_active":"2024-09-15T21:48:55.414256+0000","last_peered":"2024-09-15T21:48:55.414256+0000","last_clean":"2024-09-15T21:48:55.414256+0000","last_became_active":"2024-09-15T21:48:17.621860+0000","last_became_peered":"2024-09-15T21:48:17.621860+0000","last_unstale":"2024-09-15T21:48:55.414256+0000","last_undegraded":"2024-09-15T21:48:55.414256+0000","last_fullsized":"2024-09-15T21:48:55.414256+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:55.414151+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:55.414151+0000","last_clean_scrub_stamp":"2024-09-15T21:48:55.414151+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:32:39.534367+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"76'10","reported_seq":80,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.811637+0000","last_change":"2024-09-15T21:49:01.124047+0000","last_active":"2024-09-15T21:49:04.811637+0000","last_peered":"2024-09-15T21:49:04.811637+0000","last_clean":"2024-09-15T21:49:04.811637+0000","last_became_active":"2024-09-15T21:48:21.638300+0000","last_became_peered":"2024-09-15T21:48:21.638300+0000","last_unstale":"2024-09-15T21:49:04.811637+0000","last_undegraded":"2024-09-15T21:49:04.811637+0000","last_fullsized":"2024-09-15T21:49:04.811637+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'10","last_scrub_stamp":"2024-09-15T21:49:01.123927+0000","last_deep_scrub":"76'10","last_deep_scrub_stamp":"2024-09-15T21:49:01.123927+0000","last_clean_scrub_stamp":"2024-09-15T21:49:01.123927+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:31.924550+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"76'6","reported_seq":78,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.811530+0000","last_change":"2024-09-15T21:48:52.082835+0000","last_active":"2024-09-15T21:49:04.811530+0000","last_peered":"2024-09-15T21:49:04.811530+0000","last_clean":"2024-09-15T21:49:04.811530+0000","last_became_active":"2024-09-15T21:48:17.623246+0000","last_became_peered":"2024-09-15T21:48:17.623246+0000","last_unstale":"2024-09-15T21:49:04.811530+0000","last_undegraded":"2024-09-15T21:49:04.811530+0000","last_fullsized":"2024-09-15T21:49:04.811530+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:52.082659+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:52.082659+0000","last_clean_scrub_stamp":"2024-09-15T21:48:52.082659+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:33:52.302364+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"76'9","reported_seq":143,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:53.756214+0000","last_change":"2024-09-15T21:48:53.756242+0000","last_active":"2024-09-15T21:48:53.756214+0000","last_peered":"2024-09-15T21:48:53.756214+0000","last_clean":"2024-09-15T21:48:53.756214+0000","last_became_active":"2024-09-15T21:48:17.625642+0000","last_became_peered":"2024-09-15T21:48:17.625642+0000","last_unstale":"2024-09-15T21:48:53.756214+0000","last_undegraded":"2024-09-15T21:48:53.756214+0000","last_fullsized":"2024-09-15T21:48:53.756214+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:53.756124+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:53.756124+0000","last_clean_scrub_stamp":"2024-09-15T21:48:53.756124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:41.424624+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'11","reported_seq":155,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.810997+0000","last_change":"2024-09-15T21:48:55.608196+0000","last_active":"2024-09-15T21:49:04.810997+0000","last_peered":"2024-09-15T21:49:04.810997+0000","last_clean":"2024-09-15T21:49:04.810997+0000","last_became_active":"2024-09-15T21:48:17.622409+0000","last_became_peered":"2024-09-15T21:48:17.622409+0000","last_unstale":"2024-09-15T21:49:04.810997+0000","last_undegraded":"2024-09-15T21:49:04.810997+0000","last_fullsized":"2024-09-15T21:49:04.810997+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'11","last_scrub_stamp":"2024-09-15T21:48:55.608066+0000","last_deep_scrub":"76'11","last_deep_scrub_stamp":"2024-09-15T21:48:55.608066+0000","last_clean_scrub_stamp":"2024-09-15T21:48:55.608066+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:19:03.916606+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"76'11","reported_seq":151,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:51.395208+0000","last_change":"2024-09-15T21:48:51.395232+0000","last_active":"2024-09-15T21:48:51.395208+0000","last_peered":"2024-09-15T21:48:51.395208+0000","last_clean":"2024-09-15T21:48:51.395208+0000","last_became_active":"2024-09-15T21:48:21.636913+0000","last_became_peered":"2024-09-15T21:48:21.636913+0000","last_unstale":"2024-09-15T21:48:51.395208+0000","last_undegraded":"2024-09-15T21:48:51.395208+0000","last_fullsized":"2024-09-15T21:48:51.395208+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'11","last_scrub_stamp":"2024-09-15T21:48:51.395136+0000","last_deep_scrub":"76'11","last_deep_scrub_stamp":"2024-09-15T21:48:51.395136+0000","last_clean_scrub_stamp":"2024-09-15T21:48:51.395136+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:12.439417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"76'31","reported_seq":160,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:49.504371+0000","last_change":"2024-09-15T21:48:49.504411+0000","last_active":"2024-09-15T21:48:49.504371+0000","last_peered":"2024-09-15T21:48:49.504371+0000","last_clean":"2024-09-15T21:48:49.504371+0000","last_became_active":"2024-09-15T21:46:22.425866+0000","last_became_peered":"2024-09-15T21:46:22.425866+0000","last_unstale":"2024-09-15T21:48:49.504371+0000","last_undegraded":"2024-09-15T21:48:49.504371+0000","last_fullsized":"2024-09-15T21:48:49.504371+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'31","last_scrub_stamp":"2024-09-15T21:48:49.504271+0000","last_deep_scrub":"76'31","last_deep_scrub_stamp":"2024-09-15T21:48:49.504271+0000","last_clean_scrub_stamp":"2024-09-15T21:48:49.504271+0000","objects_scrubbed":6,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:17:14.994096+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35822,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"76'6","reported_seq":54,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:49.627048+0000","last_change":"2024-09-15T21:48:49.627091+0000","last_active":"2024-09-15T21:48:49.627048+0000","last_peered":"2024-09-15T21:48:49.627048+0000","last_clean":"2024-09-15T21:48:49.627048+0000","last_became_active":"2024-09-15T21:48:21.637151+0000","last_became_peered":"2024-09-15T21:48:21.637151+0000","last_unstale":"2024-09-15T21:48:49.627048+0000","last_undegraded":"2024-09-15T21:48:49.627048+0000","last_fullsized":"2024-09-15T21:48:49.627048+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:49.626943+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:49.626943+0000","last_clean_scrub_stamp":"2024-09-15T21:48:49.626943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:32.641079+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"76'15","reported_seq":144,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:44.723606+0000","last_change":"2024-09-15T21:48:44.723646+0000","last_active":"2024-09-15T21:48:44.723606+0000","last_peered":"2024-09-15T21:48:44.723606+0000","last_clean":"2024-09-15T21:48:44.723606+0000","last_became_active":"2024-09-15T21:46:22.427087+0000","last_became_peered":"2024-09-15T21:46:22.427087+0000","last_unstale":"2024-09-15T21:48:44.723606+0000","last_undegraded":"2024-09-15T21:48:44.723606+0000","last_fullsized":"2024-09-15T21:48:44.723606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'15","last_scrub_stamp":"2024-09-15T21:48:44.723504+0000","last_deep_scrub":"76'15","last_deep_scrub_stamp":"2024-09-15T21:48:44.723504+0000","last_clean_scrub_stamp":"2024-09-15T21:48:44.723504+0000","objects_scrubbed":6,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:25:06.067810+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"76'9","reported_seq":146,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:54.733659+0000","last_change":"2024-09-15T21:48:54.733684+0000","last_active":"2024-09-15T21:48:54.733659+0000","last_peered":"2024-09-15T21:48:54.733659+0000","last_clean":"2024-09-15T21:48:54.733659+0000","last_became_active":"2024-09-15T21:48:21.635022+0000","last_became_peered":"2024-09-15T21:48:21.635022+0000","last_unstale":"2024-09-15T21:48:54.733659+0000","last_undegraded":"2024-09-15T21:48:54.733659+0000","last_fullsized":"2024-09-15T21:48:54.733659+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:54.733535+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:54.733535+0000","last_clean_scrub_stamp":"2024-09-15T21:48:54.733535+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:14.431313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"76'9","reported_seq":81,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.811363+0000","last_change":"2024-09-15T21:48:59.065431+0000","last_active":"2024-09-15T21:49:04.811363+0000","last_peered":"2024-09-15T21:49:04.811363+0000","last_clean":"2024-09-15T21:49:04.811363+0000","last_became_active":"2024-09-15T21:48:17.625232+0000","last_became_peered":"2024-09-15T21:48:17.625232+0000","last_unstale":"2024-09-15T21:49:04.811363+0000","last_undegraded":"2024-09-15T21:49:04.811363+0000","last_fullsized":"2024-09-15T21:49:04.811363+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:59.065300+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:59.065300+0000","last_clean_scrub_stamp":"2024-09-15T21:48:59.065300+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:28.096961+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"76'15","reported_seq":63,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:54.646260+0000","last_change":"2024-09-15T21:48:54.646308+0000","last_active":"2024-09-15T21:48:54.646260+0000","last_peered":"2024-09-15T21:48:54.646260+0000","last_clean":"2024-09-15T21:48:54.646260+0000","last_became_active":"2024-09-15T21:48:21.638780+0000","last_became_peered":"2024-09-15T21:48:21.638780+0000","last_unstale":"2024-09-15T21:48:54.646260+0000","last_undegraded":"2024-09-15T21:48:54.646260+0000","last_fullsized":"2024-09-15T21:48:54.646260+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'15","last_scrub_stamp":"2024-09-15T21:48:54.646156+0000","last_deep_scrub":"76'15","last_deep_scrub_stamp":"2024-09-15T21:48:54.646156+0000","last_clean_scrub_stamp":"2024-09-15T21:48:54.646156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:32:55.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"76'8","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:47.468826+0000","last_change":"2024-09-15T21:48:47.468865+0000","last_active":"2024-09-15T21:48:47.468826+0000","last_peered":"2024-09-15T21:48:47.468826+0000","last_clean":"2024-09-15T21:48:47.468826+0000","last_became_active":"2024-09-15T21:46:22.425237+0000","last_became_peered":"2024-09-15T21:46:22.425237+0000","last_unstale":"2024-09-15T21:48:47.468826+0000","last_undegraded":"2024-09-15T21:48:47.468826+0000","last_fullsized":"2024-09-15T21:48:47.468826+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:47.468727+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:47.468727+0000","last_clean_scrub_stamp":"2024-09-15T21:48:47.468727+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:01:19.617491+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"76'6","reported_seq":133,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:50.463015+0000","last_change":"2024-09-15T21:48:50.463078+0000","last_active":"2024-09-15T21:48:50.463015+0000","last_peered":"2024-09-15T21:48:50.463015+0000","last_clean":"2024-09-15T21:48:50.463015+0000","last_became_active":"2024-09-15T21:46:23.427116+0000","last_became_peered":"2024-09-15T21:46:23.427116+0000","last_unstale":"2024-09-15T21:48:50.463015+0000","last_undegraded":"2024-09-15T21:48:50.463015+0000","last_fullsized":"2024-09-15T21:48:50.463015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:50.462904+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:50.462904+0000","last_clean_scrub_stamp":"2024-09-15T21:48:50.462904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:26.307690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"76'26","reported_seq":155,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:45.425256+0000","last_change":"2024-09-15T21:48:45.425316+0000","last_active":"2024-09-15T21:48:45.425256+0000","last_peered":"2024-09-15T21:48:45.425256+0000","last_clean":"2024-09-15T21:48:45.425256+0000","last_became_active":"2024-09-15T21:46:22.419620+0000","last_became_peered":"2024-09-15T21:46:22.419620+0000","last_unstale":"2024-09-15T21:48:45.425256+0000","last_undegraded":"2024-09-15T21:48:45.425256+0000","last_fullsized":"2024-09-15T21:48:45.425256+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'26","last_scrub_stamp":"2024-09-15T21:48:45.425176+0000","last_deep_scrub":"76'26","last_deep_scrub_stamp":"2024-09-15T21:48:45.425176+0000","last_clean_scrub_stamp":"2024-09-15T21:48:45.425176+0000","objects_scrubbed":8,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:25.986850+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"76'6","reported_seq":54,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:51.620646+0000","last_change":"2024-09-15T21:48:51.620674+0000","last_active":"2024-09-15T21:48:51.620646+0000","last_peered":"2024-09-15T21:48:51.620646+0000","last_clean":"2024-09-15T21:48:51.620646+0000","last_became_active":"2024-09-15T21:48:21.637855+0000","last_became_peered":"2024-09-15T21:48:21.637855+0000","last_unstale":"2024-09-15T21:48:51.620646+0000","last_undegraded":"2024-09-15T21:48:51.620646+0000","last_fullsized":"2024-09-15T21:48:51.620646+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:51.620560+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:51.620560+0000","last_clean_scrub_stamp":"2024-09-15T21:48:51.620560+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:04.449106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"76'4","reported_seq":136,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:46.526998+0000","last_change":"2024-09-15T21:48:46.527028+0000","last_active":"2024-09-15T21:48:46.526998+0000","last_peered":"2024-09-15T21:48:46.526998+0000","last_clean":"2024-09-15T21:48:46.526998+0000","last_became_active":"2024-09-15T21:46:22.429250+0000","last_became_peered":"2024-09-15T21:46:22.429250+0000","last_unstale":"2024-09-15T21:48:46.526998+0000","last_undegraded":"2024-09-15T21:48:46.526998+0000","last_fullsized":"2024-09-15T21:48:46.526998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'4","last_scrub_stamp":"2024-09-15T21:48:46.526911+0000","last_deep_scrub":"76'4","last_deep_scrub_stamp":"2024-09-15T21:48:46.526911+0000","last_clean_scrub_stamp":"2024-09-15T21:48:46.526911+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:07:44.627253+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"76'5","reported_seq":53,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:48.597831+0000","last_change":"2024-09-15T21:48:48.597881+0000","last_active":"2024-09-15T21:48:48.597831+0000","last_peered":"2024-09-15T21:48:48.597831+0000","last_clean":"2024-09-15T21:48:48.597831+0000","last_became_active":"2024-09-15T21:48:21.635051+0000","last_became_peered":"2024-09-15T21:48:21.635051+0000","last_unstale":"2024-09-15T21:48:48.597831+0000","last_undegraded":"2024-09-15T21:48:48.597831+0000","last_fullsized":"2024-09-15T21:48:48.597831+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'5","last_scrub_stamp":"2024-09-15T21:48:48.597739+0000","last_deep_scrub":"76'5","last_deep_scrub_stamp":"2024-09-15T21:48:48.597739+0000","last_clean_scrub_stamp":"2024-09-15T21:48:48.597739+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:24:36.880475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'7","reported_seq":94,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:50.499076+0000","last_change":"2024-09-15T21:48:50.499112+0000","last_active":"2024-09-15T21:48:50.499076+0000","last_peered":"2024-09-15T21:48:50.499076+0000","last_clean":"2024-09-15T21:48:50.499076+0000","last_became_active":"2024-09-15T21:48:27.659750+0000","last_became_peered":"2024-09-15T21:48:27.659750+0000","last_unstale":"2024-09-15T21:48:50.499076+0000","last_undegraded":"2024-09-15T21:48:50.499076+0000","last_fullsized":"2024-09-15T21:48:50.499076+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:50.498981+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:50.498981+0000","last_clean_scrub_stamp":"2024-09-15T21:48:50.498981+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:51.266968+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"76'9","reported_seq":57,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:56.679379+0000","last_change":"2024-09-15T21:48:56.679410+0000","last_active":"2024-09-15T21:48:56.679379+0000","last_peered":"2024-09-15T21:48:56.679379+0000","last_clean":"2024-09-15T21:48:56.679379+0000","last_became_active":"2024-09-15T21:48:21.639328+0000","last_became_peered":"2024-09-15T21:48:21.639328+0000","last_unstale":"2024-09-15T21:48:56.679379+0000","last_undegraded":"2024-09-15T21:48:56.679379+0000","last_fullsized":"2024-09-15T21:48:56.679379+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:56.679287+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:56.679287+0000","last_clean_scrub_stamp":"2024-09-15T21:48:56.679287+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:47:11.980903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'4","reported_seq":136,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.811144+0000","last_change":"2024-09-15T21:48:45.563143+0000","last_active":"2024-09-15T21:49:04.811144+0000","last_peered":"2024-09-15T21:49:04.811144+0000","last_clean":"2024-09-15T21:49:04.811144+0000","last_became_active":"2024-09-15T21:46:22.424771+0000","last_became_peered":"2024-09-15T21:46:22.424771+0000","last_unstale":"2024-09-15T21:49:04.811144+0000","last_undegraded":"2024-09-15T21:49:04.811144+0000","last_fullsized":"2024-09-15T21:49:04.811144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'4","last_scrub_stamp":"2024-09-15T21:48:45.563061+0000","last_deep_scrub":"76'4","last_deep_scrub_stamp":"2024-09-15T21:48:45.563061+0000","last_clean_scrub_stamp":"2024-09-15T21:48:45.563061+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:53:22.110654+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"76'6","reported_seq":74,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.811940+0000","last_change":"2024-09-15T21:48:48.087226+0000","last_active":"2024-09-15T21:49:04.811940+0000","last_peered":"2024-09-15T21:49:04.811940+0000","last_clean":"2024-09-15T21:49:04.811940+0000","last_became_active":"2024-09-15T21:48:17.617701+0000","last_became_peered":"2024-09-15T21:48:17.617701+0000","last_unstale":"2024-09-15T21:49:04.811940+0000","last_undegraded":"2024-09-15T21:49:04.811940+0000","last_fullsized":"2024-09-15T21:49:04.811940+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:48.087039+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:48.087039+0000","last_clean_scrub_stamp":"2024-09-15T21:48:48.087039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:54:45.723067+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"76'8","reported_seq":144,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:46.465764+0000","last_change":"2024-09-15T21:48:46.465791+0000","last_active":"2024-09-15T21:48:46.465764+0000","last_peered":"2024-09-15T21:48:46.465764+0000","last_clean":"2024-09-15T21:48:46.465764+0000","last_became_active":"2024-09-15T21:48:17.618782+0000","last_became_peered":"2024-09-15T21:48:17.618782+0000","last_unstale":"2024-09-15T21:48:46.465764+0000","last_undegraded":"2024-09-15T21:48:46.465764+0000","last_fullsized":"2024-09-15T21:48:46.465764+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:46.465703+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:46.465703+0000","last_clean_scrub_stamp":"2024-09-15T21:48:46.465703+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:48:10.554723+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"76'7","reported_seq":55,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:52.629613+0000","last_change":"2024-09-15T21:48:52.629639+0000","last_active":"2024-09-15T21:48:52.629613+0000","last_peered":"2024-09-15T21:48:52.629613+0000","last_clean":"2024-09-15T21:48:52.629613+0000","last_became_active":"2024-09-15T21:48:21.637883+0000","last_became_peered":"2024-09-15T21:48:21.637883+0000","last_unstale":"2024-09-15T21:48:52.629613+0000","last_undegraded":"2024-09-15T21:48:52.629613+0000","last_fullsized":"2024-09-15T21:48:52.629613+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:52.629527+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:52.629527+0000","last_clean_scrub_stamp":"2024-09-15T21:48:52.629527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:15.928443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'117","reported_seq":239,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:38.612564+0000","last_change":"2024-09-15T21:48:24.647129+0000","last_active":"2024-09-15T21:48:38.612564+0000","last_peered":"2024-09-15T21:48:38.612564+0000","last_clean":"2024-09-15T21:48:38.612564+0000","last_became_active":"2024-09-15T21:48:24.646783+0000","last_became_peered":"2024-09-15T21:48:24.646783+0000","last_unstale":"2024-09-15T21:48:38.612564+0000","last_undegraded":"2024-09-15T21:48:38.612564+0000","last_fullsized":"2024-09-15T21:48:38.612564+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:46:12.223400+0000","last_clean_scrub_stamp":"2024-09-15T21:46:12.223400+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"76'11","reported_seq":84,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.811353+0000","last_change":"2024-09-15T21:48:50.083153+0000","last_active":"2024-09-15T21:49:04.811353+0000","last_peered":"2024-09-15T21:49:04.811353+0000","last_clean":"2024-09-15T21:49:04.811353+0000","last_became_active":"2024-09-15T21:48:17.622159+0000","last_became_peered":"2024-09-15T21:48:17.622159+0000","last_unstale":"2024-09-15T21:49:04.811353+0000","last_undegraded":"2024-09-15T21:49:04.811353+0000","last_fullsized":"2024-09-15T21:49:04.811353+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'11","last_scrub_stamp":"2024-09-15T21:48:50.082972+0000","last_deep_scrub":"76'11","last_deep_scrub_stamp":"2024-09-15T21:48:50.082972+0000","last_clean_scrub_stamp":"2024-09-15T21:48:50.082972+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:11:27.039171+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"76'8","reported_seq":144,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:56.358872+0000","last_change":"2024-09-15T21:48:56.358952+0000","last_active":"2024-09-15T21:48:56.358872+0000","last_peered":"2024-09-15T21:48:56.358872+0000","last_clean":"2024-09-15T21:48:56.358872+0000","last_became_active":"2024-09-15T21:48:17.620738+0000","last_became_peered":"2024-09-15T21:48:17.620738+0000","last_unstale":"2024-09-15T21:48:56.358872+0000","last_undegraded":"2024-09-15T21:48:56.358872+0000","last_fullsized":"2024-09-15T21:48:56.358872+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:56.358740+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:56.358740+0000","last_clean_scrub_stamp":"2024-09-15T21:48:56.358740+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:37.880183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"76'5","reported_seq":53,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:58.628128+0000","last_change":"2024-09-15T21:48:58.628179+0000","last_active":"2024-09-15T21:48:58.628128+0000","last_peered":"2024-09-15T21:48:58.628128+0000","last_clean":"2024-09-15T21:48:58.628128+0000","last_became_active":"2024-09-15T21:48:21.636886+0000","last_became_peered":"2024-09-15T21:48:21.636886+0000","last_unstale":"2024-09-15T21:48:58.628128+0000","last_undegraded":"2024-09-15T21:48:58.628128+0000","last_fullsized":"2024-09-15T21:48:58.628128+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'5","last_scrub_stamp":"2024-09-15T21:48:58.628031+0000","last_deep_scrub":"76'5","last_deep_scrub_stamp":"2024-09-15T21:48:58.628031+0000","last_clean_scrub_stamp":"2024-09-15T21:48:58.628031+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:40.772019+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"76'5","reported_seq":132,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:49.747305+0000","last_change":"2024-09-15T21:48:49.747331+0000","last_active":"2024-09-15T21:48:49.747305+0000","last_peered":"2024-09-15T21:48:49.747305+0000","last_clean":"2024-09-15T21:48:49.747305+0000","last_became_active":"2024-09-15T21:46:23.430138+0000","last_became_peered":"2024-09-15T21:46:23.430138+0000","last_unstale":"2024-09-15T21:48:49.747305+0000","last_undegraded":"2024-09-15T21:48:49.747305+0000","last_fullsized":"2024-09-15T21:48:49.747305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'5","last_scrub_stamp":"2024-09-15T21:48:49.747222+0000","last_deep_scrub":"76'5","last_deep_scrub_stamp":"2024-09-15T21:48:49.747222+0000","last_clean_scrub_stamp":"2024-09-15T21:48:49.747222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:27.577276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"76'11","reported_seq":93,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:51.514245+0000","last_change":"2024-09-15T21:48:51.514311+0000","last_active":"2024-09-15T21:48:51.514245+0000","last_peered":"2024-09-15T21:48:51.514245+0000","last_clean":"2024-09-15T21:48:51.514245+0000","last_became_active":"2024-09-15T21:48:27.659880+0000","last_became_peered":"2024-09-15T21:48:27.659880+0000","last_unstale":"2024-09-15T21:48:51.514245+0000","last_undegraded":"2024-09-15T21:48:51.514245+0000","last_fullsized":"2024-09-15T21:48:51.514245+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'11","last_scrub_stamp":"2024-09-15T21:48:51.514143+0000","last_deep_scrub":"76'11","last_deep_scrub_stamp":"2024-09-15T21:48:51.514143+0000","last_clean_scrub_stamp":"2024-09-15T21:48:51.514143+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:51.672977+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":516,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'8","reported_seq":138,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.810802+0000","last_change":"2024-09-15T21:48:52.614278+0000","last_active":"2024-09-15T21:49:04.810802+0000","last_peered":"2024-09-15T21:49:04.810802+0000","last_clean":"2024-09-15T21:49:04.810802+0000","last_became_active":"2024-09-15T21:46:23.426185+0000","last_became_peered":"2024-09-15T21:46:23.426185+0000","last_unstale":"2024-09-15T21:49:04.810802+0000","last_undegraded":"2024-09-15T21:49:04.810802+0000","last_fullsized":"2024-09-15T21:49:04.810802+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:52.614124+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:52.614124+0000","last_clean_scrub_stamp":"2024-09-15T21:48:52.614124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:55:23.168353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"76'7","reported_seq":80,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.812117+0000","last_change":"2024-09-15T21:48:58.050362+0000","last_active":"2024-09-15T21:49:04.812117+0000","last_peered":"2024-09-15T21:49:04.812117+0000","last_clean":"2024-09-15T21:49:04.812117+0000","last_became_active":"2024-09-15T21:48:17.623936+0000","last_became_peered":"2024-09-15T21:48:17.623936+0000","last_unstale":"2024-09-15T21:49:04.812117+0000","last_undegraded":"2024-09-15T21:49:04.812117+0000","last_fullsized":"2024-09-15T21:49:04.812117+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:58.050232+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:58.050232+0000","last_clean_scrub_stamp":"2024-09-15T21:48:58.050232+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:26.762726+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"76'7","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:47.545034+0000","last_change":"2024-09-15T21:48:47.545064+0000","last_active":"2024-09-15T21:48:47.545034+0000","last_peered":"2024-09-15T21:48:47.545034+0000","last_clean":"2024-09-15T21:48:47.545034+0000","last_became_active":"2024-09-15T21:46:23.425416+0000","last_became_peered":"2024-09-15T21:46:23.425416+0000","last_unstale":"2024-09-15T21:48:47.545034+0000","last_undegraded":"2024-09-15T21:48:47.545034+0000","last_fullsized":"2024-09-15T21:48:47.545034+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:47.544941+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:47.544941+0000","last_clean_scrub_stamp":"2024-09-15T21:48:47.544941+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:34.400758+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"22'1","reported_seq":133,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.810877+0000","last_change":"2024-09-15T21:48:47.639673+0000","last_active":"2024-09-15T21:49:04.810877+0000","last_peered":"2024-09-15T21:49:04.810877+0000","last_clean":"2024-09-15T21:49:04.810877+0000","last_became_active":"2024-09-15T21:46:22.426210+0000","last_became_peered":"2024-09-15T21:46:22.426210+0000","last_unstale":"2024-09-15T21:49:04.810877+0000","last_undegraded":"2024-09-15T21:49:04.810877+0000","last_fullsized":"2024-09-15T21:49:04.810877+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'1","last_scrub_stamp":"2024-09-15T21:48:47.639518+0000","last_deep_scrub":"22'1","last_deep_scrub_stamp":"2024-09-15T21:48:47.639518+0000","last_clean_scrub_stamp":"2024-09-15T21:48:47.639518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:02.263758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"76'5","reported_seq":73,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.811837+0000","last_change":"2024-09-15T21:48:56.092263+0000","last_active":"2024-09-15T21:49:04.811837+0000","last_peered":"2024-09-15T21:49:04.811837+0000","last_clean":"2024-09-15T21:49:04.811837+0000","last_became_active":"2024-09-15T21:48:17.624852+0000","last_became_peered":"2024-09-15T21:48:17.624852+0000","last_unstale":"2024-09-15T21:49:04.811837+0000","last_undegraded":"2024-09-15T21:49:04.811837+0000","last_fullsized":"2024-09-15T21:49:04.811837+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'5","last_scrub_stamp":"2024-09-15T21:48:56.092106+0000","last_deep_scrub":"76'5","last_deep_scrub_stamp":"2024-09-15T21:48:56.092106+0000","last_clean_scrub_stamp":"2024-09-15T21:48:56.092106+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:44:19.841946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"76'7","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:49:00.540121+0000","last_change":"2024-09-15T21:49:00.540154+0000","last_active":"2024-09-15T21:49:00.540121+0000","last_peered":"2024-09-15T21:49:00.540121+0000","last_clean":"2024-09-15T21:49:00.540121+0000","last_became_active":"2024-09-15T21:48:21.639593+0000","last_became_peered":"2024-09-15T21:48:21.639593+0000","last_unstale":"2024-09-15T21:49:00.540121+0000","last_undegraded":"2024-09-15T21:49:00.540121+0000","last_fullsized":"2024-09-15T21:49:00.540121+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:49:00.540032+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:49:00.540032+0000","last_clean_scrub_stamp":"2024-09-15T21:49:00.540032+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:33.348343+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"76'9","reported_seq":139,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.810853+0000","last_change":"2024-09-15T21:48:51.635145+0000","last_active":"2024-09-15T21:49:04.810853+0000","last_peered":"2024-09-15T21:49:04.810853+0000","last_clean":"2024-09-15T21:49:04.810853+0000","last_became_active":"2024-09-15T21:46:23.426495+0000","last_became_peered":"2024-09-15T21:46:23.426495+0000","last_unstale":"2024-09-15T21:49:04.810853+0000","last_undegraded":"2024-09-15T21:49:04.810853+0000","last_fullsized":"2024-09-15T21:49:04.810853+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:51.635033+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:51.635033+0000","last_clean_scrub_stamp":"2024-09-15T21:48:51.635033+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:28:46.677592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"76'20","reported_seq":152,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.811150+0000","last_change":"2024-09-15T21:48:48.645351+0000","last_active":"2024-09-15T21:49:04.811150+0000","last_peered":"2024-09-15T21:49:04.811150+0000","last_clean":"2024-09-15T21:49:04.811150+0000","last_became_active":"2024-09-15T21:46:22.426855+0000","last_became_peered":"2024-09-15T21:46:22.426855+0000","last_unstale":"2024-09-15T21:49:04.811150+0000","last_undegraded":"2024-09-15T21:49:04.811150+0000","last_fullsized":"2024-09-15T21:49:04.811150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'20","last_scrub_stamp":"2024-09-15T21:48:48.645223+0000","last_deep_scrub":"76'20","last_deep_scrub_stamp":"2024-09-15T21:48:48.645223+0000","last_clean_scrub_stamp":"2024-09-15T21:48:48.645223+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:25:49.419359+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"76'5","reported_seq":144,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:57.343584+0000","last_change":"2024-09-15T21:48:57.343614+0000","last_active":"2024-09-15T21:48:57.343584+0000","last_peered":"2024-09-15T21:48:57.343584+0000","last_clean":"2024-09-15T21:48:57.343584+0000","last_became_active":"2024-09-15T21:48:21.639541+0000","last_became_peered":"2024-09-15T21:48:21.639541+0000","last_unstale":"2024-09-15T21:48:57.343584+0000","last_undegraded":"2024-09-15T21:48:57.343584+0000","last_fullsized":"2024-09-15T21:48:57.343584+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'5","last_scrub_stamp":"2024-09-15T21:48:57.343459+0000","last_deep_scrub":"76'5","last_deep_scrub_stamp":"2024-09-15T21:48:57.343459+0000","last_clean_scrub_stamp":"2024-09-15T21:48:57.343459+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:07:37.229275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'12","reported_seq":171,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:58.351556+0000","last_change":"2024-09-15T21:48:58.351594+0000","last_active":"2024-09-15T21:48:58.351556+0000","last_peered":"2024-09-15T21:48:58.351556+0000","last_clean":"2024-09-15T21:48:58.351556+0000","last_became_active":"2024-09-15T21:48:21.643251+0000","last_became_peered":"2024-09-15T21:48:21.643251+0000","last_unstale":"2024-09-15T21:48:58.351556+0000","last_undegraded":"2024-09-15T21:48:58.351556+0000","last_fullsized":"2024-09-15T21:48:58.351556+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'12","last_scrub_stamp":"2024-09-15T21:48:58.351467+0000","last_deep_scrub":"76'12","last_deep_scrub_stamp":"2024-09-15T21:48:58.351467+0000","last_clean_scrub_stamp":"2024-09-15T21:48:58.351467+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:41.587053+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":44,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"76'7","reported_seq":134,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:46.687688+0000","last_change":"2024-09-15T21:48:46.687711+0000","last_active":"2024-09-15T21:48:46.687688+0000","last_peered":"2024-09-15T21:48:46.687688+0000","last_clean":"2024-09-15T21:48:46.687688+0000","last_became_active":"2024-09-15T21:46:23.424007+0000","last_became_peered":"2024-09-15T21:46:23.424007+0000","last_unstale":"2024-09-15T21:48:46.687688+0000","last_undegraded":"2024-09-15T21:48:46.687688+0000","last_fullsized":"2024-09-15T21:48:46.687688+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:46.687610+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:46.687610+0000","last_clean_scrub_stamp":"2024-09-15T21:48:46.687610+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:57.560558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"76'3","reported_seq":145,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:52.781109+0000","last_change":"2024-09-15T21:48:52.781137+0000","last_active":"2024-09-15T21:48:52.781109+0000","last_peered":"2024-09-15T21:48:52.781109+0000","last_clean":"2024-09-15T21:48:52.781109+0000","last_became_active":"2024-09-15T21:48:17.623453+0000","last_became_peered":"2024-09-15T21:48:17.623453+0000","last_unstale":"2024-09-15T21:48:52.781109+0000","last_undegraded":"2024-09-15T21:48:52.781109+0000","last_fullsized":"2024-09-15T21:48:52.781109+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'3","last_scrub_stamp":"2024-09-15T21:48:52.781027+0000","last_deep_scrub":"76'3","last_deep_scrub_stamp":"2024-09-15T21:48:52.781027+0000","last_clean_scrub_stamp":"2024-09-15T21:48:52.781027+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:28.346534+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"76'2","reported_seq":132,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:49.465929+0000","last_change":"2024-09-15T21:48:49.465962+0000","last_active":"2024-09-15T21:48:49.465929+0000","last_peered":"2024-09-15T21:48:49.465929+0000","last_clean":"2024-09-15T21:48:49.465929+0000","last_became_active":"2024-09-15T21:46:23.429283+0000","last_became_peered":"2024-09-15T21:46:23.429283+0000","last_unstale":"2024-09-15T21:48:49.465929+0000","last_undegraded":"2024-09-15T21:48:49.465929+0000","last_fullsized":"2024-09-15T21:48:49.465929+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'2","last_scrub_stamp":"2024-09-15T21:48:49.465824+0000","last_deep_scrub":"76'2","last_deep_scrub_stamp":"2024-09-15T21:48:49.465824+0000","last_clean_scrub_stamp":"2024-09-15T21:48:49.465824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:28.475614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"76'11","reported_seq":173,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.810925+0000","last_change":"2024-09-15T21:48:56.592779+0000","last_active":"2024-09-15T21:49:04.810925+0000","last_peered":"2024-09-15T21:49:04.810925+0000","last_clean":"2024-09-15T21:49:04.810925+0000","last_became_active":"2024-09-15T21:48:21.635898+0000","last_became_peered":"2024-09-15T21:48:21.635898+0000","last_unstale":"2024-09-15T21:49:04.810925+0000","last_undegraded":"2024-09-15T21:49:04.810925+0000","last_fullsized":"2024-09-15T21:49:04.810925+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'11","last_scrub_stamp":"2024-09-15T21:48:56.592585+0000","last_deep_scrub":"76'11","last_deep_scrub_stamp":"2024-09-15T21:48:56.592585+0000","last_clean_scrub_stamp":"2024-09-15T21:48:56.592585+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:41:39.659727+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":180,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"76'9","reported_seq":138,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:51.405471+0000","last_change":"2024-09-15T21:48:51.405503+0000","last_active":"2024-09-15T21:48:51.405471+0000","last_peered":"2024-09-15T21:48:51.405471+0000","last_clean":"2024-09-15T21:48:51.405471+0000","last_became_active":"2024-09-15T21:46:23.428301+0000","last_became_peered":"2024-09-15T21:46:23.428301+0000","last_unstale":"2024-09-15T21:48:51.405471+0000","last_undegraded":"2024-09-15T21:48:51.405471+0000","last_fullsized":"2024-09-15T21:48:51.405471+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:51.405367+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:51.405367+0000","last_clean_scrub_stamp":"2024-09-15T21:48:51.405367+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:51:53.179487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"76'6","reported_seq":138,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.810992+0000","last_change":"2024-09-15T21:48:46.605361+0000","last_active":"2024-09-15T21:49:04.810992+0000","last_peered":"2024-09-15T21:49:04.810992+0000","last_clean":"2024-09-15T21:49:04.810992+0000","last_became_active":"2024-09-15T21:46:22.425478+0000","last_became_peered":"2024-09-15T21:46:22.425478+0000","last_unstale":"2024-09-15T21:49:04.810992+0000","last_undegraded":"2024-09-15T21:49:04.810992+0000","last_fullsized":"2024-09-15T21:49:04.810992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:46.605195+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:46.605195+0000","last_clean_scrub_stamp":"2024-09-15T21:48:46.605195+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.180764+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"76'8","reported_seq":137,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:48.371384+0000","last_change":"2024-09-15T21:48:48.371421+0000","last_active":"2024-09-15T21:48:48.371384+0000","last_peered":"2024-09-15T21:48:48.371384+0000","last_clean":"2024-09-15T21:48:48.371384+0000","last_became_active":"2024-09-15T21:46:23.425547+0000","last_became_peered":"2024-09-15T21:46:23.425547+0000","last_unstale":"2024-09-15T21:48:48.371384+0000","last_undegraded":"2024-09-15T21:48:48.371384+0000","last_fullsized":"2024-09-15T21:48:48.371384+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:48.371307+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:48.371307+0000","last_clean_scrub_stamp":"2024-09-15T21:48:48.371307+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:22:44.003854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'7","reported_seq":56,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:59.581476+0000","last_change":"2024-09-15T21:48:59.581511+0000","last_active":"2024-09-15T21:48:59.581476+0000","last_peered":"2024-09-15T21:48:59.581476+0000","last_clean":"2024-09-15T21:48:59.581476+0000","last_became_active":"2024-09-15T21:48:21.639223+0000","last_became_peered":"2024-09-15T21:48:21.639223+0000","last_unstale":"2024-09-15T21:48:59.581476+0000","last_undegraded":"2024-09-15T21:48:59.581476+0000","last_fullsized":"2024-09-15T21:48:59.581476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:59.581383+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:59.581383+0000","last_clean_scrub_stamp":"2024-09-15T21:48:59.581383+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:19.311173+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"76'5","reported_seq":132,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:53.455161+0000","last_change":"2024-09-15T21:48:53.455212+0000","last_active":"2024-09-15T21:48:53.455161+0000","last_peered":"2024-09-15T21:48:53.455161+0000","last_clean":"2024-09-15T21:48:53.455161+0000","last_became_active":"2024-09-15T21:46:23.434448+0000","last_became_peered":"2024-09-15T21:46:23.434448+0000","last_unstale":"2024-09-15T21:48:53.455161+0000","last_undegraded":"2024-09-15T21:48:53.455161+0000","last_fullsized":"2024-09-15T21:48:53.455161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'5","last_scrub_stamp":"2024-09-15T21:48:53.455056+0000","last_deep_scrub":"76'5","last_deep_scrub_stamp":"2024-09-15T21:48:53.455056+0000","last_clean_scrub_stamp":"2024-09-15T21:48:53.455056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:22.628886+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"76'15","reported_seq":92,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:48.464874+0000","last_change":"2024-09-15T21:48:48.464909+0000","last_active":"2024-09-15T21:48:48.464874+0000","last_peered":"2024-09-15T21:48:48.464874+0000","last_clean":"2024-09-15T21:48:48.464874+0000","last_became_active":"2024-09-15T21:48:27.656753+0000","last_became_peered":"2024-09-15T21:48:27.656753+0000","last_unstale":"2024-09-15T21:48:48.464874+0000","last_undegraded":"2024-09-15T21:48:48.464874+0000","last_fullsized":"2024-09-15T21:48:48.464874+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'15","last_scrub_stamp":"2024-09-15T21:48:48.464792+0000","last_deep_scrub":"76'15","last_deep_scrub_stamp":"2024-09-15T21:48:48.464792+0000","last_clean_scrub_stamp":"2024-09-15T21:48:48.464792+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:42.908766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"76'8","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:49:02.496464+0000","last_change":"2024-09-15T21:49:02.496550+0000","last_active":"2024-09-15T21:49:02.496464+0000","last_peered":"2024-09-15T21:49:02.496464+0000","last_clean":"2024-09-15T21:49:02.496464+0000","last_became_active":"2024-09-15T21:48:21.641517+0000","last_became_peered":"2024-09-15T21:48:21.641517+0000","last_unstale":"2024-09-15T21:49:02.496464+0000","last_undegraded":"2024-09-15T21:49:02.496464+0000","last_fullsized":"2024-09-15T21:49:02.496464+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:49:02.496360+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:49:02.496360+0000","last_clean_scrub_stamp":"2024-09-15T21:49:02.496360+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:47.427912+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"76'9","reported_seq":77,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-15T21:49:04.811702+0000","last_change":"2024-09-15T21:48:54.102481+0000","last_active":"2024-09-15T21:49:04.811702+0000","last_peered":"2024-09-15T21:49:04.811702+0000","last_clean":"2024-09-15T21:49:04.811702+0000","last_became_active":"2024-09-15T21:48:17.622019+0000","last_became_peered":"2024-09-15T21:48:17.622019+0000","last_unstale":"2024-09-15T21:49:04.811702+0000","last_undegraded":"2024-09-15T21:49:04.811702+0000","last_fullsized":"2024-09-15T21:49:04.811702+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:54.102314+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:54.102314+0000","last_clean_scrub_stamp":"2024-09-15T21:48:54.102314+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:10:45.955871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"76'20","reported_seq":71,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:49:01.535070+0000","last_change":"2024-09-15T21:49:01.535114+0000","last_active":"2024-09-15T21:49:01.535070+0000","last_peered":"2024-09-15T21:49:01.535070+0000","last_clean":"2024-09-15T21:49:01.535070+0000","last_became_active":"2024-09-15T21:48:21.640156+0000","last_became_peered":"2024-09-15T21:48:21.640156+0000","last_unstale":"2024-09-15T21:49:01.535070+0000","last_undegraded":"2024-09-15T21:49:01.535070+0000","last_fullsized":"2024-09-15T21:49:01.535070+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'20","last_scrub_stamp":"2024-09-15T21:49:01.534951+0000","last_deep_scrub":"76'20","last_deep_scrub_stamp":"2024-09-15T21:49:01.534951+0000","last_clean_scrub_stamp":"2024-09-15T21:49:01.534951+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:53.603363+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"76'6","reported_seq":140,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:51.729390+0000","last_change":"2024-09-15T21:48:51.729417+0000","last_active":"2024-09-15T21:48:51.729390+0000","last_peered":"2024-09-15T21:48:51.729390+0000","last_clean":"2024-09-15T21:48:51.729390+0000","last_became_active":"2024-09-15T21:48:17.622843+0000","last_became_peered":"2024-09-15T21:48:17.622843+0000","last_unstale":"2024-09-15T21:48:51.729390+0000","last_undegraded":"2024-09-15T21:48:51.729390+0000","last_fullsized":"2024-09-15T21:48:51.729390+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:51.729300+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:51.729300+0000","last_clean_scrub_stamp":"2024-09-15T21:48:51.729300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:01.496891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'2","reported_seq":133,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:47.382804+0000","last_change":"2024-09-15T21:48:47.382828+0000","last_active":"2024-09-15T21:48:47.382804+0000","last_peered":"2024-09-15T21:48:47.382804+0000","last_clean":"2024-09-15T21:48:47.382804+0000","last_became_active":"2024-09-15T21:46:22.428263+0000","last_became_peered":"2024-09-15T21:46:22.428263+0000","last_unstale":"2024-09-15T21:48:47.382804+0000","last_undegraded":"2024-09-15T21:48:47.382804+0000","last_fullsized":"2024-09-15T21:48:47.382804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'2","last_scrub_stamp":"2024-09-15T21:48:47.382727+0000","last_deep_scrub":"22'2","last_deep_scrub_stamp":"2024-09-15T21:48:47.382727+0000","last_clean_scrub_stamp":"2024-09-15T21:48:47.382727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:27.068168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"76'5","reported_seq":134,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:50.410405+0000","last_change":"2024-09-15T21:48:50.410459+0000","last_active":"2024-09-15T21:48:50.410405+0000","last_peered":"2024-09-15T21:48:50.410405+0000","last_clean":"2024-09-15T21:48:50.410405+0000","last_became_active":"2024-09-15T21:46:23.427546+0000","last_became_peered":"2024-09-15T21:46:23.427546+0000","last_unstale":"2024-09-15T21:48:50.410405+0000","last_undegraded":"2024-09-15T21:48:50.410405+0000","last_fullsized":"2024-09-15T21:48:50.410405+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'5","last_scrub_stamp":"2024-09-15T21:48:50.410294+0000","last_deep_scrub":"76'5","last_deep_scrub_stamp":"2024-09-15T21:48:50.410294+0000","last_clean_scrub_stamp":"2024-09-15T21:48:50.410294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:02.410504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"76'19","reported_seq":186,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-15T21:48:56.752248+0000","last_change":"2024-09-15T21:48:56.752292+0000","last_active":"2024-09-15T21:48:56.752248+0000","last_peered":"2024-09-15T21:48:56.752248+0000","last_clean":"2024-09-15T21:48:56.752248+0000","last_became_active":"2024-09-15T21:48:21.642406+0000","last_became_peered":"2024-09-15T21:48:21.642406+0000","last_unstale":"2024-09-15T21:48:56.752248+0000","last_undegraded":"2024-09-15T21:48:56.752248+0000","last_fullsized":"2024-09-15T21:48:56.752248+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'19","last_scrub_stamp":"2024-09-15T21:48:56.752163+0000","last_deep_scrub":"76'19","last_deep_scrub_stamp":"2024-09-15T21:48:56.752163+0000","last_clean_scrub_stamp":"2024-09-15T21:48:56.752163+0000","objects_scrubbed":6,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:32:45.460289+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1344,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":503,"ondisk_log_size":503,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43045,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":252,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":4802,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":335872,"data_stored":86090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6368,"internal_metadata":0},"log_size":315,"ondisk_log_size":315,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476774,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1507328,"data_stored":1002267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509478,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1515520,"data_stored":1003890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":946176,"data_stored":447637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":57,"seq":244813135884,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29104,"kb_used_data":924,"kb_used_omap":7,"kb_used_meta":28152,"kb_avail":94342736,"statfs":{"total":96636764160,"available":96606961664,"internally_reserved":0,"allocated":946176,"data_stored":421072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7963,"internal_metadata":28827877},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509479,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":909312,"data_stored":412355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509478,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":921600,"data_stored":412891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542183,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":925696,"data_stored":418672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":954368,"data_stored":448472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":37447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6368,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":36612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:49:08.174 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-09-15T21:46:22.402110+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:49:08.177 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-15T21:46:12.223400+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=46, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=48, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:49:08.178 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T21:49:17.182 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T21:49:17.186 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T21:49:28.179 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:49:28.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.289+0000 7f0bb9c77640 1 Processor -- start 2024-09-15T21:49:28.291 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.289+0000 7f0bb9c77640 1 -- start start 2024-09-15T21:49:28.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.290+0000 7f0bb9c77640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0bb4152fe0 0x7f0bb415cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:28.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.290+0000 7f0bb9c77640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0bb40c3b10 0x7f0bb40730f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:28.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.290+0000 7f0bb9c77640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bb40c6ad0 con 0x7f0bb4158db0 2024-09-15T21:49:28.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.290+0000 7f0bb9c77640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f0bb40c4070 con 0x7f0bb4152fe0 2024-09-15T21:49:28.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.290+0000 7f0bb9c77640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb40c4350 con 0x7f0bb40c3b10 2024-09-15T21:49:28.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.290+0000 7f0bb37fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0bb4152fe0 0x7f0bb415cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:28.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.290+0000 7f0bb37fe640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f0bb4152fe0 0x7f0bb415cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46526/0 (socket says 172.21.15.100:46526) 2024-09-15T21:49:28.292 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.290+0000 7f0bb37fe640 1 -- 172.21.15.100:0/4182269519 learned_addr learned my addr 172.21.15.100:0/4182269519 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:49:28.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.290+0000 7f0bb2ffd640 1 --2- 172.21.15.100:0/4182269519 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0bb40c3b10 0x7f0bb40730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:28.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.290+0000 7f0bb37fe640 1 -- 172.21.15.100:0/4182269519 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0bb40c3b10 msgr2=0x7f0bb40730f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:49:28.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.290+0000 7f0bb37fe640 1 --2- 172.21.15.100:0/4182269519 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0bb40c3b10 0x7f0bb40730f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.290+0000 7f0bb37fe640 1 -- 172.21.15.100:0/4182269519 >> v1:172.21.15.122:6789/0 conn(0x7f0bb4158db0 legacy=0x7f0bb40cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:49:28.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.290+0000 7f0bb37fe640 1 -- 172.21.15.100:0/4182269519 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb40c4630 con 0x7f0bb4152fe0 2024-09-15T21:49:28.293 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.290+0000 7f0bb2ffd640 1 --2- 172.21.15.100:0/4182269519 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0bb40c3b10 0x7f0bb40730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:49:28.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.291+0000 7f0bb37fe640 1 --2- 172.21.15.100:0/4182269519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb4152fe0 0x7f0bb415cd40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009870 tx=0x7f0ba402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7626a875ef62d58c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:28.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.291+0000 7f0bb27fc640 1 -- 172.21.15.100:0/4182269519 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4037070 con 0x7f0bb4152fe0 2024-09-15T21:49:28.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.291+0000 7f0bb27fc640 1 -- 172.21.15.100:0/4182269519 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba4032440 con 0x7f0bb4152fe0 2024-09-15T21:49:28.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.291+0000 7f0bb27fc640 1 -- 172.21.15.100:0/4182269519 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4032810 con 0x7f0bb4152fe0 2024-09-15T21:49:28.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.292+0000 7f0bb9c77640 1 -- 172.21.15.100:0/4182269519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb4152fe0 msgr2=0x7f0bb415cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:28.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.292+0000 7f0bb9c77640 1 --2- 172.21.15.100:0/4182269519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb4152fe0 0x7f0bb415cd40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009870 tx=0x7f0ba402a2a0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.292+0000 7f0bb9c77640 1 -- 172.21.15.100:0/4182269519 shutdown_connections 2024-09-15T21:49:28.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.292+0000 7f0bb9c77640 1 --2- 172.21.15.100:0/4182269519 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0bb40c3b10 0x7f0bb40730f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.292+0000 7f0bb9c77640 1 --2- 172.21.15.100:0/4182269519 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb4152fe0 0x7f0bb415cd40 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.295 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.292+0000 7f0bb9c77640 1 -- 172.21.15.100:0/4182269519 >> 172.21.15.100:0/4182269519 conn(0x7f0bb4067dd0 msgr2=0x7f0bb406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:28.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.292+0000 7f0bb9c77640 1 -- 172.21.15.100:0/4182269519 shutdown_connections 2024-09-15T21:49:28.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.292+0000 7f0bb9c77640 1 -- 172.21.15.100:0/4182269519 wait complete. 2024-09-15T21:49:28.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.293+0000 7f0bb9c77640 1 Processor -- start 2024-09-15T21:49:28.296 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.293+0000 7f0bb9c77640 1 -- start start 2024-09-15T21:49:28.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.294+0000 7f0bb9c77640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bb40c3b10 0x7f0bb41f1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:28.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.294+0000 7f0bb9c77640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0bb4152fe0 0x7f0bb4072a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:28.298 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.294+0000 7f0bb9c77640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb4158db0 0x7f0bb41e6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.294+0000 7f0bb9c77640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb415d280 con 0x7f0bb4158db0 2024-09-15T21:49:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.294+0000 7f0bb9c77640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb415c820 con 0x7f0bb40c3b10 2024-09-15T21:49:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.294+0000 7f0bb9c77640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb41533c0 con 0x7f0bb4152fe0 2024-09-15T21:49:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.294+0000 7f0bb3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb4158db0 0x7f0bb41e6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.294+0000 7f0bb3fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb4158db0 0x7f0bb41e6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46540/0 (socket says 172.21.15.100:46540) 2024-09-15T21:49:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.294+0000 7f0bb3fff640 1 -- 172.21.15.100:0/2311512329 learned_addr learned my addr 172.21.15.100:0/2311512329 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:49:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.294+0000 7f0bb2ffd640 1 --2- 172.21.15.100:0/2311512329 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0bb4152fe0 0x7f0bb4072a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.294+0000 7f0bb37fe640 1 --2- 172.21.15.100:0/2311512329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bb40c3b10 0x7f0bb41f1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.294+0000 7f0bb3fff640 1 -- 172.21.15.100:0/2311512329 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0bb4152fe0 msgr2=0x7f0bb4072a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.294+0000 7f0bb3fff640 1 --2- 172.21.15.100:0/2311512329 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0bb4152fe0 0x7f0bb4072a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.294+0000 7f0bb3fff640 1 -- 172.21.15.100:0/2311512329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bb40c3b10 msgr2=0x7f0bb41f1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.294+0000 7f0bb3fff640 1 --2- 172.21.15.100:0/2311512329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bb40c3b10 0x7f0bb41f1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.294+0000 7f0bb3fff640 1 -- 172.21.15.100:0/2311512329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb4071fd0 con 0x7f0bb4158db0 2024-09-15T21:49:28.299 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.294+0000 7f0bb3fff640 1 --2- 172.21.15.100:0/2311512329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb4158db0 0x7f0bb41e6930 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00b9a0 tx=0x7f0bac00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:28.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.295+0000 7f0b9bfff640 1 -- 172.21.15.100:0/2311512329 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac019070 con 0x7f0bb4158db0 2024-09-15T21:49:28.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.295+0000 7f0b9bfff640 1 -- 172.21.15.100:0/2311512329 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bac00cba0 con 0x7f0bb4158db0 2024-09-15T21:49:28.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.295+0000 7f0bb9c77640 1 -- 172.21.15.100:0/2311512329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb41e4dc0 con 0x7f0bb4158db0 2024-09-15T21:49:28.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.295+0000 7f0b9bfff640 1 -- 172.21.15.100:0/2311512329 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bac014440 con 0x7f0bb4158db0 2024-09-15T21:49:28.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.295+0000 7f0bb9c77640 1 -- 172.21.15.100:0/2311512329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb4072550 con 0x7f0bb4158db0 2024-09-15T21:49:28.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.296+0000 7f0bb9c77640 1 -- 172.21.15.100:0/2311512329 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bb4072750 con 0x7f0bb4158db0 2024-09-15T21:49:28.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.296+0000 7f0b9bfff640 1 -- 172.21.15.100:0/2311512329 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f0bac01c070 con 0x7f0bb4158db0 2024-09-15T21:49:28.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.297+0000 7f0b9bfff640 1 --2- 172.21.15.100:0/2311512329 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0ba006c2e0 0x7f0ba006e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:28.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.297+0000 7f0b9bfff640 1 -- 172.21.15.100:0/2311512329 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5886+0+0 (secure 0 0 0) 0x7f0bac08b1d0 con 0x7f0bb4158db0 2024-09-15T21:49:28.300 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.297+0000 7f0bb37fe640 1 --2- 172.21.15.100:0/2311512329 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0ba006c2e0 0x7f0ba006e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:28.301 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.297+0000 7f0bb37fe640 1 --2- 172.21.15.100:0/2311512329 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0ba006c2e0 0x7f0ba006e7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4034000 tx=0x7f0ba402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:28.302 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.301+0000 7f0b9bfff640 1 -- 172.21.15.100:0/2311512329 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f0bb4072750 con 0x7f0bb4158db0 2024-09-15T21:49:28.465 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.464+0000 7f0bb9c77640 1 -- 172.21.15.100:0/2311512329 --> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0bb40c96d0 con 0x7f0ba006c2e0 2024-09-15T21:49:28.471 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.469+0000 7f0b9bfff640 1 -- 172.21.15.100:0/2311512329 <== mgr.4107 v2:172.21.15.100:6808/3516908621 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252692 (secure 0 0 0) 0x7f0bb40c96d0 con 0x7f0ba006c2e0 2024-09-15T21:49:28.474 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:49:28.475 INFO:teuthology.orchestra.run.smithi100.stderr:dumped all 2024-09-15T21:49:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.472+0000 7f0bb9c77640 1 -- 172.21.15.100:0/2311512329 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0ba006c2e0 msgr2=0x7f0ba006e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.472+0000 7f0bb9c77640 1 --2- 172.21.15.100:0/2311512329 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0ba006c2e0 0x7f0ba006e7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4034000 tx=0x7f0ba402e000 comp rx=0 tx=0).stop 2024-09-15T21:49:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.472+0000 7f0bb9c77640 1 -- 172.21.15.100:0/2311512329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb4158db0 msgr2=0x7f0bb41e6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.472+0000 7f0bb9c77640 1 --2- 172.21.15.100:0/2311512329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb4158db0 0x7f0bb41e6930 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0bac00b9a0 tx=0x7f0bac00be60 comp rx=0 tx=0).stop 2024-09-15T21:49:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.472+0000 7f0bb9c77640 1 -- 172.21.15.100:0/2311512329 shutdown_connections 2024-09-15T21:49:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.472+0000 7f0bb9c77640 1 --2- 172.21.15.100:0/2311512329 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f0ba006c2e0 0x7f0ba006e7a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.477 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.472+0000 7f0bb9c77640 1 --2- 172.21.15.100:0/2311512329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0bb4158db0 0x7f0bb41e6930 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.472+0000 7f0bb9c77640 1 --2- 172.21.15.100:0/2311512329 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f0bb4152fe0 0x7f0bb4072a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.472+0000 7f0bb9c77640 1 --2- 172.21.15.100:0/2311512329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0bb40c3b10 0x7f0bb41f1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.472+0000 7f0bb9c77640 1 -- 172.21.15.100:0/2311512329 >> 172.21.15.100:0/2311512329 conn(0x7f0bb4067dd0 msgr2=0x7f0bb415b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:28.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.472+0000 7f0bb9c77640 1 -- 172.21.15.100:0/2311512329 shutdown_connections 2024-09-15T21:49:28.478 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.472+0000 7f0bb9c77640 1 -- 172.21.15.100:0/2311512329 wait complete. 2024-09-15T21:49:28.486 INFO:teuthology.orchestra.run.smithi100.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-09-15T21:49:27.844958+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633413,"num_objects":101,"num_object_clones":0,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":413,"num_write_kb":2538,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":1185538,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":937,"ondisk_log_size":937,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238460,"kb_used_data":8592,"kb_used_omap":18,"kb_used_meta":229677,"kb_avail":754736260,"statfs":{"total":773094113280,"available":772849930240,"internally_reserved":0,"allocated":8798208,"data_stored":4652698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19119,"internal_metadata":235189585},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003297"},"pg_stats":[{"pgid":"3.3f","version":"76'8","reported_seq":60,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.819032+0000","last_change":"2024-09-15T21:48:47.609967+0000","last_active":"2024-09-15T21:49:05.819032+0000","last_peered":"2024-09-15T21:49:05.819032+0000","last_clean":"2024-09-15T21:49:05.819032+0000","last_became_active":"2024-09-15T21:48:21.637402+0000","last_became_peered":"2024-09-15T21:48:21.637402+0000","last_unstale":"2024-09-15T21:49:05.819032+0000","last_undegraded":"2024-09-15T21:49:05.819032+0000","last_fullsized":"2024-09-15T21:49:05.819032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:47.609834+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:47.609834+0000","last_clean_scrub_stamp":"2024-09-15T21:48:47.609834+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:25:11.657264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"76'4","reported_seq":136,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.929526+0000","last_change":"2024-09-15T21:48:49.661819+0000","last_active":"2024-09-15T21:49:06.929526+0000","last_peered":"2024-09-15T21:49:06.929526+0000","last_clean":"2024-09-15T21:49:06.929526+0000","last_became_active":"2024-09-15T21:46:23.418471+0000","last_became_peered":"2024-09-15T21:46:23.418471+0000","last_unstale":"2024-09-15T21:49:06.929526+0000","last_undegraded":"2024-09-15T21:49:06.929526+0000","last_fullsized":"2024-09-15T21:49:06.929526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'4","last_scrub_stamp":"2024-09-15T21:48:49.661665+0000","last_deep_scrub":"76'4","last_deep_scrub_stamp":"2024-09-15T21:48:49.661665+0000","last_clean_scrub_stamp":"2024-09-15T21:48:49.661665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:06:35.692625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"76'7","reported_seq":138,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.162956+0000","last_change":"2024-09-15T21:48:48.719368+0000","last_active":"2024-09-15T21:49:07.162956+0000","last_peered":"2024-09-15T21:49:07.162956+0000","last_clean":"2024-09-15T21:49:07.162956+0000","last_became_active":"2024-09-15T21:46:23.426146+0000","last_became_peered":"2024-09-15T21:46:23.426146+0000","last_unstale":"2024-09-15T21:49:07.162956+0000","last_undegraded":"2024-09-15T21:49:07.162956+0000","last_fullsized":"2024-09-15T21:49:07.162956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:48.719251+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:48.719251+0000","last_clean_scrub_stamp":"2024-09-15T21:48:48.719251+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:02:17.487930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"76'8","reported_seq":78,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.167354+0000","last_change":"2024-09-15T21:48:53.078030+0000","last_active":"2024-09-15T21:49:06.167354+0000","last_peered":"2024-09-15T21:49:06.167354+0000","last_clean":"2024-09-15T21:49:06.167354+0000","last_became_active":"2024-09-15T21:48:17.623606+0000","last_became_peered":"2024-09-15T21:48:17.623606+0000","last_unstale":"2024-09-15T21:49:06.167354+0000","last_undegraded":"2024-09-15T21:49:06.167354+0000","last_fullsized":"2024-09-15T21:49:06.167354+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:53.077912+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:53.077912+0000","last_clean_scrub_stamp":"2024-09-15T21:48:53.077912+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:28:10.764425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"76'4","reported_seq":56,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.818455+0000","last_change":"2024-09-15T21:48:55.677538+0000","last_active":"2024-09-15T21:49:05.818455+0000","last_peered":"2024-09-15T21:49:05.818455+0000","last_clean":"2024-09-15T21:49:05.818455+0000","last_became_active":"2024-09-15T21:48:21.638410+0000","last_became_peered":"2024-09-15T21:48:21.638410+0000","last_unstale":"2024-09-15T21:49:05.818455+0000","last_undegraded":"2024-09-15T21:49:05.818455+0000","last_fullsized":"2024-09-15T21:49:05.818455+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'4","last_scrub_stamp":"2024-09-15T21:48:55.677402+0000","last_deep_scrub":"76'4","last_deep_scrub_stamp":"2024-09-15T21:48:55.677402+0000","last_clean_scrub_stamp":"2024-09-15T21:48:55.677402+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:42.615279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"76'6","reported_seq":76,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.167346+0000","last_change":"2024-09-15T21:48:57.067351+0000","last_active":"2024-09-15T21:49:06.167346+0000","last_peered":"2024-09-15T21:49:06.167346+0000","last_clean":"2024-09-15T21:49:06.167346+0000","last_became_active":"2024-09-15T21:48:17.624439+0000","last_became_peered":"2024-09-15T21:48:17.624439+0000","last_unstale":"2024-09-15T21:49:06.167346+0000","last_undegraded":"2024-09-15T21:49:06.167346+0000","last_fullsized":"2024-09-15T21:49:06.167346+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:57.067212+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:57.067212+0000","last_clean_scrub_stamp":"2024-09-15T21:48:57.067212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:34:24.410710+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"76'5","reported_seq":57,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.818328+0000","last_change":"2024-09-15T21:48:50.591979+0000","last_active":"2024-09-15T21:49:05.818328+0000","last_peered":"2024-09-15T21:49:05.818328+0000","last_clean":"2024-09-15T21:49:05.818328+0000","last_became_active":"2024-09-15T21:48:21.638310+0000","last_became_peered":"2024-09-15T21:48:21.638310+0000","last_unstale":"2024-09-15T21:49:05.818328+0000","last_undegraded":"2024-09-15T21:49:05.818328+0000","last_fullsized":"2024-09-15T21:49:05.818328+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'5","last_scrub_stamp":"2024-09-15T21:48:50.591846+0000","last_deep_scrub":"76'5","last_deep_scrub_stamp":"2024-09-15T21:48:50.591846+0000","last_clean_scrub_stamp":"2024-09-15T21:48:50.591846+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:37:32.407604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"76'9","reported_seq":121,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.815804+0000","last_change":"2024-09-15T21:48:49.465760+0000","last_active":"2024-09-15T21:49:05.815804+0000","last_peered":"2024-09-15T21:49:05.815804+0000","last_clean":"2024-09-15T21:49:05.815804+0000","last_became_active":"2024-09-15T21:48:27.656800+0000","last_became_peered":"2024-09-15T21:48:27.656800+0000","last_unstale":"2024-09-15T21:49:05.815804+0000","last_undegraded":"2024-09-15T21:49:05.815804+0000","last_fullsized":"2024-09-15T21:49:05.815804+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:49.465653+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:49.465653+0000","last_clean_scrub_stamp":"2024-09-15T21:48:49.465653+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:22:51.070326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.37","version":"76'11","reported_seq":63,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.818957+0000","last_change":"2024-09-15T21:48:57.641112+0000","last_active":"2024-09-15T21:49:05.818957+0000","last_peered":"2024-09-15T21:49:05.818957+0000","last_clean":"2024-09-15T21:49:05.818957+0000","last_became_active":"2024-09-15T21:48:21.641920+0000","last_became_peered":"2024-09-15T21:48:21.641920+0000","last_unstale":"2024-09-15T21:49:05.818957+0000","last_undegraded":"2024-09-15T21:49:05.818957+0000","last_fullsized":"2024-09-15T21:49:05.818957+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'11","last_scrub_stamp":"2024-09-15T21:48:57.640998+0000","last_deep_scrub":"76'11","last_deep_scrub_stamp":"2024-09-15T21:48:57.640998+0000","last_clean_scrub_stamp":"2024-09-15T21:48:57.640998+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:10.988113+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"76'16","reported_seq":154,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.817774+0000","last_change":"2024-09-15T21:48:56.402567+0000","last_active":"2024-09-15T21:49:05.817774+0000","last_peered":"2024-09-15T21:49:05.817774+0000","last_clean":"2024-09-15T21:49:05.817774+0000","last_became_active":"2024-09-15T21:48:17.621800+0000","last_became_peered":"2024-09-15T21:48:17.621800+0000","last_unstale":"2024-09-15T21:49:05.817774+0000","last_undegraded":"2024-09-15T21:49:05.817774+0000","last_fullsized":"2024-09-15T21:49:05.817774+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'16","last_scrub_stamp":"2024-09-15T21:48:56.402433+0000","last_deep_scrub":"76'16","last_deep_scrub_stamp":"2024-09-15T21:48:56.402433+0000","last_clean_scrub_stamp":"2024-09-15T21:48:56.402433+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:47.418031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"76'6","reported_seq":142,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.817681+0000","last_change":"2024-09-15T21:48:54.409819+0000","last_active":"2024-09-15T21:49:05.817681+0000","last_peered":"2024-09-15T21:49:05.817681+0000","last_clean":"2024-09-15T21:49:05.817681+0000","last_became_active":"2024-09-15T21:47:05.734921+0000","last_became_peered":"2024-09-15T21:47:05.734921+0000","last_unstale":"2024-09-15T21:49:05.817681+0000","last_undegraded":"2024-09-15T21:49:05.817681+0000","last_fullsized":"2024-09-15T21:49:05.817681+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:54.409705+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:54.409705+0000","last_clean_scrub_stamp":"2024-09-15T21:48:54.409705+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:04.597525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"76'7","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.816821+0000","last_change":"2024-09-15T21:48:50.443565+0000","last_active":"2024-09-15T21:49:05.816821+0000","last_peered":"2024-09-15T21:49:05.816821+0000","last_clean":"2024-09-15T21:49:05.816821+0000","last_became_active":"2024-09-15T21:46:23.430170+0000","last_became_peered":"2024-09-15T21:46:23.430170+0000","last_unstale":"2024-09-15T21:49:05.816821+0000","last_undegraded":"2024-09-15T21:49:05.816821+0000","last_fullsized":"2024-09-15T21:49:05.816821+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:50.443445+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:50.443445+0000","last_clean_scrub_stamp":"2024-09-15T21:48:50.443445+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:02.921106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"76'10","reported_seq":143,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.162467+0000","last_change":"2024-09-15T21:48:49.365097+0000","last_active":"2024-09-15T21:49:07.162467+0000","last_peered":"2024-09-15T21:49:07.162467+0000","last_clean":"2024-09-15T21:49:07.162467+0000","last_became_active":"2024-09-15T21:46:23.427621+0000","last_became_peered":"2024-09-15T21:46:23.427621+0000","last_unstale":"2024-09-15T21:49:07.162467+0000","last_undegraded":"2024-09-15T21:49:07.162467+0000","last_fullsized":"2024-09-15T21:49:07.162467+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'10","last_scrub_stamp":"2024-09-15T21:48:49.364982+0000","last_deep_scrub":"76'10","last_deep_scrub_stamp":"2024-09-15T21:48:49.364982+0000","last_clean_scrub_stamp":"2024-09-15T21:48:49.364982+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:25:08.887670+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"76'10","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.162614+0000","last_change":"2024-09-15T21:48:47.718675+0000","last_active":"2024-09-15T21:49:07.162614+0000","last_peered":"2024-09-15T21:49:07.162614+0000","last_clean":"2024-09-15T21:49:07.162614+0000","last_became_active":"2024-09-15T21:46:23.424447+0000","last_became_peered":"2024-09-15T21:46:23.424447+0000","last_unstale":"2024-09-15T21:49:07.162614+0000","last_undegraded":"2024-09-15T21:49:07.162614+0000","last_fullsized":"2024-09-15T21:49:07.162614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'10","last_scrub_stamp":"2024-09-15T21:48:47.718575+0000","last_deep_scrub":"76'10","last_deep_scrub_stamp":"2024-09-15T21:48:47.718575+0000","last_clean_scrub_stamp":"2024-09-15T21:48:47.718575+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:48:01.821709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"76'7","reported_seq":139,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.929246+0000","last_change":"2024-09-15T21:48:54.618833+0000","last_active":"2024-09-15T21:49:06.929246+0000","last_peered":"2024-09-15T21:49:06.929246+0000","last_clean":"2024-09-15T21:49:06.929246+0000","last_became_active":"2024-09-15T21:46:23.430070+0000","last_became_peered":"2024-09-15T21:46:23.430070+0000","last_unstale":"2024-09-15T21:49:06.929246+0000","last_undegraded":"2024-09-15T21:49:06.929246+0000","last_fullsized":"2024-09-15T21:49:06.929246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:54.618713+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:54.618713+0000","last_clean_scrub_stamp":"2024-09-15T21:48:54.618713+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:01.651899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"76'9","reported_seq":79,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.166992+0000","last_change":"2024-09-15T21:48:55.108052+0000","last_active":"2024-09-15T21:49:06.166992+0000","last_peered":"2024-09-15T21:49:06.166992+0000","last_clean":"2024-09-15T21:49:06.166992+0000","last_became_active":"2024-09-15T21:48:17.624764+0000","last_became_peered":"2024-09-15T21:48:17.624764+0000","last_unstale":"2024-09-15T21:49:06.166992+0000","last_undegraded":"2024-09-15T21:49:06.166992+0000","last_fullsized":"2024-09-15T21:49:06.166992+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:55.107907+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:55.107907+0000","last_clean_scrub_stamp":"2024-09-15T21:48:55.107907+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:44:35.180572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"76'8","reported_seq":10,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.820719+0000","last_change":"2024-09-15T21:49:05.820063+0000","last_active":"2024-09-15T21:49:05.820719+0000","last_peered":"2024-09-15T21:49:05.820719+0000","last_clean":"2024-09-15T21:49:05.820719+0000","last_became_active":"2024-09-15T21:49:05.819644+0000","last_became_peered":"2024-09-15T21:49:05.819644+0000","last_unstale":"2024-09-15T21:49:05.820719+0000","last_undegraded":"2024-09-15T21:49:05.820719+0000","last_fullsized":"2024-09-15T21:49:05.820719+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:49:02.106280+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:49:02.106280+0000","last_clean_scrub_stamp":"2024-09-15T21:49:02.106280+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:24:39.433364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"76'10","reported_seq":99,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.815941+0000","last_change":"2024-09-15T21:48:52.489084+0000","last_active":"2024-09-15T21:49:05.815941+0000","last_peered":"2024-09-15T21:49:05.815941+0000","last_clean":"2024-09-15T21:49:05.815941+0000","last_became_active":"2024-09-15T21:48:27.661441+0000","last_became_peered":"2024-09-15T21:48:27.661441+0000","last_unstale":"2024-09-15T21:49:05.815941+0000","last_undegraded":"2024-09-15T21:49:05.815941+0000","last_fullsized":"2024-09-15T21:49:05.815941+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'10","last_scrub_stamp":"2024-09-15T21:48:52.488966+0000","last_deep_scrub":"76'10","last_deep_scrub_stamp":"2024-09-15T21:48:52.488966+0000","last_clean_scrub_stamp":"2024-09-15T21:48:52.488966+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:18:52.212403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"76'7","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.818677+0000","last_change":"2024-09-15T21:48:53.606284+0000","last_active":"2024-09-15T21:49:05.818677+0000","last_peered":"2024-09-15T21:49:05.818677+0000","last_clean":"2024-09-15T21:49:05.818677+0000","last_became_active":"2024-09-15T21:48:21.639009+0000","last_became_peered":"2024-09-15T21:48:21.639009+0000","last_unstale":"2024-09-15T21:49:05.818677+0000","last_undegraded":"2024-09-15T21:49:05.818677+0000","last_fullsized":"2024-09-15T21:49:05.818677+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:53.606083+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:53.606083+0000","last_clean_scrub_stamp":"2024-09-15T21:48:53.606083+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:00:38.744519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"76'8","reported_seq":139,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.816648+0000","last_change":"2024-09-15T21:48:52.421114+0000","last_active":"2024-09-15T21:49:05.816648+0000","last_peered":"2024-09-15T21:49:05.816648+0000","last_clean":"2024-09-15T21:49:05.816648+0000","last_became_active":"2024-09-15T21:46:23.434297+0000","last_became_peered":"2024-09-15T21:46:23.434297+0000","last_unstale":"2024-09-15T21:49:05.816648+0000","last_undegraded":"2024-09-15T21:49:05.816648+0000","last_fullsized":"2024-09-15T21:49:05.816648+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:52.420959+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:52.420959+0000","last_clean_scrub_stamp":"2024-09-15T21:48:52.420959+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:41.523791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"76'11","reported_seq":152,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.163018+0000","last_change":"2024-09-15T21:48:55.715221+0000","last_active":"2024-09-15T21:49:07.163018+0000","last_peered":"2024-09-15T21:49:07.163018+0000","last_clean":"2024-09-15T21:49:07.163018+0000","last_became_active":"2024-09-15T21:48:21.641876+0000","last_became_peered":"2024-09-15T21:48:21.641876+0000","last_unstale":"2024-09-15T21:49:07.163018+0000","last_undegraded":"2024-09-15T21:49:07.163018+0000","last_fullsized":"2024-09-15T21:49:07.163018+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'11","last_scrub_stamp":"2024-09-15T21:48:55.715108+0000","last_deep_scrub":"76'11","last_deep_scrub_stamp":"2024-09-15T21:48:55.715108+0000","last_clean_scrub_stamp":"2024-09-15T21:48:55.715108+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:09:45.242437+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"76'10","reported_seq":82,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.167072+0000","last_change":"2024-09-15T21:49:00.105773+0000","last_active":"2024-09-15T21:49:06.167072+0000","last_peered":"2024-09-15T21:49:06.167072+0000","last_clean":"2024-09-15T21:49:06.167072+0000","last_became_active":"2024-09-15T21:48:21.638073+0000","last_became_peered":"2024-09-15T21:48:21.638073+0000","last_unstale":"2024-09-15T21:49:06.167072+0000","last_undegraded":"2024-09-15T21:49:06.167072+0000","last_fullsized":"2024-09-15T21:49:06.167072+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'10","last_scrub_stamp":"2024-09-15T21:49:00.105642+0000","last_deep_scrub":"76'10","last_deep_scrub_stamp":"2024-09-15T21:49:00.105642+0000","last_clean_scrub_stamp":"2024-09-15T21:49:00.105642+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:05:39.997437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"76'8","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.162469+0000","last_change":"2024-09-15T21:48:53.385859+0000","last_active":"2024-09-15T21:49:07.162469+0000","last_peered":"2024-09-15T21:49:07.162469+0000","last_clean":"2024-09-15T21:49:07.162469+0000","last_became_active":"2024-09-15T21:46:23.430719+0000","last_became_peered":"2024-09-15T21:46:23.430719+0000","last_unstale":"2024-09-15T21:49:07.162469+0000","last_undegraded":"2024-09-15T21:49:07.162469+0000","last_fullsized":"2024-09-15T21:49:07.162469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:53.385747+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:53.385747+0000","last_clean_scrub_stamp":"2024-09-15T21:48:53.385747+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:10:16.486598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"76'6","reported_seq":76,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.167268+0000","last_change":"2024-09-15T21:48:49.107693+0000","last_active":"2024-09-15T21:49:06.167268+0000","last_peered":"2024-09-15T21:49:06.167268+0000","last_clean":"2024-09-15T21:49:06.167268+0000","last_became_active":"2024-09-15T21:48:17.621639+0000","last_became_peered":"2024-09-15T21:48:17.621639+0000","last_unstale":"2024-09-15T21:49:06.167268+0000","last_undegraded":"2024-09-15T21:49:06.167268+0000","last_fullsized":"2024-09-15T21:49:06.167268+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:49.107527+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:49.107527+0000","last_clean_scrub_stamp":"2024-09-15T21:48:49.107527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:32:03.896706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"76'2","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.162517+0000","last_change":"2024-09-15T21:48:55.308557+0000","last_active":"2024-09-15T21:49:07.162517+0000","last_peered":"2024-09-15T21:49:07.162517+0000","last_clean":"2024-09-15T21:49:07.162517+0000","last_became_active":"2024-09-15T21:46:23.432317+0000","last_became_peered":"2024-09-15T21:46:23.432317+0000","last_unstale":"2024-09-15T21:49:07.162517+0000","last_undegraded":"2024-09-15T21:49:07.162517+0000","last_fullsized":"2024-09-15T21:49:07.162517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'2","last_scrub_stamp":"2024-09-15T21:48:55.308414+0000","last_deep_scrub":"76'2","last_deep_scrub_stamp":"2024-09-15T21:48:55.308414+0000","last_clean_scrub_stamp":"2024-09-15T21:48:55.308414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:14:12.319169+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"76'15","reported_seq":156,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.817788+0000","last_change":"2024-09-15T21:48:57.379036+0000","last_active":"2024-09-15T21:49:05.817788+0000","last_peered":"2024-09-15T21:49:05.817788+0000","last_clean":"2024-09-15T21:49:05.817788+0000","last_became_active":"2024-09-15T21:48:21.638212+0000","last_became_peered":"2024-09-15T21:48:21.638212+0000","last_unstale":"2024-09-15T21:49:05.817788+0000","last_undegraded":"2024-09-15T21:49:05.817788+0000","last_fullsized":"2024-09-15T21:49:05.817788+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'15","last_scrub_stamp":"2024-09-15T21:48:57.378918+0000","last_deep_scrub":"76'15","last_deep_scrub_stamp":"2024-09-15T21:48:57.378918+0000","last_clean_scrub_stamp":"2024-09-15T21:48:57.378918+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:43:18.293141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"76'6","reported_seq":138,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.929346+0000","last_change":"2024-09-15T21:48:50.631757+0000","last_active":"2024-09-15T21:49:06.929346+0000","last_peered":"2024-09-15T21:49:06.929346+0000","last_clean":"2024-09-15T21:49:06.929346+0000","last_became_active":"2024-09-15T21:46:23.421501+0000","last_became_peered":"2024-09-15T21:46:23.421501+0000","last_unstale":"2024-09-15T21:49:06.929346+0000","last_undegraded":"2024-09-15T21:49:06.929346+0000","last_fullsized":"2024-09-15T21:49:06.929346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:50.631641+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:50.631641+0000","last_clean_scrub_stamp":"2024-09-15T21:48:50.631641+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:20:22.937617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"76'4","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.162971+0000","last_change":"2024-09-15T21:48:54.340105+0000","last_active":"2024-09-15T21:49:07.162971+0000","last_peered":"2024-09-15T21:49:07.162971+0000","last_clean":"2024-09-15T21:49:07.162971+0000","last_became_active":"2024-09-15T21:46:23.432326+0000","last_became_peered":"2024-09-15T21:46:23.432326+0000","last_unstale":"2024-09-15T21:49:07.162971+0000","last_undegraded":"2024-09-15T21:49:07.162971+0000","last_fullsized":"2024-09-15T21:49:07.162971+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'4","last_scrub_stamp":"2024-09-15T21:48:54.339983+0000","last_deep_scrub":"76'4","last_deep_scrub_stamp":"2024-09-15T21:48:54.339983+0000","last_clean_scrub_stamp":"2024-09-15T21:48:54.339983+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:09:02.395350+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"76'7","reported_seq":138,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.163071+0000","last_change":"2024-09-15T21:48:45.671000+0000","last_active":"2024-09-15T21:49:07.163071+0000","last_peered":"2024-09-15T21:49:07.163071+0000","last_clean":"2024-09-15T21:49:07.163071+0000","last_became_active":"2024-09-15T21:46:23.422734+0000","last_became_peered":"2024-09-15T21:46:23.422734+0000","last_unstale":"2024-09-15T21:49:07.163071+0000","last_undegraded":"2024-09-15T21:49:07.163071+0000","last_fullsized":"2024-09-15T21:49:07.163071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:45.670897+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:45.670897+0000","last_clean_scrub_stamp":"2024-09-15T21:48:45.670897+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:30:38.392752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"76'8","reported_seq":78,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.167220+0000","last_change":"2024-09-15T21:48:51.066043+0000","last_active":"2024-09-15T21:49:06.167220+0000","last_peered":"2024-09-15T21:49:06.167220+0000","last_clean":"2024-09-15T21:49:06.167220+0000","last_became_active":"2024-09-15T21:48:17.621473+0000","last_became_peered":"2024-09-15T21:48:17.621473+0000","last_unstale":"2024-09-15T21:49:06.167220+0000","last_undegraded":"2024-09-15T21:49:06.167220+0000","last_fullsized":"2024-09-15T21:49:06.167220+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:51.065913+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:51.065913+0000","last_clean_scrub_stamp":"2024-09-15T21:48:51.065913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:05:51.774365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"76'12","reported_seq":150,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.815835+0000","last_change":"2024-09-15T21:48:47.449165+0000","last_active":"2024-09-15T21:49:05.815835+0000","last_peered":"2024-09-15T21:49:05.815835+0000","last_clean":"2024-09-15T21:49:05.815835+0000","last_became_active":"2024-09-15T21:48:17.616499+0000","last_became_peered":"2024-09-15T21:48:17.616499+0000","last_unstale":"2024-09-15T21:49:05.815835+0000","last_undegraded":"2024-09-15T21:49:05.815835+0000","last_fullsized":"2024-09-15T21:49:05.815835+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'12","last_scrub_stamp":"2024-09-15T21:48:47.449074+0000","last_deep_scrub":"76'12","last_deep_scrub_stamp":"2024-09-15T21:48:47.449074+0000","last_clean_scrub_stamp":"2024-09-15T21:48:47.449074+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:45:43.158651+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"76'9","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.816394+0000","last_change":"2024-09-15T21:48:51.414676+0000","last_active":"2024-09-15T21:49:05.816394+0000","last_peered":"2024-09-15T21:49:05.816394+0000","last_clean":"2024-09-15T21:49:05.816394+0000","last_became_active":"2024-09-15T21:46:23.429376+0000","last_became_peered":"2024-09-15T21:46:23.429376+0000","last_unstale":"2024-09-15T21:49:05.816394+0000","last_undegraded":"2024-09-15T21:49:05.816394+0000","last_fullsized":"2024-09-15T21:49:05.816394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:51.414516+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:51.414516+0000","last_clean_scrub_stamp":"2024-09-15T21:48:51.414516+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:12:44.641131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"76'7","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.817328+0000","last_change":"2024-09-15T21:48:48.503982+0000","last_active":"2024-09-15T21:49:05.817328+0000","last_peered":"2024-09-15T21:49:05.817328+0000","last_clean":"2024-09-15T21:49:05.817328+0000","last_became_active":"2024-09-15T21:46:23.429442+0000","last_became_peered":"2024-09-15T21:46:23.429442+0000","last_unstale":"2024-09-15T21:49:05.817328+0000","last_undegraded":"2024-09-15T21:49:05.817328+0000","last_fullsized":"2024-09-15T21:49:05.817328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:48.503827+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:48.503827+0000","last_clean_scrub_stamp":"2024-09-15T21:48:48.503827+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:10:11.107071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"76'8","reported_seq":165,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.162825+0000","last_change":"2024-09-15T21:48:58.317631+0000","last_active":"2024-09-15T21:49:07.162825+0000","last_peered":"2024-09-15T21:49:07.162825+0000","last_clean":"2024-09-15T21:49:07.162825+0000","last_became_active":"2024-09-15T21:48:21.636972+0000","last_became_peered":"2024-09-15T21:48:21.636972+0000","last_unstale":"2024-09-15T21:49:07.162825+0000","last_undegraded":"2024-09-15T21:49:07.162825+0000","last_fullsized":"2024-09-15T21:49:07.162825+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:58.317488+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:58.317488+0000","last_clean_scrub_stamp":"2024-09-15T21:48:58.317488+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:23:30.154332+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1019,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1756,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"76'8","reported_seq":139,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.163144+0000","last_change":"2024-09-15T21:48:50.758175+0000","last_active":"2024-09-15T21:49:07.163144+0000","last_peered":"2024-09-15T21:49:07.163144+0000","last_clean":"2024-09-15T21:49:07.163144+0000","last_became_active":"2024-09-15T21:46:23.433944+0000","last_became_peered":"2024-09-15T21:46:23.433944+0000","last_unstale":"2024-09-15T21:49:07.163144+0000","last_undegraded":"2024-09-15T21:49:07.163144+0000","last_fullsized":"2024-09-15T21:49:07.163144+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:50.758058+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:50.758058+0000","last_clean_scrub_stamp":"2024-09-15T21:48:50.758058+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:25:56.670468+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"76'10","reported_seq":143,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.817599+0000","last_change":"2024-09-15T21:48:48.512029+0000","last_active":"2024-09-15T21:49:05.817599+0000","last_peered":"2024-09-15T21:49:05.817599+0000","last_clean":"2024-09-15T21:49:05.817599+0000","last_became_active":"2024-09-15T21:46:22.425315+0000","last_became_peered":"2024-09-15T21:46:22.425315+0000","last_unstale":"2024-09-15T21:49:05.817599+0000","last_undegraded":"2024-09-15T21:49:05.817599+0000","last_fullsized":"2024-09-15T21:49:05.817599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'10","last_scrub_stamp":"2024-09-15T21:48:48.511883+0000","last_deep_scrub":"76'10","last_deep_scrub_stamp":"2024-09-15T21:48:48.511883+0000","last_clean_scrub_stamp":"2024-09-15T21:48:48.511883+0000","objects_scrubbed":3,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:14.895078+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"76'5","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.162617+0000","last_change":"2024-09-15T21:48:46.431777+0000","last_active":"2024-09-15T21:49:07.162617+0000","last_peered":"2024-09-15T21:49:07.162617+0000","last_clean":"2024-09-15T21:49:07.162617+0000","last_became_active":"2024-09-15T21:46:22.425226+0000","last_became_peered":"2024-09-15T21:46:22.425226+0000","last_unstale":"2024-09-15T21:49:07.162617+0000","last_undegraded":"2024-09-15T21:49:07.162617+0000","last_fullsized":"2024-09-15T21:49:07.162617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'5","last_scrub_stamp":"2024-09-15T21:48:46.431641+0000","last_deep_scrub":"76'5","last_deep_scrub_stamp":"2024-09-15T21:48:46.431641+0000","last_clean_scrub_stamp":"2024-09-15T21:48:46.431641+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:18:53.830908+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"76'8","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.162300+0000","last_change":"2024-09-15T21:48:52.415850+0000","last_active":"2024-09-15T21:49:07.162300+0000","last_peered":"2024-09-15T21:49:07.162300+0000","last_clean":"2024-09-15T21:49:07.162300+0000","last_became_active":"2024-09-15T21:46:23.428902+0000","last_became_peered":"2024-09-15T21:46:23.428902+0000","last_unstale":"2024-09-15T21:49:07.162300+0000","last_undegraded":"2024-09-15T21:49:07.162300+0000","last_fullsized":"2024-09-15T21:49:07.162300+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:52.415717+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:52.415717+0000","last_clean_scrub_stamp":"2024-09-15T21:48:52.415717+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:32:42.694715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"76'4","reported_seq":79,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.166822+0000","last_change":"2024-09-15T21:48:47.101716+0000","last_active":"2024-09-15T21:49:06.166822+0000","last_peered":"2024-09-15T21:49:06.166822+0000","last_clean":"2024-09-15T21:49:06.166822+0000","last_became_active":"2024-09-15T21:48:17.616704+0000","last_became_peered":"2024-09-15T21:48:17.616704+0000","last_unstale":"2024-09-15T21:49:06.166822+0000","last_undegraded":"2024-09-15T21:49:06.166822+0000","last_fullsized":"2024-09-15T21:49:06.166822+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'4","last_scrub_stamp":"2024-09-15T21:48:47.101544+0000","last_deep_scrub":"76'4","last_deep_scrub_stamp":"2024-09-15T21:48:47.101544+0000","last_clean_scrub_stamp":"2024-09-15T21:48:47.101544+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:08:28.355881+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"76'10","reported_seq":142,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.929234+0000","last_change":"2024-09-15T21:48:53.616901+0000","last_active":"2024-09-15T21:49:06.929234+0000","last_peered":"2024-09-15T21:49:06.929234+0000","last_clean":"2024-09-15T21:49:06.929234+0000","last_became_active":"2024-09-15T21:46:23.427200+0000","last_became_peered":"2024-09-15T21:46:23.427200+0000","last_unstale":"2024-09-15T21:49:06.929234+0000","last_undegraded":"2024-09-15T21:49:06.929234+0000","last_fullsized":"2024-09-15T21:49:06.929234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'10","last_scrub_stamp":"2024-09-15T21:48:53.616791+0000","last_deep_scrub":"76'10","last_deep_scrub_stamp":"2024-09-15T21:48:53.616791+0000","last_clean_scrub_stamp":"2024-09-15T21:48:53.616791+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:40:13.547220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"76'9","reported_seq":150,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.817432+0000","last_change":"2024-09-15T21:48:55.414295+0000","last_active":"2024-09-15T21:49:05.817432+0000","last_peered":"2024-09-15T21:49:05.817432+0000","last_clean":"2024-09-15T21:49:05.817432+0000","last_became_active":"2024-09-15T21:48:17.621860+0000","last_became_peered":"2024-09-15T21:48:17.621860+0000","last_unstale":"2024-09-15T21:49:05.817432+0000","last_undegraded":"2024-09-15T21:49:05.817432+0000","last_fullsized":"2024-09-15T21:49:05.817432+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:55.414151+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:55.414151+0000","last_clean_scrub_stamp":"2024-09-15T21:48:55.414151+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:32:39.534367+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"76'10","reported_seq":82,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.167270+0000","last_change":"2024-09-15T21:49:01.124047+0000","last_active":"2024-09-15T21:49:06.167270+0000","last_peered":"2024-09-15T21:49:06.167270+0000","last_clean":"2024-09-15T21:49:06.167270+0000","last_became_active":"2024-09-15T21:48:21.638300+0000","last_became_peered":"2024-09-15T21:48:21.638300+0000","last_unstale":"2024-09-15T21:49:06.167270+0000","last_undegraded":"2024-09-15T21:49:06.167270+0000","last_fullsized":"2024-09-15T21:49:06.167270+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'10","last_scrub_stamp":"2024-09-15T21:49:01.123927+0000","last_deep_scrub":"76'10","last_deep_scrub_stamp":"2024-09-15T21:49:01.123927+0000","last_clean_scrub_stamp":"2024-09-15T21:49:01.123927+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:31.924550+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"76'6","reported_seq":80,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.167140+0000","last_change":"2024-09-15T21:48:52.082835+0000","last_active":"2024-09-15T21:49:06.167140+0000","last_peered":"2024-09-15T21:49:06.167140+0000","last_clean":"2024-09-15T21:49:06.167140+0000","last_became_active":"2024-09-15T21:48:17.623246+0000","last_became_peered":"2024-09-15T21:48:17.623246+0000","last_unstale":"2024-09-15T21:49:06.167140+0000","last_undegraded":"2024-09-15T21:49:06.167140+0000","last_fullsized":"2024-09-15T21:49:06.167140+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:52.082659+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:52.082659+0000","last_clean_scrub_stamp":"2024-09-15T21:48:52.082659+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:33:52.302364+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"76'9","reported_seq":147,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.163168+0000","last_change":"2024-09-15T21:48:53.756242+0000","last_active":"2024-09-15T21:49:07.163168+0000","last_peered":"2024-09-15T21:49:07.163168+0000","last_clean":"2024-09-15T21:49:07.163168+0000","last_became_active":"2024-09-15T21:48:17.625642+0000","last_became_peered":"2024-09-15T21:48:17.625642+0000","last_unstale":"2024-09-15T21:49:07.163168+0000","last_undegraded":"2024-09-15T21:49:07.163168+0000","last_fullsized":"2024-09-15T21:49:07.163168+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:53.756124+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:53.756124+0000","last_clean_scrub_stamp":"2024-09-15T21:48:53.756124+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:35:41.424624+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"76'11","reported_seq":157,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.929473+0000","last_change":"2024-09-15T21:48:55.608196+0000","last_active":"2024-09-15T21:49:06.929473+0000","last_peered":"2024-09-15T21:49:06.929473+0000","last_clean":"2024-09-15T21:49:06.929473+0000","last_became_active":"2024-09-15T21:48:17.622409+0000","last_became_peered":"2024-09-15T21:48:17.622409+0000","last_unstale":"2024-09-15T21:49:06.929473+0000","last_undegraded":"2024-09-15T21:49:06.929473+0000","last_fullsized":"2024-09-15T21:49:06.929473+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'11","last_scrub_stamp":"2024-09-15T21:48:55.608066+0000","last_deep_scrub":"76'11","last_deep_scrub_stamp":"2024-09-15T21:48:55.608066+0000","last_clean_scrub_stamp":"2024-09-15T21:48:55.608066+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:19:03.916606+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":90,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"76'11","reported_seq":155,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.817370+0000","last_change":"2024-09-15T21:48:51.395232+0000","last_active":"2024-09-15T21:49:05.817370+0000","last_peered":"2024-09-15T21:49:05.817370+0000","last_clean":"2024-09-15T21:49:05.817370+0000","last_became_active":"2024-09-15T21:48:21.636913+0000","last_became_peered":"2024-09-15T21:48:21.636913+0000","last_unstale":"2024-09-15T21:49:05.817370+0000","last_undegraded":"2024-09-15T21:49:05.817370+0000","last_fullsized":"2024-09-15T21:49:05.817370+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'11","last_scrub_stamp":"2024-09-15T21:48:51.395136+0000","last_deep_scrub":"76'11","last_deep_scrub_stamp":"2024-09-15T21:48:51.395136+0000","last_clean_scrub_stamp":"2024-09-15T21:48:51.395136+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:12.439417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"76'31","reported_seq":164,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.816959+0000","last_change":"2024-09-15T21:48:49.504411+0000","last_active":"2024-09-15T21:49:05.816959+0000","last_peered":"2024-09-15T21:49:05.816959+0000","last_clean":"2024-09-15T21:49:05.816959+0000","last_became_active":"2024-09-15T21:46:22.425866+0000","last_became_peered":"2024-09-15T21:46:22.425866+0000","last_unstale":"2024-09-15T21:49:05.816959+0000","last_undegraded":"2024-09-15T21:49:05.816959+0000","last_fullsized":"2024-09-15T21:49:05.816959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'31","last_scrub_stamp":"2024-09-15T21:48:49.504271+0000","last_deep_scrub":"76'31","last_deep_scrub_stamp":"2024-09-15T21:48:49.504271+0000","last_clean_scrub_stamp":"2024-09-15T21:48:49.504271+0000","objects_scrubbed":6,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:17:14.994096+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35822,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":47,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"76'6","reported_seq":58,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.818917+0000","last_change":"2024-09-15T21:48:49.627091+0000","last_active":"2024-09-15T21:49:05.818917+0000","last_peered":"2024-09-15T21:49:05.818917+0000","last_clean":"2024-09-15T21:49:05.818917+0000","last_became_active":"2024-09-15T21:48:21.637151+0000","last_became_peered":"2024-09-15T21:48:21.637151+0000","last_unstale":"2024-09-15T21:49:05.818917+0000","last_undegraded":"2024-09-15T21:49:05.818917+0000","last_fullsized":"2024-09-15T21:49:05.818917+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:49.626943+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:49.626943+0000","last_clean_scrub_stamp":"2024-09-15T21:48:49.626943+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:51:32.641079+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"76'15","reported_seq":148,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.163197+0000","last_change":"2024-09-15T21:48:44.723646+0000","last_active":"2024-09-15T21:49:07.163197+0000","last_peered":"2024-09-15T21:49:07.163197+0000","last_clean":"2024-09-15T21:49:07.163197+0000","last_became_active":"2024-09-15T21:46:22.427087+0000","last_became_peered":"2024-09-15T21:46:22.427087+0000","last_unstale":"2024-09-15T21:49:07.163197+0000","last_undegraded":"2024-09-15T21:49:07.163197+0000","last_fullsized":"2024-09-15T21:49:07.163197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'15","last_scrub_stamp":"2024-09-15T21:48:44.723504+0000","last_deep_scrub":"76'15","last_deep_scrub_stamp":"2024-09-15T21:48:44.723504+0000","last_clean_scrub_stamp":"2024-09-15T21:48:44.723504+0000","objects_scrubbed":6,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:25:06.067810+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"76'9","reported_seq":150,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.162711+0000","last_change":"2024-09-15T21:48:54.733684+0000","last_active":"2024-09-15T21:49:07.162711+0000","last_peered":"2024-09-15T21:49:07.162711+0000","last_clean":"2024-09-15T21:49:07.162711+0000","last_became_active":"2024-09-15T21:48:21.635022+0000","last_became_peered":"2024-09-15T21:48:21.635022+0000","last_unstale":"2024-09-15T21:49:07.162711+0000","last_undegraded":"2024-09-15T21:49:07.162711+0000","last_fullsized":"2024-09-15T21:49:07.162711+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:54.733535+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:54.733535+0000","last_clean_scrub_stamp":"2024-09-15T21:48:54.733535+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:14.431313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"76'9","reported_seq":83,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.166909+0000","last_change":"2024-09-15T21:48:59.065431+0000","last_active":"2024-09-15T21:49:06.166909+0000","last_peered":"2024-09-15T21:49:06.166909+0000","last_clean":"2024-09-15T21:49:06.166909+0000","last_became_active":"2024-09-15T21:48:17.625232+0000","last_became_peered":"2024-09-15T21:48:17.625232+0000","last_unstale":"2024-09-15T21:49:06.166909+0000","last_undegraded":"2024-09-15T21:49:06.166909+0000","last_fullsized":"2024-09-15T21:49:06.166909+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:59.065300+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:59.065300+0000","last_clean_scrub_stamp":"2024-09-15T21:48:59.065300+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:28.096961+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"76'15","reported_seq":67,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.818729+0000","last_change":"2024-09-15T21:48:54.646308+0000","last_active":"2024-09-15T21:49:05.818729+0000","last_peered":"2024-09-15T21:49:05.818729+0000","last_clean":"2024-09-15T21:49:05.818729+0000","last_became_active":"2024-09-15T21:48:21.638780+0000","last_became_peered":"2024-09-15T21:48:21.638780+0000","last_unstale":"2024-09-15T21:49:05.818729+0000","last_undegraded":"2024-09-15T21:49:05.818729+0000","last_fullsized":"2024-09-15T21:49:05.818729+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'15","last_scrub_stamp":"2024-09-15T21:48:54.646156+0000","last_deep_scrub":"76'15","last_deep_scrub_stamp":"2024-09-15T21:48:54.646156+0000","last_clean_scrub_stamp":"2024-09-15T21:48:54.646156+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:32:55.068245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"76'8","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.816534+0000","last_change":"2024-09-15T21:48:47.468865+0000","last_active":"2024-09-15T21:49:05.816534+0000","last_peered":"2024-09-15T21:49:05.816534+0000","last_clean":"2024-09-15T21:49:05.816534+0000","last_became_active":"2024-09-15T21:46:22.425237+0000","last_became_peered":"2024-09-15T21:46:22.425237+0000","last_unstale":"2024-09-15T21:49:05.816534+0000","last_undegraded":"2024-09-15T21:49:05.816534+0000","last_fullsized":"2024-09-15T21:49:05.816534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:47.468727+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:47.468727+0000","last_clean_scrub_stamp":"2024-09-15T21:48:47.468727+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:01:19.617491+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"76'6","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.816536+0000","last_change":"2024-09-15T21:48:50.463078+0000","last_active":"2024-09-15T21:49:05.816536+0000","last_peered":"2024-09-15T21:49:05.816536+0000","last_clean":"2024-09-15T21:49:05.816536+0000","last_became_active":"2024-09-15T21:46:23.427116+0000","last_became_peered":"2024-09-15T21:46:23.427116+0000","last_unstale":"2024-09-15T21:49:05.816536+0000","last_undegraded":"2024-09-15T21:49:05.816536+0000","last_fullsized":"2024-09-15T21:49:05.816536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:50.462904+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:50.462904+0000","last_clean_scrub_stamp":"2024-09-15T21:48:50.462904+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:27:26.307690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"76'26","reported_seq":159,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.815941+0000","last_change":"2024-09-15T21:48:45.425316+0000","last_active":"2024-09-15T21:49:05.815941+0000","last_peered":"2024-09-15T21:49:05.815941+0000","last_clean":"2024-09-15T21:49:05.815941+0000","last_became_active":"2024-09-15T21:46:22.419620+0000","last_became_peered":"2024-09-15T21:46:22.419620+0000","last_unstale":"2024-09-15T21:49:05.815941+0000","last_undegraded":"2024-09-15T21:49:05.815941+0000","last_fullsized":"2024-09-15T21:49:05.815941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'26","last_scrub_stamp":"2024-09-15T21:48:45.425176+0000","last_deep_scrub":"76'26","last_deep_scrub_stamp":"2024-09-15T21:48:45.425176+0000","last_clean_scrub_stamp":"2024-09-15T21:48:45.425176+0000","objects_scrubbed":8,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:25.986850+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"76'6","reported_seq":58,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.818335+0000","last_change":"2024-09-15T21:48:51.620674+0000","last_active":"2024-09-15T21:49:05.818335+0000","last_peered":"2024-09-15T21:49:05.818335+0000","last_clean":"2024-09-15T21:49:05.818335+0000","last_became_active":"2024-09-15T21:48:21.637855+0000","last_became_peered":"2024-09-15T21:48:21.637855+0000","last_unstale":"2024-09-15T21:49:05.818335+0000","last_undegraded":"2024-09-15T21:49:05.818335+0000","last_fullsized":"2024-09-15T21:49:05.818335+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:51.620560+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:51.620560+0000","last_clean_scrub_stamp":"2024-09-15T21:48:51.620560+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:15:04.449106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"76'4","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.817307+0000","last_change":"2024-09-15T21:48:46.527028+0000","last_active":"2024-09-15T21:49:05.817307+0000","last_peered":"2024-09-15T21:49:05.817307+0000","last_clean":"2024-09-15T21:49:05.817307+0000","last_became_active":"2024-09-15T21:46:22.429250+0000","last_became_peered":"2024-09-15T21:46:22.429250+0000","last_unstale":"2024-09-15T21:49:05.817307+0000","last_undegraded":"2024-09-15T21:49:05.817307+0000","last_fullsized":"2024-09-15T21:49:05.817307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'4","last_scrub_stamp":"2024-09-15T21:48:46.526911+0000","last_deep_scrub":"76'4","last_deep_scrub_stamp":"2024-09-15T21:48:46.526911+0000","last_clean_scrub_stamp":"2024-09-15T21:48:46.526911+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:07:44.627253+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"76'5","reported_seq":57,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.818702+0000","last_change":"2024-09-15T21:48:48.597881+0000","last_active":"2024-09-15T21:49:05.818702+0000","last_peered":"2024-09-15T21:49:05.818702+0000","last_clean":"2024-09-15T21:49:05.818702+0000","last_became_active":"2024-09-15T21:48:21.635051+0000","last_became_peered":"2024-09-15T21:48:21.635051+0000","last_unstale":"2024-09-15T21:49:05.818702+0000","last_undegraded":"2024-09-15T21:49:05.818702+0000","last_fullsized":"2024-09-15T21:49:05.818702+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'5","last_scrub_stamp":"2024-09-15T21:48:48.597739+0000","last_deep_scrub":"76'5","last_deep_scrub_stamp":"2024-09-15T21:48:48.597739+0000","last_clean_scrub_stamp":"2024-09-15T21:48:48.597739+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:24:36.880475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"76'7","reported_seq":98,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.816251+0000","last_change":"2024-09-15T21:48:50.499112+0000","last_active":"2024-09-15T21:49:05.816251+0000","last_peered":"2024-09-15T21:49:05.816251+0000","last_clean":"2024-09-15T21:49:05.816251+0000","last_became_active":"2024-09-15T21:48:27.659750+0000","last_became_peered":"2024-09-15T21:48:27.659750+0000","last_unstale":"2024-09-15T21:49:05.816251+0000","last_undegraded":"2024-09-15T21:49:05.816251+0000","last_fullsized":"2024-09-15T21:49:05.816251+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:50.498981+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:50.498981+0000","last_clean_scrub_stamp":"2024-09-15T21:48:50.498981+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:51.266968+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"76'9","reported_seq":61,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.818986+0000","last_change":"2024-09-15T21:48:56.679410+0000","last_active":"2024-09-15T21:49:05.818986+0000","last_peered":"2024-09-15T21:49:05.818986+0000","last_clean":"2024-09-15T21:49:05.818986+0000","last_became_active":"2024-09-15T21:48:21.639328+0000","last_became_peered":"2024-09-15T21:48:21.639328+0000","last_unstale":"2024-09-15T21:49:05.818986+0000","last_undegraded":"2024-09-15T21:49:05.818986+0000","last_fullsized":"2024-09-15T21:49:05.818986+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:56.679287+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:56.679287+0000","last_clean_scrub_stamp":"2024-09-15T21:48:56.679287+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:47:11.980903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"76'4","reported_seq":138,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.929564+0000","last_change":"2024-09-15T21:48:45.563143+0000","last_active":"2024-09-15T21:49:06.929564+0000","last_peered":"2024-09-15T21:49:06.929564+0000","last_clean":"2024-09-15T21:49:06.929564+0000","last_became_active":"2024-09-15T21:46:22.424771+0000","last_became_peered":"2024-09-15T21:46:22.424771+0000","last_unstale":"2024-09-15T21:49:06.929564+0000","last_undegraded":"2024-09-15T21:49:06.929564+0000","last_fullsized":"2024-09-15T21:49:06.929564+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'4","last_scrub_stamp":"2024-09-15T21:48:45.563061+0000","last_deep_scrub":"76'4","last_deep_scrub_stamp":"2024-09-15T21:48:45.563061+0000","last_clean_scrub_stamp":"2024-09-15T21:48:45.563061+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:53:22.110654+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"76'6","reported_seq":76,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.167361+0000","last_change":"2024-09-15T21:48:48.087226+0000","last_active":"2024-09-15T21:49:06.167361+0000","last_peered":"2024-09-15T21:49:06.167361+0000","last_clean":"2024-09-15T21:49:06.167361+0000","last_became_active":"2024-09-15T21:48:17.617701+0000","last_became_peered":"2024-09-15T21:48:17.617701+0000","last_unstale":"2024-09-15T21:49:06.167361+0000","last_undegraded":"2024-09-15T21:49:06.167361+0000","last_fullsized":"2024-09-15T21:49:06.167361+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:48.087039+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:48.087039+0000","last_clean_scrub_stamp":"2024-09-15T21:48:48.087039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:54:45.723067+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"76'8","reported_seq":148,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.816139+0000","last_change":"2024-09-15T21:48:46.465791+0000","last_active":"2024-09-15T21:49:05.816139+0000","last_peered":"2024-09-15T21:49:05.816139+0000","last_clean":"2024-09-15T21:49:05.816139+0000","last_became_active":"2024-09-15T21:48:17.618782+0000","last_became_peered":"2024-09-15T21:48:17.618782+0000","last_unstale":"2024-09-15T21:49:05.816139+0000","last_undegraded":"2024-09-15T21:49:05.816139+0000","last_fullsized":"2024-09-15T21:49:05.816139+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:46.465703+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:46.465703+0000","last_clean_scrub_stamp":"2024-09-15T21:48:46.465703+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:48:10.554723+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"76'7","reported_seq":59,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.818617+0000","last_change":"2024-09-15T21:48:52.629639+0000","last_active":"2024-09-15T21:49:05.818617+0000","last_peered":"2024-09-15T21:49:05.818617+0000","last_clean":"2024-09-15T21:49:05.818617+0000","last_became_active":"2024-09-15T21:48:21.637883+0000","last_became_peered":"2024-09-15T21:48:21.637883+0000","last_unstale":"2024-09-15T21:49:05.818617+0000","last_undegraded":"2024-09-15T21:49:05.818617+0000","last_fullsized":"2024-09-15T21:49:05.818617+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:52.629527+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:52.629527+0000","last_clean_scrub_stamp":"2024-09-15T21:48:52.629527+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:15.928443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"76'119","reported_seq":253,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.818314+0000","last_change":"2024-09-15T21:49:03.491377+0000","last_active":"2024-09-15T21:49:05.818314+0000","last_peered":"2024-09-15T21:49:05.818314+0000","last_clean":"2024-09-15T21:49:05.818314+0000","last_became_active":"2024-09-15T21:48:24.646783+0000","last_became_peered":"2024-09-15T21:48:24.646783+0000","last_unstale":"2024-09-15T21:49:05.818314+0000","last_undegraded":"2024-09-15T21:49:05.818314+0000","last_fullsized":"2024-09-15T21:49:05.818314+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'119","last_scrub_stamp":"2024-09-15T21:49:03.491261+0000","last_deep_scrub":"76'119","last_deep_scrub_stamp":"2024-09-15T21:49:03.491261+0000","last_clean_scrub_stamp":"2024-09-15T21:49:03.491261+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:44.765894+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"76'11","reported_seq":86,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.166968+0000","last_change":"2024-09-15T21:48:50.083153+0000","last_active":"2024-09-15T21:49:06.166968+0000","last_peered":"2024-09-15T21:49:06.166968+0000","last_clean":"2024-09-15T21:49:06.166968+0000","last_became_active":"2024-09-15T21:48:17.622159+0000","last_became_peered":"2024-09-15T21:48:17.622159+0000","last_unstale":"2024-09-15T21:49:06.166968+0000","last_undegraded":"2024-09-15T21:49:06.166968+0000","last_fullsized":"2024-09-15T21:49:06.166968+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'11","last_scrub_stamp":"2024-09-15T21:48:50.082972+0000","last_deep_scrub":"76'11","last_deep_scrub_stamp":"2024-09-15T21:48:50.082972+0000","last_clean_scrub_stamp":"2024-09-15T21:48:50.082972+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:11:27.039171+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"76'8","reported_seq":148,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.163055+0000","last_change":"2024-09-15T21:48:56.358952+0000","last_active":"2024-09-15T21:49:07.163055+0000","last_peered":"2024-09-15T21:49:07.163055+0000","last_clean":"2024-09-15T21:49:07.163055+0000","last_became_active":"2024-09-15T21:48:17.620738+0000","last_became_peered":"2024-09-15T21:48:17.620738+0000","last_unstale":"2024-09-15T21:49:07.163055+0000","last_undegraded":"2024-09-15T21:49:07.163055+0000","last_fullsized":"2024-09-15T21:49:07.163055+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:56.358740+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:56.358740+0000","last_clean_scrub_stamp":"2024-09-15T21:48:56.358740+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:05:37.880183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"76'5","reported_seq":57,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.818338+0000","last_change":"2024-09-15T21:48:58.628179+0000","last_active":"2024-09-15T21:49:05.818338+0000","last_peered":"2024-09-15T21:49:05.818338+0000","last_clean":"2024-09-15T21:49:05.818338+0000","last_became_active":"2024-09-15T21:48:21.636886+0000","last_became_peered":"2024-09-15T21:48:21.636886+0000","last_unstale":"2024-09-15T21:49:05.818338+0000","last_undegraded":"2024-09-15T21:49:05.818338+0000","last_fullsized":"2024-09-15T21:49:05.818338+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'5","last_scrub_stamp":"2024-09-15T21:48:58.628031+0000","last_deep_scrub":"76'5","last_deep_scrub_stamp":"2024-09-15T21:48:58.628031+0000","last_clean_scrub_stamp":"2024-09-15T21:48:58.628031+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:40.772019+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"76'5","reported_seq":136,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.162523+0000","last_change":"2024-09-15T21:48:49.747331+0000","last_active":"2024-09-15T21:49:07.162523+0000","last_peered":"2024-09-15T21:49:07.162523+0000","last_clean":"2024-09-15T21:49:07.162523+0000","last_became_active":"2024-09-15T21:46:23.430138+0000","last_became_peered":"2024-09-15T21:46:23.430138+0000","last_unstale":"2024-09-15T21:49:07.162523+0000","last_undegraded":"2024-09-15T21:49:07.162523+0000","last_fullsized":"2024-09-15T21:49:07.162523+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'5","last_scrub_stamp":"2024-09-15T21:48:49.747222+0000","last_deep_scrub":"76'5","last_deep_scrub_stamp":"2024-09-15T21:48:49.747222+0000","last_clean_scrub_stamp":"2024-09-15T21:48:49.747222+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:13:27.577276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"76'11","reported_seq":97,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.815929+0000","last_change":"2024-09-15T21:48:51.514311+0000","last_active":"2024-09-15T21:49:05.815929+0000","last_peered":"2024-09-15T21:49:05.815929+0000","last_clean":"2024-09-15T21:49:05.815929+0000","last_became_active":"2024-09-15T21:48:27.659880+0000","last_became_peered":"2024-09-15T21:48:27.659880+0000","last_unstale":"2024-09-15T21:49:05.815929+0000","last_undegraded":"2024-09-15T21:49:05.815929+0000","last_fullsized":"2024-09-15T21:49:05.815929+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'11","last_scrub_stamp":"2024-09-15T21:48:51.514143+0000","last_deep_scrub":"76'11","last_deep_scrub_stamp":"2024-09-15T21:48:51.514143+0000","last_clean_scrub_stamp":"2024-09-15T21:48:51.514143+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:51.672977+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":516,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"76'8","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.929295+0000","last_change":"2024-09-15T21:48:52.614278+0000","last_active":"2024-09-15T21:49:06.929295+0000","last_peered":"2024-09-15T21:49:06.929295+0000","last_clean":"2024-09-15T21:49:06.929295+0000","last_became_active":"2024-09-15T21:46:23.426185+0000","last_became_peered":"2024-09-15T21:46:23.426185+0000","last_unstale":"2024-09-15T21:49:06.929295+0000","last_undegraded":"2024-09-15T21:49:06.929295+0000","last_fullsized":"2024-09-15T21:49:06.929295+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:52.614124+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:52.614124+0000","last_clean_scrub_stamp":"2024-09-15T21:48:52.614124+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:55:23.168353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"76'7","reported_seq":82,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.167189+0000","last_change":"2024-09-15T21:48:58.050362+0000","last_active":"2024-09-15T21:49:06.167189+0000","last_peered":"2024-09-15T21:49:06.167189+0000","last_clean":"2024-09-15T21:49:06.167189+0000","last_became_active":"2024-09-15T21:48:17.623936+0000","last_became_peered":"2024-09-15T21:48:17.623936+0000","last_unstale":"2024-09-15T21:49:06.167189+0000","last_undegraded":"2024-09-15T21:49:06.167189+0000","last_fullsized":"2024-09-15T21:49:06.167189+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:58.050232+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:58.050232+0000","last_clean_scrub_stamp":"2024-09-15T21:48:58.050232+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:26:26.762726+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"76'7","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.817059+0000","last_change":"2024-09-15T21:48:47.545064+0000","last_active":"2024-09-15T21:49:05.817059+0000","last_peered":"2024-09-15T21:49:05.817059+0000","last_clean":"2024-09-15T21:49:05.817059+0000","last_became_active":"2024-09-15T21:46:23.425416+0000","last_became_peered":"2024-09-15T21:46:23.425416+0000","last_unstale":"2024-09-15T21:49:05.817059+0000","last_undegraded":"2024-09-15T21:49:05.817059+0000","last_fullsized":"2024-09-15T21:49:05.817059+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:47.544941+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:47.544941+0000","last_clean_scrub_stamp":"2024-09-15T21:48:47.544941+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:34.400758+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"22'1","reported_seq":135,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.929398+0000","last_change":"2024-09-15T21:48:47.639673+0000","last_active":"2024-09-15T21:49:06.929398+0000","last_peered":"2024-09-15T21:49:06.929398+0000","last_clean":"2024-09-15T21:49:06.929398+0000","last_became_active":"2024-09-15T21:46:22.426210+0000","last_became_peered":"2024-09-15T21:46:22.426210+0000","last_unstale":"2024-09-15T21:49:06.929398+0000","last_undegraded":"2024-09-15T21:49:06.929398+0000","last_fullsized":"2024-09-15T21:49:06.929398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'1","last_scrub_stamp":"2024-09-15T21:48:47.639518+0000","last_deep_scrub":"22'1","last_deep_scrub_stamp":"2024-09-15T21:48:47.639518+0000","last_clean_scrub_stamp":"2024-09-15T21:48:47.639518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:32:02.263758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"76'5","reported_seq":75,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.167172+0000","last_change":"2024-09-15T21:48:56.092263+0000","last_active":"2024-09-15T21:49:06.167172+0000","last_peered":"2024-09-15T21:49:06.167172+0000","last_clean":"2024-09-15T21:49:06.167172+0000","last_became_active":"2024-09-15T21:48:17.624852+0000","last_became_peered":"2024-09-15T21:48:17.624852+0000","last_unstale":"2024-09-15T21:49:06.167172+0000","last_undegraded":"2024-09-15T21:49:06.167172+0000","last_fullsized":"2024-09-15T21:49:06.167172+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'5","last_scrub_stamp":"2024-09-15T21:48:56.092106+0000","last_deep_scrub":"76'5","last_deep_scrub_stamp":"2024-09-15T21:48:56.092106+0000","last_clean_scrub_stamp":"2024-09-15T21:48:56.092106+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:44:19.841946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"76'7","reported_seq":63,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.818771+0000","last_change":"2024-09-15T21:49:00.540154+0000","last_active":"2024-09-15T21:49:05.818771+0000","last_peered":"2024-09-15T21:49:05.818771+0000","last_clean":"2024-09-15T21:49:05.818771+0000","last_became_active":"2024-09-15T21:48:21.639593+0000","last_became_peered":"2024-09-15T21:48:21.639593+0000","last_unstale":"2024-09-15T21:49:05.818771+0000","last_undegraded":"2024-09-15T21:49:05.818771+0000","last_fullsized":"2024-09-15T21:49:05.818771+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:49:00.540032+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:49:00.540032+0000","last_clean_scrub_stamp":"2024-09-15T21:49:00.540032+0000","objects_scrubbed":3,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:33.348343+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":224,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"76'9","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.929348+0000","last_change":"2024-09-15T21:48:51.635145+0000","last_active":"2024-09-15T21:49:06.929348+0000","last_peered":"2024-09-15T21:49:06.929348+0000","last_clean":"2024-09-15T21:49:06.929348+0000","last_became_active":"2024-09-15T21:46:23.426495+0000","last_became_peered":"2024-09-15T21:46:23.426495+0000","last_unstale":"2024-09-15T21:49:06.929348+0000","last_undegraded":"2024-09-15T21:49:06.929348+0000","last_fullsized":"2024-09-15T21:49:06.929348+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:51.635033+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:51.635033+0000","last_clean_scrub_stamp":"2024-09-15T21:48:51.635033+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:28:46.677592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"76'20","reported_seq":154,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.929608+0000","last_change":"2024-09-15T21:48:48.645351+0000","last_active":"2024-09-15T21:49:06.929608+0000","last_peered":"2024-09-15T21:49:06.929608+0000","last_clean":"2024-09-15T21:49:06.929608+0000","last_became_active":"2024-09-15T21:46:22.426855+0000","last_became_peered":"2024-09-15T21:46:22.426855+0000","last_unstale":"2024-09-15T21:49:06.929608+0000","last_undegraded":"2024-09-15T21:49:06.929608+0000","last_fullsized":"2024-09-15T21:49:06.929608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'20","last_scrub_stamp":"2024-09-15T21:48:48.645223+0000","last_deep_scrub":"76'20","last_deep_scrub_stamp":"2024-09-15T21:48:48.645223+0000","last_clean_scrub_stamp":"2024-09-15T21:48:48.645223+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:25:49.419359+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"76'5","reported_seq":148,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.162625+0000","last_change":"2024-09-15T21:48:57.343614+0000","last_active":"2024-09-15T21:49:07.162625+0000","last_peered":"2024-09-15T21:49:07.162625+0000","last_clean":"2024-09-15T21:49:07.162625+0000","last_became_active":"2024-09-15T21:48:21.639541+0000","last_became_peered":"2024-09-15T21:48:21.639541+0000","last_unstale":"2024-09-15T21:49:07.162625+0000","last_undegraded":"2024-09-15T21:49:07.162625+0000","last_fullsized":"2024-09-15T21:49:07.162625+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'5","last_scrub_stamp":"2024-09-15T21:48:57.343459+0000","last_deep_scrub":"76'5","last_deep_scrub_stamp":"2024-09-15T21:48:57.343459+0000","last_clean_scrub_stamp":"2024-09-15T21:48:57.343459+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:07:37.229275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"76'12","reported_seq":175,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.817553+0000","last_change":"2024-09-15T21:48:58.351594+0000","last_active":"2024-09-15T21:49:05.817553+0000","last_peered":"2024-09-15T21:49:05.817553+0000","last_clean":"2024-09-15T21:49:05.817553+0000","last_became_active":"2024-09-15T21:48:21.643251+0000","last_became_peered":"2024-09-15T21:48:21.643251+0000","last_unstale":"2024-09-15T21:49:05.817553+0000","last_undegraded":"2024-09-15T21:49:05.817553+0000","last_fullsized":"2024-09-15T21:49:05.817553+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'12","last_scrub_stamp":"2024-09-15T21:48:58.351467+0000","last_deep_scrub":"76'12","last_deep_scrub_stamp":"2024-09-15T21:48:58.351467+0000","last_clean_scrub_stamp":"2024-09-15T21:48:58.351467+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:09:41.587053+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":44,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"76'7","reported_seq":138,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.162674+0000","last_change":"2024-09-15T21:48:46.687711+0000","last_active":"2024-09-15T21:49:07.162674+0000","last_peered":"2024-09-15T21:49:07.162674+0000","last_clean":"2024-09-15T21:49:07.162674+0000","last_became_active":"2024-09-15T21:46:23.424007+0000","last_became_peered":"2024-09-15T21:46:23.424007+0000","last_unstale":"2024-09-15T21:49:07.162674+0000","last_undegraded":"2024-09-15T21:49:07.162674+0000","last_fullsized":"2024-09-15T21:49:07.162674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:46.687610+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:46.687610+0000","last_clean_scrub_stamp":"2024-09-15T21:48:46.687610+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:56:57.560558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"76'3","reported_seq":150,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.162753+0000","last_change":"2024-09-15T21:48:52.781137+0000","last_active":"2024-09-15T21:49:07.162753+0000","last_peered":"2024-09-15T21:49:07.162753+0000","last_clean":"2024-09-15T21:49:07.162753+0000","last_became_active":"2024-09-15T21:48:17.623453+0000","last_became_peered":"2024-09-15T21:48:17.623453+0000","last_unstale":"2024-09-15T21:49:07.162753+0000","last_undegraded":"2024-09-15T21:49:07.162753+0000","last_fullsized":"2024-09-15T21:49:07.162753+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'3","last_scrub_stamp":"2024-09-15T21:48:52.781027+0000","last_deep_scrub":"76'3","last_deep_scrub_stamp":"2024-09-15T21:48:52.781027+0000","last_clean_scrub_stamp":"2024-09-15T21:48:52.781027+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:14:28.346534+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"76'2","reported_seq":136,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.817102+0000","last_change":"2024-09-15T21:48:49.465962+0000","last_active":"2024-09-15T21:49:05.817102+0000","last_peered":"2024-09-15T21:49:05.817102+0000","last_clean":"2024-09-15T21:49:05.817102+0000","last_became_active":"2024-09-15T21:46:23.429283+0000","last_became_peered":"2024-09-15T21:46:23.429283+0000","last_unstale":"2024-09-15T21:49:05.817102+0000","last_undegraded":"2024-09-15T21:49:05.817102+0000","last_fullsized":"2024-09-15T21:49:05.817102+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'2","last_scrub_stamp":"2024-09-15T21:48:49.465824+0000","last_deep_scrub":"76'2","last_deep_scrub_stamp":"2024-09-15T21:48:49.465824+0000","last_clean_scrub_stamp":"2024-09-15T21:48:49.465824+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:28.475614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"76'11","reported_seq":175,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.929409+0000","last_change":"2024-09-15T21:48:56.592779+0000","last_active":"2024-09-15T21:49:06.929409+0000","last_peered":"2024-09-15T21:49:06.929409+0000","last_clean":"2024-09-15T21:49:06.929409+0000","last_became_active":"2024-09-15T21:48:21.635898+0000","last_became_peered":"2024-09-15T21:48:21.635898+0000","last_unstale":"2024-09-15T21:49:06.929409+0000","last_undegraded":"2024-09-15T21:49:06.929409+0000","last_fullsized":"2024-09-15T21:49:06.929409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'11","last_scrub_stamp":"2024-09-15T21:48:56.592585+0000","last_deep_scrub":"76'11","last_deep_scrub_stamp":"2024-09-15T21:48:56.592585+0000","last_clean_scrub_stamp":"2024-09-15T21:48:56.592585+0000","objects_scrubbed":4,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:41:39.659727+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":180,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"76'9","reported_seq":142,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.162381+0000","last_change":"2024-09-15T21:48:51.405503+0000","last_active":"2024-09-15T21:49:07.162381+0000","last_peered":"2024-09-15T21:49:07.162381+0000","last_clean":"2024-09-15T21:49:07.162381+0000","last_became_active":"2024-09-15T21:46:23.428301+0000","last_became_peered":"2024-09-15T21:46:23.428301+0000","last_unstale":"2024-09-15T21:49:07.162381+0000","last_undegraded":"2024-09-15T21:49:07.162381+0000","last_fullsized":"2024-09-15T21:49:07.162381+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:51.405367+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:51.405367+0000","last_clean_scrub_stamp":"2024-09-15T21:48:51.405367+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:51:53.179487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"76'6","reported_seq":140,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.929392+0000","last_change":"2024-09-15T21:48:46.605361+0000","last_active":"2024-09-15T21:49:06.929392+0000","last_peered":"2024-09-15T21:49:06.929392+0000","last_clean":"2024-09-15T21:49:06.929392+0000","last_became_active":"2024-09-15T21:46:22.425478+0000","last_became_peered":"2024-09-15T21:46:22.425478+0000","last_unstale":"2024-09-15T21:49:06.929392+0000","last_undegraded":"2024-09-15T21:49:06.929392+0000","last_fullsized":"2024-09-15T21:49:06.929392+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:46.605195+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:46.605195+0000","last_clean_scrub_stamp":"2024-09-15T21:48:46.605195+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:57:11.180764+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"76'8","reported_seq":141,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.162739+0000","last_change":"2024-09-15T21:48:48.371421+0000","last_active":"2024-09-15T21:49:07.162739+0000","last_peered":"2024-09-15T21:49:07.162739+0000","last_clean":"2024-09-15T21:49:07.162739+0000","last_became_active":"2024-09-15T21:46:23.425547+0000","last_became_peered":"2024-09-15T21:46:23.425547+0000","last_unstale":"2024-09-15T21:49:07.162739+0000","last_undegraded":"2024-09-15T21:49:07.162739+0000","last_fullsized":"2024-09-15T21:49:07.162739+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:48:48.371307+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:48:48.371307+0000","last_clean_scrub_stamp":"2024-09-15T21:48:48.371307+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:22:44.003854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"76'7","reported_seq":60,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.818723+0000","last_change":"2024-09-15T21:48:59.581511+0000","last_active":"2024-09-15T21:49:05.818723+0000","last_peered":"2024-09-15T21:49:05.818723+0000","last_clean":"2024-09-15T21:49:05.818723+0000","last_became_active":"2024-09-15T21:48:21.639223+0000","last_became_peered":"2024-09-15T21:48:21.639223+0000","last_unstale":"2024-09-15T21:49:05.818723+0000","last_undegraded":"2024-09-15T21:49:05.818723+0000","last_fullsized":"2024-09-15T21:49:05.818723+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'7","last_scrub_stamp":"2024-09-15T21:48:59.581383+0000","last_deep_scrub":"76'7","last_deep_scrub_stamp":"2024-09-15T21:48:59.581383+0000","last_clean_scrub_stamp":"2024-09-15T21:48:59.581383+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:40:19.311173+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"76'5","reported_seq":136,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.816546+0000","last_change":"2024-09-15T21:48:53.455212+0000","last_active":"2024-09-15T21:49:05.816546+0000","last_peered":"2024-09-15T21:49:05.816546+0000","last_clean":"2024-09-15T21:49:05.816546+0000","last_became_active":"2024-09-15T21:46:23.434448+0000","last_became_peered":"2024-09-15T21:46:23.434448+0000","last_unstale":"2024-09-15T21:49:05.816546+0000","last_undegraded":"2024-09-15T21:49:05.816546+0000","last_fullsized":"2024-09-15T21:49:05.816546+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'5","last_scrub_stamp":"2024-09-15T21:48:53.455056+0000","last_deep_scrub":"76'5","last_deep_scrub_stamp":"2024-09-15T21:48:53.455056+0000","last_clean_scrub_stamp":"2024-09-15T21:48:53.455056+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:22.628886+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"76'15","reported_seq":96,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.815987+0000","last_change":"2024-09-15T21:48:48.464909+0000","last_active":"2024-09-15T21:49:05.815987+0000","last_peered":"2024-09-15T21:49:05.815987+0000","last_clean":"2024-09-15T21:49:05.815987+0000","last_became_active":"2024-09-15T21:48:27.656753+0000","last_became_peered":"2024-09-15T21:48:27.656753+0000","last_unstale":"2024-09-15T21:49:05.815987+0000","last_undegraded":"2024-09-15T21:49:05.815987+0000","last_fullsized":"2024-09-15T21:49:05.815987+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'15","last_scrub_stamp":"2024-09-15T21:48:48.464792+0000","last_deep_scrub":"76'15","last_deep_scrub_stamp":"2024-09-15T21:48:48.464792+0000","last_clean_scrub_stamp":"2024-09-15T21:48:48.464792+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:42.908766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"76'8","reported_seq":63,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.818546+0000","last_change":"2024-09-15T21:49:02.496550+0000","last_active":"2024-09-15T21:49:05.818546+0000","last_peered":"2024-09-15T21:49:05.818546+0000","last_clean":"2024-09-15T21:49:05.818546+0000","last_became_active":"2024-09-15T21:48:21.641517+0000","last_became_peered":"2024-09-15T21:48:21.641517+0000","last_unstale":"2024-09-15T21:49:05.818546+0000","last_undegraded":"2024-09-15T21:49:05.818546+0000","last_fullsized":"2024-09-15T21:49:05.818546+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'8","last_scrub_stamp":"2024-09-15T21:49:02.496360+0000","last_deep_scrub":"76'8","last_deep_scrub_stamp":"2024-09-15T21:49:02.496360+0000","last_clean_scrub_stamp":"2024-09-15T21:49:02.496360+0000","objects_scrubbed":3,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:00:47.427912+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1164,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"76'9","reported_seq":79,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:06.167108+0000","last_change":"2024-09-15T21:48:54.102481+0000","last_active":"2024-09-15T21:49:06.167108+0000","last_peered":"2024-09-15T21:49:06.167108+0000","last_clean":"2024-09-15T21:49:06.167108+0000","last_became_active":"2024-09-15T21:48:17.622019+0000","last_became_peered":"2024-09-15T21:48:17.622019+0000","last_unstale":"2024-09-15T21:49:06.167108+0000","last_undegraded":"2024-09-15T21:49:06.167108+0000","last_fullsized":"2024-09-15T21:49:06.167108+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'9","last_scrub_stamp":"2024-09-15T21:48:54.102314+0000","last_deep_scrub":"76'9","last_deep_scrub_stamp":"2024-09-15T21:48:54.102314+0000","last_clean_scrub_stamp":"2024-09-15T21:48:54.102314+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:10:45.955871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"76'20","reported_seq":75,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:05.818791+0000","last_change":"2024-09-15T21:49:01.535114+0000","last_active":"2024-09-15T21:49:05.818791+0000","last_peered":"2024-09-15T21:49:05.818791+0000","last_clean":"2024-09-15T21:49:05.818791+0000","last_became_active":"2024-09-15T21:48:21.640156+0000","last_became_peered":"2024-09-15T21:48:21.640156+0000","last_unstale":"2024-09-15T21:49:05.818791+0000","last_undegraded":"2024-09-15T21:49:05.818791+0000","last_fullsized":"2024-09-15T21:49:05.818791+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'20","last_scrub_stamp":"2024-09-15T21:49:01.534951+0000","last_deep_scrub":"76'20","last_deep_scrub_stamp":"2024-09-15T21:49:01.534951+0000","last_clean_scrub_stamp":"2024-09-15T21:49:01.534951+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:12:53.603363+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"76'6","reported_seq":144,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.163106+0000","last_change":"2024-09-15T21:48:51.729417+0000","last_active":"2024-09-15T21:49:07.163106+0000","last_peered":"2024-09-15T21:49:07.163106+0000","last_clean":"2024-09-15T21:49:07.163106+0000","last_became_active":"2024-09-15T21:48:17.622843+0000","last_became_peered":"2024-09-15T21:48:17.622843+0000","last_unstale":"2024-09-15T21:49:07.163106+0000","last_undegraded":"2024-09-15T21:49:07.163106+0000","last_fullsized":"2024-09-15T21:49:07.163106+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'6","last_scrub_stamp":"2024-09-15T21:48:51.729300+0000","last_deep_scrub":"76'6","last_deep_scrub_stamp":"2024-09-15T21:48:51.729300+0000","last_clean_scrub_stamp":"2024-09-15T21:48:51.729300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:45:01.496891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'2","reported_seq":137,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.162889+0000","last_change":"2024-09-15T21:48:47.382828+0000","last_active":"2024-09-15T21:49:07.162889+0000","last_peered":"2024-09-15T21:49:07.162889+0000","last_clean":"2024-09-15T21:49:07.162889+0000","last_became_active":"2024-09-15T21:46:22.428263+0000","last_became_peered":"2024-09-15T21:46:22.428263+0000","last_unstale":"2024-09-15T21:49:07.162889+0000","last_undegraded":"2024-09-15T21:49:07.162889+0000","last_fullsized":"2024-09-15T21:49:07.162889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'2","last_scrub_stamp":"2024-09-15T21:48:47.382727+0000","last_deep_scrub":"22'2","last_deep_scrub_stamp":"2024-09-15T21:48:47.382727+0000","last_clean_scrub_stamp":"2024-09-15T21:48:47.382727+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:24:27.068168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"76'5","reported_seq":138,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.162734+0000","last_change":"2024-09-15T21:48:50.410459+0000","last_active":"2024-09-15T21:49:07.162734+0000","last_peered":"2024-09-15T21:49:07.162734+0000","last_clean":"2024-09-15T21:49:07.162734+0000","last_became_active":"2024-09-15T21:46:23.427546+0000","last_became_peered":"2024-09-15T21:46:23.427546+0000","last_unstale":"2024-09-15T21:49:07.162734+0000","last_undegraded":"2024-09-15T21:49:07.162734+0000","last_fullsized":"2024-09-15T21:49:07.162734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'5","last_scrub_stamp":"2024-09-15T21:48:50.410294+0000","last_deep_scrub":"76'5","last_deep_scrub_stamp":"2024-09-15T21:48:50.410294+0000","last_clean_scrub_stamp":"2024-09-15T21:48:50.410294+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:49:02.410504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"76'19","reported_seq":190,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:49:07.163216+0000","last_change":"2024-09-15T21:48:56.752292+0000","last_active":"2024-09-15T21:49:07.163216+0000","last_peered":"2024-09-15T21:49:07.163216+0000","last_clean":"2024-09-15T21:49:07.163216+0000","last_became_active":"2024-09-15T21:48:21.642406+0000","last_became_peered":"2024-09-15T21:48:21.642406+0000","last_unstale":"2024-09-15T21:49:07.163216+0000","last_undegraded":"2024-09-15T21:49:07.163216+0000","last_fullsized":"2024-09-15T21:49:07.163216+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"76'19","last_scrub_stamp":"2024-09-15T21:48:56.752163+0000","last_deep_scrub":"76'19","last_deep_scrub_stamp":"2024-09-15T21:48:56.752163+0000","last_clean_scrub_stamp":"2024-09-15T21:48:56.752163+0000","objects_scrubbed":6,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:32:45.460289+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":1344,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":503,"ondisk_log_size":503,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43045,"num_objects":99,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":252,"num_write_kb":121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":58,"num_bytes_recovered":4802,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":25000,"num_omap_keys":50,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":335872,"data_stored":86090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6368,"internal_metadata":0},"log_size":315,"ondisk_log_size":315,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":16,"seq":68719476778,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32688,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1531904,"data_stored":1014476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509482,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1540096,"data_stored":1016099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509484,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":970752,"data_stored":459846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":57,"seq":244813135888,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29116,"kb_used_data":936,"kb_used_omap":7,"kb_used_meta":28152,"kb_avail":94342724,"statfs":{"total":96636764160,"available":96606949376,"internally_reserved":0,"allocated":958464,"data_stored":427166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7963,"internal_metadata":28827877},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509483,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":933888,"data_stored":424564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509482,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":946176,"data_stored":425100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542187,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27944,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343896,"statfs":{"total":96636764160,"available":96608149504,"internally_reserved":0,"allocated":937984,"data_stored":424766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574891,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32144,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339696,"statfs":{"total":96636764160,"available":96603848704,"internally_reserved":0,"allocated":978944,"data_stored":460681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":37447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6368,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":36612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:49:28.492 DEBUG:teuthology.orchestra.run.smithi100:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-15T21:49:28.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.624+0000 7ff131dcf640 1 Processor -- start 2024-09-15T21:49:28.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.624+0000 7ff131dcf640 1 -- start start 2024-09-15T21:49:28.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.625+0000 7ff131dcf640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff12c0cf110 0x7ff12c15be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:28.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.625+0000 7ff131dcf640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff12c144250 0x7ff12c1446c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:28.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.625+0000 7ff131dcf640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff12c15eb40 con 0x7ff12c0ced30 2024-09-15T21:49:28.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.625+0000 7ff131dcf640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7ff12c15c340 con 0x7ff12c144250 2024-09-15T21:49:28.627 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.625+0000 7ff131dcf640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff12c15e8f0 con 0x7ff12c0cf110 2024-09-15T21:49:28.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.625+0000 7ff12affd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff12c0cf110 0x7ff12c15be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:28.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.625+0000 7ff12bfff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff12c144250 0x7ff12c1446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:28.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.625+0000 7ff12bfff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff12c144250 0x7ff12c1446c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.100:33456/0 (socket says 172.21.15.100:33456) 2024-09-15T21:49:28.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.625+0000 7ff12bfff640 1 -- 172.21.15.100:0/1087405269 learned_addr learned my addr 172.21.15.100:0/1087405269 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:49:28.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.625+0000 7ff12bfff640 1 -- 172.21.15.100:0/1087405269 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff12c0cf110 msgr2=0x7ff12c15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:49:28.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.625+0000 7ff12bfff640 1 --2- 172.21.15.100:0/1087405269 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff12c0cf110 0x7ff12c15be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.625+0000 7ff12bfff640 1 -- 172.21.15.100:0/1087405269 >> v1:172.21.15.122:6789/0 conn(0x7ff12c0ced30 legacy=0x7ff12c0739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:28.628 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.625+0000 7ff12bfff640 1 -- 172.21.15.100:0/1087405269 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c15ee20 con 0x7ff12c144250 2024-09-15T21:49:28.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.626+0000 7ff12bfff640 1 --2- 172.21.15.100:0/1087405269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff12c144250 0x7ff12c1446c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff124009e50 tx=0x7ff12402a7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6aa80c4852de1a75 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:28.629 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.626+0000 7ff12a7fc640 1 -- 172.21.15.100:0/1087405269 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff124036070 con 0x7ff12c144250 2024-09-15T21:49:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.626+0000 7ff12a7fc640 1 -- 172.21.15.100:0/1087405269 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff124031600 con 0x7ff12c144250 2024-09-15T21:49:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.626+0000 7ff12a7fc640 1 -- 172.21.15.100:0/1087405269 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1240319d0 con 0x7ff12c144250 2024-09-15T21:49:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.627+0000 7ff131dcf640 1 -- 172.21.15.100:0/1087405269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff12c144250 msgr2=0x7ff12c1446c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.627+0000 7ff131dcf640 1 --2- 172.21.15.100:0/1087405269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff12c144250 0x7ff12c1446c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff124009e50 tx=0x7ff12402a7a0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.627+0000 7ff131dcf640 1 -- 172.21.15.100:0/1087405269 shutdown_connections 2024-09-15T21:49:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.627+0000 7ff131dcf640 1 --2- 172.21.15.100:0/1087405269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff12c144250 0x7ff12c1446c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.627+0000 7ff131dcf640 1 --2- 172.21.15.100:0/1087405269 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff12c0cf110 0x7ff12c15be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.627+0000 7ff131dcf640 1 -- 172.21.15.100:0/1087405269 >> 172.21.15.100:0/1087405269 conn(0x7ff12c06f990 msgr2=0x7ff12c070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.627+0000 7ff131dcf640 1 -- 172.21.15.100:0/1087405269 shutdown_connections 2024-09-15T21:49:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.627+0000 7ff131dcf640 1 -- 172.21.15.100:0/1087405269 wait complete. 2024-09-15T21:49:28.630 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.628+0000 7ff131dcf640 1 Processor -- start 2024-09-15T21:49:28.632 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.628+0000 7ff131dcf640 1 -- start start 2024-09-15T21:49:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.629+0000 7ff131dcf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff12c0ced30 0x7ff12c15dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.629+0000 7ff131dcf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff12c0cf110 0x7ff12c158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.629+0000 7ff131dcf640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff12c144250 0x7ff12c142e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.629+0000 7ff131dcf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c0740c0 con 0x7ff12c0ced30 2024-09-15T21:49:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.629+0000 7ff131dcf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c15f750 con 0x7ff12c0cf110 2024-09-15T21:49:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.629+0000 7ff131dcf640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7ff12c144c00 con 0x7ff12c144250 2024-09-15T21:49:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.629+0000 7ff12b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff12c0ced30 0x7ff12c15dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.629+0000 7ff12b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff12c0ced30 0x7ff12c15dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46546/0 (socket says 172.21.15.100:46546) 2024-09-15T21:49:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.629+0000 7ff12b7fe640 1 -- 172.21.15.100:0/541438889 learned_addr learned my addr 172.21.15.100:0/541438889 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:49:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.629+0000 7ff12affd640 1 --2- 172.21.15.100:0/541438889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff12c0cf110 0x7ff12c158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.629+0000 7ff12bfff640 1 --2- 172.21.15.100:0/541438889 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff12c144250 0x7ff12c142e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:28.633 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.629+0000 7ff12b7fe640 1 -- 172.21.15.100:0/541438889 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff12c144250 msgr2=0x7ff12c142e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.629+0000 7ff12b7fe640 1 --2- 172.21.15.100:0/541438889 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff12c144250 0x7ff12c142e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.629+0000 7ff12b7fe640 1 -- 172.21.15.100:0/541438889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff12c0cf110 msgr2=0x7ff12c158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.629+0000 7ff12b7fe640 1 --2- 172.21.15.100:0/541438889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff12c0cf110 0x7ff12c158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.629+0000 7ff12b7fe640 1 -- 172.21.15.100:0/541438889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c0cb980 con 0x7ff12c0ced30 2024-09-15T21:49:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.629+0000 7ff12b7fe640 1 --2- 172.21.15.100:0/541438889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff12c0ced30 0x7ff12c15dd00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00c460 tx=0x7ff11c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.629+0000 7ff113fff640 1 -- 172.21.15.100:0/541438889 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c0052d0 con 0x7ff12c0ced30 2024-09-15T21:49:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.630+0000 7ff113fff640 1 -- 172.21.15.100:0/541438889 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff11c012630 con 0x7ff12c0ced30 2024-09-15T21:49:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.630+0000 7ff113fff640 1 -- 172.21.15.100:0/541438889 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c0054b0 con 0x7ff12c0ced30 2024-09-15T21:49:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.630+0000 7ff131dcf640 1 -- 172.21.15.100:0/541438889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12c1f4f90 con 0x7ff12c0ced30 2024-09-15T21:49:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.630+0000 7ff131dcf640 1 -- 172.21.15.100:0/541438889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12c0ca2d0 con 0x7ff12c0ced30 2024-09-15T21:49:28.634 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.631+0000 7ff131dcf640 1 -- 172.21.15.100:0/541438889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff12c15be00 con 0x7ff12c0ced30 2024-09-15T21:49:28.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.631+0000 7ff113fff640 1 -- 172.21.15.100:0/541438889 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7ff11c012a20 con 0x7ff12c0ced30 2024-09-15T21:49:28.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.632+0000 7ff113fff640 1 --2- 172.21.15.100:0/541438889 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff11806c2e0 0x7ff11806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:28.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.632+0000 7ff113fff640 1 -- 172.21.15.100:0/541438889 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5886+0+0 (secure 0 0 0) 0x7ff11c08aea0 con 0x7ff12c0ced30 2024-09-15T21:49:28.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.632+0000 7ff12affd640 1 --2- 172.21.15.100:0/541438889 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff11806c2e0 0x7ff11806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:28.635 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.632+0000 7ff12affd640 1 --2- 172.21.15.100:0/541438889 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff11806c2e0 0x7ff11806e7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff12c06e8a0 tx=0x7ff120009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:28.638 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.636+0000 7ff113fff640 1 -- 172.21.15.100:0/541438889 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7ff11c058fa0 con 0x7ff12c0ced30 2024-09-15T21:49:28.804 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.802+0000 7ff131dcf640 1 -- 172.21.15.100:0/541438889 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7ff12c0c9700 con 0x7ff12c0ced30 2024-09-15T21:49:28.806 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.805+0000 7ff113fff640 1 -- 172.21.15.100:0/541438889 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7ff11c05d2a0 con 0x7ff12c0ced30 2024-09-15T21:49:28.807 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.806+0000 7ff113fff640 1 -- 172.21.15.100:0/541438889 <== mon.0 v2:172.21.15.100:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ff11c061ad0 con 0x7ff12c0ced30 2024-09-15T21:49:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.808+0000 7ff131dcf640 1 -- 172.21.15.100:0/541438889 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff11806c2e0 msgr2=0x7ff11806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:28.809 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.808+0000 7ff131dcf640 1 --2- 172.21.15.100:0/541438889 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff11806c2e0 0x7ff11806e7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff12c06e8a0 tx=0x7ff120009190 comp rx=0 tx=0).stop 2024-09-15T21:49:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.808+0000 7ff131dcf640 1 -- 172.21.15.100:0/541438889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff12c0ced30 msgr2=0x7ff12c15dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.808+0000 7ff131dcf640 1 --2- 172.21.15.100:0/541438889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff12c0ced30 0x7ff12c15dd00 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00c460 tx=0x7ff11c00c920 comp rx=0 tx=0).stop 2024-09-15T21:49:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.808+0000 7ff131dcf640 1 -- 172.21.15.100:0/541438889 shutdown_connections 2024-09-15T21:49:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.808+0000 7ff131dcf640 1 --2- 172.21.15.100:0/541438889 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7ff11806c2e0 0x7ff11806e7a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.808+0000 7ff131dcf640 1 --2- 172.21.15.100:0/541438889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff12c0cf110 0x7ff12c158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.808+0000 7ff131dcf640 1 --2- 172.21.15.100:0/541438889 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7ff12c144250 0x7ff12c142e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.808+0000 7ff131dcf640 1 --2- 172.21.15.100:0/541438889 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff12c0ced30 0x7ff12c15dd00 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.810 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.808+0000 7ff131dcf640 1 -- 172.21.15.100:0/541438889 >> 172.21.15.100:0/541438889 conn(0x7ff12c06f990 msgr2=0x7ff12c073140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:28.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.808+0000 7ff131dcf640 1 -- 172.21.15.100:0/541438889 shutdown_connections 2024-09-15T21:49:28.811 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.808+0000 7ff131dcf640 1 -- 172.21.15.100:0/541438889 wait complete. 2024-09-15T21:49:28.821 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-15T21:49:28.821 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T21:49:28.921 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.918+0000 7f238fded640 1 Processor -- start 2024-09-15T21:49:28.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.918+0000 7f238fded640 1 -- start start 2024-09-15T21:49:28.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.919+0000 7f238fded640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23880cd2f0 0x7f2388166aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:28.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.919+0000 7f238fded640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23880cd780 con 0x7f238806c2f0 2024-09-15T21:49:28.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.919+0000 7f238fded640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2388071ed0 con 0x7f23880ccf10 2024-09-15T21:49:28.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.919+0000 7f238fded640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f238815f650 con 0x7f23880cd2f0 2024-09-15T21:49:28.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.919+0000 7f238d388640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f238806c2f0 0x7f238815b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:55868/0 (socket says 172.21.15.100:55868) 2024-09-15T21:49:28.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.919+0000 7f238d388640 1 -- 172.21.15.100:0/689145084 learned_addr learned my addr 172.21.15.100:0/689145084 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:49:28.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.919+0000 7f238e38a640 1 --2- 172.21.15.100:0/689145084 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23880cd2f0 0x7f2388166aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:28.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.919+0000 7f238cb87640 1 -- 172.21.15.100:0/689145084 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 828778362 0 0) 0x7f23880cd780 con 0x7f238806c2f0 2024-09-15T21:49:28.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.919+0000 7f238cb87640 1 -- 172.21.15.100:0/689145084 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2374003520 con 0x7f238806c2f0 2024-09-15T21:49:28.922 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.919+0000 7f238cb87640 1 -- 172.21.15.100:0/689145084 <== mon.1 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3239349506 0 0) 0x7f2388071ed0 con 0x7f23880ccf10 2024-09-15T21:49:28.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.919+0000 7f238cb87640 1 -- 172.21.15.100:0/689145084 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23880cd780 con 0x7f23880ccf10 2024-09-15T21:49:28.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.919+0000 7f238e38a640 1 -- 172.21.15.100:0/689145084 >> v1:172.21.15.122:6789/0 conn(0x7f23880ccf10 legacy=0x7f2388071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:28.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.919+0000 7f238e38a640 1 -- 172.21.15.100:0/689145084 >> v1:172.21.15.100:6789/0 conn(0x7f238806c2f0 legacy=0x7f238815b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:28.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.919+0000 7f238cb87640 1 -- 172.21.15.100:0/689145084 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 422436265 0 0) 0x7f2374003520 con 0x7f238806c2f0 2024-09-15T21:49:28.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.919+0000 7f238e38a640 1 -- 172.21.15.100:0/689145084 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238815e890 con 0x7f23880cd2f0 2024-09-15T21:49:28.923 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.920+0000 7f238e38a640 1 --2- 172.21.15.100:0/689145084 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23880cd2f0 0x7f2388166aa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2380009800 tx=0x7f238002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a960831d59105e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:28.924 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.923+0000 7f238cb87640 1 -- 172.21.15.100:0/689145084 <== mon.2 v2:172.21.15.122:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2380037070 con 0x7f23880cd2f0 2024-09-15T21:49:28.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.923+0000 7f238cb87640 1 -- 172.21.15.100:0/689145084 <== mon.2 v2:172.21.15.122:3301/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f238002ac50 con 0x7f23880cd2f0 2024-09-15T21:49:28.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.923+0000 7f238cb87640 1 -- 172.21.15.100:0/689145084 <== mon.2 v2:172.21.15.122:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378001670 con 0x7f23880cd2f0 2024-09-15T21:49:28.925 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.923+0000 7f238fded640 1 -- 172.21.15.100:0/689145084 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23880cd2f0 msgr2=0x7f2388166aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:28.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.923+0000 7f238fded640 1 --2- 172.21.15.100:0/689145084 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23880cd2f0 0x7f2388166aa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2380009800 tx=0x7f238002a1c0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.923+0000 7f238fded640 1 -- 172.21.15.100:0/689145084 shutdown_connections 2024-09-15T21:49:28.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.924+0000 7f238fded640 1 --2- 172.21.15.100:0/689145084 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23880cd2f0 0x7f2388166aa0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.924+0000 7f238fded640 1 -- 172.21.15.100:0/689145084 >> 172.21.15.100:0/689145084 conn(0x7f238806d890 msgr2=0x7f238806bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:28.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.924+0000 7f238fded640 1 -- 172.21.15.100:0/689145084 shutdown_connections 2024-09-15T21:49:28.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.924+0000 7f238fded640 1 -- 172.21.15.100:0/689145084 wait complete. 2024-09-15T21:49:28.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.925+0000 7f238fded640 1 Processor -- start 2024-09-15T21:49:28.926 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.925+0000 7f238fded640 1 -- start start 2024-09-15T21:49:28.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.925+0000 7f238fded640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f238806c2f0 0x7f23880c1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:28.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.925+0000 7f238fded640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23880ccf10 0x7f23881576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:28.928 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.925+0000 7f238fded640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23880cd2f0 0x7f23881f1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:28.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.925+0000 7f238fded640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f238815ee50 con 0x7f23880ccf10 2024-09-15T21:49:28.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.925+0000 7f238fded640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f23880721b0 con 0x7f238806c2f0 2024-09-15T21:49:28.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.925+0000 7f238fded640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f238815f130 con 0x7f23880cd2f0 2024-09-15T21:49:28.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.926+0000 7f238d388640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23880ccf10 0x7f23881576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:28.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.926+0000 7f238d388640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23880ccf10 0x7f23881576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46548/0 (socket says 172.21.15.100:46548) 2024-09-15T21:49:28.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.926+0000 7f238d388640 1 -- 172.21.15.100:0/933071639 learned_addr learned my addr 172.21.15.100:0/933071639 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:49:28.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.926+0000 7f238e38a640 1 --2- 172.21.15.100:0/933071639 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23880cd2f0 0x7f23881f1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:28.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.926+0000 7f238db89640 1 --2- 172.21.15.100:0/933071639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f238806c2f0 0x7f23880c1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:28.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.926+0000 7f238d388640 1 -- 172.21.15.100:0/933071639 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23880cd2f0 msgr2=0x7f23881f1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:28.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.926+0000 7f238d388640 1 --2- 172.21.15.100:0/933071639 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23880cd2f0 0x7f23881f1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.926+0000 7f238d388640 1 -- 172.21.15.100:0/933071639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f238806c2f0 msgr2=0x7f23880c1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:28.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.926+0000 7f238d388640 1 --2- 172.21.15.100:0/933071639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f238806c2f0 0x7f23880c1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:28.929 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.926+0000 7f238d388640 1 -- 172.21.15.100:0/933071639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23880c3880 con 0x7f23880ccf10 2024-09-15T21:49:28.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.926+0000 7f238d388640 1 --2- 172.21.15.100:0/933071639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23880ccf10 0x7f23881576b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f237800c4d0 tx=0x7f237800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:28.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.926+0000 7f237e7fc640 1 -- 172.21.15.100:0/933071639 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378017070 con 0x7f23880ccf10 2024-09-15T21:49:28.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.926+0000 7f237e7fc640 1 -- 172.21.15.100:0/933071639 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2378005ce0 con 0x7f23880ccf10 2024-09-15T21:49:28.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.926+0000 7f238fded640 1 -- 172.21.15.100:0/933071639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23881f0c60 con 0x7f23880ccf10 2024-09-15T21:49:28.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.926+0000 7f237e7fc640 1 -- 172.21.15.100:0/933071639 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378007330 con 0x7f23880ccf10 2024-09-15T21:49:28.930 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.926+0000 7f238fded640 1 -- 172.21.15.100:0/933071639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23880c3e00 con 0x7f23880ccf10 2024-09-15T21:49:28.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.927+0000 7f238fded640 1 -- 172.21.15.100:0/933071639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23880727a0 con 0x7f23880ccf10 2024-09-15T21:49:28.931 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.928+0000 7f237e7fc640 1 -- 172.21.15.100:0/933071639 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f237801a070 con 0x7f23880ccf10 2024-09-15T21:49:28.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.928+0000 7f237e7fc640 1 --2- 172.21.15.100:0/933071639 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f236c06c2b0 0x7f236c06e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:28.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.928+0000 7f238db89640 1 --2- 172.21.15.100:0/933071639 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f236c06c2b0 0x7f236c06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:28.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.929+0000 7f237e7fc640 1 -- 172.21.15.100:0/933071639 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5886+0+0 (secure 0 0 0) 0x7f237808a960 con 0x7f23880ccf10 2024-09-15T21:49:28.932 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.929+0000 7f238db89640 1 --2- 172.21.15.100:0/933071639 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f236c06c2b0 0x7f236c06e770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2384001560 tx=0x7f2384009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:28.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:28.933+0000 7f237e7fc640 1 -- 172.21.15.100:0/933071639 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f23880727a0 con 0x7f23880ccf10 2024-09-15T21:49:29.132 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.130+0000 7f238fded640 1 -- 172.21.15.100:0/933071639 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23880b2400 con 0x7f23880ccf10 2024-09-15T21:49:29.133 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.130+0000 7f237e7fc640 1 -- 172.21.15.100:0/933071639 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6958 (secure 0 0 0) 0x7f2378058940 con 0x7f23880ccf10 2024-09-15T21:49:29.133 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:49:29.133 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":17,"btime":"2024-09-15T21:46:52:569246+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2992633397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2992633397},{"type":"v1","addr":"172.21.15.122:6833","nonce":2992633397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T21:46:22.753977+0000","modified":"2024-09-15T21:46:52.569239+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4291,"mds_2":4290,"mds_3":4288,"mds_4":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.100:6835/1809406468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1809406468},{"type":"v1","addr":"172.21.15.100:6835","nonce":1809406468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4288":{"gid":4288,"name":"f","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.122:6837/371886422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":371886422},{"type":"v1","addr":"172.21.15.122:6837","nonce":371886422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.100:6838/276075809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":276075809},{"type":"v1","addr":"172.21.15.100:6838","nonce":276075809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4291":{"gid":4291,"name":"b","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6836/2947743626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6835","nonce":2947743626},{"type":"v1","addr":"172.21.15.122:6836","nonce":2947743626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/602064326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6837","nonce":602064326},{"type":"v1","addr":"172.21.15.100:6839","nonce":602064326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4287,4288,4290,4291,4293]},"id":1}]} 2024-09-15T21:49:29.133 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 17 2024-09-15T21:49:29.134 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.132+0000 7f238fded640 1 -- 172.21.15.100:0/933071639 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f236c06c2b0 msgr2=0x7f236c06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.132+0000 7f238fded640 1 --2- 172.21.15.100:0/933071639 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f236c06c2b0 0x7f236c06e770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2384001560 tx=0x7f2384009040 comp rx=0 tx=0).stop 2024-09-15T21:49:29.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.132+0000 7f238fded640 1 -- 172.21.15.100:0/933071639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23880ccf10 msgr2=0x7f23881576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.132+0000 7f238fded640 1 --2- 172.21.15.100:0/933071639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23880ccf10 0x7f23881576b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f237800c4d0 tx=0x7f237800c8d0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.133+0000 7f238fded640 1 -- 172.21.15.100:0/933071639 shutdown_connections 2024-09-15T21:49:29.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.133+0000 7f238fded640 1 --2- 172.21.15.100:0/933071639 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f236c06c2b0 0x7f236c06e770 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.133+0000 7f238fded640 1 --2- 172.21.15.100:0/933071639 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f23880cd2f0 0x7f23881f1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.133+0000 7f238fded640 1 --2- 172.21.15.100:0/933071639 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f23880ccf10 0x7f23881576b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.133+0000 7f238fded640 1 --2- 172.21.15.100:0/933071639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f238806c2f0 0x7f23880c1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.133+0000 7f238fded640 1 -- 172.21.15.100:0/933071639 >> 172.21.15.100:0/933071639 conn(0x7f238806d890 msgr2=0x7f238815deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:29.135 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.133+0000 7f238fded640 1 -- 172.21.15.100:0/933071639 shutdown_connections 2024-09-15T21:49:29.136 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.133+0000 7f238fded640 1 -- 172.21.15.100:0/933071639 wait complete. 2024-09-15T21:49:29.146 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-15T21:49:29.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.243+0000 7f764a665640 1 Processor -- start 2024-09-15T21:49:29.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.243+0000 7f764a665640 1 -- start start 2024-09-15T21:49:29.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.244+0000 7f764a665640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f76440c9b40 0x7f7644164830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:29.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.244+0000 7f764a665640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f764415c530 con 0x7f7644152fe0 2024-09-15T21:49:29.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.244+0000 7f764a665640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f764415d3d0 con 0x7f76440c9710 2024-09-15T21:49:29.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.244+0000 7f764a665640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f7644159ec0 con 0x7f76440c9b40 2024-09-15T21:49:29.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.244+0000 7f7643fff640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f7644152fe0 0x7f764415ccc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:55884/0 (socket says 172.21.15.100:55884) 2024-09-15T21:49:29.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.244+0000 7f7643fff640 1 -- 172.21.15.100:0/2175089863 learned_addr learned my addr 172.21.15.100:0/2175089863 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:49:29.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.244+0000 7f7648c02640 1 --2- 172.21.15.100:0/2175089863 >> v2:172.21.15.100:3300/0 conn(0x7f76440c9b40 0x7f7644164830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:29.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.244+0000 7f7642ffd640 1 -- 172.21.15.100:0/2175089863 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2551688871 0 0) 0x7f764415c530 con 0x7f7644152fe0 2024-09-15T21:49:29.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.244+0000 7f7642ffd640 1 -- 172.21.15.100:0/2175089863 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f762c003520 con 0x7f7644152fe0 2024-09-15T21:49:29.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.244+0000 7f7648c02640 1 -- 172.21.15.100:0/2175089863 >> v1:172.21.15.122:6789/0 conn(0x7f76440c9710 legacy=0x7f76440730f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.244+0000 7f7648c02640 1 -- 172.21.15.100:0/2175089863 >> v1:172.21.15.100:6789/0 conn(0x7f7644152fe0 legacy=0x7f764415ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.247 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.244+0000 7f7648c02640 1 -- 172.21.15.100:0/2175089863 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76440c3f30 con 0x7f76440c9b40 2024-09-15T21:49:29.248 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.245+0000 7f7648c02640 1 --2- 172.21.15.100:0/2175089863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76440c9b40 0x7f7644164830 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f763c009d90 tx=0x7f763c02a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b28440ff47804ca9 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:29.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.245+0000 7f7642ffd640 1 -- 172.21.15.100:0/2175089863 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c037070 con 0x7f76440c9b40 2024-09-15T21:49:29.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.245+0000 7f7642ffd640 1 -- 172.21.15.100:0/2175089863 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f763c007740 con 0x7f76440c9b40 2024-09-15T21:49:29.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.245+0000 7f7642ffd640 1 -- 172.21.15.100:0/2175089863 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f763c007af0 con 0x7f76440c9b40 2024-09-15T21:49:29.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.246+0000 7f764a665640 1 -- 172.21.15.100:0/2175089863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76440c9b40 msgr2=0x7f7644164830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.246+0000 7f764a665640 1 --2- 172.21.15.100:0/2175089863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76440c9b40 0x7f7644164830 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f763c009d90 tx=0x7f763c02a780 comp rx=0 tx=0).stop 2024-09-15T21:49:29.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.246+0000 7f764a665640 1 -- 172.21.15.100:0/2175089863 shutdown_connections 2024-09-15T21:49:29.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.246+0000 7f764a665640 1 --2- 172.21.15.100:0/2175089863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76440c9b40 0x7f7644164830 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.246+0000 7f764a665640 1 -- 172.21.15.100:0/2175089863 >> 172.21.15.100:0/2175089863 conn(0x7f7644067dd0 msgr2=0x7f764406bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:29.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.246+0000 7f764a665640 1 -- 172.21.15.100:0/2175089863 shutdown_connections 2024-09-15T21:49:29.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.246+0000 7f764a665640 1 -- 172.21.15.100:0/2175089863 wait complete. 2024-09-15T21:49:29.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.247+0000 7f764a665640 1 Processor -- start 2024-09-15T21:49:29.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.247+0000 7f764a665640 1 -- start start 2024-09-15T21:49:29.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.248+0000 7f764a665640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f76440c9710 0x7f7644155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:29.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.248+0000 7f764a665640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76440c9b40 0x7f76441f2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:29.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.248+0000 7f764a665640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7644152fe0 0x7f7644154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:29.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.248+0000 7f764a665640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f76440c44f0 con 0x7f76440c9b40 2024-09-15T21:49:29.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.248+0000 7f764a665640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f764415c830 con 0x7f7644152fe0 2024-09-15T21:49:29.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.248+0000 7f764a665640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f7644158ea0 con 0x7f76440c9710 2024-09-15T21:49:29.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.248+0000 7f76437fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76440c9b40 0x7f76441f2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:29.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.248+0000 7f76437fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76440c9b40 0x7f76441f2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46562/0 (socket says 172.21.15.100:46562) 2024-09-15T21:49:29.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.248+0000 7f76437fe640 1 -- 172.21.15.100:0/790680184 learned_addr learned my addr 172.21.15.100:0/790680184 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:49:29.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.248+0000 7f76437fe640 1 -- 172.21.15.100:0/790680184 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f76440c9710 msgr2=0x7f7644155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.248+0000 7f76437fe640 1 --2- 172.21.15.100:0/790680184 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f76440c9710 0x7f7644155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.248+0000 7f76437fe640 1 -- 172.21.15.100:0/790680184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7644152fe0 msgr2=0x7f7644154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.248+0000 7f7648c02640 1 --2- 172.21.15.100:0/790680184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7644152fe0 0x7f7644154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:29.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.248+0000 7f76437fe640 1 --2- 172.21.15.100:0/790680184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7644152fe0 0x7f7644154200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.248+0000 7f76437fe640 1 -- 172.21.15.100:0/790680184 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7644153760 con 0x7f76440c9b40 2024-09-15T21:49:29.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.249+0000 7f76437fe640 1 --2- 172.21.15.100:0/790680184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76440c9b40 0x7f76441f2720 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f763800c3c0 tx=0x7f763800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:29.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.249+0000 7f7640ff9640 1 -- 172.21.15.100:0/790680184 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76380052d0 con 0x7f76440c9b40 2024-09-15T21:49:29.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.249+0000 7f7640ff9640 1 -- 172.21.15.100:0/790680184 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f7638005490 con 0x7f76440c9b40 2024-09-15T21:49:29.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.249+0000 7f764a665640 1 -- 172.21.15.100:0/790680184 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7644153a00 con 0x7f76440c9b40 2024-09-15T21:49:29.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.249+0000 7f7640ff9640 1 -- 172.21.15.100:0/790680184 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7638007280 con 0x7f76440c9b40 2024-09-15T21:49:29.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.249+0000 7f764a665640 1 -- 172.21.15.100:0/790680184 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76440cb7a0 con 0x7f76440c9b40 2024-09-15T21:49:29.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.250+0000 7f764a665640 1 -- 172.21.15.100:0/790680184 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76440c4860 con 0x7f76440c9b40 2024-09-15T21:49:29.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.250+0000 7f7640ff9640 1 -- 172.21.15.100:0/790680184 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f7638007420 con 0x7f76440c9b40 2024-09-15T21:49:29.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.251+0000 7f7640ff9640 1 --2- 172.21.15.100:0/790680184 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f763006c2b0 0x7f763006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:29.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.251+0000 7f7640ff9640 1 -- 172.21.15.100:0/790680184 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5886+0+0 (secure 0 0 0) 0x7f763808a9a0 con 0x7f76440c9b40 2024-09-15T21:49:29.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.251+0000 7f7643fff640 1 --2- 172.21.15.100:0/790680184 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f763006c2b0 0x7f763006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:29.254 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.251+0000 7f7643fff640 1 --2- 172.21.15.100:0/790680184 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f763006c2b0 0x7f763006e770 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7634001610 tx=0x7f7634007930 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:29.257 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.255+0000 7f7640ff9640 1 -- 172.21.15.100:0/790680184 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f7638059060 con 0x7f76440c9b40 2024-09-15T21:49:29.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.452+0000 7f764a665640 1 -- 172.21.15.100:0/790680184 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7644159340 con 0x7f76440c9b40 2024-09-15T21:49:29.455 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.453+0000 7f7640ff9640 1 -- 172.21.15.100:0/790680184 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+6958 (secure 0 0 0) 0x7f7638058960 con 0x7f76440c9b40 2024-09-15T21:49:29.456 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:49:29.456 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":17,"btime":"2024-09-15T21:46:52:569246+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4285,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/2992633397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":2992633397},{"type":"v1","addr":"172.21.15.122:6833","nonce":2992633397}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-15T21:46:22.753977+0000","modified":"2024-09-15T21:46:52.569239+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":4293,"mds_1":4291,"mds_2":4290,"mds_3":4288,"mds_4":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":4,"incarnation":16,"state":"up:active","state_seq":10,"addr":"172.21.15.100:6835/1809406468","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":1809406468},{"type":"v1","addr":"172.21.15.100:6835","nonce":1809406468}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4288":{"gid":4288,"name":"f","rank":3,"incarnation":14,"state":"up:active","state_seq":9,"addr":"172.21.15.122:6837/371886422","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":371886422},{"type":"v1","addr":"172.21.15.122:6837","nonce":371886422}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4290":{"gid":4290,"name":"a","rank":2,"incarnation":12,"state":"up:active","state_seq":7,"addr":"172.21.15.100:6838/276075809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":276075809},{"type":"v1","addr":"172.21.15.100:6838","nonce":276075809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4291":{"gid":4291,"name":"b","rank":1,"incarnation":10,"state":"up:active","state_seq":6,"addr":"172.21.15.122:6836/2947743626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6835","nonce":2947743626},{"type":"v1","addr":"172.21.15.122:6836","nonce":2947743626}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4293":{"gid":4293,"name":"e","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/602064326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6837","nonce":602064326},{"type":"v1","addr":"172.21.15.100:6839","nonce":602064326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4293,"qdb_cluster":[4287,4288,4290,4291,4293]},"id":1}]} 2024-09-15T21:49:29.456 INFO:teuthology.orchestra.run.smithi100.stderr:dumped fsmap epoch 17 2024-09-15T21:49:29.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.455+0000 7f764a665640 1 -- 172.21.15.100:0/790680184 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f763006c2b0 msgr2=0x7f763006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.455+0000 7f764a665640 1 --2- 172.21.15.100:0/790680184 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f763006c2b0 0x7f763006e770 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7634001610 tx=0x7f7634007930 comp rx=0 tx=0).stop 2024-09-15T21:49:29.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.455+0000 7f764a665640 1 -- 172.21.15.100:0/790680184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76440c9b40 msgr2=0x7f76441f2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.455+0000 7f764a665640 1 --2- 172.21.15.100:0/790680184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76440c9b40 0x7f76441f2720 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f763800c3c0 tx=0x7f763800c880 comp rx=0 tx=0).stop 2024-09-15T21:49:29.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.455+0000 7f764a665640 1 -- 172.21.15.100:0/790680184 shutdown_connections 2024-09-15T21:49:29.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.455+0000 7f764a665640 1 --2- 172.21.15.100:0/790680184 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f763006c2b0 0x7f763006e770 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.457 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.455+0000 7f764a665640 1 --2- 172.21.15.100:0/790680184 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7644152fe0 0x7f7644154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.455+0000 7f764a665640 1 --2- 172.21.15.100:0/790680184 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f76440c9b40 0x7f76441f2720 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.455+0000 7f764a665640 1 --2- 172.21.15.100:0/790680184 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f76440c9710 0x7f7644155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.455+0000 7f764a665640 1 -- 172.21.15.100:0/790680184 >> 172.21.15.100:0/790680184 conn(0x7f7644067dd0 msgr2=0x7f7644156fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:29.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.455+0000 7f764a665640 1 -- 172.21.15.100:0/790680184 shutdown_connections 2024-09-15T21:49:29.458 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.455+0000 7f764a665640 1 -- 172.21.15.100:0/790680184 wait complete. 2024-09-15T21:49:29.469 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:49:29.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.566+0000 7f6dac986640 1 Processor -- start 2024-09-15T21:49:29.568 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.566+0000 7f6dac986640 1 -- start start 2024-09-15T21:49:29.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.567+0000 7f6dac986640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6da8155440 0x7f6da80cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:29.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.567+0000 7f6dac986640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6da80cb7d0 0x7f6da80cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:29.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.567+0000 7f6dac986640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6da815e7a0 con 0x7f6da806d0e0 2024-09-15T21:49:29.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.567+0000 7f6dac986640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f6da815b470 con 0x7f6da80cb7d0 2024-09-15T21:49:29.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.567+0000 7f6dac986640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f6da8155820 con 0x7f6da8155440 2024-09-15T21:49:29.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.567+0000 7f6da5d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6da80cb7d0 0x7f6da80cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:29.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.567+0000 7f6da5d9b640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6da80cb7d0 0x7f6da80cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46572/0 (socket says 172.21.15.100:46572) 2024-09-15T21:49:29.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.567+0000 7f6da5d9b640 1 -- 172.21.15.100:0/2244274303 learned_addr learned my addr 172.21.15.100:0/2244274303 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:49:29.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.567+0000 7f6da659c640 1 --2- 172.21.15.100:0/2244274303 >> v2:172.21.15.122:3300/0 conn(0x7f6da8155440 0x7f6da80cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:29.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.567+0000 7f6da5d9b640 1 -- 172.21.15.100:0/2244274303 >> v2:172.21.15.122:3300/0 conn(0x7f6da8155440 msgr2=0x7f6da80cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:49:29.570 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.567+0000 7f6da5d9b640 1 --2- 172.21.15.100:0/2244274303 >> v2:172.21.15.122:3300/0 conn(0x7f6da8155440 0x7f6da80cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.567+0000 7f6da5d9b640 1 -- 172.21.15.100:0/2244274303 >> v1:172.21.15.100:6789/0 conn(0x7f6da806d0e0 legacy=0x7f6da80cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.567+0000 7f6da5d9b640 1 -- 172.21.15.100:0/2244274303 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da815ea40 con 0x7f6da80cb7d0 2024-09-15T21:49:29.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.568+0000 7f6da5d9b640 1 --2- 172.21.15.100:0/2244274303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da80cb7d0 0x7f6da80cd960 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6da80ce040 tx=0x7f6d9002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0c5b330e3f2cd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:29.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.568+0000 7f6da559a640 1 -- 172.21.15.100:0/2244274303 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90037070 con 0x7f6da80cb7d0 2024-09-15T21:49:29.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.568+0000 7f6da559a640 1 -- 172.21.15.100:0/2244274303 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6d90032440 con 0x7f6da80cb7d0 2024-09-15T21:49:29.571 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.568+0000 7f6da559a640 1 -- 172.21.15.100:0/2244274303 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90032830 con 0x7f6da80cb7d0 2024-09-15T21:49:29.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.569+0000 7f6dac986640 1 -- 172.21.15.100:0/2244274303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da80cb7d0 msgr2=0x7f6da80cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.569+0000 7f6dac986640 1 --2- 172.21.15.100:0/2244274303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da80cb7d0 0x7f6da80cd960 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6da80ce040 tx=0x7f6d9002a1e0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.569+0000 7f6dac986640 1 -- 172.21.15.100:0/2244274303 shutdown_connections 2024-09-15T21:49:29.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.569+0000 7f6dac986640 1 --2- 172.21.15.100:0/2244274303 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da80cb7d0 0x7f6da80cd960 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.569+0000 7f6dac986640 1 --2- 172.21.15.100:0/2244274303 >> v2:172.21.15.122:3300/0 conn(0x7f6da8155440 0x7f6da80cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.572 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.569+0000 7f6dac986640 1 -- 172.21.15.100:0/2244274303 >> 172.21.15.100:0/2244274303 conn(0x7f6da80680d0 msgr2=0x7f6da806d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:29.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.569+0000 7f6dac986640 1 -- 172.21.15.100:0/2244274303 shutdown_connections 2024-09-15T21:49:29.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.569+0000 7f6dac986640 1 -- 172.21.15.100:0/2244274303 wait complete. 2024-09-15T21:49:29.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.570+0000 7f6dac986640 1 Processor -- start 2024-09-15T21:49:29.573 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.570+0000 7f6dac986640 1 -- start start 2024-09-15T21:49:29.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.571+0000 7f6dac986640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6da806d0e0 0x7f6da80b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:29.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.571+0000 7f6dac986640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da80cb7d0 0x7f6da815c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:29.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.571+0000 7f6dac986640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6da8155440 0x7f6da815ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:29.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.571+0000 7f6dac986640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6da815f2e0 con 0x7f6da80cb7d0 2024-09-15T21:49:29.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.571+0000 7f6dac986640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6da815f000 con 0x7f6da806d0e0 2024-09-15T21:49:29.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.571+0000 7f6dac986640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6da815b070 con 0x7f6da8155440 2024-09-15T21:49:29.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.571+0000 7f6da5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da80cb7d0 0x7f6da815c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:29.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.571+0000 7f6da5d9b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da80cb7d0 0x7f6da815c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46578/0 (socket says 172.21.15.100:46578) 2024-09-15T21:49:29.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.571+0000 7f6da5d9b640 1 -- 172.21.15.100:0/4257172992 learned_addr learned my addr 172.21.15.100:0/4257172992 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:49:29.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.571+0000 7f6da6d9d640 1 --2- 172.21.15.100:0/4257172992 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6da8155440 0x7f6da815ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:29.575 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.571+0000 7f6da5d9b640 1 -- 172.21.15.100:0/4257172992 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6da8155440 msgr2=0x7f6da815ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.571+0000 7f6da659c640 1 --2- 172.21.15.100:0/4257172992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6da806d0e0 0x7f6da80b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:29.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.571+0000 7f6da5d9b640 1 --2- 172.21.15.100:0/4257172992 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6da8155440 0x7f6da815ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.571+0000 7f6da5d9b640 1 -- 172.21.15.100:0/4257172992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6da806d0e0 msgr2=0x7f6da80b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.571+0000 7f6da5d9b640 1 --2- 172.21.15.100:0/4257172992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6da806d0e0 0x7f6da80b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.571+0000 7f6da5d9b640 1 -- 172.21.15.100:0/4257172992 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da81f5500 con 0x7f6da80cb7d0 2024-09-15T21:49:29.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.571+0000 7f6da5d9b640 1 --2- 172.21.15.100:0/4257172992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da80cb7d0 0x7f6da815c920 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6d90000c00 tx=0x7f6d900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:29.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.572+0000 7f6d9effd640 1 -- 172.21.15.100:0/4257172992 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90037040 con 0x7f6da80cb7d0 2024-09-15T21:49:29.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.572+0000 7f6d9effd640 1 -- 172.21.15.100:0/4257172992 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6d90042070 con 0x7f6da80cb7d0 2024-09-15T21:49:29.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.572+0000 7f6d9effd640 1 -- 172.21.15.100:0/4257172992 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d90030bf0 con 0x7f6da80cb7d0 2024-09-15T21:49:29.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.572+0000 7f6dac986640 1 -- 172.21.15.100:0/4257172992 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da81f1070 con 0x7f6da80cb7d0 2024-09-15T21:49:29.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.572+0000 7f6dac986640 1 -- 172.21.15.100:0/4257172992 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da81f4760 con 0x7f6da80cb7d0 2024-09-15T21:49:29.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.573+0000 7f6dac986640 1 -- 172.21.15.100:0/4257172992 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6da80b3940 con 0x7f6da80cb7d0 2024-09-15T21:49:29.576 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.573+0000 7f6d9effd640 1 -- 172.21.15.100:0/4257172992 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6d90047460 con 0x7f6da80cb7d0 2024-09-15T21:49:29.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.573+0000 7f6d9effd640 1 --2- 172.21.15.100:0/4257172992 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6d8806c2e0 0x7f6d8806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:29.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.574+0000 7f6d9effd640 1 -- 172.21.15.100:0/4257172992 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5886+0+0 (secure 0 0 0) 0x7f6d900ac130 con 0x7f6da80cb7d0 2024-09-15T21:49:29.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.574+0000 7f6da659c640 1 --2- 172.21.15.100:0/4257172992 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6d8806c2e0 0x7f6d8806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:29.578 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.574+0000 7f6da659c640 1 --2- 172.21.15.100:0/4257172992 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6d8806c2e0 0x7f6d8806e7a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009f30 tx=0x7f6da000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:29.579 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.578+0000 7f6d9effd640 1 -- 172.21.15.100:0/4257172992 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6da81f1070 con 0x7f6da80cb7d0 2024-09-15T21:49:29.751 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.749+0000 7f6dac986640 1 -- 172.21.15.100:0/4257172992 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6da80caed0 con 0x7f6da80cb7d0 2024-09-15T21:49:29.752 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.750+0000 7f6d9effd640 1 -- 172.21.15.100:0/4257172992 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+17006 (secure 0 0 0) 0x7f6d90032e20 con 0x7f6da80cb7d0 2024-09-15T21:49:29.752 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-09-15T21:49:29.753 INFO:teuthology.orchestra.run.smithi100.stdout:{"epoch":78,"fsid":"30e2fff4-2171-4531-8806-0e03b63f494f","created":"2024-09-15T21:45:59.746897+0000","modified":"2024-09-15T21:49:05.809103+0000","last_up_change":"2024-09-15T21:48:16.603329+0000","last_in_change":"2024-09-15T21:48:20.610986+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:46:11.837603+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-15T21:46:20.577982+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-15T21:46:21.729231+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5887900b-522d-476f-9704-cc2c5ee87a60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":62,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6819","nonce":3920435546}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6821","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6822","nonce":3920435546}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6829","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6831","nonce":3920435546}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6825","nonce":3920435546},{"type":"v1","addr":"172.21.15.100:6827","nonce":3920435546}]},"public_addr":"172.21.15.100:6819/3920435546","cluster_addr":"172.21.15.100:6822/3920435546","heartbeat_back_addr":"172.21.15.100:6831/3920435546","heartbeat_front_addr":"172.21.15.100:6827/3920435546","state":["exists","up"]},{"osd":1,"uuid":"adc17993-5c50-4251-85e1-4c7ac62e25e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6801","nonce":3802679972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6803","nonce":3802679972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6807","nonce":3802679972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":3802679972},{"type":"v1","addr":"172.21.15.100:6805","nonce":3802679972}]},"public_addr":"172.21.15.100:6801/3802679972","cluster_addr":"172.21.15.100:6803/3802679972","heartbeat_back_addr":"172.21.15.100:6807/3802679972","heartbeat_front_addr":"172.21.15.100:6805/3802679972","state":["exists","up"]},{"osd":2,"uuid":"8de3b7d5-1980-4a3b-a934-264f867c7c8d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6823","nonce":2316787974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6826","nonce":2316787974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6833","nonce":2316787974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":2316787974},{"type":"v1","addr":"172.21.15.100:6830","nonce":2316787974}]},"public_addr":"172.21.15.100:6823/2316787974","cluster_addr":"172.21.15.100:6826/2316787974","heartbeat_back_addr":"172.21.15.100:6833/2316787974","heartbeat_front_addr":"172.21.15.100:6830/2316787974","state":["exists","up"]},{"osd":3,"uuid":"93dbcd1c-6a6f-49cf-bc94-e64b883de5f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6811","nonce":1287562741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6813","nonce":1287562741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6817","nonce":1287562741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":1287562741},{"type":"v1","addr":"172.21.15.100:6815","nonce":1287562741}]},"public_addr":"172.21.15.100:6811/1287562741","cluster_addr":"172.21.15.100:6813/1287562741","heartbeat_back_addr":"172.21.15.100:6817/1287562741","heartbeat_front_addr":"172.21.15.100:6815/1287562741","state":["exists","up"]},{"osd":4,"uuid":"7caee2a0-bab4-45a9-8109-724948c863c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6801","nonce":4045445512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6803","nonce":4045445512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6807","nonce":4045445512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":4045445512},{"type":"v1","addr":"172.21.15.122:6805","nonce":4045445512}]},"public_addr":"172.21.15.122:6801/4045445512","cluster_addr":"172.21.15.122:6803/4045445512","heartbeat_back_addr":"172.21.15.122:6807/4045445512","heartbeat_front_addr":"172.21.15.122:6805/4045445512","state":["exists","up"]},{"osd":5,"uuid":"6e64070a-671a-444b-a061-b3051a29b121","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":29,"up_from":57,"up_thru":60,"down_at":30,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":822572374},{"type":"v1","addr":"172.21.15.122:6825","nonce":822572374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":822572374},{"type":"v1","addr":"172.21.15.122:6827","nonce":822572374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":822572374},{"type":"v1","addr":"172.21.15.122:6831","nonce":822572374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":822572374},{"type":"v1","addr":"172.21.15.122:6829","nonce":822572374}]},"public_addr":"172.21.15.122:6825/822572374","cluster_addr":"172.21.15.122:6827/822572374","heartbeat_back_addr":"172.21.15.122:6831/822572374","heartbeat_front_addr":"172.21.15.122:6829/822572374","state":["exists","up"]},{"osd":6,"uuid":"47565b3a-3fa6-4a46-87a3-1d78bd0f4aa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6809","nonce":1437167311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6811","nonce":1437167311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6815","nonce":1437167311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":1437167311},{"type":"v1","addr":"172.21.15.122:6813","nonce":1437167311}]},"public_addr":"172.21.15.122:6809/1437167311","cluster_addr":"172.21.15.122:6811/1437167311","heartbeat_back_addr":"172.21.15.122:6815/1437167311","heartbeat_front_addr":"172.21.15.122:6813/1437167311","state":["exists","up"]},{"osd":7,"uuid":"9a916b6d-af82-486f-af40-4a7291c900fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6817","nonce":2431377162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6819","nonce":2431377162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6823","nonce":2431377162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":2431377162},{"type":"v1","addr":"172.21.15.122:6821","nonce":2431377162}]},"public_addr":"172.21.15.122:6817/2431377162","cluster_addr":"172.21.15.122:6819/2431377162","heartbeat_back_addr":"172.21.15.122:6823/2431377162","heartbeat_front_addr":"172.21.15.122:6821/2431377162","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.168771+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.886614+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.154129+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.570531+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:08.926773+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.146320+0000","dead_epoch":30},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.409137+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:46:09.373359+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.8","osds":[0,5]}],"pg_upmap_items":[{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:49:29.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.752+0000 7f6dac986640 1 -- 172.21.15.100:0/4257172992 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6d8806c2e0 msgr2=0x7f6d8806e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.752+0000 7f6dac986640 1 --2- 172.21.15.100:0/4257172992 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6d8806c2e0 0x7f6d8806e7a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009f30 tx=0x7f6da000b040 comp rx=0 tx=0).stop 2024-09-15T21:49:29.754 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.752+0000 7f6dac986640 1 -- 172.21.15.100:0/4257172992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da80cb7d0 msgr2=0x7f6da815c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.752+0000 7f6dac986640 1 --2- 172.21.15.100:0/4257172992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da80cb7d0 0x7f6da815c920 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6d90000c00 tx=0x7f6d900046c0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.752+0000 7f6dac986640 1 -- 172.21.15.100:0/4257172992 shutdown_connections 2024-09-15T21:49:29.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.752+0000 7f6dac986640 1 --2- 172.21.15.100:0/4257172992 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6d8806c2e0 0x7f6d8806e7a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.752+0000 7f6dac986640 1 --2- 172.21.15.100:0/4257172992 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6da8155440 0x7f6da815ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.752+0000 7f6dac986640 1 --2- 172.21.15.100:0/4257172992 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6da80cb7d0 0x7f6da815c920 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.752+0000 7f6dac986640 1 --2- 172.21.15.100:0/4257172992 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6da806d0e0 0x7f6da80b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.752+0000 7f6dac986640 1 -- 172.21.15.100:0/4257172992 >> 172.21.15.100:0/4257172992 conn(0x7f6da80680d0 msgr2=0x7f6da806bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:29.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.752+0000 7f6dac986640 1 -- 172.21.15.100:0/4257172992 shutdown_connections 2024-09-15T21:49:29.755 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.752+0000 7f6dac986640 1 -- 172.21.15.100:0/4257172992 wait complete. 2024-09-15T21:49:29.765 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-15T21:49:29.867 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.865+0000 7fa3a9842640 1 Processor -- start 2024-09-15T21:49:29.868 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.865+0000 7fa3a9842640 1 -- start start 2024-09-15T21:49:29.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.866+0000 7fa3a9842640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa3a40c3be0 0x7fa3a40ca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:29.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.866+0000 7fa3a9842640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3a4073cf0 con 0x7fa3a41553e0 2024-09-15T21:49:29.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.866+0000 7fa3a9842640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3a4073fd0 con 0x7fa3a40cb190 2024-09-15T21:49:29.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.866+0000 7fa3a9842640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a40cfb60 con 0x7fa3a40c3be0 2024-09-15T21:49:29.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.866+0000 7fa3a2ffd640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7fa3a41553e0 0x7fa3a415b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:55904/0 (socket says 172.21.15.100:55904) 2024-09-15T21:49:29.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.866+0000 7fa3a2ffd640 1 -- 172.21.15.100:0/3765890802 learned_addr learned my addr 172.21.15.100:0/3765890802 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:49:29.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a37fe640 1 --2- 172.21.15.100:0/3765890802 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa3a40c3be0 0x7fa3a40ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:29.870 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a1ffb640 1 -- 172.21.15.100:0/3765890802 <== mon.0 v1:172.21.15.100:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4127270725 0 0) 0x7fa3a4073cf0 con 0x7fa3a41553e0 2024-09-15T21:49:29.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a1ffb640 1 -- 172.21.15.100:0/3765890802 --> v1:172.21.15.100:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa38c003520 con 0x7fa3a41553e0 2024-09-15T21:49:29.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a1ffb640 1 -- 172.21.15.100:0/3765890802 <== mon.0 v1:172.21.15.100:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3209318497 0 0) 0x7fa38c003520 con 0x7fa3a41553e0 2024-09-15T21:49:29.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a1ffb640 1 -- 172.21.15.100:0/3765890802 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa3a40c3be0 msgr2=0x7fa3a40ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:49:29.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a1ffb640 1 --2- 172.21.15.100:0/3765890802 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa3a40c3be0 0x7fa3a40ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a1ffb640 1 -- 172.21.15.100:0/3765890802 >> v1:172.21.15.122:6789/0 conn(0x7fa3a40cb190 legacy=0x7fa3a415b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a1ffb640 1 -- 172.21.15.100:0/3765890802 --> v1:172.21.15.100:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a40c9890 con 0x7fa3a41553e0 2024-09-15T21:49:29.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a1ffb640 1 -- 172.21.15.100:0/3765890802 <== mon.0 v1:172.21.15.100:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2704569699 0 0) 0x7fa394001670 con 0x7fa3a41553e0 2024-09-15T21:49:29.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a1ffb640 1 -- 172.21.15.100:0/3765890802 >> v1:172.21.15.100:6789/0 conn(0x7fa3a41553e0 legacy=0x7fa3a415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a1ffb640 1 --2- 172.21.15.100:0/3765890802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa38c0039e0 0x7fa38c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:29.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a1ffb640 1 --2- 172.21.15.100:0/3765890802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa38c004390 0x7fa38c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:29.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a1ffb640 1 --2- 172.21.15.100:0/3765890802 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa3a40c3be0 0x7fa38c0050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:29.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a1ffb640 1 -- 172.21.15.100:0/3765890802 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa38c0141d0 con 0x7fa38c004390 2024-09-15T21:49:29.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a1ffb640 1 -- 172.21.15.100:0/3765890802 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa38c014350 con 0x7fa38c0039e0 2024-09-15T21:49:29.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a1ffb640 1 -- 172.21.15.100:0/3765890802 --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa38c014550 con 0x7fa3a40c3be0 2024-09-15T21:49:29.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a27fc640 1 --2- 172.21.15.100:0/3765890802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa38c004390 0x7fa38c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:29.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a2ffd640 1 --2- 172.21.15.100:0/3765890802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa38c0039e0 0x7fa38c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:29.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a37fe640 1 --2- 172.21.15.100:0/3765890802 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa3a40c3be0 0x7fa38c0050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:29.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a27fc640 1 -- 172.21.15.100:0/3765890802 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa3a40c3be0 msgr2=0x7fa38c0050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:49:29.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a27fc640 1 --2- 172.21.15.100:0/3765890802 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa3a40c3be0 0x7fa38c0050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a27fc640 1 -- 172.21.15.100:0/3765890802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa38c0039e0 msgr2=0x7fa38c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:49:29.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a27fc640 1 --2- 172.21.15.100:0/3765890802 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa38c0039e0 0x7fa38c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.867+0000 7fa3a27fc640 1 -- 172.21.15.100:0/3765890802 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38c003520 con 0x7fa38c004390 2024-09-15T21:49:29.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.868+0000 7fa3a27fc640 1 --2- 172.21.15.100:0/3765890802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa38c004390 0x7fa38c004860 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa398001910 tx=0x7fa38c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0847b8771ae27c server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:29.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.868+0000 7fa3a1ffb640 1 -- 172.21.15.100:0/3765890802 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398037070 con 0x7fa38c004390 2024-09-15T21:49:29.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.868+0000 7fa3a1ffb640 1 -- 172.21.15.100:0/3765890802 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa39802ac40 con 0x7fa38c004390 2024-09-15T21:49:29.872 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.868+0000 7fa3a1ffb640 1 -- 172.21.15.100:0/3765890802 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3980014e0 con 0x7fa38c004390 2024-09-15T21:49:29.875 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.869+0000 7fa3a9842640 1 -- 172.21.15.100:0/3765890802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa38c004390 msgr2=0x7fa38c004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.869+0000 7fa3a9842640 1 --2- 172.21.15.100:0/3765890802 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa38c004390 0x7fa38c004860 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa398001910 tx=0x7fa38c005830 comp rx=0 tx=0).stop 2024-09-15T21:49:29.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.869+0000 7fa3a37fe640 1 -- 172.21.15.100:0/3765890802 reap_dead start 2024-09-15T21:49:29.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.869+0000 7fa3a9842640 1 -- 172.21.15.100:0/3765890802 shutdown_connections 2024-09-15T21:49:29.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.869+0000 7fa3a9842640 1 -- 172.21.15.100:0/3765890802 >> 172.21.15.100:0/3765890802 conn(0x7fa3a40cec80 msgr2=0x7fa3a40cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:29.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.869+0000 7fa3a9842640 1 -- 172.21.15.100:0/3765890802 shutdown_connections 2024-09-15T21:49:29.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.869+0000 7fa3a9842640 1 -- 172.21.15.100:0/3765890802 wait complete. 2024-09-15T21:49:29.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.870+0000 7fa3a9842640 1 Processor -- start 2024-09-15T21:49:29.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.870+0000 7fa3a9842640 1 -- start start 2024-09-15T21:49:29.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.871+0000 7fa3a9842640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa38c004390 0x7fa3a4158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:29.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.871+0000 7fa3a9842640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3a406fb20 0x7fa3a406ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:29.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.871+0000 7fa3a9842640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3a41eff10 0x7fa3a406f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:29.876 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.871+0000 7fa3a9842640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a415c020 con 0x7fa3a406fb20 2024-09-15T21:49:29.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.871+0000 7fa3a9842640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a40c9ee0 con 0x7fa3a41eff10 2024-09-15T21:49:29.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.871+0000 7fa3a9842640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a40cf7a0 con 0x7fa38c004390 2024-09-15T21:49:29.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.871+0000 7fa3a27fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3a406fb20 0x7fa3a406ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:29.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.871+0000 7fa3a27fc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3a406fb20 0x7fa3a406ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46606/0 (socket says 172.21.15.100:46606) 2024-09-15T21:49:29.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.871+0000 7fa3a27fc640 1 -- 172.21.15.100:0/564552305 learned_addr learned my addr 172.21.15.100:0/564552305 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:49:29.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.871+0000 7fa3a2ffd640 1 --2- 172.21.15.100:0/564552305 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa38c004390 0x7fa3a4158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:29.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.871+0000 7fa3a37fe640 1 --2- 172.21.15.100:0/564552305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3a41eff10 0x7fa3a406f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:29.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.871+0000 7fa3a27fc640 1 -- 172.21.15.100:0/564552305 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa38c004390 msgr2=0x7fa3a4158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.871+0000 7fa3a27fc640 1 --2- 172.21.15.100:0/564552305 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa38c004390 0x7fa3a4158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.871+0000 7fa3a27fc640 1 -- 172.21.15.100:0/564552305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3a41eff10 msgr2=0x7fa3a406f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:29.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.871+0000 7fa3a27fc640 1 --2- 172.21.15.100:0/564552305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3a41eff10 0x7fa3a406f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:29.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.871+0000 7fa3a27fc640 1 -- 172.21.15.100:0/564552305 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a4159aa0 con 0x7fa3a406fb20 2024-09-15T21:49:29.877 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.871+0000 7fa3a27fc640 1 --2- 172.21.15.100:0/564552305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3a406fb20 0x7fa3a406ccf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa39802a730 tx=0x7fa3980012b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:29.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.872+0000 7fa38bfff640 1 -- 172.21.15.100:0/564552305 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398037040 con 0x7fa3a406fb20 2024-09-15T21:49:29.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.872+0000 7fa38bfff640 1 -- 172.21.15.100:0/564552305 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa398005070 con 0x7fa3a406fb20 2024-09-15T21:49:29.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.872+0000 7fa38bfff640 1 -- 172.21.15.100:0/564552305 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398005420 con 0x7fa3a406fb20 2024-09-15T21:49:29.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.872+0000 7fa3a9842640 1 -- 172.21.15.100:0/564552305 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a4159d80 con 0x7fa3a406fb20 2024-09-15T21:49:29.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.872+0000 7fa3a9842640 1 -- 172.21.15.100:0/564552305 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a415a340 con 0x7fa3a406fb20 2024-09-15T21:49:29.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.873+0000 7fa3a9842640 1 -- 172.21.15.100:0/564552305 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a415c3c0 con 0x7fa3a406fb20 2024-09-15T21:49:29.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.873+0000 7fa38bfff640 1 -- 172.21.15.100:0/564552305 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7fa39803a070 con 0x7fa3a406fb20 2024-09-15T21:49:29.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.874+0000 7fa38bfff640 1 --2- 172.21.15.100:0/564552305 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa39006c2b0 0x7fa39006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:29.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.874+0000 7fa3a2ffd640 1 --2- 172.21.15.100:0/564552305 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa39006c2b0 0x7fa39006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:29.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.874+0000 7fa38bfff640 1 -- 172.21.15.100:0/564552305 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 5886+0+0 (secure 0 0 0) 0x7fa3980ab390 con 0x7fa3a406fb20 2024-09-15T21:49:29.878 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.874+0000 7fa3a2ffd640 1 --2- 172.21.15.100:0/564552305 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa39006c2b0 0x7fa39006e770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa394007c20 tx=0x7fa394009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:29.880 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:29.878+0000 7fa38bfff640 1 -- 172.21.15.100:0/564552305 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7fa3a415c3c0 con 0x7fa3a406fb20 2024-09-15T21:49:30.079 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.077+0000 7fa3a9842640 1 -- 172.21.15.100:0/564552305 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa3a4073cf0 con 0x7fa3a406fb20 2024-09-15T21:49:30.081 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.080+0000 7fa38bfff640 1 -- 172.21.15.100:0/564552305 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7fa3a4073cf0 con 0x7fa3a406fb20 2024-09-15T21:49:30.082 INFO:teuthology.orchestra.run.smithi100.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-15T21:49:30.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.083+0000 7fa3a9842640 1 -- 172.21.15.100:0/564552305 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa39006c2b0 msgr2=0x7fa39006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:30.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.083+0000 7fa3a9842640 1 --2- 172.21.15.100:0/564552305 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa39006c2b0 0x7fa39006e770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa394007c20 tx=0x7fa394009040 comp rx=0 tx=0).stop 2024-09-15T21:49:30.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.083+0000 7fa3a9842640 1 -- 172.21.15.100:0/564552305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3a406fb20 msgr2=0x7fa3a406ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:30.085 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.083+0000 7fa3a9842640 1 --2- 172.21.15.100:0/564552305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3a406fb20 0x7fa3a406ccf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa39802a730 tx=0x7fa3980012b0 comp rx=0 tx=0).stop 2024-09-15T21:49:30.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.083+0000 7fa3a9842640 1 -- 172.21.15.100:0/564552305 shutdown_connections 2024-09-15T21:49:30.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.083+0000 7fa3a9842640 1 --2- 172.21.15.100:0/564552305 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7fa39006c2b0 0x7fa39006e770 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:30.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.083+0000 7fa3a9842640 1 --2- 172.21.15.100:0/564552305 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3a41eff10 0x7fa3a406f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:30.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.083+0000 7fa3a9842640 1 --2- 172.21.15.100:0/564552305 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa3a406fb20 0x7fa3a406ccf0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:30.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.083+0000 7fa3a9842640 1 --2- 172.21.15.100:0/564552305 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7fa38c004390 0x7fa3a4158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:30.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.083+0000 7fa3a9842640 1 -- 172.21.15.100:0/564552305 >> 172.21.15.100:0/564552305 conn(0x7fa3a40cec80 msgr2=0x7fa3a40c6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:30.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.083+0000 7fa3a9842640 1 -- 172.21.15.100:0/564552305 shutdown_connections 2024-09-15T21:49:30.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.083+0000 7fa3a9842640 1 -- 172.21.15.100:0/564552305 wait complete. 2024-09-15T21:49:30.088 INFO:tasks.ceph.mds.b.smithi122.stderr: -5576> 2024-09-15T21:46:40.469+0000 7ff533659640 -1 quiesce.mds.1 failed to decode message of type 1285 v1: End of buffer [buffer:2] 2024-09-15T21:49:30.098 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-15T21:49:30.122 INFO:tasks.ceph.mds.a.smithi100.stdout:starting mds.a at 2024-09-15T21:49:30.122 INFO:tasks.ceph.mds.b.smithi122.stdout:starting mds.b at 2024-09-15T21:49:30.122 INFO:tasks.ceph.mds.f.smithi122.stdout:starting mds.f at 2024-09-15T21:49:30.123 INFO:tasks.ceph.mds.f.smithi122.stderr:2024-09-15T21:49:30.121+0000 7f824cb4d640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-15T21:49:30.127 INFO:tasks.ceph.mds.c.smithi100.stdout:starting mds.c at 2024-09-15T21:49:30.135 INFO:tasks.ceph.mds.e.smithi100.stdout:starting mds.e at 2024-09-15T21:49:30.204 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.202+0000 7f6e802dd640 1 Processor -- start 2024-09-15T21:49:30.205 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.202+0000 7f6e802dd640 1 -- start start 2024-09-15T21:49:30.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.203+0000 7f6e802dd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6e781552a0 0x7f6e7815b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:30.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.203+0000 7f6e802dd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6e78155680 0x7f6e780c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:30.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.203+0000 7f6e802dd640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e7815b7f0 con 0x7f6e780c6020 2024-09-15T21:49:30.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.203+0000 7f6e802dd640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f6e7815bab0 con 0x7f6e78155680 2024-09-15T21:49:30.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.203+0000 7f6e802dd640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f6e7815bd90 con 0x7f6e781552a0 2024-09-15T21:49:30.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.204+0000 7f6e7d878640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6e78155680 0x7f6e780c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:30.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.204+0000 7f6e7d878640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f6e78155680 0x7f6e780c5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46656/0 (socket says 172.21.15.100:46656) 2024-09-15T21:49:30.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.204+0000 7f6e7d878640 1 -- 172.21.15.100:0/2592051296 learned_addr learned my addr 172.21.15.100:0/2592051296 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:49:30.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.204+0000 7f6e7e079640 1 --2- 172.21.15.100:0/2592051296 >> v2:172.21.15.122:3300/0 conn(0x7f6e781552a0 0x7f6e7815b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:30.207 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.204+0000 7f6e7d878640 1 -- 172.21.15.100:0/2592051296 >> v2:172.21.15.122:3300/0 conn(0x7f6e781552a0 msgr2=0x7f6e7815b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:49:30.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.204+0000 7f6e7d878640 1 --2- 172.21.15.100:0/2592051296 >> v2:172.21.15.122:3300/0 conn(0x7f6e781552a0 0x7f6e7815b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:30.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.204+0000 7f6e7d878640 1 -- 172.21.15.100:0/2592051296 >> v1:172.21.15.122:6789/0 conn(0x7f6e780c6020 legacy=0x7f6e780cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:49:30.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.204+0000 7f6e7d878640 1 -- 172.21.15.100:0/2592051296 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7815c070 con 0x7f6e78155680 2024-09-15T21:49:30.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.205+0000 7f6e7d878640 1 --2- 172.21.15.100:0/2592051296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e78155680 0x7f6e780c5ae0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6e78075b80 tx=0x7f6e6c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724d3be70ffc4343 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:30.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.205+0000 7f6e7d077640 1 -- 172.21.15.100:0/2592051296 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c037070 con 0x7f6e78155680 2024-09-15T21:49:30.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.205+0000 7f6e7d077640 1 -- 172.21.15.100:0/2592051296 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6e6c032440 con 0x7f6e78155680 2024-09-15T21:49:30.208 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.205+0000 7f6e7d077640 1 -- 172.21.15.100:0/2592051296 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c032830 con 0x7f6e78155680 2024-09-15T21:49:30.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.205+0000 7f6e802dd640 1 -- 172.21.15.100:0/2592051296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e78155680 msgr2=0x7f6e780c5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:30.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.205+0000 7f6e802dd640 1 --2- 172.21.15.100:0/2592051296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e78155680 0x7f6e780c5ae0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f6e78075b80 tx=0x7f6e6c02a2c0 comp rx=0 tx=0).stop 2024-09-15T21:49:30.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.205+0000 7f6e802dd640 1 -- 172.21.15.100:0/2592051296 shutdown_connections 2024-09-15T21:49:30.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.205+0000 7f6e802dd640 1 --2- 172.21.15.100:0/2592051296 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e78155680 0x7f6e780c5ae0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:30.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.205+0000 7f6e802dd640 1 --2- 172.21.15.100:0/2592051296 >> v2:172.21.15.122:3300/0 conn(0x7f6e781552a0 0x7f6e7815b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:30.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.205+0000 7f6e802dd640 1 -- 172.21.15.100:0/2592051296 >> 172.21.15.100:0/2592051296 conn(0x7f6e7806d9a0 msgr2=0x7f6e7806ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:30.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.205+0000 7f6e802dd640 1 -- 172.21.15.100:0/2592051296 shutdown_connections 2024-09-15T21:49:30.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.205+0000 7f6e802dd640 1 -- 172.21.15.100:0/2592051296 wait complete. 2024-09-15T21:49:30.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.206+0000 7f6e802dd640 1 Processor -- start 2024-09-15T21:49:30.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.207+0000 7f6e802dd640 1 -- start start 2024-09-15T21:49:30.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.207+0000 7f6e802dd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e780c6020 0x7f6e780b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:30.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.207+0000 7f6e802dd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6e781552a0 0x7f6e781f1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:30.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.207+0000 7f6e802dd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e78155680 0x7f6e781f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:30.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.207+0000 7f6e802dd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f6e780cc020 con 0x7f6e780c6020 2024-09-15T21:49:30.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.207+0000 7f6e802dd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6e780c6820 con 0x7f6e78155680 2024-09-15T21:49:30.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.207+0000 7f6e802dd640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f6e780cc1a0 con 0x7f6e781552a0 2024-09-15T21:49:30.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.207+0000 7f6e7e079640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e780c6020 0x7f6e780b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:30.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.207+0000 7f6e7e079640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e780c6020 0x7f6e780b4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46664/0 (socket says 172.21.15.100:46664) 2024-09-15T21:49:30.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.207+0000 7f6e7e079640 1 -- 172.21.15.100:0/700194806 learned_addr learned my addr 172.21.15.100:0/700194806 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:49:30.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.207+0000 7f6e7e87a640 1 --2- 172.21.15.100:0/700194806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e78155680 0x7f6e781f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:30.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.207+0000 7f6e7d878640 1 --2- 172.21.15.100:0/700194806 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6e781552a0 0x7f6e781f1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:30.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.208+0000 7f6e7e079640 1 -- 172.21.15.100:0/700194806 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6e781552a0 msgr2=0x7f6e781f1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:30.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.208+0000 7f6e7e079640 1 --2- 172.21.15.100:0/700194806 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6e781552a0 0x7f6e781f1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:30.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.208+0000 7f6e7e079640 1 -- 172.21.15.100:0/700194806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e78155680 msgr2=0x7f6e781f1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:30.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.208+0000 7f6e7e079640 1 --2- 172.21.15.100:0/700194806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e78155680 0x7f6e781f1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:30.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.208+0000 7f6e7e079640 1 -- 172.21.15.100:0/700194806 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7814b600 con 0x7f6e780c6020 2024-09-15T21:49:30.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.208+0000 7f6e7e079640 1 --2- 172.21.15.100:0/700194806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e780c6020 0x7f6e780b4f20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000d830 tx=0x7f6e7000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:30.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.208+0000 7f6e76ffd640 1 -- 172.21.15.100:0/700194806 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7000b6c0 con 0x7f6e780c6020 2024-09-15T21:49:30.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.208+0000 7f6e76ffd640 1 -- 172.21.15.100:0/700194806 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6e70019070 con 0x7f6e780c6020 2024-09-15T21:49:30.215 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.208+0000 7f6e76ffd640 1 -- 172.21.15.100:0/700194806 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7000baa0 con 0x7f6e780c6020 2024-09-15T21:49:30.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.208+0000 7f6e802dd640 1 -- 172.21.15.100:0/700194806 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e7814b8c0 con 0x7f6e780c6020 2024-09-15T21:49:30.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.208+0000 7f6e802dd640 1 -- 172.21.15.100:0/700194806 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e78154000 con 0x7f6e780c6020 2024-09-15T21:49:30.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.209+0000 7f6e802dd640 1 -- 172.21.15.100:0/700194806 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e78154aa0 con 0x7f6e780c6020 2024-09-15T21:49:30.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.209+0000 7f6e76ffd640 1 -- 172.21.15.100:0/700194806 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f6e7001e460 con 0x7f6e780c6020 2024-09-15T21:49:30.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.210+0000 7f6e76ffd640 1 --2- 172.21.15.100:0/700194806 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6e6406c2e0 0x7f6e6406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:30.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.210+0000 7f6e76ffd640 1 -- 172.21.15.100:0/700194806 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6316+0+0 (secure 0 0 0) 0x7f6e7008bd10 con 0x7f6e780c6020 2024-09-15T21:49:30.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.210+0000 7f6e7d878640 1 --2- 172.21.15.100:0/700194806 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6e6406c2e0 0x7f6e6406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:30.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.210+0000 7f6e7d878640 1 --2- 172.21.15.100:0/700194806 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6e6406c2e0 0x7f6e6406e7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c034000 tx=0x7f6e6c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:30.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.214+0000 7f6e76ffd640 1 -- 172.21.15.100:0/700194806 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f6e70059b40 con 0x7f6e780c6020 2024-09-15T21:49:30.414 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:30.413+0000 7f6e802dd640 1 -- 172.21.15.100:0/700194806 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6e7815b0f0 con 0x7f6e780c6020 2024-09-15T21:49:31.083 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.081+0000 7f6e76ffd640 1 -- 172.21.15.100:0/700194806 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f6e7005de40 con 0x7f6e780c6020 2024-09-15T21:49:31.086 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.084+0000 7f6e802dd640 1 -- 172.21.15.100:0/700194806 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6e6406c2e0 msgr2=0x7f6e6406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:31.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.084+0000 7f6e802dd640 1 --2- 172.21.15.100:0/700194806 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6e6406c2e0 0x7f6e6406e7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c034000 tx=0x7f6e6c02e000 comp rx=0 tx=0).stop 2024-09-15T21:49:31.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.084+0000 7f6e802dd640 1 -- 172.21.15.100:0/700194806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e780c6020 msgr2=0x7f6e780b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:31.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.084+0000 7f6e802dd640 1 --2- 172.21.15.100:0/700194806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e780c6020 0x7f6e780b4f20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f6e7000d830 tx=0x7f6e7000dcf0 comp rx=0 tx=0).stop 2024-09-15T21:49:31.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.084+0000 7f6e802dd640 1 -- 172.21.15.100:0/700194806 shutdown_connections 2024-09-15T21:49:31.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.084+0000 7f6e802dd640 1 --2- 172.21.15.100:0/700194806 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f6e6406c2e0 0x7f6e6406e7a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:31.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.084+0000 7f6e802dd640 1 --2- 172.21.15.100:0/700194806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e78155680 0x7f6e781f1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:31.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.084+0000 7f6e802dd640 1 --2- 172.21.15.100:0/700194806 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f6e781552a0 0x7f6e781f1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:31.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.084+0000 7f6e802dd640 1 --2- 172.21.15.100:0/700194806 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6e780c6020 0x7f6e780b4f20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:31.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.084+0000 7f6e802dd640 1 -- 172.21.15.100:0/700194806 >> 172.21.15.100:0/700194806 conn(0x7f6e7806d9a0 msgr2=0x7f6e780c6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:31.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.084+0000 7f6e802dd640 1 -- 172.21.15.100:0/700194806 shutdown_connections 2024-09-15T21:49:31.087 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.084+0000 7f6e802dd640 1 -- 172.21.15.100:0/700194806 wait complete. 2024-09-15T21:49:31.100 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-15T21:49:31.243 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.242+0000 7f21d44af640 1 Processor -- start 2024-09-15T21:49:31.244 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.242+0000 7f21d44af640 1 -- start start 2024-09-15T21:49:31.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.243+0000 7f21d44af640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f21cc151d60 0x7f21cc06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:31.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.243+0000 7f21d44af640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f21cc152140 0x7f21cc072f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:31.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.243+0000 7f21d44af640 1 -- --> v1:172.21.15.100:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21cc157e30 con 0x7f21cc072ab0 2024-09-15T21:49:31.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.243+0000 7f21d44af640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f21cc06e4c0 con 0x7f21cc152140 2024-09-15T21:49:31.245 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.243+0000 7f21d44af640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 -- 0x7f21cc158a00 con 0x7f21cc151d60 2024-09-15T21:49:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.243+0000 7f21d2a4c640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f21cc152140 0x7f21cc072f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.243+0000 7f21d1a4a640 1 --1- >> v1:172.21.15.100:6789/0 conn(0x7f21cc072ab0 0x7f21cc06ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.100:6789/0 says I am v1:172.21.15.100:55946/0 (socket says 172.21.15.100:55946) 2024-09-15T21:49:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.243+0000 7f21d1a4a640 1 -- 172.21.15.100:0/513286705 learned_addr learned my addr 172.21.15.100:0/513286705 (peer_addr_for_me v1:172.21.15.100:0/0) 2024-09-15T21:49:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.243+0000 7f21d224b640 1 --2- 172.21.15.100:0/513286705 >> v2:172.21.15.122:3300/0 conn(0x7f21cc151d60 0x7f21cc06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.243+0000 7f21d2a4c640 1 -- 172.21.15.100:0/513286705 >> v2:172.21.15.122:3300/0 conn(0x7f21cc151d60 msgr2=0x7f21cc06dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:49:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.243+0000 7f21d2a4c640 1 --2- 172.21.15.100:0/513286705 >> v2:172.21.15.122:3300/0 conn(0x7f21cc151d60 0x7f21cc06dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.243+0000 7f21d2a4c640 1 -- 172.21.15.100:0/513286705 >> v1:172.21.15.100:6789/0 conn(0x7f21cc072ab0 legacy=0x7f21cc06ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.243+0000 7f21d2a4c640 1 -- 172.21.15.100:0/513286705 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc1580d0 con 0x7f21cc152140 2024-09-15T21:49:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.243+0000 7f21d224b640 1 --2- 172.21.15.100:0/513286705 >> v2:172.21.15.122:3300/0 conn(0x7f21cc151d60 0x7f21cc06dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-15T21:49:31.246 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.244+0000 7f21d2a4c640 1 --2- 172.21.15.100:0/513286705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21cc152140 0x7f21cc072f70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f21cc069b00 tx=0x7f21c402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4527d54b5faa5636 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:31.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.244+0000 7f21d1249640 1 -- 172.21.15.100:0/513286705 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c4007740 con 0x7f21cc152140 2024-09-15T21:49:31.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.244+0000 7f21d1249640 1 -- 172.21.15.100:0/513286705 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f21c40078e0 con 0x7f21cc152140 2024-09-15T21:49:31.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.244+0000 7f21d1249640 1 -- 172.21.15.100:0/513286705 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c4007cd0 con 0x7f21cc152140 2024-09-15T21:49:31.249 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.244+0000 7f21d44af640 1 -- 172.21.15.100:0/513286705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21cc152140 msgr2=0x7f21cc072f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:31.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.244+0000 7f21d44af640 1 --2- 172.21.15.100:0/513286705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21cc152140 0x7f21cc072f70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f21cc069b00 tx=0x7f21c402a680 comp rx=0 tx=0).stop 2024-09-15T21:49:31.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.245+0000 7f21d44af640 1 -- 172.21.15.100:0/513286705 shutdown_connections 2024-09-15T21:49:31.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.245+0000 7f21d44af640 1 --2- 172.21.15.100:0/513286705 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21cc152140 0x7f21cc072f70 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:31.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.245+0000 7f21d44af640 1 --2- 172.21.15.100:0/513286705 >> v2:172.21.15.122:3300/0 conn(0x7f21cc151d60 0x7f21cc06dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:31.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.245+0000 7f21d44af640 1 -- 172.21.15.100:0/513286705 >> 172.21.15.100:0/513286705 conn(0x7f21cc0cd930 msgr2=0x7f21cc0cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:31.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.245+0000 7f21d44af640 1 -- 172.21.15.100:0/513286705 shutdown_connections 2024-09-15T21:49:31.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.245+0000 7f21d44af640 1 -- 172.21.15.100:0/513286705 wait complete. 2024-09-15T21:49:31.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.246+0000 7f21d44af640 1 Processor -- start 2024-09-15T21:49:31.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.246+0000 7f21d44af640 1 -- start start 2024-09-15T21:49:31.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.246+0000 7f21d44af640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21cc072ab0 0x7f21cc0cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:31.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.246+0000 7f21d44af640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21cc151d60 0x7f21cc154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:31.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.246+0000 7f21d44af640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21cc152140 0x7f21cc1f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:31.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.246+0000 7f21d44af640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc06d400 con 0x7f21cc151d60 2024-09-15T21:49:31.250 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.246+0000 7f21d44af640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc158690 con 0x7f21cc072ab0 2024-09-15T21:49:31.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.246+0000 7f21d44af640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f21cc06d580 con 0x7f21cc152140 2024-09-15T21:49:31.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.247+0000 7f21d1a4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21cc151d60 0x7f21cc154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:31.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.247+0000 7f21d1a4a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21cc151d60 0x7f21cc154a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46674/0 (socket says 172.21.15.100:46674) 2024-09-15T21:49:31.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.247+0000 7f21d1a4a640 1 -- 172.21.15.100:0/3288822815 learned_addr learned my addr 172.21.15.100:0/3288822815 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:49:31.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.247+0000 7f21d224b640 1 --2- 172.21.15.100:0/3288822815 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21cc072ab0 0x7f21cc0cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:31.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.247+0000 7f21d2a4c640 1 --2- 172.21.15.100:0/3288822815 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21cc152140 0x7f21cc1f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:31.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.247+0000 7f21d1a4a640 1 -- 172.21.15.100:0/3288822815 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21cc152140 msgr2=0x7f21cc1f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:31.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.247+0000 7f21d1a4a640 1 --2- 172.21.15.100:0/3288822815 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21cc152140 0x7f21cc1f1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:31.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.247+0000 7f21d1a4a640 1 -- 172.21.15.100:0/3288822815 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21cc072ab0 msgr2=0x7f21cc0cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:31.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.247+0000 7f21d1a4a640 1 --2- 172.21.15.100:0/3288822815 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21cc072ab0 0x7f21cc0cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:31.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.247+0000 7f21d1a4a640 1 -- 172.21.15.100:0/3288822815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc14a4c0 con 0x7f21cc151d60 2024-09-15T21:49:31.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.247+0000 7f21d224b640 1 --2- 172.21.15.100:0/3288822815 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21cc072ab0 0x7f21cc0cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:49:31.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.247+0000 7f21d2a4c640 1 --2- 172.21.15.100:0/3288822815 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21cc152140 0x7f21cc1f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-15T21:49:31.251 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.247+0000 7f21d1a4a640 1 --2- 172.21.15.100:0/3288822815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21cc151d60 0x7f21cc154a80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00c470 tx=0x7f21bc001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:31.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.247+0000 7f21c2ffd640 1 -- 172.21.15.100:0/3288822815 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc017070 con 0x7f21cc151d60 2024-09-15T21:49:31.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.247+0000 7f21c2ffd640 1 -- 172.21.15.100:0/3288822815 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f21bc005d20 con 0x7f21cc151d60 2024-09-15T21:49:31.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.247+0000 7f21c2ffd640 1 -- 172.21.15.100:0/3288822815 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21bc007330 con 0x7f21cc151d60 2024-09-15T21:49:31.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.247+0000 7f21d44af640 1 -- 172.21.15.100:0/3288822815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21cc14a760 con 0x7f21cc151d60 2024-09-15T21:49:31.252 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.247+0000 7f21d44af640 1 -- 172.21.15.100:0/3288822815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21cc145690 con 0x7f21cc151d60 2024-09-15T21:49:31.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.248+0000 7f21d44af640 1 -- 172.21.15.100:0/3288822815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21cc0c4700 con 0x7f21cc151d60 2024-09-15T21:49:31.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.248+0000 7f21c2ffd640 1 -- 172.21.15.100:0/3288822815 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f21bc01a070 con 0x7f21cc151d60 2024-09-15T21:49:31.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.249+0000 7f21c2ffd640 1 --2- 172.21.15.100:0/3288822815 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f21b006c2b0 0x7f21b006e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:31.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.249+0000 7f21c2ffd640 1 -- 172.21.15.100:0/3288822815 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6316+0+0 (secure 0 0 0) 0x7f21bc08aa10 con 0x7f21cc151d60 2024-09-15T21:49:31.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.249+0000 7f21d224b640 1 --2- 172.21.15.100:0/3288822815 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f21b006c2b0 0x7f21b006e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:31.253 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.249+0000 7f21d224b640 1 --2- 172.21.15.100:0/3288822815 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f21b006c2b0 0x7f21b006e770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f21c800aa30 tx=0x7f21c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:31.255 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.253+0000 7f21c2ffd640 1 -- 172.21.15.100:0/3288822815 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f21bc058840 con 0x7f21cc151d60 2024-09-15T21:49:31.420 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:31.418+0000 7f21d44af640 1 -- 172.21.15.100:0/3288822815 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f21cc0c7090 con 0x7f21cc151d60 2024-09-15T21:49:32.088 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.086+0000 7f21c2ffd640 1 -- 172.21.15.100:0/3288822815 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) ==== 181+0+0 (secure 0 0 0) 0x7f21bc05cb40 con 0x7f21cc151d60 2024-09-15T21:49:32.088 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_metadata' removed 2024-09-15T21:49:32.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.088+0000 7f21d44af640 1 -- 172.21.15.100:0/3288822815 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f21b006c2b0 msgr2=0x7f21b006e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:32.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.088+0000 7f21d44af640 1 --2- 172.21.15.100:0/3288822815 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f21b006c2b0 0x7f21b006e770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f21c800aa30 tx=0x7f21c8008040 comp rx=0 tx=0).stop 2024-09-15T21:49:32.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.088+0000 7f21d44af640 1 -- 172.21.15.100:0/3288822815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21cc151d60 msgr2=0x7f21cc154a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:32.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.088+0000 7f21d44af640 1 --2- 172.21.15.100:0/3288822815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21cc151d60 0x7f21cc154a80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f21bc00c470 tx=0x7f21bc001e70 comp rx=0 tx=0).stop 2024-09-15T21:49:32.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.088+0000 7f21d44af640 1 -- 172.21.15.100:0/3288822815 shutdown_connections 2024-09-15T21:49:32.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.088+0000 7f21d44af640 1 --2- 172.21.15.100:0/3288822815 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f21b006c2b0 0x7f21b006e770 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:32.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.088+0000 7f21d44af640 1 --2- 172.21.15.100:0/3288822815 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f21cc152140 0x7f21cc1f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:32.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.088+0000 7f21d44af640 1 --2- 172.21.15.100:0/3288822815 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f21cc151d60 0x7f21cc154a80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:32.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.088+0000 7f21d44af640 1 --2- 172.21.15.100:0/3288822815 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f21cc072ab0 0x7f21cc0cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:32.091 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.088+0000 7f21d44af640 1 -- 172.21.15.100:0/3288822815 >> 172.21.15.100:0/3288822815 conn(0x7f21cc0cd930 msgr2=0x7f21cc06db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:32.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.089+0000 7f21d44af640 1 -- 172.21.15.100:0/3288822815 shutdown_connections 2024-09-15T21:49:32.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.089+0000 7f21d44af640 1 -- 172.21.15.100:0/3288822815 wait complete. 2024-09-15T21:49:32.105 DEBUG:teuthology.orchestra.run.smithi100:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-15T21:49:32.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.207+0000 7f2f4d88d640 1 Processor -- start 2024-09-15T21:49:32.209 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.207+0000 7f2f4d88d640 1 -- start start 2024-09-15T21:49:32.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.208+0000 7f2f4d88d640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2f480c4120 0x7f2f4815f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:32.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.208+0000 7f2f4d88d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f480c4500 0x7f2f480ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:32.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.208+0000 7f2f4d88d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f4815f7b0 con 0x7f2f480c4900 2024-09-15T21:49:32.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.208+0000 7f2f4d88d640 1 -- --> v2:172.21.15.100:3300/0 -- mon_getmap magic: 0 -- 0x7f2f4806ed40 con 0x7f2f480c4120 2024-09-15T21:49:32.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.208+0000 7f2f4d88d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2f48071a40 con 0x7f2f480c4500 2024-09-15T21:49:32.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.208+0000 7f2f46ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2f480c4120 0x7f2f4815f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:32.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.208+0000 7f2f46ffd640 1 --2- >> v2:172.21.15.100:3300/0 conn(0x7f2f480c4120 0x7f2f4815f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46688/0 (socket says 172.21.15.100:46688) 2024-09-15T21:49:32.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.208+0000 7f2f46ffd640 1 -- 172.21.15.100:0/598576961 learned_addr learned my addr 172.21.15.100:0/598576961 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:49:32.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.208+0000 7f2f467fc640 1 --2- 172.21.15.100:0/598576961 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f480c4500 0x7f2f480ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:32.211 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.209+0000 7f2f46ffd640 1 -- 172.21.15.100:0/598576961 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f480c4500 msgr2=0x7f2f480ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-15T21:49:32.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.209+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/598576961 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f480c4500 0x7f2f480ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:32.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.209+0000 7f2f46ffd640 1 -- 172.21.15.100:0/598576961 >> v1:172.21.15.122:6789/0 conn(0x7f2f480c4900 legacy=0x7f2f48154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:32.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.209+0000 7f2f46ffd640 1 -- 172.21.15.100:0/598576961 --> v2:172.21.15.100:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f48071760 con 0x7f2f480c4120 2024-09-15T21:49:32.212 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.209+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/598576961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f480c4120 0x7f2f4815f270 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009870 tx=0x7f2f3802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4257cca31b19dac4 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:32.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.209+0000 7f2f45ffb640 1 -- 172.21.15.100:0/598576961 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f38037070 con 0x7f2f480c4120 2024-09-15T21:49:32.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.210+0000 7f2f45ffb640 1 -- 172.21.15.100:0/598576961 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2f38032440 con 0x7f2f480c4120 2024-09-15T21:49:32.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.210+0000 7f2f45ffb640 1 -- 172.21.15.100:0/598576961 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f38032830 con 0x7f2f480c4120 2024-09-15T21:49:32.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.210+0000 7f2f4d88d640 1 -- 172.21.15.100:0/598576961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f480c4120 msgr2=0x7f2f4815f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:32.213 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.210+0000 7f2f4d88d640 1 --2- 172.21.15.100:0/598576961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f480c4120 0x7f2f4815f270 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009870 tx=0x7f2f3802a270 comp rx=0 tx=0).stop 2024-09-15T21:49:32.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.210+0000 7f2f4d88d640 1 -- 172.21.15.100:0/598576961 shutdown_connections 2024-09-15T21:49:32.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.210+0000 7f2f4d88d640 1 --2- 172.21.15.100:0/598576961 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f480c4500 0x7f2f480ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:32.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.210+0000 7f2f4d88d640 1 --2- 172.21.15.100:0/598576961 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f480c4120 0x7f2f4815f270 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:32.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.210+0000 7f2f4d88d640 1 -- 172.21.15.100:0/598576961 >> 172.21.15.100:0/598576961 conn(0x7f2f48067980 msgr2=0x7f2f48067d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:32.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.210+0000 7f2f4d88d640 1 -- 172.21.15.100:0/598576961 shutdown_connections 2024-09-15T21:49:32.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.210+0000 7f2f4d88d640 1 -- 172.21.15.100:0/598576961 wait complete. 2024-09-15T21:49:32.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.211+0000 7f2f4d88d640 1 Processor -- start 2024-09-15T21:49:32.214 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.211+0000 7f2f4d88d640 1 -- start start 2024-09-15T21:49:32.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.212+0000 7f2f4d88d640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f480c4120 0x7f2f480b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:32.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.212+0000 7f2f4d88d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f480c4500 0x7f2f480b2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:32.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.212+0000 7f2f4d88d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f480c4900 0x7f2f48157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:32.216 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.212+0000 7f2f4d88d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4815b940 con 0x7f2f480c4900 2024-09-15T21:49:32.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.212+0000 7f2f4d88d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4815b660 con 0x7f2f480c4500 2024-09-15T21:49:32.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.212+0000 7f2f4d88d640 1 -- --> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] -- mon_getmap magic: 0 -- 0x7f2f4815bc20 con 0x7f2f480c4120 2024-09-15T21:49:32.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.212+0000 7f2f477fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f480c4900 0x7f2f48157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:32.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.212+0000 7f2f46ffd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f480c4120 0x7f2f480b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:32.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.212+0000 7f2f46ffd640 1 --2- >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f480c4120 0x7f2f480b23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3301/0 says I am v2:172.21.15.100:48236/0 (socket says 172.21.15.100:48236) 2024-09-15T21:49:32.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.212+0000 7f2f477fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f480c4900 0x7f2f48157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:46690/0 (socket says 172.21.15.100:46690) 2024-09-15T21:49:32.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.212+0000 7f2f46ffd640 1 -- 172.21.15.100:0/4217052717 learned_addr learned my addr 172.21.15.100:0/4217052717 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-09-15T21:49:32.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.212+0000 7f2f467fc640 1 --2- 172.21.15.100:0/4217052717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f480c4500 0x7f2f480b2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:32.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.212+0000 7f2f477fe640 1 -- 172.21.15.100:0/4217052717 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f480c4120 msgr2=0x7f2f480b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:32.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.212+0000 7f2f477fe640 1 --2- 172.21.15.100:0/4217052717 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f480c4120 0x7f2f480b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:32.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.212+0000 7f2f477fe640 1 -- 172.21.15.100:0/4217052717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f480c4500 msgr2=0x7f2f480b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:32.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.212+0000 7f2f477fe640 1 --2- 172.21.15.100:0/4217052717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f480c4500 0x7f2f480b2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:32.217 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.213+0000 7f2f477fe640 1 -- 172.21.15.100:0/4217052717 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f480b0550 con 0x7f2f480c4900 2024-09-15T21:49:32.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.213+0000 7f2f477fe640 1 --2- 172.21.15.100:0/4217052717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f480c4900 0x7f2f48157600 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000c730 tx=0x7f2f4000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:32.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.213+0000 7f2f2ffff640 1 -- 172.21.15.100:0/4217052717 <== mon.0 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40003b70 con 0x7f2f480c4900 2024-09-15T21:49:32.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.213+0000 7f2f2ffff640 1 -- 172.21.15.100:0/4217052717 <== mon.0 v2:172.21.15.100:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2f40003d10 con 0x7f2f480c4900 2024-09-15T21:49:32.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.213+0000 7f2f4d88d640 1 -- 172.21.15.100:0/4217052717 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f480b0830 con 0x7f2f480c4900 2024-09-15T21:49:32.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.213+0000 7f2f2ffff640 1 -- 172.21.15.100:0/4217052717 <== mon.0 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f40005330 con 0x7f2f480c4900 2024-09-15T21:49:32.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.213+0000 7f2f4d88d640 1 -- 172.21.15.100:0/4217052717 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f480b1bc0 con 0x7f2f480c4900 2024-09-15T21:49:32.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.214+0000 7f2f4d88d640 1 -- 172.21.15.100:0/4217052717 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f480ce3c0 con 0x7f2f480c4900 2024-09-15T21:49:32.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.214+0000 7f2f2ffff640 1 -- 172.21.15.100:0/4217052717 <== mon.0 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 5) ==== 90846+0+0 (secure 0 0 0) 0x7f2f400056e0 con 0x7f2f480c4900 2024-09-15T21:49:32.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.215+0000 7f2f2ffff640 1 --2- 172.21.15.100:0/4217052717 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2f3406c2e0 0x7f2f3406e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-15T21:49:32.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.215+0000 7f2f2ffff640 1 -- 172.21.15.100:0/4217052717 <== mon.0 v2:172.21.15.100:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5849+0+0 (secure 0 0 0) 0x7f2f4008b3b0 con 0x7f2f480c4900 2024-09-15T21:49:32.218 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.215+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/4217052717 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2f3406c2e0 0x7f2f3406e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-15T21:49:32.219 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.215+0000 7f2f46ffd640 1 --2- 172.21.15.100:0/4217052717 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2f3406c2e0 0x7f2f3406e7a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2f380090b0 tx=0x7f2f3802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-15T21:49:32.220 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.219+0000 7f2f2ffff640 1 -- 172.21.15.100:0/4217052717 <== mon.0 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187697 (secure 0 0 0) 0x7f2f480ce3c0 con 0x7f2f480c4900 2024-09-15T21:49:32.389 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:32.387+0000 7f2f4d88d640 1 -- 172.21.15.100:0/4217052717 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2f4815c140 con 0x7f2f480c4900 2024-09-15T21:49:33.090 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:33.088+0000 7f2f2ffff640 1 -- 172.21.15.100:0/4217052717 <== mon.0 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) ==== 169+0+0 (secure 0 0 0) 0x7f2f40059440 con 0x7f2f480c4900 2024-09-15T21:49:33.090 INFO:teuthology.orchestra.run.smithi100.stderr:pool 'cephfs_data' removed 2024-09-15T21:49:33.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:33.090+0000 7f2f4d88d640 1 -- 172.21.15.100:0/4217052717 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2f3406c2e0 msgr2=0x7f2f3406e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:33.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:33.090+0000 7f2f4d88d640 1 --2- 172.21.15.100:0/4217052717 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2f3406c2e0 0x7f2f3406e7a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2f380090b0 tx=0x7f2f3802e000 comp rx=0 tx=0).stop 2024-09-15T21:49:33.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:33.090+0000 7f2f4d88d640 1 -- 172.21.15.100:0/4217052717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f480c4900 msgr2=0x7f2f48157600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-15T21:49:33.092 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:33.090+0000 7f2f4d88d640 1 --2- 172.21.15.100:0/4217052717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f480c4900 0x7f2f48157600 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000c730 tx=0x7f2f4000cbf0 comp rx=0 tx=0).stop 2024-09-15T21:49:33.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:33.090+0000 7f2f4d88d640 1 -- 172.21.15.100:0/4217052717 shutdown_connections 2024-09-15T21:49:33.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:33.090+0000 7f2f4d88d640 1 --2- 172.21.15.100:0/4217052717 >> [v2:172.21.15.100:6808/3516908621,v1:172.21.15.100:6809/3516908621] conn(0x7f2f3406c2e0 0x7f2f3406e7a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:33.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:33.090+0000 7f2f4d88d640 1 --2- 172.21.15.100:0/4217052717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2f480c4500 0x7f2f480b2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:33.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:33.090+0000 7f2f4d88d640 1 --2- 172.21.15.100:0/4217052717 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2f480c4900 0x7f2f48157600 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:33.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:33.090+0000 7f2f4d88d640 1 --2- 172.21.15.100:0/4217052717 >> [v2:172.21.15.122:3301/0,v1:172.21.15.122:6790/0] conn(0x7f2f480c4120 0x7f2f480b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-15T21:49:33.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:33.090+0000 7f2f4d88d640 1 -- 172.21.15.100:0/4217052717 >> 172.21.15.100:0/4217052717 conn(0x7f2f48067980 msgr2=0x7f2f480cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-15T21:49:33.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:33.090+0000 7f2f4d88d640 1 -- 172.21.15.100:0/4217052717 shutdown_connections 2024-09-15T21:49:33.093 INFO:teuthology.orchestra.run.smithi100.stderr:2024-09-15T21:49:33.090+0000 7f2f4d88d640 1 -- 172.21.15.100:0/4217052717 wait complete. 2024-09-15T21:49:33.108 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-15T21:49:33.108 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-15T21:49:33.108 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:49:33.119 INFO:tasks.ceph.mds.a:Stopped 2024-09-15T21:49:33.119 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-15T21:49:33.119 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:49:33.170 INFO:tasks.ceph.mds.c:Stopped 2024-09-15T21:49:33.171 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-09-15T21:49:33.171 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:49:33.221 INFO:tasks.ceph.mds.e:Stopped 2024-09-15T21:49:33.221 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-15T21:49:33.221 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:49:33.233 INFO:tasks.ceph.mds.b:Stopped 2024-09-15T21:49:33.233 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-15T21:49:33.233 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:49:33.285 INFO:tasks.ceph.mds.d:Stopped 2024-09-15T21:49:33.285 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-09-15T21:49:33.285 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:49:33.337 INFO:tasks.ceph.mds.f:Stopped 2024-09-15T21:49:33.338 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-15T21:49:33.338 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T21:49:33.338 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:49:33.369 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T21:49:33.369 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T21:49:33.369 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:49:33.438 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T21:49:33.438 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T21:49:33.438 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:49:33.512 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T21:49:33.512 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T21:49:33.512 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:49:33.586 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T21:49:33.586 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-15T21:49:33.587 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:49:33.620 INFO:tasks.ceph.osd.4:Stopped 2024-09-15T21:49:33.620 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-15T21:49:33.621 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:49:33.697 INFO:tasks.ceph.osd.5:Stopped 2024-09-15T21:49:33.698 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-15T21:49:33.698 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:49:33.771 INFO:tasks.ceph.osd.6:Stopped 2024-09-15T21:49:33.772 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-15T21:49:33.772 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:49:33.845 INFO:tasks.ceph.osd.7:Stopped 2024-09-15T21:49:33.845 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-15T21:49:33.845 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-15T21:49:33.845 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:49:33.872 INFO:tasks.ceph.mgr.y:Stopped 2024-09-15T21:49:33.873 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-15T21:49:33.873 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:49:33.914 INFO:tasks.ceph.mgr.x:Stopped 2024-09-15T21:49:33.914 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-15T21:49:33.914 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T21:49:33.914 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:49:33.927 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T21:49:33.927 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:49:33.927 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:49:33.968 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:49:33.968 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T21:49:33.968 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:49:34.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-15T21:49:34.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-15T21:49:34.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-09-15T21:49:34.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-15T21:49:34.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-15T21:49:34.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-15T21:49:34.023 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-09-15T21:49:34.023 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-09-15T21:49:34.023 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-09-15T21:49:34.023 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T21:49:34.023 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-15T21:49:34.023 DEBUG:teuthology.orchestra.run.smithi100:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v MDS_SLOW_METADATA_IO | egrep -v MDS_TRIM | head -n 1 2024-09-15T21:49:34.069 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-15T21:49:34.070 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-15T21:49:34.358 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-15T21:49:34.358 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-15T21:49:34.461 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-15T21:49:34.461 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-15T21:49:34.517 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi100.front.sepia.ceph.com 2024-09-15T21:49:34.517 DEBUG:teuthology.orchestra.run.smithi100:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-15T21:49:34.576 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T21:49:34.576 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-15T21:49:34.777 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T21:49:34.778 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-15T21:49:34.870 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T21:49:34.870 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-15T21:49:34.925 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T21:49:34.925 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-15T21:49:34.997 INFO:tasks.ceph:Archiving mon data... 2024-09-15T21:49:34.998 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906191/data/mon.a.tgz 2024-09-15T21:49:34.998 DEBUG:teuthology.orchestra.run.smithi100:> mktemp 2024-09-15T21:49:35.017 INFO:teuthology.orchestra.run.smithi100.stdout:/tmp/tmp.zuqlLXvCH5 2024-09-15T21:49:35.017 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.zuqlLXvCH5 2024-09-15T21:49:35.199 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 0666 /tmp/tmp.zuqlLXvCH5 2024-09-15T21:49:35.286 DEBUG:teuthology.orchestra.remote:smithi100:/tmp/tmp.zuqlLXvCH5 is 359KB 2024-09-15T21:49:35.385 DEBUG:teuthology.orchestra.run.smithi100:> rm -fr /tmp/tmp.zuqlLXvCH5 2024-09-15T21:49:35.401 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906191/data/mon.b.tgz 2024-09-15T21:49:35.401 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-15T21:49:35.420 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.wbIdNFKXQb 2024-09-15T21:49:35.420 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.wbIdNFKXQb 2024-09-15T21:49:35.594 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.wbIdNFKXQb 2024-09-15T21:49:35.679 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.wbIdNFKXQb is 493KB 2024-09-15T21:49:35.777 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.wbIdNFKXQb 2024-09-15T21:49:35.793 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906191/data/mon.c.tgz 2024-09-15T21:49:35.793 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-15T21:49:35.847 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.SNAcSEej3M 2024-09-15T21:49:35.847 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.SNAcSEej3M 2024-09-15T21:49:36.003 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.SNAcSEej3M 2024-09-15T21:49:36.085 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.SNAcSEej3M is 493KB 2024-09-15T21:49:36.184 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.SNAcSEej3M 2024-09-15T21:49:36.201 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-15T21:49:36.201 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T21:49:36.204 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T21:49:36.289 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T21:49:36.320 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T21:49:36.320 INFO:tasks.ceph:Archiving crash dumps... 2024-09-15T21:49:36.322 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906191/remote/smithi100/crash 2024-09-15T21:49:36.323 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T21:49:36.365 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906191/remote/smithi122/crash 2024-09-15T21:49:36.366 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T21:49:36.390 INFO:tasks.ceph:Shutting down logrotate 2024-09-15T21:49:36.391 DEBUG:teuthology.orchestra.run.smithi100:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T21:49:36.408 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-15T21:49:36.486 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-15T21:49:36.514 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-15T21:49:36.538 INFO:tasks.ceph:Compressing logs... 2024-09-15T21:49:36.539 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T21:49:36.558 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T21:49:36.617 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.43984.log 2024-09-15T21:49:36.617 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37465.log 2024-09-15T21:49:36.618 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-15T21:49:36.618 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/objectstore_tool.43984.log: /var/log/ceph/ceph-client.admin.37465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37505.log 2024-09-15T21:49:36.618 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.7.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37465.log.gz 2024-09-15T21:49:36.618 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-15T21:49:36.618 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.37505.log: -- /var/log/ceph/ceph-client.admin.37625.log 2024-09-15T21:49:36.618 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37545.log 2024-09-15T21:49:36.618 INFO:teuthology.orchestra.run.smithi122.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37505.log.gz 2024-09-15T21:49:36.618 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37625.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-15T21:49:36.618 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37625.log.gz 2024-09-15T21:49:36.618 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37545.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.46711.log 2024-09-15T21:49:36.618 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T21:49:36.619 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.4.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37545.log.gz 2024-09-15T21:49:36.619 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/objectstore_tool.46711.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-15T21:49:36.619 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.45762.log 2024-09-15T21:49:36.619 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-15T21:49:36.619 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/objectstore_tool.45762.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T21:49:36.619 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-15T21:49:36.619 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.6.log: 95.1% -- replaced with /var/log/ceph/objectstore_tool.43984.log.gz 2024-09-15T21:49:36.619 INFO:teuthology.orchestra.run.smithi122.stderr: 94.9% -- replaced with /var/log/ceph/objectstore_tool.46711.log.gz 2024-09-15T21:49:36.619 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T21:49:36.619 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-15T21:49:36.619 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph.log: 95.1% -- replaced with /var/log/ceph/objectstore_tool.45762.log.gz 2024-09-15T21:49:36.619 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-15T21:49:36.619 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mds.d.log: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T21:49:36.619 INFO:teuthology.orchestra.run.smithi122.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-15T21:49:36.620 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37756.log 2024-09-15T21:49:36.624 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-15T21:49:36.625 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.admin.37756.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.37756.log.gz 2024-09-15T21:49:36.625 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.44830.log 2024-09-15T21:49:36.625 INFO:teuthology.orchestra.run.smithi122.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-15T21:49:36.625 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37585.log 2024-09-15T21:49:36.629 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/objectstore_tool.44830.log: 95.2% -- replaced with /var/log/ceph/objectstore_tool.44830.log.gz 2024-09-15T21:49:36.629 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37716.log 2024-09-15T21:49:36.630 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37585.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.37585.log.gz 2024-09-15T21:49:36.637 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-09-15T21:49:36.637 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37716.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37716.log.gz 2024-09-15T21:49:36.637 INFO:teuthology.orchestra.run.smithi122.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-15T21:49:36.637 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37674.log 2024-09-15T21:49:36.637 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mds.f.log: /var/log/ceph/ceph-client.admin.37674.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37674.log.gz 2024-09-15T21:49:36.654 INFO:teuthology.orchestra.run.smithi122.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-09-15T21:49:36.716 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-15T21:49:36.717 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46960.log 2024-09-15T21:49:36.717 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44244.log 2024-09-15T21:49:36.717 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50270.log 2024-09-15T21:49:36.717 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44244.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46960.log.gzgzip 2024-09-15T21:49:36.717 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42563.log 2024-09-15T21:49:36.717 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44244.log.gz 2024-09-15T21:49:36.717 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55232.log 2024-09-15T21:49:36.717 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50270.log: /var/log/ceph/ceph-client.admin.42563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43121.log 2024-09-15T21:49:36.717 INFO:teuthology.orchestra.run.smithi100.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50270.log.gz 2024-09-15T21:49:36.717 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54126.log 2024-09-15T21:49:36.717 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55232.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42563.log.gz 2024-09-15T21:49:36.718 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43121.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55232.log.gz 2024-09-15T21:49:36.718 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49721.log 2024-09-15T21:49:36.718 INFO:teuthology.orchestra.run.smithi100.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43121.log.gz 2024-09-15T21:49:36.718 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56314.log 2024-09-15T21:49:36.718 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49721.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54126.log.gz 2024-09-15T21:49:36.718 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50768.log 2024-09-15T21:49:36.718 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56314.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.49721.log.gz 2024-09-15T21:49:36.718 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56692.log 2024-09-15T21:49:36.718 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56314.log.gz 2024-09-15T21:49:36.718 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54336.log 2024-09-15T21:49:36.718 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57116.log 2024-09-15T21:49:36.718 INFO:teuthology.orchestra.run.smithi100.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.50768.log.gz 2024-09-15T21:49:36.718 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56692.log.gz 2024-09-15T21:49:36.718 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46551.log 2024-09-15T21:49:36.719 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54336.log.gz 2024-09-15T21:49:36.719 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51365.log 2024-09-15T21:49:36.719 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46551.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57116.log.gz 2024-09-15T21:49:36.719 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56188.log 2024-09-15T21:49:36.719 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46551.log.gz 2024-09-15T21:49:36.719 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43457.log 2024-09-15T21:49:36.719 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42437.log 2024-09-15T21:49:36.719 INFO:teuthology.orchestra.run.smithi100.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.51365.log.gz 2024-09-15T21:49:36.719 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43457.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56188.log.gz 2024-09-15T21:49:36.719 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54294.log 2024-09-15T21:49:36.719 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43457.log.gz 2024-09-15T21:49:36.719 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42732.log 2024-09-15T21:49:36.719 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4%/var/log/ceph/ceph-client.admin.54294.log: -- replaced with /var/log/ceph/ceph-client.admin.42437.log.gz 2024-09-15T21:49:36.719 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44806.log 2024-09-15T21:49:36.720 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54294.log.gz 2024-09-15T21:49:36.720 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55559.log 2024-09-15T21:49:36.720 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42732.log.gz 2024-09-15T21:49:36.720 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56524.log 2024-09-15T21:49:36.720 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55559.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-15T21:49:36.720 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56524.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55559.log.gz 2024-09-15T21:49:36.720 INFO:teuthology.orchestra.run.smithi100.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44806.log.gz 2024-09-15T21:49:36.720 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42774.log 2024-09-15T21:49:36.720 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56524.log.gz 2024-09-15T21:49:36.720 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52392.log 2024-09-15T21:49:36.720 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47664.log 2024-09-15T21:49:36.720 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42774.log: /var/log/ceph/ceph-client.admin.52392.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42774.log.gz 2024-09-15T21:49:36.720 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45508.log 2024-09-15T21:49:36.721 INFO:teuthology.orchestra.run.smithi100.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.52392.log.gz 2024-09-15T21:49:36.721 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55218.log 2024-09-15T21:49:36.721 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47664.log: /var/log/ceph/ceph-client.admin.45508.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30108.log 2024-09-15T21:49:36.721 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45508.log.gz 2024-09-15T21:49:36.721 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55218.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.47664.log.gz 2024-09-15T21:49:36.721 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42816.log 2024-09-15T21:49:36.721 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.55218.log.gz 2024-09-15T21:49:36.721 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45466.log 2024-09-15T21:49:36.721 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.tmp-client.admin.30108.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30108.log.gz 2024-09-15T21:49:36.721 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55226.log 2024-09-15T21:49:36.721 INFO:teuthology.orchestra.run.smithi100.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42816.log.gz 2024-09-15T21:49:36.721 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54895.log 2024-09-15T21:49:36.722 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45466.log.gz 2024-09-15T21:49:36.722 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56440.log 2024-09-15T21:49:36.722 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55226.log: /var/log/ceph/ceph-client.admin.54895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46836.log 2024-09-15T21:49:36.722 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55226.log.gz 2024-09-15T21:49:36.722 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54895.log.gz 2024-09-15T21:49:36.722 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44131.log 2024-09-15T21:49:36.722 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56440.log.gz 2024-09-15T21:49:36.722 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48136.log 2024-09-15T21:49:36.722 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42521.log 2024-09-15T21:49:36.723 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44131.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46836.log.gz 2024-09-15T21:49:36.723 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48136.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44131.log.gz 2024-09-15T21:49:36.723 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57074.log 2024-09-15T21:49:36.723 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56878.log 2024-09-15T21:49:36.723 INFO:teuthology.orchestra.run.smithi100.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48136.log.gz 2024-09-15T21:49:36.723 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42521.log: /var/log/ceph/ceph-client.admin.57074.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42521.log.gz 2024-09-15T21:49:36.723 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43802.log 2024-09-15T21:49:36.723 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57074.log.gz 2024-09-15T21:49:36.723 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55568.log 2024-09-15T21:49:36.723 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56878.log: /var/log/ceph/ceph-client.admin.43802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55806.log 2024-09-15T21:49:36.723 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56878.log.gz 2024-09-15T21:49:36.723 INFO:teuthology.orchestra.run.smithi100.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43802.log.gz 2024-09-15T21:49:36.723 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42131.log 2024-09-15T21:49:36.723 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42395.log 88.7% 2024-09-15T21:49:36.724 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55568.log.gz 2024-09-15T21:49:36.724 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55806.log: /var/log/ceph/ceph-client.admin.42131.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55806.log.gz 2024-09-15T21:49:36.724 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54210.log 2024-09-15T21:49:36.724 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42131.log.gz 2024-09-15T21:49:36.724 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46712.log 2024-09-15T21:49:36.724 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42395.log: /var/log/ceph/ceph-client.admin.54210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54084.log 2024-09-15T21:49:36.724 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42395.log.gz 2024-09-15T21:49:36.724 INFO:teuthology.orchestra.run.smithi100.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54210.log.gz 2024-09-15T21:49:36.724 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51480.log 2024-09-15T21:49:36.724 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54084.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46712.log.gz 2024-09-15T21:49:36.724 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42311.log 2024-09-15T21:49:36.724 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54084.log.gz 2024-09-15T21:49:36.725 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42219.log 2024-09-15T21:49:36.725 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51480.log.gz 2024-09-15T21:49:36.725 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44412.log 2024-09-15T21:49:36.725 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42311.log: /var/log/ceph/ceph-client.admin.42219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43460.log 2024-09-15T21:49:36.725 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42311.log.gz 2024-09-15T21:49:36.725 INFO:teuthology.orchestra.run.smithi100.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42219.log.gz 2024-09-15T21:49:36.725 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52289.log 2024-09-15T21:49:36.725 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47022.log 2024-09-15T21:49:36.725 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44412.log.gz 2024-09-15T21:49:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43460.log: /var/log/ceph/ceph-client.admin.52289.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43460.log.gz 2024-09-15T21:49:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52169.log 2024-09-15T21:49:36.726 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52289.log.gz 2024-09-15T21:49:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49083.log 2024-09-15T21:49:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45424.log 2024-09-15T21:49:36.726 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47022.log.gz 2024-09-15T21:49:36.726 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.52169.log.gz 2024-09-15T21:49:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51066.log 2024-09-15T21:49:36.726 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54892.log 2024-09-15T21:49:36.726 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0%/var/log/ceph/ceph-client.admin.45424.log: -- replaced with /var/log/ceph/ceph-client.admin.49083.log.gz 2024-09-15T21:49:36.727 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51066.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45424.log.gz 2024-09-15T21:49:36.727 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56734.log 2024-09-15T21:49:36.727 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44046.log 2024-09-15T21:49:36.727 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54892.log: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.51066.log.gz 2024-09-15T21:49:36.727 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46650.log 2024-09-15T21:49:36.727 INFO:teuthology.orchestra.run.smithi100.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54892.log.gz 2024-09-15T21:49:36.728 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44046.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56734.log.gz 2024-09-15T21:49:36.728 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57242.log 2024-09-15T21:49:36.728 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44046.log.gz 2024-09-15T21:49:36.728 INFO:teuthology.orchestra.run.smithi100.stderr:gzip/var/log/ceph/ceph-client.admin.46650.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43441.log 2024-09-15T21:49:36.728 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54252.log 2024-09-15T21:49:36.729 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46650.log.gz 2024-09-15T21:49:36.729 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43441.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57242.log.gz 2024-09-15T21:49:36.729 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49776.log 2024-09-15T21:49:36.729 INFO:teuthology.orchestra.run.smithi100.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43441.log.gz 2024-09-15T21:49:36.729 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50155.log 2024-09-15T21:49:36.729 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54252.log: /var/log/ceph/ceph-client.admin.49776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43456.log 2024-09-15T21:49:36.730 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54252.log.gz 2024-09-15T21:49:36.730 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50155.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49776.log.gz 2024-09-15T21:49:36.730 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46374.log 2024-09-15T21:49:36.730 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43456.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.50155.log.gz 2024-09-15T21:49:36.730 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42353.log 2024-09-15T21:49:36.730 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43456.log.gz 2024-09-15T21:49:36.731 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53851.log 2024-09-15T21:49:36.731 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46374.log: /var/log/ceph/ceph-client.admin.42353.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T21:49:36.731 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53851.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42353.log.gz 2024-09-15T21:49:36.731 INFO:teuthology.orchestra.run.smithi100.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.46374.log.gz 2024-09-15T21:49:36.731 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42175.log 2024-09-15T21:49:36.731 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53851.log.gz 2024-09-15T21:49:36.732 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50575.log 2024-09-15T21:49:36.732 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.42175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54887.log 2024-09-15T21:49:36.733 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54504.log 2024-09-15T21:49:36.733 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42175.log.gz 2024-09-15T21:49:36.733 INFO:teuthology.orchestra.run.smithi100.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50575.log.gz 2024-09-15T21:49:36.733 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42089.log 2024-09-15T21:49:36.733 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.54887.log.gz 2024-09-15T21:49:36.733 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54378.log 2024-09-15T21:49:36.734 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54504.log: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T21:49:36.734 INFO:teuthology.orchestra.run.smithi100.stderr:gzip/var/log/ceph/ceph-client.admin.42089.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.53953.log 2024-09-15T21:49:36.734 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54504.log.gz 2024-09-15T21:49:36.734 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54378.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42089.log.gz 2024-09-15T21:49:36.735 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44173.log 2024-09-15T21:49:36.735 INFO:teuthology.orchestra.run.smithi100.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54378.log.gz 2024-09-15T21:49:36.735 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43808.log 2024-09-15T21:49:36.735 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53953.log: /var/log/ceph/ceph-client.admin.44173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55848.log 2024-09-15T21:49:36.735 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44173.log.gz 2024-09-15T21:49:36.737 INFO:teuthology.orchestra.run.smithi100.stderr: 91.4%/var/log/ceph/ceph-client.admin.43808.log: -- replaced with /var/log/ceph/ceph-client.admin.53953.log.gz 2024-09-15T21:49:36.737 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56230.log 2024-09-15T21:49:36.737 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43808.log.gz 2024-09-15T21:49:36.737 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43117.log 2024-09-15T21:49:36.737 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55848.log: /var/log/ceph/ceph-client.admin.56230.log: gzip -5 --verbose -- 89.2% /var/log/ceph/ceph-client.admin.50113.log -- replaced with /var/log/ceph/ceph-client.admin.55848.log.gz 2024-09-15T21:49:36.737 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-15T21:49:36.737 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56230.log.gz 2024-09-15T21:49:36.737 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57200.log 2024-09-15T21:49:36.738 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43117.log: /var/log/ceph/ceph-client.admin.50113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54000.log 2024-09-15T21:49:36.738 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43117.log.gz 2024-09-15T21:49:36.738 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50113.log.gz 2024-09-15T21:49:36.738 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49400.log 2024-09-15T21:49:36.738 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57200.log.gz 2024-09-15T21:49:36.738 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45706.log/var/log/ceph/ceph-client.admin.54000.log: 2024-09-15T21:49:36.738 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56818.log 2024-09-15T21:49:36.738 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54000.log.gz 2024-09-15T21:49:36.738 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45706.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49400.log.gz 2024-09-15T21:49:36.738 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46898.log 2024-09-15T21:49:36.738 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54630.log 2024-09-15T21:49:36.738 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56818.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.45706.log.gz 2024-09-15T21:49:36.738 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56650.log 2024-09-15T21:49:36.738 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56818.log.gz 2024-09-15T21:49:36.739 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46898.log.gz 2024-09-15T21:49:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45707.log 2024-09-15T21:49:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56650.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54630.log.gz 2024-09-15T21:49:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48774.log 2024-09-15T21:49:36.739 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56650.log.gz 2024-09-15T21:49:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43120.log 2024-09-15T21:49:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45707.log: /var/log/ceph/ceph-client.admin.48774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36589.log 2024-09-15T21:49:36.739 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45707.log.gz 2024-09-15T21:49:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43120.log: gzip 89.5% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.48774.log.gz -- 2024-09-15T21:49:36.739 INFO:teuthology.orchestra.run.smithi100.stderr: /var/log/ceph/ceph-client.admin.45382.log 2024-09-15T21:49:36.739 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43120.log.gz 2024-09-15T21:49:36.739 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43668.log 2024-09-15T21:49:36.739 INFO:teuthology.orchestra.run.smithi100.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36589.log.gz 2024-09-15T21:49:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44286.log 2024-09-15T21:49:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45382.log: /var/log/ceph/ceph-client.admin.43668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46212.log 2024-09-15T21:49:36.740 INFO:teuthology.orchestra.run.smithi100.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45382.log.gz 2024-09-15T21:49:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44286.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43668.log.gz 2024-09-15T21:49:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42648.log 2024-09-15T21:49:36.740 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44286.log.gz 2024-09-15T21:49:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36488.log 2024-09-15T21:49:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46212.log: /var/log/ceph/ceph-client.admin.42648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48987.log 2024-09-15T21:49:36.740 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46212.log.gz 2024-09-15T21:49:36.740 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42648.log.gz 2024-09-15T21:49:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43452.log 2024-09-15T21:49:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-15T21:49:36.740 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4%/var/log/ceph/ceph-client.admin.48987.log: -- replaced with /var/log/ceph/ceph-client.admin.36488.log.gz 2024-09-15T21:49:36.740 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44674.log 2024-09-15T21:49:36.741 INFO:teuthology.orchestra.run.smithi100.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.48987.log.gz 2024-09-15T21:49:36.741 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43452.log.gz 2024-09-15T21:49:36.741 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49183.log 2024-09-15T21:49:36.741 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55562.log 2024-09-15T21:49:36.741 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52548.log 2024-09-15T21:49:36.741 INFO:teuthology.orchestra.run.smithi100.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44674.log.gz 2024-09-15T21:49:36.741 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55562.log: 89.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43803.log 2024-09-15T21:49:36.741 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49183.log.gz 2024-09-15T21:49:36.741 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43123.log 2024-09-15T21:49:36.741 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52548.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55562.log.gz 2024-09-15T21:49:36.741 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43803.log: gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.52548.log.gz --verbose 2024-09-15T21:49:36.741 INFO:teuthology.orchestra.run.smithi100.stderr: -- /var/log/ceph/ceph-client.admin.47405.log 2024-09-15T21:49:36.741 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43803.log.gz 2024-09-15T21:49:36.741 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54894.log 2024-09-15T21:49:36.742 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43123.log.gz 2024-09-15T21:49:36.742 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44370.log 2024-09-15T21:49:36.742 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41945.log 2024-09-15T21:49:36.742 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54894.log.gz 2024-09-15T21:49:36.742 INFO:teuthology.orchestra.run.smithi100.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.47405.log.gz 2024-09-15T21:49:36.742 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44328.log 2024-09-15T21:49:36.742 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54672.log 2024-09-15T21:49:36.742 INFO:teuthology.orchestra.run.smithi100.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44370.log.gz 2024-09-15T21:49:36.742 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44328.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41945.log.gz 2024-09-15T21:49:36.742 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43112.log 2024-09-15T21:49:36.742 INFO:teuthology.orchestra.run.smithi100.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44328.log.gz 2024-09-15T21:49:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42858.log 2024-09-15T21:49:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54672.log: /var/log/ceph/ceph-client.admin.43112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43118.log 2024-09-15T21:49:36.743 INFO:teuthology.orchestra.run.smithi100.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54672.log.gz 2024-09-15T21:49:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42858.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47562.log 2024-09-15T21:49:36.743 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43112.log.gz 2024-09-15T21:49:36.743 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42858.log.gz 2024-09-15T21:49:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43807.log 2024-09-15T21:49:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47084.log 2024-09-15T21:49:36.743 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43118.log.gz 2024-09-15T21:49:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47562.log: /var/log/ceph/ceph-client.admin.43807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54714.log 2024-09-15T21:49:36.743 INFO:teuthology.orchestra.run.smithi100.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47562.log.gz 2024-09-15T21:49:36.743 INFO:teuthology.orchestra.run.smithi100.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43807.log.gz 2024-09-15T21:49:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54168.log 2024-09-15T21:49:36.743 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56962.log 2024-09-15T21:49:36.744 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47084.log.gz 2024-09-15T21:49:36.744 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54714.log: /var/log/ceph/ceph-client.admin.54168.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54714.log.gz 2024-09-15T21:49:36.744 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55224.log 2024-09-15T21:49:36.744 INFO:teuthology.orchestra.run.smithi100.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.54168.log.gz 2024-09-15T21:49:36.744 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51183.log 2024-09-15T21:49:36.744 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56962.log: /var/log/ceph/ceph-client.admin.55224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50968.log 2024-09-15T21:49:36.744 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56962.log.gz 2024-09-15T21:49:36.744 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51183.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55224.log.gz 2024-09-15T21:49:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56104.log 2024-09-15T21:49:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50968.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51183.log.gz 2024-09-15T21:49:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-15T21:49:36.745 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50968.log.gz 2024-09-15T21:49:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52845.log 2024-09-15T21:49:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36224.log 2024-09-15T21:49:36.745 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56104.log.gz 2024-09-15T21:49:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.52845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54042.log 2024-09-15T21:49:36.745 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52845.log.gz 2024-09-15T21:49:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.36224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53276.log 2024-09-15T21:49:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54042.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36224.log.gz 2024-09-15T21:49:36.745 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43124.log 2024-09-15T21:49:36.745 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.54042.log.gz 2024-09-15T21:49:36.746 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46592.log 2024-09-15T21:49:36.746 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53276.log.gz 2024-09-15T21:49:36.746 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43124.log: gzip -5 --verbose -- 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43124.log.gz 2024-09-15T21:49:36.746 INFO:teuthology.orchestra.run.smithi100.stderr: /var/log/ceph/ceph-client.admin.42690.log 2024-09-15T21:49:36.746 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55563.log 2024-09-15T21:49:36.746 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42690.log: gzip -5 91.4% -- replaced with /var/log/ceph/ceph-client.admin.46592.log.gz 2024-09-15T21:49:36.746 INFO:teuthology.orchestra.run.smithi100.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55922.log 2024-09-15T21:49:36.746 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55563.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42690.log.gz 2024-09-15T21:49:36.746 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54896.log 2024-09-15T21:49:36.746 INFO:teuthology.orchestra.run.smithi100.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55563.log.gz 2024-09-15T21:49:36.747 INFO:teuthology.orchestra.run.smithi100.stderr:gzip/var/log/ceph/ceph-client.admin.55922.log: -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-15T21:49:36.747 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54896.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55922.log.gz 2024-09-15T21:49:36.747 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44088.log 2024-09-15T21:49:36.747 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54896.log.gz 2024-09-15T21:49:36.747 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44539.log 2024-09-15T21:49:36.747 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48382.log 2024-09-15T21:49:36.747 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0%/var/log/ceph/ceph-client.admin.44539.log: -- replaced with /var/log/ceph/ceph-client.admin.44088.log.gz 2024-09-15T21:49:36.747 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53305.log 2024-09-15T21:49:36.747 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.48382.log: -- /var/log/ceph/ceph-client.admin.45570.log 2024-09-15T21:49:36.747 INFO:teuthology.orchestra.run.smithi100.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44539.log.gz 2024-09-15T21:49:36.747 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53305.log: 90.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41236.log 2024-09-15T21:49:36.747 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48382.log.gz 2024-09-15T21:49:36.747 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51817.log 2024-09-15T21:49:36.748 INFO:teuthology.orchestra.run.smithi100.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.53305.log.gz 2024-09-15T21:49:36.748 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.41236.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.45570.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54886.log 2024-09-15T21:49:36.748 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-15T21:49:36.748 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41236.log.gz 2024-09-15T21:49:36.748 INFO:teuthology.orchestra.run.smithi100.stderr:gzip/var/log/ceph/ceph-client.admin.51817.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56062.log 2024-09-15T21:49:36.748 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56566.log 2024-09-15T21:49:36.748 INFO:teuthology.orchestra.run.smithi100.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.51817.log.gz 2024-09-15T21:49:36.748 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54886.log.gz 2024-09-15T21:49:36.748 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50514.log 2024-09-15T21:49:36.748 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56062.log: /var/log/ceph/ceph-client.admin.56566.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56062.log.gz 2024-09-15T21:49:36.748 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56566.log.gz 2024-09-15T21:49:36.748 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43122.log 2024-09-15T21:49:36.748 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50514.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-15T21:49:36.749 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50514.log.gz 2024-09-15T21:49:36.749 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44004.log 2024-09-15T21:49:36.749 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43122.log.gz/var/log/ceph/ceph-mgr.y.log: 2024-09-15T21:49:36.749 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42942.log 2024-09-15T21:49:36.749 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51777.log 2024-09-15T21:49:36.749 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44004.log: /var/log/ceph/ceph-client.admin.42942.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44004.log.gz 2024-09-15T21:49:36.749 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42942.log.gz 2024-09-15T21:49:36.749 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51621.log 2024-09-15T21:49:36.749 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42269.log 2024-09-15T21:49:36.749 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.51621.log: /var/log/ceph/ceph-client.admin.51777.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51777.log.gz 2024-09-15T21:49:36.749 INFO:teuthology.orchestra.run.smithi100.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.51621.log.gz 2024-09-15T21:49:36.750 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T21:49:36.750 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57034.log 2024-09-15T21:49:36.750 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42269.log: /var/log/ceph/ceph.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42269.log.gz 2024-09-15T21:49:36.750 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44740.log 2024-09-15T21:49:36.750 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57034.log: 91.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-15T21:49:36.750 INFO:teuthology.orchestra.run.smithi100.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57034.log.gz 2024-09-15T21:49:36.750 INFO:teuthology.orchestra.run.smithi100.stderr: 85.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T21:49:36.750 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43459.log 2024-09-15T21:49:36.750 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47873.log 2024-09-15T21:49:36.751 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55567.log 2024-09-15T21:49:36.751 INFO:teuthology.orchestra.run.smithi100.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.44740.log.gz 2024-09-15T21:49:36.751 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43459.log.gz 2024-09-15T21:49:36.751 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57383.log 2024-09-15T21:49:36.751 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49958.log 2024-09-15T21:49:36.751 INFO:teuthology.orchestra.run.smithi100.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47873.log.gz 2024-09-15T21:49:36.752 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42047.log 2024-09-15T21:49:36.752 INFO:teuthology.orchestra.run.smithi100.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55567.log.gz 2024-09-15T21:49:36.752 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54546.log 2024-09-15T21:49:36.752 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57383.log.gz 2024-09-15T21:49:36.752 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49958.log: /var/log/ceph/ceph-client.admin.42047.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42047.log.gz 2024-09-15T21:49:36.752 INFO:teuthology.orchestra.run.smithi100.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.49958.log.gz 2024-09-15T21:49:36.753 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54420.log 2024-09-15T21:49:36.753 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57312.log 2024-09-15T21:49:36.754 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54420.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54546.log.gz 2024-09-15T21:49:36.754 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46155.log 2024-09-15T21:49:36.754 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54420.log.gz 2024-09-15T21:49:36.754 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48495.log 2024-09-15T21:49:36.754 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.57312.log: /var/log/ceph/ceph-client.admin.46155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55566.log 2024-09-15T21:49:36.755 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48495.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57312.log.gz 2024-09-15T21:49:36.755 INFO:teuthology.orchestra.run.smithi100.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.46155.log.gz 2024-09-15T21:49:36.755 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50840.log 2024-09-15T21:49:36.755 INFO:teuthology.orchestra.run.smithi100.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48495.log.gz 2024-09-15T21:49:36.755 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50372.log 2024-09-15T21:49:36.755 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55566.log: /var/log/ceph/ceph-client.admin.50840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56776.log 2024-09-15T21:49:36.755 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55566.log.gz 2024-09-15T21:49:36.755 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.50372.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50840.log.gz 2024-09-15T21:49:36.756 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42605.log 2024-09-15T21:49:36.756 INFO:teuthology.orchestra.run.smithi100.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50372.log.gz 2024-09-15T21:49:36.756 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45709.log 2024-09-15T21:49:36.756 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56776.log: /var/log/ceph/ceph-client.admin.42605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52114.log 2024-09-15T21:49:36.756 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45709.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56776.log.gz 2024-09-15T21:49:36.756 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42605.log.gz 2024-09-15T21:49:36.756 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55428.log 2024-09-15T21:49:36.757 INFO:teuthology.orchestra.run.smithi100.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45709.log.gz 2024-09-15T21:49:36.757 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44497.log 2024-09-15T21:49:36.757 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52114.log: gzip/var/log/ceph/ceph-client.admin.55428.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44908.log 2024-09-15T21:49:36.757 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44497.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55428.log.gz 2024-09-15T21:49:36.757 INFO:teuthology.orchestra.run.smithi100.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.52114.log.gz 2024-09-15T21:49:36.757 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44497.log.gz 2024-09-15T21:49:36.757 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54883.log 2024-09-15T21:49:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43800.log 2024-09-15T21:49:36.758 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44908.log.gz 2024-09-15T21:49:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56482.log 2024-09-15T21:49:36.758 INFO:teuthology.orchestra.run.smithi100.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.54883.log.gz 2024-09-15T21:49:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53911.log 2024-09-15T21:49:36.758 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43800.log: /var/log/ceph/ceph-client.admin.56482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53500.log 2024-09-15T21:49:36.759 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53911.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43800.log.gz 2024-09-15T21:49:36.759 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56482.log.gz 2024-09-15T21:49:36.759 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.53911.log.gz 2024-09-15T21:49:36.759 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44608.log 2024-09-15T21:49:36.759 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56146.log 2024-09-15T21:49:36.759 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53500.log: /var/log/ceph/ceph-client.admin.44608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55229.log 2024-09-15T21:49:36.759 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56146.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.53500.log.gz 2024-09-15T21:49:36.760 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56146.log.gz 91.2% -- replaced with /var/log/ceph/ceph-client.admin.44608.log.gz 2024-09-15T21:49:36.760 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-15T21:49:36.760 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43454.log 2024-09-15T21:49:36.760 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56356.log 2024-09-15T21:49:36.760 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55229.log: /var/log/ceph/ceph-client.admin.43454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44454.log 2024-09-15T21:49:36.760 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56356.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55229.log.gz 2024-09-15T21:49:36.760 INFO:teuthology.orchestra.run.smithi100.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43454.log.gz 2024-09-15T21:49:36.760 INFO:teuthology.orchestra.run.smithi100.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56356.log.gz 2024-09-15T21:49:36.760 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41830.log 2024-09-15T21:49:36.760 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42479.log 2024-09-15T21:49:36.760 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.44454.log: /var/log/ceph/ceph-client.admin.41830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43805.log 2024-09-15T21:49:36.761 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.42479.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44454.log.gz 2024-09-15T21:49:36.761 INFO:teuthology.orchestra.run.smithi100.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41830.log.gz 2024-09-15T21:49:36.761 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56608.log 2024-09-15T21:49:36.761 INFO:teuthology.orchestra.run.smithi100.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42479.log.gz 2024-09-15T21:49:36.761 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45886.log 2024-09-15T21:49:36.761 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43805.log: /var/log/ceph/ceph-client.admin.56608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43455.log 2024-09-15T21:49:36.761 INFO:teuthology.orchestra.run.smithi100.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43805.log.gz 2024-09-15T21:49:36.761 INFO:teuthology.orchestra.run.smithi100.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56608.log.gz/var/log/ceph/ceph-client.admin.45886.log: 2024-09-15T21:49:36.761 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55231.log 88.5% 2024-09-15T21:49:36.762 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45886.log.gz 2024-09-15T21:49:36.762 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55222.log 2024-09-15T21:49:36.762 INFO:teuthology.orchestra.run.smithi100.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43455.log.gz 2024-09-15T21:49:36.762 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54885.log 2024-09-15T21:49:36.762 INFO:teuthology.orchestra.run.smithi100.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55231.log.gz/var/log/ceph/ceph-client.admin.55222.log: 2024-09-15T21:49:36.762 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42900.log 2024-09-15T21:49:36.762 INFO:teuthology.orchestra.run.smithi100.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55222.log.gz 2024-09-15T21:49:36.762 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54588.log 2024-09-15T21:49:36.762 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54885.log: /var/log/ceph/ceph-client.admin.42900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49424.log 2024-09-15T21:49:36.762 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54588.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54885.log.gz 2024-09-15T21:49:36.762 INFO:teuthology.orchestra.run.smithi100.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.42900.log.gz 2024-09-15T21:49:36.762 INFO:teuthology.orchestra.run.smithi100.stderr:gzip 87.7% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.54588.log.gz -- 2024-09-15T21:49:36.762 INFO:teuthology.orchestra.run.smithi100.stderr: /var/log/ceph/ceph-client.admin.57425.log 2024-09-15T21:49:36.762 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56272.log 2024-09-15T21:49:36.763 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.49424.log: /var/log/ceph/ceph-client.admin.57425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52630.log 2024-09-15T21:49:36.763 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56272.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57425.log.gz 90.0% -- replaced with /var/log/ceph/ceph-client.admin.49424.log.gz 2024-09-15T21:49:36.763 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-15T21:49:36.763 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.56272.log.gz 2024-09-15T21:49:36.763 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46774.log 2024-09-15T21:49:36.763 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.52630.log: /var/log/ceph/ceph-client.admin.55764.log 2024-09-15T21:49:36.763 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.46774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56920.log 2024-09-15T21:49:36.763 INFO:teuthology.orchestra.run.smithi100.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.52630.log.gz 2024-09-15T21:49:36.763 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46774.log.gz 2024-09-15T21:49:36.763 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47200.log 2024-09-15T21:49:36.763 INFO:teuthology.orchestra.run.smithi100.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55764.log.gz 2024-09-15T21:49:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53689.log 2024-09-15T21:49:36.764 INFO:teuthology.orchestra.run.smithi100.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56920.log.gz 2024-09-15T21:49:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57158.log 2024-09-15T21:49:36.764 INFO:teuthology.orchestra.run.smithi100.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47200.log.gz 2024-09-15T21:49:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-09-15T21:49:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.53689.log: gzip -5 --verbose/var/log/ceph/ceph-client.admin.57158.log: -- /var/log/ceph/ceph-client.admin.55227.log 2024-09-15T21:49:36.764 INFO:teuthology.orchestra.run.smithi100.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53689.log.gz 2024-09-15T21:49:36.764 INFO:teuthology.orchestra.run.smithi100.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57158.log.gz 2024-09-15T21:49:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47206.log 2024-09-15T21:49:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55558.log 2024-09-15T21:49:36.764 INFO:teuthology.orchestra.run.smithi100.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55227.log.gz 2024-09-15T21:49:36.764 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.47206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43806.log 2024-09-15T21:49:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.55558.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.47206.log.gz 2024-09-15T21:49:36.765 INFO:teuthology.orchestra.run.smithi100.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55558.log.gzgzip 2024-09-15T21:49:36.765 INFO:teuthology.orchestra.run.smithi100.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-15T21:49:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.43806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48639.log 2024-09-15T21:49:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-osd.3.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43806.log.gz 2024-09-15T21:49:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-15T21:49:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.48639.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.48639.log.gz 2024-09-15T21:49:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54462.log 2024-09-15T21:49:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52927.log 2024-09-15T21:49:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.54462.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54462.log.gz 2024-09-15T21:49:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56398.log 2024-09-15T21:49:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.52927.log: 90.2% 92.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-15T21:49:36.765 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45247.log 2024-09-15T21:49:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.56398.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56398.log.gz 2024-09-15T21:49:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45969.log 2024-09-15T21:49:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.0.45247.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45247.log.gz 2024-09-15T21:49:36.766 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/ceph-client.admin.45969.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45969.log.gz 2024-09-15T21:49:36.766 INFO:teuthology.orchestra.run.smithi100.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52927.log.gz 2024-09-15T21:49:36.766 INFO:teuthology.orchestra.run.smithi100.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-15T21:49:36.770 INFO:teuthology.orchestra.run.smithi100.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-09-15T21:49:36.770 INFO:teuthology.orchestra.run.smithi122.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-15T21:49:36.770 INFO:teuthology.orchestra.run.smithi122.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-15T21:49:36.803 INFO:teuthology.orchestra.run.smithi100.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-15T21:49:36.809 INFO:teuthology.orchestra.run.smithi100.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-15T21:49:36.849 INFO:teuthology.orchestra.run.smithi122.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-15T21:49:36.868 INFO:teuthology.orchestra.run.smithi100.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-15T21:49:36.883 INFO:teuthology.orchestra.run.smithi122.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-15T21:49:36.890 INFO:teuthology.orchestra.run.smithi122.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-15T21:49:36.890 INFO:teuthology.orchestra.run.smithi122.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-15T21:49:36.892 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-15T21:49:36.892 INFO:teuthology.orchestra.run.smithi122.stderr:real 0m0.299s 2024-09-15T21:49:36.892 INFO:teuthology.orchestra.run.smithi122.stderr:user 0m1.373s 2024-09-15T21:49:36.892 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m0.080s 2024-09-15T21:49:36.912 INFO:teuthology.orchestra.run.smithi100.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-15T21:49:36.933 INFO:teuthology.orchestra.run.smithi100.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-15T21:49:36.936 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-09-15T21:49:36.936 INFO:teuthology.orchestra.run.smithi100.stderr:real 0m0.364s 2024-09-15T21:49:36.936 INFO:teuthology.orchestra.run.smithi100.stderr:user 0m1.488s 2024-09-15T21:49:36.936 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m0.217s 2024-09-15T21:49:36.937 INFO:tasks.ceph:Archiving logs... 2024-09-15T21:49:36.937 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906191/remote/smithi100/log 2024-09-15T21:49:36.938 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T21:49:37.525 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906191/remote/smithi122/log 2024-09-15T21:49:37.526 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T21:49:37.798 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T21:49:37.810 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T21:49:37.810 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T21:49:37.813 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T21:49:37.897 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T21:49:37.910 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T21:49:37.911 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:49:37.915 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:49:37.929 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-09-15T21:49:37.930 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-09-15T21:49:37.935 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T21:49:37.936 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-09-15T21:49:37.936 INFO:teuthology.orchestra.run.smithi100.stdout:^* hv01.front.sepia.ceph.com 3 6 377 48 +1257us[+1257us] +/- 50ms 2024-09-15T21:49:37.936 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 47 -493us[ -493us] +/- 60ms 2024-09-15T21:49:37.936 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 -1259us[-1259us] +/- 70ms 2024-09-15T21:49:37.936 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:49:37.937 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T21:49:37.937 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-09-15T21:49:37.938 INFO:teuthology.orchestra.run.smithi122.stdout:^* hv01.front.sepia.ceph.com 3 6 377 47 +1264us[+1270us] +/- 50ms 2024-09-15T21:49:37.938 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 46 -486us[ -486us] +/- 60ms 2024-09-15T21:49:37.938 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 48 -1247us[-1240us] +/- 70ms 2024-09-15T21:49:37.938 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:49:37.938 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T21:49:37.948 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T21:49:37.948 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T21:49:37.958 DEBUG:teuthology.orchestra.run.smithi100:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T21:49:38.052 DEBUG:teuthology.orchestra.run.smithi122:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T21:49:38.118 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T21:49:38.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:49:38.150 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-15T21:49:38.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:49:38.178 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T21:49:38.188 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T21:49:38.199 INFO:teuthology.task.internal:Duration was 617.889290 seconds 2024-09-15T21:49:38.199 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T21:49:38.214 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T21:49:38.215 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T21:49:38.218 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T21:49:38.255 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T21:49:38.258 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T21:49:38.552 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T21:49:38.553 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-09-15T21:49:38.553 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T21:49:38.574 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-09-15T21:49:38.574 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T21:49:38.614 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T21:49:38.614 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T21:49:38.618 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T21:49:38.681 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T21:49:38.681 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T21:49:38.686 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T21:49:38.826 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T21:49:38.843 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T21:49:38.843 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T21:49:38.869 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T21:49:38.895 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T21:49:38.906 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:49:38.912 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:49:38.933 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-09-15T21:49:38.957 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-09-15T21:49:38.997 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:49:39.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:49:39.013 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:49:39.023 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-15T21:49:39.024 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-15T21:49:39.024 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-09-15T21:49:39.024 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-15T21:49:39.024 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-15T21:49:39.024 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-15T21:49:39.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-09-15T21:49:39.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-09-15T21:49:39.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-09-15T21:49:39.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:49:39.053 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T21:49:39.079 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T21:49:39.079 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906191/remote/smithi100 2024-09-15T21:49:39.080 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T21:49:39.114 DEBUG:teuthology.misc:Transferring archived files from smithi122:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906191/remote/smithi122 2024-09-15T21:49:39.114 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T21:49:39.147 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T21:49:39.147 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T21:49:39.151 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T21:49:39.197 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T21:49:39.209 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T21:49:39.209 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T21:49:39.223 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T21:49:39.223 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T21:49:39.226 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T21:49:39.238 INFO:teuthology.orchestra.run.smithi100.stdout: 265760 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 21:49 /home/ubuntu/cephtest 2024-09-15T21:49:39.253 INFO:teuthology.orchestra.run.smithi122.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 21:49 /home/ubuntu/cephtest 2024-09-15T21:49:39.254 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T21:49:39.271 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T21:49:39.305 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} msgr-failures/osd-mds-delay objectstore-ec/bluestore-bitmap overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/5 tasks/{1-thrash/osd 2-workunit/fs/trivial_sync}} duration: 617.8892900943756 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-15T21:49:39.306 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T21:49:39.395 INFO:teuthology.run:pass